Watermark embedding method and system for large language model based on entropy value self-adaptive adjustment

Abstract

The application provides an entropy value self-adaptive adjustment-based watermark embedding and detection method and system for a large language model, and belongs to the technical field of large language models. The method solves the problem of low reliability and influence on text quality of low-entropy text detection in the prior art. The method comprises the following steps: S1, calculating an entropy value of a token to be generated in real time through an entropy analysis module, and dynamically generating a watermark strength control parameter; S2, adaptively adjusting a watermark embedding process according to the watermark strength control parameter, and outputting a watermark text; and S3, calling a weight calculation module, and quantifying the confidence of a watermark signal in combination with the watermark strength control parameter. The method is mainly used in the field of text source identification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of large language models, and particularly relates to a large language model watermark embedding method based on entropy value self-adaptive adjustment. BACKGROUND

[0002] With the rapid development and wide application of large language models (LLMs), their popularity in text generation, content creation, and other fields has significantly increased. However, this popularity has also brought serious abuse problems: malicious actors use large language models to generate false news, spam, or infringing content, making it impossible to reliably distinguish the source of the text. The root of this problem lies in the fact that traditional text source identification mechanisms rely on external metadata or simple fingerprint technology, which can be easily tampered with or removed, and cannot provide reliable identity authentication. For example, attackers can hide the traces of model generation by simple text modification (such as synonym replacement or syntactic restructuring), making it difficult to trace malicious content.

[0003] To address the above problems, text watermarking technology is proposed, which embeds hidden features in generated text to achieve source authentication. Traditional watermarking methods (such as rule-based keyword replacement or statistical pattern insertion) attempt to implant watermarks without significantly affecting the naturalness of the text. However, these methods have fundamental flaws: due to the limitations of algorithmic semantic understanding, watermark embedding is often achieved by directly modifying, inserting, or replacing text content, making it difficult to balance semantic coherence and text quality, which can lead to semantic bias or readability decline in the watermark text, for example, forced word replacement that disrupts context logic, or inserted noise characters that reduce reading fluency. More seriously, traditional watermarking technology lacks deep integration with the large language model generation process, making it less robust against simple attacks such as synonym replacement, deletion, and modification, resulting in a significant decline in detection accuracy and limiting its practical application value.

[0004] In recent years, with the breakthrough of large language models in language semantics and context understanding ability, watermarking technology based on large language models has emerged, realizing more complex and covert embedding strategies. This method uses the internal mechanism of the model (such as logit distribution or probability sampling) to implant watermarks, reducing interference on the surface of the text and improving concealment and naturalness. However, the document points out that existing methods still have significant shortcomings: first, in terms of detection accuracy, especially for low-entropy text (i.e. high certainty of vocabulary), the watermark signal is easily covered by the model's inherent high probability selection, resulting in reduced detection reliability; second, in terms of robustness, in the face of complex watermark removal attacks (such as text synonym replacement, deletion or adversarial reconstruction), the detection effect of existing watermark technology decreases significantly, making it difficult to provide lasting protection in real-world environments. For example, the entropy value of high-frequency words in low-entropy text is low, and the existing static watermark strength parameter cannot be adaptively adjusted, causing excessive interference with text fluency or signal omission, which weakens the practicality of the watermark. SUMMARY

[0005] Therefore, the present application aims to provide a large language model watermark embedding method based on entropy adaptive adjustment to solve the problem of low detection reliability of low-entropy text and the impact on text quality in existing methods.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] The large language model watermark embedding method based on entropy adaptive adjustment comprises:

[0008] S1, calculating the entropy value of the token to be generated in real time through an entropy analysis module, and dynamically generating a watermark strength control parameter;

[0009] S2, adaptively adjusting the watermark embedding process according to the watermark strength control parameter, and outputting the watermark text;

[0010] S3, calling a weight calculation module to quantify the confidence of the watermark signal in combination with the watermark strength control parameter.

[0011] Further, an optimal mode is also proposed, wherein the entropy analysis module comprises:

[0012] The input prompt sequence p1, p2, p3…p k-1 is input into the large language model to obtain the non-normalized score distribution V k of the kth token to be generated.

[0013] The entropy value E(k) of the kth token is calculated using the entropy calculation formula, and E(k) is mapped to the watermark strength value range [δ1, δ2], where δ1 is the minimum value of the watermark strength and δ2 is the maximum value of the watermark strength.

[0014] Further, it is proposed that the entropy calculation formula is:

[0015]

[0016] wherein v represents a token in the vocabulary table of the current generated token, p v represents the probability distribution value of v, and τ is a constant.

[0017] Further, it is proposed that the function of the mapping watermark strength value is:

[0018]

[0019] wherein k is a scaling factor, and e is a natural constant.

[0020] Further, it is proposed that the adaptive adjustment watermark embedding process includes:

[0021] When the kth token is generated, the key sk and the previous token k-1 are selected as the input of the pseudo-random function F(), to generate a random number seed sd;

[0022] The vocabulary table V is divided into subsets V1 and V2 using the random number seed sd, and an adaptive adjustment watermark strength value δ(t) is added to the logit value of each vocabulary in V1;

[0023] The logit values of all vocabularies are re-normalized to achieve watermark embedding.

[0024] Further, it is proposed that the weight calculation module includes:

[0025] For each token in the to-be-detected text T: t1, t2, t3…t m , a weight value W: w1, w2, w3…w m is generated, and the weight value W represents the detection importance;

[0026] The original key sk is used to generate random numbers for each word in the to-be-detected text in an embedding manner, and the vocabulary table division is completed, the number of vocabularies from V1 is counted, and the z value is calculated in combination with the entropy value to quantify the watermark confidence.

[0027] Further, it is proposed that the z value calculation formula is:

[0028]

[0029] wherein |C| V1 represents the number of vocabularies from V1, |N| represents the size of the vocabulary table, and W iThe weight value representing the current vocabulary, and gamma represents the proportion parameter of the vocabulary table division.

[0030] Based on the same inventive concept, the present application also provides a large language model watermark embedding system based on entropy value adaptive adjustment, which comprises:

[0031] A watermark strength control parameter generation module is configured to calculate the entropy value of the token to be generated in real time through the entropy analysis module and dynamically generate the watermark strength control parameter;

[0032] A watermark text acquisition module is configured to adaptively adjust the watermark embedding process according to the watermark strength control parameter and output the watermark text.

[0033] A quantization module is configured to call the weight calculation module and quantize the confidence of the watermark signal in combination with the watermark strength control parameter.

[0034] Based on the same inventive concept, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the large language model watermark embedding method based on entropy value adaptive adjustment according to any one of the above.

[0035] Based on the same inventive concept, the present application also provides a computer readable storage medium, which stores a computer program, and when the computer program is run by a processor, the steps of the large language model watermark embedding method based on entropy value adaptive adjustment according to any one of the above are executed.

[0036] Compared with the prior art, the present application has the following advantages:

[0037] The method provided by the present application can effectively identify the low-entropy region (i.e., the high-deterministic vocabulary) of the text by performing real-time entropy value calculation on the token to be generated through the entropy analysis module. The embedding strength of the watermark can be dynamically adjusted according to the entropy value, which can avoid the watermark signal being covered by high-probability words, thereby improving the detection reliability of the watermark. For low-entropy text, the watermark strength can be automatically adjusted to ensure the detectability of the watermark signal.

[0038] The traditional watermark embedding method may interfere with the natural fluency of the text when the watermark strength is set unreasonably. However, the present application avoids the negative impact of excessive watermark strength on the quality of the text by dynamically adjusting the watermark strength control parameter, so that the watermark embedding process is more concealed and natural, and the readability and fluency of the text are guaranteed.

[0039] The method disclosed by the present application can maintain high robustness of the watermark signal when facing complex removal attacks (such as synonym replacement, deletion or counter-reconstruction) by adaptively adjusting the watermark embedding process. Dynamic watermark strength adjustment can better adapt to different text content and environmental changes, reduce the risk of attack removal, and improve the effectiveness of the watermark in real application environments. The entropy analysis method is used to adaptively adjust the watermark embedding process according to the characteristics of different texts. For example, low-entropy text and high-entropy text are processed differently, and the watermark strength is intelligently adjusted according to the actual situation to provide more flexible and accurate watermark embedding strategies. The weight calculation module is called to quantify the confidence of the watermark signal in combination with the watermark strength control parameter. This quantization can effectively determine the effectiveness and reliability of the watermark.

[0040] The method disclosed by the present application does not rely on model parameter modification or complex post-processing process, and can be embedded as a lightweight module in the existing inference process, with low computational overhead and deployment cost, facilitating large-scale deployment and practical application. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations on the present application. In the drawings:

[0042] Figure 1 The entropy value adaptive adjustment based large language model watermark embedding method described in the present application is shown in the flowchart. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0044] Embodiment one, see Figure 1 This embodiment describes an entropy value adaptive adjustment based large language model watermark embedding method. The method comprises:

[0045] S1, calculating the entropy value of the token to be generated in real time through an entropy analysis module, and dynamically generating a watermark strength control parameter;

[0046] S2, adaptively adjusting the watermark embedding process according to the watermark strength control parameter, and outputting a watermark text;

[0047] S3, calling a weight calculation module, and quantifying the confidence of the watermark signal in combination with the watermark strength control parameter.

[0048] The method provided by the embodiment can effectively identify the low-entropy region (i.e., the word with high certainty) of the text by performing real-time entropy value calculation on the token to be generated by the entropy analysis module. The embedding strength of the watermark can be dynamically adjusted according to the entropy value, so that the watermark signal can be avoided to be covered by the word with high probability, thereby improving the detection reliability of the watermark. For the low-entropy text, the watermark strength can be automatically adjusted to ensure the detectability of the watermark signal.

[0049] The conventional watermark embedding method may interfere with the natural fluency of the text when the watermark strength is set unreasonably. The method provided by the embodiment can avoid the negative impact of the excessive watermark strength on the quality of the text by dynamically adjusting the watermark strength control parameter, so that the watermark embedding process is more natural and hidden, and the readability and fluency of the text are guaranteed.

[0050] The method provided by the embodiment can make the watermark signal maintain high robustness when facing complex removal attacks (such as synonym replacement, deletion, or counter-reconstruction) by adaptively adjusting the watermark embedding process. The dynamic watermark strength adjustment can better adapt to different text content and environmental changes, reduce the risk of being attacked and removed, and improve the effectiveness in the real application environment of the watermark. The entropy value analysis method is adopted, so that the watermark embedding process can be adaptively adjusted according to the characteristics of different texts. For example, the processing methods for low-entropy text and high-entropy text are different. The watermark strength is intelligently adjusted according to the actual situation to provide more flexible and accurate watermark embedding strategies. The weight calculation module is called to combine the watermark strength control parameter to quantify the confidence of the watermark signal. The quantification can effectively judge the effectiveness and reliability of the watermark.

[0051] Embodiment two, the embodiment is a further limitation of the watermark embedding method of the large language model based on entropy adaptive adjustment described in embodiment one, and the entropy analysis module comprises:

[0052] The input prompt sequence p1, p2, p3…p k-1 The input large language model is used to obtain the non-normalized score distribution V of the kth token to be generated k ;

[0053] The entropy value E(k) of the kth token is calculated by using the entropy calculation formula, and E(k) is mapped to the watermark strength value range [δ1, δ2], where δ1 is the minimum value of the watermark strength, and δ2 is the maximum value of the watermark strength.

[0054] By mapping the entropy value E(k) of the kth token to the watermark intensity value range [δ1, δ2], the embodiment can dynamically adjust the intensity of the watermark according to the different sizes of the entropy value. For low-entropy regions (i.e., parts with high certainty), the watermark intensity is stronger to ensure that the watermark signal is not easily covered by high-probability words; while in high-entropy regions (i.e., more diverse word selection regions), the watermark intensity is weaker to avoid affecting the fluency and naturalness of the text. This adjustment mechanism can improve the concealment and naturalness of the watermark.

[0055] Input prompt sequence p1, p2, p3…p k-1 The entropy value of each generated token will be affected, so the intensity of the watermark can be dynamically adjusted according to the changes in the context content. This context-aware adaptive adjustment method is more flexible than statically setting the watermark intensity, and can effectively adapt to different types of text.

[0056] In practical applications, the context of the text often changes. By calculating the entropy value of each token in real time and adjusting the watermark intensity accordingly, the robustness of the watermark technology can be improved, ensuring that the watermark can still survive well when facing complex attacks (such as synonym replacement or adversarial reconstruction). Especially in low-entropy text, the enhancement of watermark intensity can resist attacks and reduce the loss of watermark signals.

[0057] The strategy of dynamically adjusting the watermark intensity can effectively avoid the loss of text quality caused by excessive watermark intensity. In high-entropy regions, the watermark intensity is weak, reducing the interference with the fluency of the text; while in low-entropy regions, the watermark intensity is enhanced, ensuring the detectability of the watermark signal, but not causing too much interference with the naturalness of the text. Such adjustment ensures the balance between the concealment of the watermark and the quality of the text.

[0058] Embodiment Three, the embodiment is a further limitation of the large language model watermark embedding method based on entropy adaptive adjustment according to embodiment two, and the entropy calculation formula is:

[0059]

[0060] Where v represents a token in the vocabulary of the currently generated token, p v represents the probability distribution value of v, and τ is a constant.

[0061] Embodiment Four, the embodiment is a further limitation of the large language model watermark embedding method based on entropy adaptive adjustment according to embodiment three, and the function of mapping the watermark intensity value is:

[0062]

[0063] where k is a scaling factor and e is a natural constant.

[0064] Embodiment five, this embodiment is a further limitation of the entropy value self-adaptive adjustment based large language model watermark embedding method described in embodiment one, the self-adaptive adjustment watermark embedding process includes:

[0065] When generating the kth token, the key sk and the previous token k-1 are selected as the input of the pseudo-random function F(), and a random number seed sd is generated;

[0066] The vocabulary V is divided into subsets V1 and V2 using the random number seed sd, and an adaptive adjustment watermark intensity value δ(t) is added to the logit value of each word in V1;

[0067] The logit values of all words are re-normalized to achieve watermark embedding.

[0068] In this embodiment, by introducing the pseudo-random function F(), the random number seed sd is generated using the previous token and the key, which makes the embedding of the watermark not only dependent on the content of the current token, but also associated with the historical information (the previous token). This adaptive mechanism can dynamically adjust the watermark intensity δ(t) according to the changes in the model generation process, so that the embedding of the watermark is more flexible and not easily disturbed or tampered with by external interference.

[0069] By dividing the vocabulary V into V1 and V2 subsets, and only adjusting the logit values of the words in V1, the interference of the watermark on the model output can be effectively reduced, and the concealment of the watermark can be enhanced. Especially, the adjustment during the generation process is based on the context environment of the token generation, thereby avoiding overly prominent watermark embedding and reducing the impact on the model performance.

[0070] Using the random number seed and the key to generate the watermark makes the watermark have good randomness and diversity in different generation processes, which enhances the applicability of the watermark in different models or tasks. If tampered with, the identification and extraction of the watermark will become more difficult, further improving the security of the watermark.

[0071] Because the strength of the watermark is adaptively adjusted during the generation of different tokens, the flexibility and complexity of the watermark are increased, so that the subsequent watermark detection process can maintain high accuracy while effectively avoiding the predictable or repetitive patterns that are easily exposed by conventional watermark embedding methods.

[0072] The adaptive adjustment process ensures that the strength of the watermark embedding does not excessively interfere with the performance of the generated model, ensuring the quality and fluency of the generated text, and avoiding the significant decline in the quality of the model output caused by an overly strong watermark.

[0073] Embodiment six, this embodiment is further limited to the entropy value self-adaptive adjustment based large language model watermark embedding method described in embodiment one, the weight calculation module comprises:

[0074] For each token in the to-be-detected text T: t1, t2, t3…t m , the weight value W: w1, w2, w3…w m is generated, and the weight value W represents the detection importance;

[0075] The original key sk is used to generate random numbers for each word in the to-be-detected text in turn in an embedding manner and complete the vocabulary division, count the number of words from V1, and combine the entropy value to calculate the z value to quantify the watermark confidence.

[0076] In this embodiment, the weight calculation module generates a weight value for each token, which means that the detection importance of each word in the text can be dynamically adjusted according to its specific position or context. This adaptive mechanism ensures that the watermark embedding process is more flexible and accurate, effectively improving the robustness and detectability of the watermark.

[0077] The entropy value is used to quantify the watermark confidence, which can be adjusted according to the complexity or information density of the text content. The higher the entropy value of the text, the greater the uncertainty of the information, so the embedding and detection of the watermark will be more refined, thereby improving the concealment and accuracy of the watermark in different text scenarios. The original key sk is used to generate the vocabulary division in the embedding process combined with random numbers, which increases the unpredictability and security of the watermark embedding. This method effectively avoids the risk of watermark being easily cracked or forged, and improves the protection ability of the entire system. By counting the number of words from V1 and combining the entropy value for calculation, the "importance" of each word and the "visibility" of the watermark can be more accurately evaluated, which not only improves the accuracy of detection, but also makes the watermark embedding effect more balanced and natural.

[0078] This method combines random number generation, entropy calculation and weight adjustment to make the watermark have high anti-interference performance. In actual application, even if the text is edited or partially modified, the watermark can still be accurately detected, enhancing the long-term effectiveness of the watermark.

[0079] Embodiment seven, this embodiment is further limited to the entropy value self-adaptive adjustment based large language model watermark embedding method described in embodiment six, the z value calculation formula is:

[0080]

[0081] Where, |C| V1 represents the number of words from V1, |N| represents the size of the vocabulary, and Wi represents the weight value of the current vocabulary, and γ represents a proportion parameter of the vocabulary table division.

[0082] Embodiment eight, the large language model watermark embedding system based on entropy value adaptive adjustment described in this embodiment, the system comprises:

[0083] The watermark strength control parameter generation module is configured to calculate the entropy value of the token to be generated in real time through the entropy analysis module, and dynamically generate the watermark strength control parameter;

[0084] The watermark text acquisition module is configured to adaptively adjust the watermark embedding process according to the watermark strength control parameter, and output the watermark text.

[0085] The quantification module is configured to call the weight calculation module and quantize the confidence of the watermark signal in combination with the watermark strength control parameter.

[0086] Embodiment nine, the computer device described in this embodiment, comprising a memory and a processor, the memory has a computer program stored therein, when the processor runs the computer program stored in the memory, the processor executes the large language model watermark embedding method based on entropy value adaptive adjustment according to any one of embodiments one to seven.

[0087] Embodiment ten, a computer readable storage medium according to the embodiment described in this embodiment, the computer readable storage medium has a computer program stored thereon, the computer program is executed by the processor to execute the steps of the large language model watermark embedding method based on entropy value adaptive adjustment as described in any one of embodiments one to seven.

[0088] Embodiment eleven, this embodiment is a specific embodiment of the large language model watermark embedding method based on entropy value adaptive adjustment described in embodiment one, and also used to explain embodiments two to seven, specifically:

[0089] The large language model watermark embedding method based on entropy value adaptive adjustment comprises:

[0090] S1, calculating the entropy value of the token to be generated in real time through the entropy analysis module, and dynamically generating the watermark strength control parameter;

[0091] S2, adaptively adjusting the watermark embedding process according to the strength control parameter, and outputting the watermark text with high detectability and high fluency;

[0092] S3, calling the weight calculation module, and quantizing the confidence of the watermark signal in combination with the entropy control strength parameter.

[0093] The entropy analysis module calculates the watermark intensity control parameter using an entropy value calculation formula and a watermark intensity function. Then, a bias is dynamically added using the parameter, and the bias is embedded into the watermark by changing the token selection of the large language model. The weight calculation module generates a confidence weight value for each token based on the entropy control intensity parameter, and the reliability of the presence or absence of the quantized watermark is aggregated by weighting.

[0094] Specifically, the entropy analysis module includes:

[0095] The input prompt sequence prompt: p1, p2, p3…p k-1 is input into the large language model, and the large language model generates an unnormalized score distribution V: l1, l2, l3…l N .

[0096] In order to make the generated watermark text have the characteristics of high detectability and low text quality loss, the entropy analysis module adaptively adjusts the watermark intensity parameter in real time during the watermark text generation process.

[0097] In the kth step of watermark text generation: according to V k output by the large language model, the entropy analysis module calculates the entropy value E(k) of token k using the entropy calculation formula E(t); where V k represents the unnormalized score distribution of the kth token to be generated, and E(k) represents the degree of certainty of the kth token. Then, E(k) is mapped to the watermark intensity value range: [δ1, δ2] using the watermark intensity function, where δ1 and δ2 represent the minimum and maximum values of the watermark intensity parameter, respectively.

[0098] Specifically, the entropy calculation formula includes:

[0099]

[0100] where v represents a token in the vocabulary of the currently generated token, pv represents the probability distribution value of v, and τ is a constant.

[0101] Specifically, the watermark intensity function includes:

[0102]

[0103] where δ1 and δ2 represent the minimum and maximum values of the watermark intensity parameter, respectively, and k is a scaling factor.

[0104] Specifically, the adaptive adjustment of the watermark embedding process to generate watermark text with high detectability and high fluency includes:

[0105] The input prompt sequence prompt: p1, p2, p3…pk-1 The input into the large language model, the calculation result of the entropy analysis module, will adjust the original output of the large language model to the watermark text Text: t1, t2, t3…t m .

[0106] The watermark strength parameter calculated by the entropy analysis module is called in real time before generating each token of the watermark text to adjust the output of the large language model. When the kth token is generated, the key sk and the previously generated token k-1 are selected as the input of the pseudo-random function F(), and then a random seed sd is generated by using the pseudo-random function, and the vocabulary V N of the large language model is divided into two small vocabularies V1 and V2, the logit value of each word in V1 is added to the adaptively adjusted watermark strength value δ(t), and the logit of all words is normalized again to increase the probability of selecting words in V1, thereby realizing the embedding of the watermark.

[0107] Specifically, the adaptive adjustment watermark embedding process to generate a watermark text with high detectability and high fluency includes:

[0108] To ensure high detectability: the watermark strength parameter of this token is higher, indicating that the entropy value of the token is higher, and at the same time, the uncertainty of the token is higher. The adaptive strategy will maximize the guarantee that the token output by the large language model is from the small vocabulary V1; on the contrary, the watermark strength parameter of the token is lower, indicating that the entropy value of the token is lower, and at the same time, the certainty of the token is higher. The adaptive strategy will try not to affect the output of the large language model. To ensure high fluency: for tokens with high uncertainty, the adaptive strategy will try not to be too demanding on the output of the large language model within a certain range to ensure the fluency of the watermark text; for tokens with high certainty, the adaptive strategy will try to reduce the intervention on the output of the large language model to ensure the fluency of the watermark text.

[0109] Specifically, the weight calculation module includes:

[0110] For a given to-be-detected text T: t1, t2, t3…t m , the weight calculation module generates a weight value W: w1, w2, w3…w m for each word in T, representing the importance of each word in the detection process.

[0111] In order to let the watermark detection effect have high confidence, the detection result is quantified. Using the original key sk, a random number is generated for each word in the to-be-detected text in turn by using the embedding mode, and the vocabulary division is completed. Each round needs to complete: (1) judging whether the current word is from the small vocabulary V1; (2) calculating the entropy value when the current word is generated; according to the entropy value of the current word, the words from the small vocabulary V1 are forward entropy values, the words are divided by weight, and then the current z value is calculated according to the z value calculation formula according to the number of words and the weight list; when the current z value is greater than the set threshold value, it is indicated that the current text is embedded with the watermark.

[0112] Specifically, the z value calculation formula includes:

[0113]

[0114] Wherein, |C| V1 represents the number of words from V1, |N| represents the size of the vocabulary, W i represents the weight value of the current word, and γ represents the proportion parameter of the vocabulary division.

[0115] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system or a computer program product. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1the function specified in the one or more blocks.

[0117] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable devices provide processes for implementing the flowcharts Figure 1 one or more flows and / or blocks Figure 1 the function specified in the one or more blocks.

[0118] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present disclosure, but not to limit the scope of protection, although the present disclosure has been described in detail with reference to the above examples, those skilled in the art should understand: after reading the present disclosure, the person skilled in the art can make various changes, modifications or equivalent replacements to the specific embodiments of the application, but these changes, modifications or equivalent replacements are all within the protection scope of the disclosed claims.

Claims

1. A large language model watermark embedding method based on entropy value self-adaptive adjustment, characterized in that, The method comprises: S1, calculating the entropy value of the token to be generated in real time through an entropy analysis module, and dynamically generating a watermark intensity control parameter; S2, adaptively adjusting the watermark embedding process according to the watermark intensity control parameter, and outputting a watermark text; S3, calling a weight calculation module, and quantifying the confidence of the watermark signal in combination with the watermark intensity control parameter; The entropy analysis module comprises: Input prompt sequence p1, p2, p3…p k-1 Input a large language model to obtain an unnormalized score distribution V of the kth token to be generated k ; The entropy value E(k) of the kth token is calculated by using an entropy calculation formula, and E(k) is mapped to a watermark intensity value range [δ1, δ2], wherein δ1 is the minimum value of the watermark intensity, and δ2 is the maximum value of the watermark intensity; The entropy calculation formula is: where v represents a token in the vocabulary currently generating the token, p v the probability distribution value representing v, and τ is a constant; The function of mapping the watermark intensity value is: Wherein, k is a scaling factor, and e is a natural constant; The adaptive adjustment of the watermark embedding process comprises: When the kth token is generated, the key sk and the previous token k-1 are selected as the input of the pseudo-random function F(), and a random number seed sd is generated; The vocabulary V is divided into subsets V1 and V2 by using the random number seed sd, and the logit value of each vocabulary in V1 is added with the adaptively adjusted watermark intensity value δ(t); The logit values of all vocabularies are re-normalized to realize watermark embedding; The weight calculation module comprises: For the text T to be detected: t1, t2, t3…t m Each token in the text T: t1, t2, t3…t m The weight value W represents the detection importance. The original key sk is used to generate random numbers for each word in the to-be-detected text in an embedding manner, and the vocabulary division is completed, the number of vocabularies from V1 is counted, and the z value is quantified in combination with the entropy value to quantify the watermark confidence; The z value calculation formula is: Among them, |C| V1 |N| represents the number of words from V1, |N| represents the vocabulary size, and W represents the number of words from V1. i This represents the weight value of the current word, where γ represents the proportional parameter for dividing the vocabulary.

2. A large language model watermark embedding system based on entropy value self-adaptive adjustment, characterized in that, The system comprises: A watermark intensity control parameter generation module is configured to calculate the entropy value of the token to be generated in real time through an entropy analysis module, and dynamically generate a watermark intensity control parameter; A watermark text acquisition module is configured to adaptively adjust the watermark embedding process according to the watermark intensity control parameter, and output a watermark text; A quantification module is configured to call a weight calculation module, and quantify the confidence of the watermark signal in combination with the watermark intensity control parameter; The entropy analysis module comprises: Input prompt sequence p1, p2, p3…p k-1 Input a large language model to obtain an unnormalized score distribution V of the kth token to be generated k ; The entropy value E(k) of the kth token is calculated by using an entropy calculation formula, and E(k) is mapped to a watermark intensity value range [δ1, δ2], wherein δ1 is the minimum value of the watermark intensity, and δ2 is the maximum value of the watermark intensity; The entropy calculation formula is: where v represents a token in the vocabulary currently generating the token, p v the probability distribution value representing v, and τ is a constant; The function of mapping the watermark intensity value is: Wherein, k is a scaling factor, and e is a natural constant; The adaptive adjustment of the watermark embedding process comprises: When the kth token is generated, the key sk and the previous token k-1 are selected as the input of the pseudo-random function F(), and a random number seed sd is generated; The vocabulary V is divided into subsets V1 and V2 by using the random number seed sd, and the logit value of each vocabulary in V1 is added with the adaptively adjusted watermark intensity value δ(t); The logit values of all vocabularies are re-normalized to realize watermark embedding; The weight calculation module comprises: For each token in the text T to be detected: t1, t2, t3...t m a weight value W: w1, w2, w3...w m is generated, the weight value W representing the detection importance; The original key sk is used to generate random numbers for each word in the to-be-detected text in an embedding manner, and the vocabulary division is completed, the number of vocabularies from V1 is counted, and the z value is quantified in combination with the entropy value to quantify the watermark confidence; The z value calculation formula is: where |C| denotes the number of words in V1, |N| denotes the vocabulary size, W V1 denotes the number of words in V1, |N| denotes the vocabulary size, W i denotes the weight value of the current word, and γ represents the proportion parameter of the vocabulary division.

3. A computer device, characterized by: The application discloses a computer readable storage medium, and the computer readable storage medium stores a computer program.

4. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by a processor to execute the steps of the large language model watermark embedding method based on entropy value self-adaptive adjustment.

Images