A watermark generation method, device and equipment based on a large model and a medium

Abstract

The application relates to the technical field of watermarking, in particular to a watermark generation method and device based on a large model, equipment and a medium. The method comprises the following steps: obtaining a target prompt word sequence, and obtaining a word sequence to be generated according to the target prompt word sequence; selecting one or more words in the word sequence as target words according to a preset random selection seed, and embedding watermark information into each target word according to a preset watermark strength to obtain each watermark marked word; splicing each watermark marked word and each word except the watermark marked word in the order of each word to obtain a watermark generation text; and generating a target watermark text according to the target prompt word sequence and the watermark generation text. The method can embed watermarking without affecting the quality of the model generation text as much as possible, and can quickly detect the watermark generation text.

Description

Technical Field

[0001] This application relates to the field of watermarking technology, and in particular to a method, apparatus, device and medium for generating watermarks based on a large model. Background Technology

[0002] Large models can generate highly realistic and logical text, but their powerful generation capabilities also bring potential risks of intellectual property abuse. Watermarking technology, as an information hiding method, has been introduced into the field of intellectual property protection in the era of large models. Large model watermarking, in particular, has attracted much attention due to its unique embedding and detection methods. How to embed watermarks without significantly affecting the quality of the text generated by the model has become a pressing problem to be solved. Summary of the Invention

[0003] This application provides a watermark generation method, apparatus, device, and medium based on a large model to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a watermark generation method based on a large model is provided, the method comprising:

[0006] Obtain the target prompt word sequence, and obtain the word sequence to be generated based on the target prompt word sequence, wherein the word sequence includes multiple words in an ordered manner;

[0007] According to a preset random seed, one or more words are selected from the word sequence as target words, and watermark information is embedded into each target word according to a preset watermark strength to obtain each watermarked word;

[0008] According to the order of the words, the words marked with watermarks and all other words are concatenated to obtain the watermark-generated text;

[0009] Generate the target watermark text based on the target prompt word sequence and the watermark generation text.

[0010] In one embodiment of this application, based on the foregoing scheme, the target prompt word sequence is obtained through the following steps:

[0011] Get the target prompt text;

[0012] The target prompt word text is split into multiple prompt words based on a preset large language model;

[0013] The target prompt word sequence is generated based on each of the prompt words.

[0014] In one embodiment of this application, based on the foregoing scheme, obtaining the word sequence to be generated according to the target prompt word sequence includes:

[0015] Each word is obtained sequentially based on the target prompt word sequence and the vocabulary set of the large language model;

[0016] Generate the word sequence based on each of the stated words.

[0017] In one embodiment of this application, based on the foregoing scheme, the step of selecting one or more words as target words from the word sequence according to a preset random selection seed includes:

[0018] Based on the number of words in the word sequence, each word is grouped to obtain multiple groups of word subsequences, each group of word subsequences including at least two words;

[0019] For each group of word subsequences, a random word is selected from each word in the word subsequence as the target word according to the random selection seed.

[0020] In one embodiment of this application, based on the foregoing scheme, the step of embedding watermark information into each of the target words according to a preset watermark strength to obtain each watermark-marked word includes:

[0021] Generate a watermark attack matrix based on the vocabulary set;

[0022] The target words are attacked according to the watermark attack matrix and the watermark strength to embed watermark information, thereby obtaining the watermarked words.

[0023] In one embodiment of this application, based on the foregoing scheme, generating the target watermark text according to the target prompt word sequence and the watermark generation text includes:

[0024] The target prompt word text corresponding to the target prompt word sequence and the watermark generated text are concatenated to obtain the target watermark text.

[0025] In one embodiment of this application, based on the foregoing scheme, after generating the target watermark text according to the target prompt word sequence and the watermark generation text, the method further includes:

[0026] Obtain the first total distribution probability of each word in the watermark-generated text and the second total distribution probability of each word in the word sequence;

[0027] If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not contain watermark information.

[0028] If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not have a watermark.

[0029] If the first total distribution probability is less than the second total distribution probability, then the generated target watermark text is determined to have a watermark.

[0030] According to one aspect of the embodiments of this application, a watermark generation apparatus based on a large model is provided, the apparatus comprising:

[0031] The acquisition unit is used to acquire a target prompt word sequence and acquire a word sequence to be generated based on the target prompt word sequence, wherein the word sequence includes multiple words in an ordered manner;

[0032] The watermark embedding unit is used to select one or more words as target words in the word sequence according to a preset random seed, and embed watermark information into each target word according to a preset watermark strength to obtain each watermark-marked word.

[0033] The splicing unit is used to splice the watermarked words and other words in the order of the words to obtain the watermarked text.

[0034] The watermark text generation unit is used to generate target watermark text based on the target prompt word sequence and the watermark generation text.

[0035] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided that stores a computer program thereon, the computer program including executable instructions that, when executed by a processor, implement the method described in the above embodiments.

[0036] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a memory for storing executable instructions of the processors, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in the above embodiments.

[0037] The beneficial effects of this application are as follows: This application can generate a corresponding word sequence using a pre-given target prompt word sequence. By selecting a portion of the words in the word sequence as target words for watermark embedding, watermarked words are obtained. Then, by sequentially concatenating the watermarked words with the words that have not been watermarked in the word sequence according to their order, watermarked text is generated.

[0038] Furthermore, by concatenating the target prompt word sequence with the watermark-generated text, the complete target watermark text is obtained. Therefore, this application only embeds watermarks on a subset of words, rather than all words, which reduces the impact of the watermark on the overall quality of the generated text.

[0039] Both the generated watermark text and the target watermark text obtained by splicing have watermarks, and the text quality is guaranteed, so there will be no significant deviation in the understanding of the prompt word sequence.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0042] Figure 1 This is a conventional token generation method illustrated in the embodiments of this application;

[0043] Figure 2 This is a flowchart illustrating a watermark generation method based on a large model according to an embodiment of this application;

[0044] Figure 3 This is a comparison diagram of grouped watermark embedding and global watermark embedding, which involves grouping words in a word sequence and embedding watermarks based on randomly selected seeds, according to embodiments of this application.

[0045] Figure 4 This is an architecture diagram of the TWT-LLM model network architecture shown in the embodiments of this application;

[0046] Figure 5 This is a comparison diagram of text quality under different watermark intensities according to embodiments of this application;

[0047] Figure 6 This is a block diagram illustrating a watermark generation apparatus based on a large model according to an embodiment of this application;

[0048] Figure 7 This is a structural diagram of an electronic device according to this application. Detailed Implementation

[0049] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0050] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0051] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller node devices.

[0052] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0053] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0054] The background technology of the embodiments of this application is described in detail below:

[0055] With the rapid development of artificial intelligence technology, especially the widespread application of large language models such as GPT, Claude, and LLaMA, human-like text can be generated rapidly. Text generated by large language models appears highly realistic, making it vulnerable to malicious use to create and spread misleading information such as fake news and misinformation. Content generated by large models may contain unauthorized personal information, infringe on personal privacy, and even harass or threaten specific individuals. As these large models become more prevalent, the risk of them being used for malicious purposes is increasing. There are instances of using large models to infringe on the copyrights of many authors' creative works online and to generate a large amount of false information on major social media platforms. Simultaneously, on social media, news websites, and other platforms, malicious users or organizations may use large models to generate false information to mislead public opinion, disrupt social stability, or seek illicit gains. This false information often spreads rapidly and widely, causing confusion and panic among the public, and even affecting government decision-making and judicial fairness. How to effectively protect the copyright and human intellectual property rights of these complex models has become an urgent problem to be solved. Therefore, the ability to detect and audit the use of machine-generated text and to detect and flag text generated by LLMs has become a key method for reducing the harm caused by large language models.

[0056] One effective method for accurately detecting text generated by large learning models is through watermarking. Watermarking in large models involves embedding specific information or identifiers into the model, subtly marking the generated text and allowing its origin to be determined. The aim is to achieve copyright protection, source tracing, or integrity verification without significantly impacting model performance. Watermarked text with feature information will exhibit inherent differences in characteristics compared to normally generated text. Figure 1This study demonstrates that the large language model GPT2-XL generates normal text and watermarked text from input prompt text sequences with different intrinsic dimensions. Furthermore, it reveals that text generated without watermarks exhibits relatively more regularity in word generation compared to watermarked text, while the latter, due to the interference from watermark information, appears more chaotic. This presents a challenge to our research: minimizing the impact of watermarking on the semantics of the generated text. In this application, the proposed large-model-based watermark generation method can be run independently on social media platforms or privately as an API running in the background. This allows for algorithmic detection without knowledge of model parameters or access to language model APIs, and the open-source nature of the detection algorithm makes detection cheaper and faster, as the LLM does not require loading or running. Moreover, the watermarked text can be generated using standard language models without retraining. Simultaneously, the target watermarked text detected in this application exhibits continuity, ensuring its presence is still detectable even when placed within large documents.

[0057] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0058] According to one aspect of this application, a watermark generation method based on a large model is provided. Figure 2 The flowchart below illustrates a watermark generation method based on a large model according to an embodiment of this application. This method includes at least steps S1 to S4, which are described in detail below:

[0059] In step S1, a target prompt word sequence is obtained, and a word sequence to be generated is obtained based on the target prompt word sequence. The word sequence includes multiple words in an ordered manner.

[0060] Specifically, the target cue word sequence is obtained through the following steps:

[0061] Get the target prompt text;

[0062] The target prompt word text is split into multiple prompt words based on a preset large language model;

[0063] The target prompt word sequence is generated based on each of the prompt words.

[0064] Target prompt text can be like Figure 1The prompt text shown, namely "We're so glad we have a holiday now! The weather is nice today," can be segmented into 12 prompt words using a large language model (represented by the letter L in this application). These 12 prompt words are: "very," "happy," "we," "already," "have," "holiday," "!", "today," "weather," "nice," and ",". After these 12 prompt words are input into the large language model L, they are converted into a corresponding prompt word sequence, namely the target prompt word sequence described in this application. The target prompt word sequence is represented by... express.

[0065] The watermark generation method based on a large model proposed in this application is called TWT-LLM. The TWT method embeds a certain amount of watermark information during the sampling process when generating the subsequent word sequence based on the target prompt word sequence.

[0066] In one embodiment of this application, obtaining the word sequence to be generated based on the target prompt word sequence includes:

[0067] Each word is obtained sequentially based on the target prompt word sequence and the vocabulary set of the large language model;

[0068] Generate the word sequence based on each of the stated words.

[0069] Following on from the above, the sequence obtained before inputting the 12 cue words into the large language model L can be used as follows: This indicates that it can be found in the vocabulary set of a large language model. The probability distribution is generated above The size of this vocabulary set is Therefore, it is possible to... The probability distribution for obtaining the next token. Here, the token is the word described in this application, and the word sequence described in this application is composed of multiple tokens in sequence.

[0070] Mathematical symbols can be used The watermark generation function is actually a function of the model. Even subtle changes can negatively impact text quality. Larger changes can disrupt the generated probability distribution, thus affecting text quality. This is achieved through the input sequence. ,use It is possible to obtain the probability distribution of the next watermarked token.

[0071] In step S2, one or more words are selected from the word sequence as target words according to a preset random seed, and watermark information is embedded into each target word according to a preset watermark strength to obtain each watermarked word.

[0072] In one embodiment of this application, the step of selecting one or more words as target words from the word sequence according to a preset random seed includes:

[0073] Based on the number of words in the word sequence, each word is grouped to obtain multiple groups of word subsequences, each group of word subsequences including at least two words;

[0074] For each group of word subsequences, a random word is selected from each word in the word subsequence as the target word according to the random selection seed.

[0075] Specifically, globally embedding a watermark throughout the entire process of word generation by the large language model L can not only affect the quality of the generated text but also compromise the security of the TWT-LLM model. Therefore, this application proposes an optimization strategy to improve the quality of the generated text after embedding, while ensuring that the hidden watermark information is difficult to detect with the naked eye. For example... Figure 3 As shown, two different watermark embedding strategies are used to generate text. The upper part of the diagram illustrates the global watermark embedding strategy, which embeds watermarks on all tokens during the generation process, slightly impacting the quality of the generated text. The lower part of the diagram illustrates the strategy based on group size. The tokens to be generated are divided into k groups. It should be noted that the TWT-LLM model, which is also the appendix to this application, is... Figure 4 The model network architecture is shown.

[0076]

[0077] For each group Apply watermark embedding function The result was obtained. Each group Random selection Each token is used for watermark embedding, and the random seed setting for selecting the token is set to... ,

[0078]

[0079] Because global embedding embeds watermarks onto all tokens, errors accumulate during generation, potentially causing the TWT-LLM model to exhibit hallucinations. In contrast, group-based local watermark embedding produces minimal interference, and the large language model generates tokens based on the preceding cue word sequence, tending to understand the correct meaning and generate subsequent tokens accordingly. Therefore, it avoids error accumulation. Thus, it is clear that compared to global embedding, group-based local embedding has a smaller impact on text generation.

[0080] In one embodiment of this application, the step of embedding watermark information into each of the target words according to a preset watermark strength to obtain each watermark-marked word includes:

[0081] Generate a watermark attack matrix based on the vocabulary set;

[0082] The target words are attacked according to the watermark attack matrix and the watermark strength to embed watermark information, thereby obtaining the watermarked words.

[0083] Specifically, see Figure 4 As shown, Figure 4 The logic diagram of the entire watermark embedding in this application is as follows: First, the target prompt text is input into the large language model, namely "I'm glad we have a holiday now! The weather is nice today," and then the target prompt sequence that can be recognized and used by the watermark generator is obtained. Furthermore, by adding watermark strength interference to the generation of each token, the generated words with watermarks (i.e., the watermarked words described in this application) can be obtained.

[0084] In one embodiment of this application, the step of embedding watermark information into each of the target words according to a preset watermark strength to obtain each watermark-marked word includes:

[0085] Generate a watermark attack matrix based on the vocabulary set;

[0086] The target words are attacked according to the watermark attack matrix and the watermark strength to embed watermark information, thereby obtaining the watermarked words.

[0087] Specifically, the steps for generating a watermark are as follows:

[0088] 1: Input: Watermark generation model: Watermark strength: Prompt word sequence: .Generation length:

[0089] 2: Calculate a given word sequence The output logit The probability distribution of the next word sequence.

[0090] Generate watermark attack matrix The size of this matrix is ​​determined by the vocabulary of the large model. Decide.

[0091] 4: Use watermark strength To carry out attacks and calculations When the watermark intensity When the value is 0, it is equivalent to not using a watermarking algorithm. When the watermark strength... When the value is 1, the quality of the generated text will be very poor.

[0092] 5: Concatenate the prompt word sequence and the generated watermark sequence

[0093] 6: Obtain a reconstructed large model watermark model. .

[0094] 7: Output: Fine-tuned large model with watermark. .

[0095] In step S3, the watermarked words and all other words are concatenated according to the order of the words to obtain the watermarked text.

[0096] Continue to refer to Figure 4 As shown, by concatenating the various watermarked words and other words in sequence, the target watermarked text "We're so glad we have a holiday now! The weather is nice today, but we don't want to go shopping together" can be obtained. "But we don't want to go shopping together" is the watermarked text described in this application.

[0097] In step S4, the target watermark text is generated based on the target prompt word sequence and the watermark generation text.

[0098] The step of generating the target watermark text based on the target prompt word sequence and the watermark generation text includes:

[0099] The target prompt word text corresponding to the target prompt word sequence and the watermark generated text are concatenated to obtain the target watermark text.

[0100] pass Figure 4It can be seen that "I'm so glad we have a holiday now! The weather is nice today, but we don't want to go shopping together" is the complete target watermark text. According to... Figure 4 As shown, firstly, the target prompt word sequence is used to generate a token sequence through a large language model. This token sequence represents the expression that the large language model can understand. Then, a watermark generator embeds watermark information, influencing the distribution of the next word (token) while minimizing impact on text generation quality. Next, the target prompt word sequence and the watermark-generated text are concatenated and sent to the watermark detection module. Finally, the watermark detection module can detect the concatenated complete text based on output patterns, detecting and determining the existence of the watermark.

[0101] In one embodiment of this application, after generating the target watermark text based on the target prompt word sequence and the watermark generation text, the method further includes:

[0102] Obtain the first total distribution probability of each word in the watermark-generated text and the second total distribution probability of each word in the word sequence;

[0103] If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not contain watermark information.

[0104] If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not have a watermark.

[0105] If the first total distribution probability is less than the second total distribution probability, then the generated target watermark text is determined to have a watermark.

[0106] Specifically, it can be done using mathematical symbols. This function represents a watermark detection method that can quickly and effectively detect watermarks in the generated text. It works by providing input prompt text. ,calculate .

[0107] Assumption It is the probability calculated when a watermark is used, that is, the first total distribution probability, and This is calculated assuming there is no watermark, that is, the total probability distribution. If the following formula is satisfied, then a watermark exists; otherwise, no watermark exists.

[0108]

[0109] Furthermore, this application uses a custom detection method to determine the presence of the watermark. Given a fixed large language model... Watermark strength and the number of words generated Let the probability of each token be... Calculate the total probability distribution of the word sequence with and without watermark, denoted as . The formula is as follows:

[0110]

[0111] when The existence of a watermark can be determined when one of the following relationships exists.

[0112]

[0113] Considering that as the scale of a large language model increases, the vocabulary size of that model also increases, such as the vocabulary size of GPT2. one of its words The value might be very small. If the above detection method is used directly, the two evaluation values ​​obtained will be... It is difficult to make a comparison. In order to amplify the inherent feature differences between unwatermarked and watermarked text, this application improves the evaluation and detection method.

[0114] Similarly, let the probability of each token be... Calculate the total probability distribution of the word sequence with and without watermark, denoted as . The formula is as follows:

[0115]

[0116] When we use an improved detection strategy to obtain The existence of a watermark can be determined when one of the following relationships exists.

[0117]

[0118] Furthermore, the preset watermark strength The watermark strength is 0.15. The size will affect Figure 4 The model's network architecture has a direct and significant impact on its effectiveness. For example... Figure 5 This can indicate the watermark strength. If the watermark strength is too high, it will affect the quality of the generated text. If the watermark is too small, it will be very difficult to detect its presence. Experiments have shown that when... The effect is best when the value is 0.15.

[0119] Figure 5 This demonstrates how different watermark intensities can be set when GPT is selected as the original model. The value of is used to generate watermarked text based on the given prompt text. You can see that when... When [the text is incomplete], the generated text content is largely unaffected. However, when [the text is incomplete], the generated text content is largely unaffected. At that moment, the model resembled a drunkard spouting nonsense.

[0120] In summary, the method proposed in this application can embed watermarks without significantly affecting the quality of the text generated by the model, and can detect the watermarked text relatively quickly.

[0121] According to one aspect of the embodiments of this application, a watermark generation device 300 based on a large model is proposed. Figure 6 This is a schematic diagram of a watermark generation device 300 based on a large model proposed in an embodiment of this application. The device 300 includes: an acquisition unit 301, a watermark embedding unit 302, a splicing unit 303, and a watermark text generation unit 304.

[0122] The acquisition unit 301 is used to acquire a target prompt word sequence and acquire a word sequence to be generated based on the target prompt word sequence, wherein the word sequence includes multiple words in order;

[0123] The watermark embedding unit 302 is used to select one or more words as target words in the word sequence according to a preset random seed, and embed watermark information into each target word according to a preset watermark strength to obtain each watermarked word;

[0124] The splicing unit 303 is used to splice the watermarked words and other words in the order of the words to obtain the watermarked text.

[0125] The watermark text generation unit 304 is used to generate target watermark text based on the target prompt word sequence and the watermark generation text.

[0126] In another aspect, this application also provides a computer-readable storage medium storing a program product capable of implementing the methods provided above in this specification. In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Embodiment Methods" section of this specification according to various exemplary embodiments of this application.

[0127] According to the embodiments of this application, the program product used to implement the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0130] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0131] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0132] The following reference Figure 7 To describe an electronic device 400 according to this embodiment of the present application. Figure 7 The electronic device 400 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0133] like Figure 7 As shown, the electronic device 400 is manifested in the form of a general-purpose computing device. The components of the electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one storage unit 420, and a bus 430 connecting different system components (including storage unit 420 and processing unit 410).

[0134] The storage unit stores program code that can be executed by the processing unit 410, causing the processing unit 410 to perform the steps described in the "Embodiment Methods" section above according to various exemplary embodiments of this application.

[0135] Storage unit 420 may include readable media in the form of volatile storage units, such as random access memory (RAM) 421 and / or cache memory 422, and may further include read-only memory (ROM) 423.

[0136] Storage unit 420 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of these examples or some combination thereof may include an implementation of a network environment.

[0137] Bus 430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell control node, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0138] Electronic device 400 can also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 400, and / or with any device that enables electronic device 400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 450. Furthermore, electronic device 400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 460. As shown, network adapter 460 communicates with other modules of electronic device 400 via bus 430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this application.

[0140] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0141] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A watermark generation method based on a large model, characterized in that, The method includes: Obtain the target prompt word sequence, and obtain the word sequence to be generated based on the target prompt word sequence, wherein the word sequence includes multiple words in an ordered manner; According to a preset random seed, one or more words are selected from the word sequence as target words, and watermark information is embedded into each target word according to a preset watermark strength to obtain each watermarked word; According to the order of the words, the words marked with watermarks and all other words are concatenated to obtain the watermark-generated text; Generate the target watermark text based on the target prompt word sequence and the watermark generation text; The step of selecting one or more words as target words from the word sequence according to a preset random seed includes: Based on the number of words in the word sequence, each word is grouped to obtain multiple groups of word subsequences, each group of word subsequences including at least two words; For each group of word subsequences, a random word is selected from each word in the word subsequence as the target word according to the random selection seed.

2. The watermark generation method based on a large model according to claim 1, characterized in that, The target prompt word sequence is obtained through the following steps: Get the target prompt text; The target prompt word text is split into multiple prompt words based on a preset large language model; The target prompt word sequence is generated based on each of the prompt words.

3. The watermark generation method based on a large model according to claim 2, characterized in that, The step of obtaining the word sequence to be generated based on the target prompt word sequence includes: Each word is obtained sequentially based on the target prompt word sequence and the vocabulary set of the large language model; Generate the word sequence based on each of the stated words.

4. The watermark generation method based on a large model according to claim 3, characterized in that, The process of embedding watermark information into each of the target words according to a preset watermark strength yields each watermark-marked word, including: Generate a watermark attack matrix based on the vocabulary set; The target words are attacked according to the watermark attack matrix and the watermark strength to embed watermark information, thereby obtaining the watermarked words.

5. The watermark generation method based on a large model according to claim 4, characterized in that, The step of generating the target watermark text based on the target prompt word sequence and the watermark generation text includes: The target prompt word text corresponding to the target prompt word sequence and the watermark generated text are concatenated to obtain the target watermark text.

6. The watermark generation method based on a large model according to claim 5, characterized in that, After generating the target watermark text based on the target prompt word sequence and the watermark generation text, the method further includes: Obtain the first total distribution probability of each word in the watermark-generated text and the second total distribution probability of each word in the word sequence; If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not contain watermark information. If the first total distribution probability is greater than the second total distribution probability, then it is determined that the generated target watermark text does not have a watermark. If the first total distribution probability is less than the second total distribution probability, then the generated target watermark text is determined to have a watermark.

7. A watermark generation device based on a large model, characterized in that, The device includes: The acquisition unit is used to acquire a target prompt word sequence and acquire a word sequence to be generated based on the target prompt word sequence, wherein the word sequence includes multiple words in an ordered manner; The watermark embedding unit is used to select one or more words as target words in the word sequence according to a preset random seed, and embed watermark information into each target word according to a preset watermark strength to obtain each watermark-marked word. The splicing unit is used to splice the watermarked words and other words in the order of the words to obtain the watermarked text. A watermark text generation unit is used to generate target watermark text based on the target prompt word sequence and the watermark generated text. The step of selecting one or more words as target words from the word sequence according to a preset random seed includes: Based on the number of words in the word sequence, each word is grouped to obtain multiple groups of word subsequences, each group of word subsequences including at least two words; For each group of word subsequences, a random word is selected from each word in the word subsequence as the target word according to the random selection seed.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the watermark generation method based on a large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the watermark generation method based on a large model as described in any one of claims 1 to 6.

Images