Method and device for embedding watermark in generated text, electronic equipment and medium

Abstract

The embodiment of the invention provides a method and device for embedding a watermark in a generated text, electronic equipment and a medium. The method comprises the steps that text prompt information used for generating a target text is acquired; inputting the text prompt information into a pre-trained target language model to generate a first text corresponding to the text prompt information based on the target language model, and determining a second text corresponding to the first text when the first text is consistent with a calibrated part-of-speech in a preset part-of-speech set; and in response to the detected output completion event of the target language model, obtaining the output target text. According to the technical scheme, the influence of the watermark on the text quality is reduced, the accuracy and robustness of watermark detection are improved, the calculation complexity of the watermark embedding and detection process is reduced, and the concealment and capacity of the watermark are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of information security, and in particular to a method, apparatus, electronic device and medium for embedding watermarks in generated text. Background Technology

[0002] With the widespread application of Large Language Models (LLMs), the possibility of misuse of this model has gradually emerged. Therefore, watermarking technology has been introduced to LLMs to detect and monitor text generated using Large Language Models.

[0003] Currently, methods for embedding watermarks in generated text include: constructing a thesaurus and embedding the watermark by replacing it with synonyms; or, finding and replacing the original words in the text with target words whose tone information matches the watermark data to achieve watermark embedding; or, analyzing grammatical dependencies, selecting relevant words and replacing them with synonyms, and then concatenating and converting the keywords with the selected synonyms before embedding; or, using hash values ​​to divide the vocabulary to improve the sampling probability to achieve watermark embedding.

[0004] However, these methods are highly dependent on the structural features of the generated text. When faced with perturbations such as text deletion or replacement, the stability of the watermark information is poor and its robustness is insufficient. At the same time, the computational process of watermark embedding and detection is relatively complex and computationally intensive, and it introduces a lot of intervention in the text generation process, which impairs the naturalness and readability of the text and reduces the quality of the generated text. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and medium for embedding watermarks in generated text, thereby reducing the impact of watermarks on text quality, improving the accuracy and robustness of watermark detection, reducing the computational complexity of watermark embedding and detection processes, and improving the concealment and capacity of watermarks.

[0006] In a first aspect, embodiments of the present invention provide a method for embedding a watermark in generated text, comprising:

[0007] Obtain the text prompt information used to generate the target text;

[0008] The text prompt information is input into the pre-trained target language model so that when the target language model generates a first text corresponding to the text prompt information and the first text is consistent with the labeled part of speech in the preset part of speech set, the second text corresponding to the first text is determined.

[0009] In response to detecting a target language model output completion event, the output target text is obtained; wherein the target text consists of at least one first text and / or a second text, and the second text includes watermark information.

[0010] Secondly, embodiments of the present invention also provide a device for embedding a watermark in generated text, the device comprising:

[0011] The text prompt information acquisition module is used to acquire text prompt information used to generate the target text;

[0012] The second text determination module is used to input text prompt information into a pre-trained target language model, so as to determine the second text corresponding to the first text when the first text is generated based on the target language model and the first text is consistent with the labeled part of speech in the preset part of speech set.

[0013] The target text output module is used to obtain the output target text in response to the detection of the target language model output completion event;

[0014] The target text consists of at least one first text and / or a second text, and the second text includes watermark information.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0016] At least one processor; and

[0017] A memory that is communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform a method for embedding a watermark in generated text as provided in any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to perform a method for embedding a watermark in generated text as provided in any embodiment of the present invention.

[0020] This invention, through embodiments thereof, acquires text prompt information for generating target text; inputs the text prompt information into a pre-trained target language model, and when a first text corresponding to the text prompt information is generated based on the target language model, and the first text matches the tagged part of speech in a preset part-of-speech set, determines a second text corresponding to the first text; in response to detecting a target language model output completion event, acquires the output target text; wherein the target text consists of at least one first text and / or a second text, and the second text includes watermark information. This reduces the impact of watermarks on text quality, improves the accuracy and robustness of watermark detection, reduces the computational complexity of watermark embedding and detection processes, and enhances the concealment and capacity of watermarks.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for embedding a watermark in generated text, provided by an embodiment of the present invention;

[0024] Figure 2 This is an overall framework diagram of a method for embedding watermarks in generated text, provided by an embodiment of the present invention.

[0025] Figure 3 A flowchart illustrating a method for embedding a watermark in generated text, provided by an embodiment of the present invention;

[0026] Figure 4 This invention provides a flowchart for determining the watermark embedding confidence of a text to be detected based on the actual part-of-speech and the labeled part-of-speech in a preset set of words in the text to be detected.

[0027] Figure 5 This is a schematic diagram of a device for embedding watermarks in generated text, provided by an embodiment of the present invention.

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Figure 1 This is a flowchart illustrating a method for embedding a watermark in generated text according to an embodiment of the present invention. This embodiment is applicable to situations where watermarks are embedded during text generation using a large language model. The method can be executed by a device for embedding watermarks in generated text, which can be implemented in hardware and / or software and can be configured in a computing device. Figure 1 As shown, the method includes:

[0032] S110. Obtain the text prompt information used to generate the target text.

[0033] The target text is the content that the user expects to generate through the large language model, which is usually a piece of text. The text prompt information can be understood as keywords or topic guidance provided by the user. As input to the large language model, it is used to guide the generation process and limit the core topic of the output text. The text prompt information can be a single word or multiple related words.

[0034] Specifically, users can determine one or more text prompts and input them into a large language model to generate text output that meets the expected theme and language requirements.

[0035] S120. Input the text prompt information into the pre-trained target language model, so that when the target language model generates a first text corresponding to the text prompt information, and the first text is consistent with the labeled part of speech in the preset part of speech set, determine the second text corresponding to the first text.

[0036] The target language model is a deep learning model selected and invoked to perform text generation and watermark embedding tasks, such as a large language model based on the Transformer architecture. These models are typically pre-trained on large text datasets. The first text is the text generated by the language model based on input prompts. The first text can be the output vocabulary. When the part-of-speech or structure of the first text matches the labeled part-of-speech defined by a preset part-of-speech set, the system can trigger the watermark embedding mechanism, that is, execute the watermark embedding method provided in this embodiment of the invention to determine the second text corresponding to the first text, thereby achieving the effect of adding a watermark. In this embodiment, the first text mainly refers to vocabulary. The length of the vocabulary is not limited in this embodiment. Processing the first text to generate the second text is the process of adding a watermark to the first text.

[0037] The preset part-of-speech tag set is a collection of parts of speech used for embedding the watermark. This set must match the parts of speech of the first text. Therefore, the preferred parts of speech tags are those with high frequency of occurrence in the generated text and crucial to sentence structure, including but not limited to verbs (VERB), nouns (NOUN), and determiners (DET). In this embodiment, the parts of speech in the preset part-of-speech tag set can be set according to actual needs during the development phase. The assigned part of speech is the selected part of speech from the preset part-of-speech tag set. When the part of speech of the first text matches the assigned part of speech, the first text needs to be processed to determine a second text that matches it. The resulting second text is the text after adding the watermark to the first text.

[0038] Specifically, the user inputs text prompts into a pre-trained target language model. The model, through inference, generates the original text without a watermark. At this point, the system can integrate a watermark embedding method to analyze and process the original text to obtain the watermarked target text. Both the target text and the original text are related to the prompts. The difference lies in that the original text is the unwatermarked text, while the target text is the second text generated by processing a first text whose part-of-speech tagging matches the part-of-speech tagging in a preset set of part-of-speech tags. In this embodiment, when generating the first text (original text), part-of-speech tagging can be performed on the first text and compared with a preset set of part-of-speech tags. If the part-of-speech tagging matches the tagged part-of-speech tagging, the system triggers a replacement mechanism to determine the second text associated with the first text.

[0039] Optionally, when a first text corresponding to the text prompt information is generated based on the target language model, and the first text matches the tagged part of speech in a preset part-of-speech set, a second text corresponding to the first text is determined, including:

[0040] When a first text corresponding to the text prompt information is detected based on the target language model, a preset part-of-speech tag set is retrieved; when the first text matches the tag in the preset part-of-speech tag set, a second text corresponding to the first text is determined based on the first text and the third text; wherein, the third text is a text that has been determined before the first text and includes a preset number of characters, and the second text is the text that follows the first text.

[0041] The preset number of texts refers to the number of words in the third text. This number can be set by the user. For example, if the preset number of texts is N, then the third text is the set of N texts that are preceding the first text and whose last text is adjacent to the first text.

[0042] Specifically, after the target language model generates the first text, it first determines the starting boundary of the first text, that is, by recognizing the starting token of the newly generated text. Then, the system obtains the part-of-speech tag of the text.

[0043] Specifically, after generating the first text, the system first matches it with a preset set of parts of speech. If the parts of speech match, it combines the first text with a third text consisting of a preset number of characters to determine the second text corresponding to the first text, thereby achieving the effect of embedding a watermark. In this embodiment, the process of generating the second text is the process of embedding a watermark.

[0044] Optionally, based on the first text and the third text, determining the second text corresponding to the first text includes: determining a text hash value based on the first text and the third text; dividing multiple candidate text words corresponding to the first text predicted by the target language model into a first word list or a second word list based on the text hash value; and determining the second text corresponding to the first text based on the probability value corresponding to each candidate text word in the first word list or the second word list. Here, the text hash value is the output result of a hash function applied to text information. The same text generates the same hash value each time it is calculated, and this is irreversible; the original text content cannot be derived from the hash value. Candidate text words are words in the vocabulary list generated by the target language model. These candidate text words can be understood as words concatenated from the first text based on its determination. Candidate text words can be divided according to the hash value to obtain two sets of lists. The two sets of lists have no intersection, and each element in each set of lists is one of the candidate text words. In this embodiment, the number of candidate text words can be one or more, and the specific data is related to the words corresponding to the first text and the prediction results of the target language prediction model. The two sets of tables can be referred to as the first vocabulary and the second vocabulary, respectively.

[0045] It should be noted that for each first text, as long as its part of speech matches the part of speech in the preset part of speech set, there exists a first vocabulary and a second vocabulary associated with it.

[0046] Specifically, first, a set of part-of-speech tags I is selected. While the target model generates each first text, the system obtains the part-of-speech tags for that first text. The system retrieves multiple previously determined characters from the first text and constructs the third text. A text hash value is obtained by calculating the hash values ​​of the first and third texts. For example, when the target language model generates the t-th text, it is based on the current context text. That is, calculate the hash value of the first N texts of the current text. .

[0047] Simultaneously, the target language model can output multiple candidate text words associated with the first text, resulting in a vocabulary V containing all candidate text words. Based on the target language model, the candidate text words in vocabulary V are vectorized, and then a random number generator is initialized with text hash values. The random number generator is used to segment the candidate text words in the vocabulary into a segment of size [size missing]. The first vocabulary list and a size of The second vocabulary list. Among them, The first vocabulary list represents the proportion of the vocabulary list V. Based on the probability values ​​of the text words in the first or second vocabulary list, the second text associated with the first text is determined.

[0048] In this embodiment, determining the second text corresponding to the first text based on the probability value corresponding to each candidate text word in the first or second vocabulary list can be as follows:

[0049] A preset constant is added to the matching degree of each candidate text word in the pre-selected first target vocabulary list; the target vocabulary list is a pre-configured list of words selected from the first vocabulary list or the second vocabulary list; the matching vector corresponding to each candidate text word after accumulation is processed based on the activation function in the target language model to obtain the target probability value corresponding to each candidate text word; based on the target probability value of each candidate text word in the first vocabulary list and the second vocabulary list, the second text corresponding to the first text is determined.

[0050] The target vocabulary list is a list of words to be obtained from a pre-determined vocabulary list. Optionally, if the pre-configured word list is selected from a first vocabulary list, then the first vocabulary list is the target vocabulary list; if the pre-configured word list is selected from a second vocabulary list, then the second vocabulary list is the target vocabulary list.

[0051] In this embodiment, the vocabulary list includes not only the vectors corresponding to candidate words but also the matching degree between the candidate word and the first text. In this embodiment, the matching degree can be represented numerically. The matching degree is used to characterize the fit between the candidate word and the first text. The preset constant is a fixed value predefined by the algorithm; it is a configurable hyperparameter used to be superimposed on the matching values ​​of all candidate words in the first target vocabulary list. The matching vector is the original score list composed of the matching degrees generated by the target language model for all candidate words. The activation function is a mathematical function in neural networks, such as the Softmax function, which in this embodiment is mainly used to transform the matching vector to obtain a formal probability distribution, facilitating the model's final output.

[0052] Specifically, the user selects either a first vocabulary list or a second vocabulary list as the first target vocabulary list, and obtains the matching vector from which the model outputs matching values ​​for all candidate words. A preset constant is then added to the matching value corresponding to each candidate text word in the matching vector of the first target vocabulary list. The accumulated matching vector is normalized using the activation function in the target language model, ensuring that the matching value for each candidate text word falls within the range of 0 to 1, thus obtaining the probability distribution on the vocabulary. A second text that matches the first text can be selected from the first target vocabulary list.

[0053] S130. In response to the detection of the target language model output completion event, obtain the output target text.

[0054] The target text consists of at least one first text and / or a second text, and the second text includes watermark information.

[0055] Specifically, when the part-of-speech tag of the first text does not match the tagging part-of-speech tag of the preset part-of-speech tag set, the target text is the first text output by the target language model. When the part-of-speech tag of the first text matches the tagging part-of-speech tag of the preset part-of-speech tag set, the second text is replaced and watermark information is embedded to obtain the target text composed of at least one first text and one second text.

[0056] This can be understood as follows: when the part-of-speech tag of the first text output by the target language model is inconsistent with the tagged part-of-speech tag in the preset part-of-speech tag set, it can be directly output. When it is consistent with the tagged part-of-speech tag in the preset part-of-speech tag set, the scheme provided in this embodiment of the invention can be executed to obtain the second text, until the target language model outputs the complete text. The final output text is taken as the target text, which is the text after the watermark is embedded.

[0057] The technical solution provided in this invention obtains text prompt information and inputs it into a pre-trained target language model to obtain the corresponding first text. A preset part-of-speech tagging set is constructed based on pre-defined part-of-speech tags. The first text is then matched against the tags in the preset part-of-speech tagging set. If they match, the corresponding second text is determined. This reduces the impact of watermarks on text quality, improves the accuracy and robustness of watermark detection, reduces the computational complexity of watermark embedding and detection processes, and enhances the concealment and capacity of the watermark.

[0058] Figure 2 This is an overall framework diagram of a method for embedding watermarks in generated text, provided by an embodiment of the present invention. Figure 2 Understand the technical solutions of the embodiments of the present invention.

[0059] like Figure 2 As shown, this embodiment explains the overall implementation of the solution based on the above optional implementation methods. Specifically, it includes:

[0060] First, the system obtains a priori token and determines whether the part-of-speech (POS) of the current word (first text) matches the required POS, i.e., whether the POS of the current word matches the POS tag in the preset POS set. If so, the candidate words predicted by the target language model and associated with the first text are divided into two lists: a red / green list (first word list / second word list), and one of these lists is selected as the target word list, such as the green list. Next, the vector values ​​representing the matching degree with the first text in the vectors corresponding to all candidate words in the green list are reconfigured. For example, the reconfiguration process adds a fixed constant, thus biasing the sampling process to select the next token from the green list. The next token is the second text. If the POS of the current word is not the required POS, the list division and accumulation process is not performed; instead, the next first text is directly output based on the target language model. After generating a new first text, the above POS judgment and sampling process is repeated. This watermark embedding loop continues until the target language model completes text generation, and finally outputs the text result containing the statistical watermark (target text).

[0061] This invention obtains text prompt information and inputs it into a pre-trained target language model to obtain the corresponding first text. A preset part-of-speech tagging set is constructed based on pre-defined part-of-speech tags. The first text is then matched against the tags in the preset part-of-speech tagging set. If they match, the corresponding second text is determined. This reduces the impact of watermarks on text quality, improves the accuracy and robustness of watermark detection, reduces the computational complexity of watermark embedding and detection processes, and enhances the concealment and capacity of the watermark.

[0062] Figure 3 This is a flowchart illustrating a method for embedding a watermark in generated text, provided by an embodiment of the present invention. This embodiment, based on the above embodiments, detects whether the generated text includes watermark information. Figure 3 As shown, the method includes:

[0063] S210. In response to a detection event that determines whether the text to be detected contains watermark information, obtain the text to be detected.

[0064] The text to be detected is the output content generated by the target language model based on the given text prompts. It can be a word, a phrase, or a paragraph.

[0065] Specifically, when the system detects a piece of generated text, it can perform an inspection to determine whether the text contains preset watermark information.

[0066] S220. Input the text to be detected into the target language model so that the target language model can determine the watermark embedding confidence of the text to be detected based on the actual part of speech of each word in the text to be detected and the labeled part of speech in the preset part of speech set.

[0067] Here, the actual part-of-speech tag refers to the true part-of-speech tag of each word in the text to be detected, as detected by the system. The watermark embedding confidence score can be understood as the probability value of whether the text to be detected contains watermark information. When the probability exceeds a predetermined threshold, the system determines that the text to be detected contains watermark information.

[0068] Specifically, after inputting the text to be detected into the target language model, the part-of-speech of each word in the text is determined and compared with the labeled part-of-speech in the preset part-of-speech set, thereby obtaining the watermark embedding probability value of the text to be detected.

[0069] Specifically, Figure 4 A flowchart for determining the watermark embedding confidence of a text to be detected based on the actual part-of-speech and the labeled part-of-speech in a preset set of words in the text to be detected, as provided in this embodiment of the invention, specifically includes the following steps:

[0070] S221. For each word in the text to be detected, when the actual part of speech of the current word matches the labeled part of speech, obtain the next multiple candidate words of the current word predicted by the target language model.

[0071] The actual part-of-speech of the current word is the true part-of-speech value obtained by the system from detecting the words in the text to be detected.

[0072] Specifically, the system determines the part-of-speech tag for each word in the text to be detected and compares it with the tagged part-of-speech tag in a preset set. When the part-of-speech tag of the current word matches the tagged part-of-speech tag, the system determines the vocabulary list for the next word predicted by the target language model. The vocabulary list includes multiple candidate words for the next word.

[0073] S222. Based on the hash value of the word division corresponding to the current word and the historical words with a preset number of characters before the current word in the text to be detected, divide multiple candidate words into the third word list or the fourth word list.

[0074] The preset text quantity is the number of context texts to be acquired, and the historical vocabulary includes all words preceding the current word. The third and fourth vocabulary lists are two lists generated during the detection process after dividing the vocabulary predicted by the target language model. All candidate text words in the predicted vocabulary are assigned to the third and fourth vocabulary lists respectively. The second target vocabulary list is one of the third and fourth vocabulary lists and can be selected by the user.

[0075] Specifically, once a preset number of characters is determined, the system retrieves the historical vocabulary preceding the current character, which is also a preset number of characters. A hash value is calculated based on the current word and the historical vocabulary, and multiple candidate words are then grouped into the third or fourth vocabulary list.

[0076] S223. When the next word of the current word in the text to be detected is in the second target word list, record the correct watermark addition information.

[0077] The method for determining the second target vocabulary list is the same as that for determining the first target vocabulary list, and it is selected from either the third vocabulary list or a list of vocabulary lists. The watermark embedding confidence level of the text to be detected is determined based on the correct addition of watermark information and the number of words in the text to be detected that match the identified parts of speech.

[0078] It should be noted that the first target vocabulary list selected when adding the watermark and the second target vocabulary list selected when verifying the watermark should be the same.

[0079] The correct watermark addition information can be understood as the rules followed during the watermark embedding stage. Based on the text to be detected and the correct watermark addition information, it can be used to infer how many words should theoretically undergo watermark addition during the generation process. The correct watermark addition information includes, but is not limited to, the total number of watermark positions and the number of generated words in the target vocabulary. The watermark embedding confidence score is the statistical probability value that the text to be detected contains a watermark. For example, if the third vocabulary is set as the second target vocabulary list, when the system detects that the next word of a word in the text to be detected is located in the third vocabulary, and that position meets the watermark addition conditions, this match is considered valid evidence of correct watermark addition and is recorded.

[0080] Specifically, when the next word of the current word in the text to be detected is in the second target word list, this match is regarded as evidence of the existence of a watermark and is recorded by the system as information for correctly adding the watermark.

[0081] For example, in the watermark detection process, the positions of words used for watermark embedding are first determined. These positions are typically marked with part-of-speech tags (POS tags), such as verbs, nouns, or determiners. Watermarks are embedded after the words corresponding to these POS tags. During detection, only the word tokens at these specific positions need to be considered. A third vocabulary is set as the target vocabulary. At each selected position, the same hash function used for watermark embedding is used to generate a third vocabulary G. This list contains all possible candidate word tokens, which are the legitimate choices in the watermark embedding process. For each possible watermark embedding position, it is checked whether the generated token is in the third vocabulary G. If it is, it is counted as a green token. The number of all these green tokens is counted. And the total number T of watermark locations.

[0082] Optionally, the watermark embedding confidence of the text to be detected is determined based on the correct addition of watermark information and the number of words in the text to be detected that match the identified part-of-speech tags, including:

[0083] Substitute the correct number of correctly added watermark information and the number of part-of-speech tags into the watermark embedding confidence function to output the watermark embedding confidence of the text to be detected.

[0084] The correct number of times the watermark is correctly added can be understood as the total number of times the next word in the second target vocabulary list is actually located in all positions that meet the condition of "the current word matches the part of speech in the preset part-of-speech set". The watermark embedding function is used to calculate the watermark embedding confidence score, taking the proportion of words detected in the second target vocabulary list, the number of words detected in the second target vocabulary list, and the total number of words in the watermark embedding position as input. This confidence score indicates the probability that the text to be detected contains a watermark; the higher the confidence score, the greater the probability of watermark embedding.

[0085] Specifically, the watermark embedding confidence score Z is calculated based on the proportion of words detected in the second target vocabulary list, the number of words detected in the second target vocabulary list, and the total number of words at the watermark embedding location, and is used to detect the presence of the watermark.

[0086]

[0087] in It is the proportion of words detected in the second target vocabulary list. To detect the number of words in the second target vocabulary list, T is the total number of words at the watermark embedding location.

[0088] S230. Based on the watermark embedding confidence level, determine whether the text to be detected has embedded watermark information.

[0089] Specifically, the confidence level is calculated based on the watermark embedding function described above. In the formula, the numerator is the deviation between the observed value and the expected value, and the denominator represents the standard error. The Z-value probability can be calculated based on a Gaussian distribution. When the probability is less than a certain preset significance level, it can be considered that the observed value is unlikely to occur under the null hypothesis. A threshold for the Z-value can be set; if the calculated Z-value exceeds the predetermined threshold, the system will determine that the text contains a watermark.

[0090] For example, we can assume that the text is watermark-free and calculate the expected number of times green words appear under this assumption. The resulting Z-value corresponds to a probability. We set a preset threshold of 0.05. If the probability is less than 0.05, it indicates that the number of words currently observed in the second target word list is unlikely to occur in a random situation without a watermark, that is, the text is determined to contain a watermark with high confidence.

[0091] The technical solution provided in this invention involves acquiring the text to be detected and inputting it into a target language model. For each word in the text, when the actual part-of-speech matches the labeled part-of-speech, the target language model and its predicted next multiple candidate words are obtained. Based on the hash values ​​of the words corresponding to the current word and a preset number of historical words preceding it in the text, the candidate words are divided into two lists: a third word list and a fourth word list. When it is detected that the current word will appear in the second target word list next time, the correct watermark addition information is recorded. Finally, based on the watermark embedding confidence, it is determined whether the text to be detected has embedded watermark information. By focusing on specific locations where the watermark is embedded in the text for statistical detection, the computational complexity of the detection process is reduced, ensuring a high detection rate of the watermark and minimal impact on text quality.

[0092] Figure 5 This is a schematic diagram of a device for embedding watermarks in generated text, provided as an embodiment of the present invention. Figure 5 As shown, the device includes: a text prompt information acquisition module 310, a second text determination module 320, and a target text output module 330.

[0093] The text prompt information acquisition module 310 is used to acquire text prompt information for generating target text; the second text determination module 320 is used to input the text prompt information into a pre-trained target language model, so as to determine the second text corresponding to the first text when the first text is generated based on the target language model and the first text is consistent with the labeled part of speech in the preset part of speech set; the target text output module 330 is used to acquire the output target text in response to the detection of the target language model output completion event.

[0094] The technical solution provided in this invention involves: acquiring text prompt information for generating target text; inputting the text prompt information into a pre-trained target language model; determining a second text corresponding to the first text when the first text generates a first text corresponding to the text prompt information based on the target language model, and the first text matches the tagged part of speech in a preset part-of-speech set; and acquiring the output target text in response to detecting a target language model output completion event. The target text consists of at least one first text and / or one second text, and the second text includes watermark information. This reduces the impact of watermarks on text quality, improves the accuracy and robustness of watermark detection, reduces the computational complexity of watermark embedding and detection processes, and enhances the concealment and capacity of watermarks.

[0095] Based on the above technical solutions, the second text determination module includes:

[0096] A preset part-of-speech tagging unit is used to retrieve a preset part-of-speech tagging set when the first text corresponding to the text prompt information is generated based on the target language model.

[0097] The second text determination unit is used to determine the second text corresponding to the first text based on the first text and the third text when the first text matches the tagged part of speech in the preset part-of-speech set. The third text is a text containing a preset number of characters that was previously determined before the first text, and the second text is the text following the first text.

[0098] Based on the above technical solutions, the second text determination unit includes:

[0099] The text hash value determination subunit is used to determine the text hash value based on the first text and the third text;

[0100] The candidate text word segmentation subunit is used to divide multiple candidate text words corresponding to the first text predicted by the target language model into the first word list or the second word list based on the text hash value.

[0101] The second text determination subunit is used to determine the second text corresponding to the first text based on the probability value corresponding to each candidate text word in the first or second vocabulary list.

[0102] Based on the above technical solutions, the second text defines sub-units, including:

[0103] The matching degree accumulation subunit is used to accumulate the matching degree of each candidate text word in the pre-selected first target word list by a preset constant; the target word list is a pre-configured word list selected from the first word list or the second word list;

[0104] The target probability value acquisition subunit is used to process the matching vector corresponding to each candidate text word after accumulation based on the activation function in the target language model, so as to obtain the target probability value corresponding to each candidate text word.

[0105] The second text final determination subunit is used to determine the second text corresponding to the first text based on the target probability value of each candidate text word in the first and second word lists.

[0106] Based on the above technical solutions, the device also includes:

[0107] The text to be detected module is used to obtain the text to be detected in response to a detection event that determines whether the text to be detected contains watermark information.

[0108] The watermark embedding confidence determination module is used to input the text to be detected into the target language model, so that the target language model can determine the watermark embedding confidence of the text to be detected based on the actual part of speech of each word in the text and the labeled part of speech in the preset part of speech set.

[0109] The watermark information determination module is used to determine whether the text to be detected has embedded watermark information based on the watermark embedding confidence level.

[0110] Based on the above technical solutions, the watermark embedding confidence determination module includes:

[0111] The candidate word acquisition unit is used to acquire the next multiple candidate words predicted by the target language model for each word in the text to be detected, when the actual part of speech of the current word matches the labeled part of speech.

[0112] The candidate word segmentation unit is used to segment multiple candidate words into the third word list or the fourth word list based on the word segmentation hash value corresponding to the current word and the historical words with a preset number of characters before the current word in the text to be detected.

[0113] The watermark correct addition information recording unit is used to record the watermark correct addition information when the next word of the current word in the text to be detected is in the second target word list; wherein, the method for determining the second target word list is the same as the method for determining the first target word list, and it is selected from the third word list or the fourth word list;

[0114] The watermark embedding confidence determination unit is used to determine the watermark embedding confidence of the text to be detected based on the correct addition of watermark information and the number of words in the text to be detected that are consistent with the tagged parts of speech.

[0115] Based on the above technical solutions, the watermark embedding confidence determination unit includes:

[0116] The watermark embedding confidence output subunit is used to input the correct number of correctly added watermark information and the number of part-of-speech tags into the watermark embedding confidence function to output the watermark embedding confidence of the text to be detected.

[0117] The watermark embedding device in generated text provided in the embodiments of the present invention can execute the watermark embedding method in generated text provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0118] Figure 6 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0119] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0120] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method of embedding a watermark in generated text.

[0122] In some embodiments, the method of embedding a watermark in generated text can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method of embedding a watermark in generated text described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method of embedding a watermark in generated text by any other suitable means (e.g., by means of firmware).

[0123] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0127] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0128] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0129] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0130] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for embedding watermarks in generated text, characterized in that, include: Obtain the text prompt information used to generate the target text; The text prompt information is input into a pre-trained target language model so that when a first text corresponding to the text prompt information is generated based on the target language model and the first text is consistent with the labeled part of speech in the preset part of speech set, a second text corresponding to the first text is determined. In response to the detection of the target language model output completion event, the output target text is obtained; The target text consists of at least one first text and / or a second text, and the second text includes watermark information.

2. The method according to claim 1, characterized in that, The step of generating a first text corresponding to the text prompt information based on the target language model, and determining a second text corresponding to the first text when the first text matches the tagged part of speech in a preset part-of-speech set, includes: When a first text corresponding to the text prompt information is detected based on the target language model, a preset part-of-speech tag set is retrieved. When the first text matches the tagged part of speech in the preset part-of-speech set, a second text corresponding to the first text is determined based on the first text and the third text; The third text is a text containing a preset number of characters that was determined before the first text, and the second text is the text following the first text.

3. The method according to claim 2, characterized in that, The step of determining the second text corresponding to the first text based on the first text and the third text includes: Based on the first text and the third text, determine the text hash value; Based on the text hash value, the multiple candidate text words predicted by the target language model that correspond to the first text are divided into a first word list or a second word list; Based on the probability value corresponding to each candidate text word in the first word list or the second word list, the second text corresponding to the first text is determined.

4. The method according to claim 3, characterized in that, The step of determining the second text corresponding to the first text based on the probability value corresponding to each candidate text word in the first or second vocabulary list includes: A preset constant is added to the matching degree of each candidate text word in the pre-selected first target word list; the target word list is a pre-configured word list selected from the first word list or the second word list; Based on the activation function in the target language model, the matching vector corresponding to each candidate text word after accumulation is processed to obtain the target probability value corresponding to each candidate text word; Based on the target probability value of each candidate text word in the first word list and the second word list, the second text corresponding to the first text is determined.

5. The method according to claim 1, characterized in that, The method further includes: In response to a detection event that determines whether the text to be detected contains watermark information, the text to be detected is acquired; The text to be detected is input into the target language model so that the target language model determines the watermark embedding confidence of the text to be detected based on the actual part of speech of each word in the text to be detected and the labeled part of speech in the preset part of speech set; Based on the watermark embedding confidence level, it is determined whether the text to be detected has embedded watermark information.

6. The method according to claim 5, characterized in that, The step of determining the watermark embedding confidence level of the text to be detected based on the actual part-of-speech and the labeled part-of-speech in the preset part-of-speech set includes: For each word in the text to be detected, when the actual part of speech of the current word matches the labeled part of speech, the next multiple candidate words predicted by the target language model for the current word are obtained. Based on the word division hash value corresponding to the current word and the historical words with a preset number of characters preceding the current word in the text to be detected, multiple candidate words are divided into a third word list or a fourth word list. When the next word of the current word in the text to be detected is in the second target word list, the watermark is correctly added information is recorded; wherein, the method for determining the second target word list is the same as the method for determining the first target word list, and it is selected from the third word list or the fourth word list; The watermark embedding confidence level of the text to be detected is determined based on the correct watermark addition information and the number of words in the text to be detected that match the identified part of speech.

7. The method according to claim 6, characterized in that, The step of determining the watermark embedding confidence of the text to be detected based on the correctly added watermark information and the number of words in the text to be detected that match the identified part-of-speech tagging includes: Substitute the correct number of correctly added watermark information and the number of part-of-speech tags into the watermark embedding confidence function to output the watermark embedding confidence of the text to be detected.

8. A device for embedding watermarks in generated text, characterized in that, include: The text prompt information acquisition module is used to acquire text prompt information used to generate the target text; The second text determination module is used to input the text prompt information into a pre-trained target language model, so as to determine the second text corresponding to the first text when a first text corresponding to the text prompt information is generated based on the target language model and the first text is consistent with the labeled part of speech in the preset part of speech set. The target text output module is used to obtain the output target text in response to the detection of the target language model output completion event; The target text consists of at least one first text and / or a second text, and the second text includes watermark information.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the method of embedding a watermark in generated text as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method of embedding a watermark in generated text as described in any one of claims 1-7.

Images