Text watermarking method based on synonym generation and hash operation

Abstract

The application discloses a text watermarking method based on synonym generation and hash operation, and finds out a key word for each single sentence. Whether the part of speech of the word in the adjacent position of the key word is in the part of speech exclusion list is judged; if yes, the word with the highest syntax dependency relationship with the key word in the sentence is selected as a related word to generate a synonym candidate set of the related word; otherwise, the adjacent word is used as the related word to generate the synonym candidate set; a random number generating method is used to generate a numerical value r as an index to select a synonym from the synonym candidate set to replace the related word. The key word and the selected synonym are spliced and hash operation is performed, and a numerical value obtained by taking 4 as a remainder is converted into a first binary code. The key word and the related word of the suspicious text are found, and a second binary code is calculated by using the same method. The second binary code is compared with the first binary code, a bit error rate is calculated, and ownership of the suspicious text is declared when the error rate is less than a threshold value. The application is convenient for extracting multi-bit watermark information.

Description

Technical Field

[0001] This invention relates to the field of watermark generation technology, and in particular to a text watermarking method based on synonym generation and hash operation. Background Technology

[0002] To protect the important information in text data, adding watermarks to protect intellectual property rights is essential. Existing watermarking methods for natural language text include adjusting text formatting, inserting spaces, modifying word characters, and synonym replacement. Compared to other methods, synonym replacement is more covert and has less impact on the meaning of the original text. The BERT model is a commonly used synonym generation model in recent research. Using the BERT model to dynamically generate synonyms can effectively capture the semantic information of sentences. To improve the robustness of synonym replacement, current research includes synonym replacement methods based on neural network language models and keyword-based synonym replacement methods. The first method is a zero-bit watermarking method, which can only verify whether the text contains a watermark, but cannot verify the ownership of the text; the second method is a multi-bit watermarking method, which requires generating synonyms for the same word in both the watermark embedding and verification stages, and then comparing whether the synonyms generated in the two stages are the same. If the synonyms generated in the two stages are the same, ownership of the questionable text can be claimed. This type of method heavily relies on the synonyms generated in the two separate instances, making it poorly robust against attacks such as word deletion, insertion, and replacement. Existing research lacks robust methods capable of extracting multi-bit watermark encoding information.

[0003] Furthermore, to avoid compromising the semantics of the text, attackers typically choose to attack sentences with low semantic importance. To address this issue, this invention considers the varying importance of different sentences within the text and assigns different encodings to the bit error counts of different sentences during the bit error rate calculation process.

[0004] The existing technology, "A Language Model Protection Method, Apparatus, and Computing Device Cluster," acquires the request text input by the user. Next, if the request belongs to the target category, the target instruction and request text are input into a language model for processing, resulting in a first response message with an added watermark, which is then output. Then, if the request text does not belong to the target category, the request text is input into the target language model for processing, resulting in a second response message, which is also output. This method automatically generates watermarked response messages when the language model processes specific types of requests, improving the language model's copyright protection capabilities without compromising the quality of the generated text. Meanwhile, "A Natural Language Watermarking Method" is a word substitution natural language watermarking method based on a neural network language model. This method utilizes a neural network language model to automatically generate appropriate replacements for the text content to embed the watermark. Summary of the Invention

[0005] The purpose of this invention is to provide a text watermarking method based on synonym generation and hash operation, so as to protect the text generated by the language model.

[0006] The technical solution adopted in this invention is:

[0007] A text watermarking method based on synonym generation and hash operation includes the following steps:

[0008] Step S1: For the i-th sentence Sen in the text i Find the keyword w s(j) ;w s(j) Indicates a simple sentence, Sen i The j-th word;

[0009] Step S2: Determine the keyword w s(j) The word w in the adjacent position s(j+1) Is Sen a member of the part-of-speech exclusion list? If not, then pos = j + 1; otherwise, select Sen. i Chinese and keyword w s(j) The related word w with the strongest grammatical dependency s(x) Let pos = x; where w s(x) x represents the associated word with the highest implied score; w represents the associated word. s(x) Location;

[0010] Step S3: Target the selected word w s(pos) A pre-trained padding model is used to generate a candidate set of watermarks, resulting in the synonym set can_w;

[0011] Step S4: Generate a numerical value r that is less than or equal to the length v of the candidate set using a random number generation method; select a synonym w from the candidate set with index r as the index. cr Used to replace the selected related words. s(pos) This yields the text T' containing the "watermark";

[0012] Step S5: For each single sentence Sen in T' i 'keyword w s(j) and new related words w cr After concatenation, a hash calculation is performed. The resulting hash value is modulo 4 and converted to obtain the first binary code. This first binary code is then appended to the single sentence Sen. i The text T containing the watermark and hash value is obtained at the end of the '.

[0013] Step S6: Obtain suspicious text T sus The i-th suspicious sentence Sen was found sus_iThe keywords and related terms in the text are subjected to the same hash and modulo calculation to obtain the corresponding Sen. sus_i The second binary code;

[0014] Step S7: Compare the second binary code with the first binary code and calculate the bit error rate; when the bit error rate is less than the threshold ε, claim ownership of the suspicious text.

[0015] Furthermore, in step S1, the unsupervised method YAKE is used to extract the single sentence Sen. i Find the keyword w in s(j) .

[0016] Furthermore, the specific steps of step S2 are as follows:

[0017] Step S2-1: First examine the keyword w s(j) Adjacent words w s(j+1) Determine the part of speech of the given word; check if the part of speech is a member of the ExList excluded by the part of speech list. s(j+1) If the part of speech of w is not a member of ExList, then let pos = j + 1, and end step S2; when w s(j+1) If the part of speech is a member of ExList, then proceed to step S2-2.

[0018] Step S2-2: Use the spay model as a syntactic dependency component to calculate the keyword w. s(j) The implied score of the grammatical dependencies between the word and other words in the sentence is recorded in relation to the keyword w. s(j) There exists a word w with the highest implied score. s(x) Let pos = x, given the position x.

[0019] Furthermore, in step S2-1, the part-of-speech exclusion list ExList = {pronouns, prepositions, conjunctions, proper nouns, punctuation marks, classifiers, personal names, place names, and other proper terms}.

[0020] Furthermore, the implication score of the grammatical dependencies in step S2-2 is constructed by masking and replacing each type of dependency word using an imputation model, and by calculating the implication score of the original sentence using an NLI model.

[0021] Furthermore, the first binary code at the end of each sentence in step S5 will be used as a watermark code for subsequent text knowledge copyright verification.

[0022] Furthermore, in step S5, the single sentence Sen... i The binary encoding process of ' is to convert Sen' into 'Sen'. i The keyword w in ' s(j) With the replacement w s (pos)The word w cr The data is concatenated, followed by SHA-256 hash calculation; the specific formula for the binary encoding algorithm is as follows:

[0023] b i =Binary(h(w) s(j) +w cr )%4)

[0024] Where b i The ith representation represents the binary encoded information of the i-th sentence, h represents the hash function, and w represents the hash function. s(j) Indicates keywords, w cr Indicates that it is used to replace w s(j) Synonyms for related terms, Binary is a function that encodes hash values ​​in binary form.

[0025] Furthermore, in step S6, for the suspicious text T sus ={Sen sus_1 Sen sus_2 ,...,Sen sus_m Each sentence in} sus_i Use the YAKE method to find the keywords and corresponding related words in a sentence.

[0026] Furthermore, in step S7, different weights are set for the number of bit errors when comparing different single sentences.

[0027] Furthermore, the calculation of the bit error rate in step S7 is as follows:

[0028] Step S7-1: Use the TF-IDF algorithm to extract keywords from the entire text, and then calculate the relevance between the keywords of each sentence and the keywords in the text, denoted as Corr = {c r(1) ,c r(2) ,...,c r(m)};

[0029] Step S7-2: Set the weight of the number of bit errors in each sentence during the comparison process as Wei = {c r(1) ,c r(2) ,...,c r(m) Let err be the number of bit errors in the i-th single sentence during the comparison process. i The formula for calculating the text bit error rate is as follows:

[0030]

[0031] Among them, c r(i) This represents the weight of the i-th sentence.

[0032] This invention employs the above technical solution to apply watermarking to text for verifying ownership. Each piece of text has its own unique embedded bit sequence. This invention primarily aims to protect text containing important information. Unlike watermark generation through controlling the output of a language generation model, this invention embeds watermarks by replacing related words in the original sentence with synonyms, and then uses hash operations to binary-encode the embedded watermark based on the keywords and synonyms. This invention is more practically significant because, generally, we cannot access the structure and parameters of a language model. Modifying the output probability distribution of a language model can only verify whether a text contains a watermark, but cannot extract the watermark itself. This invention focuses on extracting multi-bit watermark information from text watermarks. Attached Figure Description

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0034] Figure 1 This is a schematic diagram illustrating the principle framework of the text watermarking method based on synonym generation and hash operation of the present invention. Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, this invention discloses a text watermarking method based on synonym generation and hash operation, which identifies keywords for each sentence. It determines whether the part-of-speech tag of the word adjacent to the keyword is in the part-of-speech exclusion list; if so, it selects the word in the sentence with the highest grammatical dependency on the keyword as a related word, generating a candidate set of related word synonyms; otherwise, it generates a candidate set of synonyms using adjacent words as related words; it uses a random number generation method to generate a value r as an index to select a synonym from the candidate set to replace the related word. The keyword and the selected synonym are concatenated and hashed, and the result is converted into a first binary code by taking the remainder after dividing by 4. Keywords and related words of suspicious text are found, and a second binary code is calculated using the same method. The second binary code is compared with the first binary code to calculate the bit error rate; if the error rate is less than a threshold, ownership of the suspicious text is claimed. The method of this invention specifically includes the following steps:

[0037] Step S1: Let the selected text T = {Sen1, Sen2, ..., Sen...} m}, where Sen i Let Sen represent the i-th sentence in T, and m represent the total number of sentences in T. For each sentence, Sen... i ={w s(1) ,ws(2) ,...,w s( j) ,...,w s(k) }, w s(k) For the single sentence Sen i For the k-th word in the dataset, the unsupervised method YAKE (Yet Another Keyword Extractor) is used to find stable and invariant key features (keywords) w. s(j) .

[0038] Step S2: Selection of the synonym replacement position pos. For each sentence Sen... i Determine its keyword w s(j) The word w in the adjacent position s(j+1) Is the part of speech of 'Sen' a member of the exclusion list? If not, then pos = j + 1; otherwise, use the spacy model and select Sen. i Chinese and keyword w s(j) The related word w with the strongest grammatical dependency (highest implication score) s(x) Let pos = x.

[0039] Step S3: Target the selected word w s(pos) A pre-trained padding model is used to generate candidates for the watermark set, i.e., w. s (pos) The set of synonyms can_w = {w c1 ,w c1 ,...,w cv}

[0040] Specifically, by using a pre-trained model to generate a replacement word for the word at the current position by masking the word at that position, the contextual information can be better understood. Furthermore, this invention filters candidate encrypted words, excluding words like prepositions that could cause grammatical errors due to replacement.

[0041] Step S4: Using a random number generation method, generate a numerical value r that is less than or equal to the candidate set length v. Select the synonym w indexed by r from can_w. cr , to replace w s (pos) This leads to the sentence Sen containing the "watermark". i And finally, the text T' containing the "watermark" is obtained as {Sen1', Sen2', ..., Sen...} m '}.

[0042] Step S5: For each single sentence Sen in T' i 'keyword w s(j) and its new related words wcr After concatenation, perform hash calculation h(w) s(j) ,w cr The resulting hash value is modulo 4, and the remainder is converted into a binary code of length 2, which is then appended to the single sentence Sen. i At the end of ', the final result is text T containing the "watermark" and hash value = {Sen 1* ',Sen 2* ',...,Sen m* The binary code at the end of each sentence will be used as a watermark for subsequent text copyright verification.

[0043] Step S6: For the suspicious text T sus ={Sen sus_1 Sen sus_2 ,...,Sen sus_m Each sentence in} sus_i The YAKE method is used to find keywords and related words in the sentence (the method for determining the position of related words is the same as in step S2). The same hashing and modulo operations are then performed to obtain the corresponding words for Sen. sus_i A binary code.

[0044] Step S7: Obtain the result corresponding to Sen from step S6 sus_i The binary encoding and step S5 are stored in Sen i* The watermark encoding in the text is compared to obtain the number of bit errors for the i-th sentence, and finally the number of bit errors for the entire text. Considering that the watermarked text may be subject to word insertion, deletion, and substitution attacks, and that attackers may focus on attacking sentences with low semantic importance to improve the stealth of the attack, this invention calculates the number of bit errors for the suspicious text T. sus The bit error rate between the encrypted text T and the encrypted text T is such that if the error rate is less than a threshold ε, then the text T can be declared as suspicious. sus Ownership.

[0045] Furthermore, the selection of encrypted words in existing technologies lacks robustness. When attacks such as deletion, replacement, or insertion of words occur, the extracted encrypted information and its length will change. To address these issues, the present invention selects the synonym replacement position in step S2. Synonym replacement is performed on a locatable position in each sentence (the adjacent position of the keyword or the position of the word with the highest implication score that has a grammatical dependency on the keyword). Even under the aforementioned attacks, this position can still be located. To facilitate verification, the keywords of a single sentence are considered first. Keywords are the semantic core of a sentence and should remain unchanged even under attack; otherwise, the original meaning of the sentence would be affected. After extracting the keywords of a single sentence, they are used as anchors to determine the synonym replacement position pos using a heuristic method. The specific steps are as follows:

[0046] Step S2-1: Prioritize examining keywords w s(j) Adjacent words w s(j+1) Determine the part of speech of the given word, and whether it is a member of the ExList in the part-of-speech exclusion list: if w s(j+1) If the part of speech of w is not a member of ExList, then let pos = j + 1, and end step S2; if w s(j+1) If the part of speech is a member of ExList, then proceed to step S2-2.

[0047] Step S2-2: Use the spay model as a syntactic dependency component to calculate the keyword w. s(j) The implied score of the grammatical dependencies between the word and other words in the sentence, recorded in relation to the keyword w. s(j) There exists a word w with the highest implied score. s(x) Let pos = x, given the position x.

[0048] Further, in step S2-1, the part-of-speech exclusion list ExList = {pronouns, prepositions, conjunctions, proper nouns, punctuation marks, quantifiers, personal names, place names, and other proper terms}. The implication score of the grammatical dependencies in step S2-2 is constructed by masking and replacing each type of dependency word using an imputation model, and then calculating the implication score with the original sentence using an NLI model.

[0049] Furthermore, in step S5, the single sentence Sen... i The binary encoding process of ' is to convert Sen' into 'Sen'. i The keyword w in ' s(j) With the replacement w s (pos) The word w cr The data is concatenated and then subjected to SHA-256 hash calculation. The specific formula for the binary encoding algorithm is as follows:

[0050] b i=Binary(h(w) s(j) +w cr )%4)

[0051] Where b i The ith representation represents the binary encoded information of the i-th sentence, h represents the hash function, and w represents the hash function. s(j) Indicates keywords, w cr Indicates that it is used to replace w s(j) Synonyms for related terms, Binary is a function that encodes hash values ​​in binary form.

[0052] Furthermore, considering the following potential attack scenario—if an attacker attacks sentences with high semantic importance in the text, it will severely affect the readability of the text. In order to minimize the impact on the semantics and readability of the source text and to hide the attack to the greatest extent, the attacker attempts to attack sentences with lower semantic importance in the text. In step S7, the present invention sets different weights for the number of bit errors during different single-sentence comparisons.

[0053] Furthermore, considering the following potential attack scenario—if an attacker attacks sentences with high semantic importance in the text, it will severely affect the text's readability. To minimize the impact on the source text's semantics and readability and to conceal the attack to the greatest extent possible, the attacker attempts to attack sentences with lower semantic importance. This invention sets different weights for the number of bit errors during different single-sentence comparisons. The calculation of the bit error rate in step S7 is as follows:

[0054] Step S7-1: Use the TF-IDF algorithm to extract keywords from the entire text, and then calculate the relevance between the keywords of each sentence and the keywords in the text, denoted as Corr = {c r(1) ,c r(2) ,...,c r(m) If a keyword c in a certain sentence r(i) The larger the value, the more important the i-th sentence is to the text.

[0055] Step S7-2: Set the weight of the number of bit errors in each sentence during the comparison process as Wei = {c r(1) ,c r(2) ,...,c r(m) Let err be the number of bit errors in the i-th single sentence during the comparison process. i The formula for calculating the text bit error rate is as follows:

[0056]

[0057] Among them, c r(i) This represents the weight of the i-th sentence.

[0058] This invention employs the above technical solution, primarily aiming to protect text containing important information. Unlike watermark generation through controlling the output of a language generation model, this invention embeds watermarks by replacing related words of keywords in the original sentence with synonyms. Based on keywords and synonyms, it uses hash operations to binary encode the embedded watermark. This invention is more practically significant because, generally, we cannot access the structure and parameters of a language model. Modifying the output probability distribution of a language model can only verify whether a text contains a watermark, but cannot extract the watermark itself. This invention targets text watermarks, aiming to extract multi-bit watermark information.

[0059] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A text watermarking method based on synonym generation and hash operation, characterized in that: It includes the following steps: Step S1: For the i-th sentence Sen in the text i Find the keyword w s(j) ;w s(j) Indicates a simple sentence, Sen i The j-th word; Step S2: Determine the keyword w s(j) The word w in the adjacent position s(j+1) Is the part of speech of pos a member of the part-of-speech exclusion list? If not, then pos = j + 1. Otherwise, select Sen. i Chinese and keyword w s(j) The related word w with the strongest grammatical dependency s(x) Let pos = x; where w s(x) x represents the associated word with the highest implied score; w represents the associated word. s(x) Location; Step S3: For the selected word w s(pos) A pre-trained padding model is used to generate a candidate set of watermarks, resulting in the synonym set can_w; Step S4: Generate a numerical value r less than or equal to the candidate set length v using a random number generation method; select the synonym w indexed by r from the candidate set. cr Used to replace the selected related words. s(pos) The text T containing the "watermark" is obtained. ’ ; Step S5: For T ’ Each single sentence in the text (Sen) i ’ Keywords w s(j) and new related words w cr After concatenation, a hash calculation is performed. The resulting hash value is modulo 4 and converted to obtain the first binary code. This first binary code is then appended to the single sentence Sen. i ’ The end of the result is a text T containing a "watermark" and a hash value. ’’ In step S5, the single sentence Sen... i ’ The binary encoding process is to convert Sen... i ’ The keyword w s(j) With the replacement w s (pos) The word w cr Concatenate the data, then perform a SHA-256 hash calculation; binary... system The specific formula for the encoding algorithm is as follows: ； in Indicates the first The binary encoded information of each sentence Represents a hash function. Indicates keywords, Indicates that it is used to replace w s (pos) Synonyms It is a function that performs binary encoding on hash values; Step S6: Obtain suspicious text T sus The i-th suspicious sentence Sen was found sus_i The keywords and related terms in the text are subjected to the same hashing and modulo calculation to obtain the corresponding Sen. sus_i The second binary code; Step S7: Compare the second binary code with the first binary code and calculate the bit error rate; when the bit error rate is less than the threshold... In that case, ownership of the suspicious text is declared.

2. The text watermarking method based on synonym generation and hash operation according to claim 1, characterized in that: In step S1, the unsupervised method YAKE is used to extract the single sentence Sen. i Find the keyword w in s(j) .

3. The text watermarking method based on synonym generation and hash operation according to claim 1, characterized in that: The specific steps of step S2 are as follows: Step S2-1: First examine the keyword w s(j) Adjacent words w s(j+1) Determine the part of speech of the given word; check if the part of speech is a member of the ExList excluded by the part of speech list. s(j+1) If the part of speech is not a member of ExList, then let pos = j + 1 and end step S2; when w s(j+1) If the part of speech is a member of ExList, then proceed to step S2-2; Step S2-2: Use the spay model as a syntactic dependency component to calculate the keyword w. s(j) The implied score of the grammatical dependencies between the word and other words in the sentence is recorded in relation to the keyword w. s(j) There exists a word w with the highest implied score. s(x) Let pos = x at position x.

4. The text watermarking method based on synonym generation and hash operation according to claim 3, characterized in that: The part-of-speech exclusion list ExList in step S2-1 is {pronouns, prepositions, conjunctions, proper nouns, punctuation marks, classifiers, personal names, place names, and other proper terms}.

5. The text watermarking method based on synonym generation and hash operation according to claim 3, characterized in that: The implication score of the grammatical dependencies in step S2-2 is constructed by masking and replacing each type of dependency word using an imputation model, and then calculating the implication score of the original sentence using an NLI model.

6. The text watermarking method based on synonym generation and hash operation according to claim 1, characterized in that: In step S5, the first binary code at the end of each sentence will be used as a watermark code for subsequent text knowledge copyright verification.

7. The text watermarking method based on synonym generation and hash operation according to claim 1, characterized in that: In step S6, for the suspicious text T sus ={Sen sus_1 Sen sus_2 , ..., Sen sus_m Each sentence in} sus_i Use the YAKE method to find the keywords and corresponding related words in a sentence.

8. The text watermarking method based on synonym generation and hash operation according to claim 1, characterized in that: In step S7, different weights are set for the number of bit errors when comparing different single sentences.

9. The text watermarking method based on synonym generation and hash operation according to claim 1 or 8, characterized in that: The calculation of the bit error rate in step S7 is as follows: Step S7-1: Use the TF-IDF algorithm to extract keywords from the entire text, and then calculate the relevance between the keywords of each sentence and the keywords in the text, denoted as Corr={c r(1) ,c r(2) ,...,c r(m) }; Step S7-2: Set the weight of the number of bit errors in each sentence during the comparison process as Wei={c r(1) ,c r(2) ,...,c r(m) Let err be the number of bit errors in the i-th single sentence during the comparison process. i The formula for calculating the text bit error rate is as follows: ； Among them, c r(i) This represents the weight of the i-th sentence.

Images