Text watermarking method and device, storage medium and electronic equipment

Abstract

The invention discloses a text watermarking method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence and digital watermarking, and the method comprises the steps: obtaining an original text and a watermark, inputting the original text and the watermark into an embedded watermark model, and carrying out the watermark embedding operation to obtain a watermark text. The watermark embedding model comprises a gating modification mechanism, the gating modification mechanism is used for replacing a segmented word with a threshold value greater than a preset threshold value in the original text with a candidate segmented word with the same meaning as the segmented word, the watermark is embedded in the candidate segmented word, and the threshold value is used for representing the feasibility of embedding the watermark at the position of the segmented word. According to the technical scheme, through the gating modification mechanism and the candidate word segmentation generation strategy, the text modification degree is strictly controlled, the semantic coherence of the text embedded with the watermark is improved, and the watermark robustness is improved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and digital watermarking technology, and in particular to a text watermarking method, apparatus, storage medium and electronic device. Background Technology

[0002] Against the backdrop of rapid development in artificial intelligence, AI-generated content models are widely used in text creation. To prevent the spread of false content and enable traceability, watermarking technology can be used to add watermarks to the text content generated by AI-generated content models, achieving implicit identification and traceability of the model-generated text content.

[0003] Text watermarking technologies can be broadly categorized into three types. Traditional statistical analysis methods embed watermarks by altering the statistical characteristics of the generated text, such as adjusting word frequency distribution, sentence length, and character frequency. Natural language processing (NLP) methods utilize language models and semantic analysis to embed watermarks through synonym substitution, sentence transformation, and semantic rewriting. Deep learning-based methods achieve text watermark embedding and detection through end-to-end model architectures. For example, the Adversarial Watermarking Transformer (AWT) method employs an encoder-decoder framework, embedding watermark information while maintaining semantic consistency through joint training. Postmark methods use modular post-processing watermarking schemes to insert specific word sets after text generation to achieve watermark embedding.

[0004] The aforementioned text watermarking technologies suffer from several drawbacks. Traditional statistical analysis methods generate text whose statistical characteristics change after format conversion and content editing, leading to watermark information loss or incorrect extraction. Furthermore, these methods are poorly adaptable to diverse text types. Natural language processing (NLP) methods are constrained by the rules and semantics of natural language expression. The AWT method has limited capacity for embedding watermark information, and even slight semantic deviations or text processing differences can significantly reduce extraction accuracy. Deep learning-based methods exhibit low accuracy when dealing with longer watermark information. The PostMark method is limited by text semantics and structure, resulting in low embedding capacity, and the inserted words can easily disrupt semantic coherence, affecting text quality. Summary of the Invention

[0005] In view of this, this application provides a text watermarking method, apparatus, storage medium, and electronic device to improve the semantic coherence of the text after watermarking and to improve the robustness of the watermark.

[0006] Firstly, this application provides a text watermarking method, including:

[0007] Get the original text and watermark;

[0008] The original text and the watermark are input into the watermarking model to perform watermark embedding operation to obtain watermarked text; the watermarking model includes a gating modification mechanism, which is used to replace words in the original text with a threshold value greater than a preset threshold value with candidate words with the same semantics as the word segmentation, and the watermark is embedded in the candidate words. The threshold value is used to characterize the feasibility of embedding the watermark at the word segmentation position.

[0009] In some embodiments of this disclosure, the watermark embedding operation includes:

[0010] The original text is encoded to obtain the text representation information, which is word-level semantic information of the original text that incorporates contextual associations;

[0011] The watermark is encoded, and the watermark interacts with the text representation information based on a cross-attention mechanism to obtain the watermark representation information, which is a watermark feature that is fused with the original text context.

[0012] The watermark text is generated based on the text representation information, the watermark representation information, and the gating modification mechanism.

[0013] In some embodiments of this disclosure, the step of encoding the original text is preceded by:

[0014] The original text is segmented into multiple words;

[0015] Add text position encoding to the word segmentation, the text position encoding being used to represent the word segmentation sequence;

[0016] Map the watermark to the same dimension as the word segmentation;

[0017] Add a watermark position code to the watermark, the watermark position code being used to characterize the watermark sequence;

[0018] Convert the word segmentation and the watermark into vector form.

[0019] In some embodiments of this disclosure, generating the watermark text based on the text representation information, the watermark representation information, and the gating modification mechanism includes:

[0020] A watermark distribution map is constructed based on the text representation information and the watermark representation information, and the watermark distribution map includes the association probability between the watermark and the word segmentation in the original text;

[0021] The word segments in the original text are traversed in descending order of the association probabilities shown in the watermark distribution diagram.

[0022] Generate candidate words with the same semantics as the previously defined word segmentation;

[0023] The word segmentation threshold is calculated based on the text representation information and the watermark representation information;

[0024] In response to the threshold value being greater than a preset threshold, the word segment is replaced with the candidate word segment in the original text, and the watermark is embedded in the candidate word segment to obtain watermarked text.

[0025] In some embodiments of this disclosure, generating candidate words with the same semantic meaning as the word segmentation includes:

[0026] Generate multiple candidate words with the same semantics as the first word segmentation;

[0027] The language model generation probability, semantic similarity, and watermark alignment contribution of the candidate word segment are calculated. The language model generation probability is used to characterize the fluency of the candidate word segment in the original text, the semantic similarity is used to characterize the semantic similarity between the candidate word segment and the original word segment, and the watermark alignment contribution is used to characterize the association probability between the candidate word segment and the watermark.

[0028] The applicability of the candidate word segmentation is calculated based on the generation probability of the language model, the semantic similarity of the word segmentation, and the watermark alignment contribution.

[0029] The candidate word segments are sorted in descending order according to their applicability, and the first preset number of candidate word segments are selected.

[0030] In some embodiments of this disclosure, the step of replacing the word segmentation in the original text with the candidate word segmentation and embedding a watermark in the candidate word segmentation to obtain watermarked text includes:

[0031] The original text is replaced with the candidate words, and the watermark is embedded in the candidate words to obtain multiple candidate texts;

[0032] The language model score, text semantic similarity, and watermark alignment score of the candidate text are calculated. The language model score is used to characterize the fluency of the candidate text, the text semantic similarity is used to characterize the semantic similarity between the candidate text and the original text, and the watermark alignment score is used to characterize the degree of embedding of the watermark in the candidate text.

[0033] A comprehensive score is calculated based on the language model score, the text semantic similarity score, and the watermark alignment score.

[0034] The candidate text with the highest comprehensive score is selected as the watermark text.

[0035] In some embodiments of this disclosure, the embedded watermarking model is trained using a multi-task loss function; the multi-task loss function includes text reconstruction loss, watermark alignment loss, and semantic preservation loss, wherein the text reconstruction loss is used to guide the text reconstruction capability of the embedded watermarking model, the watermark alignment loss is used to guide the watermark information preservation capability of the embedded watermarking model, and the semantic preservation loss is used to guide the semantic consistency of the embedded watermarking model.

[0036] In some embodiments of this disclosure, the method further includes:

[0037] The watermarked text is input into the watermark extraction model to perform watermark extraction, and the watermark is obtained. The watermark extraction model is a pre-trained neural network model for watermark extraction.

[0038] In some embodiments of this disclosure, the watermark extraction operation includes:

[0039] Decode the watermarked text to predict the initial watermark;

[0040] Calculate the confidence level of the initial watermark, whereby the confidence level is used to characterize the confidence probability of the predicted initial watermark;

[0041] In response to the confidence level being less than a preset confidence threshold, the initial watermark is corrected according to the watermark structure rules to obtain the watermark.

[0042] In some embodiments of this disclosure, the method includes:

[0043] Acquire training data, which includes the watermark to be embedded, text without watermark, and watermarked text with embedded watermark output by the watermark embedding model.

[0044] The watermark extraction model is trained based on the training data using a loss function, which includes a binary cross-entropy loss and a confidence loss. The binary cross-entropy loss is used to optimize the accuracy of the watermark prediction by the watermark extraction model, and the confidence loss term is used to optimize the reliability of the watermark prediction by the watermark extraction model. During the training process of the watermark extraction model, text perturbation processing is performed on the watermark text with embedded watermarks. The text perturbation processing includes at least one of text replacement, text deletion, and text rewriting.

[0045] Secondly, this application provides a text watermarking device, comprising:

[0046] The acquisition module is used to acquire the original text and watermark;

[0047] The watermark module is used to embed the original text and the watermark input into the watermark model to perform watermark embedding operations and obtain watermarked text. The watermark embedding model includes a gating modification mechanism, which is used to replace word segments in the original text with a threshold value greater than a preset threshold value with candidate word segments that have the same semantics as the word segments. The watermark is embedded in the candidate word segments, and the threshold value is used to characterize the feasibility of embedding the watermark at the word segment position.

[0048] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0049] Fourthly, this application provides an electronic device including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0050] Fifthly, this application provides a computer program product having a computer program stored thereon, wherein the computer program product, when executed by a processor, implements the method described in the first aspect.

[0051] By employing the above technical solutions, this application provides a text watermarking method, apparatus, storage medium, and electronic device. The method includes: acquiring original text and a watermark; inputting the original text and watermark into an embedding watermark model; performing a watermark embedding operation to obtain watermarked text. The embedding watermark model includes a gating modification mechanism, which replaces word segments in the original text with a threshold greater than a preset threshold with candidate word segments that have the same semantic meaning. The watermark is embedded in the candidate word segments, and the threshold is used to characterize the feasibility of embedding the watermark at the word segmentation position. In the technical solution of this application, the degree of text modification is strictly controlled through the gating modification mechanism and the candidate word segmentation generation strategy, thereby improving the semantic coherence of the text after watermark embedding and enhancing the robustness of the watermark.

[0052] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This illustration shows a schematic diagram of watermark embedding provided in an embodiment of this application;

[0056] Figure 2 A schematic diagram of the encoding process provided in an embodiment of this application is shown;

[0057] Figure 3 This document illustrates a flowchart of the candidate word segmentation replacement process provided in an embodiment of this application.

[0058] Figure 4 This illustration shows a flowchart of the candidate word segmentation generation process provided in an embodiment of this application;

[0059] Figure 5 A schematic diagram of the process for generating watermarked text provided in an embodiment of this application is shown;

[0060] Figure 6 This illustration shows a schematic diagram of watermark extraction provided in an embodiment of this application;

[0061] Figure 7 A flowchart illustrating the watermark extraction operation provided in an embodiment of this application is shown.

[0062] Figure 8 The system architecture diagram of the multi-bit text watermarking system based on the encoder-decoder architecture provided in this application embodiment is shown. Detailed Implementation

[0063] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0064] Against the backdrop of rapid development in artificial intelligence, AI-generated content models are widely used in text creation. To prevent the spread of false content and enable traceability, watermarking technology can be used to add watermarks to the text content generated by AI-generated content models, achieving implicit identification and traceability of the model-generated text content.

[0065] Traditional statistical analysis methods embed watermarks by altering the statistical characteristics of the generated text, such as adjusting word frequency distribution, sentence length, and character frequency. For example, watermark information can be encoded into the frequency variations of specific words by pre-setting frequency patterns; or watermark information can be hidden by controlling the distribution of sentence length.

[0066] Natural language processing-based methods utilize language models and semantic analysis to embed watermarks through synonym replacement, sentence transformation, and semantic rewriting. During the text generation process of AI-generated content models, pre-trained language models are used to replace original words with semantically similar synonyms or restructure sentences, hiding watermark information while preserving semantic meaning.

[0067] Deep learning-based methods have been applied in the field of generative artificial intelligence models for text watermarking, achieving text watermark embedding and detection through an end-to-end model architecture. For example, the Adversarial Watermarking Transformer (AWT) method uses an encoder-decoder framework and embeds watermark information while maintaining the semantic consistency of the text through joint training; the PostMark method uses a modular post-processing watermarking scheme to insert a specific vocabulary set after text generation to achieve watermark embedding.

[0068] In the above embodiments, traditional statistical analysis methods have poor robustness, and the generated text is easily subject to format conversion and content editing in practical use. Once these transformations occur, the statistical characteristics of the text will change significantly, leading to the loss of watermark information or its inability to be extracted correctly. For example, after simple paragraph deletion or word replacement in the generated text, the accuracy of watermarks embedded based on statistical characteristics drops sharply, making it impossible to trace the source of the text output by the AI-generated content model. Furthermore, statistical analysis methods are poorly adaptable to the diverse text output by AI-generated content models, making it difficult to meet the needs of complex text scenarios.

[0069] Due to the constraints of natural language expression rules and semantics, the capacity for embedding watermark information using natural language processing methods is very limited. Furthermore, due to the complexity and diversity of natural language, even slight semantic deviations or differences in text processing can significantly reduce extraction accuracy during watermark extraction.

[0070] The AWT method has a limited watermark embedding length, resulting in low watermark extraction accuracy when dealing with longer watermark information. Due to text semantics and structure limitations, the PostMark method restricts the number and position of inserted words, leading to low embedding capacity. Furthermore, inserted words can easily disrupt the semantic coherence of the text, affecting the quality of the generated text.

[0071] Therefore, in order to solve the above problems, this embodiment provides a text watermarking method, such as... Figure 1 The diagram shown is a schematic representation of the watermark embedding process provided in an embodiment of this application. The method includes:

[0072] S101, retrieve the original text and watermark.

[0073] S102: Embed the original text and watermark into the watermark model, perform watermark embedding operation, and obtain the watermark text.

[0074] Among them, the watermark embedding model is a pre-trained neural network model for embedding watermarks. The watermark embedding model includes a gating modification mechanism, which is used to replace words in the original text with a gate value greater than a preset threshold with candidate words that have the same semantics as the word segmentation. The watermark is embedded in the candidate words, and the gate value is used to characterize the feasibility of embedding the watermark at the word segmentation position.

[0075] During the watermark embedding process, the degree of text modification is strictly controlled through a gating modification mechanism and a candidate word segmentation generation strategy, thereby enhancing the semantic coherence of the watermarked text and improving the robustness of the watermark.

[0076] In some embodiments, the watermark embedding model employs an encoder-decoder architecture, where the encoder encodes the original text and the watermark to be embedded in parallel, and the decoder generates the watermarked text while ensuring semantic naturalness. This deep neural network structure integrates cross-modal attention mechanisms, gating modification mechanisms, and constrained bundle decoding strategies to ensure that the watermark embedding is stable, flexible, and semantically faithful.

[0077] In some embodiments, during the input processing stage of the watermark embedding model, the original text can be segmented into multiple words, and text position encoding can be added to each word segment to represent the word segment sequence. The watermark is mapped to the same dimension as the word segmentation, and watermark position encoding is added to the watermark to represent the watermark sequence. The word segmentation and watermark are then converted into vector form for subsequent watermark embedding operations.

[0078] In this embodiment, the embedded watermark model can segment the input raw text using a word segmenter, such as Byte Pair Encoding (BPE) or WordPiece Tokenization (WordPiece), and convert the segmented words into embedded vectors. Positional encoding is added to preserve word order information. The input watermark (such as a binary watermark bit sequence) is also embedded as a fixed-dimensional vector, mapped to the same dimension as the text embedding vector through linear projection, and positional encoding is added to the watermark bits to maintain the sequence structure of the watermark information.

[0079] For example, for the original text "I love eating apples", the embedded watermarking model first uses a word segmenter to divide the original text into "I", "love", "eat", and "apple" words, and then converts each word into a string of numbers (vectors) that can be recognized by the machine. At the same time, each word is coded with a position tag, for example, "I" is tagged as the 1st position and "love" is tagged as the 2nd position.

[0080] For the watermark to be embedded, the watermark can be a binary watermark bit sequence, such as "010110". The watermark embedding model converts the watermark into a number string with the same length as the text embedding vector through linear projection, and also marks the position encoding of each watermark bit.

[0081] In some embodiments, for the encoding module, the embedded watermark model includes a text encoder and a watermark encoder. For example... Figure 2 The diagram illustrates the encoding process provided in this embodiment. The text encoder encodes the original text to obtain text representation information, which is word-level semantic information fused with contextual association in the original text (S201). The watermark encoder encodes the watermark and, based on a cross-attention mechanism, enables the watermark to interact with the text representation information to obtain watermark representation information (S202). The watermark representation information is watermark features fused with the contextual association of the original text. The subsequent watermark embedding model can generate watermarked text based on the text representation information, watermark representation information, and a gating modification mechanism.

[0082] In some embodiments, the text encoder employs a multi-layer Transformer encoder to perform contextual encoding on the original text. Each layer contains a multi-head self-attention and feedforward network module, and combines residual connections and layer normalization mechanisms to generate word-level text representation information that preserves contextual information. For example, taking the original text mentioned above, the text representation information generated by the text encoder includes the fact that "apple" is a fruit in "I love eating apples".

[0083] It should be noted that the embodiments of this application do not specifically limit the number of layers of the Transformer encoder, which can be set according to actual needs, for example, it can be set to 6 layers.

[0084] In some embodiments, the watermark encoder is a standalone Transformer encoder that introduces a cross-attention mechanism to allow the watermark vector to interact with the text representation information, outputting a watermark representation information that incorporates contextual information. This watermark representation information reveals where the watermark is best embedded in the original text or what semantic relationships it is associated with.

[0085] In some embodiments, to accurately determine where each watermark bit should be embedded in the original text, such as... Figure 3As shown, the embedded watermark model sets up a text watermark fusion model, which can construct a watermark distribution map based on text representation information and watermark representation information (S301). The watermark distribution map includes the association probability between the watermark and word segments in the original text.

[0086] In the text watermark fusion stage, the watermark embedding model can construct the correspondence between text representation information and watermark representation information through a cross-attention mechanism. Specifically, using the watermark embedding vector as the query and the text embedding vector as the key / value pair, the attention distribution of which text positions each watermark bit should be embedded in is calculated. Based on the attention distribution, a watermark distribution map is constructed to predict the probability of the preferred embedding position for each watermark bit, thus providing embedding targets for the rewriting in the decoder stage.

[0087] By calculating the attention distribution, a watermark distribution map is generated, clarifying the association probability between each watermark bit and each text position in the original text. For example, the association probability between the first watermark bit and the text position "apple" is 80%, and the association probability with the text position "eat" is 15%. This helps the model understand which text positions are suitable for embedding the corresponding watermark bits, providing a basis for subsequent watermark embedding.

[0088] In some embodiments, such as Figure 3 As shown, after obtaining the watermark distribution map, the words in the original text can be traversed in descending order of association probability in the watermark distribution map (S302) to generate candidate words with the same semantics as the original word (S303). The threshold value of the word segmentation is calculated based on the text representation information and the watermark representation information (S304). If the threshold value is less than or equal to the preset threshold, the word segmentation is not replaced, and the step of traversing the words in the original text continues to the next word segmentation. If the threshold value is greater than the preset threshold, the original word segmentation is replaced with the candidate word segmentation, and the watermark is embedded in the candidate word segmentation to obtain the watermarked text (S305).

[0089] In the above embodiments, the watermark distribution map indicates the text position where the watermark should be embedded, the candidate word segmentation generation part provides candidate word segmentation materials for the text position, and the gating modification mechanism determines whether the text position should be replaced by candidate word segmentation, thereby ensuring the accuracy of watermark embedding and the semantic coherence of the text.

[0090] In some embodiments, for the generation of candidate words, the watermark embedding module generates several semantically similar candidate words at each text position, and evaluates the applicability of these candidate words by combining the generation probability of the comprehensive language model, the semantic similarity of the words, and their alignment contribution to the target watermark bits.

[0091] That is, Figure 4The diagram shows a flowchart of candidate word segmentation generation provided in this embodiment. For each traversed word, multiple candidate words with the same semantics are generated (S401). The language model generation probability, semantic similarity, and watermark alignment contribution of the candidate words are calculated (S402). The applicability of the candidate words is calculated based on the language model generation probability, semantic similarity, and watermark alignment contribution (S403). The multiple candidate words are sorted in descending order according to their applicability, and the first preset number of candidate words are selected (S404) for subsequent word replacement.

[0092] Among them, the language model generation probability is used to characterize the fluency of the candidate word segment in the original text, that is, whether the candidate word segment is fluent in the original text; the word segment semantic similarity is used to characterize the semantic similarity between the candidate word segment and the original word, that is, whether the candidate word segment is semantically similar to the original word; and the watermark alignment contribution is used to characterize the association probability between the candidate word segment and the watermark, that is, whether the candidate word segment can stably embed the watermark.

[0093] In some embodiments, the Top-K candidate method can be used to select the top K candidate words with the highest comprehensive scores from multiple candidate words. For example, K=5 can be set to select the top 5 candidate words with the highest comprehensive scores.

[0094] In some embodiments, a gating modification mechanism is used to determine whether to replace the word segment with the candidate word segment. The gating function in the embedded watermark module is set as shown below:

[0095] gate i =sigmoid(W g ·[h i ;w j ]+b g );

[0096] Among them, gate i W represents the gate value at the i-th text position in the original text. g b is a learnable weight matrix g For learnable bias terms, h i w represents the text representation information at the i-th text position. j The watermark representation information is represented by the i-th watermark bit.

[0097] The aforementioned gating function calculates a learnable threshold for each text location to determine whether to modify that location. The threshold is jointly determined by the text representation information and the watermark representation information. Only when the threshold exceeds a preset threshold (e.g., 0.7) is candidate word replacement allowed for that text location.

[0098] The core of the gating modification mechanism is to determine whether the text position should be modified by using a threshold value. When the threshold value exceeds the preset threshold, it means that the text position is suitable for modification, and candidate word replacement is allowed. When the threshold value is lower than or equal to the preset threshold, it means that the text position is not suitable for modification, and even if there are semantically similar candidate words, candidate word replacement is not allowed.

[0099] For example, if the watermark distribution map indicates that the position of "apple" in the original text has the strongest correlation probability with the first position of the watermark, then the candidate word "red apple" with semantic similarity to "apple" is generated. The threshold value of the position of "apple" is calculated based on the gating function. If the threshold value is greater than the preset threshold of 0.7, then "apple" in the original text is modified to "red apple", and the first position of the watermark is embedded in "red apple".

[0100] In some embodiments, such as Figure 5 The diagram illustrates the process of generating watermarked text according to an embodiment of this application. For multiple candidate word segments, the original text is replaced with the candidate word segments, and a watermark is embedded in the candidate word segments to obtain multiple candidate texts (S501). The language model score, text semantic similarity, and watermark alignment score of the candidate texts are calculated (S502). A comprehensive score is calculated based on the language model score, text semantic similarity, and watermark alignment score (S503), and the candidate text with the highest comprehensive score is selected as the watermarked text (S504).

[0101] Among them, the language model score is used to characterize the fluency of the candidate text, the text semantic similarity is used to characterize the semantic similarity between the candidate text and the original text, and the watermark alignment score is used to characterize the degree of embedding of the watermark in the candidate text, that is, whether the watermark can be stably embedded in the candidate text.

[0102] Understandably, the gating modification mechanism allows the modified text positions to generate multiple candidate word segments, and these replacements combine to form multiple candidate texts. The core of the decoding stage is to select from these candidate texts the candidate text that can stably embed the watermark, is fluent, and closely matches the semantics of the original text as the final watermark text.

[0103] In some embodiments, the embedded watermarking model can filter the final watermark text based on constrained beam search. During the generation of the final watermark text, the embedded watermarking model employs a constrained beam search algorithm. This decoding strategy retains B candidate text sequences (e.g., setting B=10) and sorts the candidate text sequences according to a multi-metric scoring function. The scoring function is shown below:

[0104] score=α·LM score +β·watermark alignment +γ·sematicsimilarity ;

[0105] Among them, LM score Watermark for language model scoring alignment For watermark alignment scores, sematic similarity Let represent the semantic similarity of the text, α, β, and γ be the weights of each factor, and γ be an adjustable hyperparameter.

[0106] In some embodiments, during the training phase of the embedded watermarking model, a multi-task joint training objective function can be adopted, i.e., the embedded watermarking model can be trained using a multi-task loss function. This multi-task loss function includes text reconstruction loss, watermark alignment loss, and semantic preservation loss. The text reconstruction loss guides the text reconstruction capability of the embedded watermarking model, the watermark alignment loss guides the watermark information preservation capability, and the semantic preservation loss guides the semantic consistency of the embedded watermarking model.

[0107] The above embodiments provide a technical solution for watermark embedding operations. This application also provides a technical solution for watermark extraction operations. In some embodiments, a watermark extraction model can be set, which is a pre-trained neural network model for watermark extraction. For example... Figure 6 The diagram shown is a schematic diagram of watermark extraction provided in the embodiment of this application. The watermark text is obtained (S601), the watermark text is input into the watermark extraction model, and the watermark extraction operation is performed (S602) to obtain the watermark.

[0108] In some embodiments, the watermark extraction model employs an encoder-decoder architecture. The main task of the deep neural network is to recover the original watermark bit information from the watermarked text with high accuracy without the aid of the original text. Its focus is on pattern recognition and bit order inference of the watermark signal.

[0109] In some embodiments, during the input processing stage of the watermark extraction model, the input watermarked text is first transformed into an embedding vector representation using a word segmentation and positional encoding strategy consistent with the embedding deep neural network. Subsequently, the watermarked text is fed into a context encoder consisting of multiple Transformer layers, specifically optimized to be more sensitive to the watermark embedding signal. The representation of each text position output by the encoder will contain the potential watermark trace.

[0110] In some embodiments, such as Figure 7The diagram shows a watermark extraction process provided in this embodiment. The watermark extraction model decodes the watermark text and predicts the initial watermark (S701). The confidence level of the initial watermark is calculated (S702). The confidence level characterizes the confidence probability of the predicted initial watermark, such as 0.5. If the confidence level is greater than or equal to a preset confidence threshold, no modification is made. If the confidence level is less than the preset confidence threshold, the initial watermark is corrected according to watermark structure rules (such as watermark length, parity check, sequence dependency, etc.) to obtain the watermark (S703).

[0111] In this embodiment, a watermark location detection module is set up to extract the watermark model and a feature extraction path is established to extract the pattern left by the watermark embedding vector. Several convolutional filters are used to identify the local signals of each word in the watermarked text, and a multi-head attention mechanism is used to aggregate relevant information from the global context.

[0112] Building upon this, the watermark extraction model also constructs a sequential bit decoder (watermark bit decoder), which predicts the initial watermark information bit by bit based on the Transformer decoder structure. Each bit is converted from the current decoder state to a 0 / 1 probability prediction using a softmax function, where the weight parameters are optimized through training to improve prediction accuracy.

[0113] To enhance the robustness of watermark extraction, a confidence estimation module is included in the watermark extraction model to quantify the reliability of each predicted watermark bit. Specifically, the confidence score is calculated as the degree of uncertainty of the bit deviating from a preset probability (e.g., 0.5). This score can be used to weight, correct, and post-process the extraction results. Furthermore, the watermark extraction model incorporates the Viterbi algorithm to perform structural decoding of the entire watermark bit sequence, combined with prior knowledge of watermark structure, such as parity checks, to improve the accuracy of watermark bit extraction.

[0114] To address this, the watermark extraction model can employ a sequential bit decoder to decode the watermark text, predict the initial watermark, and use confidence estimation and error correction mechanisms. By quantifying the confidence probability of each bit of the predicted binary watermark, weighting and correcting the result, an accurate watermark can be output.

[0115] For example, the initial watermark bit sequence predicted by the sequential bit decoder is "0, 1, 1", with confidence levels of 0.9, 0.85, and 0.6 for each watermark bit. The confidence estimation module quantifies reliability using 0.5 as the uncertainty threshold; the further the confidence level of a watermark bit deviates from 0.5, the higher the reliability. For the third watermark bit in the sequence, its confidence level is low. Therefore, the Viterbi algorithm and watermark structure rules are used to correct the low-confidence watermark bit. For example, based on watermark structure rules such as a watermark length of 3 and an odd number of 1s, the watermark bit sequence can be corrected to "0, 1, 0" as the final watermark. This ensures that the final generated watermark sequence conforms to both local traces and global structural rules, improving the robustness of watermark extraction.

[0116] In some embodiments, during the watermark extraction model training phase, the deep neural network is trained under supervision using watermarked text and watermarks generated by embedding the deep neural network. The training data includes various text transformation styles, allowing for the random addition of perturbations to the embedded watermarked text, including replacement, deletion, and rewriting, to simulate the variability and attack possibilities in real-world application scenarios, thereby enhancing the accuracy and robustness of the watermark extraction model. Additional counterexamples without embedded watermarks can also be added to improve the system's discriminative ability. The overall loss function consists of a binary cross-entropy loss and a confidence loss term, comprehensively optimizing extraction accuracy and reliability.

[0117] This involves acquiring training data, which includes the watermark to be embedded, the text without an embedded watermark, and the watermarked text output by the watermarking model. Based on a loss function, a watermark extraction model is trained using the training data. The loss function includes binary cross-entropy loss and confidence loss. The binary cross-entropy loss optimizes the accuracy of the watermark prediction by the extraction model, while the confidence loss optimizes the reliability of the prediction. During the training process, text perturbation processing is performed on the watermarked text with embedded watermarks. This text perturbation processing includes at least one of text replacement, text deletion, and text rewriting.

[0118] Understandably, the watermark embedding model and the watermark extraction model can be trained simultaneously. The original text without an embedded watermark and the watermark information to be embedded can be used as input to the watermark embedding model, while the watermarked text output by the watermark embedding model can be used as input to the watermark extraction model. The loss of the watermark extraction model can be used as part of the loss of the watermark embedding model, promoting the generation of watermarked text with a high watermark extraction rate during the iterative optimization process.

[0119] Based on the above method, this embodiment proposes a multi-bit text watermarking system based on an encoder-decoder architecture. The aim is to embed multi-bit watermark information into text content generated by an AI-generated content model while maintaining semantic naturalness, and to extract the watermark content from the embedded watermark information with high accuracy. The system architecture diagram is shown below. Figure 8 As shown, the overall structure consists of two main deep neural network components: an embedding deep neural network and an extraction deep neural network. Both are built on the Transformer architecture and have been customized and optimized in terms of structure and objective function for watermark embedding and extraction tasks, respectively. The embedding deep neural network is responsible for integrating the multi-bit binary watermark information to be embedded into the original text generation process, achieving a covert and semantically consistent rewriting. The extraction deep neural network is used to recover the original watermark information from the rewritten text without the assistance of the original text. The entire system is designed with robustness of embedding, semantic consistency, and extractability as its core principles, and is suitable for implicit identification and source tracing applications of text generated by AI-generated content models. It achieves the goal of adding watermarks to text generated by AI-generated content models and enabling source tracing.

[0120] First, by optimizing the encoder-decoder architecture and employing innovative watermark embedding and extraction strategies, the stability of the watermark is enhanced under various text transformation conditions, ensuring that the watermark can be completely preserved in complex text processing environments. Second, a text watermark fusion model is used to achieve efficient embedding of multi-bit watermarks. Then, a deep neural network for watermark extraction and a confidence evaluation mechanism are designed to improve the accuracy of watermark information extraction, providing accurate and effective evidence for text tracing and source identification. Finally, during the watermark embedding process, a gating modification mechanism and a candidate word segmentation generation strategy are used to strictly control the degree of text modification, ensuring that the text after watermark embedding maintains good semantic coherence and does not affect text quality or user experience.

[0121] Compared to existing text watermarking technologies, this application proposal has significant technical advantages, particularly in adding watermarks and tracing the source of text generated by AI-generated content models. First, it improves embedding capacity. Based on an encoder-decoder framework and text watermark fusion technology, it can efficiently embed multi-bit watermarks into text generated by AI-generated content models. Second, it significantly enhances robustness. Innovative watermark embedding and extraction strategies ensure the watermark remains stable under various text transformations, even after multiple edits and format conversions. Third, it maintains good semantic consistency. The use of candidate word segmentation and gating modification mechanisms ensures the semantic coherence of the AI-generated content model-generated text after watermark embedding, without affecting text quality or user experience. Fourth, it improves extraction accuracy. Confidence estimation and error correction methods enable accurate extraction of the watermark from the AI-generated content model-generated text even in complex environments, providing reliable assurance for tracing the source of AI-generated text and accurately determining the text's origin and related information.

[0122] The technical solution provided in this application can safeguard content copyright and ensure accountability: For content creators and enterprises, this proposal clarifies the ownership of generated content, providing strong evidence in copyright disputes. When plagiarism or infringement occurs, watermarking can quickly trace the source to the specific generation model and related usage records, thereby protecting the legitimate rights and interests of creators and reducing the cost and uncertainty of copyright litigation. It can also enhance the security and credibility of content platforms: Content platforms can effectively prevent the spread of false information and harmful content by leveraging the watermarking technology proposed in this application. For example, social media platforms can screen and control potentially risky information by identifying watermarks in AI-generated content, ensuring a healthy platform ecosystem, increasing user trust in the quality of platform information, and thus enhancing platform competitiveness and user stickiness. Simultaneously, in industries such as finance and healthcare, where the accuracy and security of information are extremely important, watermarking technology can prevent serious consequences from AI-generated errors or misleading information, ensuring the reliability of critical business data. Finally, it can promote the standardization and innovation of the AI ​​model market: In the AI ​​model trading market, the watermarking technology proposed in this application can protect the rights and interests of model developers. Model developers can use watermarks to track model usage, preventing illegal copying, resale, or misuse, and protecting their intellectual property and commercial interests.

[0123] Based on the above method, this embodiment also provides a text watermarking device, including an acquisition module and a watermark module, wherein the acquisition module is used to acquire the original text and the watermark.

[0124] The watermarking module is used to embed the original text and watermark input into the watermarking model, perform watermark embedding operations, and obtain the watermarked text. The watermark embedding model includes a gating modification mechanism, which replaces word segments in the original text with a threshold value greater than a preset threshold with candidate word segments that have the same semantics as the word segments. The watermark is embedded in the candidate word segments, and the threshold value is used to characterize the feasibility of embedding the watermark at the word segmentation position.

[0125] It should be noted that the device structure provided in this embodiment does not constitute a limitation on the device, and may include more or fewer component modules, or combine certain modules, or have different module arrangements. Other corresponding descriptions of the functional units involved in the text watermarking device provided in this embodiment can be found in the corresponding descriptions in the text watermarking method, and will not be repeated here.

[0126] Based on the above method, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which implements the above method when executed by a processor.

[0127] Based on the above method, this embodiment also provides a computer program product on which a computer program is stored, and which implements the above method when executed by a processor.

[0128] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0129] Based on the above method, this application embodiment also provides an electronic device, such as a terminal device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the above method.

[0130] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0131] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0132] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented using hardware. Compared with the current related technologies, by applying the technical solution of this embodiment, while maintaining semantic naturalness, it is possible to embed multi-bit watermark information into text content generated by an artificial intelligence-generated content model, and to extract watermark content from the text with embedded watermark information with high accuracy.

[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A text watermarking method, characterized in that, include: Get the original text and watermark; The original text and the watermark are input into the watermark model, and the watermark embedding operation is performed to obtain the watermarked text. The embedded watermark model includes a gating modification mechanism, which is used to replace word segments in the original text with a threshold value greater than a preset threshold value with candidate word segments that have the same semantics as the word segments. The watermark is embedded in the candidate word segments, and the threshold value is used to characterize the feasibility of embedding the watermark at the word segment position.

2. The method according to claim 1, characterized in that, The watermark embedding operation includes: The original text is encoded to obtain the text representation information, which is word-level semantic information of the original text that incorporates contextual associations; The watermark is encoded, and the watermark interacts with the text representation information based on a cross-attention mechanism to obtain the watermark representation information, which is a watermark feature that is fused with the original text context. The watermark text is generated based on the text representation information, the watermark representation information, and the gating modification mechanism.

3. The method according to claim 2, characterized in that, Before the step of encoding the original text, the following steps are included: The original text is segmented into multiple words; Add text position encoding to the word segmentation, the text position encoding being used to represent the word segmentation sequence; Map the watermark to the same dimension as the word segmentation; Add a watermark position code to the watermark, the watermark position code being used to characterize the watermark sequence; Convert the word segmentation and the watermark into vector form.

4. The method according to claim 2, characterized in that, The step of generating the watermark text based on the text representation information, the watermark representation information, and the gating modification mechanism includes: A watermark distribution map is constructed based on the text representation information and the watermark representation information, and the watermark distribution map includes the association probability between the watermark and the word segmentation in the original text; The word segments in the original text are traversed in descending order of the association probabilities shown in the watermark distribution diagram. Generate candidate words with the same semantics as the previously defined word segmentation; The word segmentation threshold is calculated based on the text representation information and the watermark representation information; In response to the threshold value being greater than a preset threshold, the word segment is replaced with the candidate word segment in the original text, and the watermark is embedded in the candidate word segment to obtain watermarked text.

5. The method according to claim 4, characterized in that, The generation of candidate words with the same semantics as the word segmentation includes: Generate multiple candidate words with the same semantics as the first word segmentation; The language model generation probability, semantic similarity, and watermark alignment contribution of the candidate word segment are calculated. The language model generation probability is used to characterize the fluency of the candidate word segment in the original text, the semantic similarity is used to characterize the semantic similarity between the candidate word segment and the original word segment, and the watermark alignment contribution is used to characterize the association probability between the candidate word segment and the watermark. The applicability of the candidate word segmentation is calculated based on the generation probability of the language model, the semantic similarity of the word segmentation, and the watermark alignment contribution. The candidate word segments are sorted in descending order according to their applicability, and the first preset number of candidate word segments are selected.

6. The method according to claim 4, characterized in that, The step of replacing the word segmentation with the candidate word segmentation in the original text, and embedding a watermark in the candidate word segmentation to obtain watermarked text, includes: The original text is replaced with the candidate words, and the watermark is embedded in the candidate words to obtain multiple candidate texts; The language model score, text semantic similarity, and watermark alignment score of the candidate text are calculated. The language model score is used to characterize the fluency of the candidate text, the text semantic similarity is used to characterize the semantic similarity between the candidate text and the original text, and the watermark alignment score is used to characterize the degree of embedding of the watermark in the candidate text. A comprehensive score is calculated based on the language model score, the text semantic similarity score, and the watermark alignment score. The candidate text with the highest comprehensive score is selected as the watermark text.

7. The method according to claim 1, characterized in that, The method includes: The embedded watermarking model is trained using a multi-task loss function, which includes text reconstruction loss, watermark alignment loss, and semantic preservation loss. The text reconstruction loss guides the text reconstruction capability of the embedded watermarking model, the watermark alignment loss guides the watermark information preservation capability of the embedded watermarking model, and the semantic preservation loss guides the semantic consistency of the embedded watermarking model.

8. The method according to claim 1, characterized in that, The method further includes: The watermarked text is input into the watermark extraction model to perform watermark extraction, and the watermark is obtained. The watermark extraction model is a pre-trained neural network model for watermark extraction.

9. The method according to claim 8, characterized in that, The watermark extraction operation includes: Decode the watermarked text to predict the initial watermark; Calculate the confidence level of the initial watermark, whereby the confidence level is used to characterize the confidence probability of the predicted initial watermark; In response to the confidence level being less than a preset confidence threshold, the initial watermark is corrected according to the watermark structure rules to obtain the watermark.

10. The method according to claim 8, characterized in that, The method includes: Acquire training data, which includes the watermark to be embedded, text without watermark, and watermarked text with embedded watermark output by the watermark embedding model. The watermark extraction model is trained based on the training data using a loss function, which includes a binary cross-entropy loss and a confidence loss. The binary cross-entropy loss is used to optimize the accuracy of the watermark prediction by the watermark extraction model, and the confidence loss term is used to optimize the reliability of the watermark prediction by the watermark extraction model. During the training process of the watermark extraction model, text perturbation processing is performed on the watermark text with embedded watermarks. The text perturbation processing includes at least one of text replacement, text deletion, and text rewriting.

11. A text watermarking device, characterized in that, include: The acquisition module is used to acquire the original text and watermark; The watermark module is used to embed the original text and the watermark input into the watermark model, perform watermark embedding operations, and obtain watermarked text. The embedded watermark model includes a gating modification mechanism, which is used to replace word segments in the original text with a threshold value greater than a preset threshold value with candidate word segments that have the same semantics as the word segments. The watermark is embedded in the candidate word segments, and the threshold value is used to characterize the feasibility of embedding the watermark at the word segment position.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

13. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 10.

14. A computer program product having a computer program stored thereon, characterized in that, When the computer program product is executed by a processor, it implements the method of any one of claims 1 to 10.

Images