Text watermark information processing method and device, equipment, medium and product

Abstract

The invention discloses a text watermark information processing method and device, equipment, a medium and a product, and the method comprises the steps: carrying out the real-time sentence boundary detection of a text outputted by a language model, so as to obtain a currently detected original sentence; performing parallel computing on the original sentence by adopting a plurality of preset rewriting models with different text styles to generate a plurality of candidate sentences; wherein each rewriting model corresponds to unique watermark information, and the rewriting models are used for rewriting a text style of an original sentence so as to realize watermark information embedding; determining a target rewriting model; and obtaining a candidate sentence corresponding to the target rewriting model as a target sentence of the original sentence. By the adoption of the method, the information capacity and the semantic retentivity of the text in the watermark information embedding process can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and product for processing watermark information in text. Background Technology

[0002] Large-scale model output content identification technology is an important research direction in the field of AI security. It primarily addresses how to embed detectable identity markers into text generated by large-scale language models to achieve functions such as content traceability, copyright protection, and credibility verification. With the widespread application of large-scale language models such as ChatGPT and GPT-4, the recognition and identification of AI-generated content has become an urgent technical requirement. In practical applications, identification technology needs to embed sufficient identity information while maintaining text quality, and simultaneously possess anti-attack capabilities and real-time processing capabilities. However, due to the discreteness and semantic complexity of text, traditional methods struggle to achieve efficient and reliable content identification, especially in scenarios involving large-scale model streaming output and real-time interaction.

[0003] In existing technologies, the main methods for identity authentication, copyright protection, and credibility verification of large language model output content are lexical replacement watermarking, syntactic structure rewriting, and semantic space embedding. These methods are limited by issues such as information capacity, real-time processing, robustness, and semantic preservation. For example, lexical replacement methods struggle to accommodate complex identity information, syntactic structure rewriting affects text fluency, and semantic space embedding methods are computationally complex and lack reversibility. Therefore, existing technologies are ill-suited to the demands of large-scale model streaming output and real-time interaction, exhibiting limitations in information capacity, high processing latency, low detection accuracy, and poor semantic preservation. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, device, medium, and product for processing watermark information in text, which can effectively improve the information capacity and semantic preservation of text during the watermark information embedding process.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for processing watermark information in text, comprising: Perform real-time sentence boundary detection on the text output by the language model to obtain the original sentence that is currently detected; The original sentence is processed in parallel using multiple pre-defined rewriting models with different text styles to generate multiple candidate sentences; each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. Define the target and rewrite the model; Obtain the candidate sentences corresponding to the target rewriting model, and use them as the target sentences of the original sentences.

[0006] As an improvement to the above scheme, the step of performing real-time sentence boundary detection on the text output by the language model to obtain the currently detected original sentence includes: When the text is received from the streaming output of the language model, a sentence boundary detection algorithm using a sliding window mechanism is used to perform real-time sentence boundary detection during the streaming output of text units, so as to obtain the original sentence that is currently detected.

[0007] As an improvement to the above scheme, the determination of the target rewriting model includes: Based on the preset client-specific key, a key index is generated to represent the target watermark information; Obtain the rewriting model corresponding to the key index, and use it as the target rewriting model.

[0008] As an improvement to the above scheme, the watermark information is in the form of n-bit encoding, where n≥2; The step of generating a key index to represent the target watermark information based on a preset client-specific key includes: Based on the client-specific key, a pseudo-random function is used to generate an n-bit key index that represents the target watermark information.

[0009] As an improvement to the above solution, the method further includes: When a target sentence is received from a client, it is determined whether the target sentence contains watermark information, and if watermark information is present, the target rewriting model to be used. When the target sentence contains watermark information, the target restoration model corresponding to the target rewriting model is obtained; wherein, each rewriting model has a one-to-one corresponding restoration model; The target sentence is inversely processed using the target restoration model to obtain the original sentence corresponding to the target sentence.

[0010] As an improvement to the above scheme, the step of determining whether the target sentence contains watermark information when the target sentence is received from the client, and the target rewriting model used when watermark information is present, includes: When the target sentence is received from the client, the target rewritten sentence is converted into a semantic vector; The semantic vector is identified using a pre-defined detection and classification model to determine whether the target sentence contains watermark information, and the target rewriting model used when watermark information is present.

[0011] As an improvement to the above solution, the step of using a preset detection and classification model to identify the semantic vector to determine whether the target sentence contains watermark information, and the target rewriting model used when watermark information is present, includes: The semantic vector is identified using a preset detection and classification model to obtain a probability sequence; wherein the probability sequence consists of multiple probability values, wherein the first probability value represents the probability of watermark information being present, and the other multiple probability values ​​represent the probability of using the rewriting model; Based on the probability sequence, determine whether the target sentence contains watermark information, and determine the target rewriting model used when watermark information is present.

[0012] As an improvement to the above solution, the method further includes: Multiple rewriting models and corresponding restoration models with different text styles are pre-built and trained.

[0013] As an improvement to the above scheme, the pre-construction and training of multiple rewriting models with different text styles and corresponding restoration models includes: Obtain the first training dataset; wherein, the first training dataset includes several first training data, and the first training data are original sample sentences under different application scenario types; Construct the first neural network model; The first neural network model is trained based on the first training dataset to obtain the rewritten model after training. The original sample sentences and the rewritten sample sentences obtained by the rewriting model are used as the second training data to construct the second training dataset. Construct a second neural network model; The second neural network model is trained using the second training dataset to obtain the trained reconstruction model.

[0014] As an improvement to the above scheme, the loss function of the rewriting model includes a pre-training loss function, a contrastive learning loss function, a semantic preservation loss function, and a fluency loss function.

[0015] As an improvement to the above scheme, the loss function of the restoration model includes the inverse mapping loss function, the cycle consistency loss function, and the semantic consistency loss function.

[0016] As an improvement to the above scheme, the pre-construction and training of multiple rewriting models with different text styles and corresponding restoration models further includes: Calculate the quality assessment score based on the original sentence and the target sentence; When the quality assessment score is less than the preset quality assessment score threshold, the rewritten model and the restored model are jointly optimized. As an improvement to the above scheme, the step of calculating the quality assessment score based on the original sentence and the target sentence includes: Based on the original sentence and the target sentence, calculate the semantic similarity score, text overlap score, and text fluency score; A quality assessment score is calculated based on the semantic similarity score, the text overlap score, and the text fluency score.

[0017] As an improvement to the above solution, the method further includes: Pre-build and train the detection and classification model.

[0018] As an improvement to the above scheme, the pre-built and trained detection classification model includes: Obtain a third training dataset; wherein, the third training dataset includes several third training datasets, which consist of original sample sentences, rewritten sample sentences obtained by rewriting models with different text styles, and their rewriting model labels; Construct a third neural network model; The third neural network model is trained using the third training dataset to obtain a trained detection and classification model.

[0019] As an improvement to the above scheme, the loss function of the detection classification model includes a multi-classification loss function, an adversarial training loss function, and a focus loss function.

[0020] This invention also provides a text watermark information processing device, comprising: The original sentence acquisition module is used to perform real-time sentence boundary detection on the text output by the language model in order to obtain the currently detected original sentence. The candidate sentence generation module is used to perform parallel computation on the original sentence using multiple preset rewriting models with different text styles to generate multiple candidate sentences; wherein, each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. The rewrite model determination module is used to determine the target rewrite model; The target sentence determination module is used to obtain the candidate sentences corresponding to the target rewriting model, which are then used as the target sentences of the original sentence.

[0021] As an improvement to the above solution, the device further includes: The target sentence receiving module is used to determine whether the target sentence has watermark information when it receives a target sentence sent by the client, and the target rewriting model used when watermark information exists. The restoration model determination module is used to obtain the target restoration model corresponding to the target rewriting model when the target sentence contains watermark information; wherein, each rewriting model has a one-to-one corresponding restoration model; The original sentence acquisition module is used to perform inverse processing on the target sentence using the target restoration model to obtain the original sentence corresponding to the target sentence.

[0022] As an improvement to the above solution, the device further includes: The model training module is used to pre-build and train multiple rewriting models and corresponding restoration models with different text styles, as well as pre-build and train detection and classification models.

[0023] This invention also provides a text watermark information processing device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the text watermark information processing method as described in any of the above embodiments.

[0024] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the text watermark information processing method as described in any of the above embodiments.

[0025] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the text watermark information processing method as described in any of the above embodiments.

[0026] Compared with existing technologies, the watermark information processing method, apparatus, device, medium, and product disclosed in this invention achieve watermark information embedding by rewriting the text style of sentences. Based on the unique mapping relationship between text style and watermark information and the detectability of text style, invisible watermark information is converted into detectable text style features, ensuring the concealment of watermark information while ensuring that the machine can reverse-parse the watermark information through style features. This invention integrates multi-model collaborative coding, streaming sentence-level processing, and reversible watermarking mechanisms into a unified technical framework. By using multiple differentiated rewriting models to embed watermark information in sentences, it effectively addresses information capacity limitations, increases the information capacity of text during watermark information embedding, and improves the semantic preservation of text. A streaming processing pipeline is introduced to reduce latency and ensure the real-time nature of watermark information embedding. This invention not only improves the detection accuracy and robustness of the system under complex attack environments but also achieves reversible restoration of watermark information, significantly improving the overall performance and applicability of the system. Especially when facing large-scale real-time text generation scenarios, the embodiments of the present invention can effectively reduce processing latency, improve system scalability, and at the same time ensure the quality of watermark embedding and the accuracy of detection. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a text watermark information processing method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a preferred method for processing watermark information in text according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a text watermark information processing device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a preferred text watermark information processing device in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0030] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0031] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0032] See Figure 1 This is a flowchart illustrating a text watermark information processing method provided by an embodiment of the present invention. The embodiment of the present invention provides a text watermark information processing method, including steps S11 to S14: S11. Perform real-time sentence boundary detection on the text output by the language model to obtain the original sentence that is currently detected; S12. The original sentence is processed in parallel using multiple pre-defined rewriting models with different text styles to generate multiple candidate sentences; wherein each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. S13. Determine the target rewriting model; S14. Obtain the candidate sentence corresponding to the target rewriting model, and use it as the target sentence of the original sentence.

[0033] In this embodiment of the invention, several rewriting models M with different text styles are pre-constructed. Each rewriting model can rewrite the text style of the input original sentence and output the rewritten sentence. Since each rewriting model corresponds to a unique watermark, the watermark information of the original sentence can be embedded through the one-to-one correspondence between the watermark information and the text style.

[0034] Preferably, the watermark information is in the form of n-bit encoding, where n≥2.

[0035] As an example, suppose four text rewriting models with different styles are pre-built, namely M0, M1, M2, and M3, and the watermark information is encoded in 2 bits. Among them, the text style of rewriting model M0 is formal written, and its rewriting strategy is synonym replacement rewriting, and the corresponding watermark information is encoded as 00; the text style of rewriting model M1 is colloquial, and its rewriting strategy is sentence structure adjustment rewriting, and the corresponding watermark information is encoded as 01; the text style of rewriting model M2 is logically emphasized, and its rewriting strategy is word order optimization rewriting, and the corresponding watermark information is encoded as 10; the text style of rewriting model M3 is concise and compact, and its rewriting strategy is concise compression rewriting, and the corresponding watermark information is encoded as 11.

[0036] The correspondence between the four models and 2-bit encoding is as follows:

[0037] Suppose that the 5G messaging agent of the language model needs to generate the original sentence "Today's weather in City A is sunny, with a temperature of 25-32℃, suitable for outdoor activities." The system first initializes four rewriting models (M0, M1, M2, M3) and inputs the original sentence into the four rewriting models for processing, resulting in the rewritten sentences output by the four rewriting models, as follows: Rewrite model M0 (encoding 00): Synonym replacement rewriting, output "Today, the weather in City A is sunny, with a temperature of 25 to 32 degrees Celsius, suitable for outdoor activities"; Rewrite model M1 (encoding 01): Adjust and rewrite the sentence structure to output "City A is sunny today, and the temperature of 25-32℃ makes outdoor activities a good choice"; Rewrite Model M2 (encoding 10): Optimize word order and output "In City A, with a temperature of 25-32℃ and sunny weather, today is suitable for outdoor activities"; Rewrite model M3 (encoding 11): Simplify the rewrite and output "City A is sunny today, 25-32℃, suitable for outdoor travel".

[0038] In practical applications, the large language model generates and outputs text content in real time. By receiving the text output by the language model in real time and performing real-time sentence boundary detection on the received text, real-time watermark embedding is achieved during the text generation process.

[0039] To ensure the real-time performance of streaming processing, the system employs a pre-computation strategy. During the original sentence generation process, multiple candidate rewriting results are computed in parallel using multiple different rewriting models. Once the sentence is complete, the target rewriting model is determined to select the final rewritten version as the target sentence corresponding to the original sentence. The system then sends the target sentence to the client.

[0040] Understandably, the core objective of the embodiments of the present invention in adjusting the text style of the original sentence through multiple differentiated rewriting models is to conceal the watermark information (corresponding to the model encoding) through text style differences. For example, rewriting the original sentence with M2 (encoding 10) is essentially encoding the watermark information "10" as "word order optimized text style", making the rewritten sentence the carrier of the watermark information.

[0041] Taking four types of watermark information, with the watermark information using 2-bit encoding, as an example, the theoretical information capacity of this embodiment of the invention is:

[0042] Let n be the theoretical information capacity, and n be the total number of sentences in the text. The number of bits that can be embedded per sentence is increased by 100% compared to traditional methods.

[0043] This invention employs techniques to embed watermark information by rewriting the text style of sentences. Based on the unique mapping relationship between text style and watermark information and the detectability of text style, invisible watermark information is converted into detectable text style features, ensuring the concealment of the watermark information while also ensuring that the machine can reverse-parse the watermark information through style features. This invention integrates multi-model collaborative coding, streaming sentence-level processing, and a reversible watermarking mechanism into a unified technical framework. By using multiple differentiated rewriting models to embed watermark information into sentences, it effectively addresses information capacity limitations, increases the information capacity of the text during watermark embedding, and improves the semantic preservation of the text. A streaming processing pipeline is introduced to reduce latency and ensure the real-time nature of watermark embedding. This invention not only improves the detection accuracy and robustness of the system under complex attack environments but also achieves reversible watermark information restoration, significantly improving the overall performance and applicability of the system. Especially in large-scale real-time text generation scenarios, this invention can effectively reduce processing latency, improve system scalability, and simultaneously ensure the quality of watermark embedding and the accuracy of detection.

[0044] As a preferred embodiment, this embodiment of the invention further improves upon the above embodiments by real-time step S11, namely, performing real-time sentence boundary detection on the text output by the language model to obtain the currently detected original sentence, including step S111: S111. When the text is received from the streaming output of the language model, a sentence boundary detection algorithm with a sliding window mechanism is used to perform real-time sentence boundary detection during the streaming output of text units to obtain the currently detected original sentence.

[0045] In this embodiment of the invention, the message agent of the language model adopts a streaming text output method. During the streaming output of text units (tokens), the system uses a sliding window mechanism for real-time sentence boundary detection to ensure accurate identification of sentence completion time as tokens are gradually generated. Sentence segmentation employs an improved boundary detection algorithm, whose decision function is:

[0046] in, This is the function for determining the boundary conditions. For the first Each token (text unit) For the context window size, The threshold for boundary determination, For The context token window is centered, with EOS serving as the sentence end marker. This represents the sentence termination probability based on the context window.

[0047] The sentence boundary detection algorithm analyzes the semantic features of the current text unit and its surrounding context, and combines them with a pre-trained sentence boundary classifier to perform sentence boundary detection, enabling high-precision sentence segmentation of mixed Chinese and English text.

[0048] As an example, the 5G messaging agent uses streaming output, generating three sentences sequentially: S1 "Today's weather in City A is sunny, with temperatures ranging from 25-32℃, suitable for outdoor activities.", S2 "The temperature difference between day and night is 7℃...", and S3 "Please take precautions against sunburn...". When the system detects "S1 + period" through a sliding window (w=3 tokens), the model calculates P(EOS|"Suitable for outdoor activities.") = 0.96, which exceeds the boundary judgment threshold. This triggers boundary determination, which determines that the sentence is complete (IsBoundary=1), and obtains the current original sentence S1. Subsequent sentences S2 and S3 are obtained in the same way, realizing real-time sentence boundary detection.

[0049] In a preferred embodiment, step S13, namely determining the target rewriting model, includes steps S131 and S132: S131. Generate a key index to represent the target watermark information based on the preset client-specific key; S132. Obtain the rewriting model corresponding to the key index as the target rewriting model.

[0050] Preferably, generating a key index to represent the target watermark information based on a preset client-specific key includes: Based on the client-specific key, a pseudo-random function is used to generate an n-bit key index that represents the target watermark information.

[0051] In this embodiment of the invention, the system adopts a pre-computation strategy to compute multiple candidate rewriting results in parallel during sentence generation, and selects the corresponding rewriting version according to the key index when the sentence is completed.

[0052] The key index is generated using a pseudo-random function (PRF):

[0053] in, For key indexing, is the seed, representing the client-specific key, which can be dynamically updated; N is the number of rewritten models; and i is the original sentence number.

[0054] Preferably, latency control for streaming processing is achieved through an asynchronous queue mechanism, ensuring that the watermark embedding process does not significantly impact the user experience. The system maintains a fixed-size buffer, and batch processing of watermark embedding is performed when the buffer reaches a preset threshold, balancing the requirements of real-time performance and processing efficiency.

[0055] Taking the above four rewriting models, with the watermark information in 2-bit encoded form as an example, the model selection function is defined as:

[0056] in, Choose a function for the model. For the first One original sentence, Let be the key index of the i-th original sentence, and the corresponding 2-bit decimal representation of the key. Key index The corresponding rewriting model.

[0057] The parameter configuration is as follows: Seed: Client-specific key 123456; Key range: k i∈{0,1,2,3} (corresponding to 4 models); Generation rule: k i = PRF (seed, i) mod 4.

[0058] Each key The correspondence between the 2-bit encoded representation and the rewritten model is as follows:

[0059] Mapping relationship between key, model, and encoding: parameter k i (Decimal key) is a connection between 2-bit encoding and rewriting models. The core hub, for example, the 2-bit encoding "01" corresponds to the decimal k. i =1, thus selecting the model. This ensures that the watermark encoding for each sentence of text can be uniquely determined through model selection.

[0060] Suppose the original sentence is S1, "Today in City A, the weather is sunny with a temperature of 25-32℃, suitable for outdoor activities." When k1=2 (binary 10) is generated using the pseudo-random function PRF (seed=12345, i=1), the corresponding target rewriting model is M2. The system immediately calls the candidate sentence S1' rewritten by M2 as the target sentence and outputs the target sentence "S1': In City A, with a temperature of 25-32℃ and sunny weather, today is suitable for outdoor activities." The buffer only caches one sentence, with an embedding delay of ≤10ms, which is imperceptible to the user. Subsequent sentences S2 and S3 repeat this process (k2=0→M0, k3=3→M3).

[0061] It should be noted that the streaming watermark embedding process employs a pipelined processing architecture. When a sentence boundary is detected, the system immediately triggers the watermark embedding process. The mathematical expression for watermark embedding is:

[0062] in, This is a collection of watermarked text, containing all rewritten sentences with embedded watermark information. This is the original text collection, containing all complete original sentences generated by the language model in streaming.

[0063] Linkage between boundary detection and watermark embedding: When When =1 (sentence boundary detected), the system starts from... Extract the corresponding complete sentence and generate a key Then call the rewritten model right Rewrite it, and finally... join in It enables a pipelined operation of "detection-extraction-embedding-storage" to ensure the real-time performance of streaming processing (single sentence latency ≤10ms).

[0064] Using the technical means of embodiments of the present invention, a streaming processing protocol considering real-time constraints and quality assurance is proposed, and a capacity optimization method based on information theory is designed to achieve efficient real-time watermarking and information embedding through a streaming consistency protocol and a capacity optimization function.

[0065] Further, see Figure 2 This is a flowchart illustrating a preferred method for processing watermark information in text according to an embodiment of the present invention. The embodiments of the present invention are further implemented based on any of the above embodiments, and the method further includes steps S21 to S23: S21. When a target sentence is received from the client, determine whether the target sentence has watermark information and the target rewriting model used when watermark information exists. S22. When the target sentence contains watermark information, obtain the target restoration model corresponding to the target rewriting model; wherein, each rewriting model has a one-to-one corresponding restoration model; S23. The target sentence is reverse-processed using the target restoration model to obtain the original sentence corresponding to the target sentence.

[0066] In this embodiment of the invention, several restoration models R corresponding to the rewriting model M are pre-constructed, and the restoration model R and the rewriting model M form a bidirectional mapping relationship:

[0067] The restoration model can restore the text style of the rewritten sentence obtained by the rewriting model and output the restored original sentence.

[0068] When a client receives a 5G message (target sentence), it needs to verify the credibility of the content and input the text into the system. When the system receives the target sentence sent by the client, it determines whether the target sentence contains watermark information and, if so, the target rewriting model to be used.

[0069] The client needs to extract the original content of the 5G message for archiving. When the target sentence contains watermark information, the target reconstruction model corresponding to the target rewriting model is obtained, and the target sentence is input into the target reconstruction model for inverse processing to obtain the original sentence.

[0070] In a preferred embodiment, step S21, namely, determining whether the target sentence contains watermark information when the target sentence is received from the client, and the target rewriting model used when watermark information is present, includes steps S211 and S212: S211. When the target sentence sent by the client is received, the target rewritten sentence is converted into a semantic vector; S212. The semantic vector is identified using a preset detection and classification model to determine whether the target sentence has watermark information and the target rewriting model used when watermark information exists.

[0071] In this embodiment of the invention, a detection and classification model (MLP neural network) is pre-constructed to determine the probability that the target sentence contains watermark information, and the probability of each rewriting model used when watermark information is present. The input of the detection and classification model is the semantic vector of the target sentence, and the output is the probability that the target sentence contains watermark information, and the probability of each rewriting model used when watermark information is present.

[0072] Preferably, the step of using a preset detection and classification model to identify the semantic vector to determine whether the target sentence contains watermark information, and the target rewriting model used when watermark information exists, includes: The semantic vector is identified using a preset detection and classification model to obtain a probability sequence; wherein the probability sequence consists of multiple probability values, wherein the first probability value represents the probability of watermark information being present, and the other multiple probability values ​​represent the probability of using the rewriting model; Based on the probability sequence, determine whether the target sentence contains watermark information, and determine the target rewriting model used when watermark information is present.

[0073] As an example, taking the above four rewriting models (M0, M1, M2, M3) and their corresponding four restoration models (R0, R1, R2, R3) as examples, when the target sentence S1' sent by the client is received, the encoder (BERT) is first used to convert the target sentence into a semantic vector h1, and then the semantic vector h1 is input into the detection and classification model for recognition, outputting a probability sequence P1=[ p 10 p 11 p 12 p 13 p 14 ], where p 10 p represents the probability that watermark information exists. 11 p represents the probability that the target sentence has been rewritten by the rewriting model M0. 12 This represents the probability that the target sentence has been rewritten by the rewriting model M1, and so on.

[0074] Assuming the probability sequence P1 = [0.9, 0.03, 0.03, 0.91, 0.01], it indicates that the target sentence S1' contains watermark information and has been rewritten by the target rewriting model M2. The watermark information is a 2-bit encoding of 10. Based on bidirectional mapping matching, the corresponding restoration model R2 is obtained from the rewriting model M2, and it is used as the target restoration model. The restoration model R2 is called to perform inverse processing on S1'. Based on the word order optimization-word order restoration processing logic, the restored sentence "S1'': Today, the weather in City A is sunny, with a temperature of 25-32℃, suitable for outdoor activities".

[0075] Using the technical methods of this invention, a multi-model collaborative encoding-decoding architecture and a three-layer processing framework (encoding layer, detection layer, and restoration layer) are designed. Efficient watermark embedding, detection, and restoration are achieved through a multi-model collaborative mechanism and hierarchical architecture, ensuring system stability and consistency. A reversible watermarking mechanism based on deep learning is established, and a multi-dimensional system performance evaluation and optimization framework is built. High-precision content restoration and continuous performance optimization are achieved through restoration optimization functions and comprehensive system performance evaluation functions.

[0076] As a preferred embodiment, the present invention is further implemented based on any of the above embodiments, and the method further includes steps S31 and S32: S31. Pre-build and train multiple rewriting models and corresponding restoration models for different text styles.

[0077] S32. Pre-build and train the detection and classification model.

[0078] As a preferred implementation, step S31, namely the pre-construction and training of multiple rewriting models with different text styles and corresponding restoration models, includes steps S311 to S336: S311. Obtain the first training dataset; wherein, the first training dataset includes several first training data, and the first training data are original sample sentences under different application scenario types; S312. Construct the first neural network model; S313. Train the first neural network model according to the first training dataset to obtain the rewritten model after training; S314. Use the original sample sentence and the rewritten sample sentence obtained by the rewriting model as the second training data to construct the second training dataset; S315. Construct the second neural network model; S316. Train the second neural network model according to the second training dataset to obtain the trained restoration model.

[0079] Preferably, the loss function of the rewriting model includes a pre-training loss function, a contrastive learning loss function, a semantic preservation loss function, and a fluency loss function.

[0080] Preferably, the loss function of the restoration model includes an inverse mapping loss function, a cycle consistency loss function, and a semantic consistency loss function.

[0081] In this embodiment of the invention, the rewriting model employs a phased training strategy to ensure that each model has unique rewriting style features, while also being specifically optimized for streaming processing scenarios. The training process fully considers real-time requirements, employing techniques such as model compression, knowledge distillation, and inference acceleration to ensure that the rewriting model can complete the sentence rewriting task within millisecond-level latency.

[0082] The training process of model M is rewritten as follows: Pre-training phase: Each model is first pre-trained on a large-scale corpus. The training data covers texts from multiple fields, including news, literature, science and technology, and dialogue, ensuring that the model possesses broad language understanding and generation capabilities. The pre-training process employs a joint training strategy of masked language modeling and sentence-level rewriting tasks. The pre-training loss function is:

[0083] in, The pre-training loss function of the k-th rewriting model, where k is the index of the rewriting model, N is the number of samples in the sentence rewriting task, and i is the sample index of the sentence rewriting task. for The probability of generating the target sentence given a source sentence, M is the total number of tokens in the masked language modeling task, and j is the token index of the masked task. Let j be the j-th token in the sentence. This represents all tokens preceding the j-th token, indicated by the mask. Let i be the source sentence of the i-th rewritten sample. To rewrite the sentences in the target sample, The second term is the masked language modeling loss. A balancing factor is used to model the loss for masked language. To adapt to the demands of streaming processing, latency-sensitive samples are specifically added during training, and reinforcement learning methods are used to optimize the balance between the model's inference speed and rewriting quality.

[0084] Streaming Style Differentiation Training: To ensure that multiple rewriting models generate different rewriting styles during streaming processing, while maintaining consistency and coherence, this invention introduces a temporally aware contrastive learning loss. This loss function considers not only the rewriting quality of a single sentence but also the semantic coherence and style consistency between sentences. The contrastive learning loss function is defined as follows:

[0085] in, Let j be the contrastive learning loss function for the k-th rewritten model, and j be the index of the contrastive model. The coefficient representing the balancing factor for coherence loss is denoted by T, where T is the total number of sentences in the streaming text, and t is the index of the streaming sentence. For the t-th original sentence in the streaming text, This is a positive sample (the expected rewrite result). For temperature parameters, The coherence loss function is used to ensure the semantic coherence of sentences during streaming rewriting. This training strategy optimizes three objectives simultaneously—rewriting quality, style diversity, and streaming coherence—through a multi-task learning framework, ensuring stable model performance in real-time processing scenarios.

[0086] Quality-constrained training: To ensure rewriting quality, a semantic preservation loss function and a fluency loss function are added, and the overall loss function is optimized to balance style differences and text quality. The semantic preservation loss function is defined as follows:

[0087] in, Let be the semantic preservation loss function for the k-th rewritten model. This is a semantic embedding function.

[0088] The fluency loss function is defined as:

[0089] in, The k-th rewritten model's smoothness loss function, For the k-th rewritten model, Given the preceding context, the probability of generating the i-th token for the language model (LM) is given. The value ranges from (0,1]. The closer the value is to 1, the more the token fits the context. This quantifies the fluency of each token and reflects the grammatical and logical rationality of the sentence. LM is a pre-trained language model.

[0090] The rewritten comprehensive training loss function of the model is:

[0091] in, , , These are the balance coefficients for each loss function.

[0092] The training process of model R is as follows: Paired initialization: Each reduction model R k With the corresponding rewritten model M k Binding, with M k The correspondence between the rewritten sentence and the original sentence serves as the training sample, initializing the inverse mapping capability.

[0093] Inverse mapping training: Minimize the original sentence and R k The L2 norm loss of the restored sentence ensures R k Able to accurately learn M k The inverse operation of the rewriting logic. The inverse mapping loss function is:

[0094] in, The inverse mapping loss function of the kth restoration model. For the k-th restoration model, The k-th rewriting model, s is the original sentence, Let be the mathematical expectation of all original sentences s in dataset D, where D is the original text dataset containing a large number of original sentences.

[0095] Constraint-based reinforcement training: Incorporating cycle consistency loss (ensuring the original sentence → M) k Rewrite → R k Restore → M k The closed-loop consistency of the rewritten sentence and semantic consistency loss (semantic similarity between the restored sentence and the original sentence ≥ 0.98) are used to improve the restoration accuracy.

[0096] The cycle consistency loss function is:

[0097] in, The cycle consistency loss function of the kth reduction model. It is an L1 norm.

[0098] The semantic consistency loss function is:

[0099] in, Let be the semantic consistency loss function for the k-th restoration model.

[0100] The total loss function of the reduction model is:

[0101] in, , This is the balance coefficient for each loss.

[0102] Joint optimization: Training is performed synchronously with the rewritten model, and parameters are collaboratively adjusted through a joint loss function to ensure M... k When rewriting style changes, R k It can still be restored stably.

[0103] Core collaborative logic: The two types of models first complete basic capability training independently, and then form a strong binding bidirectional mapping through joint training of "rewrite-restore" pairing. Finally, in the end-to-end fine-tuning of the whole system, they are optimized in collaboration with the detection classifier, taking into account the watermark embedding quality, restoration accuracy and detection robustness.

[0104] In a preferred embodiment, step S32, namely the pre-built and trained detection and classification model, includes: Obtain a third training dataset; wherein, the third training dataset includes several third training datasets, which consist of original sample sentences, rewritten sample sentences obtained by rewriting models with different text styles, and their rewriting model labels; Construct a third neural network model; The third neural network model is trained using the third training dataset to obtain a trained detection and classification model.

[0105] Preferably, the loss function of the detection classification model includes a multi-classification loss function, an adversarial training loss function, and a focus loss function.

[0106] In this embodiment of the invention, the detection classification model adopts a multi-classification architecture, which can identify whether the text contains a watermark and the corresponding watermark type.

[0107] Taking four watermarks, with each watermark encoded using 2 bits, as an example, the training dataset is constructed as follows:

[0108] in, For the training dataset of the detection classifier, For the i-th original unwatermarked sentence, the labels are 0~4, where 0 = no watermark, 1 = M0 watermark, 2 = M1 watermark, 3 = M2 watermark, and 4 = M3 watermark.

[0109] The correspondence between the tags and the 2-bit codes is as follows:

[0110] The multi-class loss function is:

[0111] in, To test the multi-class loss of the classifier, c is the class index.

[0112] To improve robustness, adversarial example training is incorporated, and the adversarial training loss function is:

[0113] in, To detect the adversarial training loss function of the classifier, To counteract the disturbance vector, Let be the norm of the perturbation vector. This is the upper limit of the disturbance amplitude. Classifier against adversarial examples The prediction result is given by x, which is the embedding vector of the original sample.

[0114] To address the class imbalance problem, focus loss is used for optimization. The focus loss function is as follows:

[0115] in, For focus loss function, Let i be the class weight of the i-th sample. To focus parameters, Let be the true category of the i-th sample.

[0116] The following are joint training optimization strategies, including multi-stage training processes and end-to-end fine-tuning processes.

[0117] Multi-stage training process: Phase 1: Independent pre-training of each rewritten model:

[0118] For the k-th rewritten model, the parameters of the pre-trained neural network are... To rewrite the parameter variables of the model.

[0119] Phase Two: Joint Training of Rewritten and Reconstructed Models

[0120] For the k-th rewritten model after joint training, The parameters are those of the k-th restored model after joint training. To restore the parameter variables of the model.

[0121] Phase 3: Training the detection classifier:

[0122] To detect the parameters of the classifier after training, To detect the parameter variables of the classifier. This is the balance function for focus loss.

[0123] End-to-end fine-tuning: The final stage involves end-to-end joint optimization:

[0124] The joint loss function is:

[0125] in, These are the final parameters after end-to-end optimization. For the combined loss of the entire system, , , , This is the balance coefficient for the losses of each module.

[0126] In a preferred embodiment, step S31, namely the pre-construction and training of multiple rewriting models with different text styles and corresponding restoration models, further includes steps S317 and S318: S317. Calculate the quality assessment score based on the original sentence and the target sentence; S318. When the quality assessment score is less than the preset quality assessment score threshold, the rewritten model and the restored model are jointly optimized. Preferably, calculating the quality assessment score based on the original sentence and the target sentence includes: Based on the original sentence and the target sentence, calculate the semantic similarity score, text overlap score, and text fluency score; A quality assessment score is calculated based on the semantic similarity score, the text overlap score, and the text fluency score.

[0127] Specifically, the quality assessment score is defined as:

[0128] The quality assessment scores for the original sentence and the rewritten sentence are given, where s is the original sentence. To rewrite the sentence, , , This is the balance coefficient for each quality indicator. The semantic similarity score between the original sentence and the rewritten sentence is scored. A score is given for the text overlap between the original sentence and the rewritten sentence. The text fluency score for rewritten sentences.

[0129] The semantic similarity score is calculated as follows:

[0130] This is the semantic embedding vector of the sentence obtained through the BERT model.

[0131] The text overlap score is calculated as follows:

[0132] This is a text overlap metric based on the longest common subsequence (LCS). It is the longest common subsequence of the original sentence and the rewritten sentence.

[0133] Text fluency scores are based on language model perplexity:

[0134] To rewrite the sentence The text fluency score.

[0135] Using the technical means of this invention, the training process is divided into three stages: pre-training each rewriting model, jointly training the rewriting and restoration models, and training the detection classifier. Contrastive learning loss is used to ensure style differences, cyclic consistency constraints guarantee restoration accuracy, and adversarial training improves robustness. The system achieves high-capacity text watermark embedding and detection while maintaining semantics and fluency.

[0136] See Figure 3 This is a schematic diagram of the structure of a text watermark information processing device provided in an embodiment of the present invention. The present invention also provides a text watermark information processing device 10, comprising: The original sentence acquisition module 11 is used to perform real-time sentence boundary detection on the text output by the language model in order to obtain the currently detected original sentence. The candidate sentence generation module 12 is used to perform parallel computation on the original sentence using multiple preset rewriting models with different text styles to generate multiple candidate sentences; wherein, each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. Rewrite model determination module 13 is used to determine the target rewrite model; The target sentence determination module 14 is used to obtain the candidate sentences corresponding to the target rewriting model as the target sentences of the original sentences.

[0137] For a preferred embodiment, see Figure 4This is a schematic diagram of a preferred text watermark information processing device in an embodiment of the present invention. The device 10 further includes: The target sentence receiving module 15 is used to determine whether the target sentence has watermark information when it receives the target sentence sent by the client, and the target rewriting model used when watermark information exists. The restoration model determination module 16 is used to obtain the target restoration model corresponding to the target rewriting model when the target sentence contains watermark information; wherein, each rewriting model has a one-to-one corresponding restoration model. The original sentence acquisition module 17 is used to perform inverse processing on the target sentence using the target restoration model to obtain the original sentence corresponding to the target sentence.

[0138] In a preferred embodiment, the device 10 further includes: The model training module is used to pre-build and train multiple rewriting models and corresponding restoration models with different text styles, as well as pre-build and train detection and classification models.

[0139] It should be noted that the text watermark information processing device provided in this embodiment of the invention is used to execute all the process steps of the text watermark information processing method in the above embodiment. The working principle and beneficial effect of the two are one-to-one, so they will not be described again.

[0140] This invention also provides a text watermark information processing device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the text watermark information processing method as described in any of the above embodiments.

[0141] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform a text watermark information processing method as described in any of the above embodiments.

[0142] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the text watermark information processing method as described in any of the above embodiments.

[0143] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0144] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for processing watermark information in text, characterized in that, include: Perform real-time sentence boundary detection on the text output by the language model to obtain the original sentence that is currently detected; The original sentence is processed in parallel using multiple pre-defined rewriting models with different text styles to generate multiple candidate sentences; each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. Define the target and rewrite the model; Obtain the candidate sentences corresponding to the target rewriting model, and use them as the target sentences of the original sentences.

2. The text watermark information processing method as described in claim 1, characterized in that, The real-time sentence boundary detection of the text output by the language model to obtain the currently detected original sentence includes: When the text is received from the streaming output of the language model, a sentence boundary detection algorithm using a sliding window mechanism is used to perform real-time sentence boundary detection during the streaming output of text units, so as to obtain the original sentence that is currently detected.

3. The text watermark information processing method as described in claim 1, characterized in that, The target rewriting model includes: Based on the preset client-specific key, a key index is generated to represent the target watermark information; Obtain the rewriting model corresponding to the key index, and use it as the target rewriting model.

4. The text watermark information processing method as described in claim 3, characterized in that, The watermark information is encoded in n bits, where n ≥ 2; The step of generating a key index to represent the target watermark information based on a preset client-specific key includes: Based on the client-specific key, a pseudo-random function is used to generate an n-bit key index that represents the target watermark information.

5. The text watermark information processing method according to any one of claims 1 to 4, characterized in that, The method further includes: When a target sentence is received from a client, it is determined whether the target sentence contains watermark information, and if watermark information is present, the target rewriting model to be used. When the target sentence contains watermark information, the target restoration model corresponding to the target rewriting model is obtained; wherein, each rewriting model has a one-to-one corresponding restoration model; The target sentence is inversely processed using the target restoration model to obtain the original sentence corresponding to the target sentence.

6. The text watermark information processing method as described in claim 5, characterized in that, The step of determining whether the target sentence contains watermark information upon receiving the target sentence from the client, and the target rewriting model used when watermark information is present, includes: When the target sentence is received from the client, the target rewritten sentence is converted into a semantic vector; The semantic vector is identified using a pre-defined detection and classification model to determine whether the target sentence contains watermark information, and the target rewriting model used when watermark information is present.

7. The text watermark information processing method as described in claim 6, characterized in that, The step of using a preset detection and classification model to identify the semantic vector to determine whether the target sentence contains watermark information, and the target rewriting model used when watermark information is present, includes: The semantic vector is identified using a preset detection and classification model to obtain a probability sequence; wherein the probability sequence consists of multiple probability values, wherein the first probability value represents the probability of watermark information being present, and the other multiple probability values ​​represent the probability of using the rewriting model; Based on the probability sequence, determine whether the target sentence contains watermark information, and determine the target rewriting model used when watermark information is present.

8. The text watermark information processing method as described in claim 5, characterized in that, The method further includes: Multiple rewriting models and corresponding restoration models with different text styles are pre-built and trained.

9. The text watermark information processing method as described in claim 8, characterized in that, The pre-built and trained rewriting models and corresponding restoration models for different text styles include: Obtain the first training dataset; wherein, the first training dataset includes several first training data, and the first training data are original sample sentences under different application scenario types; Construct the first neural network model; The first neural network model is trained based on the first training dataset to obtain the rewritten model after training. The original sample sentences and the rewritten sample sentences obtained by the rewriting model are used as the second training data to construct the second training dataset. Construct a second neural network model; The second neural network model is trained using the second training dataset to obtain the trained reconstruction model.

10. The text watermark information processing method as described in claim 9, characterized in that, The loss function of the rewriting model includes a pre-training loss function, a contrastive learning loss function, a semantic preservation loss function, and a fluency loss function.

11. The text watermark information processing method as described in claim 9, characterized in that, The loss functions of the restoration model include the inverse mapping loss function, the cycle consistency loss function, and the semantic consistency loss function.

12. The text watermark information processing method as described in claim 9, characterized in that, The pre-built and trained rewriting models with different text styles and corresponding restoration models also include: Calculate the quality assessment score based on the original sentence and the target sentence; When the quality assessment score is less than the preset quality assessment score threshold, the rewritten model and the restored model are jointly optimized.

13. The text watermark information processing method as described in claim 12, characterized in that, The step of calculating a quality assessment score based on the original sentence and the target sentence includes: Based on the original sentence and the target sentence, calculate the semantic similarity score, text overlap score, and text fluency score; A quality assessment score is calculated based on the semantic similarity score, the text overlap score, and the text fluency score.

14. The text watermark information processing method as described in claim 6, characterized in that, The method further includes: Pre-build and train the detection and classification model.

15. The text watermark information processing method as described in claim 14, characterized in that, The pre-built and trained detection and classification model includes: Obtain a third training dataset; wherein, the third training dataset includes several third training datasets, which consist of original sample sentences, rewritten sample sentences obtained by rewriting models with different text styles, and their rewriting model labels; Construct a third neural network model; The third neural network model is trained using the third training dataset to obtain a trained detection and classification model.

16. The text watermark information processing method as described in claim 14, characterized in that, The loss function of the detection and classification model includes a multi-classification loss function, an adversarial training loss function, and a focus loss function.

17. A text watermark information processing device, characterized in that, include: The original sentence acquisition module is used to perform real-time sentence boundary detection on the text output by the language model in order to obtain the currently detected original sentence. The candidate sentence generation module is used to perform parallel computation on the original sentence using multiple preset rewriting models with different text styles to generate multiple candidate sentences; wherein, each rewriting model corresponds to a unique watermark information, and the rewriting model is used to rewrite the text style of the original sentence to achieve watermark information embedding. The rewrite model determination module is used to determine the target rewrite model; The target sentence determination module is used to obtain the candidate sentences corresponding to the target rewriting model, which are then used as the target sentences of the original sentence.

18. The text watermark information processing apparatus as described in claim 17, characterized in that, The device further includes: The target sentence receiving module is used to determine whether the target sentence has watermark information when it receives a target sentence sent by the client, and the target rewriting model used when watermark information exists. The restoration model determination module is used to obtain the target restoration model corresponding to the target rewriting model when the target sentence contains watermark information; wherein, each rewriting model has a one-to-one corresponding restoration model; The original sentence acquisition module is used to perform inverse processing on the target sentence using the target restoration model to obtain the original sentence corresponding to the target sentence.

19. The text watermark information processing apparatus as described in claim 17, characterized in that, The device further includes: The model training module is used to pre-build and train multiple rewriting models and corresponding restoration models with different text styles, as well as pre-build and train detection and classification models.

20. A text watermark information processing device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the text watermark information processing method as described in any one of claims 1 to 16.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the text watermark information processing method as described in any one of claims 1 to 16.

22. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the text watermark information processing method as described in any one of claims 1 to 16.

Images