Text Generation Watermarking for Verifiable Source Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative models face challenges in preventing misuse and misattribution of generated text content, as existing watermarking methods are ineffective in tracing the source due to loss of watermark information during content generation.
Innovation Solution
A method involving embedding a unique key-based watermark in text content generated by a generative pre-trained model, using a key to divide the vocabulary and adjust probabilities, ensuring the watermark is imperceptible yet verifiable, allowing for source identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a watermark is embedded in text content generated by a generative model, then the source identification capability is improved, but the watermark information is lost during content generation, making the watermark unverifiable
Solution Approach 1:
The patent segments the vocabulary into multiple groups and assigns different selection probabilities to each group based on the watermark key. Instead of embedding a single watermark token, the watermark is distributed across multiple vocabulary segments, making it more robust to generation variations and preventing information loss during content generation.
Solution Approach 2:
The patent applies different selection probabilities to different vocabulary items locally, creating a non-uniform probability distribution that encodes the watermark key. This local quality variation allows the watermark to be embedded subtly in the text generation process without being perceptible, while maintaining verifiability through the unique probability pattern.
2Reliability
If the selection probability of each token is adjusted to embed a watermark, then the watermark becomes verifiable, but the text quality may deteriorate
Solution Approach 1:
The patent changes the probability parameter of token selection dynamically based on the watermark key and generation context. By adjusting selection probabilities rather than hardcoding watermark tokens, the method maintains text quality while embedding verifiable watermark information through subtle probability variations that are imperceptible in the final text.
Solution Approach 2:
The patent implements dynamic probability adjustment during text generation, where the selection probability of each token is adaptively modified based on the current generation context and the watermark key. This dynamic approach allows the watermark to be embedded flexibly without creating static, perceptible artifacts that would degrade text quality.
3Reliability
If a key-based watermarking method is used to prevent misuse, then the source tracking capability is improved, but the complexity of the generation process increases
Solution Approach 1:
The patent makes the generative model multi-functional by integrating both text generation and watermark embedding into a single unified process. The same probability adjustment mechanism serves dual purposes: generating coherent text while simultaneously encoding the watermark key, thereby improving source tracking without requiring separate watermarking systems.
Solution Approach 2:
The generative model performs self-watermarking by automatically embedding the watermark key during the normal text generation process without requiring external watermarking operations. The model uses its own probability distribution mechanism to encode the watermark, making the process self-sufficient and reducing overall system complexity.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for generating text content. The method includes determining a set containing a plurality of tokens based on a key. The method further includes based on the set, generating text content for indicating the key by a generative pre-trained model, wherein a source of the text content is verified by the text content. According to embodiments of the present disclosure, a unique watermark can be formed in the generated text content in a way that is not perceptible to a user, thereby identifying and verifying the source of the text content according to the formed watermark.