Machine-Generated Text Detection Through LLM Rewriting Across Domains
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting machine-generated text are domain-dependent and lack a universal detection standard, struggling to generalize across different domains, and are vulnerable to evasion by text generation sources aware of the detection mechanism.
Innovation Solution
A rewrite-based detection framework, referred to as 'Raidar,' leverages the inherent tendency of large language models (LLMs) to make fewer edits on machine-generated text, utilizing a Learn-to-Rewrite (L2R) process to fine-tune LLMs to perform more edits on human-generated content and fewer edits on AI-generated content, capturing the rich structure of LLM content through diverse training datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional classifier implementations with hand-crafted features are used, then detection can be performed in specific domains, but the method lacks universality and struggles to generalize across different domains
Solution Approach 1:
The patent uses rewriting operations to create transformed copies of the input text. By generating multiple rewritten versions and analyzing the consistency of detection results across these copies, the system achieves both domain independence and maintained detection accuracy. The rewriting process creates variations that preserve semantic meaning while changing surface features, allowing the detector to learn domain-invariant patterns.
Solution Approach 2:
Instead of directly detecting whether text is AI-generated, the patent inverts the approach by detecting whether text is human-written through rewriting operations. The system prompts an LLM to rewrite the input text and then analyzes the rewritten version to infer the origin of the original text, achieving better generalization across domains.
2Measurement precision
If detection thresholds are established for specific domains, then detection accuracy can be optimized for those domains, but the thresholds become highly domain-dependent and cannot serve as universal standards
Solution Approach 1:
The patent implements a universal detection framework that works across multiple domains without domain-specific threshold tuning. The system uses rewriting operations combined with LLM-based detection that can handle diverse text types (academic, journalistic, social media, etc.) with a single set of parameters, achieving both accuracy and universality simultaneously.
Solution Approach 2:
The system changes the detection parameters dynamically based on the input text characteristics rather than using fixed domain-specific thresholds. By using LLM-based detection with rewriting operations, the system adapts to different domains through parameter changes in the detection process itself rather than through pre-set thresholds.
3Measurement precision
If rewriting operations are applied to detect AI-generated text, then detection accuracy improves significantly, but the system must process multiple rewritten versions increasing computational complexity
Solution Approach 1:
The patent applies rewriting operations selectively rather than exhaustively. The system generates a limited number of rewritten versions (typically 2-5) which is sufficient to achieve high detection accuracy without excessive computational overhead. This partial action approach balances detection performance with system complexity.
Solution Approach 2:
The patent introduces an LLM-based intermediary that simplifies the detection process. Instead of implementing complex detection algorithms directly, the system uses an LLM to perform the rewriting and analysis, leveraging the LLM's inherent capabilities to handle the complexity while maintaining high detection accuracy.
4Reliability
If LLMs are used for rewriting operations, then the system achieves high reliability and robustness, but the computational resources required increase compared to traditional methods
Solution Approach 1:
The patent uses the LLM's own rewriting capability to detect AI-generated text. By prompting the LLM to rewrite its own output or similar text, the system leverages the model's inherent understanding of its generation patterns without requiring external detection tools, achieving high reliability while minimizing additional computational overhead.
Data Source
AI summary
Disclosed are systems, methods, and other implementations, including a method for detecting machine-generated content that includes receiving written source input at a machine learning system configured to transform written content into resultant transformed content, and generating by the machine learning system one or more rewritten versions of the written source input, with the one or more rewritten versions being semantically similar to the written source input. The method further includes deriving one or more rewriting change measurements, for the one or more rewritten versions, representing extent of differences between the one or more rewritten versions and the written source input, and determining likelihood that the written source input was machine generated based at least on the derived one or more rewriting change measurements.