Method, system, and medium for detecting generated text based on lightweight paraphrase transformations

Abstract

The application discloses a text generation detection method and system based on lightweight rewriting conversion and a medium. The method comprises the following steps: performing AI removal conversion on the to-be-detected text through a forward rewriting model; inputting the to-be-detected text and the generated human-like equivalent text into a classifier after fusion, and outputting a classification result; the training process of the forward rewriting model is as follows: a paired sample set is constructed by using artificially written text and machine-generated text; the forward rewriting model is fine-tuned by minimizing cross-entropy loss; the machine-generated text is obtained by at least one large language model and filtering redundant samples; the generation text detection method, system and medium completely do not need to access any external, closed-source or proprietary large language model API, which not only reduces the cost and delay, but also greatly simplifies the system architecture and deployment process, and solves the problem that the watermark technology and the similarity regeneration method are limited by the closed-source model and are difficult to be actually deployed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, and medium for generating text detection based on lightweight rewriting and transformation. Background Technology

[0002] In recent years, the rapid development of large-scale language models (LLMs) has made it increasingly difficult to distinguish between machine text and human text, leading to increased risks such as the spread of misinformation and academic misconduct. To address this problem, existing technologies mainly employ four types of detection methods:

[0003] Supervised classification methods (such as RoBERTa-based models) rely on labeled data for training, but they have poor generalization ability on unknown models or domain samples (accuracy drops by more than 32%) and are easily evaded by rewrite attacks (detection accuracy drops by 41.7%).

[0004] Zero-shot statistical methods (such as DetectGPT) do not require training data, but they are sensitive to short texts (false detection rate >22% when less than 50 words) and have poor cross-language performance.

[0005] Watermarking technology requires modification of the generation process, cannot be adapted to closed-source models (such as ChatGPT API), and watermarks are easily removed (removal success rate 89.3%).

[0006] Similarity regeneration methods like SimLLM (e.g., SimLLM: Detecting Sentences Generated by Large Language Models Using Similarity between the Generation and its Re-generation) rely on external LLMs to generate rewritten text, resulting in high latency (greater than or equal to 33 seconds per thousand words), high cost ($3.50 per thousand attempts), and performance fluctuations caused by model version updates (e.g., a 14% decrease in detection accuracy after ChatGPT upgrade).

[0007] The aforementioned methods all suffer from significant drawbacks: supervised classification and zero-shot methods face a trade-off between efficiency and robustness (high-precision methods are costly, while low-cost methods are inaccurate); watermarking and similarity methods are limited by closed-source models, making practical deployment difficult; and existing technologies lack the ability to resist iterative rewriting attacks. Therefore, there is an urgent need for an efficient, stable, and self-contained machine text detection scheme. Summary of the Invention

[0008] Based on the technical problems existing in the background technology, this invention proposes a text generation detection method, system and medium based on lightweight rewriting conversion, which not only reduces cost and latency, but also greatly simplifies system architecture and deployment process, and solves the problem that watermarking technology and similarity regeneration methods are limited by closed-source models and are difficult to deploy in practice.

[0009] The generated text detection method based on lightweight rewriting transformation proposed in this invention includes:

[0010] The positive rewriting model performs deterministic rewriting of the text to be detected by removing AI, and then merges the text to be detected with the human-like equivalent text generated by removing AI and inputs it into the classifier to output the classification result.

[0011] The training process of the forward rewriting model is as follows:

[0012] A paired sample set is constructed using human-written text and machine-generated text. The positive rewriting model is fine-tuned by minimizing cross-entropy loss. The machine-generated text is the text obtained by generating it through at least one large language model and filtering out redundant samples.

[0013] Furthermore, the process of generating the machine-generated text is as follows:

[0014] ChatGPT was used to generate machine-generated sentences from human-written text.

[0015] The machine-generated sentence is rewritten using the Meta open-source large language model in multiple styles, and redundant samples with a BLEU similarity greater than 0.95 are filtered out to obtain the machine-generated text. The BLEU similarity is the similarity between the machine-generated sentence and the style-rewritten sentence.

[0016] Furthermore, the forward rewriting model adopts the T5-large model;

[0017] The forward rewriting model employs an adversarial regularization mechanism during training, using a discriminator network to guide the rewriting process and eliminate machine-generated text features.

[0018] Furthermore, after performing AI-free transformation on the text to be detected using a positive rewriting model,

[0019] Calculate the BART similarity score between the text to be detected and the generated human-like equivalent text. ;when If the similarity is less than the set similarity threshold, the text to be detected will be directly used as the human-like equivalent text.

[0020] Furthermore, the classification result is specifically as follows:

[0021] The source of the text to be detected is determined based on the probability value output by the classifier: if the probability value is greater than the preset probability value, it is determined to be machine-generated text; otherwise, it is determined to be human-written text.

[0022] Furthermore, the classifier is a dynamically masked optimized RoBERTa-base model, and the classifier training employs a bidirectional adversarial mechanism, specifically:

[0023] Train an independent reverse rewriting model to convert human text into machine-style text;

[0024] A four-dimensional adversarial sample library is constructed, consisting of standard human samples, standard machine samples, pseudo-machine-generated text, and pseudo-human text. The pseudo-machine-generated text is generated by injecting machine traces into standard human samples through a reverse rewriting model, and adding formulaic phrases and passive voice to generate pseudo-machine-generated text labeled as machine. The pseudo-human text is generated by removing AI from machine-generated text through a forward rewriting model to generate pseudo-human text labeled as human.

[0025] The four-dimensional adversarial sample library is used to train a classifier to identify style transfer and adversarial perturbation scenarios.

[0026] Furthermore, during the training of the classifier, a dynamic masking technique that randomly discards 15% of the words is used to enhance generalization; at the same time, the classifier's attention mechanism automatically weights key difference signals: word substitution, syntactic reconstruction, and logical explicitation.

[0027] Furthermore, both the reverse rewriting model and the forward rewriting model are independent T5-large models, and the hyperparameters of the two rewriting models are consistent.

[0028] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0029] A computer-readable storage medium storing a plurality of classification programs, the plurality of classification programs being invoked by a processor to execute the method described above.

[0030] The advantages of the lightweight rewriting transformation-based text detection method, system, and medium provided by this invention are as follows: the entire framework contains only a lightweight forward rewriting model (such as T5-large) and a basic classifier (such as RoBERTa-base), completely eliminating the need to access any external, closed-source, or proprietary large language model APIs. This self-contained characteristic not only reduces costs and latency but also greatly simplifies the system architecture and deployment process, solving the problems of watermarking technology and similarity regeneration methods being limited by closed-source models and difficult to deploy in practice. The modular design (forward rewriting model and classifier) ​​allows it to easily adapt to resource-constrained edge computing environments, providing a practical solution for scenarios requiring localized, low-latency responses, such as education, content moderation, and network security. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of the present invention;

[0032] Figure 2 and Figure 3 To explore the differences in the output results generated by DeepSeek-R1 and the classification results under different prompts and models, changes in the output results are highlighted in bold, while the differences between human-written text and machine-generated text are highlighted in underline; Figure 2 This is a schematic diagram illustrating the differences in classification results under SimLLM. Figure 3 This is a schematic diagram illustrating the differences in classification results under the T5 rewriting model (i.e., the forward rewriting model). Detailed Implementation

[0033] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0034] like Figures 1 to 3 As shown, the generated text detection method based on lightweight rewriting transformation proposed in this invention includes:

[0035] The positive rewriting model performs deterministic rewriting of the text to be detected by removing AI, and then merges the text to be detected with the human-like equivalent text generated by removing AI and inputs it into the classifier to output the classification result.

[0036] The training process of the forward rewriting model is as follows:

[0037] A paired sample set is constructed using human-written text and machine-generated text. The forward rewriting model is fine-tuned by minimizing cross-entropy loss. The machine-generated text is the text obtained by inversely generating a large language model and filtering redundant samples.

[0038] An adversarial regularization mechanism is adopted to guide the rewriting process through a discriminator network to eliminate machine-generated text features.

[0039] The entire framework of this embodiment contains only a lightweight forward rewriting model (such as T5-large) and a base classifier (such as RoBERTa-base), completely eliminating the need to access any external, closed-source, or proprietary large language model APIs. This self-contained nature not only reduces cost and latency but also greatly simplifies system architecture and deployment processes, solving the problems of watermarking technology and similarity regeneration methods being limited by closed-source models and difficult to deploy in practice. The modular design (forward rewriting model and classifier) ​​allows it to easily adapt to resource-constrained edge computing environments, providing a practical solution for scenarios requiring localized, low-latency responses, such as education, content moderation, and cybersecurity.

[0040] Therefore, the entire framework of this embodiment detects machine-generated text by analyzing the differences between the input text to be detected and its rewritten human-like equivalent text. Unlike traditional methods that rely on computationally intensive query large language models, this embodiment uses a compact forward rewriting model to rewrite the input text, effectively decoupling the detection function from proprietary large language models. The entire framework of this embodiment is divided into two consecutive stages: rewriting transformation and classification. In the first stage, a forward rewriting model is trained to "de-AI-ize" machine-generated text, rewriting the original input text (i.e., the text to be detected) into de-AI-ized human-like equivalent text while retaining the semantic meaning of human-created content. This transformation eliminates the inherent potential traces (such as repetitive sentence structures and grammatical rigidity) in text generated by existing large language models. In the second stage, the original text and the human-like equivalent text are concatenated and input into a RoBERTa-base-based classifier. The source of the original text is predicted by evaluating style and structural differences. That is, the source of the text to be detected is determined based on the probability value output by the classifier: if the probability value is greater than a preset probability value (e.g., 0.5), it is determined to be machine-generated text; otherwise, it is determined to be human-written text.

[0041] In one embodiment, to replace the external LLM (large model) calling mechanism of SimLLM, this embodiment designs a localized forward rewriting model to achieve a self-contained "de-AI" transformation, wherein the forward rewriting model adopts the T5-large model.

[0042] The forward rewriting model in this embodiment is trained to transform machine-generated text into human-like equivalent text by learning stylistic and structural patterns in paired data. Specifically:

[0043] We construct human-written text using the XSum dataset, which contains human-written news articles, and then use the rewritten text as machine-generated text.

[0044] For each manually written text ChatGPT (GPT-3.5-turbo) is used to generate corresponding machine-generated sentences. This method ensures both the diversity of machine-generated samples and a high degree of consistency with the original content. To enhance the robustness of the forward rewriting model, other language models (such as the Meta open-source large language model, LLaMA 2 70B) are further used to refine the machine-generated sentences. Multi-style rewriting is performed, and redundant samples with a BLEU similarity greater than 0.95 are filtered out. The filtered text is then used as machine-generated text. The BLEU similarity is the similarity between the machine-generated sentence and the style-rewritten sentence to avoid redundancy.

[0045] A dual-discriminator supervision mechanism is introduced during the training of the forward rewriting model: Discriminator 1 constrains semantic consistency (preventing conceptual shifts such as "experimental results" being mistakenly changed to "lab data"), while Discriminator 2 forces the output to conform to the distribution characteristics of human-written text, significantly suppressing lexical repetition (e.g., the frequency of "it is noteworthy that" is reduced by 62%) and syntactic rigidity (passive voice proportion reduced from 71% to 34%) in machine-generated text. With this optimization, the forward rewriting model maintains semantic fidelity (BLEU=0.934) while compressing the single rewriting latency to 7.36 seconds / thousand words and completely eliminating reliance on commercial APIs.

[0046] In one embodiment, after the text to be detected is de-AI-enhanced using a forward rewriting model, the BART similarity score between the text to be detected and the generated human-like equivalent text is calculated. ,judge Does it exceed a set similarity threshold (e.g., -1.85)? If the similarity is less than the set threshold, the text to be detected will be directly used as the human-like equivalent text.

[0047] It should be noted that the forward rewriting model is based on the T5-large model architecture, chosen for its excellent performance in text-to-text tasks and controllable computational cost. The aforementioned fine-tuning training process uses a sequence-to-sequence objective function, training the model to learn the mapping by minimizing cross-entropy loss. Training hyperparameters include the AdamW optimizer and its learning rate settings. This embodiment specifically uses a training rate of 3×10^5 and a batch size of 32. Furthermore, to address issues such as repetitive wording or unnatural coherence in machine-generated text, the forward rewriting model is specifically trained using adversarial regularization to suppress these features. During this process, the discriminator network guides the rewriting process towards a direction closer to human fluency. This dual-objective training mechanism ensures that the rewritten output not only remains consistent with the original text content but also conforms to human stylistic norms.

[0048] It should be noted that this embodiment uses a lightweight rewriting model process to replace the LLM-based proofreading of SimLLM (Similarity Regeneration Method). The pre-trained forward T5-large model (i.e., the forward rewriting model) is used to process the input text to be detected. Rewritten as human-like equivalent text This process eliminates AI-specific artifacts while preserving semantics. The conversion is performed in three stages:

[0049] (a1) First stage, deterministic transformation, input text to be detected The trained forward rewriting model Processing and generating :

[0050] ;

[0051] Forward rewriting model The fixed hyperparameters ensure stable output, with specific goals including: lexical simplification: replacing formal terms (e.g., “utilize” → “use”); structural diversification: breaking repetitive sentence patterns; and artifact removal: eliminating unnatural modifiers and other AI markers.

[0052] (a2) In the second stage, negligible transformations are handled using a similarity threshold, and the BART similarity score is calculated. :

[0053] ;

[0054] in, This is for similarity score operations.

[0055] right After setting a similarity threshold (For example Filtering out minute changes:

[0056] ;

[0057] That is, when If the similarity is less than the set threshold, the text to be detected will be directly used as the human-like equivalent text.

[0058] (a3) The third stage,

[0059] Divergent coding pairs Encoded detection signal:

[0060] Human-written text: Minimal differences ;

[0061] Machine-generated text: significant differences .

[0062] In one embodiment, to address the coarse-grained defect of SimLLM relying on BART similarity scores, the classifier in this embodiment is a dynamically masked optimized RoBERTa-base model, and the knowledge distillation is a 6-layer simplified architecture (the number of parameters is reduced from 110M to 67M), with an accuracy loss of <2%. In actual testing on a Raspberry Pi 4B (ARM Cortex-A72), it only takes 12.3 seconds to process 1,000 words of text, which meets the needs of real-time scenarios such as educational anti-cheating systems.

[0063] The classifier training process includes:

[0064] (b1) Construct a four-dimensional adversarial sample library: standard human samples, standard machine samples, pseudo-machine generated text, and pseudo-human text; the pseudo-machine generated text is generated by injecting machine traces into standard human samples through a reverse rewriting model, and generating pseudo-machine generated text labeled as machine by adding formulaic phrases and passive voice; the pseudo-human text is generated by removing AI from machine generated text through a forward rewriting model, and generating pseudo-human text labeled as human.

[0065] In this embodiment, standard human samples and pseudo-machine-generated text constitute adversarial samples labeled as machine-generated, while standard machine samples and pseudo-human text constitute adversarial samples labeled as human-generated.

[0066] To overcome the vulnerability of SimLLM to advanced adversarial attacks, this embodiment proposes a symmetric adversarial training architecture. First, an independent reverse rewriting model (e.g., a reverse T5-large model) is trained to inject machine traces into manually written text: the input "Vaccines effectively prevent diseases" is transformed into "As evidenced by clinicaltrials, vaccines demonstrate efficacy in disease prevention." Formulaic phrases ("As evidenced by...") and passive voice (increasing the proportion to 68%) are added to simulate LLM generation features.

[0067] It should be noted that both the reverse rewriting model and the forward rewriting model employ independent T5-large models, and their hyperparameters are identical. Therefore, the training processes for both models are similar. The forward rewriting model maps machine-generated text to human-like equivalent text, while the reverse rewriting model maps human-generated text to machine-like equivalent text. Simply put, the forward rewriting model is a machine-generated text to human-generated text rewriting model, aiming to obtain AI-free pseudo-human text; the reverse rewriting model is a human-generated text to machine-generated text rewriting model, aiming to obtain pseudo-machine-generated text containing human-added machine traces. Both rewriting models achieve lightweighting for edge devices: 4-bit quantization technology compresses the file size from 1.8GB to 0.45GB with a precision loss of <1%.

[0068] Specifically, standard human samples and pseudo-machine-generated text spliced ​​as The composite input, and setting the label "machine", and the standard machine sample With pseudo-human texts spliced ​​as The system takes a composite input and labels it "human". Accurate detection is achieved by quantifying the differentiated reconstruction patterns of the two types of text: standard human text, after rewriting, exhibits highly stable structure (BLEU>0.95), requiring only minor grammatical adjustments (e.g., "behaviour→behavior"); standard machine-generated text, however, undergoes significant reconstruction (BLEU<0.4), manifested as lexical simplification ("utilize→use"), syntactic compression ("exhibit optimal performance→showbest results"), and logical explicitness ("suggest a correlation→prove a link") (e.g., ...). Figure 3 (As shown).

[0069] (b2) Enhance generalization by using a dynamic masking technique that randomly discards 15% of the words;

[0070] (b3) The classifier’s attention mechanism automatically weights key difference signals: lexical substitution, syntactic reconstruction, and logical explicitness;

[0071] Its attention mechanism automatically weights key difference signals such as lexical substitution (weight ≥ 0.7), syntactic reconstruction (weight ≥ 0.5), and logical explicitation (weight ≥ 0.6).

[0072] (b4) Optimize the predefined hyperparameter space through grid search, evaluate the performance of different combinations of hyperparameters by cross-validation, and finally select the optimal hyperparameter configuration.

[0073] After grid search optimization, an output probability greater than 0.85 was set as the preset probability value for machine-generated text judgment. This design achieved an ROC-AUC of 0.931 on realistic text generated by Gemini (an 8.4% improvement over SimLLM's 0.859) and suppressed the false positive rate of short text (<50 words) to 8.3% (DetectGPT's was 22.1%). Gemini is a generative artificial intelligence model series launched by Google. ROC-AUC is an important indicator for evaluating the performance of binary classification models. The ROC (Receiver Operating Characteristic) curve is a graph on a two-dimensional plane, with the horizontal axis representing the false positive rate (FPR) and the vertical axis representing the true positive rate (TPR). AUC is the area under the ROC curve, used to quantify model performance.

[0074] The RoBERTa-based classifier is fine-tuned on these sequences using cross-entropy loss, optimizing hyperparameters: a learning rate of 2×10^5, a batch size of 64, and early stopping enabled on the 10% validation set. Dynamic masking (15% dropout rate) prevents overfitting during training. The classifier's attention mechanism automatically weights key dissimilarity signals—lexical substitutions, sentence reordering, and normalized artifacts—enabling the detection of subtle AI-generated text and thus circumventing traditional similarity thresholds.

[0075] In this embodiment, the classifier is trained using the aforementioned four-dimensional adversarial example library, exposing it to complex attack scenarios. The resulting classifier can handle the following three key scenarios: 1) Style transfer samples: manually written text with artificially added LLM artifacts (e.g., increasing perplexity, grammatical rigidity); 2) Adversarial trajectories: sequence modifications simulating evasion attempts. , To determine the number of rewrites, the rewrite methods include: first round of synonym replacement ("effective → efficacious"), second round of syntactic restructuring (active → passive), and final round of deleting formulaic phrases to eliminate traces. 3) Decision boundary challenge: There are ambiguities near the human-machine classification threshold.

[0076] After this training, the system's avoidance rate against iterative rewriting attacks (e.g., k=5) decreased from 68.4% to 30.2%, and the success rate of synonym substitution attacks decreased from 41.7% to 23.8%. In particular, the false detection rate was reduced by 52% when detecting academic misconduct text generated by ChatGPT.

[0077] Data validation;

[0078] The DeTinyLLM generative text detection framework of this embodiment was evaluated using the XSum dataset, which contains human-written news articles, to test the detection performance of the twelve mainstream large language models (LLMs) listed in Table 1. Due to computational resource constraints, 5000 sentences were randomly selected for processing and divided into training (80%), validation (10%), and test (10%) sets, with the temperature set to 0 to ensure output stability. All experiments used the ROC metric as the primary evaluation metric, consistent with standard practice in this field.

[0079] This embodiment (DeTinyLLM) was benchmarked against ten existing detection techniques, including supervised classifiers (RoBbase and RoB-large), zero-shot statistical methods (log p(x), Rank, LogRank, Entropy), perturbation methods (DetectGPT), similarity estimators (BART), baseline fine-tuning models (Baseline), and similarity regeneration methods (SimLLM). As shown in Table 1, the performance of these methods varied: traditional statistical methods such as log p(x) and entropy showed near-random accuracy (average ROCs of 0.504 and 0.551, respectively), reflecting their sensitivity to sentence-level details and limited ability to detect subtle machine artifacts. In contrast, BART and the baseline model achieved modest improvements by utilizing semantic similarity and intrinsic text features (average ROCs of 0.761 and 0.837, respectively), but still remained suboptimal.

[0080] Notably, DeTinyLLM consistently outperformed all comparison models, achieving an average ROC of 0.932 across various language models. This model exhibits exceptional robustness, particularly when handling challenging models like Phi (ROC 0.997) and Gemini (ROC 0.931)—effectively "de-AI-izing" machine-generated text by leveraging the discriminative patterns generated by its compact rewriting transformation. This resulted in significant lexical and structural adjustments (e.g., reducing similarity scores below 0.4 in adversarial cases) while maintaining the coherence of human-generated text. Therefore, DeTinyLLM in this embodiment successfully mitigates the stability issues of methods relying on external language models (such as SimLLM): despite higher computational costs, its average ROC still lags behind by 4.3%. Subsequent analyses focus on comparing the relative performance of DeTinyLLM, BART, and benchmark models.

[0081] Table 1. Text generated using a single large language model.

[0082]

[0083] In Table 1, ChatGPT is the GTP3.5-turbo version developed by OpenAI, GPT-4o is the GPT-4o 2024-05-13 version developed by OpenAI, Yi is the Yi 34B version developed by 0.1AI, OpenChat is the 3.5 1210 7B version developed by Alignment AI, Gemini is the Gemini 1.5 Pro version developed by Google, LLaMa is the LLaMa 2 70B version developed by Meta, Phi is the Phi 2 developed by Microsoft, Mixtral is the 8x7BInstruct v0.1 version developed by Mistral AI, Qwen is the QWen 1.5 72B version developed by Alibaba Group, OLMo is the 7B Instruct version developed by Allen AI, WizardLM is the 13B V1.2 version developed by Microsoft, and Vicuna is the 13B version developed by LMSYS (Large Model Systems Organization). Version 1.5 of LMSYS was initiated by students and faculty of the University of California, Berkeley, the University of California, San Diego and Carnegie Mellon University. It was founded in March 2023 and officially launched the Chatbot Arena in April 2024.

[0084] The comparative analysis in Table 2 shows that DeTinyLLM demonstrates superior scalability and detection efficiency compared to the baseline method in tests with progressively larger sample sizes (1000 to 5000 sentences). While BART only achieved a slight improvement (ROC-AUC from 0.70 to 0.78), and the baseline model only saw a moderate increase (0.75 to 0.82), both SimLLM and DeTinyLLM achieved significantly higher performance ceilings. SimLLM's ROC-AUC steadily climbed from 0.83 to 0.92, reflecting its ability to utilize expanded datasets—consistent with research findings that similarity-based methods benefit from sample diversity. However, DeTinyLLM outperformed all the comparison models, with its ROC-AUC accelerating from 0.84 to 0.94. This 0.10-point improvement at 5000 sentences highlights its strong data utilization: unlike BART, which plateaued early on with its single similarity metric, DeTinyLLM fully leverages a larger sample size to optimize the detection boundary by fusing syntactic and semantic difference patterns. It is worth noting that the advantage of DeTinyLLM over SimLLM increases with the sample size, with the model error widening from 0.01 to 0.02 points, highlighting its stronger generalization ability in complex text distributions. These results are consistent with the advantages of the framework design—through localized rewriting transformation and threshold-optimized inference, it can efficiently extract discriminative features while avoiding the latency bottleneck of methods that rely on large language models.

[0085] Table 2. Detecting Generated Text Through Changes in Sample Size

[0086]

[0087] Furthermore, comparing SimLLM with DeTinyLLM in this embodiment, Table 3 shows that the average ROC-AUC of DeTinyLLM (0.932) is greater than that of SimLLM (0.889). In addition, DeTinyLLM has lower latency, lower cost, and lower iterative attack avoidance rate than SimLLM. Therefore, DeTinyLLM achieves industry-leading scalability, making it practically valuable in real-world scenarios where both accuracy and computational efficiency are required.

[0088] Table 3

[0089]

[0090] Therefore, the DeTinyLLM generative text detection framework in this embodiment replaces the resource-intensive LLM query method with a compact, self-contained rewriting transformation technology (i.e., rewriting model), creating a new paradigm of efficient and robust machine-generated text detection. It solves the latency bottleneck problem caused by the need to repeatedly call large models in traditional methods. This efficiency improvement stems from two innovative designs: (1) device-side AI-free technology, which allows the rewriting models (forward rewriting model and reverse rewriting model) to run locally without relying on the network; (2) threshold-accelerated inference mechanism, which skips the resource-intensive high similarity pair regeneration process. As a result, DeTinyLLM achieves a detection latency of less than one second, and its accuracy is more than 38% higher than the statistical baseline. This efficiency makes deployment in latency-sensitive scenarios readily available.

[0091] Meanwhile, DeTinyLLM uses the RoBERTa model to fuse the original text with the rewritten human-like equivalent text for classification. DeTinyLLM reveals the basic pattern of text differences: human-generated text has only minor modifications (BLEU value > 0.95), while machine-generated text has undergone significant reconstruction (BLEU value < 0.4). Crucially, bidirectional adversarial training significantly improves the model's robustness, reducing the escape rate by 38.2% under iterative rewriting attacks, eliminating AI-specific artifacts while preserving semantics.

[0092] like Figure 2 and Figure 3 As shown, changes in the output are highlighted in bold, while differences between human-written text and machine-generated text are highlighted with underlines.

[0093] Figure 2 and Figure 3By exploring the differences in the output of DeepSeek-R1 and the classification results under different prompts and models, the core mechanism of this embodiment is intuitively demonstrated. For input human text (such as "Research results show that there is a correlation"), SimLLM's LLM-based proofreading method only causes minor lexical changes (such as changing "suggests" to "indicates"), and the similarity remains high (BLEU≈0.92), demonstrating stable rewriting results. In contrast, the forward rewriting model (corresponding to the T5 rewriting model in the attached figure) maintains structural coherence while the output results are almost completely consistent (BLEU>0.95), with negligible editing, such as grammatical optimization (such as replacing "organisation" with "organization"). For machine-generated text (such as "The frontrunners exhibit optimal performance"), the regenerated version of SimLLM still maintains high similarity (BLEU value exceeds 0.85) and also has masking capabilities. However, the forward rewriting model revealed significant differences: it reconstructed the grammatical structure and simplified vocabulary (e.g., "fron-trunners" → "leaders", "optimal" → "best"), causing the similarity with BLEU to drop to <0.4. Key modifications are highlighted in bold, while human-machine differences are underlined, highlighting how the deterministic "de-AI-ization" process of DeTinyLLM exposes potential byproducts—such as lexical rigidity and repetitive expressions—through measurable differences. This difference, quantified by BART scores, forms the basis of the RoBERTa-based classification model, enabling efficient detection without relying on external language models. Figure 2 and Figure 3 The study validated that the forward rewriting model can replace the costly and unstable SimLLM regeneration technique with lightweight, self-contained transformation methods—these transformations effectively enhance the integrity of machine-generated text while preserving the integrity of human-written text.

[0094] Ultimately, DeTinyLLM in this embodiment, by employing a compact forward rewriting model technique, replaces the resource-intensive LLM query approach, pioneering a new paradigm for efficient and robust machine-generated text detection. This framework completely eliminates reliance on proprietary models such as GPT-4 and Gemini. Compared to SimLLM, it reduces inference latency by 77.2%, lowers the cost per thousand queries from $3.5 to $0.15, and reduces operating costs by 96%. Simultaneously, it improves the ROC-AUC value by 4.3% across various LLM models (average ROC value reaches 0.932). By training a lightweight T5-large model to "de-AI-encode" the text—that is, removing AI traces through lexical simplification and structural diversification—and using a RoBERTa-base model as a classifier to fuse the original text (i.e., the text to be detected) and the transformed text (i.e., human-like equivalent text) for classification, DeTinyLLM reveals the fundamental patterns of text differences: human-generated text undergoes only minor modifications (BLEU value > 0.95), while machine-generated text experiences significant reconstruction (BLEU value < 0.4). Crucially, bidirectional adversarial training of the classifier significantly improved its robustness, reducing the escape rate by 38.2% under iterative rewrite attacks.

[0095] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting generated text based on lightweight rewriting transformation, characterized in that, include: The positive rewriting model performs deterministic rewriting of the text to be detected by removing AI, and then merges the text to be detected with the human-like equivalent text generated by removing AI and inputs it into the classifier to output the classification result. The training process of the forward rewriting model is as follows: A paired sample set is constructed using human-written text and machine-generated text. The positive rewriting model is fine-tuned by minimizing cross-entropy loss. The machine-generated text is the text obtained by generating it through at least one large language model and filtering out redundant samples. The classifier is a dynamically masked optimized RoBERTa-base model, and the classifier training employs a two-way adversarial mechanism, specifically: Train an independent reverse rewriting model to convert human text into machine-style text; A four-dimensional adversarial sample library is constructed, consisting of standard human samples, standard machine samples, pseudo-machine-generated text, and pseudo-human text. The pseudo-machine-generated text is generated by injecting machine traces into standard human samples through a reverse rewriting model, and adding formulaic phrases and passive voice to generate pseudo-machine-generated text labeled as machine. The pseudo-human text is generated by removing AI from machine-generated text through a forward rewriting model to generate pseudo-human text labeled as human. A classifier was trained using a four-dimensional adversarial example library to identify style transfer and adversarial perturbation scenarios.

2. The text generation detection method according to claim 1, characterized in that, The process of generating the machine-generated text is as follows: ChatGPT was used to generate machine-generated sentences from human-written text. The machine-generated sentence is rewritten using the Meta open-source large language model in multiple styles, and redundant samples with a BLEU similarity greater than 0.95 are filtered out to obtain the machine-generated text. The BLEU similarity is the similarity between the machine-generated sentence and the style-rewritten sentence.

3. The text generation detection method according to claim 1, characterized in that, The forward rewriting model adopts the T5-large model; The forward rewriting model employs an adversarial regularization mechanism during training, using a discriminator network to guide the rewriting process and eliminate machine-generated text features.

4. The text generation detection method according to claim 1, characterized in that, After the text to be detected is de-AI-ified using a positive rewriting model, Calculate the BART similarity score between the text to be detected and the generated human-like equivalent text. ;when If the similarity is less than the set threshold, the text to be detected will be directly used as the human-like equivalent text.

5. The text generation detection method according to claim 1, characterized in that, The specific classification results are as follows: The source of the text to be detected is determined based on the probability value output by the classifier: if the probability value is greater than the preset probability value, it is determined to be machine-generated text; otherwise, it is determined to be human-written text.

6. The generated text detection method according to claim 1, characterized in that, During the training of the classifier, a dynamic masking technique that randomly discards 15% of the words is used to enhance generalization.

7. The text generation detection method according to claim 1, characterized in that, Both the reverse rewriting model and the forward rewriting model are independent T5-large models, and the hyperparameters of the two rewriting models are the same.

8. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of classification programs, which are used by a processor to execute the method as described in any one of claims 1-7.

Images