A human style text training method based on local prediction entropy fluctuation constraint

Abstract

The application discloses a human style text training method based on local prediction entropy fluctuation constraint, comprising: obtaining an input text sequence, and calculating the prediction entropy corresponding to the probability distribution output by the model at each word position, forming a prediction entropy sequence with a length of T; generating a dynamic fluctuation standard deviation target value through a preset monotonically decreasing function according to the relative position t of the word in the sequence; using a sliding window to traverse the prediction entropy sequence, calculating the standard deviation of the subsequence in each window as the local actual fluctuation value, and using a linear extrapolation method to compensate for the positions of the insufficient window at the beginning and end of the sequence; constructing a one-way hinge type fluctuation penalty loss term, and weighting and fusing it with the standard cross-entropy loss to form a total loss function; and performing end-to-end optimization on the model parameters based on the total loss function. The application can significantly improve the naturalness and style authenticity of the generated text without sacrificing semantic accuracy and logical coherence.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and deep learning, specifically to a loss function optimization method for improving the generated features of autoregressive language models and enhancing the human style fluctuation features of text. Background Technology

[0002] With the widespread application of Large Language Models (LLMs), AI-generated text has shown great potential in content creation, educational assistance, and human-computer interaction. Meanwhile, effectively recognizing AI-generated content has become an important research direction in the field of natural language processing. Current mainstream AI-generated text recognition methods can be mainly divided into two categories: methods based on statistical feature analysis, and methods based on neural classifiers or watermarking mechanisms during the generation process.

[0003] In the statistical fingerprinting paradigm, perplexity and burstiness are the earliest and most widely adopted core discriminant indicators. This method calculates the average perplexity of the input text and the standard deviation of its sentence-level perplexity (i.e., burstiness), and then inputs these two features into a logistic regression classifier to recognize AI-generated text. The basic assumption of this method is that human writing has higher unpredictability and local volatility, while LLM-generated text, due to its highly concentrated word selection in high-probability regions, often exhibits characteristics of excessively concentrated probability distribution and smooth changes in local prediction entropy.

[0004] Specifically, perplexity measures the uncertainty of a language model for a given text. AI-generated text typically exhibits significantly lower perplexity than human text due to highly optimized models. Spontaneity, on the other hand, reflects the variance of local prediction uncertainty in the text. For example, human writing often naturally switches between deterministic expressions and exploratory wording, forming obvious probability jumps, while LLM output tends to be smooth and lacks such dynamic fluctuations. Research shows that this type of detection method based on statistical fingerprints has high recognition accuracy for text generated by mainstream models such as the GPT series, LLaMA, and Claude (see Liu X, Kong L. AI text detection method based on perplexity features with strided sliding window[J]. Working notes of clef, 2024).

[0005] However, with the development of detection technology, relying solely on perplexity and burstiness is no longer sufficient to cope with new detectors. Cutting-edge research has gradually shifted towards detection methods based on contrastive perturbations (such as DetectGPT, see Mitchell E, Lee Y, Khazatsky A, et al. Detectgpt: Zero-shot machine-generated text detection using probability curvature[C] / / International conference on machine learning.PMLR,2023:24950-24962.) and watermarking mechanisms embedded in the generation process (see Kirchenbauer J, Geiping J, Wen Y, et al. A watermark for large language models[C] / / InternationalConference on Machine Learning. PMLR, 2023: 17061-17084.). Nevertheless, in real-world scenarios such as watermark-free and black-box deployments, the stability of local predicted probabilities remains a fundamental criterion shared by most detection systems, and statistical feature analysis still possesses strong practical value.

[0006] To improve the naturalness of AI-generated text and its consistency with human writing style, existing technologies have mainly explored two approaches:

[0007] (1) Adjusting the sampling strategy during the inference stage, such as increasing temperature or using top-p or top-k sampling. However, this method has obvious limitations: excessive randomness can easily lead to logical breaks and factual errors, while conservative settings cannot effectively break the inherent smooth distribution characteristics of the model; (2) Post-hoc editing, which involves rewriting the original generated result using another language model or introducing controllable perturbations. Although this method can make the statistical features of the generated text closer to human text to a certain extent, it may introduce semantic shifts, style inconsistencies or grammatical errors, and the additional rewriting process itself may affect the overall quality and consistency of the text.

[0008] In summary, existing technologies either struggle to balance generation quality and stealth, or rely on black-box post-processing, lacking fundamental improvements to the model's intrinsic generation mechanism. Currently, there is still a lack of efficient, stable, and scalable systematic methods in the published literature for directly guiding models to learn statistical patterns in human writing during the training or fine-tuning phases. Summary of the Invention

[0009] The purpose of this invention is to address the problem that existing autoregressive language models suffer from easily identifiable statistical fingerprints due to overly smoothed probability distributions during text generation. Specifically, text generated by existing models lacks the dynamic fluctuations characteristic of human writing in its local prediction entropy, making it easily identifiable by mainstream AI detection tools. Existing avoidance methods are either limited to adjusting sampling strategies during the inference stage or rely on post-processing rewriting, neither of which can fundamentally reshape the generation distribution from within the model itself.

[0010] To achieve the above-mentioned technical objectives, the technical solution of the present invention is implemented as follows:

[0011] A method for training human-style text based on local prediction entropy fluctuation constraints, the method comprising the following steps:

[0012] 1) Obtain the input text sequence and calculate the prediction entropy corresponding to the probability distribution output by the model at each word position, forming a prediction entropy sequence of length T;

[0013] 2) Based on the relative position t of the word in the sequence, a target value for the dynamic fluctuation standard deviation is generated using a preset monotonically decreasing function. ;

[0014] 3) Use a sliding window to traverse the predicted entropy sequence and calculate the standard deviation of the subsequence within each window as the local actual fluctuation value. And the position of the insufficient window at the beginning and end of the sequence is compensated by linear extrapolation;

[0015] 4) Construct a one-way hinged fluctuation penalty loss term:

[0016]

[0017] in, These are the weighting coefficients.

[0018] This is then weighted and fused with the standard cross-entropy loss to form the total loss function:

[0019]

[0020] in, Standard cross-entropy loss;

[0021] 5) Optimize the model parameters end-to-end based on the total loss function.

[0022] Preferably, step 1) includes the following steps:

[0023] Input a text sequence of length T into the model;

[0024] For each word position in the sequence The model output layer will produce a probability distribution. ,in The input words preceding position t;

[0025] Through formula The predicted entropy at that position is calculated. After traversing the entire sequence, a predicted entropy sequence of length T reflecting the model's prediction entropy along that text path is obtained. ;

[0026] Preferably, in step 2), the formula is used:

[0027]

[0028] Generate based on token location The target value of the dynamically changing standard deviation of fluctuation, where It is a monotonically decreasing function. The standard deviation of the base fluctuation.

[0029] Preferably, the sliding window size W is set to an odd number, and for each center position t, the subsequence within the window is extracted. Calculate its standard deviation as the local actual fluctuation value. The linear extrapolation method is used to compensate for insufficient positions at the beginning and end of the sequence, specifically including:

[0030] Position The missing entropy value on its left side passes through the first two valid points. Linear extrapolation yields:

[0031]

[0032] For the tail position Use the last two valid points Extrapolation:

[0033]

[0034] in, The missing entropy value.

[0035] Preferred, The default value is 0.2.

[0036] The beneficial effects of this invention are as follows: By using statistical patterns of human writing (such as perplexity and spontaneity) as explicit optimization targets during the fine-tuning stage, this invention enables large language models to endogenously generate content that is closer to human text in terms of language volatility and unpredictability. Compared to existing methods that rely on inference sampling adjustments or post-processing rewriting, this solution significantly improves the naturalness and stylistic authenticity of the generated text without sacrificing semantic accuracy and logical coherence. It is suitable for scenarios with high text quality requirements, such as educational assistance, intelligent creation, and human-computer interaction. Specifically, it includes the following three aspects:

[0037] 1. Based on a position-sensitive dynamic prediction entropy fluctuation constraint mechanism, this invention abandons the globally uniform perturbation method and proposes for the first time to link the "suddenness" constraint with the relative position of tokens in the text. This is achieved by constructing a target value for the fluctuation standard deviation that dynamically changes with position. It uses a monotonically decreasing function to simulate the natural writing habit of humans, from the initial divergent ideas of a sentence to the logical convergence at the end.

[0038] 2. This method integrates local feature extraction using sliding window and linear interpolation. A variable-length sliding window mechanism is employed to traverse the predicted entropy sequence, extracting local actual fluctuation values. A linear interpolation operator is then introduced to compensate for the edges at the beginning and end of the sequence. This approach can adapt to the unique semantic groups of different languages ​​(such as Chinese), ensuring that the fluctuation sequence is strictly equal in length to the original input sequence.

[0039] 3. Based on the standard cross-entropy loss, a one-way hinged penalty loss is incorporated. A penalty is triggered when the actual standard deviation of the local prediction entropy falls below a dynamically set threshold, by minimizing the total loss function. Guide the model parameter update. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method described in the embodiments of the invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0042] Experimental environment

[0043] This embodiment uses any autoregressive language model (such as Qwen, LLaMA, GPT, etc.) as the base model. Pre-trained weights are loaded onto a GPU server with sufficient memory, and a corpus containing tens of thousands of high-quality human writing samples is prepared as the fine-tuning dataset. This corpus covers various text types, including news, essays, and technical documents.

[0044] Predicted entropy sequence generation stage

[0045] During training iterations, a text sequence of length T is input into the model. For each word position in the sequence... The model output layer will produce a probability distribution. .

[0046] First, through the formula The predicted entropy at that position is calculated. After traversing the entire sequence, a predicted entropy sequence of length T reflecting the model's predictions along that text path is obtained. .

[0047] Setting the dynamic fluctuation threshold

[0048] To determine the relative position of the token in the sequence, use the formula:

[0049]

[0050] Generate based on token location The target value for the dynamically changing standard deviation of fluctuation. The function is monotonically decreasing, designed to provide a high tolerance for fluctuations at the beginning of sentences, allowing the model to make exploratory expressions; and to strengthen convergence constraints at the end of sentences, ensuring the stability of logical induction. This simulates the human writing habit of divergent thinking at the beginning of a sentence and logical convergence at the end.

[0051] Local fluctuation characteristic calculation

[0052] A sliding window mechanism is used, with the sliding window size W set to an odd number (default W=5), to traverse the predicted entropy sequence. For each center position t, the subsequence within the window is extracted. Calculate its standard deviation as the local actual fluctuation value. If the first and last two tokens of the sequence cannot fill the window, a linear interpolation operator is used for alignment compensation to ensure... The sequence length remains T. Specifically, for position... The missing entropy value on its left side passes through the first two valid points. Linear extrapolation yields:

[0053]

[0054] Similarly, regarding the tail position Use the last two valid points Extrapolation:

[0055]

[0056] This method maintains the continuity of the local first derivative, avoids introducing artificial abrupt changes, and ensures that the fluctuation sequence is strictly equal in length to the original input and has smooth edges.

[0057] Unidirectional hinged fluctuation penalty loss fusion and end-to-end optimization

[0058] The traditional generation loss and the fluctuation loss of this invention are weighted and fused together, and the actual standard deviation of the local prediction entropy is used. Below the dynamic threshold When necessary, impose punishment:

[0059]

[0060] in The weighting coefficients are set to 0.2 by default. The total loss function is set to:

[0061]

[0062] in The standard cross-entropy loss is minimized. Perform end-to-end optimization of model parameters.

[0063] Implementation effect evaluation

[0064] The model trained in this embodiment refines pure AI text with a human writing style while maintaining the original semantic coherence.

[0065] type text Model Evaluation Pure AI text It is worth mentioning that Hu Chenggong's use of the three versions of the *Classic of Poetry* reflects, to some extent, the academic atmosphere of the gradual revival of the New Text School of Confucianism during the Jiaqing and Daoguang periods. Although he himself belonged to the Old Text School, when faced with the three versions of the *Classic of Poetry*, he did not adhere to sectarian biases, but attempted to enhance the breadth and depth of *Mao Shi* studies through integration. #### III. Similarities and Differences in the Selection and Retention of the Three Versions of the *Classic of Poetry* by Chen Huan and Hu Chenggong A comprehensive comparison reveals that Chen Huan and Hu Chenggong had both similarities and significant differences in their approach to the three versions of the *Classic of Poetry*: First, both regarded the *Mao Commentary* as the highest authority, with the three versions of the *Classic of Poetry* serving only as supplementary materials. Neither Chen Huan nor Hu Chenggong ever wavered in the core position of the *Mao Commentary* in *Classic of Poetry* studies, reflecting the academic trend of respecting antiquity and emphasizing practicality prevalent among scholars during the Qianlong and Jiaqing periods. Tencent Zhuque AI: Suspected AI; GPT Zero: 100% AI; Paperpass: 100% Moderately Suspected AI High temperature (temperature=0.9) It is worth noting that Hu Chenggong's use of the three schools of the *Classic of Poetry* reflects, to some extent, the academic atmosphere of the gradual rise of the New Text School of Confucianism during the Jiaqing and Daoguang periods. Although he himself belonged to the Old Text School of Confucianism, he did not adhere to sectarianism when dealing with the three schools of the *Classic of Poetry*, but rather integrated them to a certain extent, thereby enhancing the breadth and depth of the *Mao Commentary on the Classic of Poetry*. ### IV. Similarities and Differences between Chen Huan and Hu Chenggong in their Choice of the Three Schools of the *Classic of Poetry* Through a comprehensive analysis of Chen Huan and Hu Chenggong's approaches to the three schools of the *Classic of Poetry*, it can be seen that they have many similarities and significant differences in this field: #### (I) Similarities: Respect for the Authority of the *Mao Commentary* Both believed that the *Mao Commentary* was an important basis for interpreting the *Classic of Poetry*, and that the three schools of the *Classic of Poetry* could only be used as supplementary materials. Neither Chen Huan nor Hu Chenggong shook the view that the *Mao Commentary* held a core position in the study of the *Classic of Poetry*, which reflects the academic atmosphere of scholars during the Qianlong and Jiaqing periods who generally advocated the tradition of ancient texts and the spirit of empirical evidence. #### (II) Differences: Selection Strategies 1. **Chen Huan**: - **Comprehensiveness**: Chen Huan emphasizes holistic interpretation, not adhering to a single school of thought. - **Logicality**: His explanations are often based on rigorous logical reasoning, helping readers understand and master the knowledge. 2. **Hu Chenggong**: - **Inclusivity**: Hu Chenggong tends to combine various schools of thought, pursuing a multi-faceted understanding. - **Flexibility**: He is more flexible in his interpretations, able to adapt to different cultural backgrounds and the needs of different eras. In short, Chen Huan and Hu Chenggong, in their treatment of the three schools of thought on the *Book of Poetry*, not only demonstrate their academic consensus but also showcase their unique perspectives and insights in specific methodologies. Tencent Zhuque AI: AIGPTZero: 100%; AIPaperpass: 50% - Moderately suspected AI. AI-generated human-style text (using cross-entropy loss) It is worth mentioning that Hu Chenggong's use of the three versions of the *Classic of Poetry* reflects, to some extent, the resurgence of the New Text School of Confucianism during the Jiaqing and Daoguang periods. Although he was a member of the Old Text School, he did not adhere to sectarian biases when facing the three versions of the *Classic of Poetry*, but instead attempted to broaden and deepen the study of the *Mao Commentary* through integration. Thirdly, Chen Huan and Hu Chenggong shared similarities and differences in their approach to the three versions of the *Classic of Poetry*: both regarded the *Mao Commentary* as the highest authority, with the three versions serving only as reference materials. Neither Chen Huan nor Hu Chenggong undermined the dominant position of the *Mao Commentary* in *Classic of Poetry* studies, reflecting the academic tradition of scholars during the Qianlong and Jiaqing periods who revered antiquity. Tencent Zhuque AI: GPT Zero: 70% of the text is AI-generated, 30% is human-written. Paperpass: 50% is moderately suspected to be AI-generated. AI-generated human-style text (using the local prediction entropy fluctuation constraint loss of this method) It is worth noting that Hu Chenggong's use of the three versions of the *Classic of Poetry* reflects, to some extent, the academic trend of the rise of the New Text School of Confucianism during the Jiaqing and Daoguang periods. Although he himself belonged to the Old Text School, he did not adhere to sectarian distinctions when dealing with the three versions of the *Classic of Poetry*, but instead attempted to expand the scope of *Mao Shi* research through a synthesis of their interpretations. Thirdly, the differences between Chen Huan and Hu Chenggong in their selection of the three versions of the *Classic of Poetry* are: firstly, both Chen Huan and Hu Chenggong regarded the *Mao Commentary* as the highest authority, with the other three versions serving only as reference books. Neither Chen Huan nor Hu Chenggong insisted that the *Mao Commentary* held the sole position in *Classic of Poetry* research, reflecting the academic trend of scholars during the Qianlong and Jiaqing periods who valued antiquity and authenticity. Tencent Zhuque AI: Human-written GPTZero: 100% human-written text. Paperpass: No suspicion.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for training human-style text based on local prediction entropy fluctuation constraints, characterized in that, The method includes the following steps: 1) Obtain the input text sequence and calculate the prediction entropy corresponding to the probability distribution output by the model at each word position, forming a prediction entropy sequence of length T; 2) Based on the relative position t of the word in the sequence, a target value for the dynamic fluctuation standard deviation is generated using a preset monotonically decreasing function. ; 3) Use a sliding window to traverse the predicted entropy sequence and calculate the standard deviation of the subsequence within each window as the local actual fluctuation value. And the position of the insufficient window at the beginning and end of the sequence is compensated by linear extrapolation; 4) Construct a one-way hinged fluctuation penalty loss term: in, These are the weighting coefficients. This is then weighted and fused with the standard cross-entropy loss to form the total loss function: in, Standard cross-entropy loss; 5) Optimize the model parameters end-to-end based on the total loss function.

2. The method according to claim 1, characterized in that, Step 1) includes the following steps: Input a text sequence of length T into the model; For each word position in the sequence The model output layer will produce a probability distribution. ,in The input words preceding position t; Through formula The predicted entropy at that position is calculated. After traversing the entire sequence, a predicted entropy sequence of length T reflecting the model's prediction entropy along that text path is obtained. .

3. The method according to claim 2, characterized in that, Step 2) uses the formula: Generate based on token location The target value of the dynamically changing standard deviation of fluctuation, where It is a monotonically decreasing function. The standard deviation of the base fluctuation.

4. The method according to claim 3, characterized in that, Set the sliding window size W to an odd number, and for each center position t, extract the subsequence within the window. Calculate its standard deviation as the local actual fluctuation value. The linear extrapolation method is used to compensate for insufficient positions at the beginning and end of the sequence, specifically including: Position The missing entropy value on its left side passes through the first two valid points. Linear extrapolation yields: For the tail position Use the last two valid points Extrapolation: in, The missing entropy value.

5. The method according to claim 4, characterized in that, The default value is 0.2.

Images