Ai-generated text detection method based on sentence length distribution and text predictability features

Abstract

The application belongs to the technical field of AI text detection and classification, and particularly relates to an AI generated text detection method based on sentence length distribution and text predictability features, which comprises the following steps: analyzing character sequences from the text to be detected, cutting into multiple sentences based on punctuation features and counting the number of characters, using a scoring model to analyze conditional generation probability matrix and extract text perplexity, constructing normalized boundary weight by extracting conditional generation probability of punctuation features, weighting and projecting the number of characters and the normalized boundary weight to obtain the sentence length variation coefficient, obtaining the local text perplexity, weighting and cross mapping the character number sequence and the local text perplexity sequence to obtain the long entropy correlation coefficient, nonlinearly mapping the sentence length variation coefficient, the text perplexity and the long entropy correlation coefficient to construct a multi-dimensional feature judgment index, and comparing the multi-dimensional feature judgment index with a true or false threshold to intercept. The application excludes the interference of mechanical forced truncation and rare word camouflage, and greatly improves the detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of AI text detection and classification technology. More specifically, this invention relates to an AI-generated text detection method based on sentence length distribution and text predictability features. Background Technology

[0002] With the development of artificial intelligence technology, text generated by large language models is widely used in daily office work and text creation. In order to distinguish between text created by real humans and text generated by large language models, existing text detection methods usually rely on the text perplexity index for judgment.

[0003] Existing detection methods calculate the text perplexity of the text after acquiring it. Since large language models tend to output common and easily predictable words, the text perplexity of the generated text is usually low. When the calculated text perplexity is low, existing techniques determine that the text is generated by a large language model.

[0004] However, the aforementioned existing technologies have serious flaws when dealing with complex spoofing methods: First, forgers can artificially inflate the text perplexity index of the generated text by forcing large language models to use obscure words in the prompts. This operation causes existing detection systems based on single text perplexity to miss detections, incorrectly identifying AI-generated text as human text. Second, forgers can also use external scripts to randomly and abruptly insert punctuation marks into long sentences generated by large language models. This forced truncation operation breaks the complete sentence into fragments, directly altering the surface structure of the text. Existing detection systems rely solely on literal word counts and completely trust the punctuation marks in the text, failing to verify whether the positions of these punctuation marks conform to the pauses of real natural language.

[0005] Therefore, when faced with a combination of deception techniques, such as altering obscure words and forcibly truncating punctuation, existing text detection methods are easily fooled, leading to a significant drop in the final detection accuracy. Summary of the Invention

[0006] To address the problem that existing detection technologies rely solely on numerical statistics of text surface segmentation features, lacking methods to verify the rationality of punctuation generation, and are easily deceived by combined spoofing techniques such as obscure word alteration and forced punctuation truncation, leading to a significant drop in detection accuracy, this invention provides an AI-generated text detection method based on sentence length distribution and text predictability features. The method includes: parsing a character sequence from the text to be detected; extracting all punctuation features as sentence segmentation boundaries; segmenting the character sequence into multiple sentences based on the punctuation features; and counting the number of characters in each sentence. A scoring model is then used to perform forward propagation on the character sequence to parse the conditional generation probability matrix, combining this matrix with information... This paper proposes a method to extract text perplexity using a cross-entropy model. It extracts the conditional generation probability of punctuation features from the perspective of a scoring model, constructs boundary naturalness weights, and transforms these weights into normalized boundary weights. The paper then performs a weighted projection of the character count of each sentence with its corresponding normalized boundary weights to obtain the sentence length variation coefficient. For all sentences, it obtains the local text perplexity corresponding to each sentence based on the conditional generation probability matrix. Using a variable correlation analysis framework, it performs a weighted cross-mapping of the character count sequence and the local text perplexity sequence to obtain the long-entropy correlation coefficient. Finally, it performs a nonlinear mapping of the sentence length variation coefficient, text perplexity, and long-entropy correlation coefficient to construct a multi-dimensional feature judgment index, which is then compared with a true / false threshold for interception.

[0007] This invention constructs normalized boundary weights by extracting punctuation features from the perspective of the scoring model and constructing conditional generation probabilities. This results in a very low weighting of the abruptly truncated punctuation features in the character count statistics, thereby eliminating the interference of false segmentation on the true sentence length distribution. Simultaneously, it obtains the local text perplexity corresponding to each sentence and performs a weighted cross-mapping between the character count sequence and the local text perplexity sequence to obtain the long entropy correlation coefficient. This can capture the inconsistency between the abnormal fluctuations in local text perplexity caused by the forger's deliberate generation of obscure words and the real expression habits. It breaks through the limitations of existing technologies that rely solely on segmentation features and single text perplexity. Even when faced with comprehensive camouflage methods such as obscure word alteration and forced punctuation truncation, it can still accurately discover the inherent generation rules of the large language model, significantly improving the detection accuracy of the text to be detected.

[0008] Preferably, the step of parsing the character sequence from the text to be detected, extracting all punctuation features as sentence segmentation boundaries, dividing the character sequence into multiple sentences based on the punctuation features, and counting the number of characters in each sentence includes: decoding the input text to be detected using a unified character encoding standard to parse the character sequence from the text to be detected; traversing the character sequence and matching it with a punctuation feature library containing periods, question marks, exclamation marks, and semicolons to extract all punctuation features as sentence segmentation boundaries; dividing the character sequence into multiple sentences based on the punctuation features, removing spaces and control characters from each sentence, and counting the number of characters in each sentence.

[0009] Preferably, the step of performing forward propagation on the character sequence using the scoring model to parse the conditional generation probability matrix includes: obtaining the vocabulary dictionary used by the scoring model in the initial training phase and the segmentation rules constructed based on the byte pair encoding algorithm; performing sub-word matching on the character sequence using the segmentation rules to convert the character sequence into a discrete sequence containing multiple text words; extracting the unique integer index value corresponding to each text word in the discrete sequence from the vocabulary dictionary to obtain a word index sequence composed of unique integer index values; inputting the word index sequence into the scoring model to perform forward propagation to obtain the output logical value vector of the last hidden layer of the scoring model; and performing a normalized exponential function operation on each element value in the output logical value vector to parse the conditional generation probability matrix containing the entire vocabulary dictionary at each time step.

[0010] Preferably, the step of extracting text perplexity using the information theory cross-entropy model includes: calculating the natural logarithm of the conditional generation probability of each word in the character sequence, summing the natural logarithms of all words, dividing the sum by the total number of words in the text to be detected and taking the opposite number, and performing a natural exponential function operation on the opposite number to obtain the text perplexity.

[0011] Preferably, the step of converting the boundary naturalness weights into normalized boundary weights includes: calculating the sum of the boundary naturalness weights of all sentences as the global weight sum; and dividing the boundary naturalness weight of each sentence by the global weight sum to obtain the normalized boundary weights.

[0012] This invention calculates the sum of the boundary naturalness weights of all sentences and divides the boundary naturalness weight of each sentence by it, transforming the absolute single-point generation probability into a relative proportional distribution. This processing method effectively eliminates the scale effect caused by the difference in the total number of sentences contained in texts of different lengths, allowing the rationality of the segmentation of each sentence to be relatively measured under a unified benchmark, and ensuring that the subsequent weighted calculation process is not affected by the overall length of the text to be detected.

[0013] Preferably, the step of weighted projection of the number of characters in each sentence with the corresponding normalized boundary weights to obtain the sentence length variation coefficient includes: multiplying the number of characters in each sentence with its corresponding normalized boundary weights and summing the results to obtain the weighted expected value of the number of characters; calculating the square of the difference between the number of characters in each sentence and the weighted expected value of the number of characters, multiplying the squared value with its corresponding normalized boundary weights and summing the results, and taking the square root of the summed result to obtain the weighted standard deviation of the number of characters; and dividing the weighted standard deviation of the number of characters by the weighted expected value of the number of characters to obtain the sentence length variation coefficient.

[0014] This invention incorporates normalized boundary weights into the derivation and calculation of expectation and standard deviation, automatically assigning extremely low weights to false short sentences that violate natural language pause habits during cumulative calculation. This effectively compresses and eliminates them. The obtained character-weighted expectation and character-weighted standard deviation can truly reflect the average length and fluctuation after eliminating forced truncation interference. Finally, the sentence length variation coefficient obtained by dividing the two accurately restores the true macro-length convergence state of the text to be detected, eliminating the interference of creating false length fluctuations by randomly inserting punctuation on the detection results.

[0015] Preferably, obtaining the local text perplexity corresponding to each sentence based on the conditional generation probability matrix includes: calculating the local text perplexity corresponding to the sentence based on the conditional generation probability of the tokens belonging to the sentence in the conditional generation probability matrix; multiplying the local text perplexity of each sentence by its corresponding normalized boundary weight and summing the results to obtain the weighted expectation of the local perplexity; calculating the square of the difference between the local text perplexity and the weighted expectation of the local perplexity of each sentence, multiplying the squared value by its corresponding normalized boundary weight and summing the results, and taking the square root of the summed result to obtain the weighted standard deviation of the local perplexity.

[0016] This invention calculates the weighted expectation and weighted standard deviation of local perplexity for each sentence by combining the product of normalized boundary weights. This allows for the evaluation of the average state and dispersion of local text perplexity after removing interference from external script segmentation. It ensures that the calculation of the predictability distribution characteristics of local words is based on semantic fragments that conform to real human logical pauses, reducing data pollution caused by fragmented short sentences and providing accurate data support for subsequent in-depth analysis of the underlying relationship between sentence length and information entropy.

[0017] Preferably, the step of performing a weighted cross-mapping of the character count sequence and the local text perplexity sequence based on the variable correlation analysis framework to obtain the long entropy correlation coefficient includes: multiplying the difference between the character count and the weighted expected value of the character count for each sentence by the difference between the local text perplexity and the weighted expected value of the local perplexity for each sentence, and then multiplying the product result by its corresponding normalized boundary weight and summing them to obtain the weighted covariance; and dividing the weighted covariance by the product of the character count weighted standard deviation and the local perplexity weighted standard deviation to obtain the long entropy correlation coefficient.

[0018] This invention obtains the weighted covariance by calculating the weighted product of character count deviation and local text perplexity deviation, and then performs standardization by dividing by the weighted standard deviation product of the two. This reflects the synergistic trend of sentence length and local text perplexity fluctuating around their respective expectations. When the large language model is limited by the fixed underlying generation mechanism, causing the correlation between sentence length and vocabulary complexity to fail, the obtained long entropy correlation coefficient will drop significantly. This directly blocks the countermeasure path of forgers who attempt to unilaterally force up the length fluctuation and vocabulary complexity to evade detection from the algorithm's underlying level.

[0019] Preferably, the multidimensional feature determination index is calculated as follows: In the formula, It is a multidimensional feature determination index; The first dimension weight constant; The coefficient of variation for sentence length; The weight constant for the second dimension; It is the natural logarithm function; For text perplexity; The third dimension weight constant; It is a natural exponential function; The correlation coefficient is the long entropy coefficient. It is a preset minimum positive number.

[0020] This invention constructs a compensation mechanism for the fixed generation characteristics of the model by performing nonlinear amplification operations on the extracted multidimensional feature variables through the mapping of reciprocals and exponents. When forgers forcibly increase the fluctuation of sentence length and text perplexity, causing some feature components to be compressed and reduced in size, the long entropy correlation coefficient decreases due to the failure of the underlying feature association, causing the corresponding nonlinear decay term to expand exponentially and rapidly, leading to a rapid increase in the overall multidimensional feature judgment index. This ensures that any local forgery operation attempting to circumvent the single-index check will trigger a mutation in the overall parameters, achieving detection coverage of highly adversarial machine text.

[0021] Preferably, the interception by comparing with the authenticity threshold includes: pre-setting an authenticity threshold to define the boundary of real human text; comparing the multi-dimensional feature judgment index with the authenticity threshold; if the multi-dimensional feature judgment index is greater than the authenticity threshold, then the text to be detected is determined to be artificial intelligence-generated text, and a corresponding alarm signal is output.

[0022] The beneficial effects of this invention are as follows: 1. This invention utilizes a scoring model to analyze the conditional generation probability of punctuation features and constructs normalized boundary weights. This significantly reduces the weight of unreasonable punctuation features generated by forced truncation in subsequent character counts and local text perplexity calculations. This effectively eliminates false pauses fabricated by forgers using external scripts in the text to be detected, restoring the true layout and structural distribution of the text to be detected. Thus, it solves the problem that existing detection technologies are easily deceived by forced truncation operations because they completely trust the surface segmentation symbols.

[0023] 2. This invention obtains local text perplexity based on a conditional generation probability matrix and performs a weighted cross-mapping with the character quantity sequence to obtain a long entropy correlation coefficient. This reflects the contradictory relationship between the sudden change in local text perplexity caused by the forger's deliberate use of obscure words and the natural number of characters in the sentence. This breaks the limitation of existing technologies that rely solely on macro-text perplexity for detection and interception, thus solving the vulnerability of forgers who maliciously inject obscure words through prompts to artificially increase perplexity.

[0024] 3. This invention delves into the logical connection between the true expression patterns of the text content to be detected and the inherent smooth generation tendency of large language models. It constructs a multi-dimensional feature judgment index by nonlinearly mapping the sentence length variation coefficient, text perplexity, and long entropy correlation coefficient. This integrates multi-dimensional character quantity statistical features with deep semantic predictability features. When facing various malicious tampering of sentence structure and comprehensive disguised adversarial methods, it builds a detection barrier and significantly improves the interception success rate of the final judgment and interception based on the truth and falsehood threshold. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the AI-generated text detection method based on sentence length distribution and text predictability features in this invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] This invention discloses an AI-generated text detection method based on sentence length distribution and text predictability features, referring to... Figure 1 This includes steps S1-S6: S1. Parse the character sequence from the text to be detected, extract all punctuation features as sentence segmentation boundaries, divide the character sequence into multiple sentences based on the punctuation features, and count the number of characters in each sentence.

[0029] It should be noted that the basic data structure of natural language consists of character sequences and segmentation boundaries. Real human text output is a set of discrete sentences with a specific length rhythm in terms of length structure. This invention transforms continuous unstructured character streams into an array of sentence units that can be discretely statistically analyzed by identifying punctuation features as semantic pause markers, thereby establishing a basic data observation benchmark for subsequent extraction of structural variation features and verification of boundary authenticity.

[0030] Specifically, the input text to be detected is obtained, and a unified character encoding standard is used to decode the text to parse out the character sequence. A punctuation feature library containing periods, question marks, exclamation marks, and semicolons is established. The character sequence is traversed and matched with the punctuation feature library to extract all punctuation features that serve as sentence segmentation boundaries. Based on the punctuation features, the character sequence is divided into multiple sentences, where the punctuation features are the end elements of the corresponding sentences. For each sentence, spaces and control characters are removed, and the total number of characters contained in each sentence is counted to obtain the number of characters in each sentence.

[0031] S2. Use the scoring model to perform forward propagation on the character sequence, parse out the conditional generation probability matrix, and extract the text perplexity by combining the information theory cross-entropy model.

[0032] It should be noted that large language models have a probability-maximizing generation mechanism when generating text, which makes the predictability of their output vocabulary significantly higher than the diversity of vocabulary generated by humans in natural contexts. This invention calculates the logarithmic average of the generation probability of each word under the perspective of the scoring model, and quantifies the conditional prediction probability of the entire text to be detected from the dimension of information entropy, thereby establishing a detection benchmark to capture the inherent generation smoothness features of the model.

[0033] To ensure the objectivity of the calculation and the alignment with the probability space, the scoring model of this invention adopts an autoregressive large language model based on a self-attention mechanism architecture. Its network structure consists of 32 stacked decoder layers, each incorporating a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism has 32 attention heads and a hidden layer dimension of 4096. The feedforward neural network uses an activation function that enhances nonlinear expressive power. During model training, a self-supervised autoregressive next-word prediction task is used, with the cross-entropy loss function and an adaptive moment estimation optimizer. Regarding parameter settings, they are optimized based on the size of the pre-training corpus and the memory capacity of the computing device. The initial learning rate is set to 0.0002 with a cosine annealing learning rate decay strategy, the batch size is set to 4 million words, and the context window length is set to 4096, thus ensuring that the scoring model has the ability to accurately predict the probability of natural language generation.

[0034] Specifically, the process involves obtaining the vocabulary dictionary and segmentation rules based on the byte-pair encoding algorithm used by the scoring model during the initial training phase. The vocabulary dictionary also includes all punctuation features. Sub-word matching is performed on the character sequence using the segmentation rules, converting the character sequence into a discrete sequence containing multiple text words. A unique integer index value is extracted from the vocabulary dictionary for each text word in the discrete sequence. This unique integer index value is then used to replace each text word in the discrete sequence, resulting in a word index sequence consisting only of unique integer index values. This word index sequence is then input into the scoring model to perform a forward propagation operation, obtaining the output logical value vector of the last hidden layer of the scoring model. For each element in the output logical value vector, a normalized exponential function operation is performed to parse out the conditional generation probability matrix containing the entire vocabulary dictionary at each time step. This conditional generation probability matrix also includes the conditional generation probability of punctuation features at the corresponding time step.

[0035] Specifically, a normalized exponential function operation is performed on each element value in the output logical value vector to obtain the nth element in the conditional generation probability matrix. The corresponding vocabulary in the dictionary at each time step The prediction probability of each word is calculated using the following formula:

[0036] In the formula, For the conditional generation probability matrix, the first... The corresponding vocabulary in the dictionary at each time step The predicted probability of each word element; It is a natural exponential function; For the corresponding number in the output logical value vector The time step and the first The element values ​​of each word element; This represents the total number of lexical units contained in the dictionary. For the corresponding number in the output logical value vector The time step and the first The element values ​​of each word.

[0037] The formula for performing the normalized exponential function operation maps the originally unbounded output logical value vector into a probability distribution vector where each element's value is between 0 and 1 and the sum of the natural exponents of all element values. This is achieved by calculating the natural exponent of a single element value and dividing it by the sum of the natural exponents of all element values. When the value of a certain element in the output logical value vector increases significantly relative to other element values, its corresponding natural exponent is geometrically amplified, causing the predicted probability of that index position to approach 1, while the predicted probabilities of other positions approach 0. The obtained predicted probabilities constitute the conditional generation probability matrix that reflects the confidence of the scoring model for each word at each position.

[0038] Furthermore, for each word in the word index sequence, based on its time step in the sequence and its corresponding index position in the vocabulary dictionary, the predicted probability of the corresponding position is extracted from the conditional generation probability matrix, and the extracted predicted probability is used as the conditional generation probability of that word.

[0039] Furthermore, combining the information-theoretic cross-entropy model, the text perplexity is calculated based on the conditional generation probability matrix. The specific calculation formula is as follows:

[0040] In the formula, For text perplexity; It is a natural exponential function; The total number of tokens in the text to be detected; It is the natural logarithm function; For the first character in the character sequence Each word element; For the first The set of all historical preceding lexical elements before each lexical element; The output of the scoring model is the first term under the condition of a known set of historical prepositions. The conditional generation probability of each word.

[0041] The formula for calculating text perplexity involves taking the natural logarithm of the conditional generation probability of each word in the character sequence and calculating its arithmetic mean. Then, the average cross-entropy in the logarithmic space is transformed back to the text perplexity value in the original probability space using a natural exponential function. When the text to be detected uses a large number of rare words or complex grammatical structures, the conditional generation probability given by the scoring model for these rare words will significantly decrease, leading to a larger absolute value of the natural logarithm in the formula, and thus increasing the text perplexity obtained after mapping by the natural exponential function. Conversely, if the words used are extremely common, the text perplexity decreases. The obtained text perplexity reflects the degree to which the words in the text deviate from the statistical distribution pattern of the scoring model, effectively assessing the predictability of words in the text generation process and providing basic feature support for qualitative judgment.

[0042] S3. Extract the conditional generation probability of punctuation features from the perspective of the scoring model, construct the boundary naturalness weights, and transform the boundary naturalness weights into normalized boundary weights.

[0043] It should be noted that real human natural language organization possesses semantic completeness and logical closed-loop characteristics. The probability of punctuation features appearing when a complete logical deduction or semantic unit statement is completed is extremely high. In contrast, the punctuation features forcibly inserted into long sentences by adversarial scripts disrupt the semantic coherence of the context. This invention extracts the prediction confidence of these segmentation boundaries by a scoring model to reflect the semantic rationality of each segmentation action, thereby accurately eliminating false boundary interference created by adversaries using forced truncation operations without semantic basis.

[0044] In specific adversarial scenarios, if a real human inputs a text with complete semantics, the scoring model predicts that the probability of generating a condition with punctuation features at the end of the text is extremely high. Conversely, if the adversarial script forcibly inserts punctuation features after an incomplete phrase, the probability of generating a condition with punctuation features at that position will be significantly reduced due to the lack of subsequent key grammatical components. This probability test based on contextual semantic features can effectively distinguish between natural pauses in natural language and forced truncation by external scripts.

[0045] Specifically, regarding the first Based on the conditional generation probability matrix, the punctuation features at the end of each sentence are used to extract the conditional generation probability of these features from the perspective of the scoring model through an index query. This conditional generation probability is then directly defined as the punctuation feature at the end of the sentence. The boundary naturalness weights of the nth sentence are calculated; the sum of the boundary naturalness weights of all sentences is used as the global weight sum; the nth sentence is then assigned a weight. The ratio of the boundary naturalness weight of the first sentence to the sum of the global weights is used as the value of the first sentence's naturalness weight. The normalized boundary weights of each sentence are calculated using the following formula:

[0046]

[0047] In the formula, For the first Boundary naturalness weights for each sentence; For the first Punctuation features at the end of a sentence; For the first The set of all historical prepositional words preceding the punctuation feature at the end of a sentence; The conditional generation probability of punctuation features output by the scoring model under the condition of a known set of historical prepositions; For the first Normalized boundary weights for each sentence; The total number of sentences; This represents the sum of global weights.

[0048] The formula for calculating the boundary naturalness weight directly extracts the forward prediction probability of the autoregressive network for a specific boundary symbol as the mapping value of the boundary naturalness weight. When the position of the punctuation feature conforms to the semantic completeness of natural language, the conditional generation probability output by the scoring model is extremely high, and the value of the obtained boundary naturalness weight increases accordingly. Conversely, when the punctuation feature abruptly and forcibly truncates continuous semantics, the conditional generation probability output by the scoring model drops sharply, and the value of the obtained boundary naturalness weight approaches 0. The obtained boundary naturalness weight represents the grammatical rationality of each sentence segmentation operation.

[0049] The formula for calculating the normalized boundary weights compares a single boundary naturalness weight with the sum of global weights. When there are many reasonable pauses in the text to be detected and a very small number of forced truncations without semantic basis are mixed in, the sum of global weights remains stable. At this time, the boundary naturalness weight corresponding to the forced truncation without semantic basis is extremely small. The value of the normalized boundary weight obtained by dividing it by the sum of global weights further decays and approaches 0. Conversely, if it is a normal natural pause, its corresponding normalized boundary weight increases. The obtained normalized boundary weights provide a quantitative benchmark for subsequently reducing the weight of forced truncation phrases without semantic basis from the statistical model and removing them.

[0050] S4. The sentence length variation coefficient is derived by weighting the number of characters in each sentence with the corresponding normalized boundary weights and projecting the result.

[0051] It should be noted that large language models are limited in their computational power allocation when dealing with complex instructions, which causes the text they generate to typically converge toward the average length in terms of length structure. Adversaries may attempt to construct a false discrete length distribution by randomly inserting punctuation features. This invention integrates the normalized boundary weights that characterize the authenticity of sentences into a statistical algorithm, replacing the equal weight ratio in the discreteness model. This allows false short sentences that violate semantic logic to be automatically eliminated in the weighted calculation, thereby restoring the true length fluctuation distribution of the text to be detected.

[0052] Specifically, the traditional statistical model, in which each sample contributes equally to the overall fluctuation, is reconstructed. The number of characters in each sentence is multiplied by its corresponding normalized boundary weights, and then summed. This sum replaces the equally weighted summation term in the arithmetic mean, and the weighted expected value of the number of characters is calculated. The specific formula is as follows:

[0053] In the formula, Weighted expectation based on the number of characters; The total number of sentences; For the first Normalized boundary weights for each sentence; For the first The number of characters in a sentence.

[0054] The formula for calculating the weighted expectation of character count replaces the equal weight coefficient with a normalized boundary weight, shifting the expected value of sentence length towards genuine, naturally paused sentences. When there are forged short sentences generated by adversarial scripts, the corresponding normalized boundary weights approach 0, greatly compressing the weight ratio of these extremely short characters during the accumulation process. This results in the calculated weighted expectation of character count stably reflecting the true length of the text, unaffected by the forged short sentences. Conversely, if all short sentences are genuine logical pauses, the weighted expectation of character count decreases with the actual length. The obtained weighted expectation of character count accurately reflects the average physical length after removing interference.

[0055] Furthermore, the square of the difference between the number of characters in each sentence and the weighted expected value of the number of characters is calculated. This squared value is then multiplied by its corresponding normalized boundary weights and summed. Finally, the square root of this sum is performed to replace the equal weighted bias calculation process in the standard deviation calculation, thus obtaining the character-weighted standard deviation. The specific calculation formula is as follows:

[0056] In the formula, Weighted standard deviation of character count; Weighted expectation based on the number of characters; The total number of sentences; For the first Normalized boundary weights for each sentence; For the first The number of characters in a sentence.

[0057] The formula for calculating the weighted standard deviation of character count uses normalized boundary weights to weight the squared terms of length deviation. When there are fake short sentences with forced segmentation, the squared terms containing the maximum length deviation of the fake short sentences are effectively compressed during the accumulation process, resulting in a significant decrease in the extracted weighted standard deviation of character count. Conversely, when the length span of the real sentences increases, the weighted standard deviation of character count increases accordingly. The obtained weighted standard deviation of character count eliminates the interference of fake fluctuations on the global dispersion assessment.

[0058] Finally, the weighted standard deviation of character count is divided by the weighted expected value of character count to calculate the coefficient of variation of sentence length. The specific formula is as follows:

[0059] In the formula, The coefficient of variation for sentence length; Weighted expectation based on the number of characters; The standard deviation of the number of characters is weighted; when the text exhibits homogeneous length convergence characteristics, the numerator of the calculation formula becomes smaller while the denominator remains relatively stable, resulting in a decrease in the calculated sentence length variation coefficient; conversely, when the text length exhibits real fluctuations, the sentence length variation coefficient increases; the obtained sentence length variation coefficient eliminates the scale influence of the overall text size and provides a basis for determining the discrete state of the text length.

[0060] S5. For all sentences, based on the variable correlation analysis framework, the character quantity sequence and the local text perplexity sequence are weighted and cross-mapped to calculate the long entropy correlation coefficient.

[0061] It should be noted that real humans are limited by the logical complexity of human thought when creating text. Short sentences are used for simple expression, making vocabulary highly predictable, while long sentences carry complex logic, making vocabulary highly unpredictable. The two show a high positive correlation. However, large language models rely on a globally consistent probability sampling mechanism for autoregressive operations. The random configuration of the generated sentences of different lengths remains unchanged, causing the predictability of sentence length and vocabulary dimension to fail at the feature level. This invention constructs a long entropy correlation coefficient to evaluate the degree of co-fluctuation between text length and information entropy, aiming to block the detection avoidance path of adversaries unilaterally forcibly increasing length fluctuation and vocabulary complexity from the algorithm's bottom layer.

[0062] It is worth noting that, taking the failure of feature association caused by the underlying fixed generation mechanism as an example, when real humans write extremely short sentences with very few characters, their vocabulary is extremely basic, resulting in very low local text perplexity. However, when conceiving complex long sentences with a large number of characters, the large vocabulary used leads to an increase in local text perplexity. However, large language models, due to the probability sampling mechanism that controls the randomness of output, remain constant during the generation process. Even when challenged by adversarial instructions to output sentences of varying lengths, they still maintain the same high probability of using complex and obscure words in both extremely short and extremely long sentences. This results in the perplexity curve remaining numerically stable even when the length curve fluctuates dramatically, and the two lose their synergistic correlation.

[0063] Specifically, for each sentence, the local text perplexity is calculated separately based on the conditional generation probability of the tokens belonging to that sentence in the conditional generation probability matrix. The local text perplexity of each sentence is multiplied by its corresponding normalized boundary weights and then summed to obtain the weighted expectation of the local perplexity. The square of the difference between the local text perplexity and the weighted expectation of the local perplexity is calculated, and this squared value is multiplied by its corresponding normalized boundary weights and summed. The summation result is then square-rooted to extract the weighted standard deviation of the local perplexity. The specific calculation formula is as follows:

[0064]

[0065] In the formula, Weighted expectation based on local perplexity; The total number of sentences; For the first Normalized boundary weights for each sentence; For the first Local text perplexity of each sentence; The standard deviation is the weighted average of the local perplexity.

[0066] The formulas for calculating the weighted expectation and weighted standard deviation of local perplexity are combined with the product of normalized boundary weights to accurately assess the average state and dispersion of local text perplexity after removing interference. When the words in a truly effective sentence are extremely complex, the local text perplexity increases, resulting in an increase in the calculated weighted expectation and weighted standard deviation of local perplexity; conversely, they decrease. The obtained parameters reflect the distribution characteristics of local lexical predictability.

[0067] Furthermore, the difference between the number of characters in each sentence and the weighted expected value of its corresponding number of characters is multiplied by the difference between the local text perplexity of each sentence and the weighted expected value of its corresponding local perplexity. This product is then multiplied by its corresponding normalized boundary weights and summed to calculate the weighted covariance. The specific formula is as follows:

[0068] In the formula, For weighted covariance; The total number of sentences; For the first Normalized boundary weights for each sentence; For the first The number of characters in each sentence; Weighted expectation based on the number of characters; For the first Local text perplexity of each sentence; Weighted expectation for local perplexity.

[0069] The formula for calculating the weighted covariance measures the cooperative direction of the fluctuations of the number of characters and the local text perplexity around their respective expectations. When the length increases and the word complexity increases, the product of the two differences with the same sign produces a positive value, which makes the calculated weighted covariance increase positively. When the fluctuations of the two are unrelated, the positive and negative values ​​cancel each other out, causing the weighted covariance to approach 0. The obtained weighted covariance reflects the cooperative trend of the two feature dimensions in the same direction.

[0070] Furthermore, the long entropy correlation coefficient is calculated by dividing the weighted covariance by the product of the character-weighted standard deviation and the local perplexity-weighted standard deviation. The specific formula is as follows:

[0071] In the formula, The correlation coefficient is the long entropy coefficient. For weighted covariance; The standard deviation is weighted by the local perplexity. Weighted standard deviation of character count; This is a preset, extremely small positive number used to prevent the denominator from being zero; its value range is typically set to 10. -8 Up to 10 -4 In this embodiment, the interval is preferably set to 10. -6 .

[0072] Specifically, when the large language model is forced to output text with fluctuating length by formatted prompts and the perplexity of the local text remains unchanged due to the temperature parameter being locked, the weighted covariance in the numerator decays rapidly due to the extremely small feature bias on one side. This ultimately leads to a significant decrease in the calculated long-entropy correlation coefficient, which approaches 0 or even becomes negative. When real human-generated text is used, the long-entropy correlation coefficient increases. The obtained long-entropy correlation coefficient accurately reflects the feature association failure phenomenon caused by the underlying operating mechanism of the large language model, providing a core quantitative basis for judgment.

[0073] S6. The sentence length variation coefficient, text perplexity, and long entropy correlation coefficient are nonlinearly mapped to construct a multidimensional feature judgment index, which is then compared with the true / false threshold for interception.

[0074] It should be noted that the text to be detected generated by the large language model exhibits high predictability of vocabulary and a fixed generation characteristic of stable convergence of macro-level sentence length. In order to transform this hidden distribution feature into an explicit warning scalar that can be intuitively monitored, it is necessary to perform inverse projection and nonlinear amplification operation on the extracted multidimensional feature quantity. This invention constructs an exponential decay compensation mechanism so that any local forgery operation that attempts to circumvent the single-index check will trigger an exponential increase in the overall parameter, thereby achieving full coverage and qualitative judgment of adversarial machine text.

[0075] Specifically, a feature data set including text perplexity, sentence length variation coefficient, and long entropy correlation coefficient is obtained. Three independent dimensional weight constants are pre-defined. The product of the sentence length variation coefficient and the first dimensional weight constant is multiplied by the reciprocal. The product of the natural logarithm of text perplexity and the second dimensional weight constant is multiplied by the reciprocal. The long entropy correlation coefficient is then multiplied by the natural exponential function and multiplied by the third dimensional weight constant, and the reciprocal is taken. The results of these three reciprocal calculations are linearly superimposed to calculate the multidimensional feature determination index. The specific calculation formula is as follows:

[0076] In the formula, It is a multidimensional feature determination index; The first dimension weight constant; The coefficient of variation for sentence length; The weight constant for the second dimension; It is the natural logarithm function; For text perplexity; The third dimension weight constant; It is a natural exponential function; The correlation coefficient is the long entropy coefficient. This is a preset, extremely small positive number used to prevent the denominator from being zero.

[0077] The calculation formula for the multidimensional feature determination index constructs a nonlinear amplification operation method targeting the fixed generation characteristics of the model by performing an inverse mapping on the multidimensional feature variables. When the adversary forcibly increases the sentence length fluctuation and vocabulary complexity, the values ​​of the first two terms, namely the sentence length variation coefficient and text perplexity, increase, causing their corresponding inverse terms to be compressed and reduced. Due to the solidification of the underlying sampling mechanism, the value of the long entropy correlation coefficient approaches 0 or a negative number, causing the natural exponential function term in the denominator of the third term of the calculation formula to show a significant decay, which in turn causes the value of the third inverse term to grow exponentially, driving the overall multidimensional feature determination index to increase rapidly and break through the true / false threshold. Conversely, if it is real human text, all parameters are normal and the third inverse term remains low, causing the value of the multidimensional feature determination index to decrease. The obtained multidimensional feature determination index cuts off the technical path of the adversary to simultaneously circumvent macro-syntactic detection and vocabulary complexity detection, and completes full-dimensional detection.

[0078] Furthermore, a threshold for determining the authenticity of real human text is pre-set. The multi-dimensional feature determination index is compared with the threshold. If the multi-dimensional feature determination index is greater than the threshold, the text to be detected is determined to be artificial intelligence-generated text, and a corresponding alarm signal is output.

[0079] The three independent dimension weight constants are set as follows: a training set is constructed by collecting massive amounts of real human text and machine-generated text sample data, a multi-objective optimization algorithm is used to perform grid search on each dimension weight constant, and the combination corresponding to the highest classification accuracy is selected as the numerical benchmark through cross-validation. The true / false threshold is set as follows: based on the historical validation data set, the distribution curves of false positive rate and false negative rate are plotted, and the intersection point of the equal error rate when the two reach a balance is found and extracted as the benchmark reference for calibration and confirmation.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An AI-generated text detection method based on sentence length distribution and text predictability features, characterized in that, include: The character sequence is parsed from the text to be detected, all punctuation features are extracted as sentence segmentation boundaries, the character sequence is divided into multiple sentences based on the punctuation features, and the number of characters in each sentence is counted. The forward propagation operation on the character sequence is performed using a scoring model to parse the conditional generation probability matrix, and the text perplexity is extracted by combining the information theory cross-entropy model. Extract the conditional generation probability of punctuation features from the perspective of the scoring model, construct boundary naturalness weights, and transform the boundary naturalness weights into normalized boundary weights; The sentence length variation coefficient is obtained by weighting the number of characters in each sentence with the corresponding normalized boundary weights and projecting the result. For all sentences, the local text perplexity corresponding to each sentence is obtained based on the conditional generation probability matrix. Based on the variable correlation analysis framework, the character quantity sequence and the local text perplexity sequence are weighted and cross-mapped to obtain the long entropy correlation coefficient. A multidimensional feature determination index is constructed by nonlinearly mapping the sentence length variation coefficient, text perplexity, and long entropy correlation coefficient, and then compared with the true / false threshold for interception.

2. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The process involves parsing a character sequence from the text to be detected, extracting all punctuation features as sentence segmentation boundaries, dividing the character sequence into multiple sentences based on the punctuation features, and counting the number of characters in each sentence, including: The input text to be detected is decoded using a unified character encoding standard to parse out the character sequence. The character sequence is then traversed and matched with a punctuation feature library containing periods, question marks, exclamation marks, and semicolons to extract all punctuation features that serve as sentence segmentation boundaries. Based on the punctuation features, the character sequence is divided into multiple sentences. After removing spaces and control characters from each sentence, the number of characters contained in each sentence is counted.

3. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The step of performing forward propagation on the character sequence using a scoring model to parse out the conditional generation probability matrix includes: Obtain the vocabulary dictionary and segmentation rules based on the byte pair encoding algorithm used by the scoring model in the initial training phase; use the segmentation rules to perform sub-word matching on the character sequence, and convert the character sequence into a discrete sequence containing multiple text words; Extract the unique integer index value corresponding to each text word in the discrete sequence from the vocabulary dictionary to obtain a word index sequence composed of unique integer index values; The word index sequence is input into the scoring model to perform forward propagation operation, and the output logical value vector of the last hidden layer of the scoring model is obtained. For each element value in the output logical value vector, the normalized exponential function operation is performed to parse out the conditional generation probability matrix containing the entire vocabulary dictionary at each time step.

4. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The text perplexity extracted using the information-theoretic cross-entropy model includes: Calculate the natural logarithm of the conditional generation probability of each word in the character sequence, sum the natural logarithms of all words, divide the sum by the total number of words in the text to be detected, take the negative number, and perform a natural exponential function operation on the negative number to obtain the text perplexity.

5. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The process of converting boundary naturalness weights into normalized boundary weights includes: Calculate the sum of the boundary naturalness weights of all sentences as the global weight sum; divide the boundary naturalness weight of each sentence by the global weight sum to obtain the normalized boundary weight.

6. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The step of weighting and projecting the number of characters in each sentence with the corresponding normalized boundary weights to obtain the sentence length variation coefficient includes: The character count of each sentence is multiplied by its corresponding normalized boundary weight, and then summed to obtain the character count weighted expectation. Calculate the square of the difference between the number of characters in each sentence and the weighted expected value of the number of characters. Multiply the squared value by its corresponding normalized boundary weight and sum them up. Then, take the square root of the sum to obtain the weighted standard deviation of the number of characters. The sentence length variation coefficient is obtained by dividing the weighted standard deviation of the number of characters by the weighted expected value of the number of characters.

7. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 6, characterized in that, The process of obtaining the local text perplexity for each sentence based on the conditional generation probability matrix includes: Based on the conditional generation probabilities of the words belonging to the sentence in the conditional generation probability matrix, the local text perplexity of the sentence is calculated. The local text perplexity of each sentence is multiplied by its corresponding normalized boundary weight and then summed to obtain the weighted expectation of the local perplexity. Calculate the square of the difference between the local text perplexity and the weighted expected value of the local perplexity for each sentence. Multiply the squared value by its corresponding normalized boundary weight and sum them up. Then, take the square root of the sum to obtain the weighted standard deviation of the local perplexity.

8. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 7, characterized in that, The variable correlation analysis framework describes a weighted cross-mapping of the character quantity sequence and the local text perplexity sequence to obtain the long-entropy correlation coefficient, including: The difference between the number of characters in each sentence and the weighted expectation of the number of characters is multiplied by the difference between the local text perplexity and the weighted expectation of the local perplexity in each sentence. The product is then multiplied by its corresponding normalized boundary weight and summed to obtain the weighted covariance. Divide the weighted covariance by the product of the character-weighted standard deviation and the local perplexity-weighted standard deviation to obtain the long entropy correlation coefficient.

9. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The formula for calculating the multidimensional feature determination index is: ； In the formula, It is a multidimensional feature determination index; The first dimension weight constant; The coefficient of variation for sentence length; The weight constant for the second dimension; It is the natural logarithm function; For text perplexity; The third dimension weight constant; It is a natural exponential function; The correlation coefficient is the long entropy coefficient. It is a preset minimum positive number.

10. The AI-generated text detection method based on sentence length distribution and text predictability features according to claim 1, characterized in that, The interception based on the comparison with the true / false threshold includes: Pre-set a threshold for authenticity to define the boundaries of real human text; The multidimensional feature determination index is compared with the authenticity threshold. If the multidimensional feature determination index is greater than the authenticity threshold, the text to be detected is determined to be artificial intelligence-generated text, and a corresponding alarm signal is output.

Images