AI-Generated Text Detection Method, System, and Media Guided by Proxy Model

Abstract

This invention relates to the field of AI-generated content and information authentication, and discloses an AI-generated text detection method, system, and medium based on a proxy model. This method uses an open-source large language model as a proxy model. The text to be detected is input into the proxy model, and the probability distribution information of word elements at each position in the text under contextual conditions is calculated. Based on the probability distribution information, key words are selected by combining the local entropy, confidence, and linguistic characteristics of each position in the text. Subsequently, a contextual template is constructed, and black-box resampling is performed using the target model to generate probability feature vectors. Finally, the probability features are fused with semantic features, and a classification network is used to determine whether the text was generated by AI or created by humans. The AI-generated text detection of this invention is efficient, accurate, and universally applicable.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-generated content and information authentication technology, specifically to an AI-generated text detection method, system, and medium based on a proxy model. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLM) have made groundbreaking progress in the field of text generation. Current mainstream LLM-generated text content has reached unprecedented levels of quality, fluency, and semantic coherence, highly mimicking human writing styles and even becoming indistinguishable from human creation in certain scenarios. While this technological advancement brings convenience to content creation and information processing, it also poses serious challenges to several key social sectors. In education, students can use AI tools to generate complete assignments and academic papers, bypassing normal learning processes and skills development; in news media, AI can quickly and massively produce seemingly authentic information content, potentially fueling the spread of misinformation and manipulation of public opinion; in academic research, AI-generated content may infiltrate scientific literature, affecting the integrity of research data and the reliability of scientific conclusions. Therefore, accurately identifying whether text is generated by AI or created by humans has become a critical technological requirement for maintaining information authenticity, intellectual property rights, and academic integrity, with broad application value and significant social implications.

[0003] Current technologies for recognizing AI-generated text can be broadly categorized into two types: black-box methods and white-box methods. Black-box methods do not rely on accessing the internal structure of the generative model; instead, they detect AI-generated content by analyzing surface features of the text. Their core lies in building a general classifier independent of specific models. However, existing black-box detection methods overly rely on static language features, failing to fully explore the probabilistic distribution characteristics of text. Traditional black-box methods primarily analyze and judge based on surface statistical features such as word frequency distribution, syntactic complexity, and lexical diversity. These features are increasingly resembling human creative patterns as AI models iterate and optimize. In particular, with the significant improvement in the semantic understanding and generation capabilities of large language models, the discriminative power of these static features is continuously declining, leading to a decrease in detection accuracy. More importantly, black-box methods lack a direct understanding and utilization of the probabilistic distribution characteristics of AI-generated text, failing to effectively capture the unique probability distribution patterns of AI models in word selection—a crucial element distinguishing AI from human creation. Existing black-box methods focus more on analyzing "what the text is" rather than deeply understanding the intrinsic mechanisms of "how the text is generated," resulting in fundamental limitations in detection capabilities.

[0004] White-box methods rely on accessing the internal mechanisms of the generative model, with their technical implementation primarily focusing on two directions: controllable generation and implicit labeling. While theoretically offering higher accuracy and interpretability, white-box detection methods face significant implementation obstacles in practical applications. Most mainstream commercial large language models currently employ closed-source design strategies, preventing third parties from accessing crucial information such as the model's internal probability distribution, parameter structure, or generation path. This necessitates active cooperation from model providers for the implementation of traditional white-box methods, hindering independent deployment and verification. Furthermore, white-box detection mechanisms designed for specific model architectures are often highly specific, making effective migration to other models or novel architectures difficult and severely limiting their adaptability. These implementation difficulties significantly reduce the feasibility of white-box methods in real-world applications, confining them primarily to the theoretical research stage and failing to meet the detection needs of multiple models and scenarios in practical applications.

[0005] Therefore, the core deficiency in the current technological system lies in the lack of effective intermediate representation methods, failing to establish a technical path that accurately captures the probabilistic characteristics of generated text without relying on the internal mechanisms of the original model. A significant technological gap exists between black-box and white-box methods, lacking an intermediate solution that balances practicality and detection efficiency. In particular, existing technologies have failed to effectively address the crucial issue of accurately simulating and utilizing the probability distribution characteristics of the model without accessing its internal mechanisms, making it difficult for detection technologies to simultaneously possess the dual advantages of high accuracy and broad applicability. Furthermore, existing methods lack general detection capabilities for content generated from models of different sources, making it difficult to adapt to the rapid iterative development of AI technology, and their long-term effectiveness is challenged. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides an AI-generated text detection method, system, and medium based on a proxy model. It successfully bridges the technical gap between black-box and white-box methods, maintaining the ease of use and wide applicability of black-box methods while utilizing the probabilistic sensitivity of white-box methods. This provides a new, efficient, accurate, and universally applicable solution for AI-generated text detection.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention discloses an AI-generated text detection method guided by a proxy model, comprising the following steps:

[0009] S1. Use an open-source large language model as a proxy model. Input the text to be detected into the proxy model and calculate the probability distribution information of the occurrence of words at each position in the text under the context conditions.

[0010] S2. Based on probability distribution information, a comprehensive score is calculated using a multidimensional evaluation framework to characterize the discriminative value of each word in the text to be detected, and the set of key words is selected by prioritizing the comprehensive scores from high to low. The comprehensive score is a weighted sum of the local entropy ratio, prediction confidence index, and linguistic characteristic index at each position in the text to be detected. The local entropy ratio is used to characterize the uncertainty differences of words in their local context, and the linguistic characteristic index is used to characterize the discriminative value of words for the text detection task across multiple linguistic dimensions.

[0011] S3. Construct a context-preserving template for each word element in the keyword element set, perform black-box resampling on the template using the target language model, statistically analyze the candidate word frequencies based on the sampling results, construct a pseudo-probability distribution, and then generate a probability feature vector;

[0012] S4. Extract the contextual semantic feature vector of the text to be detected, fuse it with the probability feature vector, and output the detection result of whether the text to be detected was generated by the target language model or created by a human through a classification network.

[0013] As a further improvement to the above scheme, in step S2, the calculation methods for the local entropy ratio index, prediction confidence index, and linguistic characteristic index are as follows:

[0014] The information entropy of the current position is calculated using the probability distribution information output by the surrogate model. A local context window centered on the current position is set, and the average value of the information entropy of all positions within the window is calculated. The ratio of the information entropy of the current position to the average value is taken as the local entropy ratio of the current position.

[0015] Calculate the prediction confidence score of the proxy model for the actual occurrence of the word at the current position; calculate the average prediction confidence score and standard deviation of the entire text to be detected, and construct an adaptive error threshold accordingly; use the adaptive error threshold to normalize the prediction confidence score at the current position to obtain the prediction confidence index.

[0016] The first importance score based on part-of-speech and the second importance score based on semantics are calculated for each lexical unit at each position. The first importance score and the second importance score are then fused by weighting to obtain a comprehensive linguistic feature index. The first importance score is obtained by assigning a preset weight to the part-of-speech tag to which the lexical unit belongs. The second importance score is obtained by calculating the similarity between the semantic embedding vector of the lexical unit and the global semantic embedding vector of the entire text to be detected.

[0017] As a further improvement to the above scheme, in step S2, position constraint processing and stability verification processing are also performed during the process of filtering the keyword meta set.

[0018] The position constraint processing includes: ensuring that the distance between any two selected candidate words in the text to be detected is not less than a preset minimum spacing value; if a candidate word does not meet the spacing requirement, then the next priority word word is selected.

[0019] The stability verification process includes: performing multiple random resampling evaluations on the text segment containing the candidate word element, statistically analyzing the mean comprehensive score and standard deviation of the comprehensive score change of the word element under different contextual conditions, and calculating a stability index based on this. This index is used to reflect the ability of the word element to maintain high discriminative value under different contextual conditions. The stability index and the comprehensive score are weighted to obtain the final score, and the word elements corresponding to the top K highest final scores are selected to form the keyword word element set.

[0020] As a further improvement to the above scheme, the value of K is set to 5% to 10% of the total length of the text to be detected, and the value of K has a lower limit and an upper limit; when performing position constraint processing, if the candidate word element cannot meet the minimum spacing requirement, the word element with the highest score is abandoned, and the word element with the second best score but can meet the position distribution requirement is selected to be added to the keyword word element set.

[0021] As a further improvement to the above scheme, step S3 specifically includes:

[0022] For each key position in the keyword metaset, while keeping the context of the rest of the text unchanged, the key position is replaced with a mask marker to construct a context-preserving template;

[0023] The context-preserving template is input into the target language model, and a random sampling strategy with a temperature parameter is used to perform multiple rounds of repeated sampling to obtain the set of candidate words output by the key position in different sampling rounds.

[0024] The occurrence frequency of each candidate word in the candidate word set is calculated, and a weighted smoothing algorithm is applied to the frequency statistics to avoid the zero probability problem and improve the stability of the pseudo-probability distribution, thereby constructing a pseudo-probability distribution that reflects the word selection tendency of the target language model in a black box state.

[0025] Based on the pseudo-probability distribution, basic frequency features and probability distribution features are extracted respectively; wherein, the basic frequency features consist of the pseudo-probability distribution of the most frequently occurring candidate words; and the probability distribution features consist of statistical indicators used to quantify the distribution characteristics.

[0026] The basic frequency features and probability distribution features corresponding to all word elements in the keyword element set are concatenated in order to form the original feature vector; then the original feature vector is subjected to feature standardization processing to obtain the text-level probability feature vector.

[0027] As a further improvement to the above scheme, the statistical indicators include the information entropy of the pseudo-probability distribution, the KL divergence between the pseudo-probability distribution and the uniform distribution, the Gini coefficient, the occurrence frequency of the most frequent candidate words, the ratio of the sum of the occurrence frequencies of the top three most frequent candidate words to the total number of samples, and the ratio of the entropy value of the pseudo-probability distribution to the maximum possible entropy value.

[0028] As a further improvement to the above scheme, in step S4, a pre-trained language model is used as a feature extractor. The text to be detected is input into the feature extractor, and the context semantic feature vector is output.

[0029] The probabilistic feature vectors are reduced in dimensionality and normalized using a multilayer perceptron to obtain the processed probabilistic feature vectors.

[0030] A cross-attention mechanism is used to fuse the contextual semantic feature vector and the processed probability feature vector, and the output fused feature vector is used for final classification.

[0031] As a further improvement to the above scheme, in step S4, the classification network supports binary classification tasks or multi-class classification tasks. The binary classification task is used to output the probability that the text to be detected is generated by a single target language model, and the multi-class classification task is used to output the probability distribution that the text to be detected is generated by multiple target language models. The cross-entropy loss function is used for optimization during the training process.

[0032] This invention also discloses an AI-generated text detection system guided by a proxy model, which applies the AI-generated text detection method guided by a proxy model as described above; the detection system includes:

[0033] The general proxy model module is used to use an open-source large language model as a proxy model. The text to be detected is input into the proxy model, and the probability distribution information of the word at each position in the text under the context conditions is calculated.

[0034] A lexical selector is used to calculate a comprehensive score based on probability distribution information and a multidimensional evaluation framework to characterize the discrimination value of each lexical in the text to be detected, and to select a set of key lexicals according to the priority of the comprehensive score from high to low; wherein, the comprehensive score is a weighted sum based on the local entropy ratio index, prediction confidence index and linguistic characteristic index of each position in the text to be detected.

[0035] The probabilistic feature extractor is used to construct a context-preserving template for each word in the keyword set, and to perform black-box resampling on the template using the target language model. Based on the sampling results, the frequency of candidate words is counted and a pseudo-probability distribution is constructed, thereby generating a probabilistic feature vector.

[0036] A multi-feature fusion classifier is used to extract the contextual semantic feature vector of the text to be detected, fuse it with the probability feature vector, and output the detection result of whether the text to be detected was generated by the target language model or created by a human through a classification network.

[0037] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the AI-generated text detection method based on the agent model as described above.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] This invention is applicable to various Chinese text content detection scenarios. Through character-level probabilistic feature analysis, it more accurately captures the features of AI-generated text, providing strong technical support for content review, academic integrity, and information authenticity verification, achieving significant results.

[0040] First, this invention innovatively constructs a universal surrogate model and an error-sensitive word selector, successfully bridging the technical gap between black-box and white-box detection methods. By simulating the probability distribution characteristics of the target language model through the surrogate model, it effectively captures the features of AI-generated text without accessing the internal structure of the original model, overcoming the technical bottleneck of traditional black-box methods in obtaining deep probabilistic features. The error-sensitive word selector accurately identifies the most discriminative key positions through information entropy analysis and prediction error sensitivity assessment, significantly improving the targeting and efficiency of detection and avoiding the waste of resources in full-text analysis. These two innovations enable this invention to achieve higher detection accuracy while maintaining ease of use.

[0041] Secondly, the black-box language model sampler and multi-feature fusion classification system of this invention achieve high-precision recognition and wide applicability of AI-generated content. The black-box sampling mechanism constructs pseudo-probabilistic feature vectors by resampling keywords multiple times, cleverly obtaining probability distribution features without accessing the model's internal structure, making this invention applicable to content detection generated by various commercial closed-source models. The multi-feature fusion classification system combines probabilistic features and contextual features, achieving deep fusion through a cross-attention mechanism, enhancing the ability to recognize high-quality AI-generated content.

[0042] This invention not only solves the problems of insufficient accuracy, poor adaptability and high resource consumption in the existing technology, but more importantly, it opens up a new paradigm for AI-generated text detection based on probability distribution analysis, providing a forward-looking solution to cope with the ever-evolving AI generation technology. Attached Figure Description

[0043] Figure 1This is a flowchart of the AI-generated text detection method guided by the proxy model in Embodiment 1 of the present invention.

[0044] Figure 2 This is a schematic diagram of the framework of the AI-generated text detection method guided by the proxy model in Embodiment 1 of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] Addressing the three core challenges of current AI-generated text detection technologies—low accuracy in black-box detection, narrow applicability of white-box detection, and difficulty in recognizing high-quality generated content—this embodiment provides an AI-generated text detection method guided by a surrogate model. The core idea is to differentiate between AI-generated text and human-created text by leveraging the differences in probability distributions exhibited during semantic selection. Although different AI large-scale language models differ in their detailed implementations, they all follow similar probability distribution patterns when generating text, patterns that are fundamentally different from those of human-created text. This invention achieves high-precision recognition of various types of AI-generated content by constructing a universal surrogate model and a probabilistic feature extraction mechanism, working effectively even without accessing the internal structure of the original model.

[0048] Please see Figure 1 and Figure 2 The detection method includes steps S1 to S4.

[0049] S1. Use an open-source large language model as a proxy model. Input the text to be detected into the proxy model and calculate the probability distribution information of the occurrence of tokens at each position in the text under the context conditions.

[0050] This invention constructs a general proxy model capable of effectively extracting common features of AI-generated text. The model captures and calculates the probability distribution patterns exhibited by various large language models during the text generation process. Since various large language models follow similar probability distribution patterns when generating text, selecting a representative large language model as a proxy can effectively capture the common features of various AI-generated texts.

[0051] This invention uses an open-source large language model as a proxy model. This model calculates the value of each position in the text. Upper word element conditional probability The selection criteria mainly consider the model's generality, stability, and inference efficiency.

[0052] The surrogate model in this invention primarily performs two core functions. First, it calculates the conditional probability distribution for each position in the input text, providing foundational data for subsequent lexical selection and feature extraction. The model not only outputs the most probable prediction but, more importantly, provides complete probability distribution information, which is crucial for capturing the distributional differences between AI-generated text and human text. Simultaneously, through its deep semantic understanding of text content, the surrogate model establishes a unified feature evaluation space, enabling comparisons of texts from different sources under the same standard, ensuring the consistency and reliability of detection results. This probability distribution-based analysis method avoids the limitations of traditional feature engineering and can adaptively identify AI-generated traces in various types of text, regardless of their surface style similarity.

[0053] S2. Based on probability distribution information, a comprehensive score is calculated using a multidimensional evaluation framework to characterize the discriminative value of each word in the text to be detected. Key word sets are then selected based on the priority of the comprehensive score, from highest to lowest. The comprehensive score is a weighted sum of the local entropy ratio, prediction confidence index, and linguistic characteristic index at each position in the text to be detected. The local entropy ratio is used to characterize the uncertainty difference of a word in its local context. By comparing the prediction entropy of the surrogate model for the same word at different context windows, the context sensitivity of the word is quantified. The linguistic characteristic index is used to characterize the discriminative value of a word in distinguishing between human-generated and machine-generated text in linguistic dimensions such as part of speech, semantic category, and word frequency. Word types with higher discriminative value, such as content words, low-frequency words, and domain-specific words, are preferentially selected.

[0054] The following is the theoretical basis for word selection in step S2:

[0055] The core innovation of this invention lies in discovering and utilizing the fundamental differences in the probability distribution of word selection between AI-generated text and human-created text. Unlike traditional methods that perform full analysis of the entire text, this invention proposes an "error-sensitive word selector." Its theoretical basis is that not all words in a text have equal discriminative value. The positions of words that the surrogate model predicts with high error rates or high uncertainty often contain key signals that distinguish AI from human creation.

[0056] The error-sensitive lexical selector accurately identifies the most discriminative key positions in the text by deeply analyzing the inference results of the surrogate model. This module uses information theory and statistical principles to construct a multi-dimensional evaluation framework to select the most representative samples that best reflect the text generation mechanism from a massive number of lexical units, thereby improving the efficiency and discriminative power of subsequent feature extraction.

[0057] Specifically, step S2 may include the following aspects, namely (2.1) to (2.5).

[0058] (2.1) Information entropy analysis mechanism

[0059] Information entropy is a key indicator for quantifying prediction uncertainty. This invention utilizes the lexical probability distribution output by the surrogate model to calculate the information entropy of each lexical:

[0060] ;

[0061] in, Indicates word elements Information entropy; Represents a lexical table, In the context Under the condition, the word element The probability of occurrence. The higher the information entropy value, the greater the prediction uncertainty at that position, and the more potential candidate words there are. The information entropy distribution of AI-generated text shows a clear regular pattern, while the information entropy distribution of human-created text is more random and fluctuates more.

[0062] This invention designs an adaptive entropy threshold filtering algorithm to dynamically determine high information entropy regions and identify the peak position of information entropy through local maximum detection, ensuring that the selected word position has significant uncertainty characteristics in the context.

[0063] Specifically, for each position in the text Calculate its local entropy ratio :

[0064] ;

[0065] in This is the width of the local window. When... At that time, the position These are marked as high-entropy points. This adaptive filtering method based on relative entropy can effectively adapt to changes in different text types and styles, ensuring that the selected word positions have significant uncertainty features in the context.

[0066] (2.2) Prediction error sensitivity analysis

[0067] In addition to information entropy, this invention also introduces prediction error sensitivity analysis to identify locations where the surrogate model's prediction results differ significantly from the actual text. For each location in the text... Calculate the prediction confidence score:

[0068] ;

[0069] In the formula, Indicates position lexical elements The prediction confidence score; This indicates that the proxy model is in a given context. Under the condition of, for the actual occurrence of word elements The predicted probability; the confidence score of this prediction reflects the surrogate model's confidence in the actual occurrence of the word. The higher the score, the less accurate the model's prediction; that is, the lower the probability the model assigns to the actual word.

[0070] The inherent complexity and predictability of different texts vary significantly. To achieve fair comparison across texts, this invention designs an adaptive error threshold algorithm:

[0071] ;

[0072] In the formula, and These are texts The average prediction confidence score and standard deviation, It is an adjustable parameter. This adaptive error threshold... In subsequent comprehensive scoring, it serves as a normalization factor, enabling the prediction error index to adapt to the difficulty and style characteristics of different texts.

[0073] (2.3) Analysis of Linguistic Characteristics

[0074] To further improve the accuracy of vocabulary selection, this invention introduces linguistic feature analysis to quantitatively evaluate the discriminative value of words from both lexical and semantic dimensions.

[0075] The distribution characteristics of words with different parts of speech differ significantly between AI-generated text and human-created text. This invention calculates an importance score based on part of speech (i.e., the first importance score):

[0076] ;

[0077] in, It is a word element Based on the importance score of part of speech, It is a word element Part-of-speech tags, It is a mapping function that assigns weights to different parts of speech.

[0078] The semantic relevance of words to the text's topic also affects their discriminative value. Semantic importance scoring is quantified by calculating the relevance of lexical units to the overall text topic (i.e., the second importance score):

[0079] ;

[0080] in, It is a word element The relevance of the overall text topic It is a word element semantic embedding vector, It is the semantic embedding vector of the entire text T, and cos represents the cosine similarity.

[0081] By integrating part-of-speech and semantic scores, a comprehensive linguistic feature score is constructed.

[0082] ;

[0083] in, It is a word element A comprehensive linguistic feature score, This is a balancing parameter, and it is recommended to set it to 0.6, which means that part-of-speech features account for 60% of the weight and semantic features account for 40% of the weight.

[0084] (2.4) Top-K vocabulary selection algorithm

[0085] Taking into account entropy, prediction error, and linguistic characteristics, this invention selects the Top-K vocabulary selection algorithm, which first calculates a comprehensive score for each position:

[0086] ;

[0087] In the formula, For position Overall score It is the ratio of local entropy, which characterizes location. Relative uncertainty in a local context; It is a normalized prediction confidence score, which characterizes the degree of prediction bias of the surrogate model for this location; It is a comprehensive linguistic feature score that characterizes the discriminative value of parts of speech and semantics; to These are the corresponding weighting coefficients, satisfying... and .

[0088] After calculating the overall score for each position, the Top-K word selection algorithm selects the k positions with the highest scores as key discrimination positions, forming a keyword metaset. . The value is typically set to 5%-10% of the text length, with a minimum of 5 and a maximum of 50, to balance efficiency and coverage.

[0089] To ensure that the selected vocabulary has sufficient coverage and representativeness, this invention introduces a positional constraint mechanism to avoid keywords being overly concentrated in specific areas of the text:

[0090] ;

[0091] in, Indicates position and location The distance between the corresponding lexical units, that is, the difference between the two position indices. It is the total length of the text. This is the maximum spacing limit (30 is recommended). This constraint ensures that the selected keywords are evenly distributed throughout the text, comprehensively reflecting the text's generation characteristics. For candidate words that do not meet the constraint, the system will select the next best words that meet the positional distribution requirements. For short texts, the constraint will automatically be relaxed to fit the text length.

[0092] (2.5) Resampling optimization mechanism

[0093] For the selected keywords, this invention designs a resampling optimization mechanism to further improve the stability and reliability of keyword selection. This mechanism involves repeatedly evaluating N randomly selected text segments, statistically analyzing the score changes of each word position under different contextual conditions, and calculating a stability index.

[0094] ;

[0095] In the formula, as a word element Stability index; It is a word element The standard deviation of the score across N samples. This is the average score. The stability index reflects the ability of a word to maintain a high discriminative value under different contextual conditions; the higher the index (closer to 1), the more stable the discriminative characteristics of the word.

[0096] In the final selection process, this invention comprehensively considers both the score height and stability, and adopts a weighted scoring method:

[0097] ;

[0098] in, as a word element Weighted score, This is a balancing parameter. Based on this final score, the Top-K words are selected to form the final keyword set, ensuring that the selected word set S has both high discriminative value and good robustness.

[0099] S3. Construct a context-preserving template for each word in the keyword set, perform black-box resampling on the template using the target language model, statistically analyze the candidate word frequencies based on the sampling results, construct a pseudo-probability distribution, and then generate a probability feature vector.

[0100] The design goal of probabilistic feature extraction is to obtain the probabilistic features of text through external interaction without directly accessing the internal mechanisms of the target language model. This step is based on the black-box sampling principle. By constructing a specific context template, multiple rounds of resampling are performed on the selected Top-K keyword positions to obtain the word selection probability distribution features of the large language model. Step S3 includes the following aspects, namely (3.1) to (3.3).

[0101] (3.1) Black-box sampling framework

[0102] The probabilistic feature extractor is designed to obtain probabilistic features of text through external interaction without directly accessing the internal mechanisms of the target language model. Based on the principle of black-box sampling, this module constructs specific context templates and performs multiple rounds of resampling on the selected Top-K keyword positions to obtain the word selection probability distribution features of the large language model.

[0103] For vocabulary set lexical units at each position in This invention constructs a context-preserving template:

[0104] ;

[0105] in This is the location where the target language model needs to be filled. n The length of the text to be detected.

[0106] By submitting this template to the target language model, the vocabulary sampling results at that location are obtained. To enhance the diversity of sampling, this invention employs a temperature sampling strategy, which involves adjusting temperature parameters. Controlling the randomness of sampling:

[0107] ;

[0108] In the formula, exp(·) is an exponential function with the natural constant e as its base. In this embodiment, experiments show that setting... A value between 1.0 and 1.2 yields the best sampling results, preserving the basic characteristics of the original probability distribution while introducing sufficient randomness to observe the occurrence of different candidate words.

[0109] Perform analysis on each keyword location Multiple sampling sessions are used to balance accuracy and efficiency. To avoid contextual interference during the sampling process, the original text context is restored after each sampling to ensure that each sampling is independent of the others.

[0110] (3.2) Frequency statistics and pseudoprobability construction

[0111] This invention performs statistical analysis on the sampling results of each keyword position, calculates the occurrence frequency of each candidate word, and constructs a pseudo-probability distribution:

[0112] ;

[0113] in, Indicate candidate words In position pseudoprobability, It is a candidate word exist The number of occurrences in each sample. This frequency-based pseudo-probability distribution is an approximate estimate of the probability distribution within the target language model, reflecting the probabilistic characteristics of word selection without accessing the model's internal mechanisms.

[0114] To improve the stability of the pseudo-probability distribution, this invention introduces a smoothing mechanism, applying an additive smoothing algorithm to all candidate words in the candidate word set:

[0115] ;

[0116] In the formula, Indicate candidate words In position The pseudo-probability after smoothing. It is a smoothing parameter. This represents the number of observed candidate word types. This smoothing process avoids the zero-probability problem and improves the stability of the feature vector.

[0117] (3.3) Based on the sampling statistics, this invention constructs two types of probabilistic feature vectors: basic frequency features and probability distribution features.

[0118] The basic frequency features are constructed directly using the observed candidate word frequencies:

[0119] ;

[0120] In the formula, It is the most frequently observed There are candidate word elements, parameters The choice is related to vocabulary size and sampling diversity.

[0121] The probability distribution characteristics are then constructed by calculating the statistical properties of the pseudo-probability distribution:

[0122] ;

[0123] In the formula, Indicates word elements The probability distribution characteristics; It is the entropy value of the pseudo-probability distribution; It is the KL divergence between the pseudo-probability distribution and the uniform distribution; It is the frequency of the most frequent candidate word; It is the Gini coefficient, which measures the degree of inequality in distribution; It is the ratio of the sum of the frequencies of the top three high-frequency candidate words to the total number of samples; It is the ratio of the pseudo-probability distribution entropy value to the maximum possible entropy value, which is the standardized entropy. These statistical properties together constitute a feature set that comprehensively describes the probability distribution pattern of word selection, and can effectively distinguish the difference in probability distribution between AI-generated and human-created text.

[0124] Finally, this invention concatenates the feature vectors of all keyword locations to form a text-level probabilistic feature vector. :

[0125] ;

[0126] in It is a feature standardization function, which can be standardized using Z-score to ensure that feature vectors generated from texts of different lengths are comparable. .

[0127] This feature vector captures the probability distribution characteristics of keyword selection in text, providing an important basis for subsequent classification. Compared with traditional static language features, this dynamic feature based on probability distribution has a significant advantage in distinguishing between AI-generated and human-created text, especially showing higher robustness when facing high-quality AI-generated content.

[0128] S4. Extract the contextual semantic feature vector of the text to be detected, fuse it with the probability feature vector, and output the detection result of whether the text to be detected was generated by the target language model or created by a human through a classification network.

[0129] Step S4, multi-feature fusion classification, is the decision-making step of this invention. It is responsible for integrating contextual language features and probability distribution features to achieve accurate identification of the text generation source. The feature fusion architecture includes three key components: a contextual feature extractor, a probabilistic feature processor, and a fusion classification network.

[0130] The context feature extractor, based on a pre-trained language model, extracts deep semantic features from text content. This invention uses the BERT-base-Chinese pre-trained model as the foundation of the feature extractor. For input text T, the context feature extractor outputs a feature vector. :

[0131] ;

[0132] The probabilistic feature processor processes the probabilistic feature vectors generated by the previous module. Compression and normalization are performed to reduce feature dimensionality and improve feature quality, resulting in the processed feature vector. :

[0133] ;

[0134] in It is a feature normalization operation. It is a multilayer perceptron used for feature dimensionality reduction and nonlinear transformation. The fusion classification network is responsible for integrating two types of features and making the final decision.

[0135] This invention employs a cross-attention fusion mechanism to perform feature fusion, resulting in a fused feature vector. :

[0136] ;

[0137] In the formula, This is a cross-attention fusion mechanism; this mechanism allows bidirectional interaction between contextual features and probabilistic features, capturing deep correlations between the two types of features, thereby forming a more expressive fusion feature representation.

[0138] This invention also designs a flexible classification task framework, which can be configured with different classification tasks according to actual application needs and training requirements:

[0139] (4.1) Implementation of binary classification task

[0140] The basic binary classification task categorizes text into "AI-generated" and "human-created" types, which is the core function of the system. The binary classification model uses fully connected layers and a sigmoid activation function, outputting the probability that the text is AI-generated.

[0141] ;

[0142] In the formula, Given the text to be detected At that time, the text belongs to the category The probability of; As category labels, in binary classification tasks ,in Indicates "AI generated", It signifies "human creation"; The text to be detected; Here is the weight matrix of the binary classification layer. This is the bias scalar for the binary classification layer.

[0143] The training process uses a binary classification cross-entropy loss function for optimization.

[0144] ;

[0145] In the formula, For the index of the training samples, , This represents the total number of samples in the training batch. For the sample The true label; Predict samples for the model The probability of generating text for AI.

[0146] Binary classification models are particularly suitable for applications requiring efficient decision-making, such as real-time content moderation and initial screening.

[0147] (4.2) Implementation of multi-classification tasks

[0148] Multi-class classification tasks further refine the target recognition, not only distinguishing between AI-generated and human-created content, but also identifying specific generation model types or the degree of text mixing. The multi-class model uses fully connected layers and the Softmax activation function to output the probability distribution for each category:

[0149] ;

[0150] The training process uses a multi-class cross-entropy loss function:

[0151] ;

[0152] In the formula, For the sample The true label one-hot encoding, when the sample The true category is hour ,otherwise ; Predict samples for the model Category The probability of.

[0153] Multi-classification models support model source identification, such as classifying text as "human creation", "GPT generation", "Claude generation", "LLaMA generation", etc.

[0154] This invention supports flexible switching between binary and multi-class classification modes according to actual needs, and can also train two models simultaneously to meet the requirements of different application scenarios. The multi-class classification capability significantly enhances the system's practicality and adaptability, enabling it to serve a wider range of application areas.

[0155] The key technical points of this invention are mainly reflected in four aspects: First, it innovatively proposes a general proxy model construction method, which simulates the probability distribution characteristics of the target language model through an open-source large language model, establishes a unified feature evaluation space, and effectively solves the problem that traditional black-box methods are difficult to obtain deep probabilistic features; Second, it designs an error-sensitive word selector, which automatically identifies the most discriminative key positions in the text based on information entropy and prediction error analysis, significantly improving the pertinence and efficiency of feature extraction; Third, it develops a black-box probabilistic feature extraction method, which constructs a pseudo-probability distribution by resampling key words in multiple rounds, cleverly capturing the probabilistic features of AI-generated text in a black-box environment; Finally, it constructs a multi-feature fusion classification system, which integrates contextual features and probabilistic features through a cross-attention mechanism to achieve high-precision recognition of AI-generated content, and supports flexible switching between binary classification (distinguishing between AI generation and human creation) and multi-classification (identifying the specific source of the generation model) tasks according to actual needs.

[0156] This invention successfully bridges the technical gap between black-box and white-box methods, maintaining the ease of use and wide applicability of black-box methods while utilizing the probabilistic sensitivity of white-box methods, providing an efficient, accurate and universal solution for AI-generated text detection.

[0157] This embodiment uses AI-generated content review on a media platform as an example to illustrate the implementation process of the present invention in detail.

[0158] Step 1: Data Collection and Preprocessing

[0159] A training dataset was collected, comprising 1000 human-written news articles and 1000 news articles generated by various large language models, including Deepseek, Wenxin Yiyan, GPT, and Claude, covering multiple fields such as politics, economics, technology, and sports. All texts underwent standardization processing, including HTML tag removal, formatting, and sentence segmentation. The dataset was then divided into training, validation, and test sets in an 8:1:1 ratio.

[0160] Step 2: Building a General Proxy Model

[0161] Select Qwen-7B as the proxy model. This model is of medium scale (7 billion parameters) and performs well in predicting the probability distribution of the next word.

[0162] For the test text "The investigation shows that there are major improper behaviors among the company's senior executives, triggering immediate actions by the regulatory authorities", the proxy model calculates the conditional probability distribution for each word position. The probability distributions at key positions in terms of words are as follows: the conditional probability of the word "调 (diào)" is 0.42, the conditional probability of the word "高 (gāo)" is 0.36, the conditional probability of the word "重 (zhòng)" is 0.35, the conditional probability of the word "大 (dà)" is 0.63, the conditional probability of the word "不 (bù)" is 0.22, the conditional probability of the word "当 (dāng)" is 0.47, the conditional probability of the word "监 (jiān)" is 0.18, the conditional probability of the word "管 (guǎn)" is 0.74, and the conditional probability of the word "采 (cǎi)" is 0.28.

[0163] Step 3: Implementation of the error-sensitive vocabulary selector

[0164] (1) Information entropy calculation: Based on the probability distribution output by the proxy model, calculate the information entropy for each word position: the entropy value of the word "调 (diào)" is 3.25, the entropy value of the word "高 (gāo)" is 2.98, the entropy value of the word "重 (zhòng)" is 3.12, the entropy value of the word "大 (dà)" is 2.45, the entropy value of the word "不 (bù)" is 3.85, the entropy value of the word "当 (dāng)" is 3.12, the entropy value of the word "监 (jiān)" is 4.02, the entropy value of the word "管 (guǎn)" is 2.78, and the entropy value of the word "采 (cǎi)" is 3.46. The higher the entropy value of a word position, the greater the uncertainty at that position and the higher its potential discriminative value.

[0165] (2) Information entropy analysis settings: Calculate the local entropy ratio for each word position, set the window width to 7, and the entropy threshold to 1.25. The local entropy ratio of the word "不 (bù)" is 1.32, exceeding the threshold, and is marked as a high-entropy point; the local entropy ratio of the word "监 (jiān)" is 1.38, exceeding the threshold, and is marked as a high-entropy point; the local entropy ratio of the word "调 (diào)" is 1.18, not exceeding the threshold, and is not marked as a high-entropy point.

[0166] (3) Prediction error sensitivity analysis: Calculate the prediction confidence score for each word position, and set β = 1.6 to adjust the dynamic threshold. According to the conditional probabilities in Step 2: the prediction confidence score of the word "高 (gāo)" is 1.76, higher than the error threshold of the text 1.72, and is marked as a high-error point; the confidence score of the word "不 (bù)" is 2.14, higher than the error threshold of the text 1.72, and is marked as a high-error point; the confidence score of the word "监 (jiān)" is 2.27, higher than the error threshold of the text 1.72, and is marked as a high-error point; the confidence score of the word "管 (guǎn)" is 1.92, higher than the error threshold of the text 1.72, and is marked as a high-error point.

[0167] (4)Linguistic importance scoring: Determine the weights according to the role of the character in the word. The weight of a noun character is set to 1.0, the weight of a verb character is 0.9, the weight of an adjective character is 0.8, and the weight of an adverb character is 0.85. For example: The weight of the character "tiao" (verb) is 0.9; the weight of the character "gao" (adjective) is 0.8; the weight of the character "guan" (noun) is 1.0; the weight of the character "cai" (verb) is 0.9. Calculate the semantic relevance of the character to the overall text theme. For example: The semantic relevance of the character "tiao" is 0.72; the semantic relevance of the character "gao" is 0.65; the semantic relevance of the character "guan" is 0.78; the semantic relevance of the character "bu" is 0.58; the semantic relevance of the character "jian" is 0.82; the semantic relevance of the character "cai" is 0.70. Set the balance parameter λ = 0.6 and calculate .

[0168] (5)Top-K character selection: The parameters in the comprehensive scoring function are set as = 0.4 (entropy weight), = 0.3 (error weight), = 0.3 (linguistic weight). For an 800-word text, the k value is set to 48 (about 6%). Set the minimum character spacing to 5 characters to ensure that the selected keywords are evenly distributed in the text.

[0169] (6)Actual selection results: For the example text, the system selects a total of 48 keyword positions. The following shows the top 7 keyword positions with the highest comprehensive scores:

[0170] "jian": Entropy value 4.02, local entropy ratio 1.38, confidence score 2.27, normalized confidence 1.32, noun (positional score 1.0), semantic relevance 0.82, linguistic score 0.93, comprehensive score 1.23;

[0171] "bu": Entropy value 3.85, local entropy ratio 1.32, confidence score 2.14, normalized confidence 1.24, adverb (positional score 0.85), semantic relevance 0.58, linguistic score 0.74, comprehensive score 1.12;

[0172] "guan": Entropy value 2.78, local entropy ratio 1.12, confidence score 1.92, normalized confidence 1.12, noun (positional score 1.0), semantic relevance 0.78, linguistic score 0.91, comprehensive score 1.06;

[0173] "cai": Entropy value 3.46, local entropy ratio 1.16, confidence score 1.69, normalized confidence 0.98, verb (positional score 0.9), semantic relevance 0.70, linguistic score 0.82, comprehensive score 1.00;

[0174] "Tiao": Entropy value 3.25, local entropy ratio 1.18, confidence score 1.58, normalized confidence 0.92, verbality (positivity score 0.9), semantic relatedness 0.72, linguistic score 0.83, comprehensive score 1.00;

[0175] "Dang": Entropy value 3.12, local entropy ratio 1.02, confidence score 1.85, normalized confidence 1.08, verbality (positivity score 0.9), semantic relatedness 0.68, linguistic score 0.81, comprehensive score 0.97;

[0176] "Gao": Entropy value 2.98, local entropy ratio 1.06, confidence score 1.76, normalized confidence 1.02, adjectivity (positivity score 0.8), semantic relatedness 0.65, linguistic score 0.74, comprehensive score 0.95.

[0177] The positions of the above 7 example characters are evenly distributed in the text, satisfying the constraint condition of a minimum spacing of 5 characters, and will be used as the key positions for subsequent probability distribution feature extraction.

[0178] Step 4: Implementation of the probability feature extractor

[0179] (1) Black-box sampling settings: Each keyword position is sampled 50 times, and the temperature parameter is set to 1.0. Taking the character "Bu" as an example for detailed explanation, a template is constructed: "The investigation shows that there are major [MASK] Dang behaviors among the company's senior executives, triggering immediate action by the regulatory authorities."

[0180] (2) Sampling statistical results: The 50 sampling results for the position of the character "Bu" are as follows: "Bu": 31 times (frequency 0.62); "Wei": 8 times (frequency 0.16); "Shi": 5 times (frequency 0.10); "Yan": 3 times (frequency 0.06); "Cuo": 2 times (frequency 0.04); Other characters: 1 time (frequency 0.02).

[0181] (3) Pseudo-probability construction: Apply Laplace smoothing to the sampling results, with the smoothing parameter α = 0.15, and the observed number of candidate word types . For example, the smoothed probability of the character "Bu" is (31 + 0.15) / (50 + 0.15×6) = 0.615.

[0182] (4) Feature vector construction: Based on the pseudo-probability distribution of the sampling results, calculate a series of statistical features, including entropy value, KL divergence, Gini coefficient, Top1 probability, Top3 cumulative probability, and entropy ratio, etc. Calculate the feature vectors for all selected keyword positions, and then connect them to form a text-level probability feature vector. For the 7 keyword positions in the example, a (6 + 6)×7 = 84-dimensional probability feature vector is finally formed, where 6 dimensions are basic frequency features, and the other 6 dimensions are probability distribution features.

[0183] Step 5: Implementation of the multi-feature fusion classifier

[0184] (1) Feature extraction and fusion: The BERT-base-chinese model is used to extract text context features, outputting a 768-dimensional vector. The probabilistic feature vector is compressed to 64 dimensions using a two-layer MLP (128 hidden layer dimensions). The context features and probabilistic features are fused using a cross-attention mechanism (4 attention heads), outputting a 256-dimensional fused feature vector.

[0185] (2) Classification model training: The binary classification model uses two fully connected layers (256 hidden layers and 1 output layer); the multi-class classification model uses four fully connected layers (the dimensions of the hidden layers are 256, 128, and 64 respectively, and the output layer is 5); the training parameters are set with a batch size of 32, a learning rate of 2e-5, and 12 epochs of training, using the AdamW optimizer; the loss function is cross-entropy loss for binary classification and weighted cross-entropy loss for multi-class classification, with the weighting coefficients being human:Deepseek:Wenxinyiyan:GPT:Claude=1:1.2:1.2:1.2:1.2.

[0186] (3) Test set validation: Evaluation was conducted on 200 test articles: the binary classification accuracy reached 89.5%, and the multi-classification accuracy reached 80.6% (classified into five categories: human, deepseek, Wenxin Yiyan, GPT, and Claude). Compared with traditional statistical feature-based methods, the accuracy was improved by 10.3%.

[0187] The method of this invention demonstrates significant advantages in content moderation scenarios on media platforms:

[0188] (1) High accuracy: It achieves an accuracy of 89.5% in detecting multi-source AI-generated content in Chinese environment, and its ability to identify high-quality content generated by various Chinese large models is superior to existing methods.

[0189] (2) General adaptability: It does not require separate training for each model and has good detection capabilities for newly emerging AI model-generated content.

[0190] This invention is applicable to various Chinese text content detection scenarios. Through character-level probabilistic feature analysis, it can more accurately capture the features of AI-generated text, providing strong technical support for content review, academic integrity, and information authenticity verification.

[0191] Example 2

[0192] This invention also discloses an AI-generated text detection system guided by a proxy model, which applies the AI-generated text detection method guided by a proxy model as described in Example 1; the detection system includes:

[0193] The general proxy model module is used to use an open-source large language model as a proxy model. The text to be detected is input into the proxy model, and the probability distribution information of the word at each position in the text under the context conditions is calculated.

[0194] A lexical selector is used to calculate a comprehensive score based on probability distribution information and a multidimensional evaluation framework to characterize the discrimination value of each lexical in the text to be detected, and to select a set of key lexicals according to the priority of the comprehensive score from high to low; wherein, the comprehensive score is a weighted sum based on the local entropy ratio index, prediction confidence index and linguistic characteristic index of each position in the text to be detected.

[0195] The probabilistic feature extractor is used to construct a context-preserving template for each word in the keyword set, and to perform black-box resampling on the template using the target language model. Based on the sampling results, the frequency of candidate words is counted and a pseudo-probability distribution is constructed, thereby generating a probabilistic feature vector.

[0196] A multi-feature fusion classifier is used to extract the contextual semantic feature vector of the text to be detected, fuse it with the probability feature vector, and output the detection result of whether the text to be detected was generated by the target language model or created by a human through a classification network.

[0197] Example 3

[0198] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of AI-generated text detection based on a proxy model as described in Embodiment 1.

[0199] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., provided on the computer device. Of course, the storage medium may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0200] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting AI-generated text based on proxy model guidance, characterized in that, The method comprises the steps of: S1. using an open-source large language model as a proxy model, inputting the text to be detected into the proxy model, and calculating the probability distribution information of each position in the text under the context condition; S2. Based on the probability distribution information, the comprehensive score for representing the discriminant value of each word element in the text to be detected is calculated through a multi-dimensional evaluation framework, and the key word element set is screened out in order of priority from high to low according to the comprehensive score; wherein the comprehensive score is a weighted sum based on the local entropy ratio index, the prediction confidence index and the linguistic characteristic index of each position in the text to be detected; the local entropy ratio index is used to represent the uncertainty difference of the word element in the local context environment, and the linguistic characteristic index is used to represent the discriminant value of the word element in multiple linguistic dimensions for the text detection task; S3. Constructing a context maintaining template for each word element in the key word element set, using a target language model to perform black box resampling on the template, and according to the sampling result, the candidate word frequency is counted and the pseudo probability distribution is constructed, and then the probability feature vector is generated; S4. Extracting the context semantic feature vector of the text to be detected, fusing it with the probability feature vector, and outputting the detection result of the text to be detected by the target language model or human creation through a classification network; Step S3 specifically comprises: For each key position in the key word element set, the context of the rest of the text is kept unchanged, the key position is replaced with a mask mark, and a context maintaining template is constructed; The context maintaining template is input into the target language model, and a random sampling strategy with a temperature parameter is used for multiple rounds of repeated sampling to obtain a candidate word set output by the key position in different sampling rounds; The frequency of each candidate word in the candidate word set is calculated, and a weighted smoothing processing algorithm is applied to the frequency statistical result to avoid zero probability problem and improve the stability of the pseudo probability distribution, so as to construct the pseudo probability distribution reflecting the vocabulary selection tendency of the target language model in the black box state; Based on the pseudo probability distribution, the basic frequency feature and the probability distribution feature are extracted respectively; wherein the basic frequency feature is composed of the pseudo probability distribution of a plurality of candidate words with the highest frequency; and the probability distribution feature is composed of statistical indicators for quantifying distribution characteristics; The basic frequency features and probability distribution features corresponding to all word elements in the key word element set are spliced in order to form an original feature vector; then the original feature vector is subjected to feature standardization processing to obtain a text-level probability feature vector.

2. The proxy model guided AI generated text detection method of claim 1, wherein, In step S2, the calculation methods of the local entropy ratio index, the prediction confidence index and the linguistic characteristic index are as follows: The information entropy of the current position is calculated using the probability distribution information output by the proxy model, a local context window centered on the current position is set, the average value of the information entropy of all positions in the window is calculated, and the ratio of the information entropy of the current position to the average value is taken as the local entropy ratio of the current position; The prediction confidence score of the actual appearing word element of the current position is calculated; The average prediction confidence score and the standard deviation of the whole text are counted, and an adaptive error threshold is constructed based on the average prediction confidence score and the standard deviation; The prediction confidence score of the current position is normalized by using the adaptive error threshold to obtain a prediction confidence index; The first importance score based on the part-of-speech of each position and the second importance score based on semantics are calculated respectively, and the first importance score and the second importance score are fused by weighting to obtain a comprehensive linguistic feature index; wherein the first importance score is obtained by assigning a preset weight to the part-of-speech label to which the word belongs; and the second importance score is obtained by calculating the similarity between the semantic embedding vector of the word and the global semantic embedding vector of the whole text.

3. The proxy model guided AI generated text detection method of claim 2, wherein, In step S2, during the screening of the key word set, position constraint processing and stability verification processing are also performed; The position constraint processing includes ensuring that the distance between any two selected candidate words in the text is not less than a preset minimum distance value, and if the candidate words do not meet the distance requirement, the next priority word is selected. The stability verification processing includes performing multiple random resampling evaluations on the text segment where the candidate word is located, counting the average score and the score change standard deviation of the candidate word under different context conditions, and calculating a stability index based on the average score and the score change standard deviation, which reflects the ability of the word to maintain high discriminative value under different context conditions. The stability index and the comprehensive score are weighted to obtain a final score, and the top K highest final scores are selected to form the key word set.

4. The proxy model guided AI generated text detection method of claim 3, wherein, The value of K is set to 5% to 10% of the total length of the text to be detected, and K has a lower limit and an upper limit. When performing position constraint processing, if the candidate word cannot meet the minimum distance requirement, the word with the highest score is discarded, and the word with the second highest score that meets the position distribution requirement is selected to join the key word set.

5. The proxy model guided AI generated text detection method of claim 1, wherein, The statistical indicators include the information entropy of the pseudo probability distribution, the KL divergence between the pseudo probability distribution and the uniform distribution, the Gini coefficient, the frequency of the highest frequency candidate word, the ratio of the sum of the frequencies of the top three high frequency candidate words to the total sampling number, and the ratio of the entropy value of the pseudo probability distribution to the maximum possible entropy value.

6. The proxy model guided AI generated text detection method of claim 1, wherein, In step S4, a pre-trained language model is used as a feature extractor, and the text to be detected is input into the feature extractor to output a context semantic feature vector; The probability feature vector is reduced and normalized by using a multi-layer perception machine to obtain a processed probability feature vector; The context semantic feature vector and the processed probability feature vector are fused by using a cross-attention mechanism to output a fusion feature vector for final classification.

7. The proxy model guided AI generated text detection method of claim 1, wherein, In step S4, the classification network supports a binary classification task or a multi-classification task. The binary classification task is used to output the probability that the text to be detected is generated by a single target language model, and the multi-classification task is used to output the probability distribution that the text to be detected is generated by multiple target language models. Cross-entropy loss function is used for optimization in the training process.

8. An AI-generated text detection system based on proxy model guidance, characterized in that, The application discloses an AI-generated text detection method based on agent model guidance, and belongs to the technical field of text detection. The general agent model module is used for inputting the to-be-detected text into an agent model by using an open-source large language model as the agent model, and calculating probability distribution information of each position of the text under a context condition. The word element selector is used for calculating a comprehensive score for representing a discrimination value of each word element in the to-be-detected text based on the probability distribution information through a multi-dimensional evaluation framework, and screening a key word element set in a priority order from high to low according to the comprehensive score. The probability feature extractor is used for constructing a context maintaining template of each word element in the key word element set, performing black-box resampling on the template by using a target language model, statistically analyzing candidate word frequencies according to a sampling result, and constructing a pseudo probability distribution, and then generating a probability feature vector. The multi-feature fusion classifier is used for extracting a context semantic feature vector of the to-be-detected text, fusing the context semantic feature vector with the probability feature vector, and outputting a detection result of whether the to-be-detected text is generated by the target language model or created by a human being through a classification network.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps of the AI-generated text detection method based on agent model guidance according to any one of claims 1 to 7.

Images