An ai-generated text detection method based on graph structure features

Abstract

This invention presents an AI-generated text detection method based on graph structure features, belonging to the fields of artificial intelligence and natural language processing. The method includes: dataset construction, entity relation extraction and graph structure construction, graph structure feature extraction, graph structure feature model training, and text detection. It further incorporates traditional text feature extraction and model training, adaptively fusing the traditional text feature model and the graph feature model based on confidence-weighted entropy, and then performing text detection based on the fused model. This invention is the first to perform AI text detection from the perspective of graph structure features, breaking through the limitations of existing research that focuses on surface features such as vocabulary, syntax, and perplexity. The fusion strategy dynamically adjusts the fusion weights by quantifying the uncertainty of model predictions, maintaining a high level of performance on both original data and adversarial examples, achieving a balance between detection accuracy and adversarial robustness. It can be widely applied to the detection of AI-generated content such as news content and academic papers.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and relates to text content detection and recognition technology, especially to an AI-generated text detection method based on graph structure features. Background Technology

[0002] With the significant improvement in the quality of natural language generation by large language models, generative models such as DeepSeek, ChatGPT, and Doubao have been widely used in news reporting, content creation, and academic assistance. While the rapid development of AI text generation has facilitated information dissemination and content production, it has also brought potential risks such as the spread of misinformation, academic misconduct, and commercial fraud, seriously threatening the authenticity of information and social trust.

[0003] To address the aforementioned issues, several studies and inventions have been developed for detecting AI-generated text. Guo Xin et al., taking the introduction of an information science journal as an example, first analyzed the text structure features at the word, sentence, and discourse levels. They then integrated these features with text vector features to perform detection using various classification models, and conducted interpretability analysis on models with good classification performance. (Guo Xin, Wang Yibo, Wang Jimin; Analysis of ChatGPT-Generated Chinese Academic Content—Taking Information Science as an Example [J]. Library Forum, 2024, 44(03):134-143.). Wang Shiqi et al., targeting Chinese papers in the humanities and social sciences, extracted text structure features and text vector features. After model classification training and fusion, they achieved detection of the paper-generating module of a large language model (Peking University, Chongqing Fanyu Technology Co., Ltd., A Detection Method for the Chinese Paper-Generating Module of a Large Language Model: 202311363657.X [P]. 2025-10-10.).

[0004] However, the aforementioned studies suffer from an over-reliance on surface text features during detection. Their performance deteriorates significantly when faced with adversarial operations such as text polishing and rewriting, resulting in insufficient robustness. Currently, various techniques have been developed to intentionally circumvent AI text detection, including text transformation methods (paraphrasing, word substitution, deliberate grammatical errors) and adversarial techniques utilizing the detection mechanism itself. Existing classifiers lack robustness against adversarially generated text that has been rewritten and paraphrased, easily leading to missed detections, with performance drops exceeding 30%. More importantly, scholars have explicitly pointed out that in real-world scenarios, the proportion of human-machine hybrid text (such as human-written drafts + AI polishing) far exceeds that of purely AI-generated text. However, existing detectors lack robustness and will be unable to address issues such as academic integrity and content traceability arising from human-machine collaborative creation, highlighting the necessity of robustness for the practical application of detectors.

[0005] Therefore, existing technologies do not pay enough attention to the logical structure of text content and fail to discriminate from deep semantic dimensions such as entity relationships and knowledge organization, resulting in severe performance degradation of detection methods in real adversarial environments. Developing robust and interpretable AI-generated text detection technologies has become an important frontier direction in information governance and trustworthy AI research. Summary of the Invention

[0006] To overcome the shortcomings of the existing technology, this invention provides an AI-generated text detection method based on graph structure features, which solves the technical problems of existing detection methods relying too much on surface features and lacking robustness against attacks.

[0007] The principle of this invention is to map text into a graph structure representation using entity relation extraction technology, capturing the relationship network between entities in the text and its topological organization. AI-generated text tends to construct a complete and coherent knowledge network in its generation mechanism, striving to cover all entities and relationships related to the topic, resulting in a graph structure with high density and high centralization. In contrast, human-written text is more selective, focusing on core information, and exhibits moderately sparse and time-driven characteristics. This difference in deep logical patterns is difficult to completely eliminate through surface text polishing, thus providing a theoretical basis for robust detection. This invention further adopts an adaptive fusion strategy based on confidence-weighted entropy, combining graph structure features with traditional text features, significantly enhancing robustness while maintaining high detection accuracy.

[0008] The technical solution provided by this invention is:

[0009] An AI-generated text detection method based on graph structure features includes the following steps:

[0010] 1) Dataset construction: Collect manually written text sample data, use multiple large language models to generate AI text samples corresponding to the manually written text samples, and build a training dataset;

[0011] 2) Entity Relationship Extraction and Graph Structure Construction: Based on the semantic characteristics of the domain to which the text sample data belongs, a graph pattern is defined to clarify the entity type and relation type; a large language model is used to extract entity relations, and the text is transformed into structured entity relation triples according to the specified graph pattern, including entity identifier, entity type, entity name, relation type, and confidence information; a directed graph is constructed based on the extracted entity relation triples, with entities as nodes and relations as edges, and each edge has relation type and confidence attributes;

[0012] 3) Graph Structure Feature Extraction: Multi-dimensional feature extraction is performed on the graph structure constructed in step 2). The multi-dimensional features of the graph structure include graph topological statistical features, node centrality features, entity relationship distribution features, and graph embedding features. Among them, graph topological statistical features reflect the organization and connection pattern of the entity relationship network in the text; node centrality features measure the importance and structural position of different entities in the graph; entity relationship distribution features statistically analyze the quantity distribution and confidence of different types of entities and relationships; graph embedding features are obtained by extracting the vector representation of the graph, obtaining the average value of all node embeddings as the graph embedding, and obtaining entity embeddings and relationship embeddings respectively by semantically encoding the entity text and relationship descriptions.

[0013] 4) Training the graph structure feature model: Select multiple machine learning algorithms to train a classification model based on the graph structure features extracted in step 3), and select the optimal graph structure feature model through cross-validation and hyperparameter optimization;

[0014] 5) Detection of text to be detected: For the text to be detected, entity relations are extracted and graph structure is constructed according to step 2), graph structure features are extracted according to step 3), the extracted features are input into the graph structure feature model trained in step 4), the detection results are output, and it is determined whether the text is generated by AI.

[0015] On the other hand, this invention proposes an AI-generated text detection method based on an adaptive fusion model of confidence-weighted entropy. Building upon the aforementioned AI-generated text detection method based on graph structure features, it introduces traditional text feature extraction and model training, and adaptively fuses the traditional text feature model with the graph feature model based on confidence-weighted entropy. AI-generated text detection is then performed based on this adaptive fusion model, specifically as follows:

[0016] The traditional text feature extraction and model training are as follows: traditional text feature extraction is performed on the text, which includes text statistical features and Term Frequency-Inverse Document Frequency (TF-IDF) lexical features; multiple machine learning algorithms are selected to train a classification model based on traditional text features, and the optimal model is selected through cross-validation and hyperparameter optimization;

[0017] The adaptive fusion strategy based on confidence-weighted entropy is as follows: Based on the uncertainty of the model prediction quantified by information entropy, the fusion weights of the graph feature model and the text feature model are dynamically adjusted. Specifically, this involves: obtaining the prediction probability distributions of the graph feature model and the text feature model for the test samples; calculating the prediction entropy of each model; and then calculating the average entropy; dynamically adjusting the weights of the graph feature model according to the relationship between the average entropy and the entropy threshold, relying more on the text feature model when the average entropy is low, and relying more on the graph feature model when the average entropy is high; obtaining the final prediction probability through weighted fusion; if the prediction probability is greater than the classification threshold, it is determined to be AI-generated text.

[0018] The adaptive fusion model is optimized by hyperparameters: the hyperparameters of the fusion strategy, including entropy threshold, basic weight, maximum weight, and classification threshold, are tuned by grid search, and the optimal model is selected by cross-validation and hyperparameter optimization.

[0019] For step 5) Detection of the text to be detected: For the text to be detected, entity relations are extracted and graph structure is constructed according to step 2), graph structure features are extracted according to step 3), and traditional text features are extracted at the same time. The extracted graph structure features and traditional text features are input into the adaptive fusion model, and the detection result is output to determine whether the text is generated by AI.

[0020] Furthermore, the large language models used in step 1) include DeepSeek, Wenxin Yiyan, Zhipu GLM, and Tongyi Qianwen; the AI ​​text sample generation adopts a continuation-style generation strategy, retaining the original title and beginning as context, and requiring the model to generate subsequent content.

[0021] Furthermore, in step 3), different types of entities are counted, including people (PERSON), organizations (ORG), locations (LOC), events (EVENT), and times (TIME); different types of relationships include participation (participate_in), located (located_at), happened (happen_at), cause (cause), related (related_to), and belong (belong_to).

[0022] Furthermore, the graph topological statistical features in step 3) include the number of nodes, the number of edges, graph density, the number of connected components, connectivity, clustering coefficient, average shortest path, graph diameter, edge-to-node ratio, number of two-hop paths, number of three-hop paths, number of triangles, and number of linear chains; the node centrality features include average degree centrality, maximum degree centrality, average degree, maximum degree, degree standard deviation, height-to-node ratio, degree skewness, average betweenness centrality, maximum betweenness centrality, and average proximity centrality; the entity relation distribution features include the number distribution of five types of entities, the number distribution of six types of relations, and the mean and standard deviation of relation confidence; the graph embedding features are extracted using the Node2Vec algorithm and the BGE-M3 model.

[0023] Furthermore, the machine learning algorithms selected in step 4) include support vector machine, random forest, gradient boosting tree, XGBoost, LightGBM, logistic regression, Naive Bayes, decision tree, and K-nearest neighbors; K-fold cross-validation is used for model evaluation, and grid search is used for hyperparameter optimization, with F1 score as the evaluation metric.

[0024] Furthermore, the text statistical features include text size features, lexical diversity features, punctuation features, and complexity indicators; the TF-IDF lexical features are generated through Chinese word segmentation, stop word filtering, and a TF-IDF vectorizer.

[0025] Compared with the prior art, the beneficial effects of the present invention are:

[0026] 1. This invention is the first to perform AI text detection from the perspective of graph structure features, breaking through the limitations of existing research that focuses on surface features such as vocabulary, syntax, and perplexity. Its performance on adversarial examples is significantly better than that of traditional text feature models, proving that the detection method based on deep logic features has strong anti-perturbation ability and significantly improves the adversarial robustness of the detection model.

[0027] 2. The adaptive fusion strategy based on confidence-weighted entropy proposed in this invention dynamically adjusts the fusion weights by quantifying the uncertainty of the model prediction, maintaining a high level on both the original data and adversarial examples, and achieving a balance between detection accuracy and adversarial robustness.

[0028] 3. This invention is simple to operate and can be widely used for AI-generated content detection in various text types such as news content, academic papers, and social media, and has high application value. Attached Figure Description

[0029] Figure 1 This is a flowchart of the AI-generated text detection method based on an adaptive fusion model with confidence weighted entropy in an embodiment of the present invention.

[0030] Figure 2Example graphs of prompt words constructed for the training dataset;

[0031] Figure 3 Example diagram of prompt words constructed for the anti-robustness test dataset;

[0032] Figure 4 Example diagram of prompt words constructed for entity relation extraction. Detailed Implementation

[0033] To make the objectives, technical solutions, and technical effects of the present invention clearer, the specific implementation process of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. Obviously, what is described here is only an exemplary embodiment of the present invention, and not all embodiments, nor is it intended to limit the present invention.

[0034] This embodiment uses news text as the object to describe in detail the specific implementation process of the AI-generated text detection method based on the confidence-weighted entropy adaptive fusion model of the present invention. The flowchart is as follows: Figure 1 As shown.

[0035] Step 1: Dataset Construction

[0036] 1.1 Training Dataset Construction. Historical news articles from January 2015 to January 2017 were collected from mainstream Chinese news websites. From these, 1000 high-quality, manually written news reports were selected, covering multiple topics including sports, education, science, society, and finance. The news texts exhibit thematic diversity and rich entity relationships, facilitating the construction of a graph structure representation.

[0037] 1.2 AI Sample Generation. Four mainstream domestic large language models were used to generate corresponding AI news samples, specifically DeepSeek-V3, Wenxin Yiyan 4.5, Zhipu GLM-4, and Tongyi Qianwen 3. A continuation-style generation strategy was adopted: the title and opening of the original news were retained as context, and the model was required to generate subsequent content based on this information. The target length of the generated content was set to be similar to the original text (randomly fluctuating within 90%-110% of the original text length) to maintain consistency in text length. This resulted in a training set of 1000 AI-generated news articles. The training dataset included prompts such as... Figure 2 As shown.

[0038] Step 2: Entity Relationship Extraction and Graph Structure Construction

[0039] 2.1 Graph Pattern Definition. Based on the semantic characteristics of the news domain, the graph pattern is defined as follows:

[0040] Entity types include: PERSON (person, including name and job title), ORG (organization, including government, company, group), LOC (location, including country, city, region), EVENT (event, including meeting, accident, event), and TIME (time, including date and time period).

[0041] Relationship types include: participate_in (participation relationship, a person participates in an event), located_at (location relationship, an event or organization is located in a certain place), happened_at (time relationship, the time when the event occurred), cause (cause relationship, one event causes another event), related_to (related relationship, general association), and belong_to (belong to relationship, a person belongs to an organization).

[0042] The graph pattern design follows the principle of minimum sufficiency, which ensures that the semantics of the news are fully expressed while avoiding extraction difficulties caused by excessive refinement.

[0043] 2.2 Entity Relation Extraction. Entity relations are extracted using a large language model, through the design of prompt words, such as... Figure 4 As shown, the model is required to transform text into structured entity-relation triples according to a specified graph pattern, including entity identifier, entity type, entity name, relation type, confidence level, and other information, and output the entity list and relation list in JSON format.

[0044] 2.3 Graph Structure Construction. Based on the extracted entity relation triples, a directed graph G=(V,E) is constructed, where V is the set of nodes (each node represents an entity), E is the set of edges (each edge represents a relation between two entities), and each edge has relation type and confidence attributes.

[0045] Step 3: Graph Structure Feature Extraction and Model Training

[0046] 3.1 Graph Topological Statistical Feature Extraction. Graph topological statistical features are defined based on the overall structural information of the graph. These features reflect the organization and connection patterns of the entity relationship network in the text. For the constructed directed graph G, the following topological statistical features are calculated:

[0047] (1) Basic topological features

[0048] ① Number of nodes: The total number of entity nodes in the graph. This feature reflects the richness of entities involved in the text.

[0049] ② Number of edges: The total number of relational edges in the graph. This feature reflects the richness of the relationships between entities in the text.

[0050] ③ Graph density: The ratio of the actual number of edges to the maximum possible number of edges, ranging from 0 to 1. This feature measures the density of the graph. AI-generated text usually has a higher graph density, reflecting that AI models tend to build denser entity relationship networks.

[0051] ④ Number of connected components: The number of weakly connected subgraphs. This feature reflects the degree of fragmentation of the graph. The more connected components there are, the more independent topics or events there are in the text. AI text usually has fewer connected components and shows stronger overall coherence.

[0052] ⑤ Connectivity: Whether the graph is weakly connected, with a value of 0 or 1. This feature determines whether all entities are directly or indirectly connected through relations.

[0053] (2) Advanced topological features

[0054] ① Clustering coefficient: The average degree of tightness among the connections between nodes' neighbors, ranging from 0 to 1. This feature measures the local clustering properties of the graph.

[0055] ② Average Shortest Path: The average length of the shortest path between all pairs of nodes, calculated only for connected graphs. This feature reflects the reachability of the graph. AI text typically has a shorter average shortest path, indicating that the logical distance between entities is closer and information propagation is more direct.

[0056] ③ Graph diameter: The length of the longest and shortest path in the graph. This feature reflects the maximum span of the graph.

[0057] ④ Edge-to-node ratio: The ratio of the number of edges to the number of nodes. This feature comprehensively reflects the connection density of the graph.

[0058] (3) Path and structural features

[0059] ① Number of two-hop paths: The total number of paths with a length of 2, that is, the number of entity pairs connected through an intermediate node. This feature reflects the richness of indirect associations.

[0060] ② Number of three-hop paths: The total number of paths with a length of 3, i.e. the number of entity pairs connected by two intermediate nodes. This feature further reflects remote associations.

[0061] ③ Number of triangles: The number of closed loops between three nodes in the diagram. This feature reflects the ternary relationship between entities. The more triangles there are, the more cyclical relationships exist between entities.

[0062] ④ Number of linear chains: The number of endpoint node pairs with a degree of 1, i.e., isolated entity pairs with only one relation. This feature reflects the edge structure of the graph. Human-generated texts often have more linear chains, exhibiting a more flexible narrative style.

[0063] 3.2 Node Centrality Feature Extraction. Node centrality features measure the importance and structural position of different nodes (entities) in the graph; these features reflect the differences in status among different entities in the text. Calculation of various centrality features:

[0064] (1) Degree centrality feature

[0065] ① Average degree centrality: The average degree centrality of all nodes. Degree centrality reflects the number of direct connections between nodes, and this feature measures the average degree of connection between entities in the text.

[0066] ② Maximum degree centrality: The maximum value of degree centrality. This feature reflects the degree of connectivity of the most core entities in the graph. AI text often has highly centralized core entities, so this value is higher.

[0067] ③ Average Degree: The average degree of all nodes. Degree is the number of edges a node has, and this feature directly measures the average number of relations an entity has.

[0068] ④ Maximum degree: The maximum value of the degree. This feature reflects the number of relationships of the core entity.

[0069] ⑤ Degree Standard Deviation: The degree of dispersion in the degree distribution. This feature reflects the degree of differentiation in the importance of entities.

[0070] ⑥ Height node ratio: The proportion of nodes with a degree ≥ 75th percentile. This feature reflects the proportion of highly connected entities.

[0071] ⑦ Degree skewness: The degree of skewness in the degree distribution. This feature reflects the symmetry of the degree distribution.

[0072] (2) Other centrality features

[0073] ① Average betweenness centrality: The average betweenness centrality of all nodes. Betweenness centrality measures the bridging role of a node in a graph, reflecting the pivotal position of an entity in information transmission.

[0074] ② Maximum betweenness centrality: The maximum value of betweenness centrality. This feature reflects the most important bridging nodes. AI text often contains core entities with extremely high betweenness centrality, and this value is significantly higher than that of human text.

[0075] ③ Average proximity centrality: The average proximity centrality of all nodes. Proximity centrality reflects the average distance from a node to other nodes; a higher value indicates that the node is closer to the center of the graph.

[0076] 3.3 Entity Relationship Distribution Feature Extraction. The distribution of the number of five types of entities (people, organizations, locations, events, and time) and six types of relationships (participation, location, time, causality, correlation, and attribution) is statistically analyzed. Simultaneously, the mean and standard deviation of relationship confidence scores are calculated to reflect the semantic composition characteristics of the text. AI text typically shows a higher number of location entities and causal relationships, while human-generated text places greater emphasis on temporal relationships.

[0077] 3.4 Graph Embedding Feature Extraction. Graph embedding features were extracted using the Node2Vec algorithm and the BGE-M3 model. Node2Vec learned node vector representations through random walks and obtained the average value of all node embeddings as the graph embedding; the BGE-M3 model semantically encoded entity text and relation descriptions to obtain entity embeddings and relation embeddings, respectively.

[0078] 3.5 Training the Graph Structure Feature Model. A classification model was trained using various machine learning algorithms based on the extracted graph structure features. These algorithms included Support Vector Machine, Random Forest, Gradient Boosting Tree, XGBoost, LightGBM, Logistic Regression, Naive Bayes, Decision Tree, and K-Nearest Neighbors. The graph structure features were standardized, and K-fold cross-validation was used for model evaluation. Hyperparameter optimization was then performed to select the optimal graph structure feature model.

[0079] Step 4: Traditional Text Feature Extraction and Model Training

[0080] 4.1 Text Statistical Feature Extraction. Text statistical features are defined based on the surface statistical information of the text. These features reflect the basic structure and writing style of the text. The following statistical features are extracted:

[0081] (1) Text size characteristics

[0082] ① Text length: Total number of characters in the text. This feature reflects the overall length of the text.

[0083] ② Number of sentences: The total number of sentences in the text, separated by periods, question marks, and exclamation marks. This feature reflects the sentence organization of the text.

[0084] ③ Average sentence length: Text length divided by the number of sentences. This feature reflects the average complexity of sentences.

[0085] ④ Number of words: The total number of words after word segmentation. This feature reflects the vocabulary size of the text and is obtained after Chinese word segmentation.

[0086] ⑤ Average word length: Total number of characters in all words divided by the total number of words. This feature reflects the complexity of the vocabulary.

[0087] (2) Lexical diversity characteristics

[0088] ① Number of unique words: The number of words that are not repeated. This feature reflects the richness of the vocabulary; artificial texts typically have higher lexical diversity.

[0089] ② Lexical diversity: The ratio of unique words to the total number of words, ranging from 0 to 1. This feature measures the degree of word repetition. AI text typically has slightly lower lexical diversity, tending to reuse similar word expressions.

[0090] (3) Features of punctuation marks

[0091] The data includes the quantity and proportion of various punctuation marks, such as the total number of punctuation marks, the proportion of punctuation marks, the number of commas, periods, question marks, exclamation marks, and quotation marks. These features reflect the punctuation usage patterns in the text, and AI-generated text typically exhibits more standardized and stable punctuation usage.

[0092] (4) Complexity index

[0093] ① Type tag ratio: The ratio of unique words to the total number of words, measuring lexical richness. This feature is similar to lexical diversity, reflecting word repetition from another perspective.

[0094] ②Word length standard deviation: The degree of dispersion in the distribution of word lengths. This feature reflects the range of variation in word length. The word length standard deviation of artificial texts is usually larger, showing richer variations in word length.

[0095] 4.2 TF-IDF Lexical Feature Extraction. Term Frequency–Inverse Document Frequency (TF-IDF) is a classic text representation method that reflects the importance of words in a text. The text is segmented into Chinese using the jieba word segmentation tool, which accurately identifies Chinese word boundaries. A TF-IDF vectorizer is used to generate 100-dimensional lexical weight vectors, where each dimension corresponds to an important word, and the numerical value represents the TF-IDF weight of that word in the text. TF-IDF features effectively reflect the text's thematic tendency and word preference; the TF-IDF weights of certain words in the AI ​​text exhibit clear regularities.

[0096] Textual statistical features (approximately 30 dimensions) and TF-IDF features (100 dimensions) are combined to form a complete textual feature vector, totaling approximately 130 dimensions. These features characterize the compositional features of the text from a surface linguistic perspective and have strong discriminative power on raw data, but they are prone to failure when faced with adversarial operations such as text polishing.

[0097] 4.3 Training of Traditional Text Feature Models. A classification model was trained based on traditional text features using the same multiple machine learning algorithms. Text features were standardized, K-fold cross-validation was used for model evaluation, and hyperparameter optimization was employed to select the optimal graph structure feature model.

[0098] Step 5: Adaptive Fusion Based on Confidence-Weighted Entropy

[0099] 5.1 Fusion Model Design. Based on the concept of entropy in Shannon's information theory, the fusion weights are dynamically adjusted by quantifying the uncertainty of the model's predictions. For binary classification problems, the predicted entropy of the model's output probability distribution is used to measure uncertainty. The fusion strategy is as follows: for low-entropy samples, the performance advantages of the text feature model are fully utilized; for high-entropy samples, the adversarial robustness of the graph feature model is relied upon more.

[0100] 5.2 Fusion Model Implementation. Obtain the prediction probability distributions of the graph feature model and the text feature model for the test samples, calculate the prediction entropies H_graph and H_text of the two models respectively, and then calculate the average entropy H_avg; dynamically adjust the weight w_graph of the graph feature model based on the relationship between the average entropy and the entropy threshold τ_entropy. When the average entropy is low, more reliance is placed on the text feature model; when the average entropy is high, more reliance is placed on the graph feature model; obtain the final prediction probability through weighted fusion. If the prediction probability is greater than the classification threshold, it is determined to be AI-generated text.

[0101] 5.3 Hyperparameter Tuning. The hyperparameters of the fusion strategy were tuned on the validation set using grid search. These included the entropy threshold τ_entropy (range 0.5-0.7), base weights w_base (range 0.3-0.5), maximum weights w_max (range 0.6-0.75), and classification threshold τ_classification (range 0.4-0.55). K-fold cross-validation was used to evaluate the performance of different parameter combinations, and the optimal hyperparameter combination was used to construct an adaptive fusion model based on confidence-weighted entropy.

[0102] Step Six: Model Performance Validation

[0103] To verify the effectiveness of the AI-generated text detection method described in this invention, this embodiment conducts a comprehensive performance test on the trained model.

[0104] 6.1 Detection Process. For the text to be detected, entity relations are extracted and graph structure is constructed according to the methods described above. Graph structure features are extracted and input into the graph structure feature model. The output detection result determines whether the text is AI-generated. Further, traditional text features are extracted. At the same time, the graph features and traditional text features are input into the confidence-weighted entropy fusion model. The output result of the fusion model determines whether the text is human-written or AI-generated.

[0105] 6.2 Robustness Assessment. To comprehensively evaluate the effectiveness of the graph structure feature model and adaptive fusion model proposed in this invention in real-world application scenarios, an additional 800 news samples (400 human-written news items and 400 AI-generated news items) were collected and polished separately. For human-written news items, a professional polishing strategy was employed, with polishing instructions as follows: Figure 3 As shown, the requirements are to optimize paragraph transitions, improve logical coherence, replace repetitive words, enhance expression accuracy, correct inappropriate word choice, and improve readability, while maintaining the core facts to achieve a higher level of journalistic professionalism. For AI-generated news, a de-mechanical polishing strategy is adopted, requiring the removal of overly formal and templated expressions, avoidance of fixed wording patterns and formulaic expressions, the addition of natural transition sentences and explanatory statements, and the use of more colloquial and flexible expressions, maintaining the content while eliminating typical AI-generated features. The data before and after polishing are tested according to the detection process described in 6.1, using graph structure feature models, traditional text feature models, and confidence-weighted entropy fusion models respectively, to evaluate the performance of each model on the original and polished data.

[0106] 6.3 Performance Comparison

[0107] Table 1 Performance Comparison of Different Detection Methods

[0108]

[0109] Table 1 shows the performance of the three models on the data before and after polishing. In terms of average F1 score, the confidence-weighted entropy fusion model achieves 0.8404, an improvement of 11.1% compared to the text feature model and 7.7% compared to the graph structure feature model, demonstrating the best overall performance. The decrease in F1 score of the fusion model falls between that of the two individual models, but the decrease rate is significantly lower than that of the text feature model. This indicates that the fusion strategy successfully utilizes the high accuracy advantage of text features and the adversarial robustness advantage of graph structure features, achieving a balance between detection accuracy and adversarial robustness.

[0110] In summary, the confidence-weighted entropy-based adaptive fusion model proposed in this invention significantly enhances adversarial robustness while maintaining high detection accuracy, providing an effective solution for building practical AI text detection systems.

[0111] This embodiment verifies the effectiveness of the method of the present invention on a news text dataset, and the experimental results fully demonstrate the significant advantages of the method. The method of the present invention can be widely applied to the detection of AI-generated content of various text types, such as news content, academic papers, and social media, providing an effective solution for building a reliable AI content detection system, and has significant theoretical and practical application value.

[0112] Finally, it should be noted that the purpose of disclosing the embodiments is to help those skilled in the art further understand the technical solutions and effects of the present invention, rather than to limit the scope of the present invention. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made to the specific implementation schemes according to the needs of actual applications. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of the present invention, and the scope of protection of the present invention is determined by the scope defined in the appended claims.

Claims

1. An AI-generated text detection method based on graph structure features, characterized in that, Includes the following steps: 1) Dataset construction: Collect manually written text sample data, use multiple large language models to generate AI text samples corresponding to the manually written text samples, and build a training dataset; 2) Entity Relationship Extraction and Graph Structure Construction: Based on the semantic characteristics of the domain to which the text sample data belongs, define the graph pattern and clarify the entity type and relation type; Entity relation extraction is performed using a large language model, and the text is transformed into structured entity relation triples according to the specified graph pattern, including entity identifier, entity type, entity name, relation type, and confidence information. A directed graph is constructed based on the extracted entity-relation triples, with entities as nodes and relations as edges, and each edge carrying relation type and confidence attributes; 3) Graph structure feature extraction: Multi-dimensional feature extraction is performed on the graph structure constructed in step 2). The multi-dimensional features of the graph structure include graph topological statistical features, node centrality features, entity relationship distribution features, and graph embedding features. Among them, the graph topology statistical features reflect the organization and connection patterns of the entity relationship network in the text; the node centrality features measure the importance and structural position of different entities in the graph; the entity relationship distribution features statistically analyze the quantity distribution and confidence of different types of entities and relationships; the graph embedding features extract the vector representation of the graph, obtain the average value of all node embeddings as the graph embedding, and obtain the entity embedding and relationship embedding respectively by semantically encoding the entity text and relationship description. 4) Training the graph structure feature model: Select multiple machine learning algorithms to train a classification model based on the graph structure features extracted in step 3), and select the optimal graph structure feature model through cross-validation and hyperparameter optimization; 5) Detection of text to be detected: For the text to be detected, entity relations are extracted and graph structure is constructed according to step 2), graph structure features are extracted according to step 3), the extracted features are input into the graph structure feature model trained in step 4), the detection results are output, and it is determined whether the text is generated by AI.

2. An AI-generated text detection method based on an adaptive fusion model with confidence-weighted entropy, as described in claim 1, characterized in that, Traditional text feature extraction and model training are introduced, and the traditional text feature model and graph feature model are adaptively fused based on confidence-weighted entropy. AI-generated text detection is then performed based on this adaptive fusion model. Specifically: The traditional text feature extraction and model training are as follows: traditional text feature extraction is performed on the text, and the traditional text features include text statistical features and word frequency-inverse document frequency (TF-IDF) lexical features; Multiple machine learning algorithms were selected to train a classification model based on traditional text features, and the optimal model was selected through cross-validation and hyperparameter optimization. The adaptive fusion strategy based on confidence-weighted entropy is as follows: Based on the uncertainty of the model prediction quantified by information entropy, the fusion weights of the graph feature model and the text feature model are dynamically adjusted. Specifically, this involves: obtaining the prediction probability distributions of the graph feature model and the text feature model for the test samples; calculating the prediction entropy of each model; and then calculating the average entropy; dynamically adjusting the weights of the graph feature model according to the relationship between the average entropy and the entropy threshold, relying more on the text feature model when the average entropy is low, and relying more on the graph feature model when the average entropy is high; obtaining the final prediction probability through weighted fusion; if the prediction probability is greater than the classification threshold, it is determined to be AI-generated text. Hyperparameter optimization of the adaptive fusion model: The hyperparameters of the fusion strategy, including entropy threshold, basic weight, maximum weight, and classification threshold, are tuned through grid search. The optimal model is selected through cross-validation and hyperparameter optimization. For step 5) Detection of the text to be detected: For the text to be detected, entity relations are extracted and graph structure is constructed according to step 2), graph structure features are extracted according to step 3), and traditional text features are extracted at the same time. The extracted graph structure features and traditional text features are input into the adaptive fusion model, and the detection result is output to determine whether the text is generated by AI.

3. The method as described in claims 1 and 2, characterized in that, The large language models used in step 1) include DeepSeek, Wenxin Yiyan, Zhipu GLM, and Tongyi Qianwen; the AI ​​text sample generation adopts a continuation-style generation strategy, retaining the original title and beginning as context, and requiring the model to generate subsequent content.

4. The method as described in claims 1 and 2, characterized in that, In step 3), different types of entities are counted, including people, organizations, locations, events, and times; different types of relationships include participation relationships, location relationships, time relationships, causal relationships, correlation relationships, and affiliation relationships.

5. The method as described in claims 1 and 2, characterized in that, The topological statistical features of the graph in step 3) include the number of nodes, the number of edges, graph density, the number of connected components, connectivity, clustering coefficient, average shortest path, graph diameter, edge-to-node ratio, number of two-hop paths, number of three-hop paths, number of triangles, and number of linear chains; the node centrality features include average degree centrality, maximum degree centrality, average degree, maximum degree, degree standard deviation, height-to-node ratio, degree skewness, average betweenness centrality, maximum betweenness centrality, and average closeness centrality. The entity relationship distribution features include the quantity distribution of five types of entities, the quantity distribution of six types of relationships, and the mean and standard deviation of relationship confidence; graph embedding features are extracted using the Node2Vec algorithm and the BGE-M3 model.

6. The method as described in claims 1 and 2, characterized in that, The machine learning algorithms selected in step 4) include support vector machine, random forest, gradient boosting tree, XGBoost, LightGBM, logistic regression, Naive Bayes, decision tree, and K-nearest neighbors; K-fold cross-validation is used for model evaluation, and grid search is used for hyperparameter optimization, with F1 score as the evaluation metric.

7. The method as described in claim 2, characterized in that, The text statistical features include text size features, lexical diversity features, punctuation features, and complexity index; TF-IDF lexical features are generated through Chinese word segmentation, stop word filtering, and a TF-IDF vectorizer.

Images