An AI-generated news identification method based on ensemble learning

Abstract

In the embodiment of the present invention, an AI-generated news identification method of integrated learning is provided, which belongs to the field of data processing technology, and specifically includes: inputting a sample news text set into a random forest module for TF-IDF vectorization, BERT vectorization and additional feature extraction respectively; splicing the extracted feature vectors; training the random forest module using the spliced ​​feature vectors; inputting the news text to be detected into the trained random forest module to obtain a first probability value; inputting the news text to be detected into a text consistency module based on the continuation of the first paragraph, calculating the first average similarity and the first average logarithmic probability ratio; inputting the news text to be detected into a text consistency verification module based on the rewriting of the central sentence, calculating the second average similarity and the second average logarithmic probability ratio; performing weighted summation to obtain a comprehensive score and generating the identification result of the news text to be detected accordingly. Through the scheme of the present invention, the recognition efficiency, accuracy and adaptability are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to an AI-generated news identification method based on ensemble learning. Background Art

[0002] At present, large-scale generative language models have shown unprecedented capabilities, such as ChatGPT, which has swept the world and is able to write coherent and relevant long texts in response to user-specified prompts. However, since AI-generated texts may lack real factual basis, the information therein may be false or misleading, which may cause readers to be misled or have wrong cognition. For example, criminals use ChatGPT to generate relevant news about people's livelihood, create false news to spread rumors, etc. These behaviors will have a negative impact on social harmony. In addition, due to the explosive growth of information on online news portals and social media websites, it has become more difficult to distinguish true from false news. The widespread dissemination of false news may cause social panic, unrest and chaos, and have serious negative impacts on the entire society. The detection efficiency and results of existing artificial intelligence text detection schemes are not ideal.

[0003] It can be seen that there is an urgent need for an AI-generated news identification method based on ensemble learning with strong recognition efficiency, accuracy and adaptability. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an AI-generated news identification method based on ensemble learning, which at least partially solves the problems of poor recognition efficiency, accuracy and adaptability in the prior art.

[0005] The embodiment of the present invention provides an AI-generated news identification method based on ensemble learning, comprising:

[0006] Step 1: Input the sample news text set into the random forest module, and perform TF-IDF vectorization, BERT vectorization, and additional feature extraction respectively;

[0007] Step 2, concatenate the feature vectors after TF-IDF vectorization, BERT vectorization and additional feature extraction to obtain a concatenated feature vector;

[0008] Step 3, train the random forest module using the concatenated feature vector;

[0009] Step 4, input the news text to be detected into the trained random forest module to obtain a first probability value;

[0010] Step 5, inputting the news text to be detected into a text consistency module based on the continuation of the first paragraph, and calculating the first average similarity and the first average logarithmic probability ratio;

[0011] Step 6, inputting the news text to be detected into the text consistency verification module based on the central sentence rewriting, and calculating the second average similarity and the second average logarithmic probability ratio;

[0012] Step 7, performing weighted summation based on the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability to obtain a comprehensive score and generate an identification result of the news text to be detected based on the score.

[0013] According to a specific implementation of the embodiment of the present invention, step 1 specifically includes:

[0014] Step 1.1, count the frequency of each word in the sample news text set, then multiply it by an inverse document frequency weight, and finally represent each text as a sparse feature vector, where each dimension of the feature vector corresponds to a word, and the value represents the importance of the word in the text;

[0015] Step 1.2, use the BERT model to convert the text in the sample news text set into a feature vector of fixed dimension;

[0016] Step 1.3, extract text features related to text structure and content information from the sample news text set.

[0017] According to a specific implementation of the embodiment of the present invention, step 3 specifically includes:

[0018] Step 3.1, train the random forest module using the concatenated feature vector and calculate the loss function of the random forest module, where the expression of the loss function is

[0019]

[0020] in, is the total loss function of the random forest module, N is the total number of samples, is the loss function for each individual sample, where is the actual label of the i-th sample, is the random forest module for the i-th sample input The predicted output of , using the parameter set ;

[0021] Step 3.2, based on the loss function, a grid search strategy is used to find the optimal parameter set of the random forest module, where the expression of the grid search strategy is

[0022]

[0023] in, Indicates that for a given set of parameters in the k-fold cross validation process The average loss calculated;

[0024] Step 3.3, using the k-fold cross-validation method to verify the optimal parameter set, to obtain a trained random forest module, wherein the expression of the k-fold cross-validation method is:

[0025]

[0026] Where k represents the number of folds in cross validation. Indicates that the parameter set is used on the k-th fold data loss.

[0027] According to a specific implementation of the embodiment of the present invention, the expression of the first probability value is:

[0028]

[0029] in, Represents the news text to be detected is the first probability value generated by AI, represents the best model obtained by the random forest model during training, Used to detect news text Convert to a vector.

[0030] According to a specific implementation of the embodiment of the present invention, step 5 specifically includes:

[0031] Step 5.1: Extract the news text to be tested The first paragraph of the content is input into the large language model to generate the continuation text ;

[0032] Step 5.2, calculate the continuation text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is

[0033]

[0034] The vocabulary similarity The expression is

[0035]

[0036] in, and Respectively represent and A vocabulary collection;

[0037] Step 5.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is

[0038]

[0039] in, and Represent the weights of content similarity and vocabulary similarity respectively;

[0040] Step 5.4: Use the large language model to calculate the continued text Relative to the news text to be detected The log-odds ratio

[0041]

[0042] in, It is a continuation text The probability of Is the news text to be tested probability;

[0043] Step 5.5, respectively call multiple large language models to execute steps 5.1 to 5.3 to obtain a first average similarity, and respectively call multiple large language models to execute step 5.4 to obtain a first average logarithmic probability ratio, wherein the expression of the first average similarity is

[0044]

[0045] The expression of the first average log probability ratio is:

[0046]

[0047] Where n represents the number of large language models used.

[0048] According to a specific implementation of the embodiment of the present invention, step 6 specifically includes:

[0049] Step 6.1: Extract the news text to be tested using a large language model The central sentence of the sentence is then used to regenerate a rewritten text with a similar number of words. ;

[0050] Step 6.2, calculate the rewritten text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is

[0051]

[0052] The vocabulary similarity The expression is

[0053]

[0054] in, and Respectively represent and A vocabulary collection;

[0055] Step 6.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is

[0056] ;

[0057] Step 6.4: Rewrite the text using the large language model Relative to the news text to be detected The log-odds ratio

[0058]

[0059] in, Rewrite the text The probability of Is the news text to be tested probability;

[0060] Step 6.5, respectively call multiple large language models to execute steps 6.1 to 6.3 to obtain a second average similarity, and respectively call multiple large language models to execute step 6.4 to obtain a second average logarithmic probability ratio, wherein the expression of the second average similarity is

[0061]

[0062] The expression of the second average log probability ratio is:

[0063]

[0064] Where n represents the number of large language models used.

[0065] According to a specific implementation of the embodiment of the present invention, the expression of the comprehensive score is:

[0066]

[0067] in, Represents the comprehensive score, , , , , Respectively represent the weights of the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability.

[0068] The AI-generated news identification scheme based on ensemble learning in the embodiment of the present invention includes: step 1, inputting the sample news text set into the random forest module, and performing TF-IDF vectorization, BERT vectorization and additional feature extraction respectively; step 2, splicing the feature vectors after TF-IDF vectorization, BERT vectorization and additional feature extraction to obtain a spliced ​​feature vector; step 3, training the random forest module using the spliced ​​feature vector; step 4, inputting the news text to be detected into the trained random forest module to obtain a first probability value; step 5, inputting the news text to be detected into the text consistency module based on the continuation of the first paragraph, and calculating the first average similarity and the first average logarithmic probability ratio; step 6, inputting the news text to be detected into the text consistency verification module based on the rewriting of the central sentence, and calculating the second average similarity and the second average logarithmic probability ratio; step 7, performing weighted summation according to the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability, to obtain a comprehensive score and generate an identification result of the news text to be detected accordingly.

[0069] The beneficial effects of the embodiments of the present invention are as follows: through the scheme of the present invention, based on the self-consistency of continuation and generation prediction, a variety of technical means such as random forest, text feature extraction, large model generation, similarity calculation, average logarithmic probability ratio, TF-IDF and BERT are combined, the prediction results of multiple basic classifiers are integrated, and the synergistic effect of ensemble learning is utilized to improve the accuracy and robustness of AI-generated text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 A flowchart of an AI-generated news identification method based on ensemble learning provided in an embodiment of the present invention;

[0072] Figure 2A system framework diagram of an AI-generated news identification method based on ensemble learning provided in an embodiment of the present invention;

[0073] Figure 3 A structural flow chart of a random forest module provided by an embodiment of the present invention;

[0074] Figure 4 A structural flow chart of a text consistency module based on first paragraph continuation provided by an embodiment of the present invention;

[0075] Figure 5 A structural flow chart of a text consistency verification module based on central sentence rewriting provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0077] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0078] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present invention, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0079] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the form, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0080] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0081] In the field of computational science and natural language processing, the remarkable progress and wide application of generative large language models have become a research hotspot in recent years. With their excellent text generation capabilities, these models have demonstrated unprecedented creativity and flexibility, stimulating widespread interest and practical exploration in academia and industry. At the same time, the rapid popularization of social networking platforms and the convenience of information dissemination have also spawned a series of emerging challenges. In particular, the phenomenon of using large language models such as ChatGPT to create and spread rumors and false news has become increasingly serious, which has had a profound and complex impact on the network information ecology and seriously threatened the credibility of information and the knowledge construction of the public domain. At present, the traditional manual review mechanism is still the main means of identifying the authenticity of news content. Although this method guarantees the quality of information to a certain extent, its inherent limitations cannot be ignored. On the one hand, the human review process is time-consuming and labor-intensive, and it is difficult to cope with the real-time monitoring needs of massive information; on the other hand, the intervention of personal subjective factors may introduce bias and errors, thereby weakening the objectivity and accuracy of the review results. Therefore, the development of an automated, high-precision AI-generated news recognition technology has become an urgent need to improve the credibility of news dissemination and maintain the health of the information ecology.

[0082] Therefore, how to detect AI-generated news has received widespread attention from researchers. In order to avoid unnecessary losses and negative impacts, we need to develop methods to help identify AI-generated news so that readers can obtain more authentic information.

[0083] At present, some general AI text detectors can be found on the Internet, but none of them are very effective in the field of people's livelihood news detection. In foreign research, Chakraborty et al. verified the possibility of identifying AI-generated English text through experiments. In addition, there are already some tools for detecting AI text in reality, such as GPTZero, OpenAI's text classifier, DetectGPT, etc. Although these tools can check whether English text is generated by a large model, sometimes the detection results are not always accurate. At the same time, compared with English-based AI text detection, these tools are generally effective in detecting Chinese text.

[0084] An embodiment of the present invention provides an AI-generated news identification method based on ensemble learning, which can be applied to the false news identification process in Internet scenarios.

[0085] See also Figure 1 , is a flow chart of a method for identifying news generated by AI based on ensemble learning provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown, the method mainly comprises the following steps:

[0086] Step 1: Input the sample news text set into the random forest module, and perform TF-IDF vectorization, BERT vectorization, and additional feature extraction respectively;

[0087] Furthermore, the step 1 specifically includes:

[0088] Step 1.1, count the frequency of each word in the sample news text set, then multiply it by an inverse document frequency weight, and finally represent each text as a sparse feature vector, where each dimension of the feature vector corresponds to a word, and the value represents the importance of the word in the text;

[0089] Step 1.2, use the BERT model to convert the text in the sample news text set into a feature vector of fixed dimension;

[0090] Step 1.3, extract text features related to text structure and content information from the sample news text set.

[0091] In specific implementation, the random forest module, as one of the core components of our detector framework, uses machine learning algorithms to classify and estimate the probability of text. The reason for choosing random forest is based on its high efficiency in processing large-scale data sets and its good processing ability for high-dimensional data. In addition, random forest can provide feature importance evaluation, which combines the prediction results of multiple decision trees to improve the overall prediction accuracy and stability, and helps to further understand the impact of text features on classification decisions. Through the trained random forest model, we can calculate the first probability value of the news text being generated by AI, so as to perform preliminary text classification and analysis. The module structure is as follows Figure 3 shown.

[0092] For example, the embodiment of the present invention collects 1,000 livelihood news from website A as a human text dataset, and uses multiple LLMs such as GPT3.5 to generate 1,000 livelihood news as an AI text dataset based on the news titles in the human text dataset. We divide the dataset into a training set and a test set with a ratio of 80% and 20% to ensure the performance of the model on unseen data. At the same time, we use the jieba library to perform Chinese word segmentation on the text and convert the text into a sequence of words to facilitate subsequent feature extraction and vectorization.

[0093] In the random forest module, feature extraction is a crucial step, which provides the basis for the training and prediction of the random forest model by converting text into numerical features. Feature extraction is mainly completed through three steps: TF-IDF vectorization, BERT vectorization, and additional feature extraction, and these features are spliced ​​together to form the final feature vector.

[0094] TF-IDF (Term Frequency-Inverse Document Frequency) is a commonly used vectorization method for text data, which can reflect the importance of words in a document. In the TF-IDF vectorization process, we first count the term frequency (TF) of each word in the text, and then multiply it by an inverse document frequency (IDF) weight to reduce the importance of common words to the text. Finally, each text is represented as a sparse vector, where each dimension corresponds to a word and the value represents the importance of the word in the text, as shown below.

[0095]

[0096] in, TF-IDF vectorization of text, Indicates that the dimension of the vector is n.

[0097] BERT is a pre-trained language representation model proposed by Google AI in 2018. The core feature of BERT is the bidirectional encoder using the Transformer architecture, which allows the model to simultaneously consider the left and right contexts of each word in the text, thereby producing richer and more accurate word representations.

[0098] BERT vectorization refers to using the BERT model to convert text (such as words, phrases, or entire sentences) into vectors of fixed dimensions, as shown below.

[0099]

[0100] in, BERT vectorization of text representation, Indicates that the dimension of the vector is m. These vectors can capture the semantic information of the text, making them useful for various natural language processing tasks such as text classification, question answering systems, sentiment analysis, etc.

[0101] In addition to the features of TF-IDF vectorization and BERT vectorization, we extracted a series of additional features by analyzing a large amount of AI news text and human news. These include (1) the average number of sentences per paragraph, (2) the presence of sentences with more than 130 words, (3) vocabulary diversity, (4) the number of numbers in each news article, (5) whether there are brackets, (6) whether there are book title marks, (7) whether there are ellipsis marks, (8) whether there are punctuation marks, (9) whether there are percentage signs, and whether the text contains the following words ((10) promote, (11) provide, (12) China, (13) say, (14) introduce, (15) report, (16) economy, (17) with, (18) exceed, (19) grow, (20) year, (21) improve, and (22) with. These additional features can provide additional information about the structure and content of the text and help the model better understand the text, as shown below.

[0102]

[0103] in, Additional feature vectorization representing text, Indicates that the dimension of the vector is 22.

[0104] Step 2, concatenate the feature vectors after TF-IDF vectorization, BERT vectorization and additional feature extraction to obtain a concatenated feature vector;

[0105] In specific implementation, the sparse vector obtained by TF-IDF vectorization, the word vector obtained by BERT vectorization, and the additional feature concatenation vector are concatenated together to form the final feature vector, as shown below.

[0106]

[0107] We can see that for each article in the training set It can be represented by a vector of n+m+23 dimensions. Each article in the dataset represents a row of vectors. Assuming that the dataset has x articles, all the articles will constitute a The comprehensive feature matrix is ​​used to train the random forest model.

[0108] Step 3, train the random forest module using the concatenated feature vector;

[0109] Based on the above embodiment, step 3 specifically includes:

[0110] Step 3.1, train the random forest module using the concatenated feature vector and calculate the loss function of the random forest module, where the expression of the loss function is

[0111]

[0112] in, is the total loss function of the random forest module, N is the total number of samples, is the loss function for each individual sample, where is the actual label of the i-th sample, is the random forest module for the i-th sample input The predicted output of , using the parameter set ;

[0113] Step 3.2, based on the loss function, a grid search strategy is used to find the optimal parameter set of the random forest module, where the expression of the grid search strategy is

[0114]

[0115] in, Indicates that for a given set of parameters in the k-fold cross validation process The average loss calculated;

[0116] Step 3.3, using the k-fold cross-validation method to verify the optimal parameter set, to obtain a trained random forest module, wherein the expression of the k-fold cross-validation method is:

[0117]

[0118] Where k represents the number of folds in cross validation. Indicates that the parameter set is used on the k-th fold data loss.

[0119] In specific implementation, the random forest algorithm is used for model training, and the specific loss function is shown below.

[0120]

[0121] in, It is the total loss function of the model, which represents the average loss of the model on the entire training data set, and N is the total number of samples. is the loss function for each individual sample, where is the actual label of the i-th sample, is the model's input to the i-th sample The predicted output of , using the parameter set .

[0122] In order to find the optimal model parameters, a grid search strategy is adopted as shown below.

[0123]

[0124] The main parameters adjusted include: (1) n_estimators: the number of decision trees, considering three options: 100, 200, and 300. (2) max_depth: the maximum depth of the tree, including 10, 20, and unlimited depth, to find the best model configuration.

[0125] in, represents the optimal parameter set that minimizes the cross-validation loss. is the k-fold cross validation process for a given parameter Calculate the average loss.

[0126] The k-fold cross-validation method is used to verify the performance of the model under different configurations to ensure that the selected model can maintain good predictive ability on unseen data, as shown below.

[0127]

[0128] Here, k represents the number of folds in cross-validation, indicating how many subsets the data set is divided into, usually k=5. Indicates that on the k-th fold data, use the parameter This method can reduce the risk of model overfitting and provide a measure of model stability.

[0129] Step 4, input the news text to be detected into the trained random forest module to obtain a first probability value;

[0130] Furthermore, the expression of the first probability value is:

[0131]

[0132] in, Represents the news text to be detected is the first probability value generated by AI, represents the best model obtained by the random forest model during training, Used to detect news text Convert to a vector.

[0133] In specific implementation, once the model training is completed, we input the text to be detected into the best model trained by the random forest model, and use the classification results of each decision tree to calculate the first probability value of the text to be detected.

[0134]

[0135] in Represents the text to be detected This is the first probability value generated by AI. This probability value reflects the probability that the model believes the text to be detected is AI-generated text, and can be used as the first basis for judging the text generation method. represents the best model obtained by the random forest model during training, Used to detect the text Converted into a vector, Represents a set of parameters for the random forest model, including the number of trees, tree depth, and the minimum number of samples required to split a node.

[0136] Step 5, inputting the news text to be detected into a text consistency module based on the continuation of the first paragraph, and calculating the first average similarity and the first average logarithmic probability ratio;

[0137] Based on the above embodiment, step 5 specifically includes:

[0138] Step 5.1: Extract the news text to be tested The first paragraph of the content is input into the large language model to generate the continuation text ;

[0139] Step 5.2, calculate the continuation text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is

[0140]

[0141] The vocabulary similarity The expression is

[0142]

[0143] in, and Respectively represent and A vocabulary collection;

[0144] Step 5.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is

[0145]

[0146] in, and Represent the weights of content similarity and vocabulary similarity respectively;

[0147] Step 5.4: Use the large language model to calculate the continued text Relative to the news text to be detected The log-odds ratio

[0148]

[0149] in, It is a continuation text The probability of Is the news text to be tested probability;

[0150] Step 5.5, respectively call multiple large language models to execute steps 5.1 to 5.3 to obtain a first average similarity, and respectively call multiple large language models to execute step 5.4 to obtain a first average logarithmic probability ratio, wherein the expression of the first average similarity is

[0151]

[0152] The expression of the first average log probability ratio is:

[0153]

[0154] Where n represents the number of large language models used.

[0155] In specific implementation, the text consistency verification module is a key component of the system framework corresponding to the method of the present invention, which aims to evaluate the consistency between the generated text and the original text by using a large language model (LLM) and a specific consistency analysis method. This module includes two submodules, namely, a text consistency module based on the continuation of the first paragraph and a text consistency module based on the rewriting of the central sentence.

[0156] In the text consistency module based on the first paragraph continuation, we observed that the AI-generated text showed a high degree of consistency in the continuation prediction of the large model. Based on this finding, we first extracted the text to be tested. The first paragraph of the generated text is input into the large language model LLMi. Based on the prediction ability of the large language model, multiple evaluation indicators such as TF-IDF similarity and vocabulary similarity are combined to set the weight of each indicator, and the similarity and logarithmic probability ratio of the generated text and the original text are comprehensively evaluated, such as Figure 4 shown.

[0157]

[0158] in Represents the continuation text generated by a single large language model LLMi With original text The comprehensive similarity value of . and Represent the weights of their respective similarities. The two similarities are:

[0159] (1) TF-IDF content similarity: , used to determine the content similarity in the text and help identify the key information of the text. The similarity between two documents is usually calculated using cosine similarity, as shown below.

[0160]

[0161] (2) Lexical similarity: , which is mainly calculated based on the vocabulary sets of two texts and is used to quickly detect scenarios with duplicate content or very similar texts.

[0162]

[0163] in, and Respectively represent and A collection of words.

[0164] We calculated the log odds ratio using a specific large model. LLMi generated text The probability of Is the original text The probability of generating text for a single large language model LLMi is Relative to the original text The log-odds ratio can be expressed as:

[0165]

[0166] By applying different LLMi multiple times, this module finally outputs the first average similarity value and the first average log probability ratio, which will serve as the second basis for judging the text generation method, as shown below.

[0167]

[0168]

[0169] in represents the first average similarity, represents the first mean log-odds ratio, and n is the number of different LLMs used.

[0170] Step 6, inputting the news text to be detected into the text consistency verification module based on the central sentence rewriting, and calculating the second average similarity and the second average logarithmic probability ratio;

[0171] Based on the above embodiment, step 6 specifically includes:

[0172] Step 6.1: Extract the news text to be tested using a large language model The central sentence of the sentence is then used to regenerate a rewritten text with a similar number of words. ;

[0173] Step 6.2, calculate the rewritten text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is

[0174]

[0175] The vocabulary similarity The expression is

[0176]

[0177] in, and Respectively represent and A vocabulary collection;

[0178] Step 6.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is

[0179] ;

[0180] Step 6.4: Rewrite the text using the large language model Relative to the news text to be detected The log-odds ratio

[0181]

[0182] in, Rewrite the text The probability of Is the news text to be tested probability;

[0183] Step 6.5, respectively call multiple large language models to execute steps 6.1 to 6.3 to obtain a second average similarity, and respectively call multiple large language models to execute step 6.4 to obtain a second average logarithmic probability ratio, wherein the expression of the second average similarity is

[0184]

[0185] The expression of the second average log probability ratio is:

[0186]

[0187] Where n represents the number of large language models used.

[0188] In the specific implementation, in the consistency analysis module of the text based on the central sentence rewriting, we found that the news articles generated by AI have significant consistency. After extracting the key information in the AI ​​news text, the model can still generate new texts with similar content and vocabulary style. Based on the ability of the large language model to maintain the consistency of the original content and vocabulary usage, this module first uses the large language model LLMi to extract the central sentence of the original AI text, and then regenerates an article with a similar number of words based on this central sentence. . Similar to the above submodule, text is generated by comparison With original text The TF-IDF similarity and vocabulary similarity of the generated text are used to comprehensively evaluate the similarity and logarithmic probability ratio between the generated text and the original text. Figure 5 shown.

[0189]

[0190]

[0191] Represents the rewritten text generated by a single large language model LLMi With original text The comprehensive similarity value of middle, yes The probability of yes The probability of . and The formula is as follows.

[0192] (1) ,

[0193] (2) , and Respectively represent and A collection of words.

[0194] By applying different LLMi multiple times, this module finally outputs the second average similarity value and the second average log probability ratio as the third basis for judging the text generation method, as shown below.

[0195]

[0196]

[0197] in represents the second average similarity, represents the second mean log-odds ratio, and n is the number of different LLMs used.

[0198] Step 7, performing weighted summation based on the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability to obtain a comprehensive score and generate an identification result of the news text to be detected based on the score.

[0199] Furthermore, the expression of the comprehensive score is:

[0200]

[0201] in, Represents the comprehensive score, , , , , Respectively represent the weights of the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability.

[0202] In the specific implementation, the final integrated calculation adopts a multi-index fusion strategy, which comprehensively considers the first probability value provided by the random forest module, the first average similarity value and the first average logarithmic probability ratio provided by the text consistency module based on the continuation of the first paragraph, and the second average similarity value and the second average logarithmic probability ratio provided by the text consistency module based on the rewriting of the central sentence. By weighting these indicators, we can get a comprehensive score that can more accurately reflect the consistency and authenticity of the text, as shown below.

[0203]

[0204] in, Represents the comprehensive score, , , , , Represent the weights of the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity, and the second average logarithmic probability respectively. Then the comprehensive score can be compared with the preset threshold. If the comprehensive score is greater than the threshold, it can be determined that the news text is AI-generated news.

[0205] This comprehensive score not only enhances the model's ability to detect AI-generated news texts, but also improves the accuracy and robustness of recognition. In addition, the application of the integrated method ensures that the advantages of different modules can be fully utilized, thereby achieving better performance in practical applications.

[0206] The AI-generated news identification method based on ensemble learning provided in this embodiment integrates the prediction results of multiple basic classifiers and uses brief prompts to allow multiple LLMs to continue writing and generating. Finally, multiple indicators are used to judge the text similarity, the average log probability ratio, and a series of synergistic effects of ensemble learning such as the random forest model, so as to be competent for more complex AI detection tasks, aiming to improve the accuracy and robustness of false news identification. The experimental design and analysis will verify the effectiveness of the proposed method and its applicability on large-scale data sets, laying a theoretical and practical foundation for building a more reliable information filtering mechanism, thereby promoting the authenticity of news reports and the healthy development of the social information environment.

[0207] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof.

[0208] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An AI-generated news identification method based on ensemble learning, characterized in that: include: Step 1: Input the sample news text set into the random forest module, and perform TF-IDF vectorization, BERT vectorization, and additional feature extraction respectively; Step 2, concatenate the feature vectors after TF-IDF vectorization, BERT vectorization and additional feature extraction to obtain a concatenated feature vector; Step 3, train the random forest module using the concatenated feature vector; Step 4, input the news text to be detected into the trained random forest module to obtain a first probability value; Step 5, inputting the news text to be detected into a text consistency module based on the continuation of the first paragraph, and calculating the first average similarity and the first average logarithmic probability ratio; The step 5 specifically includes: Step 5.1: Extract the news text to be tested The first paragraph of the content is input into the large language model to generate the continuation text ; Step 5.2, calculate the continuation text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is The vocabulary similarity The expression is in, and Respectively represent and A vocabulary collection; Step 5.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is in, and Represent the weights of content similarity and vocabulary similarity respectively; Step 5.4: Use the large language model to calculate the continued text Relative to the news text to be detected The log-odds ratio in, It is a continuation text The probability of Is the news text to be tested probability; Step 5.5, respectively call multiple large language models to execute steps 5.1 to 5.3 to obtain a first average similarity, and respectively call multiple large language models to execute step 5.4 to obtain a first average logarithmic probability ratio, wherein the expression of the first average similarity is The expression of the first average log probability ratio is: Where n represents the number of large language models used; Step 6, inputting the news text to be detected into the text consistency verification module based on the central sentence rewriting, and calculating the second average similarity and the second average logarithmic probability ratio; The step 6 specifically includes: Step 6.1: Extract the news text to be tested using a large language model The central sentence of the sentence is then used to regenerate a rewritten text with a similar number of words. ; Step 6.2, calculate the rewritten text With the news text to be detected Content similarity Similarity to vocabulary , wherein the content similarity The expression is The vocabulary similarity The expression is in, and Respectively represent and A vocabulary collection; Step 6.3, based on content similarity Similarity to vocabulary Calculate the comprehensive similarity , wherein the comprehensive similarity The expression is ； Step 6.4: Rewrite the text using the large language model Relative to the news text to be detected The log-odds ratio in, Rewrite the text The probability of Is the news text to be tested probability; Step 6.5, respectively call multiple large language models to execute steps 6.1 to 6.3 to obtain a second average similarity, and respectively call multiple large language models to execute step 6.4 to obtain a second average logarithmic probability ratio, wherein the expression of the second average similarity is The expression of the second average log probability ratio is: Where n represents the number of large language models used; Step 7, performing weighted summation based on the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability to obtain a comprehensive score and generate an identification result of the news text to be detected based on the score.

2. The AI-generated news identification method based on ensemble learning according to claim 1 is characterized in that , the step 1 specifically includes: Step 1.1, count the frequency of each word in the sample news text set, then multiply it by an inverse document frequency weight, and finally represent each text as a sparse feature vector, where each dimension of the feature vector corresponds to a word, and the value represents the importance of the word in the text; Step 1.2, use the BERT model to convert the text in the sample news text set into a feature vector of fixed dimension; Step 1.3, extract text features related to text structure and content information from the sample news text set.

3. The AI-generated news identification method based on ensemble learning according to claim 2 is characterized in that , the step 3 specifically includes: Step 3.1, train the random forest module using the concatenated feature vector and calculate the loss function of the random forest module, where the expression of the loss function is in, is the total loss function of the random forest module, N is the total number of samples, is the loss function for each individual sample, where is the actual label of the i-th sample, is the random forest module input to the i-th sample The predicted output of , using the parameter set ; Step 3.2, based on the loss function, a grid search strategy is used to find the optimal parameter set of the random forest module, where the expression of the grid search strategy is: in, Indicates that for a given set of parameters in the k-fold cross validation process The average loss calculated; Step 3.3, use the k-fold cross-validation method to verify the optimal parameter set to obtain a trained random forest module, where the expression of the k-fold cross-validation method is: Where k represents the number of folds in cross validation. Indicates that the parameter set is used on the k-th fold data loss.

4. The AI-generated news identification method based on ensemble learning according to claim 3 is characterized in that , the expression of the first probability value is in, Represents the news text to be detected is the first probability value generated by AI, represents the best model obtained by the random forest model during training, Used to detect news text Convert to a vector.

5. The AI-generated news identification method based on ensemble learning according to claim 4 is characterized in that , the expression of the comprehensive score is in, Represents the comprehensive score, , , , , Respectively represent the weights of the first probability value, the first average similarity, the first average logarithmic probability, the second average similarity and the second average logarithmic probability.

Images