A content detection method based on retrieval enhancement large language model generation

Abstract

This disclosure provides a retrieval-enhanced method for detecting content generated by a large language model. It comprises three parts: the first part involves the construction and caching of a corpus, constructing a human-written corpus and a large-model-rewritten corpus offline; the label for human-written text is set to y=0, and the label for text generated by the large language model is set to y=1; the second part involves retrieving relevant and rewritten texts and using them as prefixes for the text to be detected; the third part involves calculating the generation probability difference of the text under different contextual conditions using a detection method, and combining this with the log-likelihood detection features of the text to finally obtain a judgment on whether the text was generated by the large language model.

Description

Technical Field

[0001] This disclosure relates to the field of computer science, and more specifically, to a method for content detection based on a large language model generated by retrieval enhancement. Background Technology

[0002] In recent years, with the rapid development of large language model technology, these models can generate logically rigorous and structurally fluent text, which has been applied in various real-world scenarios. However, this has also raised public concerns about issues such as academic plagiarism, copyright infringement, and news fabrication. Therefore, content detection generated by large language models has become an important means of maintaining content authenticity, protecting intellectual property rights, and preventing the spread of misinformation. Effective detection methods not only help identify the source of generated content but also support the establishment of a more transparent and credible content creation environment, attracting considerable attention and research from scholars. Existing detection techniques mainly employ classification detectors based on fine-tuned pre-trained models and zero-shot detectors based on large models.

[0003] Fine-tuning pre-trained model classification detectors: This approach treats the detection of content generated by large language models as a traditional binary classification task in machine learning, where a label of 1 represents text generated by the large language model, and a label of 0 represents human-written text. For example, pioneering work such as OpenAI has used large-scale internet corpora and paired text generated by GPT-2 to train the RoBRETa pre-trained model for classification tasks. Furthermore, HC3 and MPU have introduced broader datasets and more complex training processes to enhance the robustness of classification detectors to the domain and length of the input text. However, classification detectors trained on limited data remain limited to detecting data from specific domains and large language models, and their accuracy tends to decrease when detecting more advanced content generated by large language models or text outside the training data domain.

[0004] Zero-Shot Detectors Based on Large Language Models: To enhance detection robustness, existing research focuses on analyzing the feature differences in the generation probability distribution between human-written text and text generated by large language models. These methods rely on large language model detectors to compute the output probability distribution of the text to be detected. However, discovering general and effective detection features remains a challenge. For example... Figure 1As shown, many researchers have pioneered the exploration of effective fundamental features, such as the log-likelihood, entropy, and rank (LogRank) of the next word in the predicted probability distribution of the text to be detected. Based on this, subsequent works such as DetLLM and GhostBuster have investigated how to combine these fundamental features to achieve better detection results. Furthermore, methods such as DNA-GPT obtain finer-grained detection features by randomly extracting several fragments of the text to be detected and comparing them with the continuation content generated by large language model detectors. In addition, studies such as DetLLM, DetectGPT, and Fast-DetectGPT generate a large number of perturbed texts of the text to be detected and improve detection performance by comparing the likelihood difference between the text to be detected and the perturbed version. Despite these advances, detecting text generated by large language models still faces robustness challenges in scenarios with short input lengths. This highlights the necessity of further exploring more effective methods for detecting content generated by large language models.

[0005] Despite the progress made in existing large language model-generated content detection techniques, several challenges remain in practical applications. First, the text to be detected may be short, limiting the number of detection features it contains. Second, the text may involve domain-specific knowledge; if the detector's training data lacks relevant domain knowledge, detection performance will be negatively impacted. To quantify this issue, this invention explores detection performance under different text lengths (input lengths of 32, 64, 128, 256, 384, and 512) and various detection scenarios (such as news, novels, and knowledge-based question answering). The results show that almost all detectors experience a significant performance drop when the input length is less than 128, and it is difficult to maintain consistently high performance across different scenarios. This indicates that the difference between the information in the text to be detected and the detector's knowledge base hinders detection effectiveness. Unfortunately, current research on large language model-generated content detection largely focuses on the text itself, neglecting the writing features of the vast amount of human-written text available in corpora. The above challenges can be summarized into two points: (1) how to achieve more effective detection performance when the text to be detected is short and lacks information; (2) how to use known human writing features to enhance the detection capability of the detector. Summary of the Invention

[0006] The purpose of this disclosure is to provide a content detection method based on retrieval enhancement using a large language model, addressing the problem of low detection accuracy due to short length or insufficient information in the text to be detected. By utilizing known human writing features, this method effectively improves the accuracy of content detection using a large language model. In a general sense, given a text T = {t1,…,t...} containing l tokens to be detected... l The relationship between the predicted detection score y and the pre-calculated threshold ∈ is determined by judging the relationship between the predicted score y>∈. The text to be detected is judged as text generated by the large language model if the predicted score satisfies y>∈, otherwise it is judged as text written by humans.

[0007] The judgment calculation process is as follows: using a retrieval instrument R and an external corpus C, the top k relevant human-written texts P of the text to be detected T are retrieved. 1:k =R C (T,k), further obtain the large language model rewritten text corresponding to the retrieved text. The text rewritten by the large language model and the text to be detected are used together to calculate the probability of the text generated by the large language model. Finally, a threshold is used to determine whether the text is generated by the large language model. When the value is greater than ∈, it is determined to be generated by a large language model;

[0008] The process of calculating the probability of text generated by the large language model includes three parts:

[0009] The first part is the construction and caching of the corpus. In the offline stage, a human-written corpus and a large-scale model-rewritten corpus are constructed. The labels of human-written text are set to y=0, and the labels of text generated by the large language model are set to y=1.

[0010] The second part involves retrieving relevant text and rewriting text, which are then used as prefixes for the text to be detected.

[0011] The third part involves calculating the generation probability differences of the text to be detected under different contextual conditions using detection methods, and combining this with the log-likelihood detection features of the text to be detected.

[0012] The specific method for retrieving relevant texts is as follows: First, a manually compiled corpus C is introduced. This manually compiled corpus includes a Wikipedia dump from December 2018 and the MS MARCO corpus. The search engine R is then used to obtain the k most relevant texts to the text to be detected.

[0013] P 1:k ={P1,…,P k}=R C (T,k)

[0014] These retrieved texts are used as prefixes to the text to be detected, providing context that offers additional relevant information.

[0015] The method for rewriting the text is as follows: using LLaMA3-8B-Instruct as the large model rewriter, defined as:

[0016]

[0017] To accelerate online detection, an additional offline processing flow is introduced. The vLLM large model inference acceleration framework is used to pre-generate large model rewritten versions of relevant human-written texts that may be used and store them as a cache.

[0018] The detection method is implemented as follows: during online detection, the retrieval system obtains relevant human-written text P related to the text to be detected. 1:k Rewriting text Using the two text segments as prefixes, the difference in probability distribution of the text to be detected under different contexts is calculated using cross-entropy:

[0019]

[0020] Where V represents the vocabulary size of the detector based on a large model, t ij This represents the sum of the cross-entropies at each position in the text to be detected for the j-th token in the prediction vocabulary at the i-th position in the text to be detected. This represents the difference in probability distribution under the influence of contextual factors, namely, human-written relevant text and text rewritten by large models.

[0021] Then, the probability distribution differences of the text to be detected under different contexts are combined with the log-likelihood detection features of the text to be detected:

[0022]

[0023] To address the issue of the test text being a mixture of large language model-generated text and human-written text, a large language model-based detector is used. The score of the i-th token in the test text is calculated to represent the probability that it is text generated by the large language model: s(t i ) = s i (T)-s i-1 (T).

[0024] The technical effects to be achieved by the embodiments of the present invention are as follows:

[0025] (1) From the perspective of zero-shot detector based on large language model, a new method is designed to assist detection by using several retrieved texts related to the text to be detected as prefixes. These retrieved texts provide additional contextual information as prefixes, so that the output probability distribution of the text to be detected changes under specific prefix conditions, thereby providing richer detection features.

[0026] (2) This invention utilizes two retrieval corpora, MS MARCO and Wiki, which contain rich human writing features, to provide more representative human text. A large language model rewriter is also used to rewrite the retrieved relevant text to maintain semantic similarity to the text to be detected, while introducing subtle differences in structure and word choice. By analyzing the probability distribution differences between the original and rewritten retrieved texts, this invention can identify finer-grained generation features, providing a basis for subsequent detection.

[0027] (3) A HALO detection method based on a large language model detector is proposed. This is the first method for detecting generated content using a large language model that incorporates retrieval information. The HALO method not only utilizes the contextual information provided by relevant retrieval text, but also enhances the detection capability of generated content by introducing rewritten text for comparison. By integrating multi-source information, this method not only improves the recognition accuracy of complex generated features, but also enhances the robustness of detection under different domains and text length conditions. Attached Figure Description

[0028] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0029] Figure 1 This is a schematic diagram illustrating an example of a zero-shot detector based on a large model in the prior art;

[0030] Figure 2 This is a schematic diagram illustrating the HALO (Large Language Model Generation Content Detection Framework) according to an embodiment of the present disclosure;

[0031] Figure 3 This is a schematic diagram illustrating online detection using the HALO detection framework according to an embodiment of the present disclosure. Detailed Implementation

[0032] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0033] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0034] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0035] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0036] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0037] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0038] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0039] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0040] Figure 2 This is a schematic diagram illustrating a content detection method based on a large language model generated by retrieval enhancement according to an embodiment of the present disclosure.

[0041] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 2 As shown. This invention proposes a Retrieval-Enhanced Large Language Model Generated Content Detection (HALO) method, which combines the retrieval of relevant human-written text and the enhancement of rewritten text. The HALO framework can be roughly divided into three parts: (1) construction and caching of human-written corpora and large model rewritten corpora; (2) retrieval of relevant text and rewritten text as prefixes of the text to be detected; (3) calculation of the generation probability difference of the text to be detected under different contextual conditions, and using the log-likelihood of the text to be detected divided by the generation probability difference as the final detection score. The specific implementation details will be discussed in detail below.

[0042] Large language model-generated content detection task definition:

[0043] Given a text T = {t1, ...,t2} containing l tokens to be detected. l The goal of the detection task is to predict a detection score y. This invention sets the label for human-written text as y=0 and the label for text generated by a large language model as y=1. Using a pre-calculated threshold ∈ (which can be calculated using methods such as Youden's J statistic), this invention can classify text to be detected with a predicted score satisfying y>∈ as text generated by a large language model; otherwise, it classifies it as human-written text.

[0044] In this paper, we propose a HALO detection method. The core of the method is to use a retrieval tool R and an external corpus C to retrieve the top k relevant human-written texts P of the text to be detected T. 1:k =R C(T,k). This invention can further obtain the large language model rewritten text corresponding to the retrieved text. These texts, along with the text to be detected, are used together in the HALO detection method to determine whether the text was generated by a large language model based on a threshold.

[0045] Related text retrieval:

[0046] To obtain effective relevant context to enhance detection performance, this invention first introduces a manually compiled corpus C. Specifically, this invention introduces two widely used corpora: (1) a Wikipedia dump from December 2018; and (2) the MS MARCO corpus. These two corpora were chosen because they were built earlier than the release date of GPT-2, the first large-scale pre-trained model capable of generating fluent text (early 2019), and they contain a large amount of high-quality human knowledge and question-answering data, thus providing effective additional reference information for the text to be detected. Using a retrieval tool R, this invention can obtain the top k most relevant texts to the text to be detected:

[0047] P 1:k ={P1,…,P k}=P C (T,k)

[0048] These retrieved texts serve as prefixes to the text to be detected, and thus provide richer features for subsequent detection by providing additional relevant context.

[0049] Related text rewriting:

[0050] Human-written text and its large-model rewritten version show significant differences in grammatical structure and word choice, but maintain semantic similarity. Based on this observation, this invention uses both human-written relevant text and large-model rewritten relevant text as different prefixes to analyze the differences in probability distribution of the text to be detected under the influence of different relevant prefixes. This invention uses LLaMA3-8B-Instruct as the large-model rewriter, defined as:

[0051]

[0052] To accelerate online detection, this invention introduces an additional offline processing flow. This invention utilizes a vLLM large-model inference acceleration framework to pre-generate large-model rewritten versions of potentially used human-written relevant text and stores them as a cache. In actual detection, the retrieved human-written relevant text first searches for the corresponding rewritten text in the cache, and if no results are found, the large-model rewriter is invoked to generate the rewritten text, thereby improving the efficiency of online detection.

[0053] HALO detection method flow:

[0054] The core of HALO's detection framework is based on the observation that the probability distribution of the text to be detected differs under different relevant contexts. Specifically, because text generated by large models typically has high semantic coherence, the probability distribution of the text to be detected changes relatively little when given multiple contexts containing sufficient information. Human-written text, on the other hand, is relatively weaker in semantic coherence, and therefore may tend to use more diverse word candidates under the influence of different contexts, leading to greater differences in probability distribution. The framework of this invention is as follows... Figure 2 As shown. Specifically, this invention first constructs a human-written corpus and a large-model rewritten corpus in an offline phase. In the online detection phase, the retrieval device obtains relevant human-written text P of the text to be detected. 1:k Rewriting text Using the two text segments as prefixes, the difference in probability distribution of the text to be detected under different contexts is calculated using cross-entropy:

[0055]

[0056] Where V represents the vocabulary size of the detector based on a large model, t ij This represents the j-th token in the predicted vocabulary at the i-th position in the text to be detected. Therefore, the sum of the cross-entropies at each position in the text to be detected is: This represents the difference in probability distribution under the influence of context when relevant text is written by humans and when text is rewritten by large models.

[0057] Furthermore, this invention combines it with log-likelihood detection features:

[0058]

[0059] A smaller cross-entropy indicates a smaller difference in probability distribution across different contexts, suggesting that the text to be detected is more consistent with the characteristics of text generated by a large model. Therefore, a higher prediction score s(T) indicates a higher probability that the text to be detected is generated by a large model.

[0060] In real-world detection scenarios, the text to be detected may simultaneously contain text generated by a large language model and human-written text. Therefore, this invention requires further acquisition of fine-grained detection features at the word level of the text to be detected. This invention uses a detector based on a large language model to calculate the score of the i-th token in the text to be detected:

[0061] s(t i ) = s i (T)-s i-1 (T)

[0062] This score represents the "gain" of the i-th token to the final detection score. Therefore, a higher gain indicates that the text is more likely generated by a large language model, affecting the overall judgment of the text to be detected. This invention further demonstrates the impact of different tokens in a text on the final detection score through an interactive page using the HALO detection method. For example... Figure 3 As shown, the input text to be detected is a GPT-4o-generated introduction to the Python language. The HALO detection method can identify which text fragments are more likely to have been generated by a large language model. Figure 3 This indicates that the features generated by the word-level large language model are more significant in the first half of the text, demonstrating higher detection confidence.

[0063] Evaluation of HALO detection methods:

[0064] This invention evaluates the performance of the HALO detection method from three perspectives: cross-domain detection capability, cross-source generation detection capability, and robustness to the detection of the length of the text to be detected.

[0065] (1) Cross-domain detection capability: In order to evaluate the detection performance in various scenarios, this invention uses four cross-domain detection datasets: HC3, XSum, WritingPrompts, and SQuAD, which cover a variety of common detection scenarios such as finance, medicine, forum Q&A, news articles, novel sequels, and Wikipedia Q&A. Therefore, it can evaluate the detection generalization ability of the detection method in the case of a mixture of various possible large language model generated content and human-written text, and verify its actual performance in real applications.

[0066] (2) Cross-source detection capability: Real-world detection scenarios are typically black-box detection, where a fixed detector based on a large model is used to detect text generated from different large models and human-written text. In the initial exploration of this invention, we found that different sources of generation significantly affect detection performance. For example, OpenAI's closed-source GPT-4o model exhibits stronger resistance to detection compared to open-source models such as LLaMA and QWen. Therefore, cross-source detection capability is also an important benchmark for evaluation, aiming to explore whether the detection method can maintain high robustness in detection scenarios with different sources.

[0067] (3) Robustness to the length of the text to be detected: In practical applications, the length of the text to be detected may vary. Existing research shows that the performance of content detection methods generated by large language models drops significantly when processing shorter texts. To further analyze the robustness of the detection method to changes in input length, this invention truncated the input text to different lengths of 32, 64, 128, 256, 384, and 512, and tested its detection performance. A robust detection method should be able to maintain relatively stable and consistent detection results when dealing with shorter texts.

[0068] For the evaluation from the three perspectives mentioned above, this invention uses Area Under the Receiver Operating Characteristic Curve (AUROC) to measure detection capability. A higher AUROC value indicates that the detector tends to assign more discriminative scores to text generated by large models and text written by humans, and achieves higher classification accuracy at a given classification threshold ∈.

[0069] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A content detection method based on retrieval enhancement using a large language model, characterized in that, The process of calculating the probability of text generated by the large language model includes three steps: The first step is the construction and caching of the corpus, which involves constructing a human-written corpus and a large model-rewritten corpus in the offline stage. Set the tags of human-written text to The tags for the text generated by the large language model are set to ; The second step is to retrieve relevant and rewritten texts and use them as prefixes for the text to be detected. The third step is to calculate the generation probability difference of the text to be detected under different context conditions through detection methods, and combine the log likelihood of the text to be detected with the generation probability difference to detect features. The specific method for the third step is as follows: by giving a given... The text to be detected for each token The predicted detection score is obtained by comparison with a pre-calculated threshold. The relationship between the predicted scores and the threshold will determine whether the predicted scores are greater than a certain value. If the text to be detected is determined to be generated by a large language model, it is determined to be written by a human. The judgment calculation process is as follows: using a retrieval device and external corpus Retrieve the text to be detected The former Text written by relevant humans Further obtain the large language model rewritten text corresponding to the retrieved text. The large language model rewritten text and the text to be detected are used together to calculate the probability of the text generated by the large language model. Finally, a threshold is used to determine whether the text is generated by the large language model. Greater than When the language is generated by a large language model, it is determined to be generated by that model. The detection method is implemented as follows: during online detection, the retrieval system obtains relevant human-written texts related to the text to be detected. Rewriting text Using the two text segments as prefixes, the probability distribution difference of the text to be detected under different contexts is calculated using cross-entropy: in, This represents the vocabulary size of a detector based on a large model. Indicates the first... The position in the predicted vocabulary is the [number]th position. The sum of the cross-entropy of each token at each position in the text to be detected: This represents the difference in probability distribution under the influence of contextual factors, namely, human-written relevant text and text rewritten by large models. Then, the probability distribution differences of the text to be detected under different contexts are combined with the log-likelihood detection features of the text to be detected: Using a detector based on a large language model, the first... Each token is used to calculate its score, which represents the probability of generating text for a large language model. .

2. The content detection method based on retrieval enhancement and large language model generation as described in claim 1, characterized in that, The specific method for retrieving relevant texts is as follows: First, a manually compiled corpus is introduced. The manually compiled corpus includes a Wikipedia dump from December 2018 and the MS MARCO corpus, using a search engine. Obtain the top k most relevant texts to the text to be detected: These retrieved texts are used as prefixes to the text to be detected, providing context that offers additional relevant information.

3. The content detection method based on retrieval enhancement of a large language model as described in claim 2, characterized in that, The method for rewriting the text is as follows: using LLaMA3-8B-Instruct as the large model rewriter, defined as: The vLLM large model inference acceleration framework is used to pre-generate large model rewritten versions of relevant human-written texts and store them as cache.

Images