Large language model generation content detection method based on retrieval enhancement

Abstract

The invention provides a large language model generation content detection method based on retrieval enhancement. Comprising three parts: the first part is construction and caching of a corpus, and a human writing corpus and a large model rewriting corpus are constructed in an offline stage; setting the label of the human writing text as y = 0, and setting the label of the large language model generation text as y = 1; the second part is used for retrieving related texts and rewritten texts and respectively taking the related texts and the rewritten texts as prefixes of to-be-detected texts; and the third part is that the generation probability difference of the to-be-detected text under different context conditions is calculated through a detection method, and is combined with the log-likelihood detection features of the to-be-detected text to finally obtain a judgment conclusion that whether the text is generated by the large language model or not.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and more specifically, to a method for generating content detection using a large language model based on retrieval enhancement. Background Art

[0002] In recent years, with the rapid development of large language model technology, large language models can generate texts with rigorous logic and smooth structure. They have been applied to a variety of real-world scenarios, but have also caused public concerns about issues such as academic plagiarism, copyright infringement, and news fabrication. Therefore, large language model generated content detection has become an important means to maintain content authenticity, protect intellectual property rights, and prevent the spread of false information. Effective detection methods not only help to identify the source of generated content, but also provide support for establishing a more transparent and credible content creation environment, which has attracted the attention and research of many scholars. Existing detection technology solutions are mainly achieved by fine-tuning the classification detector of the pre-trained model and the zero-shot detector based on the large model.

[0003] Fine-tuning the pre-trained model classification detector: The detection of content generated by the large language model is regarded as a traditional machine learning binary classification task, where the label 1 indicates that the large language model generates text; the label 0 indicates that the text is written by humans. For example, some pioneering works such as OpenAI collect a large amount of Internet corpus and paired texts generated by GPT-2 to let the RoBRETa pre-trained model learn classification tasks. In addition, HC3 and MPU further introduced a wider range of data sets and more complex training processes to enhance the robustness of classification detectors to the domain and length of the input text to be detected. However, classification detectors trained on limited data are still limited to the detection of data in specific domains and large language models, and the accuracy is easily reduced when detecting more advanced content generated by large language models and text content outside the domain of training data.

[0004] Zero-shot detectors based on large language models: In order to enhance the robustness of detection, existing research focuses on analyzing the feature differences in the generation probability distribution between human-written text and text generated by large language models. This type of method relies on the large language model detector to calculate the output probability distribution of the text to be detected. However, it is still a challenge to find universal and effective detection features. Figure 1As shown in Figure 1, many researchers have pioneered the exploration of some effective basic features, such as the log-likelihood and entropy of the output probability distribution of the text to be detected, and the ranking (LogRank) of the next word in the predicted probability distribution. Based on this, subsequent work such as DetLLM, GhostBuster, etc. studied how to combine these basic features to achieve better detection results. Furthermore, methods represented by DNA-GPT randomly intercept several fragments of the text to be detected and compare the continuation content generated by the large language model detector to obtain more fine-grained detection features. In addition, studies represented by DetLLM, DetectGPT and Fast-DetectGPT generate a large amount of perturbed text of the text to be detected, and improve the detection performance by comparing the likelihood difference between the text to be detected and the perturbed version. Despite these advances, detecting text generated by large language models still faces robustness challenges in scenarios such as short input length. This highlights the need to further explore more effective methods for detecting content generated by large language models.

[0005] Although the existing large language model generated content detection technology has made certain progress, in practical application, detection still faces some challenges. First, the text to be detected may be short, so the detection features contained are limited; in addition, the text to be detected may involve knowledge in a specific field. If the relevant field knowledge is lacking in the training data of the detector, the detection performance will be adversely affected. In order to quantitatively analyze this problem, the present invention explores the detection performance under different detection text lengths (input length is 32, 64, 128, 256, 384, 512) and multiple detection scenarios (such as news, novels, and knowledge questions and answers). The results show that the performance of almost all detectors decreases significantly when the input length is less than 128, and it is difficult to maintain high performance in detection in different scenarios. This shows that the difference between the information of the text to be detected and the knowledge base of the detector will hinder the detection effect. Unfortunately, the current research on large language model generated content detection focuses on the text to be detected itself, while ignoring the writing characteristics of a large number of human-written texts available in the corpus. The above challenges can be summarized into two points: (1) how to achieve more effective detection performance when the length of the text to be detected is short and the information is insufficient; (2) how to use known human writing features to enhance the detection ability of the detector. Summary of the invention

[0006] The purpose of the embodiments of the present disclosure is to provide a method for detecting content generated by a large language model based on retrieval enhancement, so as to solve the problem that the detection accuracy is low due to the short length or insufficient information of the text to be detected. By utilizing known human writing features, this method effectively improves the accuracy of detecting content generated by a large language model. In a general aspect, by giving a text to be detected containing l tokens T = {t1,…,t l}, by judging and calculating the relationship between the predicted detection score y and the pre-calculated threshold ∈, the text to be detected whose predicted score satisfies y>∈ is determined to be the text generated by the large language model, otherwise it is determined to be human-written text;

[0007] The judgment calculation process is: using a retriever R and an external corpus C to retrieve the first k relevant human-written texts P of the text to be detected T 1:k =R C (T,k), further obtain the large language model rewritten text corresponding to the retrieved text The large language model rewritten text and the to-be-detected text are used together to calculate the likelihood of the text generated by the large language model, and finally a threshold is used to determine whether the text is generated by the large language model. When it is greater than ∈, it is judged to be generated by a large language model;

[0008] The process of calculating the likelihood of text generated by the large language model includes three parts:

[0009] The first part is the construction and caching of the corpus. In the offline stage, the human-written corpus and the large model-rewritten corpus are constructed; the label of the human-written text is set to y=0, and the label of the text generated by the large language model is set to y=1;

[0010] The second part is to retrieve the related text and the rewritten text and use them as prefixes of the text to be detected;

[0011] The third part is to calculate the generation probability difference of the text to be detected under different context conditions through the detection method, and combine it with the log-likelihood detection feature of the text to be detected.

[0012] The specific method of retrieving relevant texts is as follows: first, introduce a manually written corpus C, which includes the dump of Wikipedia in December 2018 and the MS MARCO corpus, and use a retriever R to obtain the top k relevant texts that are most relevant to the text to be detected:

[0013] P 1:k = {P1,…,P k = R C (T,k)

[0014] These retrieved texts are used as prefixes of the text to be detected, becoming contexts that provide additional relevant information.

[0015] The method of rewriting the text is: using LLaMA3-8B-Instruct as a large model rewriter, defined as:

[0016]

[0017] In order to speed up online detection, an additional offline processing process is introduced. The vLLM large model inference acceleration framework is used to pre-generate a large model rewritten version of the human-written text that may be used and store it as a cache.

[0018] The detection method is implemented as follows: during online detection, the retriever obtains the human-written text P related to the text to be detected. 1:k With rewritten text Take the two texts as prefixes respectively, and use cross entropy to calculate the probability distribution difference of the text to be detected in different contexts:

[0019]

[0020] Where V represents the vocabulary size of the detector based on the large model, t ij It represents the sum of the cross entropy corresponding to the jth token in the predicted vocabulary at the i-th position of the text to be detected and each position of the text to be detected: Represents the difference in probability distribution under the influence of context when humans write relevant text and large models rewrite text;

[0021] After that, the probability distribution difference of the text to be detected in different contexts is combined with the log-likelihood detection feature of the text to be detected:

[0022]

[0023] In order to deal with the mixed large language model generated text and human-written text in the text to be detected, a detector based on a large language model is used to calculate the score of the i-th token in the text to be detected to represent the possibility of the text generated by the large language model: s(t i )=s i (T)-s i-1 (T).

[0024] The technical effects to be achieved by the embodiments of the present invention are:

[0025] (1) From the perspective of zero-shot detector based on large language model, a new method is designed. Several search texts related to the text to be detected are used as prefixes to assist detection. These search texts provide additional contextual information as prefixes, so that the output probability distribution of the text to be detected changes under specific prefix conditions, thereby providing richer detection features.

[0026] (2) Two retrieval corpora with rich human writing features, MS MARCO and Wiki, are used to provide more representative human texts. A large language model rewriter is also used to rewrite the relevant text retrieved to keep its semantics close to the text to be detected, but with slight differences in structure and wording. By analyzing the difference in probability distribution of the text to be detected under the original retrieval text and the rewritten retrieval text, the present invention can identify more fine-grained generation features and provide a basis for subsequent detection.

[0027] (3) A HALO detection method based on a large language model detector is proposed. This is the first large language model generated content detection method that combines retrieval information. The HALO method not only utilizes the contextual information provided by the relevant retrieval text, but also introduces rewritten text to build a comparison, further enhancing the detection ability of generated content. By integrating multi-source information, this method not only improves the recognition accuracy of complex generated features, but also enhances the detection robustness under different domains and text length conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects and features of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings.

[0029] Figure 1 is a schematic diagram showing an example of a zero-sample detector based on a large model in the prior art;

[0030] Figure 2 is a schematic diagram showing a large language model generating content detection framework HALO according to an embodiment of the present disclosure;

[0031] Figure 3 is a schematic diagram showing online detection of the detection framework HALO according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0033] The features described herein can be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, which will be clear after understanding the disclosure of the present application.

[0034] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more.

[0035] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, the first member, first component, first region, first layer, or first portion referred to in the examples may also be referred to as the second member, second component, second region, second layer, or second portion.

[0036] In the specification, when an element (such as a layer, a region or a substrate) is described as being “on”, “connected to” or “coupled to” another element, the element may be directly “on”, “connected to” or “coupled to” another element, or one or more other elements may be present therebetween. Conversely, when an element is described as being “directly on”, “directly connected to” or “directly coupled to” another element, there may be no other elements present therebetween.

[0037] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs after understanding the present disclosure. Unless explicitly defined as such herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and should not be interpreted in an idealized or overly formal manner.

[0039] Furthermore, in the description of examples, when it is considered that a detailed description of a well-known related structure or function would cause vague interpretation of the present disclosure, such a detailed description will be omitted.

[0040] Figure 2 It is a schematic diagram showing a method for generating content detection based on a large language model for retrieval enhancement according to an embodiment of the present disclosure.

[0041] In order to achieve the above-mentioned invention object, the technical framework adopted by the present invention is as follows Figure 2 As shown. The present invention proposes a large language model generated content detection method based on retrieval enhancement (HALO), a large language model generated content detection method that combines the retrieval of relevant texts written by humans and the enhancement of rewritten texts. The framework of HALO can be roughly divided into three parts: (1) Construction and caching of human-written corpus and large model rewritten corpus; (2) Retrieval of relevant texts and rewritten texts and use them as prefixes of the text to be detected respectively; (3) Calculation of the generation probability difference of the text to be detected under different context conditions, and use the log-likelihood of the text to be detected divided by the generation probability difference as the final detection score. Next, the present invention will discuss the specific implementation details in depth.

[0042] Large language model generated content detection task definition:

[0043] Given a text to be detected containing l tokens T = {t1,…,t l}, the goal of the detection task is to predict a detection score y. The present invention sets the label of human-written text to y=0, and the label of large language model-generated text to y=1. Through a pre-calculated threshold ∈ (which can be calculated by Youden's J statistic and other methods), the present invention can determine the text to be detected that satisfies the prediction score y>∈ as the text generated by the large language model, otherwise it is determined as human-written text.

[0044] In this paper, the present invention proposes a HALO detection method, the core of which is to use a retriever R and an external corpus C to retrieve the top k relevant human-written texts P of the text to be detected T 1:k =R C(T, k). The present invention can further obtain the large language model rewritten text corresponding to the search text These texts are used together with the text to be detected in the HALO detection method to determine whether the text is generated by the large language model through a threshold:

[0045] Related text search:

[0046] In order to obtain effective relevant context to enhance the detection effect, the present invention first introduces a manually written corpus C. Specifically, the present invention introduces two widely used corpora: (1) Wikipedia's dump in December 2018; (2) MS MARCO corpus. The reason why the present invention chooses these two corpora is that their construction time is earlier than the release date of GPT-2, the first large-scale pre-trained model that can generate fluent text (early 2019), and they contain a large amount of high-quality human knowledge and question-answering data, so they can introduce effective additional reference information for the text to be detected. Using a retriever R, the present invention can obtain the top k relevant texts that are most relevant to the text to be detected:

[0047] P 1:k = {P1,…,P k =P C (T,k)

[0048] These retrieved texts serve as prefixes of the text to be detected, and thus serve as contexts that provide additional relevant information, providing richer features for subsequent detection.

[0049] Relevant text rewrite:

[0050] The human-written text and its large model rewritten version show significant differences in grammatical structure and word choice, but can maintain semantic similarity. Based on this observation, the present invention uses both the human-written related text and the large model rewritten related text as different prefixes to analyze the difference in probability distribution of the text to be tested under the influence of different related prefixes. The present invention uses LLaMA3-8B-Instruct as the large model rewriter, which is defined as:

[0051]

[0052] In order to speed up online detection, the present invention additionally introduces an offline processing flow. The present invention uses the vLLM large model reasoning acceleration framework to pre-generate a large model rewritten version of the human-written related text that may be used and stores it as a cache. In actual detection, the retrieved human-written related text first searches for the corresponding rewritten text in the cache, and when the search yields no result, calls the large model rewriter to generate the rewritten text, thereby improving the efficiency of online detection.

[0053] HALO detection method process:

[0054] The core of HALO's detection framework is based on the following observation: the probability distribution of the text to be detected varies under different relevant contexts. Specifically, because the text generated by large models usually has high semantic coherence, the probability distribution of the text to be detected will change less when given multiple contexts containing sufficient information. However, human-written text is relatively weak in semantic coherence, so under the influence of different contexts, it may tend to use more diverse word candidates, resulting in greater differences in probability distribution. The framework of the present invention is as follows Figure 2 Specifically, the present invention first constructs a human-written corpus and a large model-rewritten corpus in the offline stage. In the online detection stage, the retriever obtains the human-written text P related to the text to be detected. 1:k With rewritten text Take the two texts as prefixes respectively, and use cross entropy to calculate the probability distribution difference of the text to be detected in different contexts:

[0055]

[0056] Where V represents the vocabulary size of the detector based on the large model, t ij represents the jth token in the predicted vocabulary at the i-th position of the text to be detected. Therefore, the sum of the cross entropy corresponding to each position of the text to be detected is: Represents the difference in probability distribution under the influence of context between human-written relevant text and large model-rewritten text.

[0057] Furthermore, the present invention combines it with the log-likelihood detection feature:

[0058]

[0059] The smaller the cross entropy, the smaller the difference in probability distribution under different contexts, which indicates that the text to be tested is more consistent with the characteristics of the text generated by the large model. Therefore, the higher the prediction score s(T), the higher the probability that the text to be tested is generated by the large model.

[0060] In actual detection scenarios, the text to be detected may be a mixture of text generated by a large language model and human-written text. Therefore, the present invention needs to further obtain fine-grained detection features at the word level of the text to be detected. The present invention uses a detector based on a large language model to calculate the score of the i-th token in the text to be detected:

[0061] s(t i )=s i (T)-s i-1 (T)

[0062] This score means the "gain" of the ith token to the final detection score. Therefore, if the gain is higher, it means that it is more likely to be a text generated by a large language model, which affects the judgment of the entire text to be detected. The present invention further uses the HALO detection method to display the gain effect of different tokens in a text on the final detection score through an interactive page. Figure 3 As shown in the figure, the input text to be tested is an introduction to the Python language generated by GPT-4o. The HALO detection method can mark which text fragments are more likely to be generated by the large language model. Figure 3 This shows that the features generated by the large language model at the word level are more significant in the first half of the text, reflecting a higher detection confidence.

[0063] Evaluation of HALO detection method:

[0064] This paper evaluates the performance of the HALO detection method from three perspectives: cross-domain detection capability, cross-generated source detection capability, and detection robustness to the length of the text to be detected:

[0065] (1) Cross-domain detection capability: In order to evaluate the detection performance in various scenarios, the present invention uses four cross-domain detection datasets: HC3, XSum, WritingPrompts, and SQuAD, covering a variety of common detection scenarios such as finance, medicine, forum Q&A, news articles, novel sequels, and Wikipedia entry Q&A. Therefore, it is possible to evaluate the detection generalization capability of the detection method in dealing with a variety of possible large language model-generated content mixed with human-written texts, and verify its actual performance in real applications.

[0066] (2) Cross-generation source detection capability: The actual detection scenario is usually black-box detection, that is, using a fixed large model-based detector to detect generated texts and human-written texts from different large models. In the preliminary exploration of the present invention, the present invention found that different generation sources have significant differences in detection effects. For example, the GPT-4o closed-source model developed by OpenAI has a stronger resistance to detection than open source models such as LLaMA and QWen. Therefore, cross-generation source detection capability is also an important benchmark for evaluation, which aims to explore whether the performance of the detection method can maintain a high level of robustness in detection scenarios with different generation sources.

[0067] (3) Detection robustness to the length of the text to be detected: In practical applications, the length of the text to be detected may vary. Existing studies have shown that the performance of the content detection method generated by a large language model will significantly decrease when processing shorter texts to be detected. In order to deeply analyze the robustness of the detection method to changes in input length, the present invention truncates the input text to different lengths such as 32, 64, 128, 256, 384, 512, etc., and tests its detection performance. A detection method with good robustness should be able to maintain a relatively stable and consistent detection effect when facing shorter texts.

[0068] For the evaluation of the above three perspectives, the present invention uses Area Under the Receiver Operating Characteristic Curve (AUROC) to measure the detection capability. A higher AUROC value indicates that the detector tends to give more discriminative scores to large model-generated texts and human-written texts, and can achieve higher classification accuracy when a given classification threshold ∈ is given.

[0069] Although some embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that modifications may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting content generated by a large language model based on retrieval enhancement, characterized in that: Given a text to be detected containing l tokens T = {t1,…,t l }, by judging and calculating the relationship between the predicted detection score y and the pre-calculated threshold ∈, the text to be detected whose predicted score satisfies y>∈ is determined to be the text generated by the large language model, otherwise it is determined to be human-written text; The judgment calculation process is: using a retriever R and an external corpus C to retrieve the first k relevant human-written texts P of the text to be detected T 1:k =R C (T,k), further obtain the large language model rewritten text corresponding to the retrieved text The large language model rewritten text and the to-be-detected text are used together to calculate the likelihood of the text generated by the large language model, and finally a threshold is used to determine whether the text is generated by the large language model. When it is greater than ∈, it is judged to be generated by a large language model; The process of calculating the likelihood of text generated by the large language model includes three parts: The first part is the construction and caching of the corpus. In the offline stage, the human-written corpus and the large model-rewritten corpus are constructed; the label of the human-written text is set to y=0, and the label of the text generated by the large language model is set to y=1; The second part is to retrieve the related text and the rewritten text and use them as prefixes of the text to be detected; The third part is to calculate the generation probability difference of the text to be detected under different context conditions through the detection method, and use the logarithmic likelihood of the text to be detected and the generation probability difference to divide the detection feature combination.

2. The method for detecting content generated by a large language model based on retrieval enhancement as claimed in claim 1, characterized in that: The specific method of retrieving relevant texts is as follows: first, a manually written corpus C is introduced, which includes the dump of Wikipedia in December 2018 and the MS MARCO corpus, and the retriever R is used to obtain the top k relevant texts most relevant to the text to be detected: P 1:k ={P1,…,P k }=R C (T,k) These retrieved texts are used as prefixes of the text to be detected, becoming contexts that provide additional relevant information.

3. The method for detecting content generated by a large language model based on retrieval enhancement as claimed in claim 2, characterized in that: The method of rewriting the text is: using LLaMA3-8B-Instruct as a large model rewriter, defined as: In order to speed up online detection, an additional offline processing process is introduced. The vLLM large model inference acceleration framework is used to pre-generate a large model rewritten version of the human-written text that may be used and store it as a cache.

4. The method for detecting content generated by a large language model based on retrieval enhancement as claimed in claim 3, characterized in that: The detection method is implemented as follows: during online detection, the retriever obtains the human-written text P related to the text to be detected. 1:k With rewritten text Take the two texts as prefixes respectively, and use cross entropy to calculate the probability distribution difference of the text to be detected in different contexts: Where V represents the vocabulary size of the detector based on the large model, t ij It represents the sum of the cross entropy corresponding to the jth token in the predicted vocabulary at the i-th position of the text to be detected and each position of the text to be detected: Represents the difference in probability distribution under the influence of context when humans write relevant text and large models rewrite text; After that, the probability distribution difference of the text to be detected in different contexts is combined with the log-likelihood detection feature of the text to be detected: In order to deal with the mixed large language model generated text and human-written text in the text to be detected, a detector based on a large language model is used to calculate the score of the i-th token in the text to be detected to represent the possibility of the text generated by the large language model: s(t i )=s i (T)-s i-1 (T).

Images