Method and device for large language model generation text detection, electronic equipment, storage medium and program product

By extracting keywords and regenerating text using a text regeneration model, the method enhances the accuracy of identifying text generated by large language models by ensuring a correlation with the original text.

CN119088906APending Publication Date: 2024-12-06CHINA UNITED NETWORK COMM GRP CO LTD +2
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411311948.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing methods for detecting text generated by large language models are inaccurate due to the variability in responses from different models, leading to incorrect judgments about the source of the text.

Method used

A method that involves obtaining keywords from the text, generating candidate questions using a candidate source model, and regenerating text with a text regeneration model to ensure similarity with the original text, using feature vector extraction for accurate comparison.

Benefits of technology

Improves the accuracy of determining whether text is generated by a large language model by ensuring a correlation between the regenerated text and the original text through keyword-restricted regeneration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088906A_ABST
    Figure CN119088906A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a method and device for large language model generation text detection, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-detected text, and acquiring keywords in the to-be-detected text; inputting the to-be-detected text into at least one alternative source model to obtain at least one alternative question; inputting the keywords and the alternative questions into a preset text regeneration model to obtain a regeneration text comprising the keywords; obtaining the similarity between the to-be-detected text and the regenerated text; and if the similarity between the to-be-detected text and the regenerated text is greater than a preset threshold, determining that the to-be-detected text is generated by a large language model. The method is used for achieving the effect of more accurately judging whether the to-be-detected text is generated by the large language model or not.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device, storage medium and program product for large language model generated text detection. Background Art

[0002] In the field of Natural Language Processing (NLP), large language models (LLM) have gradually matured. With the advent of large language models such as ChatGPT, large language models (LLM) have been widely used in various industries and integrated into people's work and life.

[0003] Since large language models can generate text that looks real but is actually false, if the source of the text is not clarified, the text generated by the large language model may be abused, causing the spread of false information. Therefore, it is necessary to detect the text and clarify the source of the text, so as to reduce the possibility of large language model text being abused.

[0004] The existing text detection method is to input the text to be detected into a large language model, trigger the large language model to generate a question, then input the question into the large language model again to obtain the answer information corresponding to the question, and finally determine the similarity between the answer information and the text to be detected to determine whether the text to be detected is generated by the large language model. However, since the large language model can generate completely different text content based on the same question, in this case, it may lead to incorrect judgment of the text source of the text to be detected. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device, storage medium and program product for generating text detection using a large language model, so as to achieve the effect of improving the accuracy of text detection.

[0006] In a first aspect, an embodiment of the present application provides a method for large language model generation text detection, comprising: obtaining a text to be detected, and obtaining keywords in the text to be detected;

[0007] Inputting the text to be detected into at least one candidate source model to obtain at least one candidate question;

[0008] Inputting the keywords and each of the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords;

[0009] Obtaining the similarity between the text to be detected and the regenerated text;

[0010] If the similarity between the text to be detected and the regenerated text is greater than a preset threshold, it is determined that the text to be detected is generated by a large language model.

[0011] In a possible implementation, inputting the text to be detected into at least one candidate source model to obtain candidate questions includes:

[0012] constructing prompt words for generating the alternative questions;

[0013] The prompt word and the text to be detected are input into at least one candidate source model to obtain an alternative question.

[0014] In a possible implementation, obtaining the similarity between the to-be-detected text and the regenerated text includes:

[0015] Performing word segmentation processing on the text to be detected and the regenerated text respectively to obtain a first word segmentation set corresponding to the text to be detected and a second word segmentation set corresponding to the regenerated text;

[0016] Inputting the first word segmentation set and the second word segmentation set into a preset feature vector extraction model to obtain a first feature vector corresponding to the first word segmentation set and a second feature vector corresponding to the second word segmentation set;

[0017] The similarity between the first feature vector and the second feature vector is calculated according to a cosine similarity algorithm.

[0018] In a possible implementation, the keyword and each of the candidate questions are input into a preset text regeneration model to obtain a regenerated text including the keyword, including:

[0019] If there are multiple candidate questions, the multiple candidate questions are integrated to obtain a target question;

[0020] The keywords and the target question are input into the text regeneration model to obtain a regenerated text including the keywords.

[0021] In a possible implementation, a plurality of the candidate questions are integrated to obtain a target question, including:

[0022] The plurality of candidate questions are input into a preset question integration model to obtain the target question.

[0023] In a possible implementation manner, after determining that the text to be detected is generated by a large language model, the method further includes:

[0024] Determining the similarity between the target question and each of the candidate questions;

[0025] The candidate source model corresponding to the source question is determined to be the large language model that generates the text to be detected; wherein the source question is the candidate question with the greatest similarity to the target question.

[0026] In a possible implementation manner, after obtaining the similarity between the to-be-detected text and the regenerated text, the method further includes:

[0027] If the similarity between the text to be detected and the regenerated text is less than or equal to a preset threshold, it is determined that the text to be detected is manually written.

[0028] In a second aspect, an embodiment of the present application provides a device for large language model generated text detection, including:

[0029] A first acquisition module is used to acquire a text to be detected and acquire keywords in the text to be detected;

[0030] A question acquisition module, used for inputting the text to be detected into at least one candidate source model to obtain at least one candidate question;

[0031] A text acquisition module, used for inputting the keywords and each of the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords;

[0032] A second acquisition module, used to acquire the similarity between the to-be-detected text and the regenerated text;

[0033] The determination module is used to determine that the text to be detected is generated by a large language model when the similarity between the text to be detected and the regenerated text is greater than a preset threshold.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0035] The memory stores computer-executable instructions;

[0036] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0039] The method, device, electronic device, storage medium and program product for large language model generated text detection provided in the embodiments of the present application, by inputting the text to be detected into the large language model, obtaining the question corresponding to the text to be detected, and then obtaining the keywords in the text to be detected, and then inputting the keywords and the question corresponding to the text to be detected into the large language model together, to obtain the regenerated text including the keywords. In this way, even if the large language model can generate different text contents based on the same text, the text generated by the large language model can be restricted by the keywords to ensure that the regenerated text includes the keywords, so that there is a correlation between the regenerated text and the text to be detected, and then it can be more accurately determined whether the text to be detected is generated by the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A flowchart of a method for generating text detection using a large language model provided in this application;

[0042] Figure 2 A flowchart of a method for determining the source of a text to be detected provided by the present application;

[0043] Figure 3 A schematic diagram of the structure of the device for generating text detection using a large language model provided in this application;

[0044] Figure 4 A schematic diagram of the structure of the electronic device provided in this application.

[0045] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0046] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0047] The existing text detection method inputs the text to be detected into a large language model to trigger the large language model to generate a question, then inputs the question into the large language model again to obtain answer information corresponding to the question, and finally determines the similarity between the answer information and the text to be detected to determine whether the text to be detected is generated by the large language model.

[0048] This method requires that a large language model be selected in advance as auxiliary model A. If the text to be detected is originally generated by auxiliary model A, the text to be detected can be accurately recognized. However, if the text to be detected is generated by other large language models other than auxiliary model A, since different large language models may generate different answers to the same question, using auxiliary model A to recognize the text to be detected generated by other large language models may incorrectly determine that it is written by humans, resulting in missed detection.

[0049] In addition, even if the text to be detected is indeed generated by auxiliary model A, after the text to be detected is input into auxiliary model A and the question corresponding to the text to be detected is obtained, the question corresponding to the text to be detected is input into auxiliary model A again. Since the large language model can generate completely different text content based on the same question, auxiliary model A can generate a correct regenerated text based on the question corresponding to the text to be detected, but it is irrelevant to the text to be detected. In this way, the similarity between the regenerated text and the text to be detected may be low, and it is impossible to accurately determine that the text to be detected is generated by the large language model.

[0050] At the same time, the detection accuracy of the existing text detection method also depends on the accuracy of the question generated by the auxiliary model A based on the text to be detected. Since the existing text detection method only generates a question based on the text to be detected, and since the large language model can generate multiple questions corresponding to one text, the question corresponding to the text to be detected generated by the auxiliary model A may be biased. The regenerated text generated based on the question will further deviate from the text to be detected, which will result in a lower similarity between the regenerated text and the text to be detected, thereby erroneously classifying the text to be detected as being written by humans.

[0051] In order to solve the above problems, the present application provides a text detection method, which inputs the text to be detected into a large language model to obtain the questions corresponding to the text to be detected, then obtains the keywords in the text to be detected, and then inputs the keywords and the questions corresponding to the text to be detected into the large language model together to obtain a regenerated text including the keywords. In this way, even if the large language model can generate different text contents based on the same text, the text generated by the large language model can be restricted by keywords to ensure that the regenerated text includes keywords, so that there is a correlation between the regenerated text and the text to be detected, and then it can be more accurately determined whether the text to be detected is generated by the large language model.

[0052] Large language models are pre-trained models with a large amount of text data, which may come from books, articles, web pages, etc. After being trained on large-scale text data sets, pre-trained models can learn the structure of the language, the usage of vocabulary, and the relationship between different vocabulary. This enables them to generate high-quality text and perform various natural language processing tasks. Natural language processing tasks include, but are not limited to: generating coherent and meaningful new text based on a given prompt or context, answering questions raised by users, etc.

[0053] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0054] Figure 1 A flowchart of a method for generating text detection using a large language model provided in this application is shown in FIG. Figure 1 As shown, the method includes:

[0055] Step S101, obtaining a text to be detected, and obtaining keywords in the text to be detected.

[0056] The text to be tested can be text written by humans or text generated by any large language model.

[0057] Specifically, the text to be detected can be segmented to obtain multiple words, and then the frequency of each word appearing in the text to be detected is counted, and the word with the largest frequency of appearance is used as the keyword.

[0058] The text to be detected can also be input into a preset keyword extraction model to obtain keywords in the text to be detected. The keywords in the text to be detected can exist in the text to be detected, or can be synonyms of words existing in the text to be detected.

[0059] If there are multiple keywords, the multiple keywords are separated by separators, such as commas, colons, and semicolons.

[0060] Step S102: input the text to be detected into at least one candidate source model to obtain at least one candidate question.

[0061] Among them, the alternative source model is also a large language model, which is used to generate questions corresponding to the text to be detected based on the text to be detected.

[0062] The alternative source model can be one or more. If the alternative source model is one, the text to be detected can only be input into the alternative source model, so that the alternative source model generates questions corresponding to the text to be detected. However, different alternative source models can generate different questions based on the same text to be detected, and the text regeneration model can generate different regenerated texts based on different questions, and the similarities between different regenerated texts and the text to be detected may be different. Therefore, if the alternative source model is limited to one, it may not be possible to obtain multiple different regenerated texts, that is, it is impossible to perform a more comprehensive detection of the text to be detected, and the probability of missed detection is relatively high. However, if the alternative source model is limited to multiple, multiple different questions can be obtained. Further, the text regeneration model can generate multiple different regenerated texts based on multiple different texts, and then perform a more comprehensive detection of the text to be detected, reducing the probability of missed detection.

[0063] After the text to be detected is input into the candidate source model, one or more candidate questions can be obtained. That is, the candidate source model can generate one or more candidate questions based on the text to be detected. In this way, more comprehensive questions can be obtained to reduce omissions.

[0064] Step S103: input the keywords and the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords.

[0065] Specifically, the text regeneration model is also a large language model, which is used to generate regenerated text according to the alternative questions, that is, to answer the alternative questions.

[0066] In this way, even if the text regeneration model can generate multiple texts that are completely unrelated to the text to be detected based on the alternative questions, the text generated by the text regeneration model is restricted by keywords, so that there must be a connection between the regenerated text and the text to be detected. This can avoid the text regeneration model generating text that is completely unrelated to the text to be detected, reduce the possibility of missed detection, and achieve the effect of improving the accuracy of text detection.

[0067] Step S104, obtaining the similarity between the text to be detected and the regenerated text.

[0068] Specifically, the text to be detected and the regenerated text are respectively segmented to obtain a first segmentation set corresponding to the text to be detected and a second segmentation set corresponding to the regenerated text, and the first segmentation set and the second segmentation set are input into a preset feature vector extraction model to obtain a first feature vector corresponding to the first segmentation set and a second feature vector corresponding to the second segmentation set. The similarity between the first feature vector and the second feature vector is calculated according to the cosine similarity algorithm.

[0069] Among them, the feature vector extraction model is, for example, BERT (Bidirectional Encoder Representations from Transformers, a deep learning model based on the Transformer architecture).

[0070] In some embodiments, the cosine similarity between the first feature vector and the second feature vector is 0.91, and the preset threshold is 0.8. It can be seen that if the cosine similarity between the first feature vector and the second feature vector is greater than the preset threshold, it can be determined that the text to be detected is generated by a large language model.

[0071] Step S105, determining whether the similarity between the text to be detected and the regenerated text is greater than a preset threshold, if so, executing step S106, otherwise, executing step S107.

[0072] Step S106, determining whether the text to be detected is generated by a large language model.

[0073] Step S107, determining whether the text to be detected is manually written.

[0074] The method for generating text detection using a large language model provided by the disclosed embodiment obtains the text to be detected and the keywords in the text to be detected, then inputs the text to be detected into the alternative source model to obtain the alternative questions, and then inputs the keywords and the alternative questions into the text regeneration model, so that the regeneration model can generate a regenerated text including the keywords, and finally determines whether the text to be detected has been generated by the large language model based on the similarity between the regenerated text and the text to be detected. Since the regenerated text includes keywords, and the keywords are obtained from the text to be detected, there must be a correlation between the regenerated text and the text to be detected. In this way, the text range generated by the text regeneration model can be limited to avoid the text regeneration model from generating text that is completely unrelated to the text to be detected based on the alternative questions, thereby reducing the possibility of missed detection and achieving the effect of improving the accuracy of text detection.

[0075] In addition, if there are multiple alternative questions, it is necessary to input the multiple alternative questions into the text regeneration model in sequence to obtain the regenerated text corresponding to each alternative question. Such a process is relatively cumbersome, and it is impossible to generate a regenerated text corresponding to all the alternative questions. Based on this, the embodiment of the present application provides another method for generating text detection with a large language model, that is, inputting the text to be detected into the alternative source model, obtaining multiple alternative questions, using the question integration model to integrate the multiple alternative questions to obtain a target question, and then inputting the target question into the text regeneration model to obtain the regenerated text corresponding to the target question, thereby obtaining the regenerated text corresponding to all the alternative questions. In this way, by extracting features and integrating information from multiple alternative questions, a more accurate target question is obtained, so that a more accurate regenerated text can be generated based on a more accurate target question.

[0076] The problem integration model is obtained by the following method: a plurality of first sample problems and a plurality of second sample problems are input into a preset neural network model for training to obtain the problem integration model. The second sample problem is formed by splicing and integrating a plurality of first sample problems. The neural network model is, for example, a model based on LLaMA (Large Language Model MetaAI).

[0077] Specifically, the Lora (Low-Rank Adaptation) fine-tuning method can be used to fine-tune multiple first sample questions and multiple second sample questions to obtain a set of first sample questions with sample labels, wherein the sample label is the second sample question, and the first sample question set includes the first sample question corresponding to the second sample question, and the first sample question corresponding to the second sample question can be used to integrate to obtain the second sample question. Among them, the set of first sample questions with sample labels is, for example, (X; Q1, Q2, Q3), where X is the sample label, that is, the second sample question, Q1, Q2, Q3 are all first sample questions, and the first samples Q1, Q2, Q3 are integrated and spliced ​​to obtain the second sample question X. In this way, the problem integration model can be used to integrate multiple candidate questions and output more accurate target questions, thereby improving the robustness of the regenerated text.

[0078] Furthermore, after determining that the text to be detected is generated by a large language model, it is also necessary to determine the specific source of the text to be detected, so as to determine the accuracy and scope of application of the text to be detected from the specific source of the text to be detected. Based on this, the present application provides a method for determining the source of the text to be detected.

[0079] Combination Figure 2 As shown, the method for determining the source of the text to be detected may include the following steps:

[0080] Step S201, obtaining the similarity between each candidate question and the target question respectively.

[0081] Specifically, each candidate question and the target question can be input into a preset feature vector extraction model to obtain the third feature vector corresponding to each candidate question and the fourth feature vector corresponding to the target question. According to the cosine similarity algorithm, the similarity between each third feature vector and the fourth feature vector is calculated respectively. Thus, the similarity between each candidate question and the target question is obtained.

[0082] Since the target question is formed by integrating multiple alternative questions, the greater the similarity between the target question and the alternative questions, the greater the weight of the alternative questions with greater similarity in the process of integrating the target question. At the same time, the regenerated text is generated based on the target question. Therefore, obtaining the similarity between each alternative question and the target question helps to determine the weight of the alternative question in generating the regenerated text. The regenerated text generated based on the alternative question with greater weight has greater similarity with the regenerated text generated based on the target question. Further, it can be explained that the regenerated text generated based on the alternative question with greater weight has greater similarity with the text to be detected. Therefore, the possibility of the generation of the alternative question of the text to be detected can be determined. Further, by determining the source of the alternative question, the source of the text to be detected can be determined.

[0083] Step S202: determining the candidate question with the greatest similarity to the target question as the source question.

[0084] The candidate questions may be sorted in descending order according to the similarity between the target question and the candidate questions, and then the candidate question with the highest ranking may be determined as the source question.

[0085] Alternatively, the candidate questions may be sorted in ascending order according to the similarity between the target question and the candidate questions, and then the candidate question ranked last may be determined as the source question.

[0086] Step S203, determining that the candidate source model for generating the source question is the large language model for generating the text to be detected.

[0087] Since the source problem must originate from the alternative source model, after determining the source problem, the source of the source problem is determined first, that is, the alternative source model for generating the source problem is determined, and then the large language model for generating the text to be detected can be determined.

[0088] The method for determining the source of the text to be detected provided by the embodiment of the present application, by obtaining the similarity between the target question and each alternative question, then determining the alternative question with the greatest similarity to the target question as the source question, and finally using the alternative source model of the generated source question as the large language model of the text to be detected. In this way, the specific large language model of the generated text to be detected can be determined, so as to determine the authenticity and scope of application of the text to be detected based on the large language model of its source. At the same time, by detecting the similarity between the alternative question and the target question, the weight of each alternative question can be determined in the integration process of the target question. In addition, if the similarity between the text to be detected and the regenerated text is greater than the preset threshold, it means that the possibility of the text to be detected being generated based on the target question is also greater than the preset threshold. Thus, in the integration process of the target question, the greater the weight occupied by each alternative question, the greater the possibility of the text to be detected being generated based on the alternative question, thereby determining that the alternative source model of the generated source question is the large language model of the generated text to be detected.

[0089] In addition, inputting the text to be detected into at least one alternative source model to obtain alternative questions includes: constructing prompt words for generating alternative questions; inputting the prompt words and the text to be detected into at least one alternative source model to obtain alternative questions.

[0090] Among them, the prompt words can be used to limit the identity of the alternative source model when answering the text to be detected. By limiting the identity of the alternative source model, the alternative source model can use the professional knowledge of the limited identity to output alternative questions, that is, the alternative source model can be more focused on the knowledge field related to the limited identity, thereby reducing irrelevant information and improving the relevance and accuracy of the answer.

[0091] The prompt word can also be used to limit the number of alternative questions and the output format of the alternative source model. For example, the prompt word is: Please give the j most likely alternative questions based on the text to be detected, so that the answers to the alternative questions are as close as possible to the text to be detected. The output format of the alternative questions is 1, 2, 3... In this way, the alternative source model can be guided to generate more accurate alternative questions, so that more accurate regenerated text can be generated based on more accurate alternative questions, and then the text to be detected can be detected more accurately.

[0092] In some embodiments, keywords are, for example, large language model, artificial intelligence, understanding language, generating language, suitable, completely ignorant, non-professional, super large dictionary, smart robot, computer programming, data training; the target question is, for example, to explain the working mode of the large language model. After the keywords and the target question are input into the text, the regenerated text obtained is, for example, "Of course! Let me try to explain how the large language model works with some simple metaphors and keywords, especially for non-professionals who do not understand artificial intelligence at all. Large language model (Large Big Language Model is a very powerful tool that can understand and generate language. You can think of it as a super big dictionary, but not just any dictionary. Big language models are part of artificial intelligence. Artificial intelligence is about making computers "smart" and able to do things that usually require human intelligence, such as understanding and answering questions. Imagine that a big language model is like a smart robot. This robot has read thousands of books, articles, and web pages. By reading these contents, it has learned how to understand human language. It's like you learned how to read and write, except that it learns faster and more widely. Not only that, this robot can also generate new text based on what it has learned. When you ask it a question, it will find the most appropriate words and sentences from its super big dictionary and give you a meaningful answer. Meaningful answers. This smart robot is not born so smart. It became smart because it has been trained with a lot of data. During the data training process, scientists provided it with a large amount of text data, allowing it to practice understanding and generating language repeatedly. All of this is achieved through computer programming. Scientists have written complex programs that allow this robot to learn and improve itself. These programs are like the brain of the robot, guiding it on how to process and understand text information. If you don’t understand technology at all, you can think of the Big Language Model as a super smart language assistant. It has accumulated a wealth of knowledge by studying a large number of books and articles. Now, it can talk to you, answer your questions, and even help you write articles. I hope this explanation can help you better understand how the Big Language Model works! If you have more questions, feel free to let me know."

[0093] Combination Figure 3As shown, the present embodiment provides a device 300 for large language model generated text detection, including: a first acquisition module 301, a question acquisition module 302, a text acquisition module 303, a second acquisition module 304 and a determination module 305. The first acquisition module 301 is used to obtain the text to be detected and obtain the keywords in the text to be detected. The question acquisition module 302 is used to input the text to be detected into at least one alternative source model to obtain at least one alternative question. The text acquisition module 303 is used to input the keywords and each alternative question into a preset text regeneration model to obtain a regenerated text including the keywords. The second acquisition module 304 is used to obtain the similarity between the text to be detected and the regenerated text. The determination module 305 is used to determine that the text to be detected is generated by a large language model when the similarity between the text to be detected and the regenerated text is greater than a preset threshold.

[0094] In a possible implementation, the question acquisition module is used to construct prompt words for generating candidate questions; the prompt words and the text to be detected are input into at least one candidate source model to obtain the candidate questions.

[0095] In a possible implementation, the second acquisition module is used to perform word segmentation processing on the text to be detected and the regenerated text respectively, to obtain a first word segmentation set corresponding to the text to be detected, and a second word segmentation set corresponding to the regenerated text; the first word segmentation set and the second word segmentation set are input into a preset feature vector extraction model to obtain a first feature vector corresponding to the first word segmentation set, and a second feature vector corresponding to the second word segmentation set; according to the cosine similarity algorithm, the similarity between the first feature vector and the second feature vector is calculated.

[0096] In a possible implementation, if there are multiple candidate questions, the text acquisition module is used to integrate the multiple candidate questions to obtain the target question; input the keywords and the target question into the text regeneration model to obtain the regenerated text including the keywords.

[0097] In a possible implementation, the text acquisition module is used to input multiple candidate questions into a preset question integration model to obtain a target question.

[0098] In a possible implementation, the determination module is also used to determine the similarity between the target question and each alternative question; determine that the alternative source model corresponding to the source question is a large language model that generates the text to be detected; wherein the source question is the alternative question with the greatest similarity to the target question.

[0099] In a possible implementation, if the similarity between the text to be detected and the regenerated text is less than or equal to a preset threshold, the determination module is further used to determine that the text to be detected is manually written.

[0100] The device for large language model generated text detection provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0101] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 4 As shown, the electronic device 400 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the device 400 also includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus.

[0102] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that at least one processor 401 executes the above method.

[0103] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0104] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0105] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0106] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0107] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0108] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0109] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0110] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0111] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0114] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0116] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for detecting text generated by a large language model, characterized in that: include: Obtaining a text to be detected, and obtaining keywords in the text to be detected; Inputting the text to be detected into at least one candidate source model to obtain at least one candidate question; Inputting the keywords and each of the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords; Obtaining the similarity between the text to be detected and the regenerated text; If the similarity between the text to be detected and the regenerated text is greater than a preset threshold, it is determined that the text to be detected is generated by a large language model.

2. The method according to claim 1, characterized in that Inputting the to-be-detected text into at least one candidate source model to obtain candidate questions includes: constructing prompt words for generating the alternative questions; The prompt word and the text to be detected are input into at least one candidate source model to obtain an alternative question.

3. The method according to claim 1, characterized in that Obtaining the similarity between the to-be-detected text and the regenerated text includes: Performing word segmentation processing on the text to be detected and the regenerated text respectively to obtain a first word segmentation set corresponding to the text to be detected and a second word segmentation set corresponding to the regenerated text; Inputting the first word segmentation set and the second word segmentation set into a preset feature vector extraction model to obtain a first feature vector corresponding to the first word segmentation set and a second feature vector corresponding to the second word segmentation set; The similarity between the first feature vector and the second feature vector is calculated according to a cosine similarity algorithm.

4. The method according to claim 1, characterized in that: Inputting the keywords and each of the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords, including: If there are multiple candidate questions, the multiple candidate questions are integrated to obtain a target question; The keywords and the target question are input into the text regeneration model to obtain a regenerated text including the keywords.

5. The method according to claim 4, characterized in that Integrate the multiple candidate questions to obtain the target question, including: The plurality of candidate questions are input into a preset question integration model to obtain the target question.

6. The method according to claim 4, characterized in that After determining that the text to be detected is generated by a large language model, the method further includes: Determining the similarity between the target question and each of the candidate questions; The candidate source model corresponding to the source question is determined to be the large language model that generates the text to be detected; wherein the source question is the candidate question with the greatest similarity to the target question.

7. The method according to claim 1, characterized in that After obtaining the similarity between the to-be-detected text and the regenerated text, the method further includes: If the similarity between the text to be detected and the regenerated text is less than or equal to a preset threshold, it is determined that the text to be detected is manually written.

8. A device for detecting text generated by a large language model, characterized in that: include: A first acquisition module is used to acquire a text to be detected and acquire keywords in the text to be detected; A question acquisition module, used for inputting the text to be detected into at least one candidate source model to obtain at least one candidate question; A text acquisition module, used for inputting the keywords and each of the candidate questions into a preset text regeneration model to obtain a regenerated text including the keywords; A second acquisition module, used to acquire the similarity between the to-be-detected text and the regenerated text; The determination module is used to determine that the text to be detected is generated by a large language model when the similarity between the text to be detected and the regenerated text is greater than a preset threshold.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Citation Information

Patent Citations

  • Literature retrieval method, device and system and storage medium

    CN116431838A

  • Product description generation method and system based on LLM and text similarity analysis

    CN118656548A