Detection method, device, equipment, medium and product

Abstract

The invention discloses a detection method and device, equipment, a medium and a product, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining target to-be-detected content; detecting the target to-be-detected content based on a preset plagiarism detector and / or a text detector, and outputting a statistical index and / or a content source corresponding to the target to-be-detected content; according to the statistical indexes and / or the content source, large language model generation content in the target content to be detected is determined, and the statistical indexes comprise at least one of a similarity measurement index, a syntax tree detection index, an editing distance index, a text style statistical index, a linguistic feature statistical index and a fact detection index; the method for detecting the large language model generated text is simple, convenient and accurate, the large language model generated content can be rapidly determined through the detected statistical indexes and / or content sources, and the detection effect of the large language model generated content is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to detection methods, devices, equipment, media and products. Background Art

[0002] With the widespread application of large language model generation, platforms that emphasize originality, including academic, publishing, and online content sharing platforms, face the challenge of distinguishing text generated by large language models from text created by humans. Large language models generate content very quickly and are easy to repeat. If a large number of such texts appear on the platform in a short period of time, it may lead to a decline in the quality of platform content and increase the risk of user loss.

[0003] Currently, common methods for distinguishing text generated by large language models include using large language models for distinction and manual review. However, in the face of massive content updates, using the above two methods on all content requires high resources and is relatively inefficient. In addition, for texts with a large number of references, the detection methods for phrase characteristics and grammatical structures of text generated by large language models have poor distinction effects because the pure text generated is small.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a detection method, device, equipment, medium and product, aiming to provide a simple and accurate method for detecting text generated by a large language model.

[0006] To achieve the above object, the present application provides a detection method, which comprises:

[0007] Obtain the target content to be detected;

[0008] Detecting the target content to be detected based on a preset plagiarism detector and / or text detector, and outputting statistical indicators and / or content sources corresponding to the target content to be detected;

[0009] Determine the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

[0010] In one embodiment, the step of detecting the target content to be detected based on a preset plagiarism detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected includes:

[0011] Inputting the target content to be detected into the plagiarism detector;

[0012] The plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database, and outputs at least one of the similarity measurement index, syntax tree detection index and edit distance index; and / or, the plagiarism detector performs source marking on the target content to be detected, and outputs the content source.

[0013] In one embodiment, the step of detecting the target content to be detected based on a preset text detector and outputting statistical indicators corresponding to the target content to be detected includes:

[0014] Inputting the target content to be detected into the text detector;

[0015] The text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, and outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index.

[0016] In one embodiment, the step of performing syntax tree detection on the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the syntax tree detection index includes:

[0017] Parsing the target content to be detected and constructing a corresponding target syntax tree, and parsing the reference text and constructing a corresponding reference syntax tree;

[0018] Extracting syntactic features from the target syntax tree and the reference syntax tree respectively;

[0019] Calculating feature similarity based on the syntactic features, and calculating structural similarity between the target syntax tree and the reference syntax tree;

[0020] The feature similarity and the structure similarity are weightedly calculated to obtain the syntax tree detection index.

[0021] In one embodiment, the step of calculating the edit distance between the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the edit distance index includes:

[0022] Determining the text length of the target content to be detected and the reference text;

[0023] Creating an initial matrix based on the length of the text;

[0024] Traversing each position in the initial matrix, and calculating the value corresponding to each position according to the cost of performing a replacement operation, an insertion operation, and a deletion operation on each position;

[0025] The edit distance index is determined according to the values ​​corresponding to the respective positions.

[0026] In one embodiment, the step of determining the large language model generated content in the target content to be detected according to the statistical indicators and / or the content source includes at least one of the following:

[0027] Performing weighted calculation according to at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index to obtain a weighted sum of the indicators, and determining the content whose weighted sum of the indicators exceeds a preset sum threshold as content generated by the large language model;

[0028] Inputting at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index into a pre-trained binary classification model, and determining the large language model generated content in the target content to be detected according to the prediction result of the binary classification model;

[0029] The source overlap ratio is determined according to the content sources, and the content whose source overlap ratio exceeds a preset overlap threshold is determined to be content generated by a large language model.

[0030] In one embodiment, the step of determining the source overlap ratio according to the content source and determining the content whose source overlap ratio exceeds a preset overlap threshold as the content generated by the large language model includes:

[0031] According to the content source, determine the total number of content sources and the number of content sources belonging to the large language model database;

[0032] Calculate the ratio of the number of items belonging to the large language model database to the total number of items of the content source to obtain the source overlap ratio;

[0033] Determining whether the source overlap ratio exceeds the preset overlap threshold;

[0034] If the source overlap ratio exceeds the preset overlap threshold, the target content to be detected is determined to be content generated by a large language model.

[0035] In one embodiment, before the step of detecting the target content to be detected based on a preset plagiarism detector and / or text detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected, the step further includes:

[0036] The plagiarism detector and / or text detector is obtained by training, specifically comprising:

[0037] Get a sample dataset;

[0038] Preprocessing and marking the sample data in the sample data set to obtain a processed sample data set;

[0039] The first model framework is trained according to the processed sample data set to obtain the plagiarism detector, and / or the second model framework is trained according to the processed sample data set to obtain the text detector.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a detection device, which includes:

[0041] An acquisition module is used to acquire target content to be detected;

[0042] A detection module, used to detect the target content to be detected based on a preset plagiarism detector and / or text detector, and output statistical indicators and / or content sources corresponding to the target content to be detected;

[0043] A determination module is used to determine the large language model generated content in the target content to be detected based on the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

[0044] In addition, to achieve the above objectives, the present application also proposes a detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the detection method described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the detection method described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the detection method described above are implemented.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] By acquiring target content to be detected; detecting the target content to be detected based on a preset plagiarism detector and / or text detector, and outputting statistical indicators and / or content sources corresponding to the target content to be detected; determining the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator, a method for detecting text generated by a large language model is provided in a simple and accurate manner, and the obtained target content to be detected is detected by a plagiarism detector and / or a text detector. Through detection, the statistical indicators and / or content sources corresponding to the target content to be detected can be determined, wherein the similarity measurement indicators, syntax tree detection indicators, edit distance indicators, text style statistical indicators, linguistic feature statistical indicators and fact detection indicators corresponding to the target content to be detected can reflect various aspects of the target content to be detected, and are used to quickly determine the large language model generated content in the target content to be detected; the content source corresponding to the target content to be detected can track the source information of the target content to be detected, and can also quickly determine the large language model generated content in the target content to be detected. Compared with manual review, the detection efficiency can be greatly improved, and the detection effect of the large language model generated content is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 This is a schematic diagram of the process of the first embodiment of the detection method of the present application;

[0052] Figure 2 This is a schematic diagram of the flow chart of the second embodiment of the detection method of the present application;

[0053] Figure 3 This is a schematic diagram of the process of the third embodiment of the detection method of the present application;

[0054] Figure 4 This is a schematic diagram of the flow chart of the fourth embodiment of the detection method of the present application;

[0055] Figure 5 This is a schematic diagram of the process of the fifth embodiment of the detection method of the present application;

[0056] Figure 6 This is a flow chart of the sixth embodiment of the detection method of the present application;

[0057] Figure 7 is a schematic diagram of a first detection process according to a sixth embodiment;

[0058] Figure 8 is a schematic diagram of a second detection process according to the sixth embodiment;

[0059] Fig. 9 Schematic diagram of a detection device in an embodiment of the present application;

[0060] Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the detection method in the embodiment of the present application.

[0061] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0063] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0064] The main solution of the embodiment of the present application is: by acquiring the target content to be detected; based on a preset plagiarism detector and / or text detector, detecting the target content to be detected, and outputting the statistical indicators and / or content sources corresponding to the target content to be detected; determining the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator, providing a simple and accurate method for detecting large language model generated text, by using a plagiarism detector and / or a text detector to detect the acquired target content. By detecting the target content to be detected, the statistical indicators and / or content sources corresponding to the target content to be detected can be determined, wherein the similarity measurement indicators, syntax tree detection indicators, edit distance indicators, text style statistical indicators, linguistic feature statistical indicators and fact detection indicators corresponding to the target content to be detected can reflect various aspects of the target content to be detected, and are used to quickly determine the large language model generated content in the target content to be detected; the content source corresponding to the target content to be detected can track the source information of the target content to be detected, and can also quickly determine the large language model generated content in the target content to be detected. Compared with manual review, the detection efficiency can be greatly improved, and the detection effect of the large language model generated content is improved.

[0065] In this embodiment, for the convenience of description, the following description is made with the detection device as the execution subject.

[0066] With the widespread application of Large Language Model (LLM) generation, platforms that emphasize originality, including academic, publishing, and online content sharing platforms, face the challenge of distinguishing texts generated by large language models from texts created by humans. Large language models generate content very quickly and are easy to repeat. If a large number of such texts emerge on the platform in a short period of time, it may lead to a decline in the quality of platform content and increase the risk of user loss. At present, common methods for distinguishing texts generated by large language models include using large language models for distinction and manual review. However, in the face of massive content updates, the use of the above two methods for all content has high resource requirements and relatively low efficiency. In addition, the filtering method for the phrase characteristics and grammatical structure of texts generated by large language models has poor distinction effects on texts with a large amount of reference materials because the pure text generation part is small. Therefore, a simple method that can quickly screen and is convenient for combining more advanced filters is needed.

[0067] The present application provides a solution, which uses the trained plagiarism detector for the detection of content generated by a large language model. When generating content, the existing large language model tends to splice or directly use the content of the training corpus, especially when the question involves the requirement of factual statement. The large model requires accuracy due to the prompt, and the original data content or paraphrase will be retained in the output. The training of the plagiarism detector is based on indicators including similarity measurement, syntax tree detection, edit distance, etc. Because the large language model often generates content by paraphrasing the training corpus and adding conjunctions, changing the word order and other confusing techniques, these changed texts are similar to plagiarized texts and are prone to abnormal performance in the above indicators, that is, the text generated by the large language model has a probability of being classified as suspicious by the plagiarism detector, so the plagiarism detector is naturally suitable for the detection of such large language model generated content. When performing detection, other detection methods can be combined, such as the aforementioned statistical indicator detection, linguistic detection and other models, and the plagiarism detector can be used to screen the content: if it is detected that the density of plagiarized paragraphs exceeds a certain range of the specified threshold, and the scores of the text in a specific number of other detection models exceed the suspicious threshold, it will be screened out.

[0068] In addition to the above solutions, since most plagiarism detectors have the ability to trace the sources of similar paragraphs, the public training data commonly used by existing large language models (including online encyclopedias, forum records, etc.) can be annotated and the labeled training data can be used to train plagiarism detectors. If the source of similar text in a given text appears in public training data too many times, the given text can be marked as highly suspicious.

[0069] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device and a detection device capable of realizing the above functions. The following takes the detection device as an example to illustrate this embodiment and the following embodiments.

[0070] First embodiment

[0071] The present application embodiment provides a detection method, referring to Figure 1 , Figure 1 This is a schematic diagram of the flow chart of the first embodiment of the detection method of the present application.

[0072] In this embodiment, the detection method includes steps S10 to S30:

[0073] Step S10, obtaining target content to be detected;

[0074] Specifically, a large language model (LLM) can generate various types of content, covering a wide range of text creation and language processing tasks. The content it generates may include text content, image content, audio and video content, etc. With the widespread application of large language models, it is urgent to distinguish between the content generated by large language models and the content created by humans. The embodiments of the present application are mainly explained by taking the detection of text content generated by a large language model as an example.

[0075] Exemplarily, the detection method proposed in the embodiments of the present application can be applied to platforms that emphasize originality, including academic, publishing, and online content sharing platforms, to detect a large amount of text content in the platform, and then screen and filter the text content based on the detection results. For example, content generated by a large language model can be used as screened text, and content generated by non-large language models can be used as screened text, so as to improve the quality of content created by the platform and reduce the risk of user loss.

[0076] Exemplarily, in an embodiment of the present application, the collected content to be detected or to be filtered is preprocessed to obtain the target content to be detected, so as to facilitate further rapid detection of the target content to be detected. Optionally, the method of preprocessing the content to be detected or to be filtered may include unified text format, standardized processing, feature extraction, and vectorized processing, so as to reduce noise, unify the format, and extract meaningful features, and convert the content to be detected or to be filtered into a form that is easy for the machine learning model to understand.

[0077] Step S20, detecting the target content to be detected based on a preset plagiarism detector and / or text detector, and outputting statistical indicators and / or content sources corresponding to the target content to be detected;

[0078] Furthermore, after the target content to be detected is obtained, the target content to be detected can be detected based on a preset plagiarism detector and / or text detector, and statistical indicators and / or content sources corresponding to the target content to be detected can be output.

[0079] Exemplarily, when generating content, the large language model tends to splice or directly use the content of the training corpus, especially when the question involves the requirement of factual statement. The large language model will retain the original data content or paraphrase in the output because of the prompt requirement for accuracy. Therefore, in the embodiment of the present application, at least one of the similarity measurement index, the syntax tree detection index and the edit distance index is considered as the first statistical index, and at least one of the text style statistical index, the linguistic feature statistical index and the fact detection index is used as the second statistical index. By pre-training with the first statistical index, a plagiarism detector can be obtained. Similarly, by pre-training with the second statistical index, a text detector can be obtained. The target content to be detected can be detected by the plagiarism detector and / or the text detector, and the statistical indicators corresponding to the target content to be detected can be output. Because the large language model generates texts that often use paraphrases of the training corpus and add conjunctions, change word order and other confusing techniques, these modified texts are similar to plagiarized texts and are prone to abnormal performance in the above statistical indicators, thereby improving the accuracy of the detection results.

[0080] Exemplarily, in the embodiments of the present application, it is also considered to mark the commonly used public training data of the existing large language model (such as online encyclopedias, forum records, etc.) when training the plagiarism detector. In the process of detecting the target content to be detected, the plagiarism detector is used to trace the sources of similar paragraphs, and then the large language model-generated content in the target content to be detected is determined based on the proportion of the content sources appearing in the large language model training data in the total content sources.

[0081] Step S30, determining the large language model generated content in the target content to be detected based on the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

[0082] Furthermore, after detecting the target content to be detected and outputting the statistical indicators and / or content sources corresponding to the target content to be detected, the large language model generated content in the target content to be detected can be determined based on the statistical indicators and / or content sources.

[0083] As one of the implementation modes, in an embodiment of the present application, each statistical indicator can be weightedly calculated to obtain a weighted sum of the indicators of each statistical indicator, which is then compared with a preset sum threshold. Content whose weighted sum exceeds the preset sum threshold is determined to be content generated by a large language model, and content whose weighted sum exceeds the preset sum threshold is determined to be content not generated by a large language model.

[0084] It should be noted that, in the process of weighted calculation, the weight of each statistical indicator can be set according to the actual situation, and this is not specifically limited in the embodiments of the present application. Similarly, the preset sum threshold can also be set or adjusted according to actual needs, and this is not specifically limited in the embodiments of the present application. For example, when there is too much text content emerging in the platform in a short period of time, the preset sum threshold can be lowered to provide a more stringent detection standard for the detection of text content.

[0085] As another implementation method, in an embodiment of the present application, a binary classification model can also be pre-trained, and the statistical indicators output by the plagiarism detector and / or the text detector can be input into the binary classification model for prediction to determine the large language model generated content in the target content to be detected.

[0086] Exemplarily, for the content sources output by the plagiarism detector, the source overlap ratio can be determined based on the proportion of content sources appearing in the large language model training data in the total content sources, and the source overlap ratio can be compared with a preset overlap threshold. Content whose source overlap ratio exceeds the preset overlap threshold is determined to be content generated by the large language model.

[0087] It should be noted that in the process of comparing the source overlap ratio with the preset overlap threshold, the preset overlap threshold can be set or adjusted according to actual needs. There is no specific limitation on this in the embodiment of the present application. For example, when there is too much text content emerging in the platform in a short period of time, the preset overlap threshold can be lowered to provide a more stringent detection standard for the detection of text content.

[0088] This embodiment adopts the above scheme, specifically by acquiring the target content to be detected; detecting the target content to be detected based on a preset plagiarism detector and / or text detector, and outputting the statistical indicators and / or content sources corresponding to the target content to be detected; determining the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator, thereby providing a simple and accurate method for detecting large language model generated text, wherein the plagiarism detector and / or text detector are used to detect the acquired target content. By testing the content to be detected, the statistical indicators and / or content sources corresponding to the target content to be detected can be determined, wherein the similarity measurement indicators, syntax tree detection indicators, edit distance indicators, text style statistical indicators, linguistic feature statistical indicators and fact detection indicators corresponding to the target content to be detected can reflect various aspects of the target content to be detected, and are used to quickly determine the large language model generated content in the target content to be detected; the content source corresponding to the target content to be detected can track the source information of the target content to be detected, and can also quickly determine the large language model generated content in the target content to be detected. Compared with manual review, the detection efficiency can be greatly improved, and the detection effect of the large language model generated content is improved.

[0089] Second embodiment

[0090] Based on the first embodiment of the present application, a solution of the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S10 specifically includes steps S101 to S102:

[0091] Step S101: obtaining original content to be detected;

[0092] Step S102: pre-processing the original content to be detected to obtain the target content to be detected.

[0093] Specifically, the detection method proposed in the embodiments of the present application can be applied to platforms that emphasize originality, including academic, publishing, and online content sharing platforms, and can be used to detect a large amount of text content in the platform, and then screen and filter the text content based on the detection results. For example, content generated by a large language model can be used as screened text, and content generated by non-large language models can be used as screened text, so as to improve the quality of content created by the platform and reduce the risk of user loss.

[0094] Exemplarily, in the embodiment of the present application, the collected original content to be detected is preprocessed to obtain the target content to be detected, so as to facilitate further rapid detection of the target content to be detected.

[0095] Exemplarily, methods for preprocessing the content to be detected or filtered may include methods such as unified text format, standardization processing, feature extraction, and vectorization processing, so as to reduce noise, unify the format, and extract meaningful features, and convert the content to be detected or filtered into a form that is easy for the machine learning model to understand.

[0096] For example, through tokenization, the text can be split into smaller units, such as words, phrases or sentences. This can be done by spaces, punctuation marks or specific vocabulary rules.

[0097] Exemplarily, the cleaning process can remove irrelevant characters in the text, such as punctuation marks, numbers, special symbols, etc., and convert the text to lowercase (or uppercase) to ensure consistency.

[0098] Exemplary processing includes stemming or lemmatization, where stemming is reducing words to their basic form, usually by removing suffixes; lemmatization is converting words to their lexicon form (lemma), which usually involves part of speech and context analysis.

[0099] Exemplarily, the text is converted into a numerical vector through vectorization processing so that the machine learning model can process it. Possible vectorization methods include at least one of the following:

[0100] One-Hot Encoding: Create a vector for each word, where only one position is 1, indicating that the word appears, and all other positions are 0;

[0101] Bag of Words: Create a vector for each document, where each position represents a word and the value of that position represents the number of times the word appears in the document.

[0102] TF-IDF (Term Frequency-Inverse Document Frequency): used to evaluate the importance of a word to a document set or a document in a corpus;

[0103] Word Embeddings: Convert words into fixed-size dense vectors that capture the semantic and contextual information of the words. Common word embedding techniques include Word2Vec, GloVe, and FastText.

[0104] For example, if the model needs to understand the order of words (such as a recurrent neural network or a Transformer model), a sequence vectorization method can be used, such as stacking the embedding vectors of words or characters in sequence.

[0105] This embodiment adopts the above scheme, specifically by obtaining the original content to be detected; preprocessing the original content to be detected to obtain the target content to be detected, and by adopting one or more preprocessing methods, the original content to be detected can be converted into target content to be detected that is easy to detect, which can improve the accuracy of the detection process and / or improve the detection efficiency.

[0106] Third embodiment

[0107] Based on any of the above embodiments of the present application, a third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as any of the above embodiments can be referred to the above introduction and will not be repeated later. Figure 3 , the detection method in the embodiment of the present application also includes:

[0108] Step S00: training to obtain the plagiarism detector and / or text detector.

[0109] Optionally, step S00 may be implemented before step S10, or between step S10 and step S20, and the order of implementation is not specifically limited in the embodiments of the present application.

[0110] Exemplarily, step S00 specifically includes steps S01 to S03:

[0111] Step S01, obtaining a sample data set;

[0112] Exemplarily, in the process of training the plagiarism detector and / or text detector, sample data must be collected first. In the embodiment of the present application, by collecting a variety of text data sets, including texts written by humans (such as books, papers, news articles) and texts generated by known large language models, it is ensured that the sample data sets cover various fields, styles and complexities to improve the generalization ability of the plagiarism detector and / or text detector.

[0113] Step S02, preprocessing and marking the sample data in the sample data set to obtain a processed sample data set;

[0114] Exemplarily, after obtaining the sample data set, the sample data may be preprocessed. The relevant preprocessing method may refer to the second embodiment, which will not be described in detail in this embodiment. After preprocessing the sample data set, extracted text features are obtained. These features may include vocabulary diversity, syntactic structure, text statistical properties (such as sentence length distribution), specific language patterns (such as templated expressions), etc., which are often different from human creations in machine-generated texts. Since large language models may be optimized to mimic human language, it becomes particularly important to find subtle features in model-generated texts, such as repeated patterns, unnatural transitions, specific types of errors, or patterned expressions.

[0115] Step S03: training a first model framework according to the processed sample data set to obtain the plagiarism detector, and / or training a second model framework according to the processed sample data set to obtain the text detector.

[0116] Exemplarily, the sample data in the sample data set is preprocessed and labeled, and after obtaining the processed sample data set, a suitable machine learning or deep learning model can be selected as the model architecture, such as a model of the Transformer architecture, and the labeled data set is used to train the model so that it learns to distinguish between human text and text generated by a large language model. For plagiarism detectors, the focus is on training through a first statistical indicator including at least one of a similarity metric, a syntax tree detection indicator, and an edit distance indicator; for text detectors, the focus is on training through a second statistical indicator including at least one of a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

[0117] For example, in the process of similarity measurement, text vectorization can be performed first to convert the text into a numerical vector, and then the cosine angle of two text vectors can be calculated, and the ratio of the intersection and union of two text sets can be calculated, or a pre-trained language model (such as BERT, Word2Vec) can be used to convert the text into a vector representation of words or sentences, and the similarity between these vectors can be calculated using cosine similarity or other measurement methods.

[0118] Exemplarily, by measuring the similarity of texts, the similarity of two texts in terms of content or semantics can be determined. Common methods include cosine similarity, Jaccard similarity coefficient, and semantic-based similarity. Optionally, cosine similarity is to convert texts into vectors (for example, using a bag-of-words model, TF-IDF), and then calculate the cosine angle between the two vectors to determine the similarity of the texts; Jaccard similarity coefficient is to measure their similarity by calculating the ratio of the common vocabulary of two texts to their respective total vocabulary; semantic-based similarity is to use pre-trained language models (such as BERT, Word2Vec) to obtain deep semantic representations of words or sentences and calculate the similarity between them.

[0119] For example, in the process of syntax tree detection, the text needs to be segmented into sentences first to prepare for the construction of the syntax tree, and then the syntax analysis tools or libraries (such as Stanford Parser, SpaCy) are used to build the syntax tree of each sentence. The syntax tree can be built using predefined grammar rules, or statistical methods such as probabilistic context-free grammars (PCFGs) or conditional random fields (CRFs) are used. Models such as recursive neural networks (RNNs), long short-term memory networks (LSTMs) or Transformers are used to automatically learn the grammatical structure of sentences, and then analyze the structure of the syntax tree and identify the dependencies between nodes.

[0120] Exemplarily, syntax tree detection involves analyzing the sentence structure in a text to determine whether it conforms to given grammatical rules. This analysis is usually done by building a syntax tree (also called a parse tree or a syntactic tree), which is a graphical representation that shows the hierarchy and dependencies between the components of a sentence. A syntax tree is a tree-like structure in which each node represents an element in a sentence, such as a word or phrase. The root node of the tree usually represents the entire sentence, while the leaf nodes represent the basic elements in the sentence, usually words. Non-leaf nodes represent grammatical structures, such as noun phrases, verb phrases, etc. The lines between nodes represent grammatical relationships, such as modification, coordination, subordination, etc. Methods for syntax tree detection include rule-based analysis, statistical parsing, and deep learning methods. Rule-based analysis uses predefined grammatical rules to construct a syntax tree. This method requires detailed grammatical knowledge and complex rules to handle various grammatical structures; statistical parsing uses statistical-based methods, such as probabilistic context-free grammars (PCFGs) or conditional random fields (CRFs), to learn grammatical rules from a large amount of text, and then use them to parse new sentences; deep learning methods use deep neural networks, especially recursive neural networks (RNNs), long short-term memory networks (LSTMs), or transformer models to automatically learn the grammatical structure of sentences.

[0121] For example, in the process of calculating the edit distance, the edit distance matrix is ​​initialized first, and a matrix of size (m+1)×(n+1)(m+1)×(n+1) is created, where mm and nn are the lengths of the two strings respectively. Then the matrix is ​​filled, and the first row and the first column of the matrix are filled with values ​​from 0 to n and 0 to m. To further calculate the edit distance, traverse each character of the two strings, and for each position i, j, calculate the edit distance d[i][j]:

[0122] If the characters are the same: d[i][j] = d[i-1][j-1];

[0123] If the characters are different: d[i][j] = min(d[i-1][j],d[i][j-1],d[i-1][j-1])+1.

[0124] The last element of the matrix, d[m][n], is the edit distance between the two strings.

[0125] For example, in the process of text style statistics, the word frequency distribution in the text is first counted, the use of common and rare words is identified, and vocabulary diversity indicators such as the Herfindahl index are calculated, and then the sentence length distribution is analyzed, including the proportion of short and long sentences, and complex sentence structures such as nested clauses and the frequency of use of passive voice are identified. The rhythm of the sentences is measured, for example, by analyzing pauses and the fluency of sentences. The readability of the text is evaluated, using indicators such as the Flesch-Kincaid readability test. Check whether the style in the text is consistent, and whether there are sudden style changes or inconsistent expressions.

[0126] For example, in the process of linguistic feature statistics, lexical feature analysis, language structure analysis, sentiment analysis, and pragmatic feature analysis can be performed. Specifically, the lexical features in the text are identified, such as the use of professional terms, the selection of polysemous words, etc., and the distribution of parts of speech, such as the frequency of use of nouns, verbs, and adjectives, are analyzed. Syntactic analysis tools are used to identify the grammatical structure of sentences and analyze the syntactic complexity in the text, such as the depth and diversity of clauses. Sentiment analysis tools are applied to identify the emotional tendencies in the text, such as positive, negative, or neutral, and to evaluate the consistency of emotional intensity and emotional distribution. Advanced language phenomena such as metaphors and irony in the text are analyzed to evaluate the persuasiveness and purposefulness of the language.

[0127] Exemplarily, in the process of fact checking, statements or assertions can be first extracted from the text as the objects of fact checking, and an existing fact database can be constructed or accessed to verify the statements in the text. The statements in the text can be compared with the information in the fact database to evaluate the credibility of the statements and identify possible errors or false information. The context of the statements should also be considered to evaluate their correctness in specific situations. External verification tools or APIs, such as Wikipedia and Wikidata, can be used to perform fact verification. Based on the verification results, a fact checking report can be generated to point out the non-factual content in the text.

[0128] For example, statistical indicator detection is often used for long document types, because large language model training often aims at a correct and standardized output format, and the style of its output text is relatively simple, with high predictability between words and sentences, and it has lower complexity (perplexity) than human output text; while human authors have diverse expression styles and higher perplexity. Other similar detection indicators include TF-IDF and Zipfian. This type of detection method has a certain detection effect on the content generated by the large language model, but the judgment effect on the situation of directly using human text in the training data is limited. Therefore, it is necessary to combine the aforementioned similarity measurement indicators, syntax tree detection indicators, edit distance indicators and content sources for detection, and the results will be more accurate. Linguistic feature detection focuses on distinguishing large language model text from life text based on vocabulary features, language structure, emotional meaning and other aspects. Because traditional large language model-generated content tends to ensure the integrity of language structure and remain emotionally neutral, this detection method has a certain effect on such large language model-generated text. In addition, one of the common problems currently faced by large language models is hallucination, that is, although the grammatical structure and language logic of the generated content are correct, it does not have factual correctness. Fact checking filters generated text for non-facts.

[0129] Exemplarily, these methods can be selected and adjusted according to specific needs and data conditions in practical applications. In the embodiment of the present application, similarity measurement indicators, syntax tree detection indicators and edit distance indicators are used to train a plagiarism detector; text style statistical indicators, linguistic feature statistical indicators and fact detection indicators are used to train a text detector, and the plagiarism detector is combined with the text detector to improve the accuracy of the detection result.

[0130] Exemplarily, in an embodiment of the present application, transfer learning can be applied, using a pre-trained model (such as BERT, RoBERTa, or T5) as a basis, and then fine-tuning it on a specific task, which can accelerate the training process and improve performance.

[0131] This embodiment adopts the above scheme, specifically by adopting similarity measurement indicators, syntax tree detection indicators and edit distance indicators to train a plagiarism detector; adopting text style statistical indicators, linguistic feature statistical indicators and fact detection indicators to train a text detector, and combining the plagiarism detector with the text detector to improve the accuracy of the detection process and / or improve the detection efficiency.

[0132] Fourth embodiment

[0133] Based on any of the above embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as any of the above embodiments can be referred to the above introduction and will not be repeated in the following. Figure 4 , step S20 may include steps S201 to S203:

[0134] Step S201: inputting the target content to be detected into the plagiarism detector;

[0135] Step S202: performing similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database through the plagiarism detector, and outputting at least one of the similarity measurement index, syntax tree detection index and edit distance index;

[0136] Step S203: marking the source of the target content to be detected by the plagiarism detector and outputting the source of the content.

[0137] Exemplarily, in an embodiment of the present application, before inputting the target content to be detected into a preset plagiarism detector, it is also necessary to prepare a reference database of the plagiarism detector, which includes a large number of texts with similar styles or on the same platform as the text to be detected, for comparison with the target detection content.

[0138] Exemplarily, the target content to be detected is input into a plagiarism detector, and the plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected, and then outputs a first statistical indicator including at least one of a similarity measurement indicator, a syntax tree detection indicator and an edit distance indicator.

[0139] Exemplarily, the target content to be detected is input into a plagiarism detector, and the plagiarism detector marks the paragraphs of the target content to be detected that are similar to the content in the reference database, traces the sources of the similar paragraphs, and then outputs the source of the content.

[0140] The detection method in the embodiment of the present application also includes:

[0141] Inputting the target content to be detected into a preset text detector;

[0142] The text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, and outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index.

[0143] Exemplarily, in an embodiment of the present application, before the target content to be detected is input into a preset text detector, a reference database of the text detector needs to be prepared. In a solution in which the text detector is used in combination with a plagiarism detector, the same reference database can be used, which includes a large number of texts of similar style or on the same platform as the text to be detected, for comparison with the target detection content.

[0144] Exemplarily, the target content to be detected is input into a text detector, and the text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and a reference text, and then outputs a second statistical indicator including at least one of a text style statistical indicator, a linguistic feature statistical indicator and a fact detection indicator.

[0145] Through the above scheme, this embodiment provides a simple and accurate method for detecting text generated by a large language model by inputting the target content to be detected into the plagiarism detector and outputting at least one of the similarity measurement index, the syntax tree detection index and the edit distance index; inputting the target content to be detected into the plagiarism detector and outputting the content source; inputting the target content to be detected into the text detector and outputting at least one of the text style statistical index, the linguistic feature statistical index and the fact detection index. By detecting the acquired target content to be detected, the statistical index and / or content source corresponding to the target content to be detected can be determined, wherein the statistical index corresponding to the target content to be detected can reflect various aspects of the target content to be detected, and is used to quickly determine the large language model generated content in the target content to be detected; the content source corresponding to the target content to be detected can track the source information of the target content to be detected, and can also quickly determine the large language model generated content in the target content to be detected. Compared with manual review, the detection efficiency can be greatly improved, and the detection effect of the large language model generated content is improved.

[0146] Fifth embodiment

[0147] Based on any of the above embodiments of the present application, a fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as any of the above embodiments can be referred to the above introduction, and will not be repeated in the following. Figure 5 , step S202 may include steps S2021 to S2022:

[0148] Step S2021: Performing syntax tree detection on the target content to be detected and the reference text in the preset database by the plagiarism detector, and outputting the syntax tree detection index, specifically including:

[0149] Parsing the target content to be detected and constructing a corresponding target syntax tree, and parsing the reference text and constructing a corresponding reference syntax tree;

[0150] Extracting syntactic features from the target syntax tree and the reference syntax tree respectively;

[0151] Calculating feature similarity based on the syntactic features, and calculating structural similarity between the target syntax tree and the reference syntax tree;

[0152] The feature similarity and the structure similarity are weightedly calculated to obtain the syntax tree detection index.

[0153] Exemplarily, methods for syntax tree detection include rule-based analysis, statistical parsing, and deep learning methods, wherein rule-based analysis uses predefined grammatical rules to construct a syntax tree, which requires detailed grammatical knowledge and complex rules to handle various grammatical structures; statistical parsing uses statistical-based methods, such as probabilistic context-free grammars (PCFGs) or conditional random fields (CRFs), to learn grammatical rules from a large amount of text, which are then used to parse new sentences; deep learning methods use deep neural networks, especially recursive neural networks (RNNs), long short-term memory networks (LSTMs), or transformer models to automatically learn the grammatical structure of sentences.

[0154] Exemplarily, in an embodiment of the present application, for each reference text in the target content to be detected and the database, a selected tool is used to parse and construct a syntax tree. The syntax tree represents the syntactic relationship between words in a sentence, such as subject-predicate-object relationship, modification relationship, etc. Syntactic features are extracted from the syntax tree, including but not limited to: sentence length, average subtree depth; type and frequency of use of core predicates (verbs); type of subject and object and their position in the tree; dependency type and its frequency (such as prepositional phrases, attributive clauses).

[0155] Exemplarily, the structural similarity of two syntax trees is compared. This involves the application of tree edit distance (TED), longest common subsequence (LCS) in the tree structure, or a node and edge matching algorithm, based on the extracted syntactic features, calculating the similarity between feature vectors, and using cosine similarity, Jaccard similarity or other measurement methods. The structural similarity and feature similarity are combined, and the final score is obtained by weighted average or other combination methods as a syntax tree detection indicator.

[0156] Step S2022: Calculating the edit distance between the target content to be detected and the reference text in the preset database by the plagiarism detector, and outputting the edit distance index, specifically including:

[0157] Determining the text length of the target content to be detected and the reference text;

[0158] Creating an initial matrix based on the length of the text;

[0159] Traversing each position in the initial matrix, and calculating the value corresponding to each position according to the cost of performing a replacement operation, an insertion operation, and a deletion operation on each position;

[0160] The edit distance index is determined according to the values ​​corresponding to the respective positions.

[0161] For example, create a (m+1)x(n+1) matrix, where m and n are the lengths of the target content and reference text, respectively, and the first row and column are filled with the edit distance from the empty string to the respective string (i.e., the number of steps from 0 to the current index). Traverse the matrix, for each position (i, j), calculate the cost of each operation, and take the minimum value as the value of the current position:

[0162] If the characters are equal, the cost is the value of the top left corner (i.e. the edit distance to the previous position).

[0163] If the characters are not equal, the cost is the minimum of the top-left value plus 1 (representing a replace operation), the left value plus 1 (representing an insert operation), and the top value plus 1 (representing a delete operation).

[0164] The value in the lower right corner of the matrix is ​​the minimum number of edits required to convert the target content into the reference text, which is the edit distance indicator.

[0165] Exemplarily, the method further includes measuring the similarity between the target content to be detected and the reference text in a preset database through a plagiarism detector, and outputting a similarity measurement index.

[0166] Exemplarily, methods for determining similarity metrics include cosine similarity, Jaccard similarity coefficient, and semantic-based similarity. Optionally, cosine similarity is to convert text into vectors (for example, using a bag-of-words model, TF-IDF), and then calculate the cosine angle between the two vectors to determine the similarity of the text; the Jaccard similarity coefficient is to measure their similarity by calculating the ratio of the common vocabulary of two texts to their respective total vocabulary; semantic-based similarity is to use a pre-trained language model (such as BERT, Word2Vec) to obtain deep semantic representations of words or sentences and calculate the similarity between them.

[0167] Exemplarily, the detection method also includes: using the text detector to perform text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected with reference texts in a preset database, and outputting at least one of the text style statistical indicators, linguistic feature statistical indicators and fact detection indicators.

[0168] For example, text style comparison can use NLP toolkits (such as spaCy, NLTK or Hugging FaceTransformers) to extract advanced features such as n-gram, term frequency-inverse document frequency (TF-IDF), vocabulary richness, syntactic complexity, etc.; by applying pre-trained style embedding models (such as GloVe or BERT style transfer models) to obtain the style vector of the text, and then use cosine similarity or K-means clustering to compare the style differences of different texts. This type of detection method has a certain detection effect on the content generated by large language models, but the judgment effect is limited when directly using human text in training data. Therefore, it is necessary to combine similarity measurement indicators, syntax tree detection indicators, edit distance indicators and content sources for judgment, and the results will be more accurate.

[0169] For example, linguistic feature detection focuses on distinguishing large language model texts from life texts based on lexical features, language structure, emotional meaning, etc. This includes using dependency syntax analysis, component syntax analysis and other technologies to compare the complexity of syntactic structures, the frequency of clauses, prepositional phrase patterns, etc. of different texts, analyzing the semantic role framework, comparing the expression of events, argument structure, etc., to evaluate differences at the semantic level, and using sentiment analysis tools to identify the emotional tendency and tone intensity of the text, and compare the proportion and intensity of positive, negative and neutral emotions.

[0170] Exemplarily, fact detection includes knowledge graph verification, citation source checking, cross-validation and fact checking. By building or utilizing existing large-scale knowledge graphs (such as DBpedia and Wikidata), the authenticity of the information is evaluated by comparing whether the entities and relationships in the text can be found in the graph; for texts containing data, statistics or citations, verify whether the source provided is reliable and whether it can be traced back to authoritative publication; compare the target content to be detected with multiple credible news sources or databases to identify contradictions; apply deep learning-based fact-checking systems (such as BERT-based models). These models are trained to identify false information in texts and make authenticity judgments through context understanding, logical reasoning and other means.

[0171] This embodiment adopts the above scheme, specifically, the plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database, outputs at least one of the similarity measurement index, syntax tree detection index and edit distance index, and inputs the target content to be detected into the text detector; the text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index, which can reflect various aspects of the target content to be detected and is used to quickly determine the large language model generated content in the target content to be detected. Compared with manual review, the detection efficiency can be greatly improved, and the detection effect of the large language model generated content is improved.

[0172] Sixth embodiment

[0173] Based on any of the above embodiments of the present application, a sixth embodiment of the present application is proposed. In the sixth embodiment of the present application, the same or similar contents as any of the above embodiments can be referred to the above introduction, and will not be repeated in the following. Figure 6 , step S30 may include at least one of steps S301 to S303:

[0174] Step S301: performing weighted calculation according to at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index to obtain a weighted sum of the indicators, and determining the content whose weighted sum of the indicators exceeds a preset sum threshold as content generated by the large language model;

[0175] Reference Figure 7 , Figure 7 is a schematic diagram of a first detection process according to the sixth embodiment, as shown in Figure 7 As shown, in the embodiment of the present application, the target content to be detected (i.e., the text to be filtered) is respectively input into the plagiarism detector and the text detector (i.e., other detectors), and the plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database, and outputs at least one of the similarity measurement index, syntax tree detection index and edit distance index; the text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, and outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index.

[0176] Exemplarily, in an embodiment of the present application, each statistical indicator can be weightedly calculated to obtain a weighted sum of the indicators of each statistical indicator, which is then compared with a preset sum threshold. Content whose weighted sum exceeds the preset sum threshold is determined to be content generated by a large language model, and content whose weighted sum exceeds the preset sum threshold is determined to be content not generated by a large language model.

[0177] It should be noted that, in the process of weighted calculation, the weight of each statistical indicator can be set according to the actual situation, and this is not specifically limited in the embodiments of the present application. Similarly, the preset sum threshold can also be set or adjusted according to actual needs, and this is not specifically limited in the embodiments of the present application. For example, when there is too much text content emerging in the platform in a short period of time, the preset sum threshold can be lowered to provide a more stringent detection standard for the detection of text content.

[0178] Step S302: inputting at least one of the similarity measurement index, syntax tree detection index, edit distance index, text style statistical index, linguistic feature statistical index and fact detection index into a pre-trained binary classification model, and determining the large language model generated content in the target content to be detected according to the prediction result of the binary classification model;

[0179] Exemplarily, in an embodiment of the present application, a binary classification model can also be pre-trained, and the statistical indicators output by the plagiarism detector and / or the text detector can be input into the binary classification model for prediction to determine the large language model generated content in the target content to be detected.

[0180] Exemplarily, the binary classification model can use logistic regression or Fisher linear discriminant analysis (LDA), and can also use algorithms such as decision trees, random forests, support vector machines, and neural networks, which are not specifically limited in the embodiments of the present application.

[0181] Step S303: determining a source overlap ratio according to the content sources, and determining the content whose source overlap ratio exceeds a preset overlap threshold as content generated by a large language model.

[0182] Exemplarily, the step of determining a source overlap ratio according to the content sources, and determining the content whose source overlap ratio exceeds a preset overlap threshold as content generated by a large language model includes:

[0183] According to the content source, determine the total number of content sources and the number of content sources belonging to the large language model database;

[0184] Calculate the ratio of the number of items belonging to the large language model database to the total number of items of the content source to obtain the source overlap ratio;

[0185] Determining whether the source overlap ratio exceeds the preset overlap threshold;

[0186] If the source overlap ratio exceeds the preset overlap threshold, the target content to be detected is determined to be content generated by a large language model.

[0187] Reference Figure 8 , Figure 8 is a schematic diagram of a second detection process according to the sixth embodiment, as shown in Figure 8 As shown, the text sources involved in the target text to be detected are marked one by one, including text source 1, text source 2, ..., text source n. The source overlap ratio can be determined according to the proportion of content sources in the large language model database (data that can be used for large language model training) in the total content sources. For example, among 100 text sources, 80 text sources appear in the public training data, and the source overlap ratio is 80%. The source overlap ratio is compared with a preset overlap threshold, and the content whose source overlap ratio exceeds the preset overlap threshold is determined to be content generated by the large language model.

[0188] It should be noted that in the process of comparing the source overlap ratio with the preset overlap threshold, the preset overlap threshold can be set or adjusted according to actual needs. There is no specific limitation on this in the embodiment of the present application. For example, when there is too much text content emerging in the platform in a short period of time, the preset overlap threshold can be lowered to provide a more stringent detection standard for the detection of text content.

[0189] In an embodiment of the present application, the target text to be detected may be a large amount of text to be detected. After detection by the above detection method, the large language model generated content and the non-large language model generated content in the target text to be detected can be determined. The large language model generated content is used as the screened text, and the non-large language model generated content is used as the screened text, thereby realizing rapid screening of large amounts of text to be detected in the platform.

[0190] This embodiment uses the above scheme, specifically by performing weighted calculation according to each indicator in the first statistical indicator and / or the second statistical indicator, obtaining the weighted sum of the indicators, and determining the content whose weighted sum of the indicators exceeds the preset sum threshold as the content generated by the large language model; inputting each indicator in the first statistical indicator and / or the second statistical indicator into a pre-trained binary classification model, and determining the content generated by the large language model in the target content to be detected according to the prediction result of the binary classification model; determining the source overlap ratio according to the content source, and determining the content whose source overlap ratio exceeds the preset overlap threshold as the content generated by the large language model. The large language model generated content and non-large language model generated content in the target text to be detected can be determined, and the large language model generated content is used as the screened text, and the non-large language model generated content is used as the screened text, so as to realize the rapid screening of a large number of texts to be detected in the platform. A convenient and fast auxiliary detection scheme is provided for the recent screening of content generated by the large language model, which makes up for the insensitivity of the grammar detection scheme to a large amount of quoted material text, and improves the efficiency of the comprehensive detection scheme.

[0191] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0192] This application also provides a detection device, please refer to Fig. 9 , the detection device comprises:

[0193] An acquisition module is used to acquire target content to be detected;

[0194] A detection module, used to detect the target content to be detected based on a preset plagiarism detector and / or text detector, and output statistical indicators and / or content sources corresponding to the target content to be detected;

[0195] A determination module is used to determine the large language model generated content in the target content to be detected based on the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

[0196] Exemplarily, the step of obtaining the target content to be detected includes:

[0197] Obtain the original content to be detected;

[0198] The original content to be detected is preprocessed to obtain the target content to be detected.

[0199] Exemplarily, the step of detecting the target content to be detected based on a preset plagiarism detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected includes:

[0200] Inputting the target content to be detected into the plagiarism detector;

[0201] The plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database, and outputs at least one of the similarity measurement index, syntax tree detection index and edit distance index; and / or, the plagiarism detector performs source marking on the target content to be detected, and outputs the content source.

[0202] Exemplarily, the step of detecting the target content to be detected based on a preset text detector and outputting statistical indicators corresponding to the target content to be detected includes:

[0203] Inputting the target content to be detected into the text detector;

[0204] The text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, and outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index.

[0205] Exemplarily, the step of performing syntax tree detection on the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the syntax tree detection index includes:

[0206] Parsing the target content to be detected and constructing a corresponding target syntax tree, and parsing the reference text and constructing a corresponding reference syntax tree;

[0207] Extracting syntactic features from the target syntax tree and the reference syntax tree respectively;

[0208] Calculating feature similarity based on the syntactic features, and calculating structural similarity between the target syntax tree and the reference syntax tree;

[0209] The feature similarity and the structure similarity are weightedly calculated to obtain the syntax tree detection index.

[0210] Exemplarily, the step of calculating the edit distance between the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the edit distance index includes:

[0211] Determining the text length of the target content to be detected and the reference text;

[0212] Creating an initial matrix based on the length of the text;

[0213] Traversing each position in the initial matrix, and calculating the value corresponding to each position according to the cost of performing a replacement operation, an insertion operation, and a deletion operation on each position;

[0214] The edit distance index is determined according to the values ​​corresponding to the respective positions.

[0215] Exemplarily, the step of determining the large language model generated content in the target content to be detected according to the statistical indicators and / or the content source includes at least one of the following:

[0216] Performing weighted calculation according to at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index to obtain a weighted sum of the indicators, and determining the content whose weighted sum of the indicators exceeds a preset sum threshold as content generated by the large language model;

[0217] Inputting at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index into a pre-trained binary classification model, and determining the large language model generated content in the target content to be detected according to the prediction result of the binary classification model;

[0218] The source overlap ratio is determined according to the content sources, and the content whose source overlap ratio exceeds a preset overlap threshold is determined to be content generated by a large language model.

[0219] Exemplarily, the step of determining a source overlap ratio according to the content sources, and determining the content whose source overlap ratio exceeds a preset overlap threshold as content generated by a large language model includes:

[0220] According to the content source, determine the total number of content sources and the number of content sources belonging to the large language model database;

[0221] Calculate the ratio of the number of items belonging to the large language model database to the total number of items of the content source to obtain the source overlap ratio;

[0222] Determining whether the source overlap ratio exceeds the preset overlap threshold;

[0223] If the source overlap ratio exceeds the preset overlap threshold, the target content to be detected is determined to be content generated by a large language model.

[0224] Exemplarily, before the step of detecting the target content to be detected based on a preset plagiarism detector and / or text detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected, the step further includes:

[0225] The plagiarism detector and / or text detector is obtained by training, specifically comprising:

[0226] Get a sample dataset;

[0227] Preprocessing and marking the sample data in the sample data set to obtain a processed sample data set;

[0228] The first model framework is trained according to the processed sample data set to obtain the plagiarism detector, and / or the second model framework is trained according to the processed sample data set to obtain the text detector.

[0229] The detection device provided by the present application adopts the detection method in the above embodiment to solve the technical problem of detection. Compared with the prior art, the beneficial effects of the detection device provided by the present application are the same as the beneficial effects of the detection method provided by the above embodiment, and the other technical features in the detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0230] The present application provides a detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the detection method in the above-mentioned embodiment one.

[0231] Reference below Fig.10 , which shows a schematic diagram of the structure of the detection device suitable for implementing the embodiment of the present application. The detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0232] like Fig.10As shown, the detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the detection device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the detection device to communicate with other devices wirelessly or wired to exchange data. Although the detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0233] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0234] The detection device provided by the present application adopts the detection method in the above embodiment to solve the technical problem of detection. Compared with the prior art, the beneficial effects of the detection device provided by the present application are the same as the beneficial effects of the detection method provided by the above embodiment, and the other technical features in the detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0235] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0236] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0237] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the detection method in the above-mentioned embodiment.

[0238] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0239] The computer-readable storage medium may be included in the detection device; or may exist independently without being assembled into the detection device.

[0240] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the detection device, the detection device: obtains the target content to be detected; detects the target content to be detected based on a preset plagiarism detector, and outputs the statistical indicators and / or content sources corresponding to the target content to be detected; determines the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, and an edit distance indicator.

[0241] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0242] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0243] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0244] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned detection method, and can solve the technical problems of detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0245] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned detection method when executed by a processor.

[0246] The computer program product provided in this application can solve the technical problem of detection. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the detection method provided in the above embodiment, which will not be repeated here.

[0247] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A detection method, characterized in that: The method includes: Obtain the target content to be detected; Detecting the target content to be detected based on a preset plagiarism detector and / or text detector, and outputting statistical indicators and / or content sources corresponding to the target content to be detected; Determine the large language model generated content in the target content to be detected according to the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

2. The method according to claim 1, characterized in that The step of detecting the target content to be detected based on a preset plagiarism detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected includes: Inputting the target content to be detected into the plagiarism detector; The plagiarism detector performs similarity measurement, syntax tree detection and / or edit distance calculation on the target content to be detected and the reference text in the preset database, and outputs at least one of the similarity measurement index, syntax tree detection index and edit distance index; and / or, the plagiarism detector performs source marking on the target content to be detected, and outputs the content source; The step of detecting the target content to be detected based on a preset text detector and outputting statistical indicators corresponding to the target content to be detected includes: Inputting the target content to be detected into the text detector; The text detector performs text style comparison, linguistic feature comparison and / or fact detection on the target content to be detected and the reference text in the preset database, and outputs at least one of the text style statistical index, linguistic feature statistical index and fact detection index.

3. The method as claimed in claim 2, characterized in that The step of performing syntax tree detection on the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the syntax tree detection index comprises: Parsing the target content to be detected and constructing a corresponding target syntax tree, and parsing the reference text and constructing a corresponding reference syntax tree; Extracting syntactic features from the target syntax tree and the reference syntax tree respectively; Calculating feature similarity based on the syntactic features, and calculating structural similarity between the target syntax tree and the reference syntax tree; Performing weighted calculation on the feature similarity and the structure similarity to obtain the syntax tree detection index; The step of calculating the edit distance between the target content to be detected and the reference text in the preset database by the plagiarism detector and outputting the edit distance index comprises: Determining the text length of the target content to be detected and the reference text; Creating an initial matrix based on the length of the text; Traversing each position in the initial matrix, and calculating the value corresponding to each position according to the cost of performing a replacement operation, an insertion operation, and a deletion operation on each position; The edit distance index is determined according to the values ​​corresponding to the respective positions.

4. The method according to claim 1, characterized in that The step of determining the large language model generated content in the target content to be detected according to the statistical indicators and / or the content source comprises at least one of the following: Performing weighted calculation according to at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index to obtain a weighted sum of the indicators, and determining the content whose weighted sum of the indicators exceeds a preset sum threshold as content generated by the large language model; Inputting at least one of the similarity measurement index, the syntax tree detection index, the edit distance index, the text style statistical index, the linguistic feature statistical index, and the fact detection index into a pre-trained binary classification model, and determining the large language model generated content in the target content to be detected according to the prediction result of the binary classification model; The source overlap ratio is determined according to the content sources, and the content whose source overlap ratio exceeds a preset overlap threshold is determined to be content generated by a large language model.

5. The method according to claim 4, characterized in that The step of determining the source overlap ratio according to the content sources and determining the content whose source overlap ratio exceeds a preset overlap threshold as content generated by a large language model comprises: According to the content source, determine the total number of content sources and the number of content sources belonging to the large language model database; Calculate the ratio of the number of items belonging to the large language model database to the total number of items of the content source to obtain the source overlap ratio; Determining whether the source overlap ratio exceeds the preset overlap threshold; If the source overlap ratio exceeds the preset overlap threshold, the target content to be detected is determined to be content generated by a large language model.

6. The method according to claim 1, characterized in that Before the step of detecting the target content to be detected based on a preset plagiarism detector and / or text detector and outputting statistical indicators and / or content sources corresponding to the target content to be detected, the step further includes: The plagiarism detector and / or text detector is obtained by training, specifically comprising: Get a sample dataset; Preprocessing and marking the sample data in the sample data set to obtain a processed sample data set; The first model framework is trained according to the processed sample data set to obtain the plagiarism detector, and / or the second model framework is trained according to the processed sample data set to obtain the text detector.

7. A detection device, characterized in that: The device comprises: An acquisition module is used to acquire target content to be detected; A detection module, used to detect the target content to be detected based on a preset plagiarism detector and / or text detector, and output statistical indicators and / or content sources corresponding to the target content to be detected; A determination module is used to determine the large language model generated content in the target content to be detected based on the statistical indicators and / or content sources, wherein the statistical indicators include at least one of a similarity measurement indicator, a syntax tree detection indicator, an edit distance indicator, a text style statistical indicator, a linguistic feature statistical indicator, and a fact detection indicator.

8. A detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the detection method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the detection method according to any one of claims 1 to 6 are implemented.

Images