Machine generated text detection method, terminal, medium and program product

Abstract

The invention discloses a machine generated text detection method, a terminal, a medium and a program product in the field of natural language processing, and the method comprises the steps: inputting a to-be-detected target text into a trained text detection model, and obtaining a detection result outputted by the text detection model; the text detection model comprises a first semantic feature extraction network, a second semantic feature extraction network and a fusion classification network; the first semantic feature extraction network is used for extracting semantic features of keywords in a target text to generate a first representation vector; the second semantic feature extraction network is used for extracting overall semantic features of the target text and generating a second representation vector; the fusion classification network processes the first representation vector and the second representation vector to generate a detection result about whether the target text is a machine-generated text; the training of the text detection model adopts a multi-task learning strategy. According to the method, multi-task training is introduced, so that the accuracy and robustness of machine generated text detection are effectively improved.

Description

Technical Field

[0001] The invention relates to a machine-generated text detection method, terminal, medium and program product, and belongs to the field of natural language processing. Background Art

[0002] Since the release of GPT-3.5 in 2022, the field of natural language processing has experienced a leap forward. GPT has performed well in text generation, text reasoning, text understanding, and dialogue systems with its outstanding performance, and has even reached or exceeded the human level in some tasks. This technological breakthrough has not only promoted the progress of academic research, but also had a profound impact on all walks of life, especially in text generation, which has greatly saved the time cost of content generation in some fields. However, there are many problems with machine-generated texts, especially in news, academic writing, and content review. It is difficult to distinguish whether it is a machine-generated text by manual review alone, resulting in problems such as fake news and academic fraud, and it is also very time-consuming. Therefore, an effective detection mechanism can not only maintain the credibility of the content, but also prevent the spread of bad information.

[0003] At present, many methods have been proposed to detect whether text is machine-generated, including ideas based on attention mechanism, contrastive learning and zero-shot learning. Among them, the more typical models are BERT, RoBERTa, Longformer and FastText. The first three models are based on the Transformer architecture, while Fasttext is based on the Word2Vec architecture. These models have performed well in the field of natural language processing. Based on BERT and RoBERTa, the DeBERTa model further improves the performance by introducing a decoupled attention mechanism and an enhanced mask decoder, especially in the task of detecting machine-generated text.

[0004] However, DeBERTa also has some obvious disadvantages: First, although DeBERTa has significantly improved performance, the computational complexity of the model is higher than that of traditional BERT or RoBERTa due to the introduction of additional decoupled attention mechanisms and enhanced decoding processes. This means that more computing resources are required during the training and inference stages, especially when processing long texts, the demand for memory and processing time will increase significantly. Second, DeBERTa has a large number of parameters, resulting in high storage and loading costs for the model. Although larger models generally perform better, they may also encounter deployment difficulties, especially in resource-limited environments. Third, models with large numbers of parameters are prone to overfitting on small-scale data sets, which increases the difficulty of regularization and parameter adjustment. These three shortcomings greatly increase the difficulty of the DeBERTa model in practical applications, especially in resource-limited environments. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, terminal, medium and program product for detecting machine-generated text, introduce multi-task training, and effectively improve the accuracy and robustness of machine-generated text detection.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for detecting machine-generated text, comprising:

[0008] Input the target text to be detected into the trained text detection model to obtain the detection result output by the text detection model;

[0009] The text detection model includes a first semantic feature extraction network, a second semantic feature extraction network and a fusion classification network; the first semantic feature extraction network is used to extract the semantic features of keywords in the target text and generate a first representation vector; the second semantic feature extraction network is used to extract the overall semantic features of the target text and generate a second representation vector; the fusion classification network processes the first representation vector and the second representation vector to generate a detection result of whether the target text is a machine-generated text;

[0010] The text detection model is trained using a multi-task learning strategy, including:

[0011] The main task is to classify whether the target text in the training data is machine-generated text through the fusion classification network and calculate the classification loss value. ;

[0012] The auxiliary task uses the second semantic feature extraction network to determine whether the sentence pairs in the training data are context sentence pairs and calculate the context discrimination loss value. ;

[0013] Will and Add together as the total loss for multi-task training , used to optimize the parameters of the text detection model.

[0014] Furthermore, the keywords in the target text are extracted using a term frequency-inverse document frequency (TF-IDF) algorithm, including:

[0015] Calculate the term frequency TF, inverse document frequency IDF and document frequency DF respectively. The calculation formulas are as follows:

[0016] ;

[0017] Where: word frequency TF represents word The ratio of the number of occurrences in document d to the total number of words in document d is used to measure the word The importance of appearing in document d; Represents a word in document d;

[0018] ;

[0019] Where: Inverse document frequency IDF is used to measure the word Rarity of occurrence among all documents; Represents the entire document collection; Represents a collection of documents The total number of documents in ; Indicates that it contains words The number of documents; represents a constant to ensure that the denominator will not be zero;

[0020] ;

[0021] Where: Document frequency DF is used to measure word Throughout the document collection The frequency of occurrence in Indicates the word The number of documents;

[0022] Combining term frequency TF and inverse document frequency IDF, we get the value of term frequency-inverse document frequency TF-IDF, as shown in the following formula:

[0023] ;

[0024] According to the distribution of word frequency TF and document frequency DF, define filtering conditions to remove noise words, sort the words that meet the conditions according to the TF-IDF value, and select several words with the highest TF-IDF value as the keywords of the target text.

[0025] Furthermore, the first semantic feature extraction network includes a word embedding layer, a Transformer module, and a fully connected layer connected in sequence. The first semantic feature extraction network extracts semantic features of keywords in the target text and generates a first representation vector, including:

[0026] Input the keywords extracted by the TF-IDF algorithm into the word embedding layer to generate the corresponding word vectors;

[0027] The generated word embedding vector is input into the Transformer module, the attention weight is calculated through the multi-head attention mechanism, the context information is integrated to generate the attention feature representation, and the feature representation is passed to the feedforward neural network; then the feature representation is optimized through residual connection and layer normalization;

[0028] The optimized feature representation is input into the fully connected layer, extracted and reduced in dimension to generate the first representation vector.

[0029] Furthermore, the second semantic feature extraction network includes a BERT model, a long short-term memory network LSTM module, a self-attention mechanism Attention module and a fully connected layer connected in sequence, and the second semantic feature extraction network extracts the overall semantic features of the target text and generates a second representation vector, including:

[0030] The target text is input into the Tokenizer of the BERT model. The Tokenizer decomposes the text into sub-word units called tokens. Each token is mapped to a token id, and a token id sequence is generated to represent the entire text, as shown in the following formula:

[0031] ;

[0032] Where: A sequence of token ids representing text; Indicates special marks used for classification; Indicates the text separator; n indicates the length of the text;

[0033] Convert the processed token id sequence into a matrix through forward propagation , as shown below:

[0034] ;

[0035] Where: Represents the vector corresponding to the nth word;

[0036] Encode X through the BERT model to obtain the semantic level feature H of the text, as shown in the following formula:

[0037] ;

[0038] Where: H is a matrix of dimension n×d, d is the hidden layer dimension of the BERT model;

[0039] The result H output by the BERT model is input into the LSTM module, and the hidden state is updated using the following formula:

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] Where: , , and Represent the outputs of the input gate, forget gate, output gate, and candidate memory unit respectively; , , and Represent the weight matrix of the input gate, the weight matrix of the forget gate, the weight matrix of the output gate, and the weight matrix of the candidate memory unit respectively; , , and They represent the bias term of the input gate, the bias term of the forget gate, the bias term of the output gate, and the bias term of the candidate memory unit respectively; represents the hidden state of the previous time step; represents the input data at time step t; represents the activation function; Indicates the memory status at the current moment; Represents the output of the forget gate; Indicates the memory status of the previous step; Represents the hidden state of the current time step; L represents the final output of the LSTM module;

[0048] The output L of the LSTM module is input into the Attention module. The calculation formula of the attention weight is as follows:

[0049] ;

[0050] Where: represents the attention weight of the i-th time step; represents the attention score of the i-th time step; represents the exponential sum of attention scores of all time steps; T represents the total number of time steps of the sequence;

[0051] Combining the output L of the LSTM module and the attention weight, we get the output A of the Attention module, as shown in the following formula:

[0052] ;

[0053] Where: represents the output item at each time step in L;

[0054] The output L of the LSTM module and the output A of the Attention module are concatenated to obtain the concatenated result Z, which is then transformed through a fully connected layer as shown in the following formula:

[0055] ;

[0056] ;

[0057] Where: represents the transformed vector; represents the weight of the fully connected layer; represents the bias of the fully connected layer;

[0058] After using the ReLU nonlinear activation function to perform nonlinear activation in the fully connected layer, the final result is output , as shown below:

[0059] ;

[0060] ;

[0061] Where: Represents the vector after nonlinear activation; Indicates the final result;

[0062] The final result will be output from extract The vector representation corresponding to the tag is used as the vector representation of the entire text to obtain the second representation vector.

[0063] Furthermore, the fusion classification network processes the first representation vector and the second representation vector to generate a detection result of whether the target text is a machine-generated text, including:

[0064] The first representation vector and the second representation vector are concatenated to form a final representation Representation of the text; Representation is input into a fully connected layer to obtain a vector representation output by the text detection model, wherein the first component represents the probability that the input data is written by humans, and the second component represents the probability that the input data is machine-generated; when the second component is greater than or equal to the first component, the input data is determined to be machine-generated text.

[0065] Furthermore, the total loss Obtained through the following steps:

[0066] The fusion classification network is used to classify whether the target text in the training data is machine-generated text, and the classification loss value is calculated using the cross entropy loss function. , as shown below:

[0067] ;

[0068] Where: is the true label, indicating whether the input text is written by humans or generated by machines; It is the prediction category of the text detection model, including the probability of predicting that the input text is written by humans and the probability of predicting that the input text is generated by a machine;

[0069] The second semantic feature extraction network is used to determine whether the sentence pairs in the training data are context sentence pairs, and the cross entropy loss function is used to calculate the context discrimination loss value. , as shown below:

[0070] ;

[0071] Where: is the true label, indicating whether the sentence pair belongs to the same context; is the predicted probability output by the second semantic feature extraction network, including the probability of predicting that the input sentence pair belongs to the same context and the probability of belonging to different contexts;

[0072] The loss of the main task and the loss of auxiliary tasks Add together to get the total loss , as shown below:

[0073] ;

[0074] Utilize total loss Back propagation, optimizes the parameters of the text detection model.

[0075] In a second aspect, the present invention provides an electronic terminal, comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the above-mentioned machine-generated text detection method are performed.

[0076] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the above-mentioned machine-generated text detection method are implemented.

[0077] In a fourth aspect, the present invention provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the above-mentioned machine-generated text detection method are implemented.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] The present invention provides a machine-generated text detection method based on multi-task learning. By combining a first semantic feature extraction network and a second semantic feature extraction network, the target text can be effectively analyzed from the two aspects of keyword features and overall semantic features, thereby improving the ability to distinguish between machine-generated text and human text. At the same time, by using a multi-task learning strategy and introducing auxiliary tasks to judge contextual sentence pairs, the text detection model's understanding of semantic levels and contextual relationships is enhanced, so that the text detection model has stronger generalization ability and robustness, thereby being able to maintain high accuracy and stability in different text types and complex scenarios, significantly improving the effect and efficiency of machine-generated text detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 is a flow chart of the machine-generated text detection method provided in the first embodiment of the present invention;

[0081] Figure 2 The total loss of utilization provided by the first embodiment of the present invention is Schematic diagram of the process of training a text detection model;

[0082] Figure 3 This is a flowchart of using the TF-IDF algorithm to extract keywords from a target text, provided in the first embodiment. DETAILED DESCRIPTION

[0083] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0084] The terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of this disclosure / application, unless otherwise specified, "plurality" means two or more.

[0085] Embodiment 1:

[0086] Figure 1is a flow chart of the machine-generated text detection method in Embodiment 1 of the present invention; Figure 2 The total loss in the first embodiment of the present invention is Schematic diagram of the process of training the text detection model; the above flowchart only shows the logical order of the method and text detection model training described in this embodiment. Under the premise of not conflicting with each other, in other possible embodiments of the present invention, different Figure 1 and Figure 2 The steps shown or described are performed in the order shown. Figure 2 In this embodiment, the text detection model is first trained through the following steps:

[0087] Clean the training data;

[0088] Use the term frequency-inverse document frequency TF-IDF algorithm to extract keywords from the cleaned training data, where the keywords are significant words that can characterize the content characteristics of the text;

[0089] Inputting the extracted keywords into a first semantic feature extraction network to obtain a first representation vector;

[0090] Inputting the cleaned training data into a second semantic feature extraction network to obtain a second representation vector;

[0091] The fusion classification network processes the first representation vector and the second representation vector to generate a detection result of whether the target text is machine-generated text.

[0092] Perform the main task: classify whether the target text in the training data is machine-generated text through the fusion classification network, and calculate the classification loss value ;

[0093] Perform auxiliary tasks: Use the second semantic feature extraction network to determine whether the sentence pairs in the training data are contextual sentence pairs, and calculate the contextual discrimination loss value ;

[0094] Will and Add together as the total loss for multi-task training , used to optimize the parameters of the text detection model. When the training reaches the maximum number of epochs or the total loss on the validation set within the preset number of steps When there is no decline, terminate the training.

[0095] See also Figure 1, after obtaining the trained text detection model, input the target text to be detected, and obtain the vector representation output by the text detection model, where the first component represents the probability that the input data is written by humans, and the second component represents the probability that the input data is machine-generated; when the second component is greater than or equal to the first component, it is determined that the input data is machine-generated text.

[0096] In this embodiment, the Mega dataset is selected as the research object, which is a large-scale dataset for machine-generated text in the field of natural language processing. To achieve scientific and reasonable training and evaluation, this embodiment divides the dataset into a training set, a validation set, and a test set, with proportions of 70%, 15%, and 15% respectively.

[0097] In the original dataset, there is a lot of noise, and the interference of stop words on the machine text detection task is particularly significant. Stop words usually refer to some words that lack practical meaning in text analysis, such as "de", "le", "zai", etc. The existence of these words is likely to interfere with the training and analysis of the model. Therefore, removing stop words can effectively reduce the noise in the dataset.

[0098] This embodiment uses the baidu_stopwords stop word library for data cleaning. The specific steps are as follows: First, import the stop word library and store the original data in the form of a DataFrame. Then load the stop word list and perform word segmentation on each text data in the dataset. Chinese word segmentation uses the Jieba word segmentation tool to split the sentence into independent lexical units. Based on the word segmentation results, filter out the words in the stop word list and retain the valid words. Finally, recombine the filtered words into a complete string to form a new cleaned sentence and save it as a file for subsequent steps.

[0099] Extract keywords from the cleaned dataset to better represent the text meaning. Usually, a sentence can reflect its core content through several keywords. Machine-generated text often has a high language consistency, and its expression form tends to be more formal, logical, and grammatical, while human-generated text is more colloquial. Therefore, in the machine text detection task, the corpus keywords (such as logical words) obtained by using keyword extraction technology can significantly improve the accuracy of text determination.

[0100] This embodiment uses the TF-IDF algorithm for keyword extraction. Before keyword screening, it is necessary to calculate the term frequency TF, inverse document frequency IDF, and document frequency DF first. The calculation formulas for each index are as follows:

[0101] ;

[0102] In the formula: the term frequency TF represents the word The ratio of the number of occurrences in document d to the total number of words in document d is used to measure the word The importance of appearing in document d; Represents a word in document d;

[0103] ;

[0104] Where: Inverse document frequency IDF is used to measure the word Rarity of occurrence among all documents; Represents the entire document collection; Represents a collection of documents The total number of documents in ; Indicates that it contains words The number of documents; Represents a constant to ensure that the denominator will not be zero;

[0105] ;

[0106] Where: Document frequency DF is used to measure word Throughout the document collection The frequency of occurrence in Indicates the word The number of documents;

[0107] Combining term frequency TF and inverse document frequency IDF, we get the value of term frequency-inverse document frequency TF-IDF, as shown in the following formula:

[0108] ;

[0109] like Figure 3 As shown, according to the distribution of word frequency TF and document frequency DF, this embodiment defines filtering conditions to remove noise words. The specific filtering conditions are as follows:

[0110] Condition 1: The word frequency TF appears at least 5 times in the text;

[0111] Condition 2: Document frequency DF is less than 0.85.

[0112] Words that do not meet the above conditions will be considered as noise words and removed. Then, the words that meet the screening conditions are sorted according to the TF-IDF value, and the top N words with the largest TF-IDF value are selected as the keywords of the document. In this embodiment, N is 64, that is, each document finally selects 64 keywords to characterize the text content characteristics.

[0113] The above keyword extraction step mainly extracts features from the word form level, and does not involve the representation at the semantic level. For example, words like "possible" and "probable" will be regarded as different words in this step, although they are semantically similar. To make up for this deficiency, this embodiment further uses the first semantic feature extraction network to process and represent the filtered keywords from the semantic level.

[0114] Specifically, the filtered keywords are input into the word embedding layer to obtain the word vector representation. The word vector generated here is high-dimensional (the dimension is 10,000), but high-dimensional data is prone to problems such as large amount of calculation and slow processing speed. In addition, there may be a large number of redundant features in high-dimensional data. These redundant features not only increase the consumption of computing resources, but also may have a negative impact on model performance. Therefore, by reducing the data dimension, redundant features can be effectively removed and important information can be retained, thereby improving the efficiency and expression ability of the model.

[0115] To achieve dimensionality reduction, this embodiment inputs the word vector into the Transformer module for feature extraction. Subsequently, by introducing operations such as embedding layer, multi-head self-attention mechanism, feedforward neural network, residual connection and layer normalization, the feature vector is input into the fully connected layer with bias to complete the dimensionality reduction. The data format after dimensionality reduction is more compact and can effectively represent the semantic features of keywords. Finally, the dimension of the word vector is reduced to 1024, and the vector after dimensionality reduction is recorded as the first representation vector representation1.

[0116] The sentence semantic level is further extracted through the second semantic feature extraction network to obtain the second representation vector representation2. First, the cleaned text is segmented, special tags are added, and converted into tokenids. Then, the token ids of the text are converted into dense word vectors using the word embedding matrix. The second semantic feature extraction network is a BERT model, a long short-term memory network LSTM module, a self-attention mechanism Attention module and a fully connected layer connected in sequence. Among them, the BERT model is used to extract semantic features at the sentence level; the LSTM module is used to capture long-distance dependencies; and the Attention module is used to weight input features to enhance the contextual representation ability of the model. Compared with the traditional semantic extraction using only the BERT model, the advantage of the second semantic feature extraction network used in this embodiment is that it combines the powerful feature extraction ability of BERT with the sequence modeling ability of LSTM and the focusing ability of the attention mechanism to form a flexible, efficient and high-performance model architecture. Through this combination, the second semantic feature extraction network can more comprehensively capture the semantics and contextual relationships of the text, thereby improving the completion effect of the task. The specific processing steps are as follows:

[0117] The target text is input into the Tokenizer of the BERT model. The Tokenizer decomposes the text into sub-word units called tokens. Each token is mapped to a token id, and a token id sequence is generated to represent the entire text, as shown in the following formula:

[0118] ;

[0119] Where: A sequence of token ids representing text; Indicates special marks used for classification; Indicates the text separator; n indicates the length of the text;

[0120] Convert the processed token id sequence into a matrix through forward propagation , as shown below:

[0121] ;

[0122] Where: Represents the vector corresponding to the nth word;

[0123] Encode X through the BERT model to obtain the semantic level feature H of the text, as shown in the following formula:

[0124] ;

[0125] Where: H is a matrix of dimension n×d, d is the hidden layer dimension of the BERT model;

[0126] The result H output by the BERT model is input into the LSTM module, and the hidden state is updated using the following formula:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] ;

[0132] ;

[0133] ;

[0134] Where: , , and Represent the outputs of the input gate, forget gate, output gate, and candidate memory unit respectively; , , and Represent the weight matrix of the input gate, the weight matrix of the forget gate, the weight matrix of the output gate, and the weight matrix of the candidate memory unit respectively; , , and They represent the bias term of the input gate, the bias term of the forget gate, the bias term of the output gate, and the bias term of the candidate memory unit respectively; represents the hidden state of the previous time step; represents the input data at time step t; represents the activation function; Indicates the memory status at the current moment; Represents the output of the forget gate; Indicates the memory status of the previous step; Represents the hidden state of the current time step; L represents the final output of the LSTM module;

[0135] The output L of the LSTM module is input into the Attention module. The calculation formula of the attention weight is as follows:

[0136] ;

[0137] Where: represents the attention weight of the i-th time step; represents the attention score of the i-th time step; represents the exponential sum of attention scores of all time steps; T represents the total number of time steps of the sequence;

[0138] Combining the output L of the LSTM module and the attention weight, we get the output A of the Attention module, as shown in the following formula:

[0139] ;

[0140] Where: represents the output item at each time step in L;

[0141] The output L of the LSTM module and the output A of the Attention module are concatenated to obtain the concatenated result Z, which is then transformed through a fully connected layer as shown in the following formula:

[0142] ;

[0143] ;

[0144] Where: represents the transformed vector; represents the weight of the fully connected layer; represents the bias of the fully connected layer;

[0145] After using the ReLU nonlinear activation function to perform nonlinear activation in the fully connected layer, the final result is output , as shown below:

[0146] ;

[0147] ;

[0148] Where: Represents the vector after nonlinear activation; Indicates the final result;

[0149] The final result will be output from extract The vector representation corresponding to the tag is used as the vector representation of the entire text to obtain the second representation vector representation2.

[0150] The first representation vector representation1 and the second representation vector representation2 are combined by the fusion classification network to form the final representation vector Representation of the text with a dimension of 1*2048. Representation is input into the fully connected layer with the bias matrix to obtain the vector representation of the model output, where the first component represents the probability that the input data is written by humans, and the second component represents the probability that the input data is generated by a machine. Perform multi-task training and set the loss of the main task and the loss of auxiliary tasks Add together as the total loss for multi-task training ;

[0151] The main task is to classify whether the target text in the training data is machine-generated text through the fusion classification network and calculate the corresponding loss value. , as shown below:

[0152] ;

[0153] Where: is the true label, indicating whether the input text is written by humans or generated by machines; It is the prediction category of the text detection model, including the probability of predicting that the input text is written by humans and the probability of predicting that the input text is generated by a machine;

[0154] The auxiliary task uses the second semantic feature extraction network to determine whether the sentence pairs in the training data are context sentence pairs and calculate the corresponding loss value. ; The specific steps are as follows:

[0155] First, prepare the input data: select multiple texts from the cleaned training data, such as text and , split them into multiple sentence chunks: ; in, Indicates from text Sentence chunks separated from the middle; Indicates from text The sentence chunks are divided from ; m and n are the number of sentence chunks. Next, sentence chunk pairs are constructed and labels are assigned to each pair of sentence chunks.

[0156] Two sentence chunks from the same text are labeled as 1. For example, and All from , so the sentence pair The label of is 1. The labels of two sentence chunks from different texts are recorded as 0. For example, From , From , so the sentence pair The label of is 0.

[0157] The processed data is input into the second semantic feature extraction network, and the loss function is defined as follows:

[0158] ;

[0159] Where: is the true label, indicating whether the sentence pair belongs to the same context; It is the predicted probability output by the second semantic feature extraction network, including the probability that the input sentence pair belongs to the same context and the probability that it belongs to different contexts.

[0160] Will and Add together as the total loss for multi-task training ; According to the preset termination conditions, including the training reaching the maximum number of epochs or the total loss on the validation set within the preset number of steps When there is no decline, terminate the training.

[0161] Introducing auxiliary tasks can enhance the text detection model's ability to capture semantic features. Using context prediction as one of the training tasks of the text detection model aims to help the text detection model understand the semantic relationship and contextual information of sentences more deeply. By auxiliary training context tasks, the text detection model can further optimize its ability to capture semantic hierarchical features. This high-quality semantic representation can better support the main task, that is, distinguishing between machine-generated text and human text. If the text detection model only focuses on the main task, it may tend to overfit certain specific patterns in the training data, resulting in a decrease in the generalization ability of new data. The introduction of auxiliary tasks can have a regularization effect, prompting the text detection model to learn more general and robust semantic features.

[0162] The total loss is formed by adding the loss values ​​of the main task and the auxiliary task. The model is trained by optimizing the total loss, which can effectively improve the performance of the text detection model. The main task and the auxiliary task provide supervision signals of different dimensions respectively. By optimizing the total loss, the text detection model can not only strengthen the learning of the main task objectives, but also obtain additional supervision information from the auxiliary tasks, thereby improving the expressiveness of the features. The addition of auxiliary tasks can also guide the text detection model to focus on a wider range of semantic features, avoid over-reliance on local patterns in the training data, and thus improve the generalization ability of unseen data. In addition, the regularization effect provided by the auxiliary tasks can suppress overfitting and make the text detection model more robust. Through this optimization strategy that combines the main task and the auxiliary task, the semantic representation finally generated is more informative, can more comprehensively capture the contextual relationship of the sentence, and provide strong support for the completion of the task.

[0163] Through the above multi-task training, the trained text detection model is obtained, and the text data to be detected is input to obtain the vector representation of the model output, where the first component represents the probability that the input data is written by humans, and the second component represents the probability that the input data is generated by a machine. When the second component is greater than or equal to the first component, the input data is determined to be machine-generated text; otherwise, the input data is determined to be human-written text. The final output probability component provides an intuitive classification basis, which can accurately determine whether the text is written by humans or generated by machines, and is suitable for a wide range of application needs in the field of text generation.

[0164] Embodiment 2:

[0165] An embodiment of the present invention further provides an electronic terminal, characterized in that it includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the machine-generated text detection method described in the above embodiment 1 are executed.

[0166] Embodiment three:

[0167] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the machine-generated text detection method described in the first embodiment are first implemented.

[0168] The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0169] Embodiment 4:

[0170] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the machine-generated text detection method described in Embodiment 1.

[0171] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0175] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting machine-generated text, characterized in that: include: Input the target text to be detected into the trained text detection model to obtain the detection result output by the text detection model; The text detection model includes a first semantic feature extraction network, a second semantic feature extraction network and a fusion classification network; the first semantic feature extraction network is used to extract the semantic features of keywords in the target text and generate a first representation vector; the second semantic feature extraction network is used to extract the overall semantic features of the target text and generate a second representation vector; the fusion classification network processes the first representation vector and the second representation vector to generate a detection result of whether the target text is a machine-generated text; The text detection model is trained using a multi-task learning strategy, including: The main task is to classify whether the target text in the training data is machine-generated text through the fusion classification network and calculate the classification loss value. ; The auxiliary task uses the second semantic feature extraction network to determine whether the sentence pairs in the training data are context sentence pairs and calculate the context discrimination loss value. ; Will and Add together as the total loss for multi-task training , used to optimize the parameters of the text detection model.

2. The machine-generated text detection method according to claim 1, characterized in that: The keywords in the target text are extracted using a term frequency-inverse document frequency (TF-IDF) algorithm, including: Calculate the term frequency TF, inverse document frequency IDF and document frequency DF respectively. The calculation formulas are as follows: ； Where: word frequency TF represents word The ratio of the number of occurrences in document d to the total number of words in document d is used to measure the word The importance of appearing in document d; Represents a word in document d; ； Where: Inverse document frequency IDF is used to measure the word Rarity of occurrence among all documents; Represents the entire document collection; Represents a collection of documents The total number of documents in ; Indicates that it contains words The number of documents; Represents a constant to ensure that the denominator will not be zero; ； Where: Document frequency DF is used to measure word Throughout the document collection The frequency of occurrence in Indicates the word The number of documents; Combining term frequency TF and inverse document frequency IDF, we get the value of term frequency-inverse document frequency TF-IDF, as shown in the following formula: ； According to the distribution of word frequency TF and document frequency DF, define filtering conditions to remove noise words, sort the words that meet the conditions according to the TF-IDF value, and select several words with the highest TF-IDF value as the keywords of the target text.

3. The machine-generated text detection method according to claim 2, characterized in that: The first semantic feature extraction network includes a word embedding layer, a Transformer module and a fully connected layer connected in sequence; The first semantic feature extraction network extracts the semantic features of the keywords in the target text and generates a first representation vector, including: Input the keywords extracted by the TF-IDF algorithm into the word embedding layer to generate the corresponding word vectors; The generated word embedding vector is input into the Transformer module, the attention weight is calculated through the multi-head attention mechanism, the context information is integrated to generate the attention feature representation, and the feature representation is passed to the feedforward neural network; then the feature representation is optimized through residual connection and layer normalization; The optimized feature representation is input into the fully connected layer, extracted and reduced in dimension to generate the first representation vector.

4. The machine-generated text detection method according to claim 1, characterized in that: The second semantic feature extraction network includes a BERT model, a long short-term memory network LSTM module, a self-attention mechanism Attention module and a fully connected layer connected in sequence. The second semantic feature extraction network extracts the overall semantic features of the target text and generates a second representation vector, including: The target text is input into the Tokenizer of the BERT model. The Tokenizer decomposes the text into sub-word units called tokens. Each token is mapped to a token id, and a token id sequence is generated to represent the entire text, as shown in the following formula: ； Where: A sequence of token ids representing text; Indicates special marks used for classification; Indicates the text separator; n indicates the length of the text; Convert the processed token id sequence into a matrix through forward propagation , as shown below: ； Where: Represents the vector corresponding to the nth word; Encode X through the BERT model to obtain the semantic level feature H of the text, as shown in the following formula: ； Where: H is a matrix of dimension n×d, d is the hidden layer dimension of the BERT model; The result H output by the BERT model is input into the LSTM module, and the hidden state is updated using the following formula: ； ； ； ； ； ； ； Where: , , and Represent the outputs of the input gate, forget gate, output gate, and candidate memory unit respectively; , , and Represent the weight matrix of the input gate, the weight matrix of the forget gate, the weight matrix of the output gate, and the weight matrix of the candidate memory unit respectively; , , and They represent the bias term of the input gate, the bias term of the forget gate, the bias term of the output gate, and the bias term of the candidate memory unit respectively; represents the hidden state of the previous time step; represents the input data at time step t; represents the activation function; Indicates the memory status at the current moment; represents the output of the forget gate; Indicates the memory status of the previous step; Represents the hidden state of the current time step; L represents the final output of the LSTM module; The output L of the LSTM module is input into the Attention module. The calculation formula of the attention weight is as follows: ； Where: represents the attention weight of the i-th time step; represents the attention score of the i-th time step; represents the exponential sum of attention scores of all time steps; T represents the total number of time steps of the sequence; Combining the output L of the LSTM module and the attention weight, we get the output A of the Attention module, as shown in the following formula: ； Where: represents the output item at each time step in L; The output L of the LSTM module and the output A of the Attention module are concatenated to obtain the concatenated result Z, which is then transformed through a fully connected layer as shown in the following formula: ； ； Where: represents the transformed vector; represents the weight of the fully connected layer; represents the bias of the fully connected layer; After using the ReLU nonlinear activation function to perform nonlinear activation on the fully connected layer, the final result is output , as shown below: ； ； Where: Represents the vector after nonlinear activation; Indicates the final result; The final result will be output from extract The vector representation corresponding to the tag is used as the vector representation of the entire text to obtain the second representation vector.

5. The machine-generated text detection method according to claim 1, characterized in that: The fusion classification network processes the first representation vector and the second representation vector to generate a detection result of whether the target text is machine-generated text, including: The first representation vector and the second representation vector are concatenated to form the final representation of the text; Input Representation into the fully connected layer to obtain the vector representation output by the text detection model, where the first component represents the probability that the input data is written by humans, and the second component represents the probability that the input data is generated by a machine; When the second component is greater than or equal to the first component, the input data is determined to be machine-generated text.

6. The machine-generated text detection method according to claim 1, characterized in that: Total loss Obtained through the following steps: The fusion classification network is used to classify whether the target text in the training data is machine-generated text, and the classification loss value is calculated using the cross entropy loss function. , as shown below: ； Where: is the true label, indicating whether the input text is written by humans or generated by machines; It is the prediction category of the text detection model, including the probability of predicting that the input text is written by humans and the probability of predicting that the input text is generated by a machine; The second semantic feature extraction network is used to determine whether the sentence pairs in the training data are context sentence pairs, and the cross entropy loss function is used to calculate the context discrimination loss value. , as shown below: ； Where: is the true label, indicating whether the sentence pair belongs to the same context; is the predicted probability output by the second semantic feature extraction network, including the probability of predicting that the input sentence pair belongs to the same context and the probability of belonging to different contexts; The loss of the main task and the loss of auxiliary tasks Add together to get the total loss , as shown below: ； Utilize total loss Back propagation, optimizes the parameters of the text detection model.

7. An electronic terminal, characterized in that: The method comprises a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the machine-generated text detection method according to any one of claims 1 to 6 are executed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the machine-generated text detection method described in any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the machine-generated text detection method described in any one of claims 1 to 6 are implemented.

Images