Multi-dimensional generated text detection method based on user historical data

Abstract

The invention discloses a multi-dimensional generated text detection method based on user historical data. The method comprises the following steps: firstly, performing word segmentation and continuous word merging processing on classified user historical text data, and calculating a frequency probability distribution data set of each text category by applying an N-gram algorithm; thirdly, performing text classification and similarity calculation on the to-be-detected text by using a convolutional neural network algorithm, setting a similarity threshold to obtain similarity data of different types of texts, and determining a text type classification result of the to-be-detected content; and finally, on the basis of N-gram model prediction results under different text types, processing multi-dimensional data of the to-be-detected text during submission, and performing weighted calculation on the multi-dimensional data to obtain a final text generation probability result. According to the method, the generation probability of the to-be-detected text is effectively evaluated, the method for measuring the originality of the text content of the user is provided, and the efficiency and accuracy of text generation detection are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of generated text detection, and in particular relates to a multi-dimensional generated text detection method based on user historical data, which can be used in text originality and text generation detection systems. Background Art

[0002] In the field of information technology, the identification and compliance assessment of texts generated by artificial intelligence (AI) have become a key link in ensuring the integrity and credibility of content. With the innovation of deep learning algorithms and large-scale pre-trained models, the ability of AI text generation has been significantly improved, and its application range has expanded from creative writing to professional document preparation and even scientific research, reflecting the universal applicability and flexibility of the technology.

[0003] However, this progress has also triggered many challenges, especially the abuse of text generation technology, such as forging academic papers and creating false news reports, which not only destroys the purity of the knowledge system, but also may mislead public cognition and cause potential harm to society. In view of this, academia and industry are committed to studying advanced identification mechanisms to accurately distinguish between human-created and machine-generated content, so as to maintain the health and security of the information ecosystem and build a reliable information verification system, which has become a focus of current research.

[0004] The existing research foundation of Xu Wen et al. includes: Pan S, Fan J, Yu S, et al. A Method of Behavior Evaluation Based on Web Browsing Information. Smart Grid and Electrical Automation (ICSGEA), 2017 International Conference on. IEEE, 2017: 697-700. Related patents applied, published and authorized by Xu Wen et al.: Zhu Quanyin, Pan Shuxin, Li Xiang, Jia Xiaoying et al., A multi-dimensional web browsing behavior evaluation method. Chinese Patent Publication Number: CN106886577A, 2017.06.23.

[0005] Analysis of existing technologies:

[0006] Statistical feature analysis is one of the earliest technologies used in AI text detection. This type of method mainly relies on statistical characteristics of text, such as vocabulary frequency, grammatical structure, and syntactic patterns. For example, the word frequency-inverse document frequency (TF-IDF) and the proportion of stop words in the text are calculated, and feature vectors are constructed through these statistical indicators, and then classified using machine learning algorithms such as support vector machines (SVM) and random forests (RF). With the maturity of deep learning technology, neural networks have begun to be widely used in AI text detection. Models such as convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and transformers can capture the deep semantic features of text, thereby more accurately identifying AI-generated text. In particular, the Transformer architecture, due to its powerful sequence modeling capabilities and parallel computing advantages, performs well in text generation and detection. With the development of multimodal AI technology, comprehensive detection using multimodal information (such as images, audio, etc.) has become a research hotspot. By fusing multimodal data, the accuracy and robustness of AI text generation detection can be improved.

[0007] Convolutional Neural Networks:

[0008] Convolutional Neural Network (CNN) is a feedforward neural network whose artificial neurons can respond to surrounding units within a certain coverage area, and performs well in large image processing and text classification. Convolutional neural networks consist of one or more convolutional layers and a fully connected layer at the top (corresponding to the classic neural network), as well as associated weights and pooling layers. This structure enables convolutional neural networks to exploit the two-dimensional structure of the input data. Compared with other deep learning structures, convolutional neural networks can give better results in image and speech recognition. This model can also be trained using the backpropagation algorithm. Compared with other deep, feedforward neural networks, convolutional neural networks require fewer parameters to consider, making them an attractive deep learning structure.

[0009] N-gram algorithm:

[0010] The N-gram algorithm is a statistical language model that is widely used in natural language processing (NLP). It is mainly used to predict the probability of a word appearing in a text, and then achieve various tasks such as language model construction, text classification, machine translation, speech recognition, etc. Its core idea is based on the observation that the appearance of a word is often closely related to the previous words. Therefore, by analyzing the combination frequency of adjacent words in a word sequence, the N-gram model can capture the local statistical laws in the text.

[0011] Traditional text originality detection technology mainly relies on database retrieval, and its core lies in comparing the repetition between the submitted text and the stored corpus to evaluate the originality of the text or identify whether it is synthesized by an artificial intelligence system. However, this method faces significant limitations, especially when faced with advanced large-scale pre-trained models represented by Generative Pre-trained Transformer (GPT). The text generation capabilities of these models can not only simulate a wide range of human writing styles, but also make fine modifications to the initial text according to user instructions, thereby significantly reducing the repetition rate of the text. This feature makes the traditional originality detection method - based on the statistics of the repetition rate of text fragments - ineffective. Summary of the invention

[0012] Purpose of the invention: The present invention introduces user-based historical data, combined with the N-gram analysis framework and comprehensive consideration of multi-dimensional features, to provide researchers with an effective detection method for evaluating the reliability of text and judging the originality of its content.

[0013] Technical solution: The present invention proposes a multi-dimensional generated text detection method based on user historical data, comprising the following steps:

[0014] Step 1: Segment and merge consecutive words of the classified user historical text data, and use the N-gram algorithm to calculate the frequency probability distribution data set CNF corresponding to different text categories;

[0015] Step 2: Segment the text to be classified, and remove the data in the stop word set to obtain the word set of the text to be classified. Use the convolutional neural network algorithm to classify the text to be detected, calculate the similarity of the text to be detected under different categories, set the similarity threshold, and obtain the text type of the text to be detected;

[0016] Step 3: Use the frequency probability distribution data set CNF corresponding to different text categories in step 1 to calculate the CNF prediction result of the text type corresponding to the text to be detected; process the multi-dimensional data when the text to be detected is uploaded, and perform weighted calculation based on the multi-dimensional data to obtain the final text generation probability result of the text to be detected.

[0017] Furthermore, the specific method of step one is:

[0018] Step 1.1: Define the corpus set and the segmentation corpus set;

[0019] Step 1.2: Define the smoothing parameter Alpha and the number of consecutive times of the sequence N, merge the segmented word corpus set into a continuous sub-segment of length N, and obtain the continuous word corpus set;

[0020] Step 1.3: Count the word frequencies in the continuous word corpus obtained in step 1.2, calculate the N-gram probability of each word based on the smoothing parameter Alpha, and obtain the frequency probability distribution data set under different categories.

[0021] Furthermore, the specific method of step 2 is:

[0022] Step 2.1: Define the text to be classified and the set of stop words in the text;

[0023] Step 2.2: Segment the text to be classified, and remove the data in the stop word set to obtain the word set of the text to be classified;

[0024] Step 2.3: Use the convolutional neural network algorithm to classify the text to be classified and obtain the text similarity under different categories;

[0025] Step 2.4: Define the similarity threshold K;

[0026] Step 2.5: Traverse all types, save the types corresponding to those greater than the similarity threshold, and obtain the text type set to be detected.

[0027] Furthermore, the specific method of step three is:

[0028] Step 3.1: Obtain the text type set of the text to be detected and the corresponding frequency probability distribution data set CNF;

[0029] Step 3.2: Combine the word set of the text to be classified into a continuous sub-segment of length N to obtain a continuous word set of the text to be detected;

[0030] Step 3.3: Use the frequency probability distribution data set CNF corresponding to different text categories in step 1 to calculate the continuous word set of the text to be detected obtained in step 3.2;

[0031] Step 3.4: Numerize and normalize the multi-dimensional data when the text to be detected is uploaded to obtain computable multi-dimensional elements;

[0032] Step 3.5: Use the multi-dimensional text detection formula to perform weighted calculation to obtain the text generation probability result of the text to be detected.

[0033] Furthermore, the specific calculation method of the text generation probability result of the text to be detected in step 3.5 is:

[0034] The multi-dimensional data is set to include the web page dwell time as WebTime, the mouse sliding range as MouseZone, the web page sliding range as TotalZone, and the text input area as InputZone; the text generation probability result of the text to be detected is:

[0035]

[0036] Among them, InputZone has two variables: 0 for input, 1 for input not allowed, k represents the number of types in the similarity set of the text type to be classified, SCTJ i Represents a set of consecutive prediction probabilities of the text to be detected.

[0037] The present invention adopts the above technical scheme, and has the following beneficial effects: by using the N-gram algorithm, and combining the multi-dimensional data when the text to be detected is obtained, the text generation probability is evaluated, the content credibility of the text to be detected is effectively increased, and the difference between the text to be detected and the generated text is quantified. Specifically, the present invention first performs word segmentation and continuous word merging processing on the text data classified by the user, and uses the N-gram algorithm to calculate the frequency probability distribution data set corresponding to different text categories. Then, through the convolutional neural network algorithm, the text to be detected is classified and similarity is calculated, and the similarity threshold is set to obtain the similarity results under different types of texts respectively, and then the text type classification data of the content to be detected is obtained. Finally, the N-gram model prediction results of the article to be detected under different text types are calculated, the multi-dimensional data when the content to be detected is provided is processed, and the designed text generation detection algorithm is input to obtain the final text generation probability result. In addition, the present invention proposes a multi-dimensional generated text detection algorithm, which improves the accuracy of each weight of the confidence of the text to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Generate a flowchart of the outline of the text detection method for multiple dimensions;

[0039] Figure 2 Flowchart of corpus segmentation and calculation for N-gram algorithm;

[0040] Figure 3 Flowchart for text classification to be detected;

[0041] Figure 4 This is the multi-dimensional calculation flow chart of the text to be tested. DETAILED DESCRIPTION

[0042] The present invention is further illustrated below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0043] The present invention discloses a multi-dimensional generated text detection method based on user historical data, which is described by taking the text types involving finance, Internet, health, education, military, tourism, sports, culture and other text sets as examples, as follows:

[0044] Step 1: Segment and merge consecutive words of the classified user history text data, and use the N-gram algorithm to calculate the frequency probability distribution data set CNF corresponding to different text categories. Figure 2 As shown:

[0045] Step 1.1: Define CATE1, CATE2, CATE3, CATE4, CATE5, CATE6, CATE7, CATE8, CATE9 as the text sets of finance, Internet, health, education, military, tourism, sports, culture and others respectively, and define Corpus as the user's historical text library set, Corpus = {CATE1, CATE2, ..., CATE9}, CATE i ={txt i-1 ,txt i-2 ,…,txt i-n}, where i∈[1,9], txt i-n CATE i The nth text in the text set.

[0046] Step 1.2: CATE i Perform word segmentation and obtain the corresponding word segmentation material set SCorpus = {SCATE1, SCATE2, ..., SCATE9}, SCATE i ={swd i-1 ,swd i-2 ,…,swd i-m}, where i∈[1,9], swd i-m For SCATE i The mth word in the text set.

[0047] Step 1.3: Define the smoothing parameter Alpha and the number of consecutive words in the sequence as N, and assign initial values ​​to 0.3 and 2.

[0048] Step 1.4: SCATAE i Merge into continuous word segments of length N, and obtain the corresponding continuous word corpus set LSCorpus = {LSCATE1, LSCAATE2, ..., LISCATE9}, LSCATE i ={lswd i-1 ,lswd i-2 ,…,lswd i-k}, where i∈[1,9], lswd i-kFor SCATE i The kth consecutive word in a text set.

[0049] Step 1.5: LSCATE statistics i The frequency of each word in the text is calculated, and the smoothing parameter Alpha is added to the frequency result to obtain the category-frequency probability distribution data set CNF = {CNF1, CNF2, …, CNF9}, where i∈[1,9].

[0050] Step 2: Use the convolutional neural network algorithm to classify the text to be detected, calculate the similarity of the text to be detected under different categories, set the similarity threshold, and obtain the text type of the text to be detected. Figure 3 As shown:

[0051] Step 2.1: Define the text to be classified SCT.

[0052] Step 2.2: Define the text stop word set SWORD = {sword1, sword2, ..., word sw}.

[0053] Step 2.3: Segment the text to be classified to obtain the word set SCTW = {sc1, sc2, ...sc e}.

[0054] Step 2.4: Eliminate SOWRD from SCTW to obtain a new text classification word set NSCTW = SCTW-SOWRD;

[0055] Step 2.5: Define cct1, cct2, cct3, cct4, cct5, cct6, cct7, cct8, cct9 as finance, Internet, health, education, military, tourism, sports, culture and other text types respectively, and define CCT as the text classification type set, CCT = {cct1, cct2,…, cct9}.

[0056] Step 2.6: Use the convolutional neural network algorithm to classify the NSCTW in step 2.4 according to the classification type of CCT, and obtain the corresponding text classification similarity set SCTC = {sctcc1, sctcc2…, sctcc i}, where i∈[1,9], sctcc i The text type is cct i probability.

[0057] Step 2.7: Define the loop variable k for traversing SCTC and assign it an initial value of 0.7.

[0058] Step 2.8: Define the loop variable scount to traverse SCTC and assign it an initial value of 1.

[0059] Step 2.9: When scount<=9, execute step 2.10; otherwise, execute step 2.12.

[0060] Step 2.10: When sctcc scount >=0.7, execute step 2.11; otherwise, execute step 2.12.

[0061] Step 2.11: Place stcc scount Stored in the similarity set SCTCCA of the text type to be classified.

[0062] Step 2.12: scount=scount+1.

[0063] Step 2.12: Get the similarity set of the text type to be classified SCTCCA = {sctcca1, sctcca2, ... sctcca k}.

[0064] Step 3: Calculate the CNF prediction results of the text to be detected under different text types, process the multi-dimensional data when the text to be detected is uploaded, put it into the text generation detection algorithm, and obtain the final text generation probability result of the text to be detected. Figure 4 As shown:

[0065] Step 3.1: Merge the text classification word set NSCTW into continuous word segments of length N to obtain the text continuous word set NNSCTW to be detected.

[0066] Step 3.2: Define the loop variable p for traversing SCTCCA and assign it an initial value of 1.

[0067] Step 3.3: When p is less than or equal to k, execute step 3.4; otherwise, execute steps 3 and 7.

[0068] Step 3.4: Extract the text type set sctcca to be classified p The corresponding category-frequency probability distribution dataset CNF p .

[0069] Step 3.5: Using CNF p Calculate NNSCTW. If the subsequent word of NNSCTW is the same as the subsequent word of CNF, it is marked as 1, otherwise it is marked as 0, and the continuous prediction probability set SCTJ of the text to be detected is obtained. p ={sctj1,sctj2,…,sctj g}.

[0070] Step 3.6: p=p+1.

[0071] Step 3.7: Define the web page dwell time when the text to be detected is uploaded as WebTime, the mouse sliding range is MouseZone, the web page sliding range is TotalZone, and the text input area is InputZone. Among them, InputZone has two variables: 0 for input, 1 for non-input;

[0072] Step 3.8:

[0073] Step 3.9: Get the AI ​​text generation probability result F1.

[0074] In order to better verify the effectiveness of this method, the text content of 200 users was randomly selected for analysis. First, the text sets of finance, Internet, health, education, military, tourism, sports, culture and other types that have been classified by 200 users were N-gram calculated to obtain the conditional frequency distribution (CNF) of each user under different text classifications. Next, 200 users were required to provide 3 original texts and 1 AI text generated by a large model. Then, these 800 texts were classified, and the classification threshold was set to 0.7 to determine the classification set of each text. After that, the prediction set of the text to be detected was calculated using the CNF of each user, and the AI ​​generation probability of the text to be detected was calculated in combination with the multi-dimensional data obtained when the user uploaded the text. Through the manual judgment method, 98 AI-generated texts were identified, accounting for 49% of the total, and it took 2 hours. Using the method steps of the present invention, 192 AI-generated texts were identified, accounting for 96% of the total, which was 47% higher than the accuracy of manual judgment; it took 45 minutes, which was 2.76 times less than the manual judgment time.

[0075] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-dimensional generated text detection method based on user historical data, characterized in that: The steps include: Step 1: Segment and merge consecutive words of the classified user historical text data, and use the N-gram algorithm to calculate the frequency probability distribution data set CNF corresponding to different text categories; Step 2: Segment the text to be classified, and remove the data in the stop word set to obtain the word set of the text to be classified. Use the convolutional neural network algorithm to classify the text to be detected, calculate the similarity of the text to be detected under different categories, set the similarity threshold, and obtain the text type of the text to be detected; Step 3: Use the frequency probability distribution data set CNF corresponding to different text categories in step 1 to calculate the CNF prediction result of the text type corresponding to the text to be detected; process the multi-dimensional data when the text to be detected is uploaded, and perform weighted calculation based on the multi-dimensional data to obtain the final text generation probability result of the text to be detected.

2. The multi-dimensional generated text detection method based on user historical data according to claim 1 is characterized in that: The specific method of step one is: Step 1.1: Define the corpus set and the segmentation corpus set; Step 1.2: Define the smoothing parameter Alpha and the number of consecutive times of the sequence N, merge the segmented word corpus set into a continuous sub-segment of length N, and obtain the continuous word corpus set; Step 1.3: Count the word frequencies in the continuous word corpus obtained in step 1.2, calculate the N-gram probability of each word based on the smoothing parameter Alpha, and obtain the frequency probability distribution data set under different categories.

3. The multi-dimensional generated text detection method based on user historical data according to claim 1 is characterized in that: The specific method of step 2 is: Step 2.1: Define the text to be classified and the set of stop words in the text; Step 2.2: Segment the text to be classified, and remove the data in the stop word set to obtain the word set of the text to be classified; Step 2.3: Use the convolutional neural network algorithm to classify the text to be classified and obtain the text similarity under different categories; Step 2.4: Define the similarity threshold K; Step 2.5: Traverse all types, save the types corresponding to those greater than the similarity threshold, and obtain the text type set to be detected.

4. The multi-dimensional generated text detection method based on user historical data according to claim 1 is characterized in that: The specific method of step three is: Step 3.1: Obtain the text type set of the text to be detected and the corresponding frequency probability distribution data set CNF; Step 3.2: Combine the word set of the text to be classified into a continuous sub-segment of length N to obtain a continuous word set of the text to be detected; Step 3.3: Use the frequency probability distribution data set CNF corresponding to different text categories in step 1 to calculate the continuous word set of the text to be detected obtained in step 3.2; Step 3.4: Numerize and normalize the multi-dimensional data when the text to be detected is uploaded to obtain computable multi-dimensional elements; Step 3.5: Use the multi-dimensional text detection formula to perform weighted calculation to obtain the text generation probability result of the text to be detected.

5. The multi-dimensional generated text detection method based on user historical data according to claim 4 is characterized in that: The specific calculation method of the text generation probability result of the text to be detected in step 3.5 is: The multi-dimensional data is set to include the web page dwell time as WebTime, the mouse sliding range as MouseZone, the web page sliding range as TotalZone, and the text input area as InputZone; the text generation probability result of the text to be detected is: Among them, InputZone has two variables: 0 for input, 1 for input not allowed, k represents the number of types in the similarity set of the text type to be classified, SCTJ i Represents a set of consecutive prediction probabilities of the text to be detected.

Images