Method and system for detecting fake news generated by transformer-based large language model

Abstract

This application relates to a method and system for detecting fake news generated by a large language model based on Transformer. The method preprocesses a mixed news dataset, optimizes the RoBERTa model in two stages (domain adaptation and task fine-tuning) to extract deep semantic features, and simultaneously obtains multidimensional statistical features of the text through an enhanced feature extraction tool. These two types of features are input into a cross-modal attention fusion module, where feature interaction is achieved through a bidirectional attention mechanism, and dynamic weighted fusion is performed using gating units to form a unified feature representation. A classifier is trained based on this representation, ultimately constructing a fake news detection model. This approach significantly improves the accuracy, robustness, and cross-domain generalization ability for identifying fake news generated by a large language model through deep fusion and adaptive integration of semantic and statistical features.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method and system for detecting fake news based on a large language model generated by Transformer. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models based on the Transformer architecture are now capable of generating grammatically fluent, logically coherent, and stylistically realistic text. This has driven productivity advancements, but it has also provided a powerful tool for the automated and large-scale generation of fake news. Unlike traditionally fabricated misinformation, fake news generated by large language models often approaches or even surpasses human writing quality in terms of text quality. This makes it difficult for the public and traditional detection systems to effectively distinguish them using common sense or simple rules, posing a serious challenge to the online information ecosystem and social trust.

[0003] Currently, fake news detection technologies are evolving primarily in two directions. The first is based on traditional machine learning methods, which heavily rely on expert-designed feature engineering, such as extracting shallow statistical features like vocabulary, syntax, or sentiment, and then combining these with classifiers for judgment. However, these methods struggle to capture deep semantic connections and contextual logic within the text, resulting in a significant drop in recognition rates when faced with AI-generated text that highly mimics human writing patterns. The second approach is based on deep learning methods, particularly fine-tuning pre-trained models like BERT and RoBERTa, leveraging their powerful contextual semantic understanding capabilities for classification. While these methods offer clear advantages in semantic understanding, their "black box" nature makes the decision-making process difficult to interpret, and they often overlook quantifiable and interpretable statistical patterns on the surface of the text—statistical features that have proven to be crucial clues revealing the source of text generation.

[0004] More importantly, most existing research focuses on the general problem of distinguishing between "AI-generated text" and "human-written text," rather than on the more socially harmful and difficult-to-detect specific task of distinguishing between "AI-generated fake news" and "human-written real news."

[0005] Furthermore, the effectiveness and robustness of existing detection models in the face of fake news generated by large language models that employ different prompting strategies and cover different domains have not been fully verified. The models are prone to overfitting to specific patterns in the training data, resulting in insufficient generalization ability in practical applications.

[0006] Therefore, there is an urgent need for a detection scheme that can deeply integrate deep semantic understanding and shallow statistical features, and has stronger generalization ability and a certain degree of interpretability, so as to accurately and reliably identify fake news generated by large language models. Summary of the Invention

[0007] Therefore, it is necessary to provide a method and system for detecting fake news based on a large language model generated by Transformer to address the aforementioned technical problems.

[0008] Firstly, this application provides a method for detecting fake news based on a large language model generated by Transformer, including:

[0009] S1. Obtain the original dataset containing real news text and fake news text generated by the big language model, and perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset.

[0010] S2. Based on general news domain corpus, the RoBERTa pre-trained model is fine-tuned for domain adaptation to obtain the domain-adapted model; based on the training dataset, the domain-adapted model is fine-tuned for task-specific purposes to obtain the fine-tuned semantic encoder.

[0011] S3. Use the fine-tuned semantic encoder to encode the text data in the training dataset to generate a deep semantic feature vector for each news text.

[0012] S4. Based on the enhanced statistical feature extraction tool, batch feature calculation is performed on the text data in the training dataset to obtain multidimensional statistical feature vectors; among them, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for the text generated by the large language model;

[0013] S5. Input the deep semantic feature vector and the multidimensional statistical feature vector into the multi-granularity cross-modal attention fusion module, establish the interaction between features through the bidirectional cross-attention mechanism, and dynamically weight and fuse the interactive features based on the gating mechanism to generate a unified fused feature representation.

[0014] S6. Use fused feature representations to train the classifier network to obtain a trained fake news detection model;

[0015] S7. Based on the trained fake news detection model, process the fusion feature representation corresponding to the news text to be detected, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the fusion feature representation to be processed is obtained in the same way as the fusion feature representation.

[0016] Secondly, this application also provides a fake news detection system based on a large language model generated by Transformer, for implementing the method described in the first aspect, the system comprising:

[0017] The data preprocessing optimization module is used to obtain the original dataset containing real news text and fake news text generated by the big language model, and to perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset.

[0018] The domain adaptation model building module is used to fine-tune the RoBERTa pre-trained model based on general news domain corpus to obtain the domain adaptation model; and to fine-tune the domain adaptation model based on the training dataset to obtain the fine-tuned semantic encoder.

[0019] The deep semantic encoding module is used to encode the text data in the training dataset using a fine-tuned semantic encoder to generate a deep semantic feature vector for each news text.

[0020] The multidimensional statistical feature extraction module is used to perform batch feature calculations on text data in the training dataset based on the enhanced statistical feature extraction tool to obtain multidimensional statistical feature vectors. Among them, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for text generated by large language models.

[0021] The cross-modal feature fusion module is used to input deep semantic feature vectors and multi-dimensional statistical feature vectors into the multi-granularity cross-modal attention fusion module. It establishes interaction between features through a bidirectional cross-attention mechanism and performs dynamic weighted fusion of the interacting features based on a gating mechanism to generate a unified fused feature representation.

[0022] The classifier training and optimization module is used to train the classifier network using fused feature representations to obtain a trained fake news detection model.

[0023] The real-time detection and judgment module is used to process the unprocessed fusion feature representation corresponding to the news text to be detected based on the trained fake news detection model, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the unprocessed fusion feature representation is obtained in the same way as the fusion feature representation.

[0024] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a fake news detection method based on a large language model generated by Transformer as described in the first aspect.

[0025] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a fake news detection method based on a large language model generated by Transformer as described in the first aspect.

[0026] The aforementioned method and system for detecting fake news based on a Transformer-based large language model acquires and preprocesses a text dataset containing both real and fake news. It then fine-tunes a RoBERTa pre-trained model in two stages—domain adaptation and task-specific fine-tuning—using general news corpora and specific training data to obtain an encoder capable of capturing the deep semantic authenticity of news and generating deep semantic feature vectors. Simultaneously, an enhanced statistical feature extraction tool, integrating basic statistical indicators and custom indicators designed for text generated by the large language model, extracts multi-dimensional statistical feature vectors in batches from the same text data. These two types of features are then input into a multi-granularity cross-modal attention fusion module. A bidirectional cross-attention mechanism establishes deep interactions between features, and a gating mechanism dynamically weights and fuses the interacting features, forming a unified and complementary fused feature representation. This fused feature representation is used to train a classifier network, ultimately constructing a fake news detection model capable of automatically judging the news text to be detected. This approach significantly improves the accuracy and robustness of identifying fake news generated by large language models by deeply integrating deep semantic understanding with shallow statistical patterns and using attention and gating mechanisms to achieve adaptive feature integration. It also enhances the model's generalization ability when facing different text generation strategies and diverse news domains. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a method for detecting fake news based on a large language model generated by Transformer, provided by this invention;

[0029] Figure 2 This is a schematic diagram illustrating the process of generating a unified fusion feature representation in one optional embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the structure of a fake news detection system based on a large language model generated by Transformer, provided by the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] refer to Figure 1 The document presents a flowchart illustrating a method for detecting fake news based on a large language model generated using Transformer, as provided in this application. The method includes the following steps:

[0033] S1. Obtain the original dataset containing real news text and fake news text generated by the large language model, and perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset.

[0034] Specifically, the core of this step is to construct a dataset that combines authenticity, diversity, and accurate annotation. Standardized preprocessing is used to eliminate text noise and unify data formats, providing high-quality input for subsequent model training. The construction of the original dataset requires a strict distinction between real news text and fake news text generated by the large language model, ensuring a balanced data distribution. The sources of real news text must be authoritative and diverse. News reports covering multiple core areas can be obtained by calling the public API interfaces of authoritative international media outlets. Simultaneously, a subset of real news from publicly labeled datasets is integrated, with selection criteria including complete text information, traceable sources, and no factual errors verified by human verification. Fake news text generated by the large language model needs to be generated using a multi-model, multi-prompt strategy to cover the characteristics of different generation patterns. Mainstream large language models are selected as the generation tool, and three types of prompt word templates are designed to ensure that fake news is comparable to real news in terms of domain distribution, text length, and writing style, avoiding data bias.

[0035] The standardized text cleaning process requires a step-by-step approach to eliminate invalid information and formatting differences in the text. First, regular expressions are used to match and remove HTML tags, URL links, special characters, and redundant spaces, while retaining Chinese / English text, numbers, and standard punctuation. Second, text standardization is performed, converting all English characters to lowercase, standardizing punctuation, and correcting English spelling errors and typos in Chinese text. Finally, text length filtering is applied to filter out excessively short or long texts. The rationale is that excessively short texts lack sufficient semantic and statistical features, while excessively long texts increase the computational cost of the model and easily introduce redundant information.

[0036] Sub-word serialization is implemented using the Byte-Pair Encoding segmenter based on the RoBERTa pre-trained model. The vocabulary corresponding to the RoBERTa pre-trained weights is loaded, and the cleaned text is segmented sentence by sentence. For out-of-vocabulary words, the Byte-Pair Encoding algorithm is used to split them into already included sub-word units. Special tokens are added to the beginning and end of the segmented sequence to aggregate global semantics and sentence separation, generating a complete token sequence. The token sequence is converted into corresponding token IDs, and a padding operation is used to unify the length of all sequences. Sequences exceeding the maximum length are truncated to ensure consistent input dimensions.

[0037] For text length statistics, the core metric is the number of words per text, and the specific formula is as follows: .in This indicates the total number of words in a single text. This indicates the total number of sentences in the text. Let represent the word count of the i-th sentence. This formula is used to filter text samples that meet the length requirement. Finally, the processed dataset is divided into training, validation, and test sets proportionally using stratified sampling, stored in tensor format, and corresponding label vectors are generated to obtain a standardized preprocessed training dataset.

[0038] S2. Based on the general news domain corpus, the RoBERTa pre-trained model is fine-tuned for domain adaptation to obtain the domain-adapted model; based on the training dataset, the domain-adapted model is fine-tuned for task-specific purposes to obtain the fine-tuned semantic encoder.

[0039] Specifically, this step employs a two-step strategy of domain-adaptive fine-tuning and task-specific fine-tuning to gradually adapt the RoBERTa model from general language understanding capabilities to the characteristics of the news domain and the fake news detection task. This ensures the model can accurately capture the semantic features of news texts and the unique patterns of misinformation. The core objective of domain-adaptive fine-tuning is to enable the pre-trained model to learn the specific vocabulary, sentence structures, and semantic logic of the news domain, reducing the distributional differences between general and news corpora. The selected general news domain corpora need to be large-scale and diverse, integrating news corpora from multiple sources and covering all object detection domains.

[0040] The fine-tuning process uses a masked language model as the pre-training task. The core of the masked language model task is to predict the original value of the masked token, and its loss function is cross-entropy loss, with the specific formula as follows: .in This represents the loss value for the masked language modeling task. This represents the total number of masked tokens in a single batch of samples. Indicates the size of the model vocabulary. This represents the vocabulary corresponding to the i-th mask token. The true label, This indicates that the model predicts the i-th mask token as a word. The probability is calculated using the softmax function, with the specific formula being: .in This indicates the vocabulary corresponding to the i-th mask token in the model output layer. The unnormalized score, This represents the natural exponential function. This represents the index of a word in the vocabulary list.

[0041] The fine-tuning process uses AdamW as the optimizer, and its parameter update formula is as follows:

[0042]

[0043] Among them, the deviation correction value of the first momentum This is a value obtained after correcting for the bias in the first-order momentum of the model parameter gradient (i.e., the exponential moving average of the gradient). It is used to eliminate the bias caused by the initial value of the first-order momentum being 0 in the early stage of training. Its core calculation logic is to correct it by weighting the historical gradient information and the current gradient. , Let represent the first-order momentum at step t. , Let represent the first-order momentum at step t-1. . Model parameters .

[0044] The training process employs a linear learning rate decay strategy, with the learning rate decay formula being: .in This represents the learning rate at step t. This represents the initial learning rate. Indicates the number of warm-up steps. This represents the total number of training steps. This represents the function that takes the maximum value.

[0045] During model training, some Transformer layer parameters of RoBERTa are frozen, and only the parameters of the remaining Transformer layers and pooling layers are fine-tuned. The preprocessed news corpus is input, and the tokens are randomly masked proportionally. The original tokens at the mask positions are predicted by the model, the cross-entropy loss is calculated, and the parameters are updated by backpropagation. Finally, a domain-adapted model for the news domain is obtained. This model is significantly better than the general pre-trained model in terms of semantic understanding of news texts.

[0046] Task-specific fine-tuning aims to further optimize the domain-adaptive model into a semantic encoder focused on fake news detection. The core principle is to use labeled training data to teach the model discriminative features between real news and fake news generated by large language models. The fine-tuning process targets a binary classification task, using the constructed training set as input. An attention mask is used to instruct the model to ignore the positions corresponding to pad tokens, avoiding interference from invalid information.

[0047] The loss function for the binary classification task is weighted cross-entropy loss, as shown in the formula below. .in This represents the loss value for a binary classification task. Indicates batch size, This represents the true label of the i-th sample. This represents the probability that the model predicts the i-th sample as fake news generated by the large language model. Indicates the weighting coefficient. This represents the natural logarithm function.

[0048] The model's predicted probability is calculated using the sigmoid activation function, specifically using the following formula: .in This represents the sigmoid activation function. This represents the unnormalized score of the model's output layer for the i-th sample.

[0049] In terms of model structure, all Transformer layer parameters are unfrozen, allowing the model to adjust its semantic extraction capabilities for the detection task. A temporary fully connected layer is added after the RoBERTa special token output to map the global semantic representation to the binary classification space. The calculation formula for this fully connected layer is as follows: .in This represents the output of the fully connected layer. This represents the weight matrix of the fully connected layer. The hidden state vector representing a special token. This represents the bias vector of the fully connected layer.

[0050] During training, accuracy, precision, recall, and F1 score are calculated on the validation set after each round. The formula for calculating the F1 score is as follows: .in Indicates the F1 score. Indicates accuracy. Recall is the percentage of data collected. Precision is calculated using the following formula: .in This represents the number of true positive samples, i.e., the number of samples that were predicted to be fake news and were actually fake news. Recall represents the number of false positives, i.e., the number of samples that were predicted to be fake news but were actually real news. The formula for calculating recall is... .in This represents the number of false negative samples, i.e., the number of samples that were predicted to be true news but were actually false news.

[0051] The loss curve and evaluation metric changes are recorded using tools. If overfitting is observed, the dropout rate of the Dropout layer is adjusted accordingly. The calculation logic for the Dropout layer is as follows: .in This represents the output of the Dropout layer. Indicates the discard rate. This represents a binary mask vector with the same dimension as the input vector. This represents the input vector of the Dropout layer.

[0052] Finally, the model with the highest F1 score on the validation set is saved as the fine-tuned semantic encoder, which can output a deep semantic representation that integrates news domain characteristics and fake information discrimination features.

[0053] S3. Use the fine-tuned semantic encoder to encode the text data in the training dataset to generate a deep semantic feature vector for each news text.

[0054] Specifically, the core of this step is to encode the training text using a fine-tuned semantic encoder, extracting high-dimensional feature vectors that can represent the deep semantic logic and misinformation patterns of the text. The encoding process follows the model input specification, batching the preprocessed training dataset into the fine-tuned semantic encoder. Token type IDs are used to distinguish single-sentence texts, and the attention mask is a binary vector with the same dimension as the token ID sequence, where valid token positions are 1 and pad token positions are 0, ensuring the model focuses only on the true text content.

[0055] The fine-tuned RoBERTa semantic encoder architecture comprises multiple Transformer modules, an embedding layer, and a pooling layer. The embedding layer converts the token ID into a fixed-dimensional word embedding vector. The formula for calculating the word embedding is as follows: .in This represents the word embedding vector of the i-th token. The word embedding weight matrix represents the word embedding weight matrix. This represents the vector after one-hot encoding of the token ID. This represents the ID of the i-th token. Represents the word embedding bias vector.

[0056] The input to the Transformer layer is then processed using layer normalization and dropout. The formula for layer normalization is as follows: .in The output after normalization of the representation layer, Represents the input vector. and This represents the learnable scaling and offset parameters. This represents the mean of the input vector. This represents the variance of the input vector. This represents a numerical stability parameter to avoid a denominator of 0.

[0057] Each Transformer layer consists of a multi-head self-attention sublayer and a feedforward neural network sublayer. The multi-head self-attention sublayer contains multiple attention heads, which capture long-distance dependencies and contextual semantic associations between tokens by calculating the attention weights of each token with all other tokens in the text. The complete computation process of multi-head self-attention first maps the input vector to a query vector, a key vector, and a value vector through three independent linear projection layers, with the projection formulas as follows: , , .in Represents the query vector. Represents the key vector. Represents a value vector. , , These represent the weight matrices of the three projection layers, respectively. , , These represent the bias vectors of the three projection layers, respectively.

[0058] The query vector, key vector, and value vector are then split according to the number of attention heads. The core purpose of this splitting is to allow each attention head to independently capture semantic association information of different dimensions in the text, achieving parallel extraction of multi-granular semantic features. The specific splitting method combines the number of attention heads and vector dimensions: First, the total number of attention heads is defined as follows: The projected query vector Key vector Value vector Total dimension (denoted as) ) are divided into Divide into equal parts, each with dimensions of (satisfy , (for the dimension of a single attention head); then targeting , , The vectors are split along each column dimension to obtain the sub-vectors corresponding to each attention head. The splitting formulas are as follows: , , The index in the formula is " The colon (:) is a syntax for extracting column dimensions of a vector / matrix. It is used to locate and extract the feature dimensions corresponding to each attention head. The meanings of its parts are broken down as follows: ":" indicates that all row dimensions of the vector / matrix are retained (because...). , , All are two-dimensional tensors, with dimensions of "number of samples × total feature dimension". The comma ("" corresponds to the sample dimension and must be fully retained to ensure the feature integrity of each sample) separates the row index from the column index, explicitly specifying that the subsequent expression truncates the column dimension. "" indicates the cutoff range for the column dimension, where " "This is the starting position of the interval (including the column corresponding to that position)." " represents the end position of the interval (excluding the column corresponding to that position), and the interval length is exactly 1. (i.e., the feature dimension of a single attention head) to ensure that the dimension of each segment of the extracted sub-vector is consistent.

[0059] To further clarify this, let's illustrate with a specific parameter example: Assume the total number of attention heads... Total feature dimensions Then the single attention head dimension .when (Attention point) When the first attention is drawn, the selected interval is ,Right now This corresponds to extracting features from columns 0 to 63 (a total of 64 columns) of the vector; when (Attention point 2) When the interval is selected, it is... ,Right now This corresponds to extracting features from columns 64 to 127; and so on. At that time, column 128:192 was extracted. The original projection vector (columns 0:256) is completely restored after the subvectors of the four attention heads are concatenated, with no feature loss or redundancy.

[0060] This represents the query subvector corresponding to the j-th attention head. This represents the key vector corresponding to the j-th attention head. This represents the subvector of values ​​corresponding to the j-th attention head. Indicates the attention head index (value range is 1 to 1). By precisely extracting the index expression, it can be ensured that each attention head obtains a feature sub-vector that is non-overlapping and has the same dimension. This lays the foundation for each attention head to independently perform semantic association calculations, ensuring that the multi-head self-attention mechanism can capture multi-dimensional semantic information of the text in parallel. At the same time, it ensures that all sub-vectors can be restored to the original projection vector after being concatenated, thus ensuring the consistency and integrity of subsequent attention calculations.

[0061] The attention score for a single attention head is calculated as follows: .in This represents the original attention score matrix corresponding to the j-th attention head. Let represent the transpose matrix of the key vector corresponding to the j-th attention head. This represents a scaling factor used to prevent the softmax function from saturating due to excessively large scores. This represents the dimension of a single attention head.

[0062] The scores are normalized using the softmax function to obtain the attention weights, calculated using the following formula: .in This represents the normalized attention weight matrix. This represents the softmax function. This represents the attention mask matrix, where the element corresponding to the valid token position is 0, and the element corresponding to the pad token position is negative infinity, ensuring that the pad token does not participate in the attention calculation.

[0063] The output of a single attention head is calculated as follows: .in This represents the output vector of the j-th attention head.

[0064] The outputs of all attention heads are concatenated and then integrated through a linear projection layer to obtain the final output of the multi-head self-attention sublayer. The calculation formula is as follows: .in This represents the final output of the multi-head self-attention sublayer. This represents a vector concatenation operation. Indicates the total number of attention heads. This represents the output projection weight matrix. This represents the output projection bias vector.

[0065] The feedforward neural network sublayer employs a structure of "linear transformation + GELU activation + linear transformation" to perform a non-linear transformation on the attention output, enhancing the model's feature representation capability. Its calculation formula is as follows: .in This represents the output of a sublayer in a feedforward neural network. This represents the output of the multi-head self-attention sublayer. This represents the weight matrix of the first-level linear transformation. This represents the first-level bias vector. Represents the activation function of the Gaussian error linear unit. This represents the weight matrix of the second-level linear transformation. This represents the second-layer bias vector.

[0066] The formula for the GELU activation function is: ,in The cumulative distribution function representing the standard normal distribution can be approximated as: .in Represents the hyperbolic tangent function. This indicates a fixed coefficient.

[0067] After iterative processing through multiple Transformer layers, the model outputs a fixed-dimensional hidden state vector for each token, which is represented as... .in The hidden state matrix representing all tokens. to These represent the hidden state vectors of each token. Indicates the length of the text token sequence.

[0068] The hidden state vector of the special token corresponding to the aggregated global semantics integrates the global semantic information of the entire text and is the core carrier representing the overall semantics of the text. To further improve the stability and discriminative ability of the feature vector, post-processing of this hidden state vector is required. First, layer normalization is performed, using the same formula as before. Then, L2 normalization is performed on the normalized vector, calculated using the following formula: .in This represents the hidden state vector of the special token after L2 normalization. The hidden state vector of the special token after normalization of the representation layer. Let L2 be the L2 norm of the vector after layer normalization. The formula for calculating the L2 norm is: ,in Describes the L2 norm of vector v. Representing vectors The One element, Representing vectors Dimensions.

[0069] The final generated deep semantic feature vector is the special token hidden state vector after L2 normalization. Each dimension of this vector corresponds to a semantic attribute of the text, which can accurately capture the hidden defects in the semantic logic of fake news generated by the large language model and the semantic coherence features of real news. It is stored in tensor format for subsequent fusion with statistical features.

[0070] S4. Based on the enhanced statistical feature extraction tool, perform batch feature calculation on the text data in the training dataset to obtain multidimensional statistical feature vectors; among them, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for the text generated by the large language model.

[0071] Specifically, this step uses an enhanced statistical feature extraction tool to batch extract basic statistical indicators and text-specific indicators generated by the large language model, constructing a multi-dimensional vector that quantifies the surface features of the text, complementing the deep semantic features. The enhanced statistical feature extraction tool is an extension of an optimized existing tool, retaining its basic statistical features while adding custom indicators specific to the characteristics of text generated by the large language model. This results in a total of multi-dimensional features covering multiple core dimensions such as text length, vocabulary usage, syntactic structure, sentiment expression, and generation patterns. All features have been empirically verified to have discriminative value in detecting fake news generated by the large language model.

[0072] Basic text statistical indicators specifically include categories such as text length-related indicators, lexical diversity-related indicators, word frequency features, syntactic features, and sentiment features. Text length-related indicators cover total number of characters, total number of words, number of sentences, average sentence length, average word length, longest sentence length, and shortest sentence length; lexical diversity-related indicators cover lexical richness, modified lexical richness, HDD index, and MTLD index; word frequency features cover the proportion of function words, frequency of high-frequency words, proportion of rare words, and proportion of stop words; syntactic features cover the average number of clauses, proportion of passive voice, proportion of subject-verb-object structures, proportion of relative clauses, and frequency of conjunctions; sentiment features cover sentiment polarity score, sentiment intensity score, proportion of positive words, and proportion of negative words based on a sentiment lexicon.

[0073] Custom statistical metrics are designed to address the inherent characteristics of text generated by large language models, specifically including n-gram repetition rate, mean sentence similarity, logical connector deviation, factual statement pass rate, and exclamation mark frequency in punctuation features. The core formula for calculating the n-gram repetition rate is as follows: .in Represents the repetition rate of n-grams. This represents the total number of n-grams in the text. This represents the number of unique n-grams in the text.

[0074] The formula for calculating the total number of n-grams in a text is: .in The total number of words in the text. This indicates the order of the n-gram.

[0075] The mean sentence similarity is obtained by calculating the mean of the syntactic tree edit distance between any two sentences in the generated dependency syntactic trees of all sentences in the text. The formula for calculating the syntactic tree edit distance is as follows: .in This represents the edit distance between the syntax trees of two sentences. , These are the syntax trees for two sentences respectively. Indicates will Convert to The required set of operations includes node replacement, insertion, and deletion. Indicates the cost of each operation. This represents the function that takes the minimum value.

[0076] The formula for calculating the mean sentence similarity is as follows: .in This represents the mean of sentence structure similarity. The total number of sentences in the text. This represents the syntactic tree edit distance between the i-th sentence and the j-th sentence.

[0077] The logical connector bias value is calculated based on the average usage frequency of multiple core logical connectors from a real news corpus. The calculation formula is as follows: .in Indicates the deviation value of logical connectors. This represents the category index of logical connectors. Indicates the total number of logical connector categories. This represents the frequency of use of the k-th type of conjunction in the i-th text. This represents the average frequency of use of the k-th type of conjunction in a real news corpus. This represents the absolute value function. The formula for calculating the frequency of the k-th type of conjunction in the i-th text is: .in Let be the number of occurrences of the k-th type of conjunction in the i-th text. Let be the total number of words in the i-th text.

[0078] The formula for calculating the pass rate of factual statement verification is as follows: .in This indicates the pass rate of authenticity verification. The total number of factual statements in the text. The number of statements validated by the fact-checking API.

[0079] The formula for calculating the frequency of exclamation mark usage in punctuation marks is as follows: .in This indicates the frequency of exclamation mark usage. This indicates the number of times the exclamation mark appears. This represents the total number of characters in the text.

[0080] The specific implementation process for batch feature calculation requires first dividing the preprocessed training text into data blocks, and then using multi-threaded parallel computing to improve processing efficiency. The tool has a built-in text parsing module that uses a combination of word segmentation and other methods to first segment the text into sentences, and then perform word segmentation and part-of-speech tagging on each sentence.

[0081] After all features have been calculated, the feature vectors are standardized using the Z-score standardization method, calculated as follows: .in This represents the standardized j-th feature value of the i-th sample. This represents the j-th feature value of the i-th sample before standardization. This represents the mean of the j-th feature on the training set. Let represent the standard deviation of the j-th feature on the training set. The formula for calculating the mean of the j-th feature on the training set is: .in Let be the total number of samples in the training set. The standard deviation of the j-th feature on the training set is calculated using the following formula: .

[0082] For the very few missing feature values ​​caused by text formatting errors, the median of that feature in the training set is used to fill in the gaps. The formula for calculating the median needs to distinguish between odd and even total sample sizes. If the total sample size is odd, the formula for calculating the median is as follows: If the total number of samples is even, the formula for calculating the median is: .in This represents the median of the j-th feature. This indicates rounding down to the nearest half of the total sample size. This represents the k-th value of the j-th feature after sorting in ascending order.

[0083] The resulting multidimensional statistical feature vector is stored in tensor format. This vector can quantify the surface features of the text and reveal the differences between the statistical regularities of the text generated by the large language model and real news.

[0084] S5. Input the deep semantic feature vector and the multidimensional statistical feature vector into the multi-granularity cross-modal attention fusion module, establish the interaction between features through a bidirectional cross-attention mechanism, and dynamically weight and fuse the interactive features based on the gating mechanism to generate a unified fused feature representation.

[0085] Specifically, this step achieves effective interaction and fusion of deep semantic features and multidimensional statistical features through a multi-granularity cross-modal attention fusion module, generating a unified feature representation that combines semantic depth and interpretability. The core is to address the heterogeneity problem of two different modalities and uncover fine-grained correlations between features. The module structure consists of three parts: a feature projection layer, a bidirectional cross-attention layer, and a gated fusion layer. The parameters of all layers are adaptively learned through the training process to ensure optimal fusion results.

[0086] The function of the feature projection layer is to map two heterogeneous features to the same dimensional space, eliminating the fusion barrier caused by dimensional differences. The deep semantic feature vector undergoes a linear transformation with the bias vector through the linear projection layer, calculated using the following formula: .in This represents the deep semantic feature vector after projection. This represents the original deep semantic feature vector. Represents the projection layer weight matrix. This represents the projection layer bias vector. This represents the LeakyReLU activation function. The formula for the LeakyReLU activation function is: .in Indicates a negative slope. This represents the input value of the activation function. This represents the function that takes the maximum value.

[0087] The projection layer weight matrix is ​​initialized using a Xavier normal distribution, and the initialization formula is as follows: .in Indicates a normal distribution. This represents the dimension of the input vector to the projection layer. This indicates the dimension of the output vector of the projection layer.

[0088] The multidimensional statistical eigenvector is transformed by another independent linear projection layer and a bias vector, and the calculation formula is as follows: .in This represents the projected multidimensional statistical feature vector. Represents the original multidimensional statistical feature vector. This represents the weight matrix of the projection layer. This represents the bias vector of the projection layer.

[0089] The projection layer weight matrix is ​​also initialized using a Xavier normal distribution, with the initialization formula as follows: .in This indicates the dimension of the input vector of the projection layer. This indicates the dimension of the output vector of the projection layer.

[0090] Through the two projection processes described above, two isomorphic feature vectors of the same dimension are finally obtained, namely the projected deep semantic feature vector and the projected multidimensional statistical feature vector.

[0091] The bidirectional cross-attention layer employs a multi-head attention mechanism to construct bidirectional interaction channels from semantic features to statistical features and from statistical features to semantic features, enabling multi-granularity feature association mining. The core of semantic feature-to-statistical feature attention is to guide the model to focus on relevant dimensions within the statistical features based on semantic features. Its complete computation process requires mapping the projected deep semantic feature vectors to query vectors, key vectors, and value vectors respectively through three independent weight matrices. The mapping formulas are as follows: , , .in This represents the query vector corresponding to the semantic features. This represents the key vector corresponding to the semantic features. This represents the value vector corresponding to the semantic features. , , These represent the three mapping weight matrices.

[0092] The projected multidimensional statistical feature vector is mapped to a key vector and a value vector using the same weight matrix as described above. The mapping formulas are as follows: , .in This represents the key vector corresponding to the statistical features. This represents the value vector corresponding to the statistical feature.

[0093] Then, the query vector corresponding to the semantic features and the key vector corresponding to the statistical features are split according to the number of attention heads, resulting in a sub-vector corresponding to each head. , .in This represents the semantic query subvector corresponding to the j-th attention head. This represents the statistical key vector corresponding to the j-th attention head. This indicates the attention head index.

[0094] The attention score for a single attention head is calculated as follows: .in This represents the semantic-to-statistical feature attention score matrix of the j-th attention head. This represents the transpose matrix of the statistical key vectors. Indicates the scaling factor. This represents the dimension of a single attention head.

[0095] The scores are normalized using the softmax function to obtain the attention weights, calculated using the following formula: .in Let represent the normalized weight matrix of the j-th attention head.

[0096] The output of a single attention head is calculated as follows: .in This represents the output vector of the j-th attention head. Let represent the statistical subvector corresponding to the j-th attention head.

[0097] The outputs of all attention heads are concatenated and then integrated through a linear projection layer to obtain the final output of semantic-to-statistical feature attention. The calculation formula is as follows: .in This represents the final output of semantic-to-statistical feature attention. This represents a vector concatenation operation. Indicates the total number of attention heads. This represents the output projection weight matrix. This represents the output projection bias vector.

[0098] The computational logic for attention from statistical features to semantic features is symmetrical to that from semantic features to statistical features. The core is to guide the model to focus on relevant dimensions within the semantic features based on statistical features. The final output formula is: .in This represents the final output of the attention to semantic features. This represents the output vector of the j-th attention head. This represents the weight matrix of the attention output projection. This represents the bias vector of the attention output projection.

[0099] The final outputs of semantic-to-statistical feature attention and statistical-to-semantic feature attention are concatenated and mapped back to the target dimension through a linear fusion layer to obtain the bidirectional attention integration result. The calculation formula is as follows: .in This indicates the result of bidirectional attention integration. Represents the ReLU activation function. Represents the weight matrix of the linear fusion layer. This represents the bias vector of the linear fusion layer.

[0100] The gated fusion layer employs an adaptive gating mechanism to dynamically balance the weights of attention-fused features and original projected features, preventing a single feature from dominating the fusion result. The gate vector is calculated as follows: .in Represents the gate vector, This represents the sigmoid activation function. This represents the deep semantic feature vector after projection. This represents the projected multidimensional statistical feature vector. Represents the gate weight matrix. This represents the gated bias vector. The formula for the Sigmoid activation function is: .in This represents the negative exponential form of the natural exponential function.

[0101] The final formula for calculating the fused feature representation is as follows: .in This represents the final fused feature vector. This represents element-wise multiplication (Hadamard product). This indicates the result of bidirectional attention integration. This represents the average fusion result of the original projection features.

[0102] This fusion process enables deep interaction between semantic features and statistical features, giving the fused features both logical discrimination capabilities at the semantic level and quantitative support at the statistical level.

[0103] S6. Use fused feature representations to train the classifier network to obtain a trained fake news detection model.

[0104] Specifically, this step trains a classifier network based on fused feature representations, constructing an end-to-end model capable of accurately identifying fake news generated by large language models. The classifier network adopts a "deep fully connected + regularization" structure, balancing feature fitting and generalization capabilities. The specific structure of the classifier network, from input to output, consists of an input layer, a first fully connected layer, a batch normalization layer, a Dropout layer, a second fully connected layer, another Dropout layer, and an output layer.

[0105] The input layer is used to receive the fused feature representation, and the calculation formula for the first fully connected layer is: .in This represents the output of the first fully connected layer. This represents the fused feature vector of the input. This represents the weight matrix of the first fully connected layer. This represents the bias vector of the first fully connected layer.

[0106] The weight matrix of the first fully connected layer is initialized using a Xavier normal distribution, and the initialization formula is as follows: .in This represents the dimension of the input vector of the first fully connected layer. This represents the dimension of the output vector of the first fully connected layer.

[0107] The output of the first fully connected layer is processed by an activation function, specifically GELU, whose formula is the same as before. A batch normalization layer is used to standardize the output of the first fully connected layer; the calculation formula is as follows. .in This represents the output of the batch normalization layer. This represents the mean of the output of the first fully connected layer for the current batch of samples. This represents the variance of the output of the first fully connected layer for the current batch of samples. and This represents the learnable scaling and offset parameters. This represents the numerical stability parameter.

[0108] The formula for calculating the mean of the output of the first fully connected layer of the current batch of samples is as follows: .in Indicates batch size, This represents the output of the first fully connected layer for the i-th sample. The variance of the output of the first fully connected layer for the current batch of samples is calculated using the following formula: .

[0109] The first Dropout layer is used to prevent the model from overfitting, and its calculation logic is as follows: .in This represents the output of the first Dropout layer. Indicates the discard rate. Represents a binary mask vector. This represents the output of the batch normalization layer. This indicates element-wise multiplication.

[0110] The calculation formula for the second fully connected layer is: .in This represents the output of the second fully connected layer. This represents the output of the first Dropout layer. This represents the weight matrix of the second fully connected layer. This represents the bias vector of the second fully connected layer.

[0111] The weight matrix of the second fully connected layer is initialized using a Xavier normal distribution, and the initialization formula is as follows: .in This represents the dimension of the input vector of the second fully connected layer. This represents the dimension of the output vector of the second fully connected layer.

[0112] The output of the second fully connected layer is also processed by the GELU activation function, and then fed into the second Dropout layer. The computation logic of the second Dropout layer is as follows: .in This represents the output of the second Dropout layer. This indicates the dropout rate of the second Dropout layer. This represents the second binary mask vector. This represents the output after the second fully connected layer is activated.

[0113] The output layer is a linear layer with no activation function; the calculation formula is as follows: .in This represents the unnormalized score of the output layer. This represents the output of the second Dropout layer. This represents the output layer weight matrix. This represents the output layer bias vector.

[0114] The training of the classifier network and the task-specific fine-tuning of the semantic encoder are carried out in tandem, employing an end-to-end training approach. The optimizer used is AdamW, with its parameter update formula as described above. The loss function is the cross-entropy loss function, expressed as follows: .in This represents the cross-entropy loss value. Indicates batch size, This represents the true label of the i-th sample, class c. This represents the probability that the model predicts the i-th sample belongs to class c. This represents the natural logarithm function.

[0115] The model's predicted probability is calculated using the softmax function, and the formula is as follows: .in This represents the unnormalized score of the i-th sample in category c. This represents the natural exponential function. This indicates a category index.

[0116] During training, a gradient accumulation strategy is employed, meaning that backpropagation is performed to update the parameters only after calculating the loss for multiple batches. The formula for calculating the accumulated loss is as follows: .in This represents the cumulative loss value. Indicates the cumulative number of steps. Let represent the loss value of the t-th batch. The gradient is calculated using the mean of the cumulative loss during parameter updates; the gradient calculation formula is: .in Represents the gradient value. This represents the gradient of the cumulative loss with respect to the model parameters.

[0117] After each training round, four core evaluation metrics are calculated on the validation set: accuracy, precision, recall, and F1 score. The formulas for each metric are the same as before. An early stopping strategy is employed during training: if the F1 score on the validation set does not improve for several consecutive rounds, training is stopped to avoid overfitting due to overtraining. After training, the model with the best performance on the validation set is saved as the final fake news detection model. This model includes all parameters of the semantic encoder, feature fusion module, and classifier network.

[0118] S7. Based on the trained fake news detection model, process the fusion feature representation corresponding to the news text to be detected, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the fusion feature representation to be processed is obtained in the same way as the fusion feature representation.

[0119] Specifically, this step, based on a pre-trained fake news detection model, performs standardized processing and feature extraction on the news text to be detected, outputting accurate judgment results and ensuring the consistency and reliability of the detection process. The processing flow for the news text to be detected is completely consistent with that for the training data. First, it performs normalization cleaning and sub-word serialization processing corresponding to the data preprocessing and dataset construction steps, with the core calculation formulas being the same as before, including text word count and token ID conversion. Then, it uses RoBERTa's tokenizer for word segmentation, adds special tokens for aggregating global semantics and sentence separation, converts them into tokenID sequences, pads or truncates them to a fixed length, generating a standardized sequence of text to be detected.

[0120] Subsequently, the sequence is input into the trained fine-tuned semantic encoder, and the hidden state vectors of the corresponding aggregated global semantic special tokens are extracted according to the deep semantic feature vector generation steps. The core calculation formulas include word embedding, multi-head self-attention, layer normalization, L2 normalization, etc., which are completely consistent with the training process, to obtain the deep semantic feature vector of the text to be detected.

[0121] Meanwhile, the enhanced statistical feature extraction tool in the multidimensional statistical feature vector extraction step is used to calculate the features of the text to be detected. The core calculation formulas include n-gram repetition rate, mean sentence similarity, Z-score standardization, etc., all of which use the mean and standard deviation of the training set to obtain the multidimensional statistical feature vector of the text to be detected.

[0122] The deep semantic feature vector and multidimensional statistical feature vector of the text to be detected are input into the multi-granularity cross-modal attention fusion module. The module processes the text according to the feature projection, bidirectional cross-attention calculation and gating fusion process. The core calculation formulas include projection transformation, attention score, gating vector generation, and fusion feature calculation, which are completely consistent with the training process. The module generates the fusion feature representation to be processed, ensuring that the features to be processed are consistent with the fusion features in the training process in terms of dimension and distribution.

[0123] The fused feature representation to be processed is input into the trained classifier network, and the corresponding unnormalized score is output. This score is represented as... .in This represents the unnormalized score vector of the text to be detected. The unnormalized score represents the value of the text to be detected as belonging to the category of real news. The unnormalized score represents the score by which the text to be detected belongs to the category of fake news generated by a large language model.

[0124] The unnormalized score is converted into a probability distribution using the softmax function, and the calculation formula is as follows: .in Represents the probability distribution vector. This indicates the probability that the text to be detected is real news. This indicates the probability that the text to be detected is fake news generated by a large language model.

[0125] The formula for calculating the probability of real news is: The formula for calculating the probability of a large language model generating fake news is as follows: .

[0126] A judgment threshold is set, and the core judgment logic is executed: if the probability that the text to be detected belongs to fake news generated by the large language model is greater than or equal to the judgment threshold, the judgment result is output as fake news generated by the large language model; if the probability is less than the judgment threshold, the judgment result is output as fake news not generated by the large language model. To improve the rigor of the judgment, a confidence evaluation mechanism is introduced. When the probability that the text to be detected belongs to fake news generated by the large language model is in the boundary range near the threshold, the result is output as suspected fake news generated by the large language model, and manual review is recommended.

[0127] For batch detection scenarios, the model supports multi-threaded parallel processing, accelerating preprocessing, feature extraction, fusion, and classification processes. Detection results can be exported in a specified format, including the text ID to be detected, the judgment result, confidence score, and the contribution of key features. The feature contribution is quantified by calculating the gradient value of each feature with respect to the model output, using the following formula: .in This represents the contribution of the j-th feature. Let represent the partial derivative of the probability of a large language model generating fake news with respect to the j-th feature value. This represents the absolute value function.

[0128] A higher feature contribution value indicates a greater impact of the feature on the judgment result, providing interpretability support for the detection results. This judgment process ensures standardization throughout the entire process from text input to result output, enabling accurate and efficient identification of fake news generated by large language models.

[0129] The aforementioned method for detecting fake news based on a Transformer-based large language model involves acquiring and preprocessing a text dataset containing both real and fake news. It then fine-tunes a RoBERTa pre-trained model in two stages—domain adaptation and task-specific fine-tuning—using general news corpora and specific training data to obtain an encoder capable of capturing the deep semantic authenticity of news and generating deep semantic feature vectors. Simultaneously, an enhanced statistical feature extraction tool, integrating basic statistical indicators and custom indicators designed for text generated by the large language model, extracts multi-dimensional statistical feature vectors in batches from the same text data. These two types of features are then input into a multi-granularity cross-modal attention fusion module. A bidirectional cross-attention mechanism establishes deep interactions between features, and a gating mechanism dynamically weights and fuses the interacting features, forming a unified and complementary fused feature representation. This fused feature representation is used to train a classifier network, ultimately constructing a fake news detection model capable of automatically judging the news text to be detected. This approach significantly improves the accuracy and robustness of identifying fake news generated by large language models by deeply integrating deep semantic understanding with shallow statistical patterns and using attention and gating mechanisms to achieve adaptive feature integration. It also enhances the model's generalization ability when facing different text generation strategies and diverse news domains.

[0130] In one alternative embodiment, a fine-tuned semantic encoder is used to encode the text data in the training dataset to generate a deep semantic feature vector for each news text, including the following steps:

[0131] S11. Input the text data of a single news article in the training dataset into the fine-tuned semantic encoder. After forward propagation through multiple layers of Transformer encoders, the final hidden state sequence is output.

[0132] Specifically, the input text data in this step is a preprocessed, standardized token sequence, and the input process is consistent with the input specifications of text encoding. The fine-tuned semantic encoder's multi-layer Transformer encoder performs operations according to the predetermined forward propagation logic without requiring additional adjustments to structural parameters. The final output hidden state sequence is the set of hidden states of all tokens in the text after processing by all Transformer layers, and its dimension is related to the token sequence length and the hidden layer dimension.

[0133] S12. Extract the vector corresponding to the special classification label position from the final hidden state sequence as the basic semantic feature vector.

[0134] Specifically, the special classification marker in this step is a dedicated token that has been clearly defined for aggregating global semantics. Its position in the hidden state sequence is fixed. The vector at this position is extracted as the basic semantic feature vector. This is based on the characteristic that the token has been integrated with the global semantics of the text. This step does not require additional preprocessing of the vector and can be directly extracted as the basis for subsequent fusion.

[0135] S13. Extract the hidden state vectors corresponding to the special classification label positions from the last K layers of the fine-tuned semantic encoder to obtain a multi-level semantic feature set.

[0136] Specifically, K is a preset positive integer, and its value is adaptively determined based on the model training effect. The core purpose is to capture the feature information of the text at different semantic abstraction levels. In practice, the output of the last K layers of the Transformer encoder in the fine-tuned semantic encoder is traced back. The hidden state vector of the specific classification label position is extracted from each layer. All the extracted vectors together constitute a multi-level semantic feature set. The dimension of the vectors in this set is consistent with the dimension of the hidden state vector output by a single Transformer layer.

[0137] S14. Perform a weighted summation on the vectors in the multi-level semantic feature set to obtain the weighted summation result; concatenate the weighted summation result with the basic semantic feature vector to generate the deep semantic feature vector.

[0138] Specifically, the formula for weighted summation is as follows: In the formula This represents the weighted summation result. This represents the weight coefficients of the hidden state vector corresponding to the k-th layer Transformer. This represents the hidden state vector at the special classification label position output by the k-th layer Transformer in a multi-level semantic feature set, where all weight coefficients satisfy... ,and It is obtained through adaptive learning during model training and is used to dynamically adjust the contribution of semantic features at different levels.

[0139] After the weighted summation is completed, the weighted summation result will be... With basic semantic feature vector Perform vector concatenation, with the concatenation method being dimension superposition, to generate a deep semantic feature vector. The calculation formula is In the formula This represents a vector concatenation operation. The dimension is the sum of the weighted summation result dimension and the dimension of the basic semantic feature vector. This concatenation operation achieves the fusion of multi-level semantic features and global basic semantic features, enabling the final deep semantic feature vector to capture detailed semantic information at different levels of the text while preserving global semantic relevance, further enhancing the discriminative ability of the feature vector.

[0140] refer to Figure 2 In one optional embodiment, the deep semantic feature vector and the multidimensional statistical feature vector are input into the multi-granularity cross-modal attention fusion module. An interaction between features is established through a bidirectional cross-attention mechanism, and the interacting features are dynamically weighted and fused based on a gating mechanism to generate a unified fused feature representation. This includes the following steps:

[0141] S21. Project the deep semantic feature vector through the first linear transformation layer to obtain the first projected feature; project the multidimensional statistical feature vector through the second linear transformation layer to obtain the second projected feature; wherein the first projected feature and the second projected feature have the same dimension.

[0142] Specifically, both the first and second linear transformation layers contain learnable weight matrices and bias terms. The core purpose of the projection operation is to transform the two types of original features, which may have different dimensions, into feature vectors with consistent dimensions, providing a suitable input basis for subsequent cross-attention calculations. The first and second projected features have the same dimension, and the value of this dimension is adaptively set according to the overall training effect of the model and the feature representation requirements, ensuring that the two types of features can be effectively interacted.

[0143] S22. Using the second projection feature as the query and the first projection feature as the key and value, perform a first cross-attention calculation to generate a first interaction feature; wherein, the expression for the first cross-attention calculation is:

[0144]

[0145] in, Indicates the first interaction feature, For the first projection feature, This is the second projection feature. , , The first set of learnable parameter matrices is used to generate the projections of queries, keys, and values, respectively. It is a key vector The dimension is used to scale the dot product result; This represents the matrix transpose operation.

[0146] Specifically, The first interactive feature is essentially an interactive representation obtained by focusing attention on the first projected feature by the second projected feature, which can highlight the correlation information between the two types of features. This is the first projection feature, which is the output of the deep semantic feature vector after being projected through the first linear transformation layer; This is the second projection feature, which is the output of the multidimensional statistical feature vector after being projected through the second linear transformation layer; , , The first set of learnable parameter matrices, all three are adaptively updated through model training, among which... Used for linear projection of the query vector (second projection feature). Used for linear projection of the key vector (first projective feature). Used for linear projection of the value vector (first projection feature); It is a key vector The dimension is used to scale the dot product result in the attention calculation, so as to avoid the dot product result being too large due to the high dimension, which would affect the gradient stability of the Softmax function. This represents the matrix transpose operation, used to achieve dimension matching between the query vector and the key vector, ensuring that the dot product operation is executed correctly; The function is used to convert the scaled dot product result into a probability distribution form, namely attention weights. These weights reflect the importance of each dimension in the first projected feature to the second projected feature. Finally, the first interactive feature is obtained by multiplying the attention weights with the value vector.

[0147] S23. Using the first projection feature as the query and the second projection feature as the key and value, perform a second cross-attention calculation to generate a second interaction feature; wherein, the expression for the second cross-attention calculation is:

[0148]

[0149] in, Indicates the second interaction feature, , , The second set of learnable parameter matrices is used to generate the projections of queries, keys, and values, respectively. It is a key vector The dimension is used to scale the dot product result.

[0150] Specifically, This represents the second interactive feature, which forms a two-way interaction with the first interactive feature and is used to capture the correlation and focusing information of the first projection feature to the second projection feature; , , The second set of learnable parameter matrices is independent of the first set of parameter matrices. It is also adaptively updated through model training and is used to perform linear projections on the query vector (first projection feature), key vector (second projection feature), and value vector (second projection feature) for this round, respectively. It is a key vector The dimension, its function and Consistency is used to scale the dot product result of this round to ensure computational stability; the meanings of other symbols in the formula are consistent with the meanings of the corresponding symbols in the first cross-attention calculation expression, and will not be repeated here.

[0151] S24. Perform a residual connection between the second interactive feature and the first projected feature to obtain the first residual connection result, and perform layer normalization on the first residual connection result to obtain the enhanced semantic feature; perform a residual connection between the first interactive feature and the second projected feature to obtain the second residual connection result, and perform layer normalization on the second residual connection result to obtain the enhanced statistical feature.

[0152] Specifically, the core purpose of residual connection operation is to alleviate the gradient vanishing problem during model training and retain the basic information of the original features. Its specific implementation is to perform element-wise addition operation on the interaction features and the corresponding original projected features. Layer normalization is used to standardize the feature vector after residual connection, making the feature distribution more stable and accelerating model convergence. Its calculation logic follows the conventional layer normalization implementation method, and ensures that the feature values ​​are within a reasonable range by adjusting the mean and variance of the features.

[0153] S25. Input the enhanced semantic features and enhanced statistical features into the gated fusion network to generate a dynamic gated weight vector. Then, perform a weighted summation of the enhanced semantic features and enhanced statistical features based on the gated weight vector to output the fused feature representation.

[0154] Specifically, the gated fusion network includes a linear transformation layer and an activation function. It adaptively generates a dynamic gating weight vector, the dimension of which is consistent with the dimensions of the enhanced semantic and statistical features. Each element in the vector ranges from 0 to 1, representing the importance of the corresponding feature dimension. After the dynamic gating weight vector is generated, it is element-wise multiplied with both the enhanced semantic and statistical features, and then element-wise added to the results. The final result is the unified fused feature representation. This process dynamically adjusts the weight ratio of the two types of features, enabling the fused features to adaptively focus on information more valuable to subsequent tasks, thus improving the overall discriminative performance of the features.

[0155] In one alternative embodiment, a classifier network is trained using fused feature representations to obtain a trained fake news detection model, including the following steps:

[0156] S31. Construct an initial classifier network, which consists of a fully connected layer, an activation function layer, a Dropout layer, and a classification layer connected in sequence.

[0157] Specifically, the fully connected layer is used to perform dimensionality transformation and feature mapping on the fusion feature representation of the input, the activation function layer is used to introduce non-linear feature expression capabilities, the Dropout layer is used to suppress model overfitting, and the classification layer is used to output the fake news prediction probability of the sample. The core structural parameters of each layer (such as the number of neurons in the fully connected layer, the dropout rate in the Dropout layer, etc.) are all preset with initial values ​​based on the model training requirements, and can be fine-tuned during training based on the fitting effect.

[0158] S32. The fused feature representation is input into the fully connected layer and linearly transformed to obtain the first transformed feature.

[0159] Specifically, the linear transformation process is achieved through the learnable weight matrix and bias vector of the fully connected layer. The core is to map the fused feature representation to a feature space that adapts to subsequent nonlinear processing. The dimension of the first transformed feature is determined by the number of output neurons of the fully connected layer, ensuring that it matches the input requirements of the activation function layer.

[0160] S33. Input the first transformation feature into the activation function layer for nonlinear activation to obtain the activated feature.

[0161] Specifically, the activation function layer selects a nonlinear activation function adapted to binary classification tasks. Through nonlinear transformation, the model's ability to fit complex feature relationships is enhanced, eliminating the expressive limitations of linear transformation, and enabling the activated features to more accurately capture the core information related to fake news discrimination in the fused features.

[0162] S34. Input the activated features into the Dropout layer to randomly discard neurons, and obtain the regularized features.

[0163] Specifically, the Dropout layer avoids the model's over-reliance on specific feature dimensions and reduces the risk of overfitting by randomly shutting down some neurons during training. The dropout operation is performed according to a preset dropout rate, and the feature values ​​corresponding to the dropped neurons do not participate in the backpropagation of this round of training. The dimension of the regularized features remains consistent with that of the activated features.

[0164] S35. Input the regularized features into the classification layer, and calculate the predicted probability that the news is fake news through the classification layer.

[0165] Specifically, the classification layer adopts the output logic adapted to the binary classification task, mapping the regularized features to probability values ​​in the range [0,1]. These probability values ​​directly represent the confidence that the input sample is fake news. The closer the probability value is to 1, the higher the probability that the sample is fake news, and vice versa.

[0166] S36. Calculate the main classification loss based on the predicted probabilities and the corresponding true labels; the formula for calculating the main classification loss is:

[0167]

[0168] in, Primary classification loss, For training batch size, For the first The true binary label of each sample, For the predicted first The probability that a sample is fake news.

[0169] Specifically, The main classification loss is used to quantify the deviation between the model's prediction and the sample's true label. The smaller the loss value, the higher the model's prediction accuracy. N represents the training batch size, which is the number of samples participating in a single training run. This represents the true binary label of the i-th sample. A value of 1 indicates that the sample is fake news, and a value of 0 indicates that the sample is real news. This represents the probability that the i-th sample is fake news, which is the probability value of the corresponding sample output by the classification layer. This represents the natural logarithm function. By performing a logarithmic transformation on the predicted probabilities, it amplifies the loss contribution when the prediction bias is large, thereby enhancing the guiding role of the loss in updating the model parameters.

[0170] S37. Calculate the contrastive learning auxiliary loss based on the fusion feature representation of different samples within the training batch.

[0171] Specifically, the core calculation logic of this auxiliary loss is as follows: Within the training batch, positive and negative sample pairs are constructed. Positive sample pairs are those with highly similar semantic or statistical features and consistent labels, while negative sample pairs are those with significantly different features or inconsistent labels. By calculating the similarity of the fused feature representations between sample pairs, a contrastive loss function is constructed, making the feature distance of positive sample pairs approach 0 and the feature distance of negative sample pairs greater than a preset threshold. This improves the discriminative power of the fused features and enhances the model's discrimination ability. The calculation of the contrastive learning auxiliary loss needs to be adaptively adjusted based on the distribution characteristics of the samples within the training batch to ensure a complementary constraint with the main classification loss.

[0172] S38. Combine the main classification loss and the contrastive learning auxiliary loss to obtain the total loss. Based on the total loss, update the parameters of the classifier network and the multi-granularity cross-modal attention fusion module simultaneously through the backpropagation algorithm to complete one round of training.

[0173] Specifically, the total loss is calculated as a weighted sum of the two, i.e. ,in Indicates the total loss. This represents the contrast learning auxiliary loss. The weight coefficient (within the range of [0,1]) is used to balance the loss contribution of the main classification task and the contrastive learning auxiliary task. The backpropagation algorithm follows the gradient descent logic, starting from the total loss, and sequentially calculates the parameter gradients of each layer of the classifier network (fully connected layer, activation function layer, Dropout layer, classification layer) and each layer of the multi-granularity cross-modal attention fusion module. Based on the preset optimizer (such as the AdamW optimizer), all learnable parameters are updated to achieve collaborative optimization of the two modules.

[0174] S39. Repeat steps S32 to S38 until the training termination condition is met to obtain the trained fake news detection model.

[0175] Specifically, the training termination condition is preset to one of the following two conditions: first, the core evaluation metrics (such as F1 score and accuracy) on the validation set do not improve for a preset number of consecutive rounds (or the improvement is less than a preset threshold); second, the number of training rounds reaches the preset maximum number of training rounds. After each round of training, the total loss value of the model and the validation set evaluation metrics are recorded. When the termination condition is met, parameter updates are stopped, and the current optimal model parameters (i.e., the model parameters corresponding to the optimal validation set evaluation metrics) are saved. This model is the finally trained fake news detection model, which can be directly used for the task of judging the news text to be detected.

[0176] In one optional embodiment, the contrastive learning auxiliary loss is calculated based on the fused feature representations of different samples within a training batch, including the following steps:

[0177] S41. Input the fused feature representation of each sample in the training batch into the projection head network for dimensionality reduction projection to obtain the corresponding low-dimensional projection representation; wherein, the projection head network includes a fully connected layer.

[0178] Specifically, the projection head network contains only one fully connected layer. This fully connected layer has a learnable weight matrix and bias vector. Its core function is to map the high-dimensional fused feature representation to a lower-dimensional feature space, reducing the complexity of subsequent similarity calculations while preserving the core discriminative information of the features. The dimension of the low-dimensional projection representation is determined through experimental verification, balancing computational efficiency and feature discriminability to ensure suitability for subsequent contrastive learning.

[0179] S42. For a sample within a training batch, use the low-dimensional projection representation of the sample as the anchor representation, and select another low-dimensional projection representation from the same batch of samples of the same type as the positive sample representation.

[0180] Specifically, the anchor representation is the core reference object for contrastive learning, used to compare the similarity of the feature representations of other samples. The selection of positive sample representations should follow the principle of "same batch + same type", that is, select the low-dimensional projection representation corresponding to the sample that is in the same training batch as the anchor sample and has the same true label. Its core purpose is to strengthen the model's ability to recognize the feature patterns of the same type of samples by aggregating the features of the same type of samples.

[0181] S43. Calculate the cosine similarity between the anchor representation and the low-dimensional projected representations of all other samples in the training batch.

[0182] Specifically, the cosine similarity function is used to quantify the similarity between two feature vectors. The calculated value ranges from -1 to 1; the closer the value is to 1, the higher the similarity between the two feature vectors, and the closer the value is to -1, the lower the similarity. This step requires traversing all other samples in the training batch except for the anchor sample, calculating the cosine similarity between the anchor representation and the low-dimensional projection representation of each sample, and obtaining the similarity set corresponding to the anchor sample, providing basic data for subsequent loss calculation.

[0183] S44. Based on the anchor point representation, positive sample representation, and the cosine similarity of all other samples, the normalized temperature-scaled cross-entropy loss function is used to calculate the contrastive loss component of the corresponding sample; the average of the contrastive loss components of all samples in the batch is calculated to obtain the contrastive learning auxiliary loss; the formula for calculating the contrastive learning auxiliary loss is as follows:

[0184]

[0185] in, To compare the learning aid loss, Indicates the size of the training batch; Indicates the first Anchor points for each sample; Indicates the first A positive sample representation of a sample; Indicates the first in the training batch A low-dimensional projective representation of each sample; Represents the cosine similarity function; This represents the temperature coefficient hyperparameter.

[0186] Specifically, The loss represents the contrastive learning auxiliary loss, which is used to quantify the effect of sample feature aggregation and differentiation in contrastive learning tasks. The smaller the loss value, the more aggregated the features of similar samples and the more dispersed the features of dissimilar samples, thus helping to improve the model's discriminative ability; N represents the size of the training batch, that is, the total number of samples participating in training in a single session. The anchor representation of the i-th sample is the low-dimensional projection representation of the i-th sample obtained after dimensionality reduction by the projection head network. This represents the positive sample representation of the i-th sample, which is a low-dimensional projection representation selected from samples of the same type in the same batch. Let j represent the low-dimensional projection of the j-th sample in the training batch, where the range of j covers all samples in the batch and excludes the i-th sample itself. This represents the cosine similarity function, used to calculate the similarity between two feature vectors within the parentheses. The specific calculation logic is the dot product of the two vectors divided by the product of their L2 norms. This represents the temperature coefficient hyperparameter, which is a preset positive number used to adjust the smoothness of the similarity distribution and avoid unstable loss calculation caused by excessive similarity differences. Its optimal value was determined through multiple sets of experiments. This represents the natural exponential function, used for exponential transformation of scaled similarity; The natural logarithm function is used to amplify the contribution of similarity matching bias to the loss. The numerator represents the result of the temperature-scaled exponential transformation of the anchor representation and the positive sample representation, while the denominator represents the sum of the temperature-scaled exponential transformations of the anchor representation and the low-dimensional projection representations of all other samples in the batch. The ratio of the two represents the relative similarity ratio between the anchor representation and the positive sample representation. The final contrastive learning auxiliary loss is obtained by taking the natural logarithm and negative of this ratio for all samples in the batch and averaging the results.

[0187] The aforementioned method for detecting fake news based on a Transformer-based large language model involves acquiring and preprocessing a text dataset containing both real and fake news. It then fine-tunes a RoBERTa pre-trained model in two stages—domain adaptation and task-specific fine-tuning—using general news corpora and specific training data to obtain an encoder capable of capturing the deep semantic authenticity of news and generating deep semantic feature vectors. Simultaneously, an enhanced statistical feature extraction tool, integrating basic statistical indicators and custom indicators designed for text generated by the large language model, extracts multi-dimensional statistical feature vectors in batches from the same text data. These two types of features are then input into a multi-granularity cross-modal attention fusion module. A bidirectional cross-attention mechanism establishes deep interactions between features, and a gating mechanism dynamically weights and fuses the interacting features, forming a unified and complementary fused feature representation. This fused feature representation is used to train a classifier network, ultimately constructing a fake news detection model capable of automatically judging the news text to be detected. This approach significantly improves the accuracy and robustness of identifying fake news generated by large language models by deeply integrating deep semantic understanding with shallow statistical patterns and using attention and gating mechanisms to achieve adaptive feature integration. It also enhances the model's generalization ability when facing different text generation strategies and diverse news domains.

[0188] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0189] Based on the same inventive concept, this application also provides a system for implementing the aforementioned method for detecting fake news based on a large language model generated by Transformer. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the fake news detection system based on a large language model generated by Transformer provided below can be found in the limitations of the fake news detection method based on a large language model generated by Transformer described above, and will not be repeated here.

[0190] In one exemplary embodiment, such as Figure 3As shown, a fake news detection system 30 based on a large language model generated by Transformer is provided to implement the methods in the above embodiments. The system includes:

[0191] The data preprocessing optimization module 31 is used to obtain the original dataset containing real news text and fake news text generated by the big language model, and to perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset.

[0192] The domain adaptation model building module 32 is used to fine-tune the RoBERTa pre-trained model based on general news domain corpus to obtain the domain adaptation model; and to fine-tune the domain adaptation model based on the training dataset to obtain the fine-tuned semantic encoder.

[0193] The deep semantic encoding module 33 is used to encode the text data in the training dataset using the fine-tuned semantic encoder to generate a deep semantic feature vector for each news text.

[0194] The multidimensional statistical feature extraction module 34 is used to perform batch feature calculation on the text data in the training dataset based on the enhanced statistical feature extraction tool to obtain multidimensional statistical feature vectors; wherein, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for the text generated by the large language model.

[0195] The cross-modal feature fusion module 35 is used to input deep semantic feature vectors and multi-dimensional statistical feature vectors into the multi-granularity cross-modal attention fusion module, establish interaction between features through a bidirectional cross-attention mechanism, and dynamically weight and fuse the interacting features based on a gating mechanism to generate a unified fused feature representation.

[0196] The classifier training optimization module 36 is used to train the classifier network using fused feature representations to obtain a trained fake news detection model.

[0197] The real-time detection and judgment module 37 is used to process the unprocessed fusion feature representation corresponding to the news text to be detected based on the trained fake news detection model, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the unprocessed fusion feature representation is obtained in the same way as the fusion feature representation.

[0198] Embodiments of this application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the aforementioned method embodiments.

[0199] Embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0200] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0201] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for detecting fake news based on a large language model generated by Transformer, characterized in that, The method includes: S1. Obtain the original dataset containing real news text and fake news text generated by the big language model, and perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset. S2. Based on the general news domain corpus, the RoBERTa pre-trained model is fine-tuned for domain adaptation to obtain a domain-adapted model; based on the training dataset, the domain-adapted model is fine-tuned for task-specific purposes to obtain a fine-tuned semantic encoder. S3. Use the fine-tuned semantic encoder to encode the text data in the training dataset to generate a deep semantic feature vector for each news text. S4. Perform batch feature calculation on the text data in the training dataset based on the enhanced statistical feature extraction tool to obtain a multidimensional statistical feature vector; wherein, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for text generated by large language models; S5. Input the deep semantic feature vector and the multidimensional statistical feature vector into the multi-granularity cross-modal attention fusion module, establish the interaction between features through a bidirectional cross-attention mechanism, and dynamically weight and fuse the interacting features based on a gating mechanism to generate a unified fused feature representation. S6. Use the fused feature representation to train the classifier network to obtain a trained fake news detection model; S7. Based on the trained fake news detection model, process the unprocessed fusion feature representation corresponding to the news text to be detected, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the unprocessed fusion feature representation is obtained in the same way as the fusion feature representation.

2. The method according to claim 1, characterized in that, The step of encoding the text data in the training dataset using the fine-tuned semantic encoder to generate a deep semantic feature vector for each news text includes: S11. Input the text data of a single news article in the training dataset into the fine-tuned semantic encoder, and after forward propagation through multiple layers of Transformer encoders, output the final hidden state sequence. S12. Extract the vector corresponding to the special classification marker position from the final hidden state sequence as the basic semantic feature vector; S13. Extract the hidden state vector corresponding to the special classification label position from the last K layers of the fine-tuned semantic encoder to obtain a multi-level semantic feature set. S14. Perform a weighted summation on the vectors in the multi-level semantic feature set to obtain a weighted summation result; concatenate the weighted summation result with the basic semantic feature vector to generate the deep semantic feature vector.

3. The method according to claim 1, characterized in that, The process of inputting the deep semantic feature vector and the multidimensional statistical feature vector into the multi-granularity cross-modal attention fusion module, establishing inter-feature interaction through a bidirectional cross-attention mechanism, and dynamically weighting and fusing the interacting features based on a gating mechanism to generate a unified fused feature representation includes: S21. Project the deep semantic feature vector through a first linear transformation layer to obtain a first projected feature; project the multidimensional statistical feature vector through a second linear transformation layer to obtain a second projected feature; wherein the first projected feature and the second projected feature have the same dimension; S22. Using the second projection feature as the query and the first projection feature as the key and value, perform a first cross-attention calculation to generate a first interaction feature; wherein, the expression for the first cross-attention calculation is: in, This indicates the first interaction feature. This is the first projection feature. This is the second projection feature. , , The first set of learnable parameter matrices is used to generate the projections of queries, keys, and values, respectively. It is a key vector The dimension is used to scale the dot product result; This represents the matrix transpose operation; S23. Using the first projection feature as the query and the second projection feature as the key and value, perform a second cross-attention calculation to generate a second interaction feature; wherein, the expression for the second cross-attention calculation is: in, This indicates the second interaction feature. , , The second set of learnable parameter matrices is used to generate the projections of queries, keys, and values, respectively. It is a key vector The dimension is used to scale the dot product result; S24. Perform a residual connection between the second interactive feature and the first projected feature to obtain a first residual connection result, and perform layer normalization on the first residual connection result to obtain enhanced semantic features; perform a residual connection between the first interactive feature and the second projected feature to obtain a second residual connection result, and perform layer normalization on the second residual connection result to obtain enhanced statistical features. S25. Input the enhanced semantic features and the enhanced statistical features into a gated fusion network to generate a dynamic gated weight vector, and perform a weighted summation of the enhanced semantic features and the enhanced statistical features based on the gated weight vector to output the fused feature representation.

4. The method according to any one of claims 1 to 3, characterized in that, The step of training the classifier network using the fused feature representation to obtain a trained fake news detection model includes: S31. Construct the initialized classifier network, which includes a fully connected layer, an activation function layer, a Dropout layer, and a classification layer connected in sequence. S32. Input the fused feature representation into the fully connected layer and perform a linear transformation to obtain the first transformed feature; S33. Input the first transformation feature into the activation function layer for nonlinear activation to obtain the activated feature; S34. Input the activated features into the Dropout layer to randomly discard neurons and obtain regularized features; S35. Input the regularized features into the classification layer, and calculate the predicted probability that the news is fake news through the classification layer; S36. Calculate the main classification loss based on the predicted probabilities and the corresponding true labels; wherein the formula for calculating the main classification loss is: in, The primary classification loss is... For training batch size, For the first The true binary label of each sample, For the predicted first The probability that a sample is fake news; S37. Calculate the contrastive learning auxiliary loss based on the fusion feature representation of different samples within the training batch; S38. Combine the main classification loss and the contrastive learning auxiliary loss to obtain the total loss. Based on the total loss, update the parameters of the classifier network and the multi-granularity cross-modal attention fusion module simultaneously through the backpropagation algorithm to complete one round of training. S39. Repeat steps S32 to S38 until the training termination condition is met to obtain the trained fake news detection model.

5. The method according to claim 4, characterized in that, The calculation of the contrastive learning auxiliary loss based on the fused feature representation of different samples within the training batch includes: S41. Input the fusion feature representation of each sample in the training batch into the projection head network for dimensionality reduction projection to obtain the corresponding low-dimensional projection representation; wherein, the projection head network includes a fully connected layer; S42. For a sample within a training batch, the low-dimensional projection representation of the sample is used as the anchor representation, and another low-dimensional projection representation is selected from the same batch of samples of the same type as the positive sample representation. S43. Calculate the cosine similarity between the anchor representation and the low-dimensional projection representation of all other samples in the training batch; S44. Based on the anchor point representation, the positive sample representation, and the cosine similarity of all other samples, the contrastive loss component of the corresponding sample is calculated using the normalized temperature-scaled cross-entropy loss function; the average of the contrastive loss components of all samples in the batch is calculated to obtain the contrastive learning auxiliary loss; wherein, the formula for calculating the contrastive learning auxiliary loss is: in, The contrastive learning auxiliary loss, Indicates the size of the training batch; Indicates the first The anchor point of each sample represents; Indicates the first The positive sample representation of each sample; Indicates the first in the training batch The low-dimensional projection representation of each sample; Represents the cosine similarity function; This represents the temperature coefficient hyperparameter.

6. A fake news detection system based on a large language model generated by Transformer, used to implement the method of any one of claims 1 to 5, characterized in that, The system includes: The data preprocessing optimization module is used to obtain the original dataset containing real news text and fake news text generated by the big language model, and to perform normalization cleaning and sub-word serialization processing on the news text in the original dataset to obtain the preprocessed training dataset. The domain adaptation model construction module is used to fine-tune the RoBERTa pre-trained model based on general news domain corpus to obtain a domain adaptation model; and to fine-tune the domain adaptation model based on the training dataset to obtain a fine-tuned semantic encoder. The deep semantic encoding module is used to encode the text data in the training dataset using the fine-tuned semantic encoder to generate a deep semantic feature vector for each news text. The multidimensional statistical feature extraction module is used to perform batch feature calculations on the text data in the training dataset based on the enhanced statistical feature extraction tool to obtain multidimensional statistical feature vectors; wherein, the enhanced statistical feature extraction tool is used to extract basic text statistical indicators and custom statistical indicators designed for text generated by large language models; The cross-modal feature fusion module is used to input the deep semantic feature vector and the multi-dimensional statistical feature vector into the multi-granularity cross-modal attention fusion module, establish the interaction between features through a bidirectional cross-attention mechanism, and dynamically weight and fuse the interacting features based on a gating mechanism to generate a unified fused feature representation. The classifier training optimization module is used to train the classifier network using the fused feature representation to obtain a trained fake news detection model; The real-time detection and judgment module is used to process the unprocessed fusion feature representation corresponding to the news text to be detected based on the trained fake news detection model, and output the judgment result of whether the news text to be detected is fake news generated by the large language model; wherein, the unprocessed fusion feature representation is obtained in the same way as the fusion feature representation.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images