A three-classification hybrid transfer learning method and system for "information epidemic" false information discrimination
By combining a three-class hybrid transfer learning method with TF-IDF, BERT, TextCNN and fastText models, the problem of accurately classifying false information in the "information epidemic" was solved, achieving higher accuracy and generalization ability, and adapting to information identification in different language environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF TECH
- Filing Date
- 2025-02-10
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies have failed to fully utilize the characteristics of health information and the expertise of healthcare workers during the "information epidemic," resulting in low accuracy in identifying misinformation and complex models that are prone to overfitting, making it difficult to effectively classify information into three categories: true, false, and uncertain.
A three-class hybrid transfer learning approach is adopted, combining TF-IDF, BERT, TextCNN and fastText models. Through pre-training and fine-tuning of the model, key feature words and word embedding representations are extracted, and the text output is fused for binary and three-class training. An "undetermined" category is introduced, and the professional knowledge of health workers is used for further subdivision.
It improves the accuracy and generalization ability of identifying misinformation, reduces overfitting, and can more accurately identify misinformation in the "information epidemic." It adapts to different language environments and improves recognition efficiency and accuracy.
Smart Images
Figure CN120067415B_ABST
Abstract
Description
A three-class hybrid transfer learning method and system for identifying misinformation during the "information epidemic" Technical Field
[0001] This invention relates to the field of information management technology, and in particular to a three-category hybrid transfer learning method and system for identifying false information related to "information epidemics". Background Technology
[0002] The term "infodemic" is a combination of "information" and the root word "epidemic" (-demic), referring to the phenomenon of excessive dissemination of information in the context of an infectious disease outbreak. During an infectious disease outbreak, the public's demand for health information increases dramatically, making health information a crucial component of the "infodemic."
[0003] Xu et al. proposed a health rumor detection method based on I-BERT-BiLSTM. This method extracts document-level summaries of long text sequences and feeds them into a deep neural network with a multi-layer attention mechanism for feature extraction. Finally, the summaries are fed into a BiLSTM for rumor classification. Experimental results show that the model achieves high accuracy on a self-built Chinese health rumor dataset containing 2000 health rumors and 2000 non-rumor health data.
[0004] Wang Xiaoyan et al. mined 18 features of WeChat health-related rumors in terms of textual rhetoric, posting motivation, publishing standards, and editing layout. They used a deep neural network model to obtain pre-classification results, which were then used as deep semantic feature representations. This expanded the manually constructed feature set, and combined with a machine learning model, they completed the task of detecting health-related rumors. Experimental results verified the effectiveness and innovation of this method in detecting WeChat health-related rumors. The dataset used included 1092 real articles and 1015 rumor articles.
[0005] Hussain et al. proposed a cascaded multi-head attention model for fake news detection. This model combines deep convolution and cascaded multi-head attention mechanisms, capturing information from both local and global contexts through multi-head attention to achieve a comprehensive understanding of fake news. On the Urdu Twitter dataset containing both accurate tweets (942) and fake tweets (2676), the model outperformed state-of-the-art models in classification.
[0006] Xia et al. proposed a fake news detection framework based on complex adaptive systems theory and information dissemination theory. This framework constructs a knowledge base by extracting and integrating anomalous knowledge from fake news, and applies it to a hybrid CNN-BiLSTM-AM model for fake news detection. The model's application on an English dataset (including 5600 real news items and 5100 fake news items) significantly improved the performance of various evaluation metrics.
[0007] The above studies primarily focus on categorizing "information epidemics" into true and false information, failing to fully utilize the unique characteristics of health information within these epidemics. The expertise of healthcare workers in the field of health information can provide effective support for identifying misinformation during "information epidemics." Specifically, misinformation within "information epidemics" can be further subdivided into three categories: the first category is information that healthcare workers cannot independently determine as true or false; the second category is information that healthcare workers can independently determine as false; and the third category is information that healthcare workers can independently determine as true.
[0008] To address the complexity of distinguishing between true and false information in the "information epidemic," Luo et al. proposed a three-class classification detection method based on a deep learning model. This study, combining the professional knowledge of healthcare workers in the field of health information, categorizes the "information epidemic" into three classes: true, false, and uncertain. Classification experiments were conducted on Chinese and English data using fastText, three RNN-based models, two CNN-based models, and two Transformer-based models. The results show that, due to the limited amount of data for the three-class classification task of the "information epidemic," pre-trained or simpler models outperform complex models, while complex models are prone to overfitting when handling such problems.
[0009] Few studies have focused on leveraging the characteristics of health information in an "infodemic" and the expertise of healthcare workers in the field to effectively support the identification of misinformation during an "infodemic." Furthermore, the three-class classification task for an "infodemic" is limited by insufficient data, making it difficult to improve model accuracy, and complex models are more prone to overfitting when handling such tasks.
[0010] Dividing the "information epidemic" into true and false categories fails to fully utilize the characteristics of health information within the "information epidemic" and also fails to fully leverage the professional knowledge of health workers in the field of health information. This results in a low actual accuracy rate in identifying false information about the "information epidemic," thus limiting its practical application effectiveness.
[0011] The "Information Epidemic" three-classification task, which integrates the characteristics of health information and the professional knowledge of health workers in the field of health information, is limited by insufficient data, making it difficult to improve the accuracy of the model. Moreover, complex models are more prone to overfitting when dealing with such tasks. Summary of the Invention
[0012] The technical problem to be solved by this invention is to provide a three-class hybrid transfer learning method and system for identifying false information in "information epidemics", which can more accurately identify false information in "information epidemics" and increase the practical application effect of the method.
[0013] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0014] Firstly, a three-class hybrid transfer learning system for identifying misinformation related to the "information epidemic" is provided, the system comprising: a pre-trained model and a fine-tuned model;
[0015] TF-IDF technology was used to preprocess data related to the "information epidemic" and extract key feature words from it;
[0016] Using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation.
[0017] The word embeddings of the keyword "information epidemic" are fused with the BERT word embeddings and text outputs of regular misinformation to obtain fused data; using the fused data, binary classification training is performed based on the regular "false" or "true" labels to build a pre-trained model;
[0018] In the fine-tuned model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model integrates the keywords "information epidemic" and general false information.
[0019] By combining the BERT model, TextCNN model, and fastText model, the input features of the BERT model are processed, and the fine-tuning model is used to train the processed features to subdivide the data related to the "information epidemic" into three categories: "undetermined", "false", or "real".
[0020] Furthermore, TF-IDF technology was used to preprocess the data related to the "information epidemic" to extract key feature words, including:
[0021] Preprocessing of "information epidemic" records using TF-IDF yielded the top 10% of keywords, where "information epidemic" is represented as... Each This represents a record of an "information epidemic";
[0022] For each record Word segmentation was performed. For Chinese text, the Jieba word segmentation tool was used; for English text, regular expressions were used for word segmentation. TF-IDF was then used to process the segmented "information epidemic" records, calculating the value of each term t in the document. The TF-IDF score is calculated using the following formula:
[0023] ;
[0024] in, , N represents the total number of records.
[0025] Furthermore, using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation, including:
[0026] The keyword set for "information epidemic" is represented as follows: Where k represents the number of informational epidemic keywords, and each keyword After processing by the BERT model, corresponding word embeddings are generated. The calculation formula is as follows:
[0027] ;
[0028] in, This indicates the operation of generating word embeddings using the BERT model. This indicates the result of generating word embeddings using the BERT model.
[0029] This represents the labeled sequence of the j-th instance of general misinformation, processed using the BERT model. ,generate The word embedding representation is calculated using the following formula;
[0030] ;
[0031] in, This indicates the operation of generating word embeddings using the BERT model. This indicates the result of generating word embeddings using the BERT model.
[0032] By integrating word embeddings of common misinformation and the keyword "infodemic," a unified representation is created, which embeds words related to common misinformation. Word embedding with each "information epidemic" keyword The concatenation and flattening of the result into a single vector are calculated using the following formula:
[0033] ;
[0034] in, This indicates a splicing operation. Indicates the splicing method. Indicates the flattening operation. This indicates word embedding that combines conventional misinformation with the keyword "information epidemic".
[0035] Furthermore, using the fused data and performing binary classification training based on conventional "fake" or "real" labels, a pre-trained model is constructed, including:
[0036] Processing with BERT model ,generate The text output includes an embedding vector with contextual information and a pooled output. The pooled output is calculated using the following formula:
[0037] ;
[0038] in, This indicates the operation of generating pooled output using the BERT model. This indicates the result of generating pooled output using the BERT model.
[0039] Will and By combining these methods and processing the fused feature vectors through a fully connected layer, binary classification prediction is performed on conventional misinformation incorporating the keyword "information epidemic." The calculation formula is as follows:
[0040] ;
[0041] Where fc represents a fully connected layer. It is the sigmoid activation function. In a binary classification task, predict the probability of belonging to the "true" category.
[0042] Furthermore, in the fine-tuned model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model, which integrates the keywords "information epidemic" and general misinformation, includes:
[0043] The conventional false information text output generated by the BERT layer in the pre-trained model is used to obtain a batch of "information epidemic" text containing contextual information embedding vectors and pooled outputs. The calculation formula is as follows:
[0044] ;
[0045] ;
[0046] in, This represents the labeled sequence of the input batch of "information epidemic" instances. This indicates the operation of generating pooled output using the BERT model. This indicates the result of generating pooled output using the BERT model. This indicates that the pooling output of the BERT model is a matrix, and the dimension of this matrix is... , Let B be the set of real numbers, B be the batch size, and L be the sequence length. This indicates the operation of generating contextualized embedding vectors using the BERT model. `contextualized_token_embedding` represents the result of generating contextualized embedding vectors using the BERT model. `H` indicates that the result of the BERT model generating contextualized embedding vectors is a matrix with dimension H. H is the dimension of the hidden layer;
[0047] The pooling output is processed by the dropout layer, and the embedding vector containing contextual information is processed by the TextCNN pooling layer and the fastText pooling layer. The calculation formula is as follows:
[0048] ;
[0049] ;
[0050] ;
[0051] ;
[0052] ;
[0053] in, This represents the result after applying dropout regularization to the pooled output; and This is the output of the pooling layer in TextCNN. The pooling layer applies a kernel size of [kernel size missing] to the embedding vectors containing contextual information. and The convolution operation performs adaptive max pooling. Conv1D represents a one-dimensional convolution operation, and AdaptiveMaxPool1D represents an adaptive max pooling operation. and This is the output of the fastText pooling layer. The pooling layer performs adaptive average pooling and adaptive max pooling on the embedding vectors containing context information, respectively. AdaptiveAvgPool1D represents the adaptive average pooling operation.
[0054] Furthermore, by combining the BERT, TextCNN, and FastText models, the input features of the BERT model are processed, and the fine-tuned model is used to train the processed features to subdivide the data related to the "information epidemic" into three categories: "uncertain," "false," or "true," including:
[0055] The results of the dropout layer, TextCNN pooling layer, and fastText pooling layer are concatenated to obtain the final feature representation, calculated as follows:
[0056] ;
[0057] Where F represents the concatenated feature vector;
[0058] The concatenated feature vector is passed to a fully connected layer to perform a three-class classification prediction on the "information epidemic" records. The calculation formula is as follows:
[0059] ;
[0060] in, This represents the softmax activation function. This represents the predicted probability distribution of each category in a three-class classification task.
[0061] Secondly, a three-category hybrid transfer learning method for identifying misinformation related to the "information epidemic" includes:
[0062] TF-IDF technology was used to preprocess data related to the "information epidemic" and extract key feature words from it;
[0063] Using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation.
[0064] The word embeddings of the keyword "information epidemic" are fused with the BERT word embeddings and text outputs of regular misinformation to obtain fused data; using the fused data, binary classification training is performed based on the regular "false" or "true" labels to build a pre-trained model;
[0065] In the fine-tuned model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model integrates the keywords "information epidemic" and general false information.
[0066] By combining the BERT model, TextCNN model, and fastText model, the input features of the BERT model are processed, and the fine-tuning model is used to train the processed features to subdivide the data related to the "information epidemic" into three categories: "undetermined", "false", or "real".
[0067] Thirdly, a computing device includes:
[0068] One or more processors;
[0069] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.
[0070] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.
[0071] The above-described solution of the present invention has at least the following beneficial effects:
[0072] 1. Leveraging the expertise of healthcare workers in the field of health information, “information epidemics” are categorized into three types. A fine-tuned model is developed by combining BERT, TextCNN, and fastText to classify “information epidemics” into three categories: uncertain, false, or real, thereby enhancing the practical application effectiveness of the method.
[0073] 2. Effectively utilize common misinformation labeled as false or true, combined with the keyword "information epidemic," to design a pre-trained model. By incorporating a large amount of common misinformation with the keyword "information epidemic," the accuracy and generalization ability of the classification method are improved, mitigating the overfitting phenomenon that often occurs in such tasks. Attached Figure Description
[0074] Figure 1 is a schematic diagram of a three-class hybrid transfer learning system for identifying false information related to "information epidemics" provided by an embodiment of the present invention.
[0075] Figure 2 is a schematic diagram of the confusion matrix of the three-classification task for identifying false information about the Chinese "information epidemic" of the present invention.
[0076] Figure 3 is a schematic diagram of the confusion matrix of the three-classification task for identifying false information about the English "information epidemic" in this invention. Detailed Implementation
[0077] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0078] As shown in Figure 1, an embodiment of the present invention proposes a three-class hybrid transfer learning system for identifying false information in the "information epidemic", including a pre-trained model and a fine-tuning model;
[0079] TF-IDF technology was used to preprocess data related to the "information epidemic" and extract key feature words from it;
[0080] Using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation.
[0081] The word embeddings of the keyword "information epidemic" are fused with the BERT word embeddings and text outputs of regular misinformation to obtain fused data; using the fused data, binary classification training is performed based on the regular "false" or "true" labels to build a pre-trained model;
[0082] In the fine-tuned model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model integrates the keywords "information epidemic" and general false information.
[0083] By combining the BERT model, TextCNN model, and fastText model, the input features of the BERT model are processed, and the fine-tuning model is used to train the processed features to subdivide the data related to the "information epidemic" into three categories: "undetermined", "false", or "real".
[0084] In this embodiment of the invention, by combining the BERT model, TextCNN model, and fastText model, the system can comprehensively utilize the advantages of multiple deep learning models to capture text features from different perspectives, thereby improving the accuracy of classifying data related to "information epidemics." This multi-model fusion approach helps reduce the limitations of a single model, making the classification results more reliable. Preprocessing the data using TF-IDF technology can effectively extract key feature words, which helps the system understand and analyze text data more accurately. Simultaneously, the word embeddings generated by the BERT model can capture the contextual information of words, further enhancing the system's ability to process complex text data. Through the combination of pre-trained and fine-tuned models, the system can utilize the general knowledge learned by the pre-trained model and optimize for specific tasks through fine-tuning. This transfer learning method enables the system to better adapt to different data and task requirements, improving the system's flexibility. Traditional binary classification models often only classify data into "real" or "fake," but in practical applications, the authenticity of some information may not be clearly determined. This system, by incorporating "undetermined" categories and the expertise of healthcare workers in the field of health information, can more accurately reflect the true situation of "infodemic" data, avoiding misjudgments of uncertain information and improving the efficiency of identifying misinformation. In the context of public health emergencies, misinformation about "infodemics" spreads extremely rapidly on social media and the internet; therefore, timely and accurate identification of such misinformation is crucial. Through the rapid training and inference capabilities of deep learning models, this system can quickly classify and process large amounts of "infodemic" data, improving the efficiency of misinformation identification.
[0085] The COVID-19 Information Datasets: These datasets use both Chinese and English text, containing social media text records annotated by healthcare workers. All records are categorized as uncertain, false, or true. The Chinese dataset contains 1,055 records, of which 435 are marked as uncertain, 281 as false, and 339 as true. Data sources include manually verified Weibo posts, the WeChat mini-program "Xiaozhen," and other authoritative sources. The English dataset contains 2,490 records, 830 in each category. Data sources include public fact-checking websites, the Twitter API, and other social media platforms.
[0086] The standard dataset for misinformation includes a Chinese dataset sourced from the Beijing Academy of Computing Technology's Internet Fake News Detection Challenge. After removing low-quality entries, the dataset contains 38,455 records, of which 19,170 are marked as fake and 19,284 as real. The English dataset is from the Kaggle Fake News Detection Datasets, with each record containing a title and body. Since the body text in the real category is typically longer and more formal, while the body text in the fake category is shorter, only the title was selected. Many titles in the fake category begin each word with a capital letter, a rule not followed in the real category. To ensure consistency, all capital letters in the titles of both categories were converted to lowercase. Additionally, some titles in the fake category ended with bracket labels (such as (VIDEO), (IMAGE), or (TWEET)), which were removed. After removing low-quality entries, this dataset contains 42,986 records, of which 21,569 are marked as fake and 21,417 as real.
[0087] In a preferred embodiment of the present invention, TF-IDF technology is used to preprocess data related to the "information epidemic" to extract key feature words, including:
[0088] Preprocessing of "information epidemic" records using TF-IDF yielded the top 10% of keywords, where "information epidemic" is represented as... Each This represents a record of an "information epidemic";
[0089] For each record Word segmentation was performed. For Chinese text, the Jieba word segmentation tool was used; for English text, regular expressions were used for word segmentation. TF-IDF was then used to process the segmented "information epidemic" records, calculating the value of each term t in the document. The TF-IDF score is calculated using the following formula:
[0090] ;
[0091] in, , N represents the total number of records.
[0092] In this embodiment of the invention, by extracting the top 10% of keywords, the dimensionality of the data can be significantly reduced. This facilitates subsequent model training and inference, reduces computational complexity, and improves processing efficiency. TF-IDF technology can assign higher weights to terms that frequently appear in a particular "information epidemic" record but are uncommon in other records. Thus, the extracted keywords better reflect the uniqueness and importance of each record. By extracting the keywords most relevant to "information epidemic," a large amount of redundant and irrelevant information can be removed, making the data input into the subsequent pre-trained model more refined and effective. By using the Jieba word segmentation tool to process Chinese text and regular expressions to process English text, the system can adapt to the word segmentation needs of different languages. This multilingual processing capability makes the system more flexible and versatile, capable of handling "information epidemic" data from different language environments.
[0093] In a preferred embodiment of the present invention, the BERT model is used to generate word embedding representations for the extracted keywords "information epidemic" and conventional misinformation, including:
[0094] The keyword set for "information epidemic" is represented as follows: Where k represents the number of informational epidemic keywords, and each keyword After processing by the BERT model, corresponding word embeddings are generated. The calculation formula is as follows:
[0095] ;
[0096] in, This indicates the operation of generating word embeddings using the BERT model. This indicates the result of generating word embeddings using the BERT model.
[0097] This represents the labeled sequence of the j-th instance of general misinformation, processed using the BERT model. ,generate The word embedding representation is calculated using the following formula;
[0098] ;
[0099] in, This indicates the operation of generating word embeddings using the BERT model. This indicates the result of generating word embeddings using the BERT model.
[0100] By integrating word embeddings of common misinformation and the keyword "infodemic," a unified representation is created, which embeds words related to common misinformation. Word embedding with each "information epidemic" keyword The concatenation and flattening of the result into a single vector are calculated using the following formula:
[0101] ;
[0102] in, This indicates a splicing operation. Indicates the splicing method. Indicates the flattening operation. This indicates word embedding that combines conventional misinformation with the keyword "information epidemic".
[0103] In this embodiment of the invention, the BERT model is a pre-trained deep bidirectional model capable of generating word embeddings rich in contextual information. Processing the keywords "information epidemic" and general misinformation with BERT yields more accurate and richer semantic representations, aiding subsequent classification tasks. By concatenating the word embeddings of general misinformation with the word embeddings of the "information epidemic" keyword, the system can simultaneously capture the overall semantics of the misinformation and specific keyword information related to "information epidemic." This fusion method enhances the completeness and specificity of the feature representation, helping the subsequent fine-tuning of the model to more accurately identify misinformation related to "information epidemic." Using the fused word embeddings as input features provides the subsequent classifier with more comprehensive information, thereby improving classification accuracy. This representation method fully considers the correlation between the "information epidemic" keyword and general misinformation, helping the model make more accurate judgments. Although multiple word embeddings are fused, flattening them into a single vector effectively reduces the dimensionality of the features. This helps reduce the computational complexity of the subsequent classifier and improves processing efficiency.
[0104] In a preferred embodiment of the present invention, the fused data is used, and binary classification training is performed based on conventional "fake" or "real" labels to construct a pre-trained model, including:
[0105] Processing with BERT model ,generate The text output includes an embedding vector with contextual information and a pooled output. The pooled output is calculated using the following formula:
[0106] ;
[0107] in, This indicates the operation of generating pooled output using the BERT model. This indicates the result of generating pooled output using the BERT model.
[0108] Will and By combining these methods and processing the fused feature vectors through a fully connected layer, binary classification prediction is performed on conventional misinformation incorporating the keyword "information epidemic." The calculation formula is as follows:
[0109] ;
[0110] Where fc represents a fully connected layer. It is the sigmoid activation function. In a binary classification task, predict the probability of belonging to the "true" category.
[0111] In this embodiment of the invention, the pre-trained model is trained on a large amount of data, enabling it to learn general language representations and patterns. This gives the model better generalization ability when processing new data, allowing it to make more accurate judgments even when faced with unfamiliar information. Through pre-training, the model has learned rich language knowledge and patterns, thus allowing for faster convergence and reduced training time during subsequent fine-tuning for specific tasks. For data in the specific domain of "information epidemic," by incorporating relevant keywords, the model can better capture the specific features and patterns of this domain. This makes the model more sensitive and accurate when processing information related to "information epidemic." The pre-trained model simplifies the design of subsequent task models. It provides powerful feature extraction capabilities, allowing subsequent models to focus only on the classification layer design for specific tasks, reducing the complexity of model design. Pre-trained models are typically trained on large-scale datasets, making them more robust to various linguistic phenomena and noisy data.
[0112] In a preferred embodiment of the present invention, in the fine-tuning model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model, which integrates the keywords "information epidemic" and general false information, includes:
[0113] The conventional false information text output generated by the BERT layer in the pre-trained model is used to obtain a batch of "information epidemic" text containing contextual information embedding vectors and pooled outputs. The calculation formula is as follows:
[0114] ;
[0115] ;
[0116] in, This represents the labeled sequence of the input batch of "information epidemic" instances. This indicates the operation of generating pooled output using the BERT model. This indicates the result of pooling output generated using the BERT model, where P indicates that the pooling output generated by the BERT model is a matrix with dimension 1. , Let B be the set of real numbers, B be the batch size, and L be the sequence length. This indicates the operation of generating contextualized embedding vectors using the BERT model. `contextualized_token_embedding` represents the result of generating contextualized embedding vectors using the BERT model. `H` indicates that the result of the BERT model generating contextualized embedding vectors is a matrix with dimension H. H is the dimension of the hidden layer;
[0117] The pooling output is processed by the dropout layer, and the embedding vector containing contextual information is processed by the TextCNN pooling layer and the fastText pooling layer. The calculation formula is as follows:
[0118] ;
[0119] ;
[0120] ;
[0121] ;
[0122] ;
[0123] in, This represents the result after applying dropout regularization to the pooled output; and This is the output of the pooling layer in TextCNN. The pooling layer applies a kernel size of [kernel size missing] to the embedding vectors containing contextual information. and The convolution operation performs adaptive max pooling. Conv1D represents a one-dimensional convolution operation, and AdaptiveMaxPool1D represents an adaptive max pooling operation. and This is the output of the fastText pooling layer. The pooling layer performs adaptive average pooling and adaptive max pooling on the embedding vectors containing context information, respectively. AdaptiveAvgPool1D represents the adaptive average pooling operation.
[0124] In this embodiment of the invention, by introducing an "undetermined" category, the model can handle records that cannot be clearly classified as "false" or "true" more precisely. This three-classification (false, true, undetermined) method helps reduce misjudgments and improve the accuracy of information classification. The text output generated by the BERT model, including embedding vectors with contextual information and pooled output, provides the model with richer and deeper semantic information. This helps the model make more robust judgments when dealing with complex or ambiguous information. By combining TextCNN and fastText pooling layers, the model can extract text features from multiple perspectives. TextCNN efficiently extracts local patterns and salient features from word embeddings, making it ideal for identifying keyword phrases and local dependencies. FastText captures global sequence-level features, providing a lightweight and effective solution for representing overall trends and key features in scenarios with limited data. This multi-layered feature extraction method improves the model's ability to understand textual information. Applying dropout regularization to the pooled output helps reduce the risk of overfitting during training. Dropout randomly discards some network connections, preventing the model from over-relying on certain connections during training, thereby enhancing the model's generalization ability.
[0125] In a preferred embodiment of the present invention, the input features of the BERT model are processed by combining the BERT model, the TextCNN model, and the fastText model. The processed features are then trained using a fine-tuning model to further categorize the data related to the "information epidemic" into three classes: "undetermined," "false," or "true," including:
[0126] The results of the dropout layer, TextCNN pooling layer, and fastText pooling layer are concatenated to obtain the final feature representation, calculated as follows:
[0127] ;
[0128] Where F represents the concatenated feature vector;
[0129] The concatenated feature vector is passed to a fully connected layer to perform a three-class classification prediction on the "information epidemic" records. The calculation formula is as follows:
[0130] ;
[0131] in, This represents the softmax activation function. This represents the predicted probability distribution of each category in a three-class classification task.
[0132] In this embodiment of the invention, by concatenating the processing results of the dropout layer, the TextCNN pooling layer, and the fastText pooling layer, the model can fuse feature information from different levels. This feature fusion method enhances the model's ability to represent text data, enabling it to capture semantic and structural information in the text more comprehensively. TextCNN efficiently extracts local patterns and salient features from word embeddings, while fastText captures global sequence-level features. Combining the outputs of these models with the pooled output of BERT after dropout regularization allows the fine-tuned model to understand text from multiple perspectives, improving its ability to process complex text data. Introducing an "undetermined" category and fine-tuning it based on the pre-trained model allows the model to better adapt to small datasets in three-class classification tasks, not only improving model performance but also effectively saving computational resources and time. The entire process achieves end-to-end training; from the original text input to the final three-class classification output, all components can be optimized within a unified framework. This end-to-end training approach helps the model better adapt to task requirements, improving training efficiency and model performance.
[0133] Table 1. Test Results of the Three-Classification Task for Identifying False Information about the Chinese "Information Epidemic"
[0134]
[0135] Table 2. Test Results of the Three-Category Task for Identifying False Information about the "Information Pandemic" in English
[0136]
[0137] Table 3. Test Results of Each Category in the Three-Category Task for Identifying False Information about the Chinese "Information Epidemic"
[0138]
[0139] Note: 0, 1 and 2 represent “information epidemic” data marked as “uncertain”, “false” and “true”, respectively.
[0140] Table 4. Test Results of Each Category in the Three-Category Task for Identifying False Information about the "Information Pandemic" in English
[0141]
[0142] Note: 0, 1 and 2 represent “information epidemic” data marked as “uncertain”, “false” and “true”, respectively.
[0143] Embodiments of the present invention also provide a three-class hybrid transfer learning method for identifying misinformation related to an "information epidemic," comprising:
[0144] TF-IDF technology was used to preprocess data related to the "information epidemic" and extract key feature words from it;
[0145] Using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation.
[0146] The word embeddings of the keyword "information epidemic" are fused with the BERT word embeddings and text outputs of regular misinformation to obtain fused data; using the fused data, binary classification training is performed based on the regular "false" or "true" labels to build a pre-trained model;
[0147] In the fine-tuned model, the "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true"; the text output generated by the BERT layer of the pre-trained model integrates the keywords "information epidemic" and general false information.
[0148] By combining the BERT model, TextCNN model, and fastText model, the input features of the BERT model are processed, and the fine-tuning model is used to train the processed features to subdivide the data related to the "information epidemic" into three categories: "undetermined", "false", or "real".
[0149] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.
[0150] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.
[0151] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.
Claims
1. A three-category hybrid transfer learning system for identifying misinformation related to the "information epidemic," characterized in that: The system includes a pre-trained model and a fine-tuning model. It preprocesses data related to "information epidemic" using TF-IDF technology to extract key feature words. A BERT model is used to generate word embeddings for the extracted "information epidemic" keywords and general misinformation. The word embeddings of the "information epidemic" keywords are fused with the BERT word embeddings of general misinformation and their text output. The text output is generated by processing general misinformation using the BERT model to obtain fused data. The fused data is then used for binary classification training based on the general "false" or "true" labels to construct the pre-trained model. In the fine-tuning model, an "undetermined" category is used to classify records that cannot be clearly determined as "false" or "true." The BERT layer of the pre-trained model generates a fused text output of "information epidemic" keywords and general misinformation. The BERT model, TextCNN model, and FastText model are combined to process the input features of the BERT model. The fine-tuning model is then used to train the processed features to subdivide the "information epidemic" related data into three categories: "undetermined," "false," or "true." 2. The three-category hybrid transfer learning system for identifying misinformation related to "information epidemics" as described in claim 1, characterized in that, Data related to "information epidemic" was preprocessed using TF-IDF technology to extract key feature words. This included preprocessing "information epidemic" records using TF-IDF to extract the top 10% of keywords, where "information epidemic" was represented as... Each This represents one "information epidemic" record; for each record Word segmentation was performed. For Chinese text, the Jieba word segmentation tool was used; for English text, regular expressions were used for word segmentation. TF-IDF was then used to process the segmented "information epidemic" records, calculating the value of each term t in the document. The TF-IDF score is calculated using the following formula: ;in, ， N represents the total number of records.
3. A three-category hybrid transfer learning system for identifying misinformation related to "information epidemics" as described in claim 2, characterized in that, Using the BERT model, word embedding representations are generated for the extracted keywords "information epidemic" and common misinformation, including: the set of keywords for "information epidemic" is represented as follows: Where k represents the number of informational epidemic keywords, and each keyword After processing by the BERT model, corresponding word embeddings are generated. The calculation formula is as follows: ;in, This indicates the operation of generating word embeddings using the BERT model. This indicates the result of generating word embeddings using the BERT model; This represents the labeled sequence of the j-th instance of general misinformation, processed using the BERT model. ,generate The word embedding representation is calculated using the following formula; ;in, This indicates the operation of generating word embeddings using the BERT model. This represents the result of word embeddings generated using the BERT model; by fusing word embeddings of common misinformation and the keyword "information epidemic," a unified representation is created, which includes word embeddings of common misinformation. Word embedding with each "information epidemic" keyword The concatenation and flattening of the result into a single vector are calculated using the following formula: ;in, This indicates a splicing operation. Indicates the splicing method. Indicates the flattening operation. This indicates word embedding that combines conventional misinformation with the keyword "infodemic".
4. A three-category hybrid transfer learning system for identifying misinformation related to "information epidemics" as described in claim 3, characterized in that, Using the fused data and performing binary classification training based on conventional "fake" or "real" labels, a pre-trained model is constructed, including: processing data using the BERT model. ,generate The text output includes an embedding vector with contextual information and a pooled output. The pooled output is calculated using the following formula: ;in, This indicates the operation of generating pooled output using the BERT model. This indicates the result of generating pooled output using the BERT model; and By combining these methods and processing the fused feature vectors through a fully connected layer, binary classification prediction is performed on conventional misinformation incorporating the keyword "information epidemic." The calculation formula is as follows: Where fc represents a fully connected layer, It is the sigmoid activation function. In a binary classification task, predict the probability of belonging to the "true" category.
5. A three-category hybrid transfer learning system for identifying misinformation related to "information epidemics" as described in claim 4, characterized in that, In the fine-tuning model, the "Undetermined" category is used to classify records that cannot be clearly determined as "false" or "true". The text output generated by the BERT layer of the pre-trained model, which integrates the keywords "information epidemic" and general misinformation, includes: inputting the general misinformation text output generated by the BERT layer in the pre-trained model, obtaining a batch of "information epidemic" text embedding vectors with contextual information, and pooling output. The calculation formula is as follows: ； ;in, This represents the labeled sequence of the input batch for the "Information Epidemic" instance. This indicates the operation of generating pooled output using the BERT model. This indicates the result of pooling output generated using the BERT model, where P indicates that the pooling output generated by the BERT model is a matrix with dimension 1. ， Let B be the set of real numbers, B be the batch size, and L be the sequence length. This indicates the operation of generating contextualized embedding vectors using the BERT model. `contextualized_token_embedding` represents the result of generating contextualized embedding vectors using the BERT model. `H` indicates that the result of the BERT model generating contextualized embedding vectors is a matrix with dimension H. H is the hidden layer dimension; the pooling output is processed by the dropout layer, and the embedding vector containing contextual information is processed by the TextCNN pooling layer and the fastText pooling layer. The calculation formula is as follows: ； ； ； ； ;in, This represents the result after applying dropout regularization to the pooled output; and This is the output of the pooling layer in TextCNN. The pooling layer applies a kernel size of [kernel size missing] to the embedding vectors containing contextual information. and The convolution operation performs adaptive max pooling. Conv1D represents a one-dimensional convolution operation, and AdaptiveMaxPool1D represents an adaptive max pooling operation. and This is the output of the fastText pooling layer. The pooling layer performs adaptive average pooling and adaptive max pooling on the embedding vectors containing context information, respectively. AdaptiveAvgPool1D represents the adaptive average pooling operation.
6. A three-category hybrid transfer learning system for identifying misinformation related to "information epidemics" as described in claim 5, characterized in that, This paper combines the BERT, TextCNN, and FastText models to process the input features of the BERT model. The fine-tuned model is then used to train the processed features to categorize "information epidemic" related data into three classes: "uncertain," "false," and "real." This process involves concatenating the results of dropout, TextCNN, and FastText pooling layers to obtain the final feature representation. The calculation formula is as follows: Where F represents the concatenated feature vector; the concatenated feature vector is passed to the fully connected layer to perform a three-class classification prediction on the "information epidemic" records, and the calculation formula is as follows: ;in, This represents the softmax activation function. This represents the predicted probability distribution of each category in a three-class classification task.
7. A three-class hybrid transfer learning method for identifying misinformation related to the "information epidemic," characterized in that... The system, as described in any one of claims 1 to 6, comprises: preprocessing data related to "information epidemic" using TF-IDF technology to extract key feature words; generating word embedding representations for the extracted "information epidemic" keywords and general misinformation using a BERT model; fusing the word embeddings of the "information epidemic" keywords with the BERT word embeddings of general misinformation and their text output to obtain fused data; using the fused data and performing binary classification training based on general "false" or "true" labels to construct a pre-trained model; in the fine-tuning model, using the "undetermined" category to classify records that cannot be clearly determined as "false" or "true"; using the text output generated by the BERT layer of the pre-trained model that fuses the "information epidemic" keywords and general misinformation; processing the input features of the BERT model by combining the BERT model, TextCNN model, and fastText model, and training the processed features using the fine-tuning model to subdivide the data related to "information epidemic" into three categories: "undetermined," "false," or "true." 8. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to perform the method as described in claim 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in claim 7.