LLM enhanced contrast learning-based social network false information detection method

Abstract

A social network false information detection method based on LLM enhanced comparative learning is used for detecting social network false information under a large language model LLM assisted comparative learning CL framework, and comprises the following steps: S1, data preprocessing and Prompt arrangement: capturing original data from a social network, and formatting; inputting the processed data into a Prompt composer to generate enhanced data so as to expand a training sample of the CL model; s2, LLM-assisted feature extraction and label prediction: matching the data enhanced in the step S1 with original data, and inputting the matched data into a CL model; the CL model extracts discriminative features of false information and performs label prediction; after training is completed, parameters of the CL model are frozen; in the test stage, data are input into the CL model to extract features, and labels are predicted through a classifier; and S3, joint fine tuning: guiding LLM fine tuning according to the contrast loss of the CL model.

Description

Technical Field

[0001] The invention belongs to the field of network technology, and specifically is a social network false information detection method based on LLM enhanced contrast learning. Background Art

[0002] The rapid development of the Internet has provided an unprecedented and convenient way for the widespread dissemination of information, but it has also brought new challenges to the generation and spread of false information and rumors. The spread of rumors not only increases the information burden on the public, but can also lead to serious misleading effects and even cause social panic. In order to curb the spread of rumors, social networking platforms have gradually increased their supervision and adopted measures such as manual review and account restrictions. However, these methods are costly and may be affected by the subjective bias of reviewers. In order to improve the efficiency of supervision, deep neural networks (DNN) are used to assist manual review. In this process, supervised learning uses labeled data to train models and has become the main means of rumor detection [1][2].

[0003] Although supervised learning methods have achieved remarkable results in rumor detection, their reliance on large amounts of labeled data and lack of flexibility in dealing with emerging and rapidly changing rumors have gradually become apparent. In response, how to efficiently utilize unlabeled data has begun to attract attention. Contrastive Learning (CL), as an emerging self-supervised learning paradigm, has shown broad potential and has been applied to tasks such as image classification [3][4], object detection [5][6], and semantic segmentation [7][8]. In the field of Natural Language Processing (NLP), CL has shown excellent performance in tasks such as text classification [9], named entity recognition

[10] , and machine translation

[11] . The CL model combined with Graph Neural Networks (GNN)

[12]

[13] can effectively analyze complex data structures and deeply mine information propagation characteristics, providing strong technical support for detecting false information in social networks.

[0004] The emotions and opinions of social media users play a key role in evaluating the authenticity of information

[14] . The content generated by users on the platform (such as Weibo interactions) not only reveals the propagation path of false information, but also helps to understand its diffusion pattern in social networks. Existing mainstream GNN-based CL methods mostly use strategies such as DropEdge

[15] , AdaEdge

[16] , and NodeAug

[17] . Although they can reveal the rumor propagation path and network structure characteristics, they often ignore the emotional tendencies and opinion diversity contained in user replies, thereby limiting the accuracy and expressiveness of the model. Similarly, although the NLP enhancement technology proposed in the literature

[18] (such as synonym replacement, sentence rearrangement, etc.) can enhance the model's understanding of text structure and semantics, it has limitations when dealing with subtle language expressions and complex emotional levels, resulting in deviations in the judgment of false information. Specifically, the following are the reasons: (1) the model cannot fully understand and distinguish users' diverse emotional reactions and attitudes, thereby reducing its insight into the motivations and impacts of rumor spreading; (2) it performs poorly when processing complex and controversial information, and has difficulty accurately identifying potentially misleading content; (3) key semantic clues such as sarcasm, metaphors, or expressions related to cultural background are easily overlooked.

[0005] In recent years, large language models (LLMs) have revolutionized the traditional NLP paradigm with their large parameter scale and massive pre-training data. Compared with traditional NLP models, LLMs have demonstrated superior capabilities in language understanding and generation, and can more comprehensively grasp contextual relationships and capture subtle semantic differences. This makes LLMs more advantageous in identifying key features, implicit logic, and potential contradictions in false information. However, there are still many challenges in directly applying LLMs to false information detection. The first is the phenomenon of hallucination

[19] , that is, LLMs generate content that seems reasonable but is inconsistent with the facts, which may lead to misjudgment; the second is that the potential bias in the training data may be amplified by the model, thereby affecting the objectivity and fairness of the detection. Finally, LLMs often lack flexibility when facing frequently updated rumor content and are unable to adapt to changes in emerging contexts and topics in a timely manner.

[0006] LLM's superior capabilities in feature extraction and semantic understanding can make up for CL's limitations in semantic reasoning and other aspects. It can be seen that CL has a natural dependence on LLM. Specifically, CL relies on its high-quality feature representation learning ability as the main detection mechanism; LLM's powerful data augmentation ability can provide CL with diverse but semantically consistent data, thereby enhancing CL's accuracy in identifying and classifying false information.

[0007] Despite the great potential, the synergy between LLM and CL still has some problems to be solved, including

[0008] 1. How to use LLM to break through the data limitations of the CL framework. Most of the existing rumor detection datasets

[20] come from a single social platform, which results in insufficient generalization of the model when facing multiple sources. Under the CL framework, traditional data augmentation methods are greatly limited. Although the synonym replacement and sentence rearrangement proposed in the literature

[18] can increase the amount of data, it is difficult to accurately retain the subtle semantic features in the rumor text, such as irony, metaphor or suggestion, and may even generate samples that are contrary to the original meaning. In addition, rumor propagation often depends on specific social backgrounds and event contexts. Traditional text augmentation methods cannot effectively simulate these complex contexts and cannot reproduce the subtle differences in group behavior patterns. Feng et al

[21] proposed a rumor detection framework called BiMGCL, which simulates the structural characteristics of rumor propagation and diffusion through bidirectional multi-level graph contrastive learning, and adopts three interpretable bidirectional graph data augmentation strategies as well as node-level and graph-level contrastive learning to improve the performance of rumor detection. Although the BiMGCL framework is good at learning the graph structure and propagation mode of rumors, its ability to capture and integrate fine-grained semantic features is limited when dealing with rumor content with complex contexts and multiple meanings. In particular, rumors in different cultural backgrounds often have unique expressions and propagation characteristics, and existing enhancement techniques are difficult to preserve these cultural and language specificities, limiting the cross-cultural application capabilities of the model. These problems make it difficult for the CL framework to expand the knowledge boundaries and adaptability of the model through traditional enhancement methods under single-source data.

[0009] 2. How to deeply mine deep representations with few samples. In the task of social media rumor detection, the effective mining of deep semantic representations faces significant challenges. The existing public datasets

[20] have limited sample sizes. This data scarcity seriously restricts the effective extraction of deep features by contrastive learning frameworks. Although traditional data augmentation methods

[15]

[16]

[17]

[18] can alleviate the problem of insufficient samples to a certain extent, these methods can only achieve shallow text transformation and are difficult to produce substantial enhancement at the semantic level. This results in the contrastive learning framework being unable to capture the deep semantic features in rumor texts, such as key information such as discourse logic, argument structure, and emotional tendency, when constructing positive and negative sample pairs. More importantly, rumor texts often contain complex contextual dependencies and implicit semantic connections, which require deep semantic understanding and representation learning to be effectively identified. Liu et al

[22] proposed a fake news detection framework for DCEP, which integrates news content, emotional information, and communication structure, and uses graph neural networks and edge-aware methods to enhance news graph representation. The innovation of the DCEP framework lies in the extraction and fusion of multi-dimensional emotional features, as well as the consideration of the influence of emotional interaction networks in fake news detection. Although DCEP performs well in experiments, it is still a challenge to deeply mine deep semantic and emotional features in the case of few samples, which may limit the generalization ability of the model and the accuracy of early detection in practical applications. Therefore, how to effectively mine deep semantic features through more advanced enhancement strategies and optimized contrastive learning mechanisms under limited sample conditions has become a key issue to be solved.

[0010] 3. How to align the optimization goals of LLM and CL. DNN models including CL can be integrated through joint training or model concatenation

[23] , but these methods are not ideal in terms of aligning the optimization goals of LLM and CL. A direct idea is to use the language understanding ability of LLM to directly participate in detection

[24] , but since the requirements of LLM and CL are not fully aligned, such methods cannot fully utilize the powerful language understanding ability of LLM and the feature extraction potential of CL. Hu et al

[25] proposed a fake news detection method that provides multi-angle reasoning and guiding opinions. In the study of

[26] , LLM is used to extract keywords and evaluate the relationship weights between keywords through graph Laplacian learning to automatically construct a knowledge graph (KG). Although the above methods use LLM as a key tool for data enhancement and knowledge extraction, they lack direct linkage and feedback mechanisms between classifiers. Therefore, how to design a method that can achieve LLM requirement alignment to effectively integrate the advantages of LLM and CL needs further exploration. Summary of the invention

[0011] To address the above challenges, a large language model (LLM)-assisted contrastive learning (CL) detection framework is proposed to improve the stability and accuracy of false information detection in social networks. In this framework, the powerful language understanding ability of LLM is used to make up for the shortcomings of CL in feature extraction, while the discriminative feature learning of the CL model effectively alleviates the hallucination output of LLM.

[0012] The present invention discloses a method for detecting false information on social networks based on LLM enhanced contrastive learning. The method detects false information on social networks under an LLM-assisted CL framework. The LLM-assisted CL framework includes a data preprocessing module, a Prompt arranger, a CL classification network, and an LLM optimizer.

[0013] The steps of the social network false information detection method include:

[0014] S1. Data preprocessing and Prompt orchestration: The raw data is captured from social networks and formatted. The processed data is input into the Prompt orchestrator to generate enhanced data to expand the training samples of the CL model.

[0015] S2, LLM-assisted feature extraction and label prediction: The data enhanced by S1 is paired with the original data and input into the CL classification network model; the CL model extracts the discriminative features of false information and predicts labels; after training, the parameters of the CL model are frozen; in the testing phase, the data is input into the CL model to extract features and predict labels through the classifier;

[0016] S3. Joint fine-tuning: The contrast loss of the CL model is used to guide the LLM fine-tuning; the fine-tuned LLM optimizes data augmentation and strengthens the feature extraction and classification of the CL model.

[0017] The present invention mainly includes the following three contributions:

[0018] 1. In order to overcome data limitations, a social network data augmentation method based on LLM was developed. This method focuses on generating diverse false information samples, maintaining semantic consistency, and at the same time expands the training dimension of the CL model to better capture the linguistic characteristics of false information.

[0019] 2. To achieve deep representation mining, a LLM-assisted CL feature extraction method is designed. LLM, as a data augmentation tool, replaces the traditional CL enhancement method and uses CL's discriminative feature learning ability to constrain LLM's hallucination output, thereby improving the robustness and accuracy of rumor detection. LLM's semantic understanding and generation capabilities further enhance the feature extraction effect of CL and significantly improve detection performance.

[0020] 3. To align the optimization goals of LLM and CL, a sustainable joint fine-tuning strategy is developed. This strategy guides LLM fine-tuning with the contrast loss of the CL model. The fine-tuned LLM optimizes data augmentation and strengthens the feature extraction and classification capabilities of CL. This closed-loop optimization strategy overcomes the hallucinations of LLM and the limitations of CL in processing complex semantics, improving the accuracy of false information identification, the interpretability and adaptability of the model.

[0021] Experimental results on multiple mainstream datasets verify the superiority and practicality of the proposed method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 represents the LLM-assisted language detection architecture;

[0023] Figure 2 Indicates data preprocessing;

[0024] Figure 3 Prompt indicating the design;

[0025] Figure 4 Represents a specific example of data augmentation;

[0026] Figure 5 The performance comparison chart of BiGCN, BiMGCL and this method under Twitter15 is shown;

[0027] Figure 6 The performance comparison chart of BiGCN, BiMGCL and this method under Twitter16 is shown;

[0028] Figure 7 Feature graphs showing different rounds of enhancements for Twitter15 (from left to right, 7b-1aug, 7b-2ug, and 7b-3ug);

[0029] Figure 8 Feature maps showing different rounds of enhancement under Twitter16 (from left to right, 7b-1aug, 7b-2ug, and 7b-3ug);

[0030] Fig. 9 Feature maps showing different rounds of fine-tuning for Twitter15 (from left to right, 7b-3aug-0ft, 7b-3aug-1ft, and 7b-3aug-2ft);

[0031] Fig.10 Feature maps representing different rounds of fine-tuning on Twitter15 (from left to right, 7b-3aug-0ft, 7b-3aug-1ft, and 7b-3aug-2ft). DETAILED DESCRIPTION

[0032] 1. Overview

[0033] With the rapid development of the Internet, false information on social networking platforms has become a serious social problem. Although deep neural networks have made significant progress in rumor detection, existing methods still face challenges in dealing with complex semantic features and emotional expressions. This paper proposes a new Large Language Model (LLM)-assisted Contrastive Learning (CL) framework to improve the accuracy of false information detection on social networks. The framework mainly includes three core contributions:

[0034] Firstly, a social network data augmentation strategy based on LLM is designed to effectively expand the dimension of training data by generating false information samples with consistent semantics but diverse expressions.

[0035] Secondly, a LLM-assisted CL feature extraction method is developed to enhance the data using the semantic understanding ability of LLM, so that CL can mine deeper semantic features;

[0036] Finally, a novel joint fine-tuning strategy is proposed to guide the fine-tuning of LLM through the contrastive loss generated by CL, forming a virtuous cycle of data augmentation and model optimization.

[0037] Experimental results show that the proposed framework significantly improves the performance of false information detection while maintaining model stability, providing new technical support for information governance on social network platforms.

[0038] 2. Solution Design

[0039] Figure 1 The proposed LLM-assisted social network fake content detection architecture is presented, which includes a data preprocessing module, a Prompt orchestrator, a CL classification network, and an LLM optimizer. Under this architecture, the collaborative optimization process includes the following three stages:

[0040] 1. Data preprocessing and Prompt orchestration: The raw data is captured from social networks locally through crawler technology and formatted. The processed data is input into the Prompt orchestrator to generate high-quality enhanced data to expand the training samples of the CL model.

[0041] 2. LLM-assisted feature extraction and label prediction: The LLM-enhanced data is paired with the original data and input into the CL classification network. The CL module is responsible for extracting discriminative features of false information and predicting labels. After training, the parameters of the CL model are frozen. In the testing phase, the data is input into the CL model to extract features and predict labels through the classifier.

[0042] 3. CL-assisted LLM continuous fine-tuning: The contrast loss generated during the CL model training process is used to guide the LLM optimizer to fine-tune the output direction of the LLM to ensure that the generated enhanced data better meets the discrimination requirements of CL, thereby achieving collaborative optimization of LLM and CL in detecting false information.

[0043] 2.1 Data preprocessing and prompt arrangement

[0044] The social network dataset is represented as where x i Represents the i-th social network data instance. LLM is responsible for processing Each piece of data in x. Through the preset prompt, LLM generates i Enhanced data with the same semantics but different expressions Figure 2 The structure of each piece of data is shown, which usually includes the original post and its related interactive replies. The preset prompts are used to guide and constrain LLM to ensure that it can identify and analyze the typical characteristics of false information on social networks, including logical incoherence, factual inaccuracies, strong emotional colors, and significant position tendencies. Through this guidance method, LLM can deeply capture subtle clues in the original data that may indicate that the content has been tampered with, thereby accurately identifying potential false information. After completing the analysis, LLM will reorganize and diversify the original information to generate semantically consistent and diverse enhanced data.

[0045] Prompt arrangement needs to meet the following goals:

[0046] 1. Full-view enhancement: Prompt should guide LLM to start from a global perspective rather than just enhancing local content. Through holistic analysis, LLM can better capture the characteristics and distribution of the dataset, thereby generating more realistic and rich enhanced data.

[0047] 2. Consistent output format: Prompt needs to constrain LLM to keep the output format consistent with the original data to ensure the structure and usability of the enhanced data.

[0048] 3. Diversity improvement: Prompt should encourage LLM to increase the diversity of generated content so that the enhanced data has more unique information and richness while maintaining authenticity.

[0049] 4. Semantic consistency: Prompt needs to ensure that the data generated by LLM is strictly semantically consistent with the original data. This constraint is crucial to maintaining the original meaning of the content and improving the model's ability to identify false information.

[0050] Following these principles, we have organized Figure 3 The Prompt design shown here has each feature corresponding to the above criteria to achieve high-quality data augmentation.

[0051] Figure 4 A specific example of data augmentation is shown. By comparing the original data and the augmented data, we can clearly observe the practical application of the aforementioned Prompt principle:

[0052] 1. Principle of overall enhancement: The overall enhancement is reflected in the examples through the overall reconstruction of the sentence structure rather than simple vocabulary replacement.

[0053] 2. Format consistency principle: The data before and after enhancement contain the same fields (such as uid, string_value, and replies), and each reply in replies maintains the original hierarchical structure. This format consistency ensures the structural compatibility of the enhanced data with the original data, which facilitates subsequent data processing and analysis.

[0054] 3. Diversity improvement principle: Improve data diversity through diversified language expressions while maintaining the original semantics. For example, rewriting "They love him" as "They're fully committed to him" increases the specificity and richness of the expression.

[0055] 4. Semantic preservation principle: Although the expression has changed, the core meaning is fully retained.

[0056] Through this example, we can intuitively see the effect of applying the Prompt design principle in the data enhancement process. This method not only improves the quality and diversity of the data, but also ensures that the enhanced data is consistent with the original data in structure and semantics.

[0057] 2.2 LLM-assisted feature extraction and label prediction

[0058] The CL network architecture of a single encoder is used to extract data features. The architecture consists of a feature extraction network and a projection head. The former is responsible for extracting high-dimensional feature representations from the input social media text. BERT (Bidirectional Encoder Representations from Transformers)

[27] was selected as the feature extractor for false information. The projection head is implemented by a multilayer perceptron

[32] (Multilayer Perceptron, MLP), and each layer of the perceptron contains a weight matrix W i and a nonlinear activation function σ. In this architecture, x i After the feature extraction network, the corresponding feature h is obtained i , and then through the projection head to get

[0059] z i =g(h i )=W2σ(W1h i ) (1)

[0060] In projected space, it is used to measure z i and z j The similarity function is expressed as

[0061]

[0062] The samples involved in the loss calculation include the original samples and their enhanced samples, the number of which is 2N. Applying formula (2) to InfoNCE Loss

[33] , the loss function of CL is described as

[0063]

[0064] Where τ is a temperature hyperparameter used to calculate the sensitivity of the similarity and loss function. Minimizing (3) forces samples of the same category to be tightly clustered in the projected space while maximizing the distance between samples of different categories. Due to its ability to identify subtle differences between rumors and true information, the accuracy and robustness of the CL model in rumor detection tasks are improved.

[0065] The CL classifier is described as f:G→Y(Y={y1,y2,..,y n}), where y∈{N,F,T,U}(N represents non

[0066] Rumors, F is false rumors, T is true rumors, and U is unconfirmed rumors). The classifier consists of a fully connected layer followed by a softmax activation function. In the test phase, the social network data is extracted by CL to obtain the d-dimensional features. is fed into the classifier. The weight matrix is the key factor in the classifier decision process, where k represents the number of categories. c , the d-dimensional features are mapped to the k-dimensional output space. To increase flexibility, the bias term The test data is processed by the linear transformation of the fully connected layer and the softmax activation function to complete the category classification.

[0067] In each batch, x1 and x1 T are paired as positive samples, and x2...x N and are regarded as negative samples. These positive and negative samples are input into the CL framework and processed by the feature extractor to generate the initial feature representation. Based on formula (1), these features are mapped to a high-dimensional feature space and the data distribution is optimized in this space. In this way, semantically similar data points are pulled closer to each other in the feature space, while semantically different data points are pushed away from each other. In order to accurately quantify the similarity of feature vectors, cosine similarity is adopted to ensure the accuracy and reliability of feature extraction.

[0068] As a feature extractor of false information, the CL model is frozen after training. During the testing phase, new data is input into the previously frozen CL model. The features of these data are extracted by the model and fed into the pre-trained classifier. Subsequently, the classifier maps these features to pre-defined rumor categories and finally outputs the predicted label to complete the classification of rumors.

[0069] 2.3 Alignment of CL network with LLM requirements

[0070] Fine-tuning of LLM is a key technology to improve the performance in the vertical field of rumor detection. The current mainstream fine-tuning methods include full parameter fine-tuning

[28] , prompt learning

[29] , parameter-efficient fine-tuning

[30] (Parameter-Efficient Fine-tuning, PEFT), etc. Among them, the PEFT method has attracted widespread attention due to its low computational cost and memory requirements. In the PEFT method, low-rank adaptation (LoRA)

[31] achieves efficient parameter updates by adding a low-rank decomposition matrix next to the original weight matrix. However, traditional fine-tuning methods often focus on the performance optimization of specific tasks and lack precise control over the quality of the content generated by the model.

[0071] Algorithm 1 shows the process of Adapter merging.

[0072]

[0073]

[0074] In this study, LLM is used to enhance rumor data rather than directly participate in the feature extraction and classification of rumor data. This design idea enables a fine-tuning strategy based on a dynamic dataset to be proposed: let the original LLM be θ0, and the LoRA method is used to fine-tune the LLM. At the beginning of the kth training cycle, Then, LLM enhances the data according to the preset prompts, thus generating a dataset in the form of dialogue. As the kth training cycle ends, is used as a fine-tuning dataset. and θk-1, the adapter θk obtained by fine-tuning the k-th training cycle is determined as:

[0075]

[0076] To improve the data enhancement capability of LLM, a loss-aware parameter merging method was proposed based on the literature

[35] and summarized as Algorithm 1. For the rumor detection task t, the task vector is defined as τt = θk - θk - 1. Each entry in τt (corresponding to a particular parameter in the LLM) can be thought of as an axis in the d-dimensional space, and the sign of the parameter indicates the direction along that axis (positive or negative). Therefore, a given task vector τt can be decomposed into the symbolic vector and size vector That is, τt=γt⊙ηt. Where ⊙ denotes the element-wise product. Formally, γt=sgn(τt). Where sgn(x)*|x|=x and returns +1, 0 or -1. The size vector ηt is defined as ηt=|τt|, The value of represents the required movement from initialization to the i-th dimension.

[0077] τt used for merging first needs to prune the redundant values ​​to obtain (See lines 2-3). In order to remove redundant parameters that have little impact on LLM performance, in τt, the magnitudes in the first k% will be retained, and the remaining redundant values ​​will be set to 0. Therefore, can be further decomposed into

[0078] Next, we create an aggregated elected sign vector γm for the merged LLM (see lines 4-5). This vector ensures the sign consistency of each parameter p across different models. When creating γm, the sign with the highest total magnitude is selected. For each parameter p∈{1,2,...,d}, we separate them according to their signs And take their sum to calculate the total mass in the positive and negative directions, and then distribute As a symbol of a greater total movement. is calculated as

[0079] Then, for each parameter in θ, the disjoint mean is computed by averaging the values ​​of the model parameters whose sign is the same as the one chosen for the summary. but Algorithm 2 shows the demand alignment process between CL network and LLM.

[0080]

[0081]

[0082] The requirement alignment of LLM and CL is summarized as Algorithm 2. In joint training, L k As a CL performance indicator, it not only quantifies the ability of the CL model in the false information feature learning task, but also indirectly reflects the effect of LLM data augmentation. Specifically, a smaller L k This not only means that data augmentation produces semantically rich and discriminative samples, but also shows that the CL model can effectively distinguish positive and negative samples; otherwise, it is poor. The core of this method is to, in a predetermined fine-tuning round (for example, the kth round), according to L k Dynamically adjust the adapter parameters of the fusion strategy. In order to avoid a single cycle of L k To produce excessive effects, the momentum update

[34] strategy is used to evaluate the enhancement effect of LLM (see lines 2-3), calculated as (5):

[0083]

[0084] Where μ is the influencing factor. When the value of m is small, the fine-tuned LLM parameters will change dramatically, thereby reducing the consistency of the LLM enhanced data. k Based on , the fine-tuning weights can be expressed as Finally, the combined LLM can be expressed as

[0085] θ=θ k-1 +λτ m (6)

[0086] In addition, the variable max_ft_round (initial value is set to 1) is used to control the maximum fine-tuning round of the model (see lines 8-12). When the current fine-tuning round exceeds max_ft_round, the algorithm will stop the fine-tuning process. This parameter value can be adjusted manually according to actual needs.

[0087] In this closed-loop framework, LLM's outstanding capabilities in feature extraction and semantic understanding can provide CL with diverse data, and CL's performance indicators can guide LLM fine-tuning. Together, they promote continuous optimization of the entire training process.

[0088] 3. Experimental Preparation

[0089] 3.1 Dataset selection:

[0090] This experiment uses social network rumor detection datasets Twitter15 and Twitter16 for performance evaluation. These two datasets are widely used in rumor detection research of deep learning models. Table 1 shows the category distribution and proportion of these two datasets. In terms of data division, 70% of the dataset is used for training, 20% for testing, and 10% for validation. To ensure the comprehensiveness of the data, crawler technology is used to collect original posts and all related replies from the corresponding social platforms to provide richer context information for the model.

[0091] TABLE 1Overview of Twitter15 and Twitter16

[0092]

[0093]

[0094] 3.2Baseline selection

[0095] In order to comprehensively and objectively evaluate the performance of the proposed scheme, typical GNN and CL algorithms are selected as benchmarks:

[0096] BiGCN[1]: This method considers both the depth and breadth of rumor propagation, and uses a graph convolutional network (GCN) to capture the rumor propagation pattern and diffusion structure from top-down and bottom-up directions. Its advantage is that it can effectively combine the rumor propagation path and network structure characteristics to improve the accuracy and efficiency of rumor detection.

[0097] BiMGCL

[21] : This method uses a bidirectional graph structure to simulate the propagation and diffusion of rumor information and uses multi-level CL to capture different characteristics of rumors. Its advantage is that it reduces the dependence on a large amount of labeled data and improves the model's recognition and detection performance of rumor events by designing three graph structure enhancement strategies.

[0098] 3.3 Verification indicators

[0099] Two generally accepted evaluation criteria are used to quantitatively analyze and evaluate the performance of social network rumor detection methods:

[0100]

[0101]

[0102] TP, TN, FP and FN represent true positive, false positive and false negative respectively. Accuracy intuitively reflects the correct detection results, while precision and recall emphasize the proportion of correctly classified instances.

[0103] 3.4.LLM selection and implementation details

[0104] Here, three sets of experiments are designed to comprehensively evaluate the performance of the proposed framework:

[0105] 1. Comparison of baseline models: In the first set of experiments, BiGCN and BiMGCL are selected as baseline models to evaluate the overall performance of the proposed framework.

[0106] 2. Analysis of the impact of LLM parameter volume: The second group of experiments used Qwen13B as a control to analyze the impact of LLM with different parameter volumes on detection performance. We further explored the effect of different rounds of LLM data enhancement on the performance of false information detection.

[0107] 3. Analysis of fine-tuning effect: The third set of experiments focuses on the effect of fine-tuning LLM on data enhancement quality and its effect on detection capability improvement, in order to reveal the impact of fine-tuning strategies on model performance.

[0108] In the data enhancement performance boundary analysis, the Qwen

[36] series LLM was selected as the core component. We selected Qwen7B as the main experimental object and introduced Qwen13B for comparison to study the impact of LLMs with different parameter values ​​on the quality of data enhancement.

[0109] To ensure the fairness of the experiment, the key hyperparameters (learning rate, training cycle and batch size) were set uniformly in all experiments. All experiments were completed on a high-performance server with an Intel Core i9-14900K processor, 64GB DDR5 5200MHz memory, 4TB PCIe 4.0 solid-state drive, ASUS PRIME Z790-P WIFID5 motherboard, and two Gigabyte RTX 4090 24GB WindForce graphics cards.

[0110] 4. Results Analysis

[0111] 4.1 Performance benchmarking

[0112] Table2.Performance comparison between the proposed method and the baseline on Twitter15

[0113] (Performance comparison between the proposed method and the baseline under Twitter15)

[0114]

[0115]

[0116] Table3.Performance comparison between the proposed method and the baseline on Twitter16

[0117] (Performance comparison of the proposed method and baseline under Twitter16)

[0118]

[0119] The first set of experiments examines the accuracy and F1 value of the proposed method. In the experiment using Twitter15, the detection accuracy of Qwen7B as the base reaches 79.63%, which is 3.59% and 6.59% higher than BiGCN and BiMGCL respectively. For the experiment using Twitter16, the detection accuracy of Qwen7B as the base is 3.21% and 4.33% higher than BiGCN and BiMGCL respectively. Furthermore, when Qwen13B is used as the base, its accuracy is improved by 7.29% and 10.36% compared with BiGCN and BiMGCL respectively. In the experiment using Twitter16, the role of Qwen13B as the base is also significant, increasing the leading advantage of accuracy by 8.14% and 9.26% respectively. Both BiGCN and BiMGCL models are based on graph neural networks, which simulate the diverse scenarios that may be encountered during the spread of rumors by enhancing the complexity of the graph structure. However, these methods do not fully consider the emotions and opinions in the content published by social media users. In contrast, the proposed enhancement method focuses on analyzing the emotions and opinions in the user's content, hoping to improve the accuracy of rumor detection through a deeper understanding. Therefore, it leads the baseline model in terms of average F1 value.

[0120] Regarding the F1 score of the FR category, BiGCN outperforms the Qwen7B-assisted detection model by 0.72% in the experiment using Twitter15; using Twitter16, BiGCN's gain in the F1 indicator expands to 1.97%, while BiMGCL surpasses the proposed model by 4.06%. FR refers to rumors that are proven to be false or inaccurate. One possible reason is that LLM has inherent limitations in enhancing unlabeled data. Since LLM cannot independently judge the authenticity of the input information, it may inadvertently dilute the key features that make FR false information during the enhancement process, and may even "rationalize" some FR content to make it more deceptive. Therefore, it is reasonable to believe that this processing may cause the boundary between the enhanced FR samples and the real information to become blurred, while also changing the characteristics of different FR samples to a certain extent, reducing the consistency within the FR category, and thus affecting the performance of the detection model on the FR category.

[0121] Another thing worth noting is that LLM is usually trained to generate reasonable and coherent content, which may cause it to tend to make the information that was originally FR look more credible. These factors combined may cause the enhanced FR samples to lose some of the original obvious false information features, making the model perform worse than the baseline in identifying the FR category, although the overall performance is still excellent.

[0122] 4..2. The impact of LLM data enhancement rounds on detection

[0123] Table4.Results of different augmentation rounds on Twitter15

[0124] (Results of different enhancement rounds under Twitter15)

[0125]

[0126] Table5.Results of different augmentation rounds on Twitter16

[0127] (Results of different enhancement rounds under Twitter16)

[0128]

[0129] The number of rounds of augmentation affects the detection accuracy of LLM-assisted CL. This group of experiments examines the impact of the number of augmentation rounds on the detection performance of the model. As the number of rounds of data augmentation increases, the detection performance of the model shows a slow upward trend. In the Twitter15 experiment, the detection accuracy after three rounds of data augmentation is improved by 0.67% and 1.01% compared with one and two rounds of augmentation, respectively. In the Twitter16 experiment, the detection accuracy of three rounds of augmentation is improved by 2.47% and 3.7% compared with one and two rounds of augmentation, respectively, but neither exceeds the 84.56% accuracy of the model detection assisted by the 13B model. This can be clearly seen from the feature graph. The clustering effect of the four feature points becomes more obvious as the number of augmentation rounds increases. There are two main reasons for this phenomenon. On the one hand, as the number of augmentation rounds increases, the generated data samples become more diverse. This diversity can help the model see more variants during training and help the model learn more comprehensive features. On the other hand, larger models usually have better language understanding and more accurate context understanding capabilities, and can generate higher quality samples. These samples are more useful for model training and help improve the performance of the final model.

[0130] There are fluctuations in the detection results of specific categories. In the experiment using Twitter15, for the TR category, the single-round enhancement effect of the 7B model is better than the results of other enhancement rounds. From the feature graph, the TR feature points after one enhancement are more closely clustered than those after two rounds of enhancement. For the NR category, the 2-round enhancement effect of the 7B model is better than the 3-round enhancement, but still weaker than that of the 13B model. It can be seen from the feature graph that the NR feature points after three rounds of enhancement partially overlap with the UR feature points. A similar trend was also observed in the experiment using Twitter16. This may be due to two factors. First, the quality of the enhanced data generated by LLM may vary between different categories, making it easier for some categories to generate high-quality samples; second, the enhancement process may introduce new features that conflict with the features of the original data, thereby causing performance fluctuations. 4.3. Impact of LLM fine-tuning rounds on detection

[0131] Table6.Results of different rounds of fine-tuning on Twitter15

[0132] (Results of different fine-tuning rounds under Twitter15)

[0133]

[0134] Table7.Results of different rounds of fine-tuning on Twitter16

[0135] (Results of different fine-tuning rounds under Twitter16)

[0136]

[0137] This set of experiments investigates the effect of fine-tuning rounds on detection capabilities. As the number of fine-tuning rounds of Qwen7B increases, the LLM-assisted CL detection accuracy improves. However, this improvement does not exceed the detection accuracy under 13B. In particular, for the experiment using Twitter15, the F1 values ​​of the NR and FR categories show significant fluctuations. The F1 value of the NR category decreases by 7.27% after two fine-tunings compared to one fine-tuning, while the F1 value of the FR category increases by 7.57%. The F1 values ​​of other categories show an overall upward trend. This phenomenon can be explained by the feature visualization diagram: in the initial state, the NR category and the FR category show a high degree of overlap, resulting in a relatively low accuracy of the FR category. After the second fine-tuning, the FR category shows a stronger clustering effect, and its feature distribution is more compact. In contrast, the clustering effect of the NR category is weakened, and there is a large degree of overlap with the feature space of the UR category.

[0138] 5. Summary

[0139] This study proposes a CL framework based on LLM enhancement, aiming to improve the accuracy and efficiency of false information detection in social networks. The innovations are mainly reflected in three aspects: first, a social network data enhancement strategy based on LLM is designed, which effectively expands the dimension of training data by generating false information samples with consistent semantics but diverse expressions; second, a LLM-enhanced CL feature extraction method is developed, which uses the semantic understanding ability of LLM to enhance data, enabling CL to mine deeper semantic features; finally, a novel joint fine-tuning strategy is proposed, which guides the fine-tuning and merging of LLM through the contrastive loss generated by CL, forming a virtuous cycle of data enhancement and model optimization. Experimental results on multiple authoritative datasets show that the proposed framework improves the performance of false information detection while maintaining model stability, especially outperforming existing baseline models in core indicators such as accuracy and F1 value. In addition, with the increase of data enhancement rounds, the overall detection performance of the model shows a slow upward trend, and the increase of fine-tuning rounds can further improve the detection accuracy, but it still cannot surpass the performance of the LLM model with a larger number of parameters.

[0140] refer to

[0141] [1].Bian, Tian, ​​et al. "Rumor detection on social media with bi-directional graph convolutional networks." Proceedings of the AAAI conference on artificial intelligence.Vol.34.No.01.2020.

[0142] [2].Kotteti,Chandra Mouli Madhav,Xishuang Dong,and Lijun Qian."Ensembledeep learning on time-series representation of tweets for rumor detection in social media."Applied Sciences 10.21(2020):7541.

[0143] [3].Lim,Jit Yan,et al."SCL:Self-supervised contrastive learning forfew-shot image classification."Neural Networks 165(2023):19-30.

[0144] [4].Ning,Yujie,et al."Contrastive learning based on category matchingfor domain adaptation in hyperspectral image classification."IEEETransactions on Geoscience and Remote Sensing(2023).

[0145] [5].Xia,Qiming,et al."Coin:Contrastive instance feature mining foroutdoor 3d object detection with very limited annotations."Proceedings of theIEEE / CVF International Conference on Computer Vision.2023.

[0146] [6].Zheng,Peng,et al."Memory-aided contrastive consensus learning forco-salient object detection."Proceedings of the AAAI Conference on ArtificialIntelligence.Vol.37.No.3.2023.

[0147] [7].Xie,Jinheng,et al."C2am:Contrastive learning of class-agnosticactivation map for weakly supervised object localization and semanticsegmentation."Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition.2022.

[0148] [8].Mukhoti,Jishnu,et al."Open vocabulary semantic segmentation withpatch aligned contrastive learning."Proceedings of the IEEE / CVF Conference onComputer Vision and Pattern Recognition.2023.

[0149] [9].Liu,Yonghao,et al."Improved graph contrastive learning for shorttext classification."Proceedings of the AAAI Conference on ArtificialIntelligence.Vol.38.No.17.2024.

[0150]

[10] .Zhao,Shan,et al."MCL:Multi-granularity contrastive learningframework for Chinese NER."Proceedings of the AAAI Conference on ArtificialIntelligence.Vol.37.No.11.2023

[11] .Yin,Yongjing,et al."Multi-modal graphcontrastive encoding for neural machine translation."Artificial Intelligence323(2023):103986.

[0151]

[12] .Jiang,Yangqin,Chao Huang,and Lianghao Huang."Adaptive graphcontrastive learning for recommendation."Proceedings of the 29th ACM SIGKDDconference on knowledge discovery and data mining.2023.

[0152]

[13] .Liu,Nian,et al."Hierarchical contrastive learning enhancedheterogeneous graph neural network."IEEE Transactions on Knowledge and DataEngineering 35.10(2023):10884-10896.

[0153]

[14] .Giachanou,Anastasia,Paolo Rosso,and Fabio Crestani."The impactof emotional signalson credibility assessment."Journal of the Association forInformation Science andTechnology 72.9(2021):1117-1132.

[0154]

[15] .Rong,Yu,et al."Dropedge:Towards deep graph convolutionalnetworks on nodeclassification."arXiv preprint arXiv:1907.10903(2019).

[0155]

[16] .Chen,Deli,et al."Measuring and relieving the over-smoothingproblem for graph neuralnetworks from the topological view."Proceedings ofthe AAAI conference on artificial intelligence.Vol.34.No.04.2020.

[0156]

[17] .Wang,Yiwei,et al."Nodeaug:Semi-supervised node classificationwith dataaugmentation."Proceedings of the 26th ACM SIGKDD InternationalConference on KnowledgeDiscovery&Data Mining.2020.

[0157]

[18] .Wei,Jason,and Kai Zou."Eda:Easy data augmentation techniques forboosting performanceon text classification tasks."arXiv preprint arXiv:1901.11196(2019).

[0158]

[19] .Zhang,Yue,et al."Siren’s song in the AI ocean:A survey onhallucination in large languagemodels,2023."URL https: / / arxiv.org / abs / 2309.01219(2024).

[0159]

[20] .Ma,Jing,Wei Gao,and Kam-Fai Wong."Detect rumors in microblogposts usingpropagation structure via kernel learning."Association forComputational Linguistics,2017.Kaddour,Jean,et al."Challenges andapplications of large language models."arXiv preprintarXiv:2307.10169(2023).

[0160]

[21] .Feng,Weiwei,et al."BiMGCL:rumor detection via bi-directionalmulti-level graphcontrastive learning."PeerJ Computer Science 9(2023):e1659.

[0161]

[22] .Liu,Fei,Xinsheng Zhang,and Qi Liu."An Emotion-Aware Approach forFake NewsDetection."IEEE Transactions on Computational Social Systems(2023).

[0162]

[23] .Saxena,Divya,and Jiannong Cao."Generative adversarial networks(GANs)challenges,solutions,and future directions."ACM Computing Surveys(CSUR)54.3(2021):1-42.

[0163]

[24] .Liu,Qiang,et al."Can Large Language Models Detect Rumors onSocial Media?."arXivpreprint arXiv:2402.03916(2024).

[0164]

[25] .Hu,Beizhe,et al."Bad actor,good advisor:Exploring the role oflarge language models infake news detection."Proceedings of the AAAIConference on Artificial Intelligence.Vol.38.No.20.2024.

[0165]

[26] .Chen,Bohan,and Andrea L.Bertozzi."AutoKG:Efficient automatedknowledge graphgeneration for language models."2023 IEEE InternationalConference on Big Data(BigData).IEEE,2023.

[0166]

[27] .Koroteev,Mikhail V."BERT:a review of applications in naturallanguage processing andunderstanding."arXiv preprint arXiv:2103.11943(2021).

[0167]

[28] .Lv,Kai,et al."Full parameter fine-tuning for large languagemodels with limitedresources."arXiv preprint arXiv:2306.09782(2023).

[0168]

[29] .Gu,Yuxian,et al."Ppt:Pre-trained prompt tuning for few-shotlearning."arXiv preprintarXiv:2109.04332(2021).

[0169]

[30] .Ding,Ning,et al."Parameter-efficient fine-tuning of large-scalepre-trained languagemodels."Nature Machine Intelligence 5.3(2023):220-235.

[0170]

[31] .Hu,Edward J.,et al."Lora:Low-rank adaptation of large languagemodels."arXiv preprintarXiv:2106.09685(2021).

[0171]

[32] .Popescu,Marius-Constantin,et al."Multilayer perceptron andneural networks."WSEASTransactions on Circuits and Systems 8.7(2009):579-588.

[0172]

[33] .Hadsell,Raia,Sumit Chopra,and Yann LeCun."Dimensionalityreduction by learning aninvariant mapping."2006 IEEE computer societyconference on computer vision and patternrecognition(CVPR'06).Vol.2.IEEE,2006.

[0173]

[34] .He,Kaiming,et al."Momentum contrast for unsupervised visualrepresentationlearning."Proceedings of the IEEE / CVF conference on computervision and pattern recognition.2020.

[0174]

[35] .Yadav,Prateek,et al."Ties-merging:Resolving interference whenmergingmodels."Advances in Neural Information Processing Systems 36(2024).

[0175]

[36] .Bai,Jinze,et al."Qwen technical report."arXiv preprint arXiv:2309.16609(2023).

Claims

1. A method for detecting false information in social networks based on LLM enhanced contrastive learning, characterized by This method detects false information on social networks under the LLM-assisted CL framework, where CL stands for Contrastive Learning and LLM stands for Large Language Model. The LLM-assisted CL framework includes a data preprocessing module, a Prompt orchestrator, a CL classification network, and an LLM optimizer; The steps of the social network false information detection method include: S1. Data preprocessing and Prompt orchestration: The raw data is captured from social networks and formatted. The processed data is input into the Prompt orchestrator to generate enhanced data to expand the training samples of the CL model. S2, LLM-assisted feature extraction and label prediction: The data enhanced by S1 is paired with the original data and input into the CL classification network model; the CL model extracts the discriminative features of false information and predicts labels; after training, the parameters of the CL model are frozen; in the testing phase, the data is input into the CL model to extract features and predict labels through the classifier; S3, Joint fine-tuning: Use the contrast loss of the CL model to guide LLM fine-tuning; the fine-tuned LLM optimizes data augmentation and strengthens the feature extraction and classification of the CL model; In S1: The social network dataset is represented as where x i represents the i-th social network data instance; LLM Processing Each piece of data in; Through the preset prompt, LLM generates i Enhanced data with the same semantics but different expressions The preset prompt is used to guide and constrain LLM to ensure that LLM can identify and analyze the typical characteristics of false information on social networks. After the analysis is completed, LLM reorganizes and diversifies the original information to generate semantically consistent and diverse enhanced data. The constraints of prompt editing include: full-view enhancement, consistent output format, increased diversity, and consistent semantics; In S2: The CL network architecture of a single encoder is used to extract data features. The architecture consists of a feature extraction network and a projection head. The feature extraction network extracts high-dimensional feature representations from the input social media text. BERT is selected as the feature extractor of false information. The projection head is implemented by a multi-layer perceptron MLP, and each layer of the perceptron contains a weight matrix W. i and nonlinear activation function σ; In this architecture, x i After the feature extraction network, the corresponding feature h is obtained i , and then through the projection head to get z i =g(h i )=W2σ(W1h i ) (1) In projected space, it is used to measure z i and z j The similarity function is expressed as The samples involved in the loss calculation include the original samples and their enhanced samples, the number of which is 2N. Applying formula (2) to InfoNCELoss, the loss function of the CL model is described as Among them, τ is the temperature hyperparameter, which is used to calculate the sensitivity of the similarity and loss function; minimizing formula (3) encourages samples of the same category to be closely clustered in the projection space, while maximizing the distance between samples of different categories; The CL classifier is described as f:G→Y(Y={y1,y2,..,y n }), where y is the type of false information output; the classifier consists of a fully connected layer followed by a softmax activation function; In the testing phase, the d-dimensional features extracted from social network data by the CL model is fed into the classifier; the weight matrix It is the key factor in the classifier decision process, where k represents the number of categories; after W c , the d-dimensional features are mapped to the k-dimensional output space; at the same time, the bias term is introduced The test data is processed by the linear transformation of the fully connected layer and the softmax activation function to complete the category classification; In each batch, x1 and are paired as positive samples, and x2...x N and are regarded as negative samples; these positive and negative samples are input into the CL classification network and processed by the feature extractor to generate initial feature representations; The CL model is frozen after training. In the testing phase, new data is input into the previously frozen CL model, and the features of these data are extracted by the model and fed into the pre-trained classifier. Subsequently, the classifier maps these features to pre-defined false information categories and finally outputs the predicted label to complete the classification. In S3: A fine-tuning strategy based on a dynamic data set is adopted. The method is as follows: let the original LLM be θ0, and the LoRA method is used to fine-tune the LLM; at the beginning of the kth training cycle, is introduced into LLM; then, LLM enhances the data according to the preset prompt, thus generating a data set in the form of a dialogue As the kth training cycle ends, Used as a fine-tuning dataset; based on and θ k-1 , θ obtained by fine-tuning in the kth training cycle k Determined as: The loss-aware parameter merging method is as follows: for the false information detection task t, the task vector is defined as τ t =θ k -θ k-1 ; τ t Each entry in corresponds to a particular parameter in the LLM considered as an axis in the d-dimensional space, and the sign of the parameter indicates the positive or negative direction along that axis; Therefore, a given task vector τ t is decomposed into symbolic vectors and size vector That is, τ t =γ t ⊙η t ; where ⊙ denotes the element-wise product; formally, γ t =sgn(τ t ); where sgn(x)*|x=x and returns +1, 0 or -1; the size vector η t is defined as η t =τ t |, The value of represents the required movement from initialization to the i-th dimension; τ for merging t First, trim the redundant values ​​to get Further decomposed into Next, we create an aggregated elected symbol vector γ for the merged LLMs. m , which ensures the consistency of the signs of the parameters p between different models; when creating γ m When the total amplitude is the highest, the symbol is selected; for each parameter p∈{1,2,...,d}, the symbols are separated according to their signs And take their sum to calculate the total mass in the positive and negative directions, then distribute as a symbol of a greater total movement; is calculated as Then, for each parameter in θ, the disjoint means are computed by averaging the values ​​of the model parameters whose sign is the same as the one selected for the summary; 2. According to claim 1, the method for detecting false information in social networks based on LLM enhanced contrastive learning, Its characteristics are in S1: Each enhanced data The structure of a post includes the original post and its related interactive replies. Typical characteristics of false information on social networks include incoherent logic, inaccurate facts, strong emotional colors, and significant position tendencies. The constraints of prompt arrangement are: Full-view enhancement: Prompt guides LLM to start from a global perspective rather than just enhancing local content; Consistent output format: Prompt needs to constrain LLM to keep the output format consistent with the original data to ensure the structure and usability of the enhanced data; Diversity improvement: Prompt encourages LLM to increase the diversity of generated content, so that the enhanced data has more unique information and richness while maintaining authenticity; Semantic consistency: Prompt needs to ensure that the data generated by LLM is strictly semantically consistent with the original data; According to the constraint principle, the prompt is designed as follows: "You are given data from a rumor detection dataset, and your task is to enhance the data based on the following instructions: 1.**Holistic Enhancement**:Enhance the data by considering its overallcontent,not just focusing on isolated parts.Make-sure the enhancements reflect a comprehensive understanding of the entire data piece.2.**MaintainFormat Consistency**: -Ensure the enhanced data retains the exact same format as the originaldata.No structural changes should occur. 3.**Increase Diversity**:-Focus on making the data more diverse in terms of language expression, without altering the underlying meaning. The enhanced data should be more varied but semantically identical to the original. 4.**Preserve Semantics**.-It is critical that the enhanced data preserves the exact same meaning as the original, even though the expressions maydiffer." 3. According to the method for detecting false information in social networks based on LLM enhanced contrastive learning in claim 1, it is characterized in that in S2: y∈{N,F,T,U}, false information is divided into N for non-rumor, F for false rumor, T for true rumor, and U for unconfirmed rumor.

4. The method for detecting false information in social networks based on LLM enhanced contrastive learning according to claim 1, characterized in that in S3, in τ t In the above example, the magnitudes in the first k% will be retained and the remaining redundant values ​​will be set to 0.

Images