A method for detecting false information on the Internet based on a large language model

Abstract

The present invention provides a method for detecting false information on a network based on a large language model, which belongs to the technical field of network information authenticity assessment, and comprises: establishing a data set to be detected based on original network information; using a pre-trained large language model to analyze any original information on a network platform to obtain its scoring result; calculating the scoring difference of all its scoring results to obtain its scoring difference; when the scoring difference is greater than a threshold, re-analyzing to obtain a new scoring result and calculating its new scoring difference; using the large language model to analyze the comment information to obtain its emotional tendency result; obtaining a confidence score based on the scoring result, obtaining a comment feedback score based on the emotional tendency result, and obtaining a stability score based on the scoring results of the first scoring question and the second scoring question in any question type; judging whether the original text of the current network platform is false information based on the confidence score, the comment feedback score and the stability score.

Description

Technical Field

[0001] The present invention belongs to the technical field of network information authenticity evaluation, and relates to a method for detecting network false information based on a large language model. Background Art

[0002] In today's era of information explosion, social media platforms have become an important channel for people to obtain and share information. However, these platforms have also become a hotbed for the spread of false and untrue information. Due to the fast speed and wide range of information dissemination, false information can spread quickly and cause impact.

[0003] In traditional technologies, manual review, rule-based or simple machine learning models are usually used for processing, but they have the following shortcomings:

[0004] 1) Manual review: Although manual review can provide high-quality evaluation, it is inefficient and costly, and cannot cope with massive amounts of information flow.

[0005] 2) Rule-based methods: This type of method relies on preset rules or feature extraction, has poor adaptability to complex semantics and changing content forms, and is prone to misjudgment.

[0006] 3) Simple machine learning models: Although they can automatically process large amounts of data, they often lack the ability to understand in depth, especially when dealing with situations that contain implicit meaning or require background knowledge.

[0007] Based on the shortcomings of traditional technologies, in recent years, with the advancement of natural language processing technology, especially the development of large language models, new ideas have been provided to solve the above problems. After being trained on large-scale text data, large language models have powerful semantic understanding and generation capabilities, and can capture deeper language structures and contextual information. This makes them perform well in processing complex text tasks, including but not limited to sentiment analysis, topic classification, question-answering systems, etc. Summary of the invention

[0008] The present invention aims to solve the technical problem that it is difficult to quickly and accurately evaluate the authenticity of information on social media platforms; in the current network environment, false and untrue information spreads very quickly, which may not only lead to public misunderstandings, but also have a negative impact on social order and personal safety.

[0009] The present invention provides a method for detecting false information on the Internet based on a large language model, comprising the following steps:

[0010] Step 1: Establish a data set to be tested based on the original network information;

[0011] The data set to be tested includes original information of the network platform, forwarding information related to the original information of the network platform, and comment information related to the original information of the network platform; the original information of the network platform includes text content, user information of the user who posted the original information of the network platform, and the time when the user posted the original information of the network platform; forwarding information includes the user ID who posted the forwarding, the text content of the forwarding, and the time when the forwarding was posted; comment information includes the user ID who posted the comment, the text content of the comment, and the time when the comment was posted;

[0012] Step 2: Use the pre-trained large language model to analyze the original text information of the mth network platform and obtain its score result S ij ;

[0013] Step 3: Calculate the score difference of all score results of the i-th question type of the m-th network platform original information News_m to obtain the score difference of the i-th question type of the m-th network platform original information News_m;

[0014] Step 4: Compare the score difference of the i-th question type of the m-th network platform original information News_m with the threshold θ; if the score difference is greater than the threshold θ, return to step 2 to optimize the score problem of the corresponding question type, and then regenerate the score results for all score problems of the i-th question type of the m-th network platform original information News_m through the pre-trained large language model, and recalculate the score difference through step 3 until the final score difference is less than the threshold θ, and obtain the final score result S′ ij ;

[0015] Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment;

[0016] Step 6: Based on the scoring result S′ ij Get the confidence score, based on the sentiment tendency of all forwarded information and / or comment information of the mth network platform original information News_m, get the comment feedback score of the current network platform original text, and the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 Get the stability score of the original text of the current network platform;

[0017] The comprehensive accuracy of the original text on the current online platform is calculated based on the confidence score, comment feedback score and stability score of the original text on the current online platform to determine whether the original text on the current online platform is false information.

[0018] Further, the score result S of the original text of the mth network platform is obtainedij The specific process is as follows:

[0019] S2.1, suppose there are N questions for scoring the authenticity of the mth network platform original text, and there are M network platform original text information in the data set to be tested;

[0020] S2.2. Based on the question used to score the authenticity of the mth network platform original text, use the pre-trained large language model to give N corresponding replies to the mth network platform original text, and obtain the score result S of the current network platform original text between [1, 5] ij ;

[0021] Among them, 1 is very untrue; 5 is very true; m is the mth original information of the network platform in the data set to be tested, m = (1, 2, ..., M); n is the answer to the nth question in the mth original information of the network platform, n = (1, 2, ..., N).

[0022] Further, the score result S of the original text of the mth network platform is obtained ij The specific process is as follows:

[0023] Suppose the questions used to score the authenticity of the original text on the mth network platform include scoring question types and specific questions;

[0024] The mth original text information News_m of the network platform is obtained from the data set to be detected, and the mth original text information News_m of the network platform is input into the pre-trained large language model, and the pre-trained large language model captures the semantic information in the mth original text information News_m of the network platform to fully understand the mth original text information News_m of the network platform;

[0025] Based on the scoring question type and specific questions, a series of authenticity questions Q are asked to the large language model for the original text information News_m on the mth network platform ij The query obtains a scoring result S for the mth original information News_m on the network platform. ij ; Wherein, i is the i-th question type, i = (1, 2, ..., k), k is the total number of question types; j is the j-th scoring question in the i-th question type, j = (1, 2, ..., h), h is the total number of scoring questions.

[0026] Furthermore, the specific method for obtaining the score difference of the i-th question type of the m-th network platform original information News_m is:

[0027] Suppose the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is Si2 , then the score difference of the i-th question type is |S i1 -S i2 |;

[0028] When |S i1 -S i2 When |>θ, the two scoring problems of the i-th problem are optimized and then return to step 2 for re-scoring. If the score difference is still too large, the two problems of the same type with a large score difference are discarded.

[0029] Furthermore, the specific method for obtaining the sentiment tendency result of each comment is as follows:

[0030] S5.1. Obtain the mth network platform original text News_m and P forwarding information Forward_P and E comment information Comment_E corresponding to the network platform original text from the data set to be detected;

[0031] S5.2, inputting the mth network platform original information News_m and the pth forwarding information Forward_p and the eth comment information Comment_e corresponding to the network platform original information into the pre-trained large language model for analysis to identify the stance of the current forwarding information Forward_p and the current comment information Comment_e;

[0032] And, the sentiment analysis is performed on the current forwarding information Forward_p and / or the current comment information Comment_e through the pre-trained large language model to obtain the sentiment score S (C n ), and judge the sentiment tendency of the current forwarding information Forward_p and / or the current comment information Comment_e;

[0033] Suppose that the sentiment score S(C n ) is 0.5, and the judgment rules are as follows:

[0034]

[0035] Wherein, Ar is a binary variable, which represents the position in the current forwarding information Forward_p and / or the current comment information Comment_e;

[0036] S5.3. Repeat step S5.2 until the positions of all forwarded information and / or comment information of the mth original network platform information News_m are obtained.

[0037] Furthermore, the specific process of obtaining the stability score of the original text of the current network platform is as follows:

[0038] S6.1. Assume the scoring result S′ ij The probability values ​​are X1,X2,…,X l , where l = (1, 2, ..., L), L is the total number of probabilities of the score;

[0039] Based on the scoring result S′ ij The probability value is used to calculate the entropy value Entropy, and in order to ensure that the entropy value Entropy is normalized to the interval [0, 1], the normalized entropy value Normalized_Entropy is obtained;

[0040] The confidence score is adjusted according to the normalized entropy value Normalized_Entropy to obtain the adjusted confidence score Confidence_Score;

[0041] S6.2. Statistically calculate the positions of all forwarded information and / or comment information of the mth network platform original information News_m to obtain the overall attitude value, that is, obtain the comment feedback score User_Feedback_Score of the current network platform original information;

[0042] S6.3, the scoring result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 , calculate the average score

[0043] Based on the mean score Calculate the absolute difference ij ;

[0044] Based on absolute difference ij Sum the total absolute difference TDifference between the scores of all questions of all types and the average score;

[0045] Calculate the scoring stability score N based on the total absolute difference TDifference TDifference ;

[0046] The rating stability score is used to measure the rating consistency of the original text on the same network platform for the same type of questions. The higher the rating stability score, the better the rating consistency of the original text on the same network platform for the same type of questions. The following formula is used to express its rating consistency Internal_Consistency_Score:

[0047] Internal_Consistency_Score=1-N TDifference ;

[0048] S6.4. Use confidence score Confidence_Score, comment feedback score User_Feedback_Score and rating stability score N TDifference , calculate the comprehensive accuracy Accuracy_Score of the original text of the current network platform;

[0049] S6.5. Determine whether the original text on the current network platform is false information;

[0050] Based on the comprehensive accuracy, it is judged whether the original text on the current network platform is false information; the specific judgment process is as follows:

[0051] If the comprehensive accuracy Accuracy_Score < thresold, it is considered that the original text on the current network platform is false information;

[0052] If the comprehensive accuracy Accuracy_Score ≥ thresold, it is considered that the original text on the current network platform is not false information.

[0053] Furthermore, the specific process of obtaining the comment feedback score User_Feedback_Score of the original text on the current network platform is as follows:

[0054] Perform a statistical analysis on the sentiment of all forwarded information and / or comments on the original text News_m of the mth network platform, calculate the ratio of supporting and opposing comments, and then obtain an overall attitude value A total ; Overall attitude value A total It can be expressed as a value between 0 and 1, where 0 means complete opposition, which means that the original text on the current online platform is false information; 1 means complete support, which means that the original text on the current online platform is not false information.

[0055] Furthermore, the method for detecting false information on the Internet based on a large language model provided by the present invention further includes obtaining an optimized false information detection model by optimizing the process of step 1 to step 6, and using the optimized false information detection model to verify whether the original text on the current network platform is false information, which specifically includes:

[0056] Step 7: Evaluate the performance of the process from step 1 to step 6, determine the false information judgment threshold based on the evaluation results, and adjust the weight of calculating the comprehensive accuracy to obtain an optimized false information detection model;

[0057] Step 8: Use the optimized false information detection model to re-determine whether the original text on the current network platform is false information.

[0058] Furthermore, the specific process of optimizing the false information detection model is as follows:

[0059] S7.1. Perform an initial evaluation of the false information detection model. The specific process is as follows:

[0060] ①、Preparation of validation set:

[0061] Randomly extract a portion of the dataset to be tested as the validation set, ensuring that it covers original online platform texts of different topics, types, and time periods, and that the number of non-false information samples and false information samples in the validation set is relatively balanced;

[0062] ②. Set a preliminary threshold (0.5) at the beginning, and use the validation set to calculate the classification indicators of the model under the preliminary threshold. The classification indicators include accuracy, precision, recall and F1 score;

[0063] ③. For a series of possible thresholds, calculate the true positive rate and false positive rate at each threshold respectively; then, draw the ROC curve for each threshold, and find the best threshold that can maximize the true positive rate and minimize the false positive rate through the ROC curve;

[0064] ④. Calculate the area AUC value of the ROC curve;

[0065] ⑤. Analyze the AUC value and ROC curve, and select the threshold near the highest AUC value as Threshold;

[0066] S7.2. Use the Bayesian optimization method to iteratively optimize the false information detection model. The specific process is as follows:

[0067] ①. Set initial weight:

[0068] At the beginning, set the initial weights ω1 = 0.4, ω2 = 0.3, ω3 = 0.3 as the starting point of the search to ensure that the sum of the weights is equal to 1;

[0069] ②. Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library to create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set;

[0070] ③. Execute the optimization process: Start the Bayesian optimizer to intelligently select new weight combinations for evaluation based on previous test results; and in each iteration, record the current optimal weight combination and its corresponding classification indicator performance;

[0071] ④. Introduce K-fold cross validation: Introduce K-fold cross validation in each weight adjustment process to ensure that specific data segmentation will not be overfitted; for each candidate weight combination, perform multiple verifications through K-fold cross validation, calculate its average performance index and select the most stable weights ω1′, ω2′, ω3′.

[0072] Furthermore, the specific process of applying the optimized false information detection model to re-judge whether the original text on the current network platform is false information is as follows:

[0073] S8.1. extract the mth network platform original text from the data set to be tested, and perform data preprocessing operations such as text cleaning, word segmentation, and stop word removal on the data of the mth network platform original text to ensure the data quality input to the large language model, and obtain new network platform original text data News′;

[0074] S8.2, input the new network platform original data News′ into the large language model, and generate a new scoring result AnswerScore_n′ between 1 and 5;

[0075] Input the new network platform original data News′ and the new e-th comment information Comment_e′ into the large language model to obtain the new sentiment score P(ω)′ for each word in each comment;

[0076] The sentiment tendency A of a single comment is obtained based on the sentiment score calculation formula and threshold judgment. n ′;

[0077] S8.3, calculate the overall accuracy;

[0078] According to the new scoring result AnswerScore_n′ and the sentiment tendency A of each comment n ' Calculate the new confidence score Confidence_Score', the new comment feedback score User_Feedback_Score' and the new score stability score Internal_Consistency_Score' respectively;

[0079] Based on the new confidence score Confidence_Score′, the new comment feedback score User_Feedback_Score′ and the new score stability score Internal_Consistency_Score, a new comprehensive accuracy Accuracy_Score′ is obtained;

[0080] S8.4, determine the results of false information;

[0081] Based on the new comprehensive accuracy Accuracy_Score′, it is judged whether the original text on the current network platform is false information; the specific judgment process is as follows:

[0082] If the new comprehensive accuracy Accuracy_Score′<thresold, the original text on the current network platform is considered to be false information;

[0083] If the new comprehensive accuracy Accuracy_Score′≥thresold, it is considered that the original text on the current network platform is not false information.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] Multi-dimensional authenticity scoring: Through a series of preset question templates, the information is comprehensively evaluated from different angles (such as source reliability, authority confirmation, evidence support, etc.) to ensure the comprehensiveness and accuracy of the scoring.

[0086] Integration of review analysis and user feedback: Not only does it analyze the original information, but it also uses user reviews to judge the authenticity of the information, which enhances the objectivity and comprehensiveness of the evaluation.

[0087] Rating difference detection and optimization: Difference detection is performed on the ratings of questions of the same type. When the rating difference exceeds the threshold, the large language model is returned for re-scoring. The question template is adjusted based on the questions with low-quality ratings to further improve the consistency and accuracy of the ratings.

[0088] Comprehensive accuracy calculation: Combine the confidence score, review feedback score, and rating stability score to calculate the comprehensive accuracy of each information item and ultimately determine whether the information is false.

[0089] Through these technological innovations, the present invention can significantly improve the accuracy and reliability of information authenticity assessment while maintaining efficient processing speed, providing strong support for maintaining a healthy network environment.

[0090] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0092] Figure 1 It is a flow chart of a method for detecting false information on the Internet based on a large language model in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] In order to make the above-mentioned purposes, features and advantages of the present invention more clear and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be noted that the drawings of the present invention are all simplified and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the implementation of the present invention; the "number" mentioned in the present invention is not limited to the specific number in the examples in the accompanying drawings; the directions or positional relationships indicated by "front", "middle", "back", "left", "right", "up", "down", "top", "bottom", "middle", etc. mentioned in the present invention are based on the directions or positional relationships shown in the drawings of the present invention, and do not indicate or imply that the devices or components referred to must have a specific direction, nor can they be understood as limitations on the present invention.

[0094] Example:

[0095] See also Figure 1 As shown, a method for detecting false information on the Internet based on a large language model provided by the present invention comprises the following steps:

[0096] Step 1: Establish a data set to be tested based on the original network information.

[0097] The data set to be detected is preferably a network platform data set (CED_Dataset), and the data set to be detected includes original information of the network platform, forwarding information related to the original information of the network platform, and comment information related to the original information of the network platform; the original information of the network platform includes text content, user information of the user who posted the original information of the network platform, and the time when the user posted the original information of the network platform; the forwarding information includes the user ID who posted the forwarding, the text content of the forwarding, and the time when the forwarding was published; the comment information includes the user ID who posted the comment, the text content of the comment, and the time when the comment was published.

[0098] Step 2: Use the pre-trained large language model to analyze the mth network platform original information AnswerScore_m_n and obtain its score result S ij The specific method is:

[0099] S2.1, suppose there are N questions for scoring the authenticity of the original text on the mth online platform;

[0100] S2.2. Based on the question, a large language model is used to give N corresponding replies to any original information on the network platform, and a series of scoring results S between [1, 5] are obtained. ij ;

[0101] Among them, 1 is very untrue; 5 is very true; m is the mth original information of the network platform in the data set to be tested, m = (1, 2, ..., M); n is the answer to the nth question in the mth original information of the network platform, n = (1, 2, ..., N).

[0102] Furthermore, the pre-trained large language model is used to analyze the mth original information of the network platform and obtain its scoring result S ij The specific process is as follows:

[0103] ①, Suppose the scoring question types and specific questions are as shown in Table 1;

[0104] Table 1. Scoring question types and specific questions

[0105]

[0106]

[0107] ②, obtain the mth network platform original information News_m from the data set to be tested, and input the mth network platform original information News_m into the pre-trained large language model, and the pre-trained large language model captures the semantic information in the mth network platform original information News_m to fully understand the mth network platform original information News_m;

[0108] ③ Based on the scoring question types and specific questions shown in Table 1, a series of authenticity questions Q are asked to the large language model for the original text information News_m on the mth network platform. ij The query obtains a score result S for the mth original information News_m on the network platform ij ; Wherein, i is the i-th question type, i = (1, 2, ..., k), k is the total number of question types (in this embodiment, k = 2); j is the j-th scoring question in the i-th question type, j = (1, 2, ..., h), h is the total number of scoring questions (in this embodiment, h = 2).

[0109] Step 3: Calculate the score differences of all score results of the i-th question type of the m-th network platform original information News_m to obtain the score difference of the i-th question type of the m-th network platform original information News_m.

[0110] The specific method is:

[0111] Suppose the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 , then the score difference of the i-th question type is |Si1 -S i2 |;

[0112] When |S i1 -S i2 When |>θ, the two scoring problems of the i-th problem are optimized and then return to step 2 for re-scoring. If the score difference is still too large, the two problems of the same type with a large score difference are discarded.

[0113] Step 4: Compare the score difference of the i-th question type of the m-th network platform original information News_m with the threshold θ (in this embodiment, θ=2); if the score difference is greater than the threshold θ, return to step 2 to optimize the score problem of the corresponding question type, and then regenerate the score results for all the score problems of the i-th question type of the m-th network platform original information News_m through the pre-trained large language model, and recalculate the score difference through step 3. If the score difference obtained for the second time is less than the threshold θ, the optimized score result S′ is obtained. ij , otherwise discard two questions of the same type with a large difference in scores.

[0114] Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment.

[0115] The specific method is:

[0116] S5.1. Obtain the mth network platform original text News_m and P forwarding information Forward_P and E comment information Comment_E corresponding to the network platform original text from the data set to be detected.

[0117] S5.2, inputting the mth network platform original information News_m and the pth forwarding information Forward_p and the eth comment information Comment_e corresponding to the network platform original information into the pre-trained large language model for analysis to identify the position (support or opposition) of the current forwarding information Forward_p and the current comment information Comment_e;

[0118] And, the sentiment analysis is performed on the current forwarding information Forward_p and / or the current comment information Comment_e through the pre-trained large language model to obtain the sentiment score S (C n ), and judge the sentiment tendency (support or opposition) of the current forwarding information Forward_p and / or the current comment information Comment_e;

[0119] The sentiment score S(C n ) is as follows:

[0120]

[0121] Where p = (1, 2, ..., P), P is the total number of forwarded messages corresponding to the original text on the network platform; e = (1, 2, ..., E), E is the total number of comments corresponding to the original text on the network platform; C r is the rth word in the current forwarded information and / or comment information, r = (1, 2, ..., R), R is the total number of words in the current forwarded information and / or comment information; D(ω) is the sentiment polarity score of the rth word;

[0122] Suppose that the sentiment score S(C n ) is 0.5, and the judgment rules are as follows:

[0123]

[0124] Wherein, Ar is a binary variable, which represents the position in the current forwarding information Forward_p and / or the current comment information Comment_e.

[0125] S5.3. Repeat step S5.2 until the positions of all forwarded information and / or comment information of the mth original network platform information News_m are obtained.

[0126] Step 6: Based on the scoring result S′ ij Get the confidence score, based on the position of all forwarded information and / or comment information of the mth network platform original information News_m, get the comment feedback score of the current network platform original text, and the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 The stability score of the original text of the current network platform is obtained to measure the consistency of the scores of the original text of the same network platform on the same type of questions.

[0127] The specific process is as follows:

[0128] S6.1. Assume the scoring result S′ ij The probability values ​​are X1,X2,…,X l , where l = (1, 2, [, L), L is the total number of probabilities of the score;

[0129] Calculate the entropy value Entropy, which is used to measure the uncertainty of the output of the large language model. The higher the entropy value, the more uncertain the judgment of the text on the network platform; the lower the entropy value, the more certain the judgment of the text on the network platform.

[0130] The expression of entropy is as follows:

[0131]

[0132] To ensure that the entropy value Entropy is normalized to the interval [0, 1], the entropy value Entropy is divided by the maximum possible entropy log(5) (because there are 5 possible ratings) to obtain the normalized entropy value Normalized_Entropy;

[0133] The expression of the normalized entropy value Normalized_Entropy is as follows:

[0134]

[0135] The confidence score is adjusted according to the normalized entropy value Normalized_Entropy to obtain an adjusted confidence score Confidence_Score. Specifically, the confidence score is defined as 1 minus the normalized entropy value Normalized_Entropy, so that when the entropy value is high, the confidence score is low; and vice versa;

[0136] The expression of the adjusted confidence score Confidence_Score is as follows:

[0137] Confidence_Score=1-Normalized_Entropy;

[0138] S6.2. Statistical calculation is performed based on the positions of all forwarded information and / or comment information of the mth network platform original information News_m to obtain the overall attitude value (i.e., the comment feedback score User_Feedback_Score of the current network platform original information). The specific process is as follows:

[0139] Perform a statistical analysis on the sentiment of all forwarded information and / or comments on the original text News_m of the mth network platform, calculate the ratio of supporting and opposing comments, and then obtain an overall attitude value A total ; Overall attitude value A totalIt can be expressed as a value between 0 and 1, where 0 means total opposition, which means that the original text on the current network platform is false information; 1 means total support, which means that the original text on the current network platform is not false information (that is, the original text on the current network platform is true).

[0140] Overall attitude value A total The expression is as follows:

[0141]

[0142] Where V is the total number of reposts or comments; A v is the attitude value of the vth comment, ranging from 0 to 1.

[0143] S6.3, the scoring result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 , calculate the average score

[0144] Rating average The expression is as follows:

[0145]

[0146] Based on the mean score Calculate the absolute difference ij ;

[0147] Absolute Difference ij The expression is as follows:

[0148]

[0149] Based on absolute difference ij Sum the total absolute difference TDifference between the scores of all questions of all types and the average score;

[0150] The expression of the total absolute difference TDifference is as follows:

[0151]

[0152] Calculate the scoring stability score N based on the total absolute difference TDifference TDifference ;

[0153] Rating stability score N TDifference The expression is as follows:

[0154]

[0155] The rating stability score is used to measure the rating consistency of the original text on the same network platform for the same type of questions. The higher the rating stability score, the better the rating consistency of the original text on the same network platform for the same type of questions. The following formula is used to express its rating consistency Internal_Consistency_Score:

[0156] Internal_Consistency_Score=1-N TDifference .

[0157] S6.4. Use confidence score Confidence_Score, comment feedback score User_Feedback_Score and rating stability score N TDifference , calculate the comprehensive accuracy Accuracy_Score of the original text of the current network platform;

[0158] The expression of the comprehensive accuracy of the original text of the current network platform is as follows:

[0159]

[0160] Among them, ω1, ω2, and ω3 are all weight coefficients.

[0161] S6.5. Determine whether the original text on the current network platform is false information;

[0162] ③. Based on the comprehensive accuracy score, determine whether the original text on the current network platform is false information; the specific judgment process is as follows:

[0163] If the comprehensive accuracy Accuracy_Score < thresold, it is considered that the original text on the current network platform is false information;

[0164] If the comprehensive accuracy Accuracy_Score ≥ thresold, it is considered that the original text on the current network platform is not false information.

[0165] Step 7: Evaluate and iteratively optimize the performance of the large language model;

[0166] S7.1. Set and optimize thresholds to determine whether the original text on the current network platform is false information;

[0167] ① Conduct an initial evaluation of the false information detection model; the specific process is as follows:

[0168] Validation set preparation:

[0169] Randomly select a part from the dataset to be detected as the validation set, ensuring that it covers the original texts of different topics, different types, and different time periods on the network platform, and the number of non-false information samples and false information samples in the validation set is relatively balanced to improve the reliability of the evaluation results.

[0170] Model basic performance evaluation:

[0171] Set a preliminary threshold (0.5) at the beginning, and use the validation set to calculate the classification metrics of the false information detection model under the preliminary threshold. The classification metrics include accuracy, precision, recall, F1 score, etc., and use these classification metrics as the baseline of the false information detection model performance for subsequent comparison.

[0172] ②. Optimization selection: By drawing the ROC curve and calculating the AUC value, find the threshold range with better discrimination ability; the specific process is as follows:

[0173] Draw the ROC curve:

[0174] For a series of possible thresholds (for example, from 0 to 1, with an interval of 0.01), calculate the true positive rate (TPR) and false positive rate (FPR) under each threshold respectively;

[0175] Draw the ROC curve graph for each threshold; and find the best threshold that can maximize the true positive rate and minimize the false positive rate through the ROC curve graph;

[0176] Calculate the AUC value:

[0177] Calculate the area AUC value of the ROC curve graph. The AUC value reflects the average performance of the model within the entire threshold range. The closer the AUC value is to 1, the better the false information detection model.

[0178] ③. Obtain the final threshold:

[0179] Analyze the AUC value and the ROC curve graph, and select the threshold near the highest AUC value as Threshold,

[0180] If Accuracy_Score < Thresold, then consider this event as false information.

[0181] If Accuracy_Score ≥ Thresold, then consider this event as not false information.

[0182] S7.2. Evaluate performance and perform iterative optimization

[0183] Use the Bayesian optimization method to perform iterative optimization on the false information detection model. The specific process is as follows:

[0184] Set initial weights: At the beginning, set initial weights ω1=0.4, ω2=0.3, ω3=0.3 as the starting point of the search, and ensure that the sum of the weights is equal to 1.

[0185] Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library; create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set. This objective function will guide the Bayesian optimizer in parameter search.

[0186] Execute the optimization process: Start the Bayesian optimizer, which intelligently selects new weight combinations for evaluation based on previous test results. In each iteration, record the current optimal weight combination and its corresponding classification indicator performance.

[0187] Introducing K-fold cross validation: Introducing K-fold cross validation (5-fold) in each weight adjustment process ensures that the model does not overfit a specific data segmentation and improves the stability of the evaluation. For each candidate weight combination, multiple validations are performed through K-fold cross validation, and its average performance index is calculated to select the most stable weights ω1′, ω2′, ω3′.

[0188] Step 8: Use the optimized false information detection model to re-determine whether the original text on the current network platform is false information;

[0189] The specific process is as follows:

[0190] S8.1. extract the mth network platform original text from the data set to be tested, and perform data preprocessing operations such as text cleaning, word segmentation, and stop word removal on the data of the mth network platform original text to ensure the data quality input into the pre-trained large language model, and obtain new network platform original text data News′;

[0191] The expression of the new network platform original data News′ is as follows:

[0192] News′={text1,user,time};

[0193] Comment_e′={uid, text2, data};

[0194] Among them, text1 is the text content of the original text on the network platform, user is the user information who published the original text on the network platform, time is the time when the user published the original text on the network platform, uid is the user ID who published the forwarding and comment, text2 is the text content of the forwarding and comment (if some users do not add comment content when forwarding, the item has no content), data is the publishing time of the forwarding and comment, and Comment_e′ is the new e-th comment information.

[0195] S8.2, input the new network platform original data News′ into the pre-trained large language model, and generate a new scoring result AnswerScore_n′ between 1 and 5;

[0196] Input the new network platform original data News′ and the new e-th comment information Comment_e′ into the trained large language model to obtain the new sentiment score P(ω)′ for each word in each comment;

[0197] The sentiment tendency A′ of a single comment is obtained based on the sentiment score calculation formula and threshold judgment. n .

[0198] S8.3, calculate the overall accuracy;

[0199] According to the new scoring result AnswerScore_n′ and the sentiment tendency A of each comment n ' Calculate the new confidence score Confidence_Score', the new comment feedback score User_Feedback_Score' and the new score stability score Internal_Consistency_Score' respectively;

[0200] Based on the new confidence score Confidence_Score′, the new comment feedback score User_Feedback_Score′ and the new score stability score Internal_Consistency_Score′, the new comprehensive accuracy Accuracy_Score′ is obtained;

[0201] S8.4, determine the results of false information;

[0202] The new combined accuracy Accuracy_Score′ is obtained by each score.

[0203] Based on the new comprehensive accuracy Accuracy_Score′, it is judged whether the original text on the current network platform is false information; the specific judgment process is as follows:

[0204] If the new comprehensive accuracy Accuracy_Score′ < thresold, it is considered that the original text on the current network platform is false information;

[0205] If the new comprehensive accuracy Accuracy_Score′ >= thresold, it is considered that the original text on the current network platform is not false information.

[0206] S8.5. Compare the false information judgment result obtained in step six with the false information judgment result obtained in 8.4 to verify the accuracy of the false information judgment result obtained in step six.

[0207] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting false information on the Internet based on a large language model, characterized in that: The following steps are involved: Step 1: Establish a data set to be tested based on the original network information; The data set to be tested includes original information of the network platform, forwarding information related to the original information of the network platform, and comment information related to the original information of the network platform; the original information of the network platform includes text content, user information of the user who posted the original information of the network platform, and the time when the user posted the original information of the network platform; forwarding information includes the user ID who posted the forwarding, the text content of the forwarding, and the time when the forwarding was posted; comment information includes the user ID who posted the comment, the text content of the comment, and the time when the comment was posted; Step 2: Use the pre-trained large language model to analyze the original text information of the mth network platform and obtain its score result S ij ; Step 3: Calculate the score difference of all score results of the i-th question type of the m-th network platform original information News_m to obtain the score difference of the i-th question type of the m-th network platform original information News_m; Step 4: Compare the score difference of the i-th question type of the m-th network platform original information News_m with the threshold θ; if the score difference is greater than the threshold θ, return to step 2 to optimize the score problem of the corresponding question type, and then regenerate the score results for all the score problems of the i-th question type of the m-th network platform original information News_m through the pre-trained large language model, and recalculate the score difference through step 3 until the final score difference is less than the threshold θ, and obtain the final score result S i ' j ; Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment; Step 6: Based on the scoring result S i ' j Get the confidence score, based on the sentiment tendency of all forwarded information and / or comment information of the mth network platform original information News_m, get the comment feedback score of the current network platform original text, and the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 Get the stability score of the original text of the current network platform; The comprehensive accuracy of the original text on the current online platform is calculated based on the confidence score, comment feedback score and stability score of the original text on the current online platform to determine whether the original text on the current online platform is false information.

2. The method for detecting false information on the Internet based on a large language model according to claim 1, characterized in that: Get the score result S of the original text of the mth network platform ij The specific process is as follows: S2.1, suppose there are N questions for scoring the authenticity of the mth network platform original text, and there are M network platform original text information in the data set to be tested; S2.

2. Based on the question used to score the authenticity of the mth network platform original text, use the pre-trained large language model to give N corresponding replies to the mth network platform original text, and obtain the score result S of the current network platform original text between [1, 5] ij ; Among them, 1 is very untrue; 5 is very true; m is the mth original information of the network platform in the data set to be tested, m = (1, 2, ..., M); n is the answer to the nth question in the mth original information of the network platform, n = (1, 2, ..., N).

3. The method for detecting false information on the Internet based on a large language model according to claim 2, characterized in that: Get the score result S of the original text of the mth network platform ij The specific process is as follows: Suppose the questions used to score the authenticity of the original text on the mth network platform include scoring question types and specific questions; The mth original text information News_m of the network platform is obtained from the data set to be detected, and the mth original text information News_m of the network platform is input into the pre-trained large language model, and the pre-trained large language model captures the semantic information in the mth original text information News_m of the network platform to fully understand the mth original text information News_m of the network platform; Based on the scoring question type and specific questions, a series of authenticity questions Q are asked to the large language model for the original text information News_m on the mth network platform ij The query obtains a scoring result S for the mth original information News_m on the network platform. ij ; Wherein, i is the i-th question type, i = (1, 2, ..., k), k is the total number of question types; j is the j-th scoring question in the i-th question type, j = (1, 2, ..., h), h is the total number of scoring questions.

4. The method for detecting false information on the Internet based on a large language model according to claim 3, characterized in that: The specific method for obtaining the score difference of the i-th question type of the m-th network platform original information News_m is: Suppose the score result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 , then the score difference of the i-th question type is |S i1 -S i2 |; When |S i1 -S i2 When |>θ, the two scoring problems of the i-th problem are optimized and then return to step 2 for re-scoring. If the score difference is still too large, the two problems of the same type with a large score difference are discarded.

5. The method for detecting false information on the Internet based on a large language model according to claim 1, characterized in that: The specific method to obtain the sentiment tendency results of each comment is: S5.

1. Obtain the mth network platform original text News_m and P forwarding information Forward_P and E comment information Comment_E corresponding to the network platform original text from the data set to be detected; S5.2, inputting the mth network platform original information News_m and the pth forwarding information Forward_p and the eth comment information Comment_e corresponding to the network platform original information into the pre-trained large language model for analysis to identify the stance of the current forwarding information Forward_p and the current comment information Comment_e; And, the sentiment analysis is performed on the current forwarding information Forward_p and / or the current comment information Comment_e through the pre-trained large language model to obtain the sentiment score S (C n ), and judge the sentiment tendency of the current forwarding information Forward_p and / or the current comment information Comment_e; Suppose that the sentiment score S(C n ) is 0.5, and the judgment rules are as follows: Wherein, Ar is a binary variable, which represents the position in the current forwarding information Forward_p and / or the current comment information Comment_e; S5.

3. Repeat step S5.2 until the positions of all forwarded information and / or comment information of the mth original network platform information News_m are obtained.

6. The method for detecting false information on the Internet based on a large language model according to claim 5, characterized in that: The specific process of obtaining the stability score of the original text of the current network platform is as follows: S6.1, let the scoring result S i ' j The probability values ​​are X1,X2,…,X l , where l = (1, 2, ..., L), L is the total number of probabilities of the score; Based on the scoring results S i ' j The probability value is used to calculate the entropy value Entropy, and in order to ensure that the entropy value Entropy is normalized to the interval [0, 1], the normalized entropy value Normalized_Entropy is obtained; The confidence score is adjusted according to the normalized entropy value Normalized_Entropy to obtain the adjusted confidence score Confidence_Score; S6.

2. Statistically calculate the positions of all forwarded information and / or comment information of the mth network platform original information News_m to obtain the overall attitude value, that is, obtain the comment feedback score User_Feedback_Score of the current network platform original information; S6.3, the scoring result of the first scoring question in the i-th question type is S i1 , the scoring result of the second scoring question in the i-th question type is S i2 , calculate the average score Based on the mean score Calculate the absolute difference ij ; Based on absolute difference ij Sum the total absolute difference TDifference between the scores of all questions of all types and the average score; Calculate the scoring stability score N based on the total absolute difference TDifference TDifference ; The rating stability score is used to measure the rating consistency of the original text on the same network platform for the same type of questions. The higher the rating stability score, the better the rating consistency of the original text on the same network platform for the same type of questions. The following formula is used to express its rating consistency Internal_Consistency_Score: Internal_Consistency_Score=1-N TDifference ; S6.

4. Use confidence score Confidence_Score, comment feedback score User_Feedback_Score and rating stability score N TDifference , calculate the comprehensive accuracy Accuracy_Score of the original text of the current network platform; S6.

5. Determine whether the original text on the current network platform is false information; Based on the comprehensive accuracy, it is judged whether the original text on the current network platform is false information; the specific judgment process is as follows: If the comprehensive accuracy Accuracy_Score < thresold, it is considered that the original text on the current network platform is false information; If the comprehensive accuracy Accuracy_Score ≥ thresold, it is considered that the original text on the current network platform is not false information.

7. The method for detecting false information on the Internet based on a large language model according to claim 6, characterized in that: The specific process of obtaining the comment feedback score User_Feedback_Score of the original text on the current network platform is as follows: Perform a statistical analysis on the sentiment of all forwarded information and / or comments on the original text News_m of the mth network platform, calculate the ratio of supporting and opposing comments, and then obtain an overall attitude value A total ; Overall attitude value A total It can be expressed as a value between 0 and 1, where 0 means complete opposition, which means that the original text on the current online platform is false information; 1 means complete support, which means that the original text on the current online platform is not false information.

8. The method for detecting false information on the Internet based on a large language model according to any one of claims 1 to 7, characterized in that: The method further includes obtaining an optimized false information detection model by optimizing the process of step 1 to step 6, and using the optimized false information detection model to verify whether the original text on the current network platform is false information, which specifically includes: Step 7: Evaluate the performance of the process from step 1 to step 6, determine the false information judgment threshold based on the evaluation results, and adjust the weight of calculating the comprehensive accuracy to obtain an optimized false information detection model; Step 8: Use the optimized false information detection model to re-determine whether the original text on the current network platform is false information.

9. The method for detecting false information on the Internet based on a large language model according to claim 8, characterized in that: The specific process of optimizing the false information detection model is as follows: S7.

1. Perform an initial evaluation of the false information detection model. The specific process is as follows: ①、Preparation of validation set: Randomly extract a portion of the dataset to be tested as the validation set, ensuring that it covers original online platform texts of different topics, types, and time periods, and that the number of non-false information samples and false information samples in the validation set is relatively balanced; ②, set a preliminary threshold at the beginning, and use the validation set to calculate the classification indicators of the model under the preliminary threshold. The classification indicators include accuracy, precision, recall and F1 score; ③. For a series of possible thresholds, calculate the true positive rate and false positive rate at each threshold respectively; then, draw the ROC curve for each threshold, and find the best threshold that can maximize the true positive rate and minimize the false positive rate through the ROC curve; ④. Calculate the area AUC value of the ROC curve; ⑤. Analyze the AUC value and ROC curve, and select the threshold near the highest AUC value as Threshold; S7.

2. Use the Bayesian optimization method to iteratively optimize the false information detection model. The specific process is as follows: ①. Set initial weight: At the beginning, set the initial weights ω1 = 0.4, ω2 = 0.3, ω3 = 0.3 as the starting point of the search to ensure that the sum of the weights is equal to 1; ②. Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library to create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set; ③. Execute the optimization process: Start the Bayesian optimizer to intelligently select new weight combinations for evaluation based on previous test results; and in each iteration, record the current optimal weight combination and its corresponding classification indicator performance; ④. Introduce K-fold cross validation: Introduce K-fold cross validation in each weight adjustment process to ensure that specific data segmentation will not be overfitted; for each candidate weight combination, perform multiple verifications through K-fold cross validation, calculate its average performance index and select the most stable weights ω1′, ω2′, ω3′.

10. The method for detecting false information on the Internet based on a large language model according to claim 8, characterized in that: The specific process of applying the optimized false information detection model to re-judge whether the original text on the current network platform is false information is as follows: S8.

1. Extract the mth network platform original text from the data set to be tested, and perform data preprocessing operations such as text cleaning, word segmentation, and stop word removal on the data of the mth network platform original text to ensure the data quality input to the large language model, and obtain new network platform original text data News′; S8.2, input the new network platform original data News′ into the large language model, and generate a new scoring result AnswerScore_n′ between 1 and 5; Input the new network platform original data News′ and the new e-th comment information Comment_e′ into the large language model to obtain the new sentiment score P(ω)′ for each word in each comment; The sentiment tendency A′ of a single comment is obtained based on the sentiment score calculation formula and threshold judgment. n ; S8.3, calculate the overall accuracy; According to the new scoring result AnswerScore_n′ and the sentiment tendency A of each comment n ' Calculate the new confidence score Confidence_Score', the new comment feedback score User_Feedback_Score' and the new score stability score Internal_Consistency_Score' respectively; Based on the new confidence score Confidence_Score′, the new comment feedback score User_Feedback_Score′ and the new score stability score Internal_Consistency_Score, a new comprehensive accuracy Accuracy_Score′ is obtained; S8.4, determine the results of false information; Based on the new comprehensive accuracy Accuracy_Score′, it is judged whether the original text on the current network platform is false information; the specific judgment process is as follows: If the new comprehensive accuracy Accuracy_Score′<thresold, the original text on the current network platform is considered to be false information; If the new comprehensive accuracy Accuracy_Score′≥thresold, it is considered that the original text on the current network platform is not false information.

Images