Mesh unreal information detection method based on large language model

Abstract

The invention provides a grid unreal information detection method based on a large language model, and belongs to the technical field of network information authenticity evaluation, and the method comprises the steps: building a to-be-detected data set based on network original text information; analyzing the original text information of any grid platform by using a pre-trained large language model to obtain a scoring result of the original text information; carrying out score difference calculation on all score results to obtain a score difference value; when the score difference value is greater than the threshold value, re-analyzing to obtain a new score result and calculating a new score difference value; analyzing the comment information by utilizing large language model analysis to obtain an emotional tendency result of the comment information; a confidence score is obtained based on the scoring result, a comment feedback score is obtained based on the emotional tendency result, and a stability score is obtained based on the scoring result of the first scoring question and the scoring result of the second scoring question in any question type; and based on the confidence score, the comment feedback score and the stability score, judging whether the current grid platform original text is unreal information or not.

Description

Technical Field

[0001] The invention belongs to the technical field of network information authenticity evaluation and relates to a grid false information detection method based on a large language model. Background Art

[0002] In today's era of information explosion, social media platforms have become an important channel for people to obtain and share information. However, these platforms have also become a hotbed for the spread of false and untrue information. Due to the fast speed and wide range of information dissemination, false information can spread quickly and cause impact.

[0003] In traditional technologies, manual review, rule-based or simple machine learning models are usually used for processing, but they have the following shortcomings: 1) Manual review: Although manual review can provide high-quality evaluation, it is inefficient and costly, and cannot cope with massive amounts of information flow.

[0004] 2) Rule-based methods: This type of method relies on preset rules or feature extraction, has poor adaptability to complex semantics and changing content forms, and is prone to misjudgment.

[0005] 3) Simple machine learning models: Although they can automatically process large amounts of data, they often lack the ability to understand in depth, especially when dealing with situations that contain implicit meaning or require background knowledge.

[0006] Based on the shortcomings of traditional technologies, in recent years, with the advancement of natural language processing technology, especially the development of large language models, new ideas have been provided to solve the above problems. After being trained on large-scale text data, large language models have powerful semantic understanding and generation capabilities, and can capture deeper language structures and contextual information. This makes them perform well in processing complex text tasks, including but not limited to sentiment analysis, topic classification, question-answering systems, etc. Summary of the invention

[0007] The present invention aims to solve the technical problem that it is difficult to quickly and accurately evaluate the authenticity of information on social media platforms; in the current network environment, false and untrue information spreads very quickly, which may not only lead to public misunderstandings, but also have a negative impact on social order and personal safety.

[0008] The present invention provides a grid false information detection method based on a large language model, comprising the following steps: Step 1: Establish a data set to be tested based on the original network information; The data set to be tested includes the original information of the grid platform, the forwarding information related to the original information of the grid platform, and the comment information related to the original information of the grid platform; the original information of the grid platform includes the text content, the user information of the user who posted the original information of the grid platform, and the time when the user posted the original information of the grid platform; the forwarding information includes the user ID who posted the forwarding, the text content of the forwarding, and the time when the forwarding was posted; the comment information includes the user ID who posted the comment, the text content of the comment, and the time when the comment was posted; Step 2: Use the pre-trained large language model to Analyze the original information of the grid platform and get its scoring results ; Step 3: Grid platform original information No. Calculate the score difference of all the scoring results of the question type and get the Grid platform original information No. The difference in scores between question types; Step 4: Grid platform original information No. Score difference and threshold for question types Compare; if the score difference is greater than the threshold , then return to step 2 to optimize the scoring problem of the corresponding question type, and then use the pre-trained large language model to optimize the scoring problem of the first Grid platform original information No. Regenerate the scoring results for all scoring questions of the question type, and recalculate their score differences through step 3 until the final score difference is less than the threshold , and get the final scoring result ; Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment; Step 6: Based on the scoring results Get the confidence score based on the Grid platform original information The sentiment tendency of all forwarded information and / or comment information is obtained to obtain the comment feedback score of the original text on the current grid platform. The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is Get the stability score of the original text of the current grid platform; The comprehensive accuracy of the original text of the current grid platform is calculated based on the confidence score, comment feedback score and stability score of the original text of the current grid platform to determine whether the original text of the current grid platform is false information.

[0009] Further, we get Scoring results of the original text on the grid platform The specific process is as follows: S2.1, set for There are some issues in scoring the authenticity of the original text on the grid platform. There are Grid platform original information; S2.2, based on the The problem of scoring the authenticity of the original text on the grid platform uses a pre-trained large language model to All grid platforms are original The corresponding reply is obtained from the current grid platform original text between Rating results between ; Among them, 1 is very untrue; 5 is very true; is the first Grid platform original information, ; For the The first one in the original information of the grid platform The answer to the question, .

[0010] Further, we get Scoring results of the original text on the grid platform The specific process is as follows: Designed for The questions for scoring the authenticity of the original text on the grid platform include scoring question types and specific questions; Get the first Grid platform original information , and the Grid platform original information Input into the pre-trained large language model, the pre-trained large language model is Grid platform original information The semantic information in Grid platform original information Gain a comprehensive understanding; Based on the scoring question type and the specific question, ask the large language model to Grid platform original information Ask a series of questions about authenticity The inquiry obtained Grid platform original information A rating result of ;in, For the Types of questions, , is the total number of question types; For the The first type of question Rating question, , is the total number of scoring questions.

[0011] Further, we get Grid platform original information No. The specific method for calculating the score difference of the question type is: Set up The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , then The difference in scores for the question types is ; when , then the After optimizing the two scoring problems of the same type, return to step 2 for re-scoring. If the score difference is still too large, discard the two questions of the same type with too large score difference.

[0012] Furthermore, the specific method for obtaining the sentiment tendency result of each comment is as follows: S5.1. Obtain the first Grid platform original information And the corresponding to the original text of the grid platform Forwarded messages and Comments ; S5.2. Grid platform original information And the first Forwarded messages and Comments All are input into the pre-trained large language model for analysis to identify the current forwarding information and current comment information position; And, through the pre-trained large language model, the current forwarding information and / or current comment information Perform sentiment analysis to obtain the current forwarding information and / or current comment information The sentiment score of each word in , and determine the current forwarding information and / or current comment information emotional inclination; Set to determine the current forwarding information and / or current comment information The sentiment score of each word in The threshold of the sentiment tendency is 0.5, and the judgment rules are as follows: ; in, A binary variable representing the current forwarding information and / or current comment information the position in S5.3, repeat step S5.2 until the Grid platform original information The position of all forwarded information and / or commented information.

[0013] Furthermore, the specific process of obtaining the stability score of the original text of the current grid platform is as follows: S6.1. Assume scoring result The probability value is ,in, , is the total number of probabilities scored; Based on the scoring results The probability value is used to calculate the entropy value , and in order to ensure the entropy value Normalized to In the interval, the normalized entropy value is obtained ; According to the normalized entropy value Adjust the confidence score to get the adjusted confidence score ; S6.2, based on Grid platform original information The positions of all forwarded information and / or comment information are statistically calculated to obtain the overall attitude value, that is, the comment feedback score of the original text on the current grid platform ; S6.3, based on The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , calculate the average score ; Based on the mean score Calculate the absolute difference ; Based on absolute difference Sum the total absolute difference between the ratings of all questions of all types and the average rating ; Based on total absolute difference Calculating the scoring stability score ; The scoring stability score is used to measure the scoring consistency of the original text on the same grid platform for the same type of questions. The higher the scoring stability score, the better the scoring consistency of the original text on the grid platform for the same type of questions. The following formula is used to express its scoring consistency : ; S6.4. Using confidence scores , Comment feedback score and scoring stability score , calculate the comprehensive accuracy of the original text of the current grid platform ; S6.5. Determine whether the original text of the current grid platform is false information; Based on the comprehensive accuracy, it is judged whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

[0014] Furthermore, the comment feedback score of the original text on the current grid platform is obtained. The specific process is as follows: For Grid platform original information Statistical analysis of the sentiment of all forwarded messages and / or comments is performed to calculate the ratio of supportive to opposing comments, and then an overall attitude value is obtained. ; Overall attitude value It can be expressed as a value between 0 and 1, where 0 means total opposition, which means that the original text of the current grid platform is false information; 1 means total support, which means that the original text of the current grid platform is not false information.

[0015] Furthermore, the grid false information detection method based on a large language model provided by the present invention also includes an optimized false information detection model obtained by optimizing the process of step one to step six, and the step of using the optimized false information detection model to verify whether the original text of the current grid platform is false information, which specifically includes: Step 7: Evaluate the performance of the process from step 1 to step 6, determine the false information judgment threshold based on the evaluation results, and adjust the weight of calculating the comprehensive accuracy to obtain an optimized false information detection model; Step 8: Apply the optimized false information detection model to re-determine whether the original text of the current grid platform is false information.

[0016] Furthermore, the specific process of optimizing the false information detection model is as follows: S7.1. Perform an initial evaluation of the false information detection model. The specific process is as follows: ①、Preparation of validation set: Randomly extract a portion of the dataset to be tested as the validation set, ensuring that it covers grid platform originals of different topics, types, and time periods, and that the number of non-false information samples and false information samples in the validation set is relatively balanced; ②. Set a preliminary threshold (0.5) at the beginning, and use the validation set to calculate the classification indicators of the model under the preliminary threshold. The classification indicators include accuracy, precision, recall and F1 score; ③. For a series of possible thresholds, calculate the true positive rate and false positive rate at each threshold respectively; then, draw the ROC curve for each threshold, and find the best threshold that can maximize the true positive rate and minimize the false positive rate through the ROC curve; ④. Calculate the area AUC value of the ROC curve; ⑤. Analyze the AUC value and ROC curve, and select the threshold near the highest AUC value as ; S7.2. Use the Bayesian optimization method to iteratively optimize the false information detection model. The specific process is as follows: ①. Set initial weight: At the beginning, set the initial weight As a starting point for the search, ensure that the sum of the weights is equal to 1; ②. Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library to create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set; ③. Execute the optimization process: Start the Bayesian optimizer to intelligently select new weight combinations for evaluation based on previous test results; and in each iteration, record the current optimal weight combination and its corresponding classification indicator performance; ④. Introduce K-fold cross validation: Introduce K-fold cross validation in each weight adjustment process to ensure that no overfitting of a specific data segmentation occurs; for each candidate weight combination, perform multiple validations through K-fold cross validation, calculate its average performance index, and select the most stable weight. .

[0017] Furthermore, the specific process of applying the optimized false information detection model to re-judge whether the original text of the current grid platform is false information is as follows: S8.1. Extract the first The original text of the grid platform and the The original data of the grid platform is preprocessed by text cleaning, word segmentation, and stop word removal to ensure the quality of data input into the large language model and obtain new grid platform original data. ; S8.2. New grid platform original data Input into the large language model and generate a new score between 1 and 5 ; The new grid platform original data and the new Comments Input them into the large language model to get the new sentiment score for each word in each comment ; The emotional tendency of a single comment is obtained based on the emotional score calculation formula and threshold judgment of the comment text. ; S8.3, calculate the overall accuracy; According to the new rating results and the sentiment of each comment Calculate the new confidence scores , New review feedback score and a new rating stability score ; Based on the new confidence score , New review feedback score and a new rating stability score , and obtain a new comprehensive accuracy ; S8.4, determine the results of false information; Based on the new comprehensive accuracy Determine whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the new comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the new comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

[0018] Compared with the prior art, the present invention has the following beneficial effects: Multi-dimensional authenticity scoring: Through a series of preset question templates, the information is comprehensively evaluated from different angles (such as source reliability, authority confirmation, evidence support, etc.) to ensure the comprehensiveness and accuracy of the scoring.

[0019] Integration of review analysis and user feedback: Not only does it analyze the original information, but it also uses user reviews to judge the authenticity of the information, which enhances the objectivity and comprehensiveness of the evaluation.

[0020] Rating difference detection and optimization: Difference detection is performed on the ratings of questions of the same type. When the rating difference exceeds the threshold, the large language model is returned for re-scoring. The question template is adjusted based on the questions with low-quality ratings to further improve the consistency and accuracy of the ratings.

[0021] Comprehensive accuracy calculation: Combine the confidence score, review feedback score, and rating stability score to calculate the comprehensive accuracy of each information item and ultimately determine whether the information is false.

[0022] Through these technological innovations, the present invention can significantly improve the accuracy and reliability of information authenticity assessment while maintaining efficient processing speed, providing strong support for maintaining a healthy network environment.

[0023] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 It is a flow chart of a grid false information detection method based on a large language model in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned purposes, features and advantages of the present invention more clear and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be noted that the drawings of the present invention are all simplified and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the implementation of the present invention; the "number" mentioned in the present invention is not limited to the specific number in the examples in the accompanying drawings; the directions or positional relationships indicated by "front", "middle", "back", "left", "right", "up", "down", "top", "bottom", "middle", etc. mentioned in the present invention are based on the directions or positional relationships shown in the drawings of the present invention, and do not indicate or imply that the devices or components referred to must have a specific direction, nor can they be understood as limitations on the present invention. Example

[0026] See also Figure 1 As shown, a grid false information detection method based on a large language model provided by the present invention comprises the following steps: Step 1: Establish a data set to be tested based on the original network information.

[0027] The data set to be detected is preferably a grid platform data set (CED_Dataset), and the data set to be detected includes grid platform original information, forwarding information related to the grid platform original information, and comment information related to the grid platform original information; the grid platform original information includes text content, user information that publishes the grid platform original text, and the time when the user publishes the grid platform original text; the forwarding information includes the user ID that publishes the forwarding, the text content of the forwarding, and the time when the forwarding is published; the comment information includes the user ID that publishes the comment, the text content of the comment, and the time when the comment is published.

[0028] Step 2: Use the pre-trained large language model to Grid platform original information Analyze and get the scoring results The specific method is: S2.1, set for There are some issues in scoring the authenticity of the original text on the grid platform. indivual; S2.2. Based on the question, use the large language model to analyze any original information on the grid platform. The corresponding answer is a series of Rating results between ; Among them, 1 is very untrue; 5 is very true; is the first Grid platform original information, ; For the The first one in the original information of the grid platform The answer to the question, .

[0029] Furthermore, we use the pre-trained large language model to Analyze the original information of the grid platform and get its scoring results The specific process is as follows: ①, Suppose the scoring question types and specific questions are as shown in Table 1; Table 1. Scoring question types and specific questions

[0030] ②、Get the first Grid platform original information , and the Grid platform original information Input into the pre-trained large language model, the pre-trained large language model is Grid platform original information The semantic information in Grid platform original information Gain a comprehensive understanding; ③ Based on the scoring question types and specific questions shown in Table 1, ask the large language model to Grid platform original information Ask a series of questions about authenticity The inquiry obtained Grid platform original information A rating result of ;in, For the Types of questions, , is the total type of question (in this embodiment, =2); For the The first type of question Rating question, , is the total number of scoring questions (in this embodiment, =2).

[0031] Step 3: Grid platform original information No. Calculate the score difference of all the scoring results of the question type and get the Grid platform original information No. The difference in scores for each question type.

[0032] The specific method is: Set up The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , then The difference in scores for the question types is ; when , then the After optimizing the two scoring problems of the same type, return to step 2 for re-scoring. If the score difference is still too large, discard the two questions of the same type with too large score difference.

[0033] Step 4: Grid platform original information No. Score difference and threshold for question types (In this embodiment, =2) for comparison; if the score difference is greater than the threshold , then return to step 2 to optimize the scoring problem of the corresponding question type, and then use the pre-trained large language model to optimize the scoring problem of the first Grid platform original information No. Regenerate the scoring results for all scoring questions of the question type, and recalculate the score difference through step 3. If the score difference obtained the second time is less than the threshold , and get the optimized scoring result , otherwise discard two questions of the same type with a large difference in scores.

[0034] Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment.

[0035] The specific method is: S5.1. Obtain the first Grid platform original information And the corresponding to the original text of the grid platform Forwarded messages and Comments .

[0036] S5.2. Grid platform original information And the first Forwarded messages and Comments All are input into the pre-trained large language model for analysis to identify the current forwarding information and current comment information stance (support or opposition); And, through the pre-trained large language model, the current forwarding information and / or current comment information Perform sentiment analysis to obtain the current forwarding information and / or current comment information The sentiment score of each word in , and determine the current forwarding information and / or current comment information emotional inclination (support or opposition); Current forwarding information and / or current comment information The sentiment score of each word in The expression is as follows: ; in, , The total number of forwarded messages corresponding to the original text of the grid platform; , The total number of comments corresponding to the original text on the grid platform; The first Vocabulary, , The total number of words in the current forwarded information and / or comment information; For the The sentiment polarity score of the word; Set to determine the current forwarding information and / or current comment information The sentiment score of each word in The threshold of the sentiment tendency is 0.5, and the judgment rules are as follows: ; in, A binary variable representing the current forwarding information and / or current comment information position in.

[0037] S5.3, repeat step S5.2 until the Grid platform original information The position of all forwarded information and / or commented information.

[0038] Step 6: Based on the scoring results Get the confidence score based on the Grid platform original information The position of all forwarded information and / or comment information is obtained to obtain the comment feedback score of the original text of the current grid platform, based on the first The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is The stability score of the original text of the current grid platform is obtained to measure the consistency of the scores of the original text of the same grid platform on the same type of questions.

[0039] The specific process is as follows: S6.1. Assume scoring result The probability value is ,in, , is the total number of probabilities scored; Calculating entropy , entropy value It is used to measure the uncertainty of the output of the large language model. The higher the entropy value, the more uncertain the judgment of the grid platform text; the lower the entropy value, the more certain the judgment of the grid platform text.

[0040] entropy The expression is as follows: ; To ensure the entropy Normalized to In the interval, the entropy value Divide by the maximum possible entropy (because there are 5 possible ratings), we get the normalized entropy value ; Normalized entropy value The expression is as follows: ; According to the normalized entropy value Adjust the confidence score to get the adjusted confidence score Specifically, the confidence score is defined as 1 minus the normalized entropy value , so that when the entropy value is high, the confidence score is low, and vice versa; Adjusted confidence score The expression is as follows: ; S6.2, based on Grid platform original information Calculate the positions of all forwarded information and / or comment information to obtain the overall attitude value (i.e., obtain the comment feedback score of the original text on the current grid platform The specific process is as follows: For Grid platform original information Statistical analysis of the sentiment of all forwarded messages and / or comments is performed to calculate the ratio of supportive to opposing comments, and then an overall attitude value is obtained. ; Overall attitude value It can be expressed as a value between 0 and 1, where 0 means complete opposition, which means that the original text of the current grid platform is false information; 1 means complete support, which means that the original text of the current grid platform is not false information (that is, the original text of the current grid platform is true).

[0041] Overall attitude score The expression is as follows: ; in, is the total number of reposts or comments; For the The attitude value of the comment, ranging from 0 to 1.

[0042] S6.3, based on The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , calculate the average score ; Rating average The expression is as follows: ; Based on the mean score Calculate the absolute difference ; Absolute Difference The expression is as follows: ; Based on absolute difference Sum the total absolute difference between the ratings of all questions of all types and the average rating ; Total absolute difference The expression is as follows: ; Based on total absolute difference Calculating the scoring stability score ; Rating stability score The expression is as follows: ; The scoring stability score is used to measure the scoring consistency of the original text on the same grid platform for the same type of questions. The higher the scoring stability score, the better the scoring consistency of the original text on the grid platform for the same type of questions. The following formula is used to express its scoring consistency : .

[0043] S6.4. Using confidence scores , Comment feedback score and scoring stability score , calculate the comprehensive accuracy of the original text of the current grid platform ; The overall accuracy of the original text of the current grid platform The expression is as follows: ; in, are all weight coefficients.

[0044] S6.5. Determine whether the original text of the current grid platform is false information; ③. Based on comprehensive accuracy Determine whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

[0045] Step 7: Evaluate and iteratively optimize the performance of the large language model; S7.1. Set and optimize thresholds to determine whether the original text on the current grid platform is false information; ① Conduct an initial evaluation of the false information detection model; the specific process is as follows: Validation set preparation: A part of the data set to be tested is randomly selected as the verification set to ensure that it covers the original texts of the grid platform of different topics, types and time periods, and the number of non-false information samples and false information samples in the verification set is relatively balanced to improve the reliability of the evaluation results.

[0046] Model basic performance evaluation: At the beginning, a preliminary threshold (0.5) is set, and the validation set is used to calculate the classification indicators of the false information detection model under the preliminary threshold. The classification indicators include accuracy, precision, recall and F1 score, etc. These classification indicators are used as the baseline of the performance of the false information detection model for subsequent comparison.

[0047] ② Optimization selection: By drawing the ROC curve and calculating the AUC value, find the threshold range with better discrimination ability; the specific process is as follows: Plotting the ROC curve: For a range of possible thresholds (e.g., from 0 to 1, with an interval of 0.01), calculate the true positive rate (TPR) and false positive rate (FPR) at each threshold; Draw the ROC curve for each threshold; and find the best threshold that maximizes the true positive rate and minimizes the false positive rate through the ROC curve; Calculate the AUC value: Calculate the area under the ROC curve. The AUC value reflects the average performance of the model over the entire threshold range. The closer the AUC value is to 1, the better the false information detection model is.

[0048] ③. Get the final threshold: Analyze the AUC value and ROC curve, and select the threshold near the highest AUC value as , if , then the event is considered to be false information.

[0049] if , then it is considered that the event is not false information.

[0050] S7.2. Evaluate performance and perform iterative optimization The Bayesian optimization method is used to iteratively optimize the false information detection model. The specific process is as follows: Set initial weights: At the beginning, set the initial weights , as the starting point of the search, ensuring that the sum of the weights is equal to 1.

[0051] Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library; create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set. This objective function will guide the Bayesian optimizer in parameter search.

[0052] Execute the optimization process: Start the Bayesian optimizer, which intelligently selects new weight combinations for evaluation based on previous test results. In each iteration, the current optimal weight combination and its corresponding classification indicator performance are recorded.

[0053] Introducing K-fold cross validation: Introducing K-fold cross validation (5-fold) in each weight adjustment process ensures that the model does not overfit a specific data segmentation and improves the stability of the evaluation. For each candidate weight combination, multiple validations are performed through K-fold cross validation to calculate its average performance index and select the most stable weight. .

[0054] Step 8: Use the optimized false information detection model to re-determine whether the original text on the current grid platform is false information; The specific process is as follows: S8.1. Extract the first The original text of the grid platform and the The original data of the grid platform is preprocessed by text cleaning, word segmentation, and stop word removal to ensure the data quality input into the pre-trained large language model and obtain the new grid platform original data. ; New grid platform original data The expression is as follows: ; ; in, It is the text content of the original text of the grid platform. To publish the original text of the grid platform, The time when the user published the original text of the grid platform. The ID of the user who posted the repost or comment. The text content of the repost or comment (if some users do not add comments when reposting, the project will have no content). The time of posting and commenting. For the new Comments.

[0055] S8.2. New grid platform original data Input into the pre-trained large language model and generate a new score between 1 and 5 ; The new grid platform original data and the new Comments All of them are input into the trained large language model to get the new sentiment score for each word in each comment. ; The emotional tendency of a single comment is obtained based on the emotional score calculation formula and threshold judgment of the comment text. .

[0056] S8.3, calculate the overall accuracy; According to the new rating results and the sentiment of each comment Calculate the new confidence scores , New review feedback score and a new rating stability score ; Based on the new confidence score , New review feedback score and a new rating stability score , and obtain a new comprehensive accuracy ; S8.4, determine the results of false information; The new overall accuracy of each score .

[0057] Based on the new comprehensive accuracy Determine whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the new comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the new comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

[0058] S8.5. Compare the false information judgment result obtained in step six with the false information judgment result obtained in step 8.4 to verify the accuracy of the false information judgment result obtained in step six.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A grid false information detection method based on a large language model, characterized in that: The following steps are involved: Step 1: Establish a data set to be tested based on the original network information; The data set to be tested includes the original information of the grid platform, the forwarding information related to the original information of the grid platform, and the comment information related to the original information of the grid platform; The grid platform original text information includes text content, information about the user who posted the grid platform original text, and the time when the user posted the grid platform original text; Forwarding information includes the user ID that posted the forwarding, the text content of the forwarding, and the time of posting the forwarding; commenting information includes the user ID that posted the comment, the text content of the comment, and the time of posting the comment; Step 2: Use the pre-trained large language model to Analyze the original information of the grid platform and get its scoring results ; Step 3: Grid platform original information No. Calculate the score difference of all the scoring results of the question type and get the Grid platform original information No. The difference in scores between question types; Step 4: Grid platform original information No. Score difference and threshold for question types Compare; if the score difference is greater than the threshold , then return to step 2 to optimize the scoring problem of the corresponding question type, and then use the pre-trained large language model to optimize the scoring problem of the first Grid platform original information No. Regenerate the scoring results for all scoring questions of the question type, and recalculate their score differences through step 3 until the final score difference is less than the threshold , and get the final scoring result ; Step 5: Use the large language model to analyze the comment information in the test data set to identify the sentiment tendency of each comment; Step 6: Based on the scoring results Get the confidence score based on the Grid platform original information The sentiment tendency of all forwarded information and / or comment information is obtained to obtain the comment feedback score of the original text on the current grid platform. The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is Get the stability score of the original text of the current grid platform; The comprehensive accuracy of the original text of the current grid platform is calculated based on the confidence score, comment feedback score and stability score of the original text of the current grid platform to determine whether the original text of the current grid platform is false information.

2. The grid false information detection method based on a large language model according to claim 1 is characterized in that: Get the first Scoring results of the original text on the grid platform The specific process is as follows: S2.1, set for There are some issues in scoring the authenticity of the original text on the grid platform. There are Grid platform original information; S2.2, based on the The problem of scoring the authenticity of the original text on the grid platform uses a pre-trained large language model to All grid platforms are original The corresponding reply is obtained from the current grid platform original text between Rating results between ; Among them, 1 is very untrue; 5 is very true; is the first Grid platform original information, ; For the The first one in the original information of the grid platform The answer to the question, .

3. The grid false information detection method based on a large language model according to claim 2 is characterized in that: Get the first Scoring results of the original text on the grid platform The specific process is as follows: Designed for The questions for scoring the authenticity of the original text on the grid platform include scoring question types and specific questions; Get the first Grid platform original information , and the Grid platform original information Input into the pre-trained large language model, the pre-trained large language model is Grid platform original information The semantic information in Grid platform original information Gain a comprehensive understanding; Based on the scoring question type and the specific question, ask the large language model to Grid platform original information Ask a series of questions about authenticity The inquiry resulted in Grid platform original information A rating result of ;in, For the Types of questions, , is the total number of question types; For the The first type of question Rating question, , is the total number of scoring questions.

4. The method for detecting false information in a grid based on a large language model according to claim 3, characterized in that: Get the first Grid platform original information No. The specific method for calculating the score difference of the question type is: Set up The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , then The difference in scores for the question types is ; when , then the After optimizing the two scoring problems of the same type, return to step 2 for re-scoring. If the score difference is still too large, discard the two questions of the same type with too large score difference.

5. The grid false information detection method based on a large language model according to claim 1 is characterized in that: The specific method to obtain the sentiment tendency results of each comment is: S5.

1. Obtain the first Grid platform original information And the corresponding to the original text of the grid platform Forwarded messages and Comments ; S5.

2. Grid platform original information And the first one corresponding to the original text of the grid platform Forwarded messages and Comments All are input into the pre-trained large language model for analysis to identify the current forwarding information and current comment information position; And, through the pre-trained large language model, the current forwarding information and / or current comment information Perform sentiment analysis to obtain the current forwarding information and / or current comment information The sentiment score of each word in , and determine the current forwarding information and / or current comment information emotional inclination; Set to determine the current forwarding information and / or current comment information The sentiment score of each word in The threshold of the sentiment tendency is 0.5, and the judgment rules are as follows: ； in, A binary variable representing the current forwarding information and / or current comment information the position in S5.3, repeat step S5.2 until the Grid platform original information The position of all forwarded information and / or commented information.

6. The grid false information detection method based on a large language model according to claim 5 is characterized in that: The specific process of obtaining the stability score of the current grid platform original text is as follows: S6.

1. Assume scoring result The probability value is ,in, , is the total number of probabilities scored; Based on the scoring results The probability value is used to calculate the entropy value , and in order to ensure the entropy value Normalized to In the interval, the normalized entropy value is obtained ; According to the normalized entropy value Adjust the confidence score to get the adjusted confidence score ; S6.2, based on Grid platform original information The positions of all forwarded information and / or comment information are statistically calculated to obtain the overall attitude value, that is, the comment feedback score of the original text on the current grid platform ; S6.3, based on The scoring result of the first scoring question in the question type is , No. The scoring result of the second scoring question in the question type is , calculate the average score ; Based on the mean score Calculate the absolute difference ; Based on absolute difference Sum the total absolute difference between the ratings of all questions of all types and the average rating ; Based on total absolute difference Calculating the scoring stability score ; The scoring stability score is used to measure the scoring consistency of the original text on the same grid platform for the same type of questions. The higher the scoring stability score, the better the scoring consistency of the original text on the grid platform for the same type of questions. The following formula is used to express its scoring consistency : ； S6.

4. Using confidence scores , Comment feedback score and scoring stability score , calculate the comprehensive accuracy of the original text of the current grid platform ; S6.

5. Determine whether the original text of the current grid platform is false information; Based on the comprehensive accuracy, it is judged whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

7. The grid false information detection method based on a large language model according to claim 6 is characterized in that: Get the comment feedback score of the original text on the current grid platform The specific process is as follows: For Grid platform original information Statistical analysis of the sentiment of all forwarded messages and / or comments is performed to calculate the ratio of supportive to opposing comments, and then an overall attitude value is obtained. ; Overall attitude value It can be expressed as a value between 0 and 1, where 0 means total opposition, which means that the original text of the current grid platform is false information; 1 means total support, which means that the original text of the current grid platform is not false information.

8. The grid false information detection method based on a large language model according to any one of claims 1 to 7, characterized in that: The method further includes obtaining an optimized false information detection model by optimizing the process of step 1 to step 6, and using the optimized false information detection model to verify whether the original text of the current grid platform is false information, which specifically includes: Step 7: Evaluate the performance of the process from step 1 to step 6, determine the false information judgment threshold based on the evaluation results, and adjust the weight of calculating the comprehensive accuracy to obtain an optimized false information detection model; Step 8: Apply the optimized false information detection model to re-determine whether the original text of the current grid platform is false information.

9. The grid false information detection method based on a large language model according to claim 8, characterized in that: The specific process of optimizing the false information detection model is as follows: S7.

1. Perform an initial evaluation of the false information detection model. The specific process is as follows: ①、Preparation of validation set: Randomly extract a portion of the dataset to be tested as the validation set, ensuring that it covers grid platform originals of different topics, types, and time periods, and that the number of non-false information samples and false information samples in the validation set is relatively balanced; ②, set a preliminary threshold at the beginning, and use the validation set to calculate the classification indicators of the model under the preliminary threshold. The classification indicators include accuracy, precision, recall and F1 score; ③. For a series of possible thresholds, calculate the true positive rate and false positive rate at each threshold respectively; then, draw the ROC curve for each threshold, and find the best threshold that can maximize the true positive rate and minimize the false positive rate through the ROC curve; ④. Calculate the area AUC value of the ROC curve; ⑤. Analyze the AUC value and ROC curve, and select the threshold near the highest AUC value as ; S7.

2. Use the Bayesian optimization method to iteratively optimize the false information detection model. The specific process is as follows: ①. Set initial weight: At the beginning, set the initial weight As a starting point for the search, ensure that the sum of the weights is equal to 1; ②. Build a Bayesian optimizer: Use the Scikit-Optimize Bayesian optimization library to create an objective function that receives a set of weight parameters as input and returns the F1 score of the large language model on the validation set; ③. Execute the optimization process: Start the Bayesian optimizer to intelligently select new weight combinations for evaluation based on previous test results; and in each iteration, record the current optimal weight combination and its corresponding classification indicator performance; ④. Introduce K-fold cross validation: Introduce K-fold cross validation in each weight adjustment process to ensure that no overfitting of a specific data segmentation occurs; for each candidate weight combination, perform multiple validations through K-fold cross validation, calculate its average performance index, and select the most stable weight. .

10. The grid false information detection method based on a large language model according to claim 8, characterized in that: The specific process of applying the optimized false information detection model to re-judge whether the original text of the current grid platform is false information is as follows: S8.

1. Extract the first Article Grid Platform original text, and The original data of the grid platform is preprocessed by text cleaning, word segmentation, and stop word removal to ensure the quality of data input to the large language model and obtain new grid platform original data. ; S8.

2. New grid platform original data Input into the large language model and generate a new score between 1 and 5 ; The new grid platform original data and the new Comments Input them into the large language model to get the new sentiment score for each word in each comment ; The emotional tendency of a single comment is obtained based on the emotional score calculation formula and threshold judgment of the comment text. ; S8.3, calculate the overall accuracy; According to the new rating results and the sentiment of each comment Calculate the new confidence scores , New review feedback score and a new rating stability score ; Based on the new confidence score , New review feedback score and a new rating stability score , and obtain a new comprehensive accuracy ; S8.4, determine the results of false information; Based on the new comprehensive accuracy Determine whether the original text of the current grid platform is false information; the specific judgment process is as follows: If the new comprehensive accuracy , then the original text of the current grid platform is considered to be false information; If the new comprehensive accuracy , then it is believed that the original text of the current grid platform is not false information.

Images