An emergency distortion information identification method and system

Abstract

The application belongs to the technical field of information analysis, and particularly discloses a sudden event distortion information identification method and system, which comprises the following steps: constructing a keyword table of a sudden event according to keywords of the sudden event; screening a sudden event distortion information data set to construct a sudden event distortion information training data set and a test data set; building a first-stage verification large model through a sample embedding prompt strategy and building a second-stage review large model through a chain thinking prompt strategy; constructing an XLNet-BiLSTM hybrid model to identify sudden event distortion information; constructing a model dynamic integration rule considering the consequences of illusion relief and specific distortion scores, and obtaining the distortion information identification results of the two-stage large model and the XLNet-BiLSTM hybrid model. The application ensures the consistency and understandability of the identification results and the generated explanation, and highlights the transferability of information identification in different types of sudden events.

Description

Technical Field

[0001] This invention belongs to the field of information analysis technology, specifically, it relates to a method and system for identifying distorted information of sudden events, and more specifically, it relates to a method and system for identifying distorted information of sudden events that integrates a two-stage large model and XLNet-BiLSTM. Background Technology

[0002] Existing methods for identifying distorted information fall into three categories: statistical methods, machine learning, and deep learning. These methods primarily design algorithms from the perspectives of model evolution, graph propagation, and information features. However, distorted information from sudden events differs from general distorted information, generally exhibiting complex characteristics such as misleadingness, diversity, immediacy, and dynamism. Sudden events are often related to specific times, places, and contexts, and continuously evolve as the situation develops, resulting in significant immediacy and dynamic changes in the related distorted information. However, due to their dependence on fixed training datasets, traditional identification methods based on information features often face problems such as limited data coverage and lagging knowledge updates, making it difficult to quickly adapt to changes in distorted information from sudden events. This has led to relatively few studies on distorted information identification in the field of sudden events by scholars both domestically and internationally.

[0003] Recent advancements in large language model technology have provided new solutions for identifying distorted information. Current research has demonstrated the effectiveness of large language models in verifying and detecting news headlines and rumors, showcasing their unique knowledge reserves, logical reasoning, and semantic understanding capabilities in identifying distorted information. However, most studies have overlooked the "illusion" drawback of large models—the phenomenon where the content generated by the model does not match objective facts or user input. In distorted information identification, this manifests as incorrect judgments of information and the generation of misleading explanatory text. To overcome the illusion problem, the most fundamental measure is to verify the training data and alignment data of large models to ensure the complete reliability of the data. However, due to the high manual cost, scholars tend to look for ways to mitigate the illusion: Ji Z et al. pointed out that the uncertainty in the generated text characters, words, and sentences can be reflected by token probabilities to evaluate the illusion, but this can generally only be applied to large models deployed locally and cannot be used in widely used applications that call APIs; Sun X et al. proposed a self-verification strategy by using the large model itself as an "illusion" recognizer, but whether the recognizer itself has an illusion cannot be evaluated; Manakul P et al. repeatedly sampled the text generated by the large model to detect the consistency between the test text and the generated text under multiple metrics, but random sampling of a large number of texts means higher computational burden and time cost. Therefore, the following problems remain unsolved: 1) Machine learning algorithms and pre-trained models are limited by the size and type of dataset; 2) Large language models are also not trusted in some studies due to the existence of illusions. Summary of the Invention

[0004] To address the aforementioned shortcomings or improvement needs of existing technologies, this invention provides a framework for identifying distorted information about sudden events by integrating a two-stage large model and XLNet-BiLSTM. The framework includes: compiling keywords for four types of sudden events to obtain a keyword table; using this keyword table to filter distorted information datasets from six different domains and multiple sources, followed by data preprocessing to construct training and testing datasets for distorted information; building a large-scale verification model using a sample embedding prompting strategy; building a large-scale review model using a chain-thinking prompting strategy; building a two-stage large model based on a self-verification strategy, and determining whether the large model generates hallucinations or makes misjudgments by comparing the judgment reasons of the two-stage large model with experimental results; constructing an XLNet-BiLSTM hybrid model to identify distorted information about sudden events; and constructing a dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores to obtain the final identification result. This invention demonstrates good accuracy in identifying distorted information about sudden events, ensuring consistency and comprehensibility between the identification results and the generated explanations, and highlighting the transferability of information identification across different types of sudden events.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for identifying distorted information in sudden events is proposed, comprising the following steps:

[0006] Step 1: Construct a keyword list for emergencies based on the keywords associated with them;

[0007] Step two: The dataset of distorted information about emergencies is filtered through a keyword list of emergencies, and then the data is preprocessed to construct training and testing datasets for distorted information about emergencies.

[0008] Step 3: Build the first-stage verification model using a sample embedding prompting strategy, and build the second-stage review model using a chain thinking prompting strategy. By comparing the judgment reasons of the two-stage models with the experimental results, determine whether the large model has produced illusions or made misjudgments.

[0009] Step 4: Construct an XLNet-BiLSTM hybrid model to identify distortion information in sudden events;

[0010] Step 5: Construct a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores, and use this rule to obtain the distortion information recognition results of the two-stage large model and the XLNet-BiLSTM hybrid model.

[0011] As a further preferred option, in step one, the emergencies include: natural disasters, accidents, public health emergencies, and social security incidents.

[0012] As a further optimization, in step two, distorted information datasets from different fields and sources are screened, and then duplicate texts are removed by similarity algorithm. Finally, a distorted information dataset for sudden events is constructed and used as the training dataset for each experimental model.

[0013] In addition, considering factors such as time sequence, event distribution, and label balance, the training dataset for distorted information about sudden events was filtered through a keyword list of sudden events, thus forming a test dataset.

[0014] As a further preferred step, in step three, the first-stage verification model performs information recognition on four parts: sample embedding, source declaration, text information and task instructions, and generates verification results and distortion scores, thereby extracting verification labels.

[0015] As a further optimization, in step three, the second-stage review model comprehensively evaluates the data through five aspects: source declaration, text information, verification results, task instructions, and chain thinking. It also makes adjustments and corrections based on the verification results of the first-stage verification model, and finally generates review results and review labels.

[0016] As a further preferred step, in step four, the pre-trained model is combined with a deep learning algorithm to construct an XLNet-BiLSTM hybrid model to enhance the ability to understand the bidirectional contextual semantics and extract features from information text. The XLNet model is used to capture the deep semantic features of information text, while the BiLSTM model is used to learn the contextual patterns in the semantic features, and finally to identify distorted information.

[0017] As a further preferred embodiment, in step five, the construction of a model dynamic integration rule that considers mitigating the consequences of hallucinations and a specific distortion score includes: when the results of the two-stage large model are inconsistent, or when the distortion scores of the two-stage large model are in the range of [40, 60), the XLNet-BiLSTM model is selected as a substitute to identify the distortion information of the sudden event, thereby dynamically integrating the experimental results of the two.

[0018] Specifically, the process first checks whether the results of the check model (Check_label) and the review model (Review_label) in the two-stage large model are equal. If they are equal, proceed to the next step; otherwise, set the final result (Final_label) to the XLNet-BiLSTM model result (PLM_label). Then, it checks whether the average distortion scores of the check and review models are within the range [40, 60). If they are within this range, set the final result to the XLNet-BiLSTM model result; otherwise, set the final result to the two-stage large model result (LLM_label). The dynamic model integration process then ends.

[0019] According to another aspect of the present invention, a system for identifying distorted information in sudden events is also provided, comprising:

[0020] The first main control module is used to construct a keyword table for emergencies based on the keywords of the emergencies.

[0021] The second main control module is used to filter the dataset of distorted information about emergencies through a keyword list of emergencies, and then perform data preprocessing to construct training and testing datasets for distorted information about emergencies.

[0022] The third main control module is used to build a first-stage verification model through a sample embedding prompting strategy, and to build a second-stage review model through a chain thinking prompting strategy. By comparing the judgment reasons of the two-stage models with the experimental results, it is determined whether the large model has produced illusions or made misjudgments.

[0023] The fourth main control module is used to build an XLNet-BiLSTM hybrid model to identify distortion information of sudden events;

[0024] The fifth main control module is used to construct a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores, and to obtain the distortion information recognition results of the two-stage large model and the XLNet-BiLSTM hybrid model.

[0025] As a further optimization, the first-stage verification model performs information identification in four parts: sample embedding, source declaration, text information and task instructions, and generates verification results and distortion scores to extract verification labels; the second-stage review model conducts a comprehensive evaluation through five links: source declaration, text information, verification results, task instructions and chain thinking, and makes adjustments and corrections based on the verification results of the first-stage verification model, and finally generates review results and review labels.

[0026] As a further optimization, in the fourth main control module, the pre-trained model is combined with the deep learning algorithm to construct the XLNet-BiLSTM hybrid model to enhance the ability to understand the bidirectional contextual semantics and extract features of information text. The XLNet model is used to capture the deep semantic features of information text, and the BiLSTM model is used to learn the contextual rules in the semantic features, and finally to identify distorted information.

[0027] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0028] In response to the current lack of text datasets for emergency events, this invention filters and refines multivariate distorted information datasets based on national standards, resulting in a dataset specifically designed for distorted information related to emergency events. Furthermore, it proposes a distorted information recognition framework that integrates a two-stage large language model and the XLNet-BiLSTM model. The advantages of this framework mainly include the following aspects:

[0029] First, the TLXM framework proposed in this invention achieved the best accuracy in comparative experiments with various model algorithms such as machine learning, deep learning, pre-trained models, and large language models. Two sets of model ablation experiments were also designed to explore the contribution of each unit of the framework to the recognition results and the degree of decline in recognition performance after removing each unit, thereby demonstrating the necessity of each unit and the effectiveness of the framework structure.

[0030] Second, the TLXM framework proposed in this invention mitigates the consequences of hallucinations, ensures the consistency of framework recognition results, and uses different prompting strategies to reduce the occurrence of model hallucinations in the generated text of large models, resulting in higher factual consistency and thus providing an explanatory basis for the framework results.

[0031] Third, the feasibility of the framework proposed in this invention in identifying real-time distorted information text was verified through information texts from four types of recent emergencies. This demonstrates that the framework can be transferred to different emergencies for information identification, highlighting the application potential and practical value of the framework on real-world internet platforms. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the distortion information recognition framework that integrates a two-stage large model and XLNet-BiLSTM involved in the embodiments of the present invention.

[0033] Figure 2 This is a schematic diagram of the two-stage large language model reasoning process based on different prompting strategies involved in the embodiments of the present invention;

[0034] Figure 3 This is a structural diagram of the XLNet-BiLSTM hybrid model involved in the embodiments of the present invention;

[0035] Figure 4 This is a schematic diagram of the model dynamic integration rule decision tree involved in the embodiments of the present invention;

[0036] Figure 5 This is a schematic diagram illustrating the distribution of correct and incorrect values ​​of LLM verification, LLM review, and XLNet-BiLSTM in an embodiment of the present invention.

[0037] Figure 6 This is a schematic diagram of the error distribution of the two-stage large model, the verification of LLM, the review of LLM, and the error distribution of XLNet-BiLSTM in an embodiment of the present invention;

[0038] Figure 7 This is a schematic diagram illustrating the consistency index of generated text under different prompting strategies in embodiments of the present invention;

[0039] Figure 8 This is a schematic diagram of the portability test results of the TLXM framework in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0041] Example 1

[0042] like Figure 1 As shown in the figure, this invention provides a method for identifying distorted information about sudden events by integrating a two-stage large model and XLNet-BiLSTM. The method includes: organizing keywords for four types of sudden events to obtain a keyword table; filtering distorted information datasets from six different fields and multiple sources using the keyword table, followed by data preprocessing to construct training and testing datasets for distorted information about sudden events; building a verification large model using a sample embedding prompting strategy; building a review large model using a chain-thinking prompting strategy; building a two-stage large model based on a self-verification strategy, and determining whether the large model produces hallucinations or misjudgments by comparing the judgment reasons of the two-stage large model with experimental results; constructing an XLNet-BiLSTM hybrid model to identify distorted information about sudden events; and constructing a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores to obtain the final identification result. This invention demonstrates good accuracy in identifying distorted information about sudden events, ensuring consistency and comprehensibility between the identification results and the generated explanations, and highlighting the transferability of information identification in different types of sudden events.

[0043] Specifically, the method includes:

[0044] Step 1: Organize the keywords for the four types of emergencies to obtain an emergency keyword list.

[0045] In one embodiment of the present invention, in step one, a keyword list corresponding to four types of emergencies is obtained by compiling the national standard for emergencies (GB / T35561-2017). The four types of emergencies are abbreviated as natural disaster events (N), accident disaster events (A), public health events (H), and social security events (S).

[0046] Step 2: Use the emergency keyword list to filter emergency distortion information datasets from six different fields and multiple sources, and then perform data preprocessing to construct emergency distortion information training datasets and test datasets.

[0047] In step two, six distorted information datasets from different fields and multiple sources were screened, and duplicate texts were removed using a similarity algorithm. This resulted in the construction of a distorted information dataset (MEE) for emergency events, which served as the training dataset for various experimental models. The training dataset spans from 2009 to 2021, with a generally balanced label distribution. The information texts basically cover the main areas of four types of emergencies to ensure comprehensive knowledge regarding emergencies, totaling 10,575 data entries. The original dataset and the emergency event distorted information training dataset are shown in the table below:

[0048]

[0049] Note: T / F indicates the label is true or false.

[0050] Furthermore, considering factors such as chronological order, event distribution, and label balance, the training dataset containing distorted information about emergencies was filtered using a keyword list for these events, resulting in a test dataset of 1096 data entries. See the table below:

[0051]

[0052] Note: T / F indicates the label is true or false.

[0053] Step 3: Build a large-scale verification model using a sample embedding prompting strategy. In this step, the first-stage verification model performs information recognition in four parts: sample embedding, source declaration, text information, and task instructions, generating verification results and distortion scores to extract verification labels. More specifically, in this step, the first-stage verification model adopts a few-sample embedding prompting strategy. This strategy provides the large model with a small number of examples, including task descriptions and expected results, which can improve the initial accuracy of the entire framework and enable the large model to quickly understand the task intent and output standards. The first-stage verification model performs information recognition in four parts: sample embedding, source declaration, text information, and task instructions, generating verification results and distortion scores to extract verification labels.

[0054] Step Four: Building a Comprehensive Review Model Using Chain Thinking Prompts. In this step, the second-stage review model comprehensively evaluates the model through five stages: source declaration, text information, verification results, task instructions, and chain thinking. It is adjusted and corrected based on the verification results of the first-stage verification model, ultimately generating review results and review tags. More specifically, this step employs a chain thinking prompt strategy for the second-stage review model, divided into five thinking steps: clarifying sentiment, event type, logical structure, verification explanation, and sentence format standardization. This chain thinking prompt strategy allows the model to think progressively during response generation, breaking down complex tasks into simpler problems. It also clarifies task steps in the prompts to limit the dimensions of the large model's thinking, thereby reducing the probability of the large model generating illusions and the randomness of the output results. The second-stage review model comprehensively evaluates the model through five stages: source declaration, text information, verification results, task instructions, and chain thinking. It is adjusted and corrected based on the verification results of the first-stage verification model, ultimately generating review results and review tags.

[0055] Step 5: Build a two-stage large model based on a self-verification strategy, and determine whether the large model produces illusions or misjudgments by comparing the judgment reasons of the two-stage large model with the experimental results. In this step, the reasoning flowchart of the two-stage large model based on different prompting strategies is illustrated as follows: Figure 2 As shown in the figure above, the distortion score given by the review model is 10 points lower than that of the verification model, indicating consistency in the self-verification results. However, for other distorted text information, the labels of the two-stage models may be inconsistent. Therefore, this invention suggests that the results generated by the large language model may contain illusions and lead to misjudgments. Thus, the accuracy of the recognition results cannot be guaranteed by the large language model alone, and it is necessary to consider it in conjunction with a pre-trained model.

[0056] Step Six: Construct an XLNet-BiLSTM hybrid model to identify distorted information in sudden events. This model, based on the two-stage large model's assessment of distortion, performs a judgment transfer and uses an XLNet-BiLSTM hybrid model to identify distorted information in sudden events. In this step, a pre-trained model is combined with a deep learning algorithm to construct an XLNet-BiLSTM hybrid model to enhance the ability to understand the bidirectional contextual semantics and extract features from the information text. The XLNet model can capture the deep semantic features of the information text, while the BiLSTM model can learn the contextual patterns in the semantic features, ultimately identifying distorted information. The structure of the XLNet-BiLSTM hybrid model is as follows: Figure 3 As shown in the diagram. In this step, given the dependence of pre-trained models and deep learning algorithms on the training dataset, when there is a potential for illusions to occur in the large language model, the XLNet-BiLSTM model based on a fixed dataset can be considered as an alternative to improve the reliability and accuracy of the recognition framework.

[0057] Step 7: Construct a dynamic model integration rule that considers mitigating the consequences of hallucinations and specific distortion scores to obtain the final recognition result. In this step, a model integration rule is constructed. When the results of the two large models are inconsistent, or when the distortion score of the large model is in the range [40, 60), the XLNet-BiLSTM model is selected as a substitute, thereby dynamically integrating the experimental results of the two models. The specific process is as follows: Figure 4 As shown.

[0058] Specifically, the process first checks whether the results of the check model (Check_label) and the review model (Review_label) in the two-stage large model are equal. If they are equal, proceed to the next step; otherwise, set the final result (Final_label) to the XLNet-BiLSTM model result (PLM_label). Then, it checks whether the average distortion scores of the check and review models are within the range [40, 60). If they are within this range, set the final result to the XLNet-BiLSTM model result; otherwise, set the final result to the two-stage large model result (LLM_label). The dynamic model integration process then ends.

[0059] Based on the above rules, this invention can, on the one hand, use the results of the XLNet-BiLSTM model to correct the results of the two-stage large model in a timely manner, so as to avoid the influence of illusion as much as possible; on the other hand, from the perspective of specific distortion scores, it considers and selects the advantageous range of the two-stage large model and the XLNet-BiLSTM model in identifying distortion information, thereby enhancing the robustness of distortion information identification, realizing dynamic model integration and obtaining the final prediction result.

[0060] Example 2

[0061] refer to Figure 1 This invention proposes a distortion information recognition framework that integrates a two-stage large model with XLNet-BiLSTM. The specific implementation method of the invention is described in detail below:

[0062] Step 1: Determine the model and model parameters to be used in advance.

[0063] Step 2: Given the above conditions, write code to construct the model framework proposed in this method and identify information about sudden events.

[0064] To simplify the text of this application and avoid unnecessary detail, the specific solutions and formulas involved herein can be found in the invention content section of the preceding specification, and will not be repeated here.

[0065] Example 1

[0066] The following is a detailed description based on Example 1:

[0067] This invention selects GLM-4-Air, released by Zhipu Qingyan, as the experimental model among the top three large-scale models in China during a certain period. A two-stage large-scale model structure is constructed, and information recognition is performed by calling API interfaces. Furthermore, this invention uses the XLNet benchmark Chinese model "chinese-xlnet-base" and the BiLSTM model from the PyTorch package to build the XLNet-BiLSTM model on the Google Colab platform. The model parameters are shown in the table below:

[0068]

[0069] Note: Type; Parameter.

[0070] Based on the given parameters, code was written to construct the model framework proposed in this invention and to identify information about sudden events.

[0071] To verify the performance of the model framework of this invention in distorted information recognition, a comparative experiment was designed, and the experimental results are shown in the table below:

[0072]

[0073] Note: The evaluation metrics are accuracy, precision, recall, and F1 score.

[0074] As can be seen from the table above, the TLXM framework proposed in this invention achieved the best results in the distortion information recognition experiment, with accuracy, precision, recall, and F1 score reaching 85.02%, 84.97%, 85.41%, and 84.97%, respectively. This indicates that the TLXM framework of this invention, by combining the advantages of large language models and pre-trained models, significantly outperforms other models in experimental results.

[0075] To explore the contribution of each unit of the TLXM framework to the final information recognition accuracy, this invention designed two types of ablation experiments. The first type demonstrates the information recognition results of each unit of the framework on the test set individually, as shown in the table below:

[0076]

[0077] Note: The evaluation metrics are accuracy, precision, recall, and F1 score.

[0078] As shown in the table above, the XLNet-BiLSTM hybrid model outperforms the individual XLNet and BiLSTM models in information recognition, demonstrating the necessity of combining the two. Furthermore, the accuracy of XLNet-BiLSTM, the verification model, and the review model are all lower than the TLXM framework, indicating that the method of fusing the two-stage large model structure with the XLNet-BiLSTM model in this invention can leverage the strengths of each model while mitigating its weaknesses, thus improving the accuracy of the final recognition result.

[0079] Secondly, the information recognition performance of the TLXM framework on the test set was compared after removing each unit. The results are shown in the table below:

[0080]

[0081] Note: The evaluation metrics are accuracy, precision, recall, and F1 score.

[0082] As shown in the table above, the ablation combination omitting XLNetBiLSTM, the verification large model, and the review large model units lagged behind the TLXM framework in accuracy by 1.10%, 1.37%, and 3.75%, respectively, demonstrating the indispensability of each unit and the rationality of the TLXM framework. Among these, the ablation combination retaining only the two-stage large model structure yielded results closest to the TLXM framework, highlighting the unique importance of the large language model within the framework. However, its accuracy was lower than that of the review large model alone. This is because the two large models may have produced conflicting results, making the use of the XLNet-BiLSTM model as a replacement for the two-stage large model structure reasonable. This also contributes to improving the accuracy of the final recognition results in the experimental results.

[0083] To verify the rationality and effectiveness of the dynamic integration rules, the MEE dataset was used to perform validation analysis on both stages of the rules. When the results of the two-stage large model contradicted each other, the distribution of correct and incorrect results of the large model verification, large model review, and XLNet-BiLSTM model were as follows: Figure 5 As shown:

[0084] As shown in the figure, the XLNet-BiLSTM model achieves an accuracy of 73%, significantly higher than the two large language models. This inconsistency is partly due to hallucinations, and partly due to the randomness of the large models. Therefore, when the results of the two-stage large models are inconsistent, choosing the XLNet-BiLSTM model, which performs consistent with the MEE dataset, as an alternative can ensure the consistency of recognition results and effectively mitigate the consequences of hallucinations.

[0085] Figure 6 The error patterns of the two-stage large model and the XLNet-BiLSTM model are shown separately under different distortion score ranges:

[0086] As can be seen from the figure, within the distortion score range of 40-60, the error rate of the two-stage large model is significantly higher than in other ranges, while the number of errors of the XLNet-BiLSTM model is less than that of the two large models within the same score range. However, in other distortion score ranges, the accuracy of the two-stage large model is significantly higher than its error rate, and the number of errors of the XLNet-BiLSTM model is also significantly higher than that of the two-stage large model.

[0087] This invention constructs a two-stage large model structure by employing different prompting strategies. However, the combination of prompting strategies is not unique. Therefore, this invention designs four prompting strategies: "zero-sample embedding", "zero-sample + thought chain", "few-sample embedding", and "few-sample + thought chain". The table below shows the recognition performance of the GLM-4-Air model on the MEE dataset under the guidance of different prompting strategies.

[0088]

[0089] Note: The evaluation metrics are accuracy, precision, recall, and F1 score.

[0090] As can be seen from the table above, the three large models that adopted the prompting strategy provided standard examples, which enabled them to more clearly address the task requirements and understand the verification perspective. Therefore, they all showed a certain performance improvement compared to the zero-shot embedding scenario. Among them, the group that used the few-shot embedding strategy achieved the best accuracy.

[0091] To assess the reduction in the probability of hallucination under different prompting strategies, this invention uses three methods—ROUGE, BERTScore, and FactCC based on natural language reasoning—for analysis. ROUGE focuses on determining the lexical overlap of the generated text under different prompting strategies, while BERTScore and FactCC assess the semantic similarity and factual consistency between the generated text and the original text. Greater lexical overlap between the generated texts, higher semantic similarity to the original text, and higher confidence scores for factual consistency indicate greater consistency and a lower likelihood of hallucination. The results are as follows: Figure 7 As shown:

[0092] As shown in the figure above, the large model's generated text based on the chain-thinking strategy (cot) outperformed the other two groups in all three metrics. Furthermore, in the FactCC test, the generated text based on the chain-thinking strategy exhibited an implication relationship with the original text, while the other two groups contained generated texts with some contradictions and neutral relationships. This indicates that the chain-thinking strategy effectively improves the similarity of the generated text to the original text and maintains a high degree of factual consistency with the key content of the original text, thereby reducing the possibility of introducing irrelevant or erroneous information. This is because the thought chain clearly defines the content of each step of the large model's reasoning, progressively considering the degree of distortion in the information text from multiple aspects such as emotional polarity and intensity, event type and theme, causal logic, contextual structure, and data facts. This allows the large model to think step by step based on the original text information, avoiding arbitrary conclusions based on a specific content and reducing the possibility of the large model's thinking jumping to other information, thus contributing to reducing the probability of illusions and ensuring the consistency and credibility of the large model's generated text.

[0093] To verify the portability of the TLXM framework in the complex environment of internet platforms after an emergency, this invention selects typical cases for four types of emergencies. For the three cases of natural disasters, accidents, and social security incidents, information text was crawled from Weibo based on corresponding keywords and manually labeled. This invention selects information text related to emergencies from the WBV1 dataset as event cases for the portability test. The test results are as follows: Figure 8 As shown.

[0094] As shown in the figure above, the TLXM framework achieved accuracy rates of 96.56%, 95.89%, 94.05%, and 94.47% in the four types of emergency event information texts, respectively, demonstrating the accuracy and transferability of the TLXM framework proposed in this invention in identifying different event information texts.

[0095] Although the invention has been described in considerable detail and particularly with regard to several of the described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of the claims by referring to the appended claims and taking into account the prior art, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

[0096] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying distorted information in sudden events, characterized in that, Includes the following steps: Step 1: Construct a keyword list for emergencies based on the keywords associated with them; Step two: The dataset of distorted information about emergencies is filtered through a keyword list of emergencies, and then the data is preprocessed to construct training and testing datasets for distorted information about emergencies. Step 3: Build the first-stage verification model using a sample embedding prompting strategy, and build the second-stage review model using a chain thinking prompting strategy. By comparing the judgment reasons of the two-stage models with the experimental results, determine whether the large model has produced illusions or made misjudgments. The first-stage verification model adopts a few-sample embedding prompting strategy. The first-stage verification model performs information recognition in four parts: sample embedding, source declaration, text information and task instructions, and generates verification results and distortion scores, thereby extracting verification labels. The second-stage review model employs a chain-thinking prompting strategy, consisting of five thinking steps: clarifying sentiment, event type, logical structure, verification and explanation, and sentence format standardization. The second-stage review model comprehensively evaluates the data through five links: source declaration, text information, verification results, task instructions, and chain-thinking. It is adjusted and corrected based on the verification results of the first-stage verification model, ultimately generating review results and review labels. GLM-4-Air is used as the experimental model to construct the two-stage model structure. Step 4: Construct an XLNet-BiLSTM hybrid model to identify distortion information of sudden events. Train the XLNet-BiLSTM hybrid model using the training dataset and test the trained XLNet-BiLSTM hybrid model using the test dataset. Step 5: Construct a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores, and use this rule to obtain the distortion information recognition results of the two-stage large model and the XLNet-BiLSTM hybrid model; In step five, the construction of a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores includes: when the results of the two-stage large model are inconsistent, or when the distortion scores of the two-stage large model are in the range of [40, 60), the XLNet-BiLSTM model is selected as a substitute to identify the distortion information of the sudden event, thereby dynamically integrating the experimental results of the two.

2. The method for identifying distorted information in a sudden event according to claim 1, characterized in that, In step one, the emergencies include: natural disasters, accidents, public health emergencies, and social security incidents.

3. The method for identifying distorted information in a sudden event according to claim 2, characterized in that, In step two, distorted information datasets from different fields and sources are screened, and then duplicate texts are removed using a similarity algorithm. Finally, a distorted information dataset for sudden events is constructed and used as the training dataset for each experimental model. In addition, considering factors such as time sequence, event distribution, and label balance, the training dataset for distorted information about sudden events was filtered through a keyword list of sudden events, thus forming a test dataset.

4. The method for identifying distorted information in a sudden event according to claim 3, characterized in that, In step four, the pre-trained model is combined with a deep learning algorithm to construct an XLNet-BiLSTM hybrid model to enhance the ability to understand the bidirectional contextual semantics and extract features from information text. The XLNet model is used to capture the deep semantic features of information text, while the BiLSTM model is used to learn the contextual patterns in the semantic features, and finally to identify distorted information.

5. A system for identifying distorted information during emergencies, characterized in that, A method for identifying distorted information in sudden events as described in any one of claims 1-4 includes: The first main control module is used to construct a keyword table for emergencies based on the keywords of the emergencies. The second main control module is used to filter the dataset of distorted information about emergencies through a keyword list of emergencies, and then perform data preprocessing to construct training and testing datasets for distorted information about emergencies. The third main control module is used to build a first-stage verification model through a sample embedding prompting strategy, and to build a second-stage review model through a chain thinking prompting strategy. By comparing the judgment reasons of the two-stage models with the experimental results, it is determined whether the large model has produced illusions or made misjudgments. The fourth main control module is used to build an XLNet-BiLSTM hybrid model to identify distortion information of sudden events; The fifth main control module is used to construct a model dynamic integration rule that considers mitigating the consequences of hallucinations and specific distortion scores, and thereby obtain the distortion information recognition results of the two-stage large model and the XLNet-BiLSTM hybrid model.

6. The system for identifying distorted information in emergency situations according to claim 5, characterized in that, In the fourth main control module, the pre-trained model is combined with the deep learning algorithm to construct the XLNet-BiLSTM hybrid model to enhance the ability to understand the bidirectional contextual semantics and extract features of information text. The XLNet model is used to capture the deep semantic features of information text, and the BiLSTM model is used to learn the contextual rules in the semantic features, and finally to identify distorted information.

Images