A dynamic, interactive large-scale model fallacy awareness detection method and device

Abstract

This invention provides a method and apparatus for detecting fallacies awareness in large models in dynamic interactive scenarios, belonging to the field of large model fallacies awareness detection technology. The method includes: conducting a first-stage fallacy awareness test on the large model to be tested based on discrimination prompt text and a test dataset using a fallacy awareness discrimination module; conducting a second-stage fallacy awareness test on the large model to be tested based on interviewer prompt text and discrimination prompt text, using the same fallacy awareness discrimination module; and calculating the large model fallacy awareness test result based on a fallacy awareness evaluation index, considering the test dataset, datasets of individuals aware of fallacies, datasets of individuals not aware of fallacies, and datasets where judgment is impossible. This invention provides an accurate and efficient method for detecting large model fallacy awareness in dynamic interactive scenarios.

Description

Technical Field

[0001] This invention relates to the field of large model fallacy awareness detection technology, and in particular to a dynamic interactive large model fallacy awareness detection method and apparatus. Background Technology

[0002] In recent years, with the significant improvement in computing power and the rapid development of deep learning algorithms, Large Language Models (LLMs) have made groundbreaking progress in the field of natural language processing. The core of LLMs lies in utilizing deep neural networks, especially the Transformer architecture, to capture complex patterns and semantic relationships in text. These models are trained on large-scale text datasets and are capable of understanding and generating near-human-level text content. At the same time, the widespread application of LLMs has also brought new problems and challenges, one of which is how to ensure their reliability and security.

[0003] In logic and philosophy, a fallacy is defined as flawed reasoning that appears reasonable but actually uses incorrect or invalid reasoning in constructing the argument. Fallacies are ubiquitous in human life, appearing as unintentional errors or deliberate attempts at persuasion in all kinds of human communication; almost everyone may encounter fallacy problems in their thinking and expression.

[0004] Relying on LLMs for content generation and decision-making may inadvertently amplify or create new fallacies. Given the increasingly important roles LLMs play in various fields, their fallibility becomes crucial. Research on fallacy detection, classification, and generation indicates that LLMs have potential in handling fallacies. However, the ability of LLMs to detect and classify fallacies does not equate to their fallibility awareness. While fallacies may be overlooked or underestimated in human communication, their impact becomes more significant when they permeate LLMs, posing a substantial challenge to the intelligence and reliability of AI systems. Especially with the widespread application of LLM-based agents, where LLMs play different roles in various environments and interact with other agents or users, current research primarily focuses on fallacy identification and classification in static text. The question of whether LLMs can autonomously identify fallacies and respond accurately in dynamic interactive scenarios remains insufficiently researched and resolved.

[0005] Fallacy detection refers to identifying logical errors and invalid reasoning in text. Besides detecting fallacies, it also involves classifying fallacy types. Fallacy classification methods can be categorized into those based on neural networks, pre-trained language models, and large language models to identify common fallacy types. Existing technologies include: fallacy classification at different granularities; using a classifier to calculate loss for each fallacy argument feature (including argument components and component relationships); using a unified model based on multi-task instruction prompts (T5); using large language models to progressively convert natural language into first-order logic for logical fallacy detection; utilizing the pre-trained knowledge of LLMs for zero-shot classification; proposing single-turn and multiple multi-turn prompting schemes; stimulating the fallacy-related knowledge and reasoning ability of LLMs through different task instructions (such as extraction, summarization, and chained reasoning); and improving the performance of LLMs in fallacy detection and classification tasks based on logical structure trees (LSTs).

[0006] In the field of dynamic interaction, the R-Judge benchmark dataset was developed to evaluate the security risk awareness of LLMs (Limited-Level Managers) in multi-turn interactions. This dataset assesses the ability of LLMs to identify security risks in multi-turn interactions. Risk awareness in open agent scenarios involves multi-dimensional capabilities of knowledge and reasoning, but current LLMs still show significant deficiencies in this area. LLMs often lack problem awareness, failing to distinguish between open and closed questions, making them prone to illusions and impacting model performance. Furthermore, LLMs are susceptible to logical fallacies in multi-turn debates, easily changing their perspectives. Their original correct beliefs about factual knowledge can be easily manipulated by fallacy-based persuasion strategies. These studies indicate that LLMs may lack sufficient fallacy awareness in interactive scenarios. Although there has been progress in fallacy detection for LLMs in static text, whether models can autonomously perceive and respond to fallacies in dynamic interactions remains a pressing issue.

[0007] In the existing technology, there is a lack of an accurate and efficient method for detecting large model fallacies in dynamic interactive scenarios. Summary of the Invention

[0008] To address the technical problems of existing technologies that fail to deeply explore the conscious manifestation of models during interaction and lack a systematic theoretical framework and evaluation methods, this invention provides a method and apparatus for detecting large-scale model fallacies in dynamic interactions. The technical solution is as follows:

[0009] On the one hand, a dynamic interactive method for detecting large model fallacies awareness is provided, which is implemented by a large model fallacy awareness detection device. The method includes:

[0010] Obtain a test dataset containing erroneous dialogues;

[0011] Based on the discriminative prompt text and the test dataset, the fallacy awareness discrimination module is used to conduct the first stage fallacy awareness test on the large model to be detected, and obtain the natural response dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first fallacy not awareness dataset, the first indeterminate dataset, and the first fallacy uncertain dataset.

[0012] Based on the interviewer prompt text and the discrimination prompt text, according to the test dataset, the natural response dataset and the first fallacy uncertainty dataset, the fallacy awareness discrimination module is used to conduct a second-stage fallacy awareness test on the large model to be detected, and obtain the second-aware fallacy dataset, the second-unaware fallacy dataset and the second undeterminable dataset.

[0013] Based on the fallacy awareness evaluation index, the fallacy awareness test results of the large model are obtained by calculating the test dataset, the dataset of first-time fallacy awareness, the dataset of first-time fallacy non-awareness, the dataset of first-time inability to judge, the dataset of second-time fallacy awareness, the dataset of second-time fallacy non-awareness, and the dataset of second-time inability to judge.

[0014] On the other hand, a dynamic interactive large model fallacy awareness detection device is provided, which is applied to the dynamic interactive large model fallacy awareness detection method. The device includes:

[0015] The information acquisition module is used to acquire a test dataset containing erroneous dialogues;

[0016] The first-stage testing module is used to conduct a first-stage fallacy awareness test on the large model to be detected based on the discriminative prompt text and the test dataset, through the fallacy awareness discrimination module, to obtain the natural response dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first undetermined dataset, and the first fallacy uncertainty dataset.

[0017] The second-stage testing module is used to conduct a second-stage fallacy awareness test on the large model to be detected based on the interviewer prompt text and the discrimination prompt text, according to the test dataset, the natural response dataset and the first fallacy uncertainty dataset, and the fallacy awareness discrimination module to obtain the second-stage fallacy awareness dataset, the second-stage non-awareness fallacy dataset and the second-stage indeterminate dataset.

[0018] The fallacy awareness evaluation module is used to calculate the fallacy awareness test results of a large model based on the fallacy awareness evaluation index, according to the test dataset, the dataset of first awareness of fallacy, the dataset of first non-awareness of fallacy, the dataset of first inability to judge, the dataset of second awareness of fallacy, the dataset of second non-awareness of fallacy, and the dataset of second inability to judge.

[0019] On the other hand, a large model fallacy awareness detection device is provided, the large model fallacy awareness detection device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods in the above-described dynamic interactive large model fallacy awareness detection method is implemented.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for detecting large model fallacies of dynamic interaction.

[0021] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0022] This invention proposes a method for detecting fallacy awareness in large-scale models (LLMs) in dynamic interactive scenarios. Through a two-stage testing framework and corresponding evaluation metrics, it comprehensively assesses the fallacy awareness capabilities of LLMs in dynamic interactive scenarios. Utilizing three strategies—natural response, closed-ended questioning, and open-ended questioning—it evaluates both the model's initial response and uncovers its deeper fallacy awareness capabilities. By using the fallacy awareness rate and the awareness response depth index, it quantifies the fallacy awareness capability in dynamic interactive scenarios into comparable numerical indicators, solving the problem that existing evaluation systems (such as static classification benchmarks) cannot reflect the complexity of interactions. This invention provides an accurate and efficient method for detecting fallacy awareness in large-scale models in dynamic interactive scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a dynamic interactive large-scale model fallacy awareness detection method provided in an embodiment of the present invention;

[0025] Figure 2 This is a block diagram of a dynamic interactive large-scale model fallacy awareness detection device provided in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a large model fallacy awareness detection device provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0028] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0029] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0030] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0031] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0032] This invention provides a dynamic, interactive method for detecting large model fallacies awareness. This method can be implemented using a large model fallacy awareness detection device, which can be a terminal or a server. Figure 1 The flowchart shown is for a dynamic interactive large-scale model fallacy awareness detection method. The processing flow of this method may include the following steps:

[0033] S1. Obtain a test dataset containing erroneous dialogues.

[0034] In one feasible implementation, this invention employs a two-stage fallacy awareness test framework (FATE) based on reaction and decision causes to investigate whether LLMs can recognize fallacies without fallacy-related prompts such as "whether it contains a fallacy / what type of fallacy it contains." The study also examines the degree to which the large model recognizes the fallacy, and what factors influence the large model's ability to perceive fallacies.

[0035] In this step, a test dataset containing fallacious dialogues and fallacy judgment scenario information are used. In the test dataset, each data entry contains 11 fields: scenario description, dialogue (the last person's statement contains a fallacy), fallacy type, social interaction type, topic, fallacy user, misled target, fallacy analysis, misled target's decision after being misled, role relationship, and personality traits.

[0036] The scenario description and dialogue constitute the basic test scenario; the fallacy type refers to the specific type of fallacy that appears in the scenario, which includes six common types: appeal to emotion, rash generalization, ad hominem appeal, scarecrow fallacy, red herring fallacy, and false dilemma reasoning.

[0037] Social interaction type refers to the form in which individuals influence each other's behavior and attitudes through communication in this scenario, including five interaction types: exchange, cooperation, competition, conflict, and compromise; topic refers to the specific topic around which the dialogue in this scenario revolves; fallacy user refers to the person whose statement in this scenario contains a fallacy; misleading target refers to the person that the fallacy user wants to persuade or influence, which is also the role played by the large model in the subsequent fallacy awareness test.

[0038] Fallacy analysis is the analysis of fallacies that appear in a scenario; the decision made by the misled target after being misled refers to the specific behavior or decision made by the misled target in the scenario after being misled by the fallacy; role relationship refers to the relationship between the misled target and the fallacy user, including five role relationships: family relationship, friend relationship, stranger relationship, hierarchical relationship - high and hierarchical relationship - low; personality traits refer to the specific personality trait type of the misled target, including four opposing types: rational and emotional, open and conservative, critical and compliant, confident and insecure.

[0039] The following is an example of the test dataset used:

[0040]

[0041] S2. Based on the discriminative prompt text, and according to the test dataset, the fallacy awareness discrimination module is used to conduct the first stage fallacy awareness test on the large model to be detected, and obtain the natural response dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first fallacy not awareness dataset, the first indeterminate dataset, and the first fallacy uncertainty dataset.

[0042] In one feasible implementation, in order to more accurately assess the fallacy awareness of LLMs, clues about fallacies (such as whether a fallacy is contained / what type of fallacy is contained) should not be disclosed during the assessment process to help the large model detect fallacies in the scenario.

[0043] Accurately observing and assessing the fallacy awareness ability of LLMs (Limited Management Models) without any fallacy-related prompts is a key technical challenge in achieving the technical objectives. To address this, this invention employs a "fallacy awareness questioning strategy," including natural response methods, closed-ended questioning, and open-ended questioning. For each test scenario, the evaluation process is divided into two phases: direct response testing and decision-making reasoning testing. In each phase, the behavior of LLMs in the fallacy scenario is tested sequentially, and their responses are used to determine whether the LLMs are aware of the fallacy.

[0044] The fallacy awareness detection module is constructed using GPT-4 with the LLM-as-Judge method; the fallacy awareness detection module is used to determine whether there is a clear fallacy awareness in the current response of the large model to be detected.

[0045] In one feasible implementation, this invention proposes a Fallacy Awareness Identification Module (FAIM) to determine whether the input text in a specific interaction scenario reflects awareness of a fallacy. This mainly involves determining whether the text falls under the category of "awareness."

[0046] The fallacy awareness detection module employs an automated semantic evaluation mechanism using the LLM-as-Judge method. It calls the gpt-4-0125-preview model, which acts as a reviewer by judging the responses generated by each tested model in the fallacy awareness task based on the judgment prompt text. This determines whether the responses demonstrate fallacy awareness, achieving stable and consistent automated detection on large-scale data. Furthermore, the Kappa value between the automated evaluator and the human score is 0.773, falling within the good agreement range (0.61-0.8), indicating that the automated evaluation method has good reliability in the fallacy awareness task. The judgment prompt text is shown below; the content in brackets [] should be replaced with specific content from the actual scenario.

[0047]

[0048] A fallacy is a flawed line of reasoning. It is not simply a false, erroneous, or absurd belief, proposition, or argument, but rather a logical error or flawed method of argumentation that occurs during the reasoning process, leading to a conclusion that is invalid or cannot be reasonably derived from the premises. The fallacy awareness detection module defines the types of fallacies as follows:

[0049] "Appeal to emotion": "An argumentative approach that seeks to win an argument by manipulating people's emotions rather than by sound logic. It attempts to influence people's judgment by evoking emotional responses such as fear, sympathy, or anger, rather than by basing it on facts or rational reasoning."

[0050] "Rash generalization": refers to drawing inductive conclusions based on insufficient or unrepresentative examples or samples without fully considering general situations.

[0051] "Ad hominem" refers to using personal traits unrelated to the current topic, such as personality, motives, attitudes, status, class, or circumstances, as grounds for refuting the opponent's argument or supporting one's own. It attempts to weaken the validity of the argument by attacking the speaker rather than the point itself.

[0052] "Straw Man Fallacy": "A method of argumentation that misinterprets the opponent's argument, attacks the misinterpreted argument (a substitute straw man, usually a simplified or extreme version), and then claims to have overturned the opponent's original argument."

[0053] The "red herring fallacy": "A strategy of diverting attention from the core issue by introducing an unrelated topic or argument, thus shifting the focus of the discussion."

[0054] "False Dilemma Reasoning": "Presenting a few options (usually two, but possibly three or more) and asking you to choose one, but in reality these options do not cover all possibilities, ignoring other possible choices or middle ground, simplifying a complex problem into an either-or situation."

[0055] For each type of fallacy, examples of both positive and negative awareness of the fallacy are as follows:

[0056] "Appeal to Emotion": Example: Argument: "Imagine the suffering of those innocent animals if we don't act immediately to protect this forest!" Response 1: "While the animals are pitiful, I need to consider providing concrete scientific evidence or rational arguments for protecting the forest." This response highlights the manifestation of the appeal to emotion fallacy in this scenario, illustrating its characteristic of influencing the audience by manipulating emotions rather than logic. Response 2: "This argument emphasizes the importance of protecting animals and demonstrates the necessity of protecting the forest." This response fails to highlight the manifestation of the appeal to emotion fallacy in this scenario; it does not mention how to support the argument by evoking emotions rather than providing rational evidence.

[0057] "Presumptuous generalization": Example: Argument: "I had terrible service twice at restaurants in this city, and all restaurants here have terrible service." Response 1: "It infers that all restaurants in the city have terrible service based solely on two bad experiences. This inference doesn't consider a larger sample size or other possible scenarios." This response highlights the manifestation of the presumptuous generalization fallacy in this scenario, illustrating its characteristic of drawing broad conclusions from insufficient and unrepresentative samples. Response 2: "It seems you have a very bad impression of the restaurant service in this city." This response fails to highlight the manifestation of the presumptuous generalization fallacy in this scenario; it merely reflects agreement with the argumenter's impression without recognizing the error of drawing broad conclusions based on an insufficient sample.

[0058] "Ad hominem fallacy": Example: Argument: "Xiaoming made a serious mistake, so his views on environmental protection are not trustworthy." Response 1: "Xiaoming's mistake is irrelevant to the correctness of his views on environmental protection." This response highlights the manifestation of the ad hominem fallacy in the context, demonstrating how it refutes or questions Xiaoming's viewpoint by mentioning his personal situation (i.e., his mistake). Response 2: "Everyone has the opportunity to correct their mistakes and learn; I think Xiaoming should be given that opportunity." This response fails to highlight the manifestation of the ad hominem fallacy in the context; it merely states a moral viewpoint without demonstrating how it refutes or supports an argument by mentioning a personal trait unrelated to the topic.

[0059] "The Scarecrow Fallacy": Example: Argument: A: "I think we should appropriately reduce the amount of homework for primary and secondary school students, allowing them more time for independent exploration and rest." B: "Do you want children to only play after school and not learn anything? Without strict homework management, students can easily neglect their studies." Response 1: A: "I am not against learning, much less say 'let children only play and not learn after school.' I advocate 'appropriately reducing' the amount of homework so that students can have more diverse development space. You have misunderstood my original intention." This response highlights the manifestation of the scarecrow fallacy in the scenario, demonstrating that it misinterprets the original argument "appropriately reducing homework" as "not learning at all," and attacks the characteristics of this misinterpreted argument; Response 2: A: "You are right. It is true that some children nowadays lack self-control and may really spend too much time playing." "This answer fails to pinpoint how the scarecrow fallacy manifests in the context. It agrees with B's viewpoint but fails to recognize that B distorts the original argument and attacks a false surrogate argument."

[0060] "Red Herring Fallacy": Example: Argument: A: "We should reduce plastic use to reduce ocean pollution." B: "But we must focus on economic growth; that's the most important thing." Response 1: A: "Economic growth is important, but we're discussing ocean pollution right now. Let's solve that problem first." This response highlights the manifestation of the red herring fallacy in the scenario, showing that B shifted the focus to economic growth instead of addressing how to reduce ocean pollution, thus bringing the discussion back to the original topic. Response 2: A: "You raised an important point; economic growth does indeed need attention." This response fails to highlight the manifestation of the red herring fallacy in the scenario; it doesn't recognize that B shifted the focus of the original argument and didn't address how to reduce ocean pollution.

[0061] "False Dilemma Reasoning": Example: Argument: "We must either ban all self-driving cars or completely ignore their safety issues." Response 1: "You've only presented two extreme options, but in reality, we can have strict regulations while still allowing the development of self-driving cars." This response highlights the fallacy of false dilemma reasoning in this scenario, demonstrating that the argumentant's options do not cover all possibilities and offering a middle ground solution. Response 2: "You're right, safety is indeed important." This response fails to highlight the fallacy of false dilemma reasoning in this scenario, simply agreeing with the argumentant's point while ignoring the middle ground or other solutions.

[0062] Optionally, based on the discriminative prompt text and according to the test dataset, the fallacy awareness discrimination module performs a first-stage fallacy awareness test on the large model to be detected, obtaining a natural response dataset, a dataset of first realized fallacy errors, a dataset of first unrecognized fallacy errors, a dataset of first indeterminate fallacy errors, and a dataset of first fallacy uncertainty, including:

[0063] Based on the test dataset, a direct response test is performed on the large model to be detected to obtain the natural response dataset of the large model to be detected.

[0064] Based on the discriminative prompt text, and using the natural response dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the first fallacy awareness dataset of the large model to be detected in relation to the natural response dataset.

[0065] Based on the fallacy awareness status of the first fallacy awareness status dataset, the first fallacy awareness status dataset is filtered and divided to obtain the first fallacy awareness dataset, the first fallacy non-awareness dataset, the first indeterminate dataset, and the first fallacy uncertain dataset.

[0066] In one feasible implementation, during the first phase of testing, the invention employs a natural response method, placing LLMs in a predetermined scenario where they act as specific roles interacting with erroneous discourse. This setup is designed to simulate how LLMs would react to fallacies encountered in real-world applications.

[0067] In a multi-turn dialogue interaction scenario, given a specific scenario description D, a utterance d containing a specific fallacy, and a model M, a response R=M(d;D) can be obtained. The fallacy awareness status s=FAIM(R) of model M is obtained through the fallacy awareness recognition module.

[0068] In the two-stage test, the test results include four categories: aware, unaware, unable to judge, and uncertain.

[0069] Awareness is defined as the model's response meeting one of the following three criteria:

[0070] In order to identify the existence of fallacies, a model can detect fallacies in the input. This usually requires the model to perform in-depth semantic analysis of the input data to identify expressions that do not conform to correct logic or argumentation, so as to assert that these inconsistencies and errors constitute substantial fallacies.

[0071] The model identifies the types of fallacies and can accurately classify the types of fallacies in the input. This involves a deep understanding and ability to identify the characteristics of different fallacy types. The model can match observed fallacies with specific erroneous reasoning patterns, providing more specific diagnostic information.

[0072] This involves identifying the specific manifestations of the fallacy's erroneous logical paradigm within a given scenario, i.e., the depth of awareness score for the scenario in which the fallacy is recognized. The model must be able to describe in detail how the fallacy manifests in a specific context, requiring a deep understanding of the scenario and the ability to connect the fallacy's manifestations to flawed reasoning.

[0073] Unawareness: LLMs do not demonstrate a clear awareness of the fallacy, including the following four situations:

[0074] Misled by fallacies, the model is guided by incorrect arguments or information, fails to recognize the errors, and accepts erroneous arguments, leading it to base its conclusions or actions on incorrect information. This indicates a deficiency in the model's ability to identify fallacies.

[0075] Simply adhering to existing viewpoints, a model clings to its preconceived knowledge or beliefs, ignoring or rejecting conflicting information, and failing to critically analyze fallacies. This reflects a confirmation bias in the model, where it tends to support existing knowledge frameworks rather than critically analyze new information.

[0076] Simply considering the consistency between the model's own knowledge and arguments, when processing input, the model relies on checking whether the information matches its pre-trained knowledge base; it does not perform appropriate reasoning for new or complex logical forms. This indicates that the model may lack flexibility or adaptability, failing to adapt to new scenarios or complex logical structures that are significantly different from the training data.

[0077] Semantic misunderstanding occurs when the model fails to accurately understand or interpret the true semantic content of the input data, leading to misinterpretations or responses. This typically happens when processing ambiguous, highly ambiguous, or context-dependent statements. This may be due to insufficient natural language processing capabilities of the model, making it unable to handle ambiguous, metaphorical, or complex context-dependent expressions. The result is a response to the literal meaning only, failing to grasp the implied meaning or deeper implications within the context.

[0078] Unable to Determine: Due to built-in ethical and safety mechanisms, the model refuses to express opinions on sensitive topics. This occurs when the model deems a scenario unsafe, restricting its output and making it impossible to judge the model's fallacy awareness from its responses. Such situations should be avoided as much as possible during data construction and screening.

[0079] Uncertainty does not fall into any of the above categories. It applies when there is insufficient information to determine whether the respondent is truly aware of the fallacy. A clear judgment needs to be made by obtaining more information or through further dialogue.

[0080] S3. Based on the interviewer's prompt text and the discrimination prompt text, according to the test dataset, the natural response dataset and the first fallacy uncertainty dataset, the fallacy awareness discrimination module is used to conduct a second-stage fallacy awareness test on the large model to be detected, and obtain the second-aware fallacy dataset, the second-unaware fallacy dataset and the second-undeterminable dataset.

[0081] Optionally, based on the interviewer prompt text and the discrimination prompt text, and according to the test dataset, the natural response dataset, and the first fallacy uncertainty dataset, a second-stage fallacy awareness test is conducted on the large model to be detected through the fallacy awareness discrimination module to obtain a second-awareness fallacy dataset, a second-unawareness fallacy dataset, and a second-undeterminable dataset, including:

[0082] Based on the interviewer's prompts, and using the test dataset, natural response dataset, and first fallacy uncertainty dataset, a decision cause test is performed on the large model to be tested to obtain the decision cause dataset of the large model to be tested.

[0083] Based on the discriminative prompt text, and according to the decision cause dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the second fallacy awareness dataset of the large model to be detected in relation to the decision cause dataset.

[0084] Based on the fallacy awareness status of the second fallacy awareness dataset, the second fallacy awareness dataset is filtered and divided to obtain the second fallacy awareness dataset, the second fallacy awareness dataset, and the second inability to judge dataset.

[0085] In one feasible implementation, since no restrictions are placed on the LLMs' responses in the Phase 1 direct response test, the LLMs' natural responses may not involve explicit views of fallacies, making it impossible to determine the model's fallacy awareness at this point.

[0086] To address this issue, in the Phase 2 Decision Cause Test, a new interviewer role was introduced based on the interviewer's prompts, and closed-ended and open-ended questioning techniques were used to follow up with LLMs. The interviewer was positioned as a close friend with whom the model's role had no secrets, and the model completely trusted the interviewer, sharing its true thoughts without reservation.

[0087] After the second phase of testing, LLM still did not realize the fallacy and attributed its awareness result to a lack of awareness.

[0088] S4. Based on the fallacy awareness evaluation index, the fallacy awareness test results of the large model are obtained by calculating the test dataset, the dataset of first-time fallacy awareness, the dataset of first-time fallacy non-awareness, the dataset of first-time inability to judge, the dataset of second-time fallacy awareness, the dataset of second-time fallacy non-awareness, and the dataset of second-time inability to judge.

[0089] Among them, the fallacy awareness evaluation indicators include the fallacy awareness rate and the awareness response depth index.

[0090] In a feasible implementation, a set of scientific and reasonable evaluation metrics is needed to systematically evaluate the performance of large models on the fallacy awareness task. These metrics should not only be able to quantify the model's performance but also reflect its ability differences across different dimensions. To this end, two core evaluation metrics are proposed: Fallacy Awareness Rate (FAR) and Awareness Response Depth Index (RDI).

[0091] Among them, the fallacy awareness rate refers to the proportion of fallacies in the test data that the large model recognizes;

[0092] The formula for calculating the fallacy awareness rate is as follows (1):

[0093] (1);

[0094] in, It is to recognize the total number of erroneous data; This is the total number of test data; It is impossible to determine the total amount of data.

[0095] In one feasible implementation, the fallacy awareness rate refers to the proportion of scenarios in which the model can recognize fallacies across all identifiable test scenarios. Its design aims to quantify the basic fallacy awareness capability of large language models, primarily focusing on whether the model can recognize the existence of fallacies. In dynamic interactions, models may encounter various types of fallacies; the model's ability to accurately identify and demonstrate awareness of fallacies is fundamental to measuring fallacy awareness capability. The fallacy awareness rate ranges from [0,1], with values ​​closer to 1 indicating a stronger ability to identify fallacies in interactive scenarios and better fallacy awareness performance.

[0096] Among them, the Consciousness Response Depth Index is a weighted calculation index based on the awareness of fallacies according to the large model;

[0097] The formula for calculating the depth of consciousness response index is as follows (2):

[0098] (2);

[0099] Among them, D i It is the consciousness depth score of the i-th scenario where the error is realized; the consciousness depth score is obtained based on the first and second sets of datasets where the error is realized.

[0100] In one feasible implementation, it is insufficient to merely measure whether the model is aware of the fallacy; it is also necessary to further assess the depth of the model's awareness of the fallacy. The awareness response depth index is a weighted metric calculated based on the model's specific performance when it becomes aware of the fallacy.

[0101] This invention identifies three scenarios of fallacy awareness: pointing out the existence of the fallacy, pointing out the type of the fallacy, and pointing out the flawed logical paradigm of the fallacy. These scenarios are assigned scores of 1, 2, and 3 respectively based on their specific performance in a given scenario. Only the highest score is recorded for each scenario to avoid duplicate scoring for the same scenario. The awareness response depth index ranges from [1, 3], with a higher value indicating a deeper understanding of the fallacy and providing more accurate and detailed analysis.

[0102] This invention proposes a method for detecting fallacy awareness in large-scale models (LLMs) in dynamic interactive scenarios. Through a two-stage testing framework and corresponding evaluation metrics, it comprehensively assesses the fallacy awareness capabilities of LLMs in dynamic interactive scenarios. Utilizing three strategies—natural response, closed-ended questioning, and open-ended questioning—it evaluates both the model's initial response and uncovers its deeper fallacy awareness capabilities. By using the fallacy awareness rate and the awareness response depth index, it quantifies the fallacy awareness capability in dynamic interactive scenarios into comparable numerical indicators, solving the problem that existing evaluation systems (such as static classification benchmarks) cannot reflect the complexity of interactions. This invention provides an accurate and efficient method for detecting fallacy awareness in large-scale models in dynamic interactive scenarios.

[0103] Figure 2 This is a block diagram illustrating a dynamically interactive large model fallacy awareness detection device according to an exemplary embodiment. The device is used in a dynamically interactive large model fallacy awareness detection method. (Refer to...) Figure 2 The device includes an information acquisition module 310, a first-stage testing module 320, a second-stage testing module 330, and a fallacy awareness evaluation module 340. Among them:

[0104] Information acquisition module 310 is used to acquire a test dataset containing erroneous dialogues;

[0105] The first-stage test module 320 is used to perform a first-stage fallacy awareness test on the large model to be detected based on the discriminative prompt text and the test dataset, through the fallacy awareness discrimination module, to obtain the natural response dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first fallacy not awareness dataset, the first indeterminate dataset, and the first fallacy uncertain dataset.

[0106] The second-stage test module 330 is used to conduct a second-stage fallacy awareness test on the large model to be detected based on the interviewer prompt text and the discrimination prompt text, according to the test dataset, the natural response dataset and the first fallacy uncertainty dataset, and the fallacy awareness discrimination module to obtain the second-stage fallacy awareness dataset, the second-stage non-awareness fallacy dataset and the second-stage indeterminate dataset.

[0107] The fallacy awareness evaluation module 340 is used to calculate the large model fallacy awareness test results based on the fallacy awareness evaluation index, according to the test dataset, the first fallacy awareness dataset, the first fallacy non-awareness dataset, the first inability to judge dataset, the second fallacy awareness dataset, the second non-awareness dataset, and the second inability to judge dataset.

[0108] The fallacy awareness detection module is constructed using GPT-4 with the LLM-as-Judge method; the fallacy awareness detection module is used to determine whether there is a clear fallacy awareness in the current response of the large model to be detected.

[0109] Optionally, the first-stage test module 320 is further used for:

[0110] Based on the test dataset, a direct response test is performed on the large model to be detected to obtain the natural response dataset of the large model to be detected.

[0111] Based on the discriminative prompt text, and using the natural response dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the first fallacy awareness dataset of the large model to be detected in relation to the natural response dataset.

[0112] Based on the fallacy awareness status of the first fallacy awareness status dataset, the first fallacy awareness status dataset is filtered and divided to obtain the first fallacy awareness dataset, the first fallacy non-awareness dataset, the first indeterminate dataset, and the first fallacy uncertain dataset.

[0113] Optionally, the second-stage test module 330 is further used for:

[0114] Based on the interviewer's prompts, and using the test dataset, natural response dataset, and first fallacy uncertainty dataset, a decision cause test is performed on the large model to be tested to obtain the decision cause dataset of the large model to be tested.

[0115] Based on the discriminative prompt text, and according to the decision cause dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the second fallacy awareness dataset of the large model to be detected in relation to the decision cause dataset.

[0116] Based on the fallacy awareness status of the second fallacy awareness dataset, the second fallacy awareness dataset is filtered and divided to obtain the second fallacy awareness dataset, the second fallacy awareness dataset, and the second inability to judge dataset.

[0117] Among them, the fallacy awareness evaluation indicators include the fallacy awareness rate and the awareness response depth index.

[0118] Among them, the fallacy awareness rate refers to the proportion of fallacies in the test data that the large model recognizes;

[0119] The formula for calculating the fallacy awareness rate is as follows (1):

[0120] (1);

[0121] in, It is to recognize the total number of erroneous data; This is the total number of test data; It is impossible to determine the total amount of data.

[0122] Among them, the Consciousness Response Depth Index is a weighted calculation index based on the awareness of fallacies according to the large model;

[0123] The formula for calculating the depth of consciousness response index is as follows (2):

[0124] (2);

[0125] Among them, D i It is the consciousness depth score of the i-th scenario where the error is realized; the consciousness depth score is obtained based on the first and second sets of datasets where the error is realized.

[0126] This invention proposes a method for detecting fallacy awareness in large-scale models (LLMs) in dynamic interactive scenarios. Through a two-stage testing framework and corresponding evaluation metrics, it comprehensively assesses the fallacy awareness capabilities of LLMs in dynamic interactive scenarios. Utilizing three strategies—natural response, closed-ended questioning, and open-ended questioning—it evaluates both the model's initial response and uncovers its deeper fallacy awareness capabilities. By using the fallacy awareness rate and the awareness response depth index, it quantifies the fallacy awareness capability in dynamic interactive scenarios into comparable numerical indicators, solving the problem that existing evaluation systems (such as static classification benchmarks) cannot reflect the complexity of interactions. This invention provides an accurate and efficient method for detecting fallacy awareness in large-scale models in dynamic interactive scenarios.

[0127] Figure 3 This is a schematic diagram of the structure of a large-scale fallacy awareness detection device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, a large-scale fallacy awareness detection device may include the above-mentioned... Figure 2 The large model fallacy awareness detection device shown is a dynamic interactive device. Optionally, the large model fallacy awareness detection device 310 may include a first processor 2001.

[0128] Optionally, the large model fallacy awareness detection device 310 may also include a memory 2002 and a transceiver 2003.

[0129] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0130] The following is combined Figure 3 A detailed description of each component of the large model fallacy awareness detection device 310 is provided below:

[0131] The first processor 2001 is the control center of the large model fallacy awareness detection device 310. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0132] Optionally, the first processor 2001 can perform various functions of the large model fallacy awareness detection device 310 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0133] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0134] In a specific implementation, as one example, the large model fallacy awareness detection device 310 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0135] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0136] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the large model fallacy awareness detection device 310. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0137] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0138] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0139] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the large model fallacy awareness detection device 310. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0140] It should be noted that, Figure 3 The structure of the large model fallacy awareness detection device 310 shown in the diagram does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0141] Furthermore, the technical effect of the large model fallacy awareness detection device 310 can be referred to the technical effect of the dynamic interactive large model fallacy awareness detection method described in the above method embodiments, and will not be repeated here.

[0142] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0144] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0145] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0146] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0147] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0153] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A dynamic interactive large-scale model fallacy awareness detection method, characterized in that, The method includes: Obtain a test dataset containing erroneous dialogues; Based on the discriminative prompt text and the test dataset, the fallacy awareness discrimination module performs a first-stage fallacy awareness test on the large model to be detected, obtaining the following datasets: natural response dataset, first-time fallacy awareness dataset, first-time fallacy unawareness dataset, first-time undetermined dataset, and first-time fallacy uncertainty dataset. Based on the test dataset, a direct response test is performed on the large model to be detected to obtain the natural response dataset of the large model to be detected. Based on the discriminative prompt text, and using the natural response dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the first fallacy awareness dataset of the large model to be detected in relation to the natural response dataset. Based on the fallacy awareness status of the first fallacy awareness status dataset, the first fallacy awareness status dataset is filtered and divided to obtain the first fallacy awareness dataset, the first fallacy non-awareness dataset, the first indeterminate dataset, and the first fallacy uncertain dataset. Based on the interviewer's prompts and the discrimination prompts, and according to the test dataset, the natural response dataset, and the first fallacy uncertainty dataset, the fallacy awareness discrimination module performs a second-stage fallacy awareness test on the large model to be detected, obtaining a second-awareness fallacy dataset, a second-unawareness fallacy dataset, and a second-undeterminable dataset, including: Based on the interviewer's prompts, and using the test dataset, natural response dataset, and first fallacy uncertainty dataset, a decision cause test is performed on the large model to be tested to obtain the decision cause dataset of the large model to be tested. Based on the discriminative prompt text, and according to the decision cause dataset, the fallacy awareness discrimination module performs analysis and reasoning to obtain the second fallacy awareness dataset of the large model to be detected in relation to the decision cause dataset. Based on the fallacy awareness status of the second fallacy awareness dataset, the second fallacy awareness dataset is filtered and divided to obtain the second fallacy awareness dataset, the second fallacy awareness dataset, and the second fallacy indeterminate dataset. Based on the fallacy awareness evaluation index, the fallacy awareness test results of the large model are obtained by calculating the test dataset, the dataset of first-time fallacy awareness, the dataset of first-time fallacy non-awareness, the dataset of first-time inability to judge, the dataset of second-time fallacy awareness, the dataset of second-time fallacy non-awareness, and the dataset of second-time inability to judge.

2. The dynamic interactive large-scale model fallacy awareness detection method according to claim 1, characterized in that, The fallacy awareness discrimination module is constructed using GPT-4 with the LLM-as-Judge method; the fallacy awareness discrimination module is used to determine whether there is a clear fallacy awareness in the current response of the large model to be detected.

3. The dynamic interactive large-scale model fallacy awareness detection method according to claim 1, characterized in that, The fallacy awareness evaluation indicators include the fallacy awareness rate and the awareness response depth index.

4. The dynamic interactive large-scale model fallacy awareness detection method according to claim 3, characterized in that, The fallacy awareness rate refers to the proportion of erroneous dialogues in the test data that the large model is aware of. The formula for calculating the fallacy awareness rate is as follows (1): （1）； in, It is to recognize the total number of erroneous data; This is the total number of test data; It is impossible to determine the total amount of data.

5. The dynamic interactive large-scale model fallacy awareness detection method according to claim 4, characterized in that, The consciousness response depth index refers to a weighted calculation index based on the awareness of fallacies in the large model; The formula for calculating the depth of consciousness response index is as follows (2): （2）； Among them, D i It is the consciousness depth score of the i-th scenario where the error is realized; the consciousness depth score is obtained based on the first and second datasets of the scenario where the error is realized.

6. A dynamic interactive large model fallacy awareness detection device, wherein the dynamic interactive large model fallacy awareness detection device is used to implement the dynamic interactive large model fallacy awareness detection method as described in any one of claims 1-5, characterized in that, The device includes: The information acquisition module is used to acquire a test dataset containing erroneous dialogues; The first-stage testing module is used to conduct a first-stage fallacy awareness test on the large model to be detected based on the discriminative prompt text and the test dataset, through the fallacy awareness discrimination module, to obtain the natural response dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first fallacy awareness dataset, the first undetermined dataset, and the first fallacy uncertainty dataset. The second-stage testing module is used to conduct a second-stage fallacy awareness test on the large model to be detected based on the interviewer prompt text and the discrimination prompt text, according to the test dataset, the natural response dataset and the first fallacy uncertainty dataset, and the fallacy awareness discrimination module to obtain the second-stage fallacy awareness dataset, the second-stage non-awareness fallacy dataset and the second-stage indeterminate dataset. The fallacy awareness evaluation module is used to calculate the fallacy awareness test results of a large model based on the fallacy awareness evaluation index, according to the test dataset, the dataset of first awareness of fallacy, the dataset of first non-awareness of fallacy, the dataset of first inability to judge, the dataset of second awareness of fallacy, the dataset of second non-awareness of fallacy, and the dataset of second inability to judge.

7. A large-scale fallacy awareness detection device, characterized in that, The large-scale fallacy awareness detection device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 5.

Images