Multi-field false information detection method and system based on fact information mode and large language model

Abstract

The invention belongs to the field of false information detection, and discloses a multi-field false information detection method and system based on a fact information mode and a large language model.The method comprises the steps that firstly, false information is classified, the false information is classified into a specific field, and related information is retrieved only in the specific field to serve as evidence to be input into a recognition model; and a false information identification result is obtained. By comparing the recognition effects of an open source large model before and after the method is introduced, the multi-field false information recognition effect of the large language model can be effectively improved, the problem of unbalanced knowledge field distribution of the large language model is relieved, and the large language model can better complete a false information recognition task.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of false information detection, and particularly relates to a multi-field false information detection method and system based on a fact information mode and a large language model. BACKGROUND

[0002] In recent years, short video platforms represented by Douyin and Tiktok have developed rapidly, attracting a large number of users, and also giving birth to a large number of self-media that publish works on short video platforms. These self-media are distributed all over the world and publish a large number of works, which have subverted the form of obtaining information by people. With the help of the recommendation algorithm of short videos, users can know about an event at the time of the event or within a very short time, which helps them to obtain information instantly and efficiently. However, the quality of these information is uneven, and many of them are false information deliberately fabricated by self-media to attract eyeballs and attract traffic. Moreover, these information involves various fields, which seriously affects normal production and life and social order. Therefore, in a specific field, it is very important to use artificial intelligence technology to identify these false information and further process the false information for maintaining the clear and safe cyberspace and normal social order.

[0003] In a specific field, there are generally two routes for identifying false information, a multi-field false information detection method based on features and a multi-field false information detection method based on fact information. The former mainly trains a model by using the unique features of each field and the features such as transmission and text of false information to identify false information. This technology has achieved high accuracy in the binary classification task of false information, but the experimental results show that the model has achieved high accuracy, but the accuracy rate of different fields is quite different, and there is a problem of obvious uneven distribution of accuracy rate in different fields. The latter mainly combines specific fact information in each field with the false information to be identified. If there is a contradiction between the fact information in the field and the information to be identified, it is judged that the information to be identified is false information. This method is more in line with the identification logic of artificial false information checkers, and compared with the multi-field false information identification method based on features, the phenomenon of uneven distribution of accuracy rate in different fields is significantly reduced. However, in the previous method, the model used is a pre-training language model represented by BERT, which has a parameter quantity of about 3 billion and weak logical thinking and reasoning ability. The parameter quantity of a large language model reaches hundreds of billions, which has strong logical thinking and reasoning ability and the ability to complete the task of identifying false information according to fact information. At the same time, providing fact information to the large language model can alleviate the problem of uneven distribution of knowledge fields of the large language model, so that the large language model can better complete the task of identifying false information.

[0004] In view of the above analysis, the technical problems existing in the prior art that need to be solved urgently are:

[0005] (1) Model logic thinking and reasoning ability is weak;

[0006] In the past false information recognition model, the model uses small language model, compared with the existing large language model, the parameter quantity and TB level pre-training data set of small language model has 1-3 orders of magnitude difference compared with large language model, so in the complex logical task of false information recognition, due to the lack of strong enough logical reasoning ability of small language model, it is difficult to understand the retrieved evidence and accurately identify false information

[0007] (2) Knowledge field distribution is uneven.

[0008] The key reasons affecting the accuracy and efficiency of false information recognition model in identifying false information are 1. The knowledge mastered by the model has the characteristics of uneven distribution of knowledge in different fields, which leads to the fact that the accuracy of identifying false information in the field with more knowledge mastered by the model is significantly better than that in the field with less knowledge mastered by the model. 2. The model cannot select and integrate correct reasons to identify false information. When the model obtains information related to false information as a reason, the model will classify the reason into the field with more knowledge mastered by the model instead of the field to which the reason belongs, and get the wrong identification result. SUMMARY

[0009] In view of the problems existing in the prior art, the present application provides a multi-field false information detection method and system based on fact information mode and large language model.

[0010] The present application is realized in this way, a multi-field false information detection method based on fact information mode and large language model, characterized in that the multi-field false information detection method based on fact information mode and large language model, the method first classifies false information, classifies the false information into a specific field, and only retrieves relevant information in the specific field as evidence input into the identification model to obtain the identification result of the false information, the specific steps include:

[0011] S1: classifying the information to be identified based on a large language model;

[0012] S2: extracting all documents of the field classified in step S1 from the knowledge base;

[0013] S3: using DRCV document retrieval algorithm to retrieve the top_k documents ranked from high to low in similarity with the information to be identified from the knowledge base obtained in step S2;

[0014] S4: constructing document-claim pairs with the top_k documents obtained in step S3 and the information to be identified;

[0015] S5: The document-statement pair obtained in step S4 Evidence retrieval is performed based on the keyword-attention mechanism, and the evidence retrieval results {E1, E2, ..., E top_k}∈E C ;

[0016] S6: Input the information to be recognized, the domain information obtained in step S1, and the evidence retrieval result in step S5 into the instruction-aligned large language model, and output the recognition result.

[0017] Furthermore, in S1, for the information to be identified C, the domain set {D1, D2, ..., D n}∈D, based on the large language model, classify the domain D closest to C from the domain set D i , and D i Add it to the end of the information to be identified C and output it together as a result to step S2.

[0018] Furthermore, the S2 is based on the domain result D obtained in step S1. i , extract all the knowledge base K belonging to the domain D i Documents containing only domain information D i The knowledge base K'.

[0019] Furthermore, in S4, for a piece of information to be identified C, the document set {A1, A2, ..., A top_k}∈A, the set of sentences in each document Construct the information to be identified C and document A i The document-declaration pair For {C,A i ,S1,S2,…,S n}.

[0020] Furthermore, the method for constructing the instruction-aligned large language model in S6 is as follows:

[0021] (1) Different training datasets are constructed based on the different instructions used by the large language model aligned with domain classification and the large language model aligned with recognition information for fine-tuning;

[0022] (2) Fine-tune the large model based on the LoRA method;

[0023] (3) After fine-tuning, the model is output and the effect of the model is evaluated. If the model achieves the expected effect, the model is output; if the model does not achieve the expected effect, return to step (2) and fine-tune the large model again;

[0024] (4) Use the fine-tuned model from step (3) to complete the corresponding task.

[0025] Another purpose of the present application is to provide a multi-field false information detection system based on a fact information mode and a large language model, which specifically comprises:

[0026] The field-balanced document retrieval and evidence retrieval model classifies the to-be-identified information into a specific field based on a large language model, the document retrieval step only retrieves documents consistent with the classified field from the knowledge base, ensuring the consistency of the document field and the to-be-identified information field; after the document retrieval, the document and the to-be-identified information are constructed into a document-claim pair for evidence retrieval and evidence is obtained;

[0027] The large language model after instruction alignment is performed, the instruction alignment dataset is constructed to fine-tune the large language model, and it is ensured that the large language model can correctly respond to the input instruction.

[0028] In combination with the above technical solutions and the technical problems solved, the technical solution to be protected by the present application has the following advantages and positive effects:

[0029] By comparing the recognition effects of the open source large model before and after the introduction of the present application, the present application can effectively improve the effect of the large language model in identifying false information in multiple fields, alleviate the problem of uneven distribution of knowledge fields of the large language model, and enable the large language model to better complete the false information identification task.

[0030] The present application focuses on multi-field false information identification in public emergencies, and develops a multi-field false information identification system in public emergencies. The purpose is to solve the core problems of low efficiency of traditional manual verification and insufficient cross-field recognition ability of single-field systems. The innovation of this system lies in the use of a false information identification mode based on fact information, which provides an efficient and accurate solution for false information identification through the deep integration of automation technology and large language models. This system breaks through the limitations of traditional methods and can receive false information in any field and retrieve related documents from a multi-field knowledge base. This design relies on the multi-field false information detection method based on the fact information mode and the large language model proposed in chapters 3 and 4 of this paper, ensuring that false information in different fields can be responded to and identified at any time. In the future, with the continuous development of large language model technology, this system can be further optimized and play a role in a wider range of scenarios, continuously contributing to the construction of a safe and trustworthy digital society.

[0031] The application develops a multi-field false information discovery system in a public emergency event, which accepts false information in any field, retrieves documents related to the false information in the same field from a knowledge base, retrieves evidence proving the false information from the documents, inputs the evidence and the false information into a multi-field false information recognition model based on a fact information mode and a large language model, and obtains a recognition result of the false information.

[0032] In a specific field, the false information is identified using artificial intelligence technology, and the false information is further processed, which is crucial for maintaining a clear and safe cyberspace and normal social order. However, existing false information recognition models focus on recognizing false information in a certain field, ignoring the need for false information recognition models to have the ability to recognize false information in different fields to show the universality of the recognition model. The application optimizes the false information recognition model, alleviates the problem of large model recognition accuracy in different fields, and improves the recognition accuracy of the false information recognition model in each field. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a multi-field false information detection method flowchart based on a fact information mode and a large language model provided by an embodiment of the application;

[0034] Figure 2 is a flowchart of a large language model with instruction alignment provided by an embodiment of the application;

[0035] Figure 3 is an architecture diagram of a multi-field false information detection method based on a fact information mode and a large language model provided by an embodiment of the application;

[0036] Figure 4 is a ChatGLM3 fine-tuning loss function change diagram provided by an embodiment of the application;

[0037] Figure 5 is a main page diagram of a QWEN1.5-7B large model provided by an embodiment of the application;

[0038] Figure 6 is a false information recognition page diagram using a QWEN1.5-7B large model provided by an embodiment of the application;

[0039] Figure 7 is a default false information recognition page diagram using a QWEN1.5-14B-GPTQ-INT8 large model provided by an embodiment of the application;

[0040] Figure 8 is a recognition result page figure of identifying false information using a QWEN1.5-7B large model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0042] As shown in Figure 1 A multi-field false information detection method based on a fact information mode and a large language model includes:

[0043] Step S1, classifying the to-be-identified information by field to obtain a target field;

[0044] Step S2, extracting all documents corresponding to the target field from a knowledge base to form a field knowledge base;

[0045] Step S3, retrieving top_k documents with the highest relevance to the to-be-identified information from the field knowledge base by using a document relevance calculation-sorting algorithm;

[0046] Step S4, constructing the top_k documents and the to-be-identified information into a document-claim pair;

[0047] Step S5, performing evidence retrieval on the document-claim pair based on a keyword-attention mechanism to obtain an evidence set;

[0048] Step S6, inputting the to-be-identified information, the target field, and the evidence set into a large language model with instruction alignment to output a false information recognition result.

[0049] An embodiment of the present application provides a multi-field false information detection method based on a fact information mode and a large language model. The method first classifies false information, divides the false information into a specific field, retrieves relevant information in the specific field as evidence input into a recognition model, and obtains a recognition result of the false information. The specific steps include:

[0050] S1: classifying to-be-identified information by field based on a large language model;

[0051] S2: extracting all documents of the field classified in step S1 from a knowledge base;

[0052] S3: retrieving top_k documents with the highest similarity ranking from top to bottom from the knowledge base obtained in step S2 by using a DRCV document retrieval algorithm;

[0053] S4: Construct document-claim pairs by combining the top_k documents obtained in step S3 with the information to be identified;

[0054] S5: The document-statement pair obtained in step S4 Evidence retrieval is performed based on the keyword-attention mechanism, and the evidence retrieval results {E1, E2, ..., E top_k}∈E C ;

[0055] S6: Input the information to be recognized, the domain information obtained in step S1, and the evidence retrieval result in step S5 into the instruction-aligned large language model, and output the recognition result.

[0056] Said S1, for the information to be identified C, the domain set {D1, D2, ..., D n}∈D, based on the large language model, classify the domain D closest to C from the domain set D i , and D i Add it to the end of the information to be identified C and output it together as a result to step S2.

[0057] S2, the domain result D obtained according to step S1 i , extract all the knowledge base K belonging to the domain D i Documents containing only domain information D i The knowledge base K'.

[0058] Said S4, in a to-be-identified information C, document set {A1, A2, ..., A top_k}∈A, the set of sentences in each document Construct the information to be identified C and document A i The document-declaration pair For {C,A i ,S1,S2,…,S n}.

[0059] like Figure 2 As shown, the S6 instruction-aligned large language model is constructed as follows:

[0060] (1) Different training datasets are constructed based on the different instructions used by the large language model aligned with domain classification and the large language model aligned with recognition information for fine-tuning;

[0061] (2) Fine-tune the large model based on the LoRA method;

[0062] (3) After fine-tuning, the model is output and the effect of the model is evaluated. If the model achieves the expected effect, the model is output; if the model does not achieve the expected effect, return to step (2) and fine-tune the large model again;

[0063] (4) using the fine-tuned model of step (3) to complete the corresponding task.

[0064] As Figure 3 indicated, the multi-field false information detection system based on the fact information mode and the large language model provided by the embodiment of the application specifically comprises:

[0065] The document retrieval and evidence retrieval model balanced in the field classify the to-be-identified information into specific fields such as geography and history based on the large language model. The document retrieval step only retrieves the documents consistent with the classified field from the knowledge base to ensure the consistency of the document field and the to-be-identified information field; after the document retrieval, the document and the to-be-identified information are used to construct a document-claim pair for evidence retrieval and obtain evidence;

[0066] The large language model after the instruction alignment, the instruction alignment dataset is used to fine-tune the large language model to ensure that the large language model can correctly respond to the input instruction.

[0067] The application selects the QWEN1.5-7B large model for demonstration, selects the QWEN1.5-7B large model from the model selection column, and enters the main page of the QWEN1.5-7B large model, as shown in Figure 5 From top to bottom, there are the current selected model logo, introduction and parameter quantity; the accuracy and F1 value of the current selected model for identifying false information in the dataset; the reference value of the video memory required for the local deployment of the current selected model; and the accuracy and F1 value of the current selected model for identifying false information in each field in the dataset. When the user decides to use the model to identify false information, click the “use this model to identify false information” button to jump to the false information identification page.

[0068] (1) Enter the false information identification page and input the to-be-identified false information: when the user completes step (2) and jumps to the false information identification page, the page is as shown in Figure 6

[0069] The page includes prompt information, false information identification input and recommended identified false information. The prompt information is divided into the currently used large model and the default used large model. Since the user enters the page by selecting the QWEN1.5-7B large model through the model selection page, the currently used large model is displayed as “QWEN1.5-7B”; if the user directly clicks the false information identification in the left menu bar, the currently used large model is displayed as “QWEN1.5-14B-GPTQ-INT8”, as shown in Figure 7

[0070] ​​False information recognition input: the user inputs the false information to be identified in this input, clicks start recognition, and the system calls the DLLVM model of the fourth chapter to identify the false information and returns the identification result.

[0071] The recommended false information is the false information identified by the user before in the database. The user can input these false information for secondary identification.

[0072] When the user inputs the false information to be identified in the input box and clicks start recognition, it jumps to the result display page and waits for the model to output the identification result.

[0073] (2) Enter the false information recognition result page and display the identification result of the false information to be identified: when the model returns the identification result, the false information recognition result page processes and displays the returned result, as shown in Figure 8 From top to bottom, they are the information to be identified, the retrieved documents, the field, the retrieved evidence, and the identification result.

[0074] The information to be identified is the false information to be identified input by the user in step (3); the retrieved documents are the names of the 10 documents most relevant to the information to be identified retrieved by the model; the field is the field to which the false information to be identified belongs calculated by the model; the retrieved evidence is the set of 5 sentences most relevant to the information to be identified retrieved by the model from the above documents. The identification result is the identification result of the information to be identified based on all the above information.

[0075] The above is the process from entering the system to input a piece of information to be identified to output the identification result of the information, which is also the core function of the system. In addition to this function, the system also provides the function of viewing the identification record, and the user can view the label to which each statement in the original data set belongs, the document retrieval record, the evidence retrieval record, and the statement verification result record output by the model.

[0076] Related evidence of the technical effects obtained by the embodiments of the present application.

[0077] 1. Simulation experiment environment and configuration

[0078] All simulation experiments are run on an Intel Xeon(R Platinum 8352V and NVIDIA RTX 4090 LINUX platform. The base model for training the field classification alignment model is ChatGLM3-6B. The base model for training the instruction alignment model is QWEN1.5-0.5B, QWEN1.5-1.8B, QWEN1.5-4B, ChatGLM3-6B, QWEN1.5-7B, QWEN1.5-14B-GPTQ-INT8.

[0079] 2、Dataset

[0080] The dataset is divided into two parts: large language model instruction alignment training dataset and false information recognition dataset. Both of them are based on the CFEVER dataset constructed by Lin et al. The distribution of each field and its number of statements in the dataset is shown in Table 1. According to the differences of large language model instruction alignment tasks, the present application extracts 1000 data from the training set in the CFEVER dataset as training data and constructs training datasets according to different instructions. The data examples of the dataset are shown in Table 2 and Table 3.

[0081] Table 1 Distribution of fields and their number of statements in CFEVER dataset

[0082]

[0083]

[0084] Table 2 Format of field classification alignment model training dataset

[0085]

[0086] Table 3 Format of false information recognition model training dataset

[0087]

[0088] 3、Simulation experiment process

[0089] For open source single card deployment large language model, the present application adopts three prompts to evaluate the influence of different prompts on the ability of large model to recognize false information. Examples of the three prompts are shown in Table 4:

[0090] Table 4 Three prompts constructed by open source single card deployment large language model

[0091]

[0092]

[0093] System Prompt (System Prompt with a small number of examples):

[0094] You are an expert in judging the truth and falsehood of facts. Now you need to use the evidence given to you to give the truth of the statement after thinking and reasoning

[0095] answer, you can only answer with true, false and not enough information. Below are some examples for you to learn. According to

[0096] the knowledge you have, judge the statement [In the process of burning matter, a strong reduction reaction called fire occurs. Energy will be released in the form of light and heat]

[0097] The truth of the statement [Fire must have four conditions to exist simultaneously.] You need to think step by step and then output your results from "True", "False", "Not enough information to judge." Output: True. Based on the knowledge you have, judge the statement [Ancient people generally believed that comets made of ice could bring unlimited luck.]

[0098] The truth of the statement [Fire must have four conditions to exist simultaneously.] You need to think step by step and then output your results from "True", "False", "Not enough information to judge." Output: True. Based on the knowledge you have, judge the statement [Ancient people generally believed that comets made of ice could bring unlimited luck.]

[0099] The truth of the statement [Fire must have four conditions to exist simultaneously.] You need to think step by step and then output your results from "True", "False", "Not enough information to judge." Output: True. Based on the knowledge you have, judge the statement [Ancient people generally believed that comets made of ice could bring unlimited luck.]

[0100] The truth of the statement [Fire must have four conditions to exist simultaneously.] You need to think step by step and then output your results from "True", "False", "Not enough information to judge." Output: True. Based on the knowledge you have, judge the statement [Ancient people generally believed that comets made of ice could bring unlimited luck.]

[0101] The truth of the statement [Fire must have four conditions to exist simultaneously.] You need to think step by step and then output your results from "True", "False", "Not enough information to judge." Output: True. Based on the knowledge you have, judge the statement [Ancient people generally believed that comets made of ice could bring unlimited luck.]

[0102]

[0103] Because domain alignment and instruction alignment are only supplements to the capabilities of large language models, only a part of the model needs to be involved in training when fine-tuning the model, so the present invention selects the LoRA technology to fine-tune the large language model. The hyperparameters for training the domain alignment model and the instruction alignment model are as shown in Table 5. The training time of the model increases with the number of model parameters, as shown in Table 6, when the model training loss function tends to be stable, the time of QWEN1.5-0.5B is about 30 minutes, while the time required for ChatGLM3-6B is 1.5 hours. Figure 4

[0104] Table 5 Hyperparameters for training document retrieval and evidence retrieval models

[0105]

[0106]

[0107] 4. Performance comparison

[0108] For base models, the present invention selects QWEN1.5 series and Chatglm3. The QWEN1.5 series large model is an open source large language model developed by the Alibaba Tongyi Qianwen team, and its 7B parameter model has a significant advantage in the industry, suitable for single card deployment and training scenarios. ChatGLM3 is a large language model jointly released by ZhiPai AI and KEG Laboratory of Tsinghua University.

[0109] 5. Results and analysis

[0110] ​First, the ability of open source single card deployment large language model (hereinafter referred to as large model) to identify false information was experimented, and the experimental results are shown in Table 6. It is found that after the parameters of the large model exceed 7 billion, the accuracy of using the large model for statement verification does not increase with the increase of parameters, but fluctuates around 0.460, which is consistent with the results of Lin et al. using GPT-3.5 and GPT-4 for statement verification. Using the prompts of thinking chain can further improve the accuracy of statement verification, but the improvement is limited; using a small number of example prompts can improve the accuracy of statement verification more than using thinking chain prompts, which shows that the large model has learning ability and can learn knowledge from the input examples and apply it to the output. In particular, the small parameter large model (QWEN1.5-0.5B and QWEN1.5-1.8B) using a small number of example prompts has lower accuracy than prompts containing only statements, which shows that the small parameter large model does not have learning ability, and the input examples will have a negative impact on the output of the large model. According to the above experimental results, the present application considers that the upper limit of the accuracy of directly using the large language model for statement verification is 0.460.

[0111] Table 6 Statement verification accuracy of different parameters of open source large model with different prompts

[0112]

[0113]

[0114] Further, the present application experiments on the multi-field false information recognition ability of open source large model, the prompts of large model are all based on the thinking chain prompts with the best effect shown in Table 6, the experimental results are shown in Tables 7 and 8, we use accuracy and F1 value to evaluate its recognition ability. It is found that the multi-field recognition ability of open source large model differs greatly with different fields, its effect in culture and humanities and social science is better, the average accuracy of each model reaches 0.587 and 0.561 respectively, the effect in political field is poor, the average accuracy value of each model is only 0.330, the gap between them reaches 77.9% and 70.0%, it can be concluded that there is still a lot of optimization space for the multi-field recognition ability of open source large model.

[0115] Table 7 Multi-field false information recognition accuracy of different parameters of open source large model

[0116]

[0117] Table 8 Multi-field false information recognition F1 value of different parameters of open source large model

[0118]

[0119] Table 4-8 (continued)

[0120]

[0121] Therefore, the model DLLVM proposed in the application is integrated into different parameter open source large models for false information identification experiments, and the experimental results are shown in Tables 9 and 10. The experimental results show that the accuracy and F1-value of other models are greatly improved, except that the accuracy and F1-value of QWEN1.5-0.8B are not improved, as shown in Tables 11 and 12, and the maximum improvement ratio reaches 228.81%, which indicates that the model DLLVM proposed in the application can effectively alleviate the domain imbalance problem of large language model in identifying false information and improve the ability of large language model in identifying false information in multiple domains. The reason why the model QWEN1.5-0.8B does not improve is that the model parameter is small, and the "emergence ability" of the large model is not obtained, and the logical reasoning ability is not possessed.

[0122] Table 9 Accuracy of multi-domain false information identification of DLLVM integrated into different parameter open source large models

[0123]

[0124]

[0125] Table 10 F1-value of multi-domain false information identification of DLLVM integrated into different parameter open source large models

[0126]

[0127] Table 11 Accuracy improvement value of multi-domain false information identification of DLLVM integrated into different parameter open source large models

[0128]

[0129]

[0130] Table 12 F1-value improvement value of multi-domain false information identification of DLLVM integrated into different parameter open source large models

[0131]

[0132] It should be noted that embodiments of the present application can be realized by hardware, software, or a combination of software and hardware. The hardware portion can be realized by a special logic; the software portion can be stored in a memory and executed by a proper instruction execution system, such as a microprocessor or a specially designed hardware. A person of ordinary skill in the art can understand that the above-mentioned apparatus and method can be realized by computer executable instructions and / or included in processor control codes, such as a carrier medium, such as a magnetic disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The apparatus of the present application and its modules can be realized by a hardware circuit, such as a very large scale integrated circuit or a gate array, a semiconductor, such as a logic chip, a transistor, or a programmable hardware device, such as a field programmable gate array, a programmable logic device, or the like, by software executed by various types of processors, or by a combination of the above-mentioned hardware circuit and software, such as firmware.

[0133] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement, and improvement within the technical range disclosed by the present application, and within the spirit and principle of the present application, should be included in the protection scope of the present application.

Claims

1. A multi-domain false information detection method based on factual information patterns and large language models, characterized by: The method comprises: Step S1, classify the information to be identified into domains to obtain the target domain; Step S2, extracting all documents corresponding to the target domain from the knowledge base to form a domain knowledge base; Step S3, using a document relevance calculation-ranking algorithm to retrieve the top k documents with the highest relevance to the information to be identified from the domain knowledge base; Step S4, constructing the top k documents and the information to be identified into document-claim pairs; Step S5: performing evidence retrieval on the document-claim pair based on the keyword-attention mechanism to obtain an evidence set; Step S6: input the information to be identified, the target domain, and the evidence set into a large language model aligned with instructions, and output a false information identification result.

2. The method according to claim 1, characterized in that The step S1 uses a large language model to perform text matching on a preset domain set, and appends the classification result to the end of the information to be identified for subsequent processing.

3. The method according to claim 1, characterized in that The step S3 uses a bidirectional dense residual vector retrieval algorithm to calculate the text vector similarity, and selects the top k target documents from high to low similarity.

4. The method according to claim 1, wherein The step S5 selects the sentences with the highest degree of conflict with the information to be identified as evidence by assigning keyword weights to each document-statement pair and combining them with the attention score.

5. The method according to claim 1, characterized in that The instruction alignment large language model in step S6 is trained by low-rank adaptive fine-tuning, and a performance evaluation is performed after the training is completed. If the preset indicators are not met, the fine-tuning is repeated until the indicators are met.

6. A multi-domain false information detection system, characterized in that: include: The domain classification module is used to classify the domain of the information to be identified; The document retrieval module is used to extract documents consistent with the field classification results from the knowledge base and complete the relevance ranking; an evidence retrieval module, for extracting evidence from the document based on a keyword-attention mechanism; The false information identification module is used to align the information to be identified, domain information and evidence input instructions with a large language model and output the identification results.

7. The system according to claim 6, characterized in that The document retrieval module supports hybrid retrieval of dense vector index and inverted index, and filters irrelevant documents according to relevance threshold.

8. A computer-readable storage medium, characterized in that The storage medium stores computer-executable instructions, which, when executed by a processor, enable the processor to execute the method according to any one of claims 1 to 5.

9. An electronic device, characterized in that: The system comprises a processor, a memory and a communication interface, wherein the memory stores instructions executable on the processor, and when the instructions are executed, the processor executes the method according to any one of claims 1 to 5.

10. A method for training a large language model for instruction alignment, characterized in that: include: Construct a training dataset containing domain classification instructions and false information identification instructions, and use a low-rank adaptive fine-tuning strategy to train the basic language model. After completion, evaluate the model performance according to the accuracy index and output a model that meets the index.

Images