Low-resource language open domain question answering method based on multi-dimensional answer screening

Abstract

The invention relates to a low-resource language open domain question answering method based on multi-dimensional answer screening, and belongs to the technical field of natural language processing. The method comprises the following steps: generating a diversified candidate answer set according to low-resource language query questions and documents through a large language model, and meanwhile, generating analysis and description of the low-resource language query questions by using English; performing global text verification on the candidate answers by utilizing a large language model based on all document information and English question description; extracting sentences supporting the answers from the document as evidences, and completing local evidence verification of the candidate answers; performing correlation sorting on the candidate answer set by adopting a pairwise sorting method to realize mutual verification among different candidate answers; and by calculating each candidate answer and considering the equal-weight evaluation total score of different dimensions, screening out answers which finally meet the low-resource language open domain question and answer requirements. According to the method, the performance of the large language model in the low-resource open domain question and answer task can be improved.

Description

Technical Field

[0001] The present invention relates to a low-resource language open-domain question answering method based on multi-dimensional answer screening, and belongs to the technical field of natural language processing. Background Art

[0002] Open-domain question answering usually involves retrieving multiple relevant documents from large-scale data and understanding the document content to generate answers. Currently, open-domain question answering tasks mainly adopt a retrieval-reading framework. In this framework, the retriever usually filters relevant text paragraphs from a large-scale document collection based on dense retrieval such as DPR or sparse retrieval such as the BM25 algorithm. The reader relies on a generative language model to generate answers that meet the requirements of the question through semantic understanding of the retrieved content and document analysis. Early studies mainly used pre-trained language models based on the Transformer architecture as readers, such as fine-tuned BERT or T5. In recent years, with the development of large language models, large language models represented by ChatGPT, the Llama series, and DeepSeek have become the preferred solution for current readers due to their powerful generation and reasoning capabilities.

[0003] Large language model-based reader methods can be broadly categorized into two types: single-round answer generation and candidate answer verification. In the former, the model directly generates answers or generates a single result after understanding and ranking documents. For example, Lewis et al. (2020) combined external document retrieval with a generative model for open-domain question answering; Wei et al. (2022) proposed "thought chain" prompts to guide the model's intermediate reasoning to improve accuracy; and Sun et al. (2023) used a sliding window to filter key paragraphs across multiple documents to improve answer quality. In contrast, candidate answer verification methods generate multiple candidate answers to improve accuracy and then select the final answer through a combination of reasoning and verification. Weng et al. (2023) employed multi-sampling to generate multiple answers and verified their correctness through reverse reasoning; Kim et al. (2024) proposed first generating multiple candidate answers through a prompting strategy, then generating a summary for each answer, and determining the final answer based on the verification results of the summary. These large language model-based methods have performed well in high-resource languages such as English and Chinese, but remain challenging in low-resource languages such as Burmese and Cambodian. Single-round generation methods are prone to reasoning errors due to noise in documents. This noise primarily stems from a limited knowledge base and immature retrieval technology. Furthermore, the model's limited understanding of low-resource languages further exacerbates bias. While candidate answer verification methods improve the coverage of correct answers by generating multiple answers, they are still limited by low-quality evidence and a single verification mechanism, making it difficult to effectively screen for correct answers.

[0004] To address the above problems, the present invention proposes a low-resource language open-domain question answering method based on multi-dimensional answer screening. Summary of the Invention

[0005] The technical problem solved by the present invention is: the present invention provides a low-resource language open-domain question answering method based on multi-dimensional answer screening, which is used to solve the problem of biased generation results caused by interference from noisy documents when processing low-resource open-domain question answering tasks in existing methods; at the same time, in view of the fact that existing large language models have poor context understanding capabilities in low-resource languages and are prone to reasoning errors; the method of the present invention can improve the performance of large language models in low-resource open-domain question answering tasks.

[0006] The technical solution of the present invention is: a low-resource language open-domain question answering method based on multi-dimensional answer screening, the method comprising:

[0007] Step 1: Build an open-domain question-answering dataset in a low-resource language. The dataset contains question-answer pairs and related documents in a low-resource language.

[0008] Step 2: Use the prompt learning strategy to guide the large language model to generate a diverse set of candidate answers based on low-resource language queries and documents. At the same time, guide the large language model to generate analysis and descriptions of low-resource language queries in English.

[0009] Step 3: Based on all document information and the generated English question description, a large language model is used to perform global context verification on candidate answers to assess how well the answers match the overall document content.

[0010] Step 4: For each candidate answer, extract sentences supporting the answer from the document as evidence. Combined with the generated English question description, the large language model is used to evaluate the relevance of the evidence to the low-resource language query question, completing local evidence verification for the candidate answer.

[0011] Step 5: Input candidate answers into the large language model in pairs, and use the pairwise sorting method to sort the candidate answers by relevance, so as to achieve mutual verification between different candidate answers.

[0012] Step 6: Calculate the total score of each candidate answer by considering the equal weights of different dimensions, and select the answers that ultimately meet the requirements of open domain question answering in low-resource languages.

[0013] Furthermore, the Step 1 includes:

[0014] Step 1.1: Use Selenium web crawler technology to control browser automation operations and batch convert the existing English question-answering dataset into the target low-resource language data through Google Translate. The dataset contains low-resource language questions, answers, and related documents; the related documents include noisy documents.

[0015] Step 1.2: Use the gpt-4o-mini model to perform semantic correction and consistency optimization on the translated data to obtain a low-resource language open domain question answering dataset.

[0016] Furthermore, the Step 2 includes:

[0017] Step 2.1: The low-resource language query question q in the low-resource language open domain question answering dataset and the low-resource document collection containing noise are Splice to form an input content set; generate prompts p in the designed candidate answers cans Guide the large language model M to generate a diverse set of candidate answers A can ; Generate candidate answer set A can The process is expressed as follows:

[0018]

[0019] Among them, N represents the generated candidate answer a i The number of

[0020] Step 2.2: Take the low-resource language query question q as input and prompt p with the English question description en Explicitly instruct the large language model M to use English to analyze the low-resource language query question q and generate the corresponding English question description des en ; Generate English problem description des en The process is expressed as follows:

[0021]

[0022] Furthermore, the Step 3 includes:

[0023] Using the global chapter validation function For each candidate answer a i Evaluate; each candidate answer a i , document collection Low-resource language query question q and English question description des en Input into the large language model M, based on the global verification prompt p glob Perform global paragraph verification to determine candidate answers a i As the correct answer to the low-resource language query question q is accurate and reasonable, and the global evaluation value is calculated based on the output of the large language model M. The global evaluation value score1(a i ) is calculated as follows:

[0024]

[0025] Among them, the global chapter verification function Defined as:

[0026] Furthermore, the Step 4 includes:

[0027] Step 4.1. Using evidence to extract function ε evi , each candidate answer a i , low-resource language query question q and document collection As input, in the evidence extraction prompt p evi Under the action of i Evidence of the source i As fine-grained evidence; the specific implementation is as follows:

[0028]

[0029] Among them, each candidate answer a i evidence i satisfy

[0030] Step 4.2: Verify the function using local evidence For each candidate answer a i Evaluate and assign each candidate answer a i evidence i With low-resource language query question q and English question description des en Input into the large language model M together; through the designed local verification prompt p part , guiding the large language model M to judge and support each candidate answer a i evidence i Whether it provides effective help information for low-resource language query question q, and performs local evidence verification evaluation value score2(a i ) calculation, local evidence verification evaluation value score2(a i ) The specific calculation method is as follows:

[0031]

[0032] Among them, the local evidence verification function Defined as:

[0033] Furthermore, the Step 5 includes:

[0034] The candidate answer set A cand Each candidate answer a ini With other candidate answers a k Input to the answer comparison function In the middle, the large language model M is based on the document collection And low resource language query question q in answer comparison prompt p comp Under the guidance of i or a k Which one is more suitable as the correct answer to the low-resource language query problem; Finally, based on the candidate answer a i The number of times selected, calculate the score of the candidate answer score3(a i ), and rank all candidate answers accordingly, the ranking score of the candidate answers is score3(a i ) is calculated as follows:

[0035]

[0036] Among them, the answer comparison function Defined as:

[0037]

[0038] Furthermore, the Step 6 includes:

[0039] The scores of global passage verification, local evidence verification, and relevance ranking of different candidate answers are calculated using equal weights. By summarizing the evaluation results of different dimensions of each candidate answer, the candidate answer with the highest score is selected as the final output. The specific calculation method is as follows:

[0040]

[0041] in, Indicates the final answer screened out, j indicates the evaluation dimension, Indicates the i-th one with the highest score after equal weight calculation * candidate answers.

[0042] The present invention also provides a low-resource language open domain question answering system based on multi-dimensional answer screening, which includes: a module for executing the low-resource language open domain question answering method based on multi-dimensional answer screening.

[0043] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the low-resource language open-domain question answering method based on multi-dimensional answer screening is implemented.

[0044] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the low-resource language open-domain question answering method based on multi-dimensional answer screening is implemented.

[0045] The beneficial effects of the present invention are:

[0046] 1. This paper uses a translation engine and a large language model to construct an open-domain question-answering dataset for low-resource languages in Southeast Asian languages, providing a solid data foundation for the research of open-domain question-answering in low-resource languages.

[0047] 2. This invention uses prompt learning to guide large models to transform the complex question-answering process into a multi-stage candidate answer generation and screening process, effectively overcoming the limitations of traditional methods such as susceptibility to noise interference and the limited understanding ability of large models in low-resource languages.

[0048] 3. This invention generates a variety of candidate answers through prompt learning and combines a multi-dimensional screening mechanism to ensure that the final answer has a high degree of relevance and accuracy;

[0049] 4. This invention enables open-domain question answering in low-resource languages through lightweight prompt engineering, without relying on large-scale annotated data or language-specific fine-tuning, significantly reducing the application threshold and computational cost.

[0050] 5. The present invention aims to transform the complex question-answering process into a multi-stage candidate answer generation and screening framework. First, in the answer generation stage, the model is guided to generate multiple candidate answers from multiple documents to improve the coverage of correct answers. Then, in the answer screening stage, a multi-dimensional verification strategy is designed: direct verification of each answer based on global document information, local verification of each answer by extracting relevant evidence, and ranking by comparing the correlations between different answers. Through the collaborative screening and verification mechanism of these three dimensions, the model selects the optimal answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is the framework of the low-resource language open-domain question answering method based on multi-dimensional answer screening in the present invention. DETAILED DESCRIPTION

[0052] Example 1: The following method proposed in this example was experimented on an open domain question-answering dataset in four low-resource languages: Burmese, Thai, Lao, and Khmer. The experimental dataset was obtained by translating and manually verifying the RGB question-answering dataset. Each low-resource language contains 200 test question-answering data and 1,000 documents related to question-answering. The average length information of each part of the dataset is shown in Table 1. At the same time, in the experiment, the present invention selected large language models with powerful performance and certain low-resource language processing capabilities, including GPT-4o-mini and DeepSeek-v3, to verify the effectiveness of the proposed method. When calling the API, the temperature parameter is set to 0.0 to ensure the stability of the model generation results. In the experiment, the number of candidate answers generated is fixed to N=2.

[0053] Table 1 shows the length statistics of each part of the data set (string)

[0054]

[0055]

[0056] like Figure 1 As shown, a low-resource language open-domain question answering method based on multi-dimensional answer screening includes:

[0057] Step 1: Build an open-domain question-answering dataset in a low-resource language. The dataset contains question-answer pairs and related documents in a low-resource language.

[0058] Furthermore, the Step 1 includes:

[0059] Step 1.1: Use Selenium web crawler technology to control browser automation operations and batch convert the existing English question-answering dataset into the target low-resource language data through Google Translate. The dataset contains low-resource language questions, answers, and related documents, including noisy documents.

[0060] Step 1.2: Use the GPT-4O-Mini model to perform semantic correction and consistency optimization on the translated data to obtain a low-resource language open-domain question-answering dataset, ensuring that the translated questions, answers, and related documents are semantically consistent.

[0061] Step 2: Use the prompt learning strategy to guide the large language model to generate a diverse set of candidate answers based on low-resource language queries and documents. At the same time, guide the large language model to generate analysis and descriptions of low-resource language queries in English.

[0062] Furthermore, the Step 2 includes:

[0063] Step 2.1: The low-resource language query question q in the low-resource language open domain question answering dataset and the low-resource document collection containing noise are Splice to form an input content set; generate prompts p in the designed candidate answers cans Guide the large language model M to generate a diverse set of candidate answers A can ; Generate candidate answer set A can The process is expressed as follows:

[0064]

[0065] Among them, N represents the generated candidate answer a i The number of candidate answers; the prompt p cans You are aprofessional{language}AI assistant.Below are{language}N passages related to the question at the end.After reading the passages,provide two correct and complete candidates for the answer to the question at the end.The answer should be in the form:(a)xx,(b)yy.Each candidate answer should not exceed3words.Passage#1text:{passage1text}...Passage#N text:{passageN text}Question:{question}Answer:

[0066] Translation:

[0067] You are a professional {language} AI assistant. Below are N {language} articles related to the question at the end of the article. After reading the articles, please provide two correct and complete candidate answers to the question at the end of the article. The answer format is as follows: (a) xx, (b) yy. Each candidate answer should be no more than 3 words. Paragraph #1 Text: {Paragraph 1 Text}... Paragraph #N Paragraph: {Paragraph N Text} Question: {Question} Answer:

[0068] Step 2.2: Take the low-resource language query question q as input and prompt p with the English question description en Explicitly instruct the large language model M to use English to analyze the low-resource language query question q and generate the corresponding English question description desen ; Generate English problem description des en The process is expressed as follows:

[0069]

[0070] The English question description prompts p en For: You are a professional multilingual AI assistant. Please analyze the given{language}question using English,brieflyidentifying the type or format of answer it requires.Do not answer the question directly.Question:{question}Analyze:

[0071] Translation:

[0072] You are a professional multilingual AI assistant. Please analyze the given {language} question in English and briefly describe the type or format of answer required. Do not answer the question directly. Question: {question} Analysis:

[0073] Step 3: Based on all document information and the generated English question description, a large language model is used to perform global context verification on candidate answers to assess how well the answers match the overall document content.

[0074] Furthermore, the Step 3 includes:

[0075] Using the global chapter validation function For each candidate answer a i Evaluate; each candidate answer a i , document collection Low-resource language query question q and English question description des en Input into the large language model M, based on the global verification prompt p glob Perform global paragraph verification to determine candidate answers a i As the correct answer to the low-resource language query question q is accurate and reasonable, and the global evaluation value is calculated based on the output of the large language model M. The global evaluation value score1(a i ) is calculated as follows:

[0076]

[0077] Among them, the global chapter verification function Defined as:

[0078] The global verification prompt p glob For: Passage#1text:{passage1 text}...Passage#Ntext:{passageN text}Question:{question}Question Description:{question des}Prediction:{Candidate Answer}Analyze the provided answer to determine its accuracy,reasonableness,and correctness.Provide a concise evaluation first,then conclude with either”True”or”False”based on your assessment.

[0079] Translation:

[0080] Paragraph #1 Text: {Paragraph 1 Text} ... Paragraph #N Text: {Paragraph N Text} Question: {Question} Question Description: {Question Description} Prediction: {Candidate Answer} Analyze the provided answers to determine their accuracy, plausibility, and correctness. Provide a concise assessment first, then conclude whether the answer is "True" or "False" based on your assessment.

[0081] Step 4: For each candidate answer, extract sentences supporting the answer from the document as evidence. Combined with the generated English question description, the large language model is used to evaluate the relevance of the evidence to the low-resource language query question, completing local evidence verification for the candidate answer.

[0082] Furthermore, the Step 4 includes:

[0083] Step 4.1. Using evidence extraction function ε evi , each candidate answer a i , low-resource language query question q and document collection As input, in the evidence extraction prompt p evi Under the action of i Evidence of the source i As fine-grained evidence; the specific implementation is as follows:

[0084]

[0085] Among them, each candidate answer ai evidence i satisfy

[0086] Evidence extraction prompts evi For: Passage#1text:{passage1 text}...Passage#N text:{passageN text}Question:{question}Prediction:{Candidate Answer}Your task is to extract content related to the source of the predicted answer directly from the provided{language}passages.Do not generate new content; only extractexisting content from the paragraph.Make sure the content extracted fully supports the given prediction.Preserve the original sentence structure as much as possible.When you have completed the task,write[DONE]to indicate that the task is completed.

[0087] Translation:

[0088] Paragraph #1 Text: {Paragraph 1 Text} ... Paragraph #N Text: {Paragraph 1 Text} Question: {Question} Prediction: {Candidate Answer} Your task is to extract relevant content from the provided {Language} paragraph directly to the source of the predicted answer. Do not generate new content; only extract existing content from the paragraph. Ensure that the extracted content fully supports the given prediction. Preserve the original sentence structure as much as possible. When you have completed the task, write [Done] to indicate completion.

[0089] Step 4.2: Verify the function using local evidence For each candidate answer a i Evaluate and assign each candidate answer a i evidence i With low-resource language query question q and English question description des enInput into the large language model M together; through the designed local verification prompt p part , guiding the large language model M to judge and support each candidate answer a i evidence i Whether it provides effective help information for low-resource language query question q, and performs local evidence verification evaluation value score2(a i ) calculation, local evidence verification evaluation value score2(a i ) The specific calculation method is as follows:

[0090]

[0091] Among them, the local evidence verification function Defined as:

[0092] The local verification prompt p part For:Question:{question}Question Description:{question description}Evidence:{evidence}Does the evidence provide usefulinformation to answer the question? Please explain your reasoning and provide your judgment(True or False).

[0093] Translation:

[0094] Question: {Question} Question Description: {Question Description} Evidence: {evidence} Does the evidence provide useful information to answer the question? Please explain your reasoning and provide your judgment (True or False).

[0095] Step 5: Input candidate answers into the large language model in pairs, and use the pairwise sorting method to sort the candidate answers by relevance, so as to achieve mutual verification between different candidate answers.

[0096] Furthermore, the Step 5 includes:

[0097] The candidate answer set A cand Each candidate answer a in i With other candidate answers a k Input to the answer comparison function In the middle, the large language model M is based on the document collection And low resource language query question q in answer comparison prompt pcomp Under the guidance of i or a k Which one is more suitable as the correct answer to the low-resource language query problem; Finally, based on the candidate answer a i The number of times selected, calculate the score of the candidate answer score3(a i ), and rank all candidate answers accordingly, the ranking score of the candidate answers is score3(a i ) is calculated as follows:

[0098]

[0099] Among them, the answer comparison function Defined as:

[0100]

[0101] The answer comparison prompts p comp For: Passage#1text:{passage1 text}...Passage#Ntext:{passageN text}Question:{question}Candidate Answers:(Answer1:{CandidateAnswers1}Answer2:{Candidate Answers2})Based on the information provided in the passage,determine whether[Answer 1]or[Answer 2]is the most suitable answer to the question.Respond only with”Answer1”or”Answer2”.

[0102] Translation:

[0103] Paragraph #1 Text: {Paragraph 1 Text}... Paragraph #N Text: {Paragraph N Text} Question: {Question} Candidate Answers: (Answer 1: {Candidate Answer 1} Answer 2: {Candidate Answer 2}) Based on the information provided in the passage, determine whether [Answer 1] or [Answer 1] is the most appropriate answer to this question. Please only use "Answer 1" or "Answer 2" when answering.

[0104] Step 6: Calculate the total score of each candidate answer by considering the equal weights of different dimensions, and select the answers that ultimately meet the requirements of open domain question answering in low-resource languages.

[0105] Furthermore, the Step 6 includes:

[0106] The scores of global passage verification, local evidence verification, and relevance ranking of different candidate answers are calculated using equal weights. By summarizing the evaluation results of different dimensions of each candidate answer, the candidate answer with the highest score is selected as the final output. The specific calculation method is as follows:

[0107]

[0108] in, Indicates the final answer screened out, j indicates the evaluation dimension, Indicates the i-th one with the highest score after equal weight calculation * candidate answers.

[0109] The present invention also provides a low-resource language open-domain question answering system based on multi-dimensional answer screening, the system comprising:

[0110] A dataset construction module is used to construct an open-domain question-answering dataset for low-resource languages. The open-domain question-answering dataset for low-resource languages contains question-answer pairs and related documents in low-resource languages.

[0111] A generation module, which guides the large language model to generate a diverse set of candidate answers based on low-resource language queries and documents through a prompt learning strategy, while also guiding the large language model to generate analysis and descriptions of low-resource language queries in English;

[0112] The global text verification module is used to perform global text verification on candidate answers based on all document information and the generated English question description using a large language model to evaluate the degree of match between the answer and the overall document content;

[0113] The local evidence verification module extracts sentences supporting each candidate answer from the document as evidence. Combined with the generated English question description, the module uses a large language model to evaluate the relevance of the evidence to the low-resource language query question, completing the local evidence verification of the candidate answer.

[0114] The candidate answer relevance ranking module is used to input candidate answer sets into pairs into the large language model, and use the pairwise ranking method to sort the candidate answer sets by relevance, thereby achieving mutual verification between different candidate answers;

[0115] The final answer screening module is used to calculate the total score of each candidate answer by considering the equal weights of different dimensions, and screen out the answers that ultimately meet the requirements of open domain question answering in low-resource languages.

[0116] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the low-resource language open-domain question answering method based on multi-dimensional answer screening is implemented.

[0117] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the low-resource language open-domain question answering method based on multi-dimensional answer screening is implemented.

[0118] In order to verify the effectiveness of the proposed method, the method is compared with the following baseline methods of different frameworks, as follows:

[0119] Direct generation method (Base): The Base method is to input the retrieved text and question into a large language model, and directly generate the answer to the question through the prompt model.

[0120] Chain of Thought (CoT): Based on the Base method, a zero-shot Chain of Thought strategy is adopted to guide the model to reason and think step by step before generating the final answer.

[0121] Cross-lingual prompts (CLP): Guide the large language model to think and reason about questions and passages in low-resource languages in English, and then derive answers in the target low-resource language.

[0122] Large Model Reranking Method (LLMRerank): Using a large language model as the ranking agent, the large language model is used to identify and rank the retrieved paragraphs. A sliding window strategy is implemented to select the documents most relevant to the query question from multiple documents, and then the selected documents are used to generate answers.

[0123] Self-verification method: Generate multiple candidate answers through multiple sampling, and then use a large language model to reversely verify the answers inferred by itself and select the most suitable answer.

[0124] Candidate answer summary verification (Sure): Generate multiple candidate answers by prompting the model, and generate supporting summaries based on these candidate answers. Finally, the best answer is selected by evaluating the quality of the summary.

[0125] To evaluate the performance of our method in the open-domain question answering task in low-resource languages, we selected Exact Match (EM) and F1 score as evaluation metrics. Exact Match (EM) measures the ratio of correct answers in the test dataset. If a given prediction is completely consistent with one of the correct answers, the prediction is considered correct. The specific calculation method is as follows:

[0126]

[0127] in, is the answer predicted by the model, a is the correct answer, and normalize(x) means normalizing the text x, including case processing, deleting punctuation, and removing extra spaces.

[0128] The F1 score measures the overlap between the vocabulary set of the predicted answer and the correct answer, aiming to balance the trade-off between correctly identifying the answer and avoiding missing the correct answer. The specific calculation method is as follows:

[0129]

[0130] in, is the answer predicted by the model, a is the correct answer, Is the predicted answer The number of words in common between the correct answer and a. is the total number of words in the predicted answer. |a| is the total number of words in the correct answer.

[0131] Table 2 shows the comparative experimental results on gpt-4o-mini

[0132]

[0133] Table 3 shows the comparative experimental results on deepseek-v3

[0134]

[0135] The proposed method was compared with four language benchmark datasets to verify its effectiveness. The experimental results are shown in Tables 2 and 3. The main findings are as follows:

[0136] (1) Our method outperforms most existing methods on four language datasets. Specifically, compared to the suboptimal approach, our method achieves improvements of 3.5% and 3.5% when using the GPT-4o-mini model, and 2.9% and 2.7% when using the DeepSeek-v3 model. This result fully demonstrates the effectiveness of our method.

[0137] (2) Experimental observations show that on the GPT-4o-mini model, single-round generation methods such as CoT and CLP are affected by noise interference and the limitations of low-resource language reasoning, resulting in little or no significant performance improvement or even a decline. Although the LLMRerank method has a slight performance improvement in languages other than Thai, its performance declines in Thai tasks, indicating that its ranking ability is not stable enough in low-resource tasks. Candidate answer verification-based methods such as Sure and Self-verification improve the correct answer hit rate through a multi-candidate answer mechanism, but due to reliance on a single verification strategy and low-quality summary evidence, the performance improvement is limited.

[0138] (3) The DeepSeek-v3 model performed best in most experiments, demonstrating extremely high adaptability. All baseline methods outperformed the direct answer generation method on this model, further verifying its excellent low-resource language generalization ability. The present invention believes that this advantage stems from its innovative multi-head latent attention mechanism and DeepSeekMoE architecture, which are significantly different from traditional large language models and effectively enhance reasoning ability and adaptability to low-resource tasks.

[0139] In order to verify the influence of the number N of candidate answers on the experimental results in the method proposed in the present invention, an analysis experiment was designed to explore the experimental results of N=3 under the deepseek-v3 model.

[0140] Table 4 shows the comparison results of different candidate answer numbers N on deepseek-v3

[0141]

[0142] The experimental results, shown in Table 4, show that increasing the number of candidate answers N slightly improves the probability of finding the correct answer, but it also complicates the multi-dimensional screening process and increases computational cost. Therefore, considering both accuracy and computational cost, the proposed method adopts N = 2, generating two candidate answers for screening, which is a more reasonable and efficient strategy.

[0143] In order to verify the effectiveness of each part of the low-resource language question answering method proposed in this paper, the following ablation experiments were designed for comparison. The experimental results are shown in Tables 5 and 6:

[0144] w / o desen: represents removing the English problem description from the model of the present invention,

[0145] w / o score1: represents removing the global chapter verification score score1 from the model of the present invention

[0146] w / o score2: represents the removal of local evidence verification score score2 from the model of the present invention

[0147] w / o score3: represents removing the different answer relevance ranking score score3 from the model of the present invention,

[0148] Full model: represents the complete model method proposed in this invention.

[0149] Table 5 shows the ablation test results on the gpt-4o-mini model

[0150]

[0151]

[0152] Table 6 shows the ablation test results on the deepseek-v3 model.

[0153]

[0154] The results of ablation experiments show that (1) the complete method of the present invention shows the best performance in all test languages and evaluation indicators, which fully demonstrates the effectiveness of the proposed low-resource question answering method. Experimental data show that removing any functional module will significantly affect the model performance, verifying the necessity of each component in the system. (2) Removing the English question description desen significantly affects the model performance, indicating that it is of great value as a bridge for cross-language understanding for low-resource language question answering tasks. Among the three scoring modules, the global chapter verification score score1 has a relatively small impact, but its global verification role is still indispensable; the removal of the local evidence verification score score2 has a more significant impact, confirming the key role of fine-grained verification in answer accuracy; and the absence of the relevance ranking of different answers score3 leads to the largest performance drop, indicating that the comparison and ranking mechanism of different answers is crucial to the final answer selection, which can effectively filter out wrong answers and improve the reliability of the results.

[0155] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A low-resource language open-domain question answering method based on multi-dimensional answer screening, characterized by: The method comprises: Step 1: Build an open-domain question-answering dataset in a low-resource language. The dataset contains question-answer pairs and related documents in a low-resource language. Step 2: Use the prompt learning strategy to guide the large language model to generate a diverse set of candidate answers based on low-resource language queries and documents. At the same time, guide the large language model to generate analysis and descriptions of low-resource language queries in English. Step 3: Based on all document information and the generated English question description, a large language model is used to perform global context verification on candidate answers to assess how well the answers match the overall document content. Step 4: For each candidate answer, extract sentences supporting the answer from the document as evidence. Combined with the generated English question description, the large language model is used to evaluate the relevance of the evidence to the low-resource language query question, completing local evidence verification for the candidate answer. Step 5: Input candidate answers into the large language model in pairs, and use the pairwise sorting method to sort the candidate answers by relevance, so as to achieve mutual verification between different candidate answers. Step 6: Calculate the total score of each candidate answer by considering the equal weights of different dimensions, and select the answers that ultimately meet the requirements of open domain question answering in low-resource languages.

2. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 1 includes: Step 1.1: Use Selenium web crawler technology to control browser automation operations and batch convert the existing English question-answering dataset into the target low-resource language data through Google Translate. The dataset contains low-resource language questions, answers, and related documents. Step 1.2: Use the gpt-4o-mini model to perform semantic correction and consistency optimization on the translated data to obtain a low-resource language open domain question answering dataset.

3. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 2 includes: Step 2.1: The low-resource language query question q in the low-resource language open domain question answering dataset and the low-resource document collection containing noise are Splice to form an input content set; generate prompts p in the designed candidate answers cans Guide the large language model M to generate a diverse set of candidate answers A can ; Generate candidate answer set A can The process is expressed as follows: Among them, N represents the generated candidate answer a i The number of Step 2.2: Take the low-resource language query question q as input and prompt p with the English question description en Explicitly instruct the large language model M to use English to analyze the low-resource language query question q and generate the corresponding English question description des en ; Generate English problem description des en The process is expressed as follows:

4. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 3 includes: Using the global chapter validation function For each candidate answer a i Evaluate; each candidate answer a i , document collection Low-resource language query question q and English question description des en Input into the large language model M, based on the global verification prompt p glob Perform global paragraph verification to determine candidate answers a i Whether the correct answer to the low-resource language query question q is accurate and reasonable, and the global evaluation value is calculated based on the output of the large language model M. The global evaluation value score1(a i ) is calculated as follows: Among them, the global chapter verification function Defined as:

5. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 4 includes: Step 4.

1. Using evidence to extract function ε evi , each candidate answer a i , low-resource language query question q and document collection As input, in the evidence extraction prompt p evi Under the action of i Evidence of the source i As fine-grained evidence; the specific implementation is as follows: Among them, each candidate answer a i evidence i satisfy Step 4.2: Verify the function using local evidence For each candidate answer a i Evaluate and assign each candidate answer a i evidence i With low-resource language query question q and English question description des en Input into the large language model M together; through the designed local verification prompt p part , guiding the large language model M to judge and support each candidate answer a i evidence i Whether it provides effective help information for low-resource language query question q, and performs local evidence verification evaluation value score2(a i ) calculation, local evidence verification evaluation value score2(a i ) The specific calculation method is as follows: Among them, the local evidence verification function Defined as:

6. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 5 includes: The candidate answer set A cand Each candidate answer a in i With other candidate answers a k Input to the answer comparison function In the middle, the large language model M is based on the document collection And low resource language query question q in answer comparison prompt p comp Under the guidance of i or a k Which one is more suitable as the correct answer to the low-resource language query problem; Finally, based on the candidate answer a i The number of times selected, calculate the score of the candidate answer score3(a i ), and rank all candidate answers accordingly, the ranking score of the candidate answers is score3(a i ) is calculated as follows: Among them, the answer comparison function Defined as:

7. The low-resource language open-domain question answering method based on multi-dimensional answer screening according to claim 1 is characterized by: Step 6 includes: The scores of global passage verification, local evidence verification, and relevance ranking of different candidate answers are calculated using equal weights. By summarizing the evaluation results of different dimensions of each candidate answer, the candidate answer with the highest score is selected as the final output. The specific calculation method is as follows: in, Indicates the final answer screened out, j indicates the evaluation dimension, Indicates the i-th one with the highest score after equal weight calculation * candidate answers.

8. A low-resource language open-domain question answering system based on multi-dimensional answer screening, characterized by: The system includes: a module for executing the low-resource language open-domain question answering method based on multi-dimensional answer screening according to any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the low-resource language open-domain question answering method based on multi-dimensional answer screening as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the low-resource language open-domain question answering method based on multi-dimensional answer screening as described in any one of claims 1 to 7 is implemented.

Images