Search enhanced scientific question answering method and system based on deep evidence reordering
By employing a deep evidence reordering method, combined with a large language model for intent recognition and logical relevance assessment, the problem of semantically similar but logically unrelated text segments in the RAG system is solved, improving the robustness and credibility of scientific question answering and generating high-quality scientific evidence summaries.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-05
AI Technical Summary
Existing search-enhanced generation (RAG) systems suffer from interference in scientific question answering by semantically similar but logically unrelated text segments, and lack reasoning and judgment capabilities during the reordering stage, resulting in insufficient reliability of the evidence in the answers.
A deep evidence reordering approach is adopted, which uses a large language model for intent recognition, logical relevance scoring, evidence screening and summary generation to achieve dynamic evaluation of the logical consistency and reliability of candidate evidence paragraphs.
It enhances the robustness and credibility of the RAG system in the scientific field, improves the scientific rigor, accuracy, and interpretability of the answers, reduces the impact of noisy documents, and increases generation efficiency.
Smart Images
Figure CN121979990A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing and knowledge retrieval, specifically to a retrieval-enhanced scientific question answering method and system based on deep evidence reordering, belonging to the application of retrieval-enhanced generation (RAG) technology in scientific question answering. Background Technology
[0002] Scientific Q&A, as an important means of promoting scientific research innovation and knowledge discovery, is widely used in many fields such as biomedicine, materials science, and chemical engineering. For example, molecular biologists need to systematically search relevant literature to analyze off-target effects when discussing CRISPR gene editing technology; during global public health events, clinicians need to quickly extract reliable evidence from a massive amount of newly published research to support evidence-based medicine decisions. With the rapid increase in the number of scientific documents, researchers face increasingly greater challenges in literature screening and knowledge acquisition.
[0003] Traditional information retrieval systems often rely on keyword matching or static semantic embedding models. These systems tend to return documents that are only lexically or semantically related, failing to ensure logical relevance or factual support. In recent years, large language models have shown significant progress in natural language understanding and reasoning, but they still suffer from problems such as illusion generation, outdated knowledge, and insufficient factual basis. To overcome these shortcomings, retrieval-enhanced generative techniques have been widely adopted. RAGs, by combining external retrieval with language model generation, enable the generated results to be traced and verified based on factual documents, offering significant advantages in improving answer accuracy, interpretability, and domain adaptability.
[0004] While the RAG framework effectively mitigates the illusion of fact problem in language models, its performance remains limited by the quality of context during the retrieval phase. Since initial search results often contain a large number of semantically similar but logically unrelated passages, or even noisy or outdated information, these "pseudo-relevant" contents significantly reduce the reliability of the final answer. To address this, researchers have introduced re-ranking modules that re-evaluate the relevance of candidate passages using models such as cross-encoders to improve retrieval accuracy. Typical re-ranking models include BGE, BCE, and Jina, which calculate the relevance score between the query and the document using double-sequence encoding, significantly improving ranking quality in most scenarios.
[0005] However, existing reordering models still primarily rely on word- or sentence-level vector similarity measurements, making it difficult to identify semantically similar but logically unrelated texts. This "semantic illusion" phenomenon is particularly prominent in scientific question answering: two passages may be semantically similar but do not logically answer the research question. Experiments have shown that this semantic misjudgment leads the model to retrieve a large amount of "superficially relevant" but "logically inconsistent" content, thereby inducing the language model to produce incorrect or unsupported answers during the generation stage, weakening the robustness and credibility of the RAG system.
[0006] To address this issue, researchers attempted to generate "interference segments" using large language models to evaluate the noise resistance performance of current embedding models. Experimental results show that mainstream embedding models (such as Qwen3-Embedding-8B, E5-Mistral-7B, and BGE-M3) exhibit significantly lower noise robustness scores and context discrimination rates than ideal when faced with semantically similar but logically unrelated interference samples. This indicates that existing embedding-based RAG retrieval and re-ranking mechanisms still have significant shortcomings at the logical discrimination level.
[0007] Therefore, a novel reordering mechanism is needed that combines the reasoning capabilities of large language models with multidimensional semantic understanding to surpass traditional vector similarity measures and achieve dynamic evaluation of the logical consistency and reliability of evidence for retrieved candidate text segments, thereby improving the robustness and credibility of the RAG system in the scientific field. Summary of the Invention
[0008] To address the problems of semantically similar but logically unrelated text interference, lack of reasoning and judgment ability in the reordering stage, and insufficient reliability of answer evidence in existing scientific literature question answering systems with Retrieval Enhancement Generation (RAG) frameworks, this invention proposes a retrieval enhancement scientific question answering method and system based on deep evidence reordering. It performs deep evidence reordering based on the reasoning ability of a large language model, realizing a deep evidence reordering agent system (DeepEra).
[0009] The technical solution adopted in this invention is as follows: A retrieval-enhanced scientific question answering method based on deep evidence reordering includes the following steps: The input scientific question is parsed into a structured representation of the question's intent. Scoring the logical relevance of candidate evidence paragraphs based on the intended meaning of the question; Based on the logical relevance scoring results, highly credible evidence paragraphs are selected from candidate evidence paragraphs using the logical relevance score threshold. Generate summaries for scientific questions based on the selected high-credibility evidence paragraphs.
[0010] Furthermore, the process of parsing the input scientific question into a structured question intent representation includes: The input question is submitted to a large language model for intent recognition, generating a structured question intent representation, which contains four key elements: The scientific topic to which the question belongs indicates the discipline or research direction of the question; Core entity types are used to identify the key scientific objects involved in the problem; The question intent instructs the user on the cognitive task they wish to perform. Expected answer type: Specifies the type of information the answer should provide.
[0011] Furthermore, the step of scoring the logical relevance of candidate evidence paragraphs based on the problem intent representation involves the following steps: scoring the logical relevance between each candidate evidence paragraph in the candidate evidence paragraph set and the structured problem intent representation. Each candidate evidence paragraph is used to calculate a logical relevance score through a large language model. The logical relevance score reflects the direct evidentiary value of the candidate evidence paragraph to the scientific question. The higher the score, the stronger the logical relevance. Each candidate evidence paragraph and its logical relevance score are combined into a set; The candidate evidence paragraphs in the set are sorted from high to low according to their logical relevance scores to obtain the rearranged candidate evidence paragraph sequence.
[0012] Furthermore, the step of using a logical relevance score threshold to screen highly credible evidence paragraphs from candidate evidence paragraphs includes: setting a logical relevance score threshold, selecting candidate evidence paragraphs with scores higher than the threshold from the rearranged candidate evidence paragraph sequence, and using these as the final set of highly credible paragraphs to be retained. The set of highly credible paragraphs includes candidate evidence paragraphs that are most directly related to the scientific question and have logical support.
[0013] Furthermore, the step of generating a summary for a scientific question based on the filtered high-credibility evidence paragraphs involves calling a large language model to generate a summary for the scientific question for each paragraph in the set of high-credibility paragraphs.
[0014] Furthermore, the generated summary should meet the following principles: Key information extraction: Extracting the core information from candidate evidence paragraphs that directly answer the scientific questions; Context preservation: Retain original scientific terms and entity names; Simplicity: Generates logically clear and concise evidence text.
[0015] A retrieval-enhanced scientific question-answering system based on deep evidence reordering, comprising: The intent recognition module is used to parse the input scientific question into a structured representation of the question's intent. The relevance assessment module is used to score the logical relevance of candidate evidence paragraphs based on the intent of the question. The evidence screening module is used to select highly credible evidence paragraphs from candidate evidence paragraphs based on the logical relevance scoring results and using a logical relevance score threshold. The evidence summary module is used to generate scientifically oriented summaries based on filtered, highly credible evidence paragraphs.
[0016] Furthermore, the evidence summary module includes: The key information extraction unit is used to extract the core information from candidate evidence paragraphs that directly answer scientific questions. Terminology preservation unit, used to preserve original scientific terms and entity names; The summary generation unit is used to generate logically clear and concise evidence text.
[0017] The beneficial effects of this invention are as follows: This invention realizes a novel reordering mechanism that combines the reasoning capabilities of large language models with multidimensional semantic understanding. It surpasses traditional vector similarity measures and can dynamically evaluate the logical consistency and evidence reliability of search candidate segments, thereby improving the robustness and credibility of the RAG system in the scientific field.
[0018] The method of this invention requires only one retrieval, sorting the retrieved content according to logical relevance scores and simultaneously filtering it. Existing multi-round retrieval techniques are susceptible to the influence of noisy documents, making the questions generated in subsequent rounds inaccurate and affecting the query and final results. Furthermore, multi-round retrieval significantly increases time costs. The method of this invention eliminates the need for multiple rounds of interaction, simultaneously sorting, filtering, and summarizing relevant documents to remove the influence of noisy documents, thereby improving efficiency and the quality of the generated results.
[0019] This invention uses LLM to perform multi-dimensional analysis of the problem and candidate evidence paragraphs, thereby obtaining more accurate relevance scores. Furthermore, the use of continuous scores better reflects the relationship between the problem and candidate evidence paragraphs, which also facilitates subsequent evidence screening and evidence summarization. Attached Figure Description
[0020] Figure 1 This is a flowchart of the steps of the retrieval enhancement scientific question answering method based on deep evidence reordering of the present invention.
[0021] Figure 2 This is a block diagram of the retrieval enhancement scientific question answering system based on deep evidence reordering of the present invention. Detailed Implementation
[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0023] This invention provides a retrieval-enhanced scientific question-answering method based on deep evidence reordering, such as... Figure 1 As shown, the core technical solution includes the following key steps: 1) Intent Recognition Step: The input scientific question is parsed into a structured representation of the question's intent. This step uses a large language model to identify the subject area, core entity type, cognitive intent, and expected answer type in the scientific question, forming a structured representation that provides a semantic basis for subsequent logical relevance judgments.
[0024] 2) Relevance Assessment Step: Logical relevance scores are assigned to candidate evidence paragraphs. This step leverages the scientific reasoning capabilities of a large language model to structurally score the degree of match between each candidate paragraph and the intent of the question, ensuring that high-scoring paragraphs possess clear causal, mechanistic, or structural evidence, rather than merely superficial semantic similarity.
[0025] 3) Evidence Screening Step: Based on the logical relevance scoring results, highly credible evidence paragraphs are selected from candidate evidence paragraphs according to the logical relevance score threshold. Logically irrelevant, redundant, or speculative paragraphs are automatically removed according to the preset threshold, retaining core evidence that is highly relevant to the question, ensuring that the content input into the generated model has scientific credibility and logical rigor.
[0026] 4) Evidence Summary Step: Generate a summary oriented towards the scientific question based on the selected high-credibility evidence paragraphs. This step extracts key experimental results, variable relationships, or conclusive information while keeping the original scientific terminology and entity names unchanged, generating a concise evidence text of one or two sentences, thereby reducing redundant information and controlling the input length of the generation model.
[0027] Through the above technical solution, the present invention realizes intelligent identification, screening and reordering of deep semantic evidence in scientific question answering tasks, which significantly improves the scientificity, accuracy and interpretability of the generated answers.
[0028] In one embodiment of the present invention, the retrieval-enhanced scientific question answering method based on deep evidence reordering specifically includes the following steps: 1) Problem definition.
[0029] make This indicates that a researcher's query, through a first-stage search (such as a database search or web search), yields results related to the question. The relevant set of candidate evidence paragraphs: , in, Denote the i-th candidate evidence passage, where n is generally large (e.g., n > 100) and each passage contains the text information of the paper.
[0030] The present invention aims to design a retrieval-enhanced scientific question answering method based on deep evidence re-ranking. By comprehensively evaluating the logical relevance between each passage in the candidate evidence passage set and the question , a re-ranking score function is generated: . Among them, is the corresponding relevance score of the candidate evidence passage . Re-rank according to the scores from high to low to obtain a new sequence of candidate evidence passages: , where represents re-ranking. Subsequently, select the top k highly credible evidence passages (where k << n) from the re-ranked candidate evidence passage sequence E′ as the final evidence set: . Finally, the selected evidence set will be input into the generation module to generate an answer to the scientific question : .
[0031] 2) Intent recognition step.
[0032] The intent recognition step analyzes the input scientific question q, extracts a structured query intent representation, and provides scientific semantic constraints for subsequent evidence re-ranking. Specifically, when implemented, the input question q is submitted to a large language model (LLM) for intent recognition to generate a structured representation: . Among them, contains four key elements: . Among them, t represents the scientific topic to which the question belongs, which is used to indicate the discipline or research direction of the question; e represents the core entity type, which is used to determine the key scientific objects involved in the question, such as "cell type: lymphocyte"; i represents the question intent, which indicates the cognitive task that the user hopes to perform, such as definition, mechanism, comparison, causality, fact or functional role, etc.; a represents the expected answer type, which specifies the type of information that the answer should provide, such as text description, numerical result or chart.
[0033] Through this structured intent representation, the subsequent evidence relevance evaluation and screening process can be closely aligned with the scientific topic and research goal of the question, thus avoiding incorrect ranking caused by relying solely on surface semantic matching.
[0034] 3) Relevance evaluation step.
[0035] The relevance evaluation step evaluates each candidate evidence passage in the candidate evidence passage set Structured intent of the input question Logical relevance is scored to determine the degree of match between the candidate paragraph and the question. Specifically, each candidate paragraph is input into a scoring unit, and a logical relevance score is calculated using a Large Language Model (LLM). Among them, the logical relevance score Reflecting candidate evidence paragraphs On the issue The direct evidentiary value is assessed, with higher scores indicating stronger logical relevance. Subsequently, each candidate evidentiary passage and its logical relevance score are grouped into a pairwise set: The sets are ranked from highest to lowest according to their logical relevance scores. The candidate evidence paragraphs are sorted to obtain a rearranged sequence of candidate evidence paragraphs, thereby prioritizing the paragraphs with the most scientific evidence value as input for the next step of screening and summarization.
[0036] 4) Evidence screening steps.
[0037] The evidence screening step selects highly credible evidence paragraphs from the rearranged candidate evidence paragraphs based on relevance scores, thereby removing logically irrelevant or redundant paragraphs. In practice, a logical relevance score threshold is set. And from the sorted set of candidate evidence paragraphs Paragraphs with scores greater than or equal to the threshold are selected as the final set of highly reliable paragraphs to be retained. in, This represents a set of highly reliable paragraphs after filtering, containing information related to the question. The most directly relevant and logically supported paragraphs are selected. This step effectively eliminates noisy paragraphs that do not match the query logic or are only superficially semantically related, providing high-quality input for subsequent evidence summarization.
[0038] 5) Evidence summary steps.
[0039] The evidence summary step involves filtering the set of highly credible paragraphs. A refined summary is performed to reduce redundant information and control the length of the input to the generative model. Specifically, for a set... For each paragraph, a large language model is invoked to generate a response to the question. Abstract The generated abstract should meet the following principles: a) Key information extraction: Extracting the core information from candidate evidence paragraphs that directly answer scientific questions; b) Context preservation: Retain the original scientific terms and entity names without paraphrasing or conceptual substitution to avoid semantic shift; c) Conciseness: Generate logically clear and concise text, usually limited to one or two sentences, i.e., a concise text of evidence in one or two sentences.
[0040] This step yields the final set of evidence. : in, This indicates that a large language model is invoked to generate a solution for the problem. Abstract .gather It includes selected and refined paragraphs, which ensures the logical relevance and credibility of scientific evidence while reducing redundant information, providing high-quality input for subsequent generation modules.
[0041] Another embodiment of the present invention provides a retrieval-enhanced scientific question-answering system based on deep evidence reordering, such as... Figure 2 As shown, it includes: Intent Recognition Module: This module parses the input scientific questions into a structured representation of the question's intent. It uses a large language model to identify the subject area, core entity types, cognitive intent, and expected answer type within the scientific question, forming a structured representation that provides a semantic foundation for subsequent logical relevance judgments.
[0042] The relevance assessment module scores the logical relevance of candidate evidence paragraphs. Based on the scientific reasoning capabilities of a large language model, this module provides a structured score for the degree of match between each candidate paragraph and the question's intent, ensuring that high-scoring paragraphs possess clear causal, mechanistic, or structural evidence, rather than merely superficial semantic similarity.
[0043] Evidence screening module: This module filters highly credible evidence passages from candidate evidence passages based on a logical relevance score threshold. The system automatically removes logically irrelevant, redundant, or speculative passages according to preset thresholds, retaining core evidence highly relevant to the issue, ensuring that the content input into the generated model possesses scientific credibility and logical rigor.
[0044] Evidence Summary Module: This module generates scientifically relevant summaries of highly credible evidence passages after screening. While preserving the original scientific terminology and entity names, it extracts key experimental results, variable relationships, or conclusive information to generate concise evidence text of one or two sentences, thereby reducing redundant information and controlling the input length to the generation model.
[0045] In one embodiment of the present invention, the intent recognition module includes: The intent decomposition unit is used to decompose the input question into structured elements such as scientific topic, core entity, cognitive task type, and expected answer type; The large language model invocation unit is used to extract elements from the input question based on the intent decomposition template and generate a structured representation.
[0046] In one embodiment of the present invention, the correlation evaluation module includes: The scoring unit is used to calculate the relevance score between candidate paragraphs and the structured question representation, giving priority to paragraphs containing experimental, causal, or structured evidence; The sorting unit is used to sort all candidate paragraphs from high to low according to their relevance scores to form a sorted result set.
[0047] In one embodiment of the present invention, the evidence screening module includes: Threshold filtering unit, used to filter paragraphs based on preset or adaptive relevance thresholds.
[0048] In one embodiment of the present invention, the evidence summary module includes: The key information extraction unit is used to extract the core information from candidate evidence paragraphs that directly answer the questions. Terminology preservation units are used to preserve original scientific terms and entity names to avoid semantic drift. The summary generation unit is used to generate logically clear and concise evidence text. Preferably, the generated text is a refined text of one to two sentences in length, serving as the final evidence.
[0049] The above division of modules and units is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules and units as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module and unit can be found in the corresponding process in the aforementioned method embodiments.
[0050] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.
[0051] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.
[0052] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.
[0053] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.
Claims
1. A retrieval-enhanced scientific question-answering method based on deep evidence reordering, characterized in that, Includes the following steps: The input scientific question is parsed into a structured representation of the question's intent. Scoring the logical relevance of candidate evidence paragraphs based on the intended meaning of the question; Based on the logical relevance scoring results, highly credible evidence paragraphs are selected from candidate evidence paragraphs using the logical relevance score threshold. Generate summaries for scientific questions based on the selected high-credibility evidence paragraphs.
2. The method according to claim 1, characterized in that, The process of parsing the input scientific question into a structured question intent representation includes: The input question is submitted to a large language model for intent recognition, generating a structured question intent representation, which contains four key elements: The scientific topic to which the question belongs indicates the discipline or research direction of the question; Core entity types are used to identify the key scientific objects involved in the problem; The question intent instructs the user on the cognitive task they wish to perform. Expected answer type: Specifies the type of information the answer should provide.
3. The method according to claim 2, characterized in that, The step of scoring the logical relevance of candidate evidence paragraphs based on the question intent representation involves the following steps: scoring the logical relevance between each candidate evidence paragraph in the candidate evidence paragraph set and the structured question intent representation. Each candidate evidence paragraph is used to calculate a logical relevance score through a large language model. The logical relevance score reflects the direct evidentiary value of the candidate evidence paragraph to the scientific question. The higher the score, the stronger the logical relevance. Each candidate evidence paragraph and its logical relevance score are combined into a set; The candidate evidence paragraphs in the set are sorted from high to low according to their logical relevance scores to obtain the rearranged candidate evidence paragraph sequence.
4. The method according to claim 3, characterized in that, The step of using a logical relevance score threshold to select highly credible evidence paragraphs from candidate evidence paragraphs includes: setting a logical relevance score threshold, selecting candidate evidence paragraphs with scores higher than the threshold from the rearranged candidate evidence paragraph sequence, and using these as the final set of highly credible paragraphs. The set of highly credible paragraphs includes candidate evidence paragraphs that are most directly related to the scientific question and have logical support.
5. The method according to claim 5, characterized in that, The process of generating a summary for a scientific question based on the filtered high-credibility evidence paragraphs involves calling a large language model to generate a summary for the scientific question for each paragraph in the set of high-credibility paragraphs.
6. The method according to claim 5, characterized in that, The generated summary should meet the following principles: Key information extraction: Extracting the core information from candidate evidence paragraphs that directly answer the scientific questions; Context preservation: Retain original scientific terms and entity names; Simplicity: Generates logically clear and concise evidence text.
7. A retrieval-enhanced scientific question-answering system based on deep evidence reordering, characterized in that, include: The intent recognition module is used to parse the input scientific question into a structured representation of the question's intent. The relevance assessment module is used to score the logical relevance of candidate evidence paragraphs based on the intent of the question. The evidence screening module is used to select highly credible evidence paragraphs from candidate evidence paragraphs based on the logical relevance scoring results and using a logical relevance score threshold. The evidence summary module is used to generate scientifically oriented summaries based on filtered, highly credible evidence paragraphs.
8. The system according to claim 7, characterized in that, The evidence summary module includes: The key information extraction unit is used to extract the core information from candidate evidence paragraphs that directly answer scientific questions. Terminology preservation unit, used to preserve original scientific terms and entity names; The summary generation unit is used to generate logically clear and concise evidence text.
9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.