An intelligent research and learning question and answer method and system based on a three-layer knowledge base and multi-layer search enhancement generation

Abstract

This invention relates to an intelligent research and study question-and-answer method and system based on a three-layer knowledge base and multi-layer retrieval enhancement, belonging to the fields of artificial intelligence, natural language processing, and scientific research knowledge service technology. The method includes: acquiring multi-source data from a research team; performing text extraction, OCR recognition, content cleaning, chapter recognition, and text segmentation on the multi-source data to obtain standardized text data; constructing a three-layer knowledge base based on the standardized text data, consisting of a summary index layer, a structured information layer, and a text block layer; querying and rewriting user research and study questions and recalling multi-layer candidate evidence, and obtaining a final context set through deduplication and re-sorting; generating research and study question-and-answer results based on user questions, the final context set, and evidence source information, and outputting source traceability information. This invention can improve the efficiency of scientific research data retrieval, question-and-answer generation, and knowledge reuse, and enhance the accuracy and traceability of question-and-answer results.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, natural language processing, and knowledge services, specifically to an intelligent research question-and-answer method and system based on a three-layer knowledge base and multi-layer retrieval enhancement generation. This method targets multi-source research materials within research teams, including papers, patents, code documents, process documents, and experiential knowledge. It constructs a three-layer knowledge base consisting of a summary index layer, a structured information layer, and a main text block layer, and combines this with a multi-layer retrieval enhancement generation mechanism to achieve retrieval and recall of research materials, answer generation, and source tracing. Background Technology

[0002] With the development of scientific research informatization and artificial intelligence technologies, university research teams continuously accumulate a large amount of materials such as papers, patents, code, experimental records, process documents, and experience summaries in their daily research activities. These materials include not only the team's existing research results but also research methods, experimental procedures, dataset usage, code execution methods, and project experience, playing a crucial role in helping team members conduct literature review, take over projects, reproduce experiments, and pass on experience. However, in practice, these materials are often scattered across different members, different devices, and different documents, lacking a unified method for organization and retrieval. New members often need to repeatedly consult their supervisors and senior students when searching for historical materials, understanding research directions, or reproducing projects, resulting in low efficiency in utilizing existing knowledge and difficulty in continuously accumulating team experience.

[0003] Currently, traditional keyword retrieval methods primarily rely on literal word matching, which struggles to accurately understand the deep semantics of research questions. When faced with tasks such as comparing research methods, analyzing experimental results, summarizing innovative points, and explaining code processes, inaccurate search results and unclear evidence location are common problems. In recent years, retrieval enhancement and generation technologies have provided new approaches to scientific knowledge question answering. However, existing methods often directly segment documents into fixed-length blocks and then perform retrieval based on single-layer text blocks, easily disrupting the original chapter structure of papers, patents, and process documents, leading to semantic breaks and missing evidence. Furthermore, general-purpose large models may still exhibit issues such as answer illusions, unclear basis, and untraceable results when lacking explicit evidentiary constraints. Therefore, how to construct a unified knowledge organization method for multi-source data from research teams, and combine multi-layered retrieval, candidate evidence screening, and evidence constraint generation mechanisms to improve the accuracy, relevance, and traceability of research question answering results, has become a problem that needs to be addressed in current scientific knowledge services. Summary of the Invention

[0004] To overcome the problems existing in current knowledge services for research teams, such as scattered data, inaccurate search results, low efficiency of knowledge reuse, unclear basis for question-and-answer results, and difficulty in accumulating experiential knowledge, this invention provides an intelligent research-based question-and-answer method and system based on a three-layer knowledge base and multi-layer retrieval enhancement. This method is applicable to multi-source research materials such as papers, patents, code documents, process documents, and experiential questions and answers. By constructing a summary index layer, a structured information layer, and a main text block layer, it achieves multi-granular organization of research materials. Combined with multi-layer retrieval, candidate evidence reordering, and evidence-constrained answer generation, it improves the accuracy, relevance, and traceability of research-based question-and-answer results.

[0005] According to one aspect of the present invention, an intelligent research and study question-answering method based on a three-layer knowledge base and multi-layer retrieval enhancement is provided, specifically including the following steps:

[0006] S1. Obtain multi-source data from the research team;

[0007] S2. Preprocessing multi-source scientific research data;

[0008] S3. Construct a three-layer knowledge base consisting of a summary index layer, a structured information layer, and a main text block layer;

[0009] S4. Enhance question-and-answer generation through multi-layered retrieval based on a three-layered knowledge base;

[0010] S5. Generate research and study Q&A results and output evidence source information;

[0011] S6. Generate academic profiles and academic inheritance Q&A results based on a three-layer knowledge base, and update and maintain the knowledge.

[0012] As described above and in any possible implementation, a further implementation is provided, wherein acquiring multi-source scientific research data in S1 includes:

[0013] S11. Obtain existing journal articles, conference papers, patent documents, project codes, experimental instructions, process records, environment configuration documents, and experience summary materials from the research team;

[0014] S12. Classify and organize research materials from different sources and in different formats, and archive and store them according to material type, upload time, project or research direction.

[0015] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the preprocessing of the multi-source scientific research data in S2 includes:

[0016] S21. Extract text from text-based PDF documents and perform OCR recognition on scanned PDFs or image-based materials to obtain the initial text content;

[0017] S22. The initial text content is cleaned, including removing headers and footers, merging blank lines, repairing abnormal line breaks, removing irrelevant symbols, and standardizing text format;

[0018] S23. Perform chapter structure recognition on the cleaned text, identifying chapter information such as title, abstract, introduction, methods, experiments, conclusions, and references;

[0019] S24. Divide the text into blocks according to the chapter structure to obtain the main text blocks that retain the chapter semantics.

[0020] As described above and in any possible implementation, a further implementation is provided in which the construction of the three-layer knowledge base in S3 includes:

[0021] S31. Construct a summary index layer to generate document summary information for each research document. The document summary information includes one or more of the following: document topic, research direction, main content, and core contributions.

[0022] S32. Construct a structured information layer by extracting structured academic elements from research materials. The structured academic elements include one or more of the following: research questions, research methods, datasets, innovative points, experimental indicators, and performance results.

[0023] S33. Construct a main text block layer, and store the main text content after chapter recognition and block processing as fine-grained evidence text;

[0024] S34. Vectorize the knowledge units in the summary index layer, structured information layer, and main text block layer respectively, and establish corresponding retrieval indexes.

[0025] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the multi-layer retrieval enhancement and question-answering generation based on a three-layer knowledge base in S4 includes:

[0026] S41. Perform text normalization, query rewriting, and semantic expansion on the research questions input by the user to obtain the rewritten query;

[0027] S42. Vectorize the rewritten query and candidate text, and calculate semantic similarity;

[0028] S43. Candidate evidence recall is performed at the summary index layer, structured information layer, and main text block layer, respectively;

[0029] S44. Merge the three-tiered candidate evidence sets and perform deduplication;

[0030] S45. Reorder the candidate evidence by relevance to obtain the final context set.

[0031] As described above and in any possible implementation, a further implementation is provided in which the generation of research and study question-and-answer results and the output of evidence source information in S5 include:

[0032] S51. Organize the user's original question, the final context set, and the evidence source information into generated prompt content;

[0033] S52. Input the generated prompts into the answer generation model to generate study tour Q&A results;

[0034] S53. Associate the research and study Q&A results with the source document, source chapter, structured field, or text block;

[0035] S54. Return the results of the study tour Q&A and the information on the source of the evidence to the user.

[0036] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein generating academic profiles and academic heritage Q&A results and performing knowledge updates and maintenance in S6 includes:

[0037] S61. Construct a system module that includes data acquisition, data preprocessing, three-layer knowledge base construction, multi-layer retrieval, re-ranking, answer generation, evidence tracing, academic profiling, academic inheritance, and knowledge updating;

[0038] S62. Update and maintain the knowledge base content and search index after adding new research materials or receiving user feedback.

[0039] This invention also provides an intelligent research and study question-and-answer system based on a three-layer knowledge base and multi-layer retrieval enhancement, the system being used to implement the above method, the system comprising:

[0040] The data acquisition module is used to acquire multi-source scientific research data such as papers, patents, code data, process documents, experience summaries, and preset Q&A data;

[0041] The data preprocessing module is used to perform text extraction, OCR recognition, text cleaning, chapter recognition, and text segmentation on multi-source scientific research data.

[0042] The three-layer knowledge base construction module is used to build a summary index layer, a structured information layer, and a main text block layer, and to establish corresponding retrieval indexes;

[0043] The multi-layer retrieval module is used to recall candidate evidence in a three-layer knowledge base based on the research questions input by the user.

[0044] The reordering module is used to sort and filter the candidate evidence obtained from the recall based on relevance.

[0045] The answer generation module is used to generate research and study Q&A results based on user questions and filtered candidate evidence;

[0046] The evidence tracing module is used to output the source document, structured fields, or text blocks corresponding to the research and study Q&A results;

[0047] The Academic Profiling module generates profiles based on structured information layers and document metadata, including research direction distribution, research trends, keyword statistics, and author information. The Academic Inheritance module provides Q&A services related to project takeover, environment configuration, code execution, and experiment reproduction, based on code materials, process documents, experimental records, and experience summaries. The Knowledge Update module updates the knowledge base content and search indexes after new research materials are added.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] (1) This invention constructs a summary index layer, a structured information layer, and a main text block layer to organize multi-source materials such as papers, patents, code materials, process documents, and experience summaries in the research team at multiple granularities. Compared with methods that rely solely on single-layer text segmentation and full-text retrieval, this method can better preserve the overall theme, key academic elements, and main text details of research documents, and reduce problems such as scattered data, semantic fragmentation, and inaccurate evidence location.

[0050] (2) This invention utilizes a multi-layered candidate evidence recall and relevance reordering mechanism to filter content relevant to the user's question from document-level summary information, structured academic elements, and main text blocks, respectively. Compared to directly inputting the initial retrieval results into the generative model, this method can further filter weakly relevant text and redundant content, improve the quality of evidence in the input context, and thus enhance the accuracy and relevance of the research and answering results.

[0051] (3) This invention introduces evidence constraints and source tracing mechanisms in the answer generation process, enabling the generated results to be associated with the corresponding documents, chapters, structured fields, or text blocks. Compared with the general large model that directly generates answers, this method can reduce problems such as unclear answer basis, unverifiable content, and knowledge illusion, and improve the credibility and traceability of research and learning Q&A results.

[0052] (4) This invention can integrate papers, patents, code materials, process documents and experience summaries into a unified knowledge service process, providing research teams with support for data retrieval, research Q&A, knowledge reuse and experience transfer. Compared with a single literature retrieval system or a general Q&A system, this method is more in line with the internal knowledge management scenario of research teams, helps to improve the efficiency of research data utilization and reduce the learning and project takeover costs for new members. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the description of the invention and its embodiments to explain the invention, but do not constitute a limitation thereof. In the drawings:

[0054] Figure 1 This is a schematic diagram of the intelligent research and learning question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement according to the present invention.

[0055] Figure 2 This is a schematic diagram of the three-layer knowledge base structure of the present invention.

[0056] Figure 3 This is a schematic diagram of the multi-layered retrieval enhancement question-and-answer generation process of the present invention.

[0057] Figure 4 This is a schematic diagram of the intelligent research and study question-and-answer system generated by the present invention based on a three-layer knowledge base and multi-layer retrieval enhancement. Detailed Implementation

[0058] The present invention will be described in detail below with reference to the accompanying drawings and embodiments, so that those skilled in the art can understand how the present invention solves the problems of scattered knowledge resources in research teams, low efficiency of data reuse, untraceable research and learning Q&A results, and difficulty in accumulating experiential knowledge through technical means such as three-layer knowledge base construction, multi-layer candidate evidence recall, candidate evidence reordering, and evidence-constrained answer generation. It also achieves the technical effect of improving the accuracy, relevance, and traceability of research and learning Q&A. It should be noted that, unless otherwise specified, the various embodiments and technical features of the present invention can be combined with each other, and the resulting technical solutions all fall within the protection scope of the present invention.

[0059] The research and learning question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement includes the following steps:

[0060] S1. Obtain multi-source data from the research team:

[0061] S11. Obtain multi-source research materials accumulated within the research team, including papers, patents, code materials, project process documents, experimental records, environment configuration instructions, experience summary materials, and pre-set question and answer data, etc.

[0062] S12. The research materials obtained in S11 are initially classified and stored according to material type, project, research direction, upload time, and source. Among them, papers and patent materials mainly include information such as title, abstract, main text, author, publication time, and research field; code materials mainly include README files, code comments, operating instructions, configuration files, and interface descriptions; process documents mainly include experimental procedures, environment configuration, project records, problem summaries, and experience descriptions.

[0063] S13. Establish basic metadata records for scientific research materials. The metadata includes material number, material name, material type, source path, uploading user, upload time, research group, and processing status, which are used for subsequent knowledge base construction, material tracking, and system maintenance.

[0064] S2. Preprocessing of multi-source scientific research data:

[0065] S21. Text Extraction and Processing: For text-based PDF documents such as papers and patents, PDF parsing tools are used to extract page text, titles, paragraph content, and basic layout information; for scanned PDFs, image documents, or screenshots, OCR recognition models are used to extract the text content; for code documents, README files, comments, configuration instructions, and operating instructions are extracted; for process documents, effective text related to scientific research knowledge services, such as experimental procedures, project records, environment configurations, and experience summaries, is retained.

[0066] S22. Content cleaning process: The text extracted in S21 is denoising, removing headers and footers, page numbers, duplicate blank lines, abnormal spaces, residual references and unreasonable line breaks, and the text encoding, punctuation format and paragraph boundaries are standardized.

[0067] S23. Chapter Recognition and Processing: Based on the title number, chapter keywords, and paragraph structure of the research document, identify chapters such as abstract, introduction, related work, methods, experiments, results analysis, and conclusions in the document to form text content with chapter attributes;

[0068] S24. Intelligent Block Segmentation: Based on chapter recognition, long documents are segmented into blocks. Segmentation is performed by considering chapter boundaries, paragraph semantics, and text length to ensure that the resulting text blocks maintain semantic integrity as much as possible and reduce semantic breaks caused by fixed-length segmentation.

[0069] S25, Regarding the first One original research document Its preprocessing process is expressed as follows:

[0070] In the formula, Indicates the original research document The standardized text result is obtained after text extraction, content cleaning, chapter recognition, and intelligent segmentation. This indicates the document preprocessing process.

[0071] S3. Construct a three-tiered knowledge base:

[0072] S31. Construct a summary index layer. Perform document-level summarization on the preprocessed research materials to generate summary information reflecting the document's theme, research direction, main content, and core contributions. This summary information is then stored as a document-level index. For the first... Research document Its summary index is represented as:

[0073] In the formula, Document Corresponding summary index information, This indicates the document summary generation process.

[0074] S32. Construct a structured information layer. Extract, summarize, and standardize key academic elements from research documents, such as research questions, research methods, datasets, innovative points, experimental indicators, and performance results, and store them as structured knowledge records. For the first... Research document The structured information extraction process is represented as follows:

[0075] In the formula, Indicates from document The structured information extracted from it, This represents the process of extracting structured information.

[0076] S33. Construct the main text block layer. Store the main text content, after chapter recognition and intelligent segmentation, as fine-grained text blocks, retaining the document number, chapter name, text content, text block order, and source location for each text block. For example, the first... A research document, its main text block set is represented as:

[0077] In the formula, Indicates the first The set of text blocks corresponding to each document. This indicates the first in the document. A block of main text. This indicates the number of text blocks obtained by dividing the document;

[0078] S34. Vectorize the text content in the summary index layer, structured information layer and main text block layer respectively, and store the corresponding vector results in a vector database or vector index structure for subsequent semantic retrieval and candidate evidence recall.

[0079] S35. Establish the relationship between the three-layer knowledge base so that the summary index, structured information and main text blocks can be tracked by document number, chapter number or text block number, thereby supporting the source location and evidence traceability of question and answer results.

[0080] S4. Enhanced question-and-answer generation based on a three-layer knowledge base using multi-layered retrieval:

[0081] S41. Natural Language Problems in Obtaining User Input The query is then rewritten and semantically expanded to obtain the rewritten query statement. :

[0082] In the formula, This refers to the rewritten and expanded query statement. This indicates the query rewriting process;

[0083] S42. Regarding the rewritten query statement The candidate texts in the three-layer knowledge base are vectorized. Let the candidate texts be... The text embedding model is Then the vector representation of the query and candidate text is:

[0084] In the formula, e_ represents the vector representation of the rewritten query. Indicates candidate text Vector representation of;

[0085] S43. Calculate the semantic similarity between the rewritten query and the candidate text, wherein the semantic similarity calculation formula is:

[0086] In the formula, Represents the vector dot product. Represents the vector norm;

[0087] S44. Perform document-level candidate recall in the summary index layer to obtain the candidate result set of the summary index layer. The summary index layer is used to quickly narrow down the scope of documents related to user questions;

[0088] S45. Perform key academic element matching in the structured information layer to obtain a set of candidate results for the structured information layer. The structured information layer is used to match research questions, research methods, datasets, innovative points, and performance metrics.

[0089] S46. Perform fine-grained evidence recall at the text block level to obtain the candidate result set at the text block level. The main text block layer is used to recall method details, experimental setup, result analysis, and conclusion descriptions that are directly related to the user's question;

[0090] S47. Merge the candidate recall results from the summary index layer, structured information layer, and main text block layer to obtain an initial candidate evidence set:

[0091] In the formula, This represents the set of candidate results for the summary index layer. This represents the set of candidate results for the structured information layer. This represents the set of candidate results for the main text block layer. This represents the initial set of candidate evidence retrieved jointly by the three-layer knowledge base.

[0092] S48. Reorder the initial candidate evidence set by relevance. Let the candidate evidence be... ∈ The reordering score is expressed as:

[0093] In the formula, Represents the initial set of candidate evidence. The first in One candidate piece of evidence, Indicate candidate evidence With query statement The relevance score, This indicates the process of reordering candidate evidence.

[0094] S49. To facilitate comparison of the relevance between different candidate pieces of evidence, the re-ranking scores are normalized. Represented as:

[0095] In the formula, Indicate candidate evidence The normalized relevance score, where the denominator represents the initial set of candidate evidence. The sum of the reordering score index values ​​of all candidate evidence;

[0096] S410. Select the top M candidate pieces of evidence with the highest scores based on the normalized scores to form the final context set:

[0097] In the formula, This represents the final set of highly relevant candidate evidence.

[0098] S5. Generate research and study Q&A results and output evidence source information:

[0099] S51, The user's original question Final context set and information on the source of evidence The joint organization generates prompt content. :

[0100] In the formula, This indicates the prompt content for inputting the generative model. This indicates the process of text concatenation and formatting. S52: Generate prompt content, indicating the source information corresponding to the final context set. Input the model to generate the final answer:

[0101] In the formula, Represents a generative model. This indicates the research and study Q&A results generated by the system;

[0102] S53. Add evidence constraints and source tracing mechanisms to the answer generation process so that the generated results can be linked to the corresponding documents, chapters, structured fields or text blocks, reducing the problems of unclear basis and unverifiable content in the answers;

[0103] S54. Return the results of the study tour Q&A and the source information of the evidence to the user.

[0104] As a further functional extension of S6, the intelligent research and study Q&A system may also include an academic profiling module and an academic inheritance module. The academic profiling module is used to generate profiling results such as research direction distribution, research trends, keyword statistics and author information based on the structured information layer and document metadata. The academic inheritance module is used to provide Q&A services such as project takeover, environment configuration, code running and experiment reproduction based on process documents, code materials, experimental records and experience summary materials.

[0105] S6. Build and maintain an intelligent research and study Q&A system:

[0106] S61. Construct an intelligent research and study Q&A system, the system including a user management module, a research and study Q&A module, a knowledge base module, an academic profile module, an academic inheritance module, and a system maintenance module;

[0107] S62. The User Management module is used to complete user registration and login, identity authentication, role identification, and access control; the Research and Study Q&A module is used to receive user questions, invoke the multi-layered search enhancement generation process, and return Q&A results; the Knowledge Base module is used to display document summaries, structured information, and main text evidence content; the Academic Profile module is used to display research directions, research trends, and keyword statistics; the Academic Heritage module is used to provide Q&A services based on process documents, code materials, and experience materials; and the System Maintenance module is used to support the import of research materials, knowledge base updates, log recording, and system operation and maintenance.

[0108] S63. During system operation, record user questions, system answers, sources of candidate evidence, and user feedback information, and incorporate the reviewed feedback into the knowledge base update process;

[0109] S64. When new papers, patents, code materials, or process documents are added, the system re-executes S2 to S3 to preprocess the new materials and add them to the knowledge base. When users report inaccurate answers or insufficient evidence, the system adjusts the relevant candidate evidence, structured information, and prompt word templates to continuously improve the question-and-answer effect.

[0110] Preferably, in step S21, PDF document parsing includes page traversal, text block extraction, layout order arrangement, and text result output.

[0111] Preferably, in step S21, OCR recognition includes text region detection, text direction determination, and character recognition, used to process scanned PDFs, image documents, and screenshots.

[0112] Preferably, in step S22, the content cleaning rules include, but are not limited to, merging consecutive blank lines, merging duplicate spaces, removing page numbers, repairing unreasonable line breaks, and removing residual references.

[0113] Preferably, in step S24, the text is divided into blocks not by a simple fixed length, but by combining chapter boundaries, paragraph structure, and semantic integrity.

[0114] Preferably, in step S31, the summary index layer is generated by a large language model or a summary generation model, and retains the document number, document title, research field, and summary content.

[0115] Preferably, in step S32, the structured information layer includes at least one or more of the following: research question, research method, dataset, innovation point, experimental index, and performance result.

[0116] Preferably, in step S34, the vectorized representation can be implemented using a text embedding model, and the vector index can be implemented using a vector database or an approximate nearest neighbor retrieval structure.

[0117] Preferably, in step S48, the candidate evidence reordering model takes the query statement and candidate evidence text pairs as input and outputs the relevance score between them.

[0118] Preferably, in step S52, the generated model can be a large language model adapted to the scenario, and its adaptability to scientific research question-and-answer scenarios can be improved through LoRA fine-tuning.

[0119] Preferably, in step S53, the source tracing information includes one or more of the following: document name, chapter name, structured fields, text block number, and original data path.

[0120] The present invention is illustrated by the following embodiments:

[0121] Example

[0122] This invention relates to an intelligent research and study question-answering method based on a three-layer knowledge base and multi-layer retrieval enhancement, the process of which is as follows: Figure 1 As shown, it includes:

[0123] S1. Obtain multi-source data from the research team:

[0124] S11. Obtain multi-source research data accumulated within the research team, including papers, patents, code data, project process documents, experimental records, environment configuration instructions, experience summary materials, and pre-set question and answer data, etc.

[0125] For example, in one specific embodiment, the system acquires approximately 140 journal and conference papers, nearly 40 patent documents, several project code materials, experimental process documents, and approximately 800 pre-set question-and-answer data accumulated by a research team; among them, the papers and patent documents are mainly used to construct a research and study question-and-answer knowledge base and a structured knowledge base, the code materials and process documents are mainly used to construct an academic inheritance knowledge base, and the pre-set question-and-answer data is used to supplement common question-and-answer scenarios;

[0126] S12. The research materials obtained in S11 are initially classified and stored according to material type, project, research direction, upload time, and source. Among them, paper and patent materials mainly include information such as title, abstract, main text, author, publication time, and research field; code materials mainly include README files, code comments, running instructions, configuration files, and interface descriptions; process documents mainly include experimental procedures, environment configuration, project records, problem summaries, and experience descriptions.

[0127] S13. Establish basic metadata records for research materials, including material number, material name, material type, source path, uploading user, upload time, research group, and processing status; for example, for the first... One original research document Establish corresponding metadata records :

[0128] In the formula, Indicates the document number. Indicates the name of the data. Indicates the data type. Indicates the source path of the data. Indicates the user who uploaded the content. Indicates the upload time. Indicates the research group to which they belong. Indicates the processing status.

[0129] S2. Preprocessing of multi-source scientific research data:

[0130] S21. Text Extraction and Processing: For text-based PDF documents such as papers and patents, PDF parsing tools are used to extract page text, titles, paragraph content, and basic layout information; for scanned PDFs, image documents, or screenshots, OCR recognition models are used to extract the text content; for code documents, README files, comments, configuration instructions, and operating instructions are extracted; for process documents, effective text related to scientific research knowledge services, such as experimental procedures, project records, environment configurations, and experience summaries, is retained.

[0131] S22. Content cleaning process: The text extracted in S21 is denoising, removing headers and footers, page numbers, duplicate blank lines, abnormal spaces, residual references and unreasonable line breaks, and the text encoding, punctuation format and paragraph boundaries are standardized.

[0132] For example, for the extracted original text It can be achieved through the cleaning function. The cleaned text is obtained:

[0133] In the formula, Indicates from the first The original text extracted from a research document. This indicates the cleaned text. This indicates the text cleaning process;

[0134] S23. Chapter Recognition and Processing: Based on the title number, chapter keywords, and paragraph structure of the research document, identify chapters such as abstract, introduction, related work, methods, experiments, results analysis, and conclusions in the document to form text content with chapter attributes;

[0135] S24. Intelligent block processing: Based on chapter recognition, long documents are divided into blocks. When dividing the blocks, chapter boundaries, paragraph semantics and text length are combined to make the resulting text blocks maintain semantic integrity as much as possible and reduce the semantic breakage problem caused by fixed-length segmentation.

[0136] S25, Regarding the first One original research document Its preprocessing process is expressed as follows:

[0137] In the formula, Indicates the original research document The standardized text result is obtained after text extraction, content cleaning, chapter recognition, and intelligent segmentation. This indicates the document preprocessing process.

[0138] S3. Construct a three-tiered knowledge base:

[0139] S31. Constructing a summary index layer: Document-level summarization is performed on preprocessed research materials to generate summary information reflecting the document's theme, research direction, main content, and core contributions. This summary information is then stored as a document-level index. For the first... Research document Its summary index is represented as:

[0140] In the formula, Document Corresponding summary index information, This indicates the document summary generation process;

[0141] For example, for a paper on retrieval enhancement generation methods, the summary index layer can save information such as its research topic, research background, core methods, experimental conclusions and applicable scenarios, which can be used to quickly locate the relevant document range according to user questions later;

[0142] S32. Constructing a structured information layer: Extracting, summarizing, and standardizing key academic elements from research documents, such as research questions, research methods, datasets, innovative points, experimental indicators, and performance results, and storing them as structured knowledge records; for the first... Research document The structured information extraction process is represented as follows:

[0143] In the formula, Indicates from document The structured information extracted from it, This represents the process of extracting structured information.

[0144] In one specific embodiment, structured information It can be represented as:

[0145] In the formula, Indicates the research question, Indicates the research method. Represents a dataset, Indicate the innovative point, Indicates evaluation indicators, Indicates performance results;

[0146] S33. Construct the main text block layer: Store the main text content after chapter recognition and intelligent segmentation as fine-grained text blocks, retaining the document number, chapter name, text content, text block order, and source location corresponding to each text block; for the first... A research document, its main text block set is represented as:

[0147] In the formula, Indicates the first The set of text blocks corresponding to each document. This indicates the first in the document. A block of main text. S34. Represent the number of text blocks obtained by dividing the document; S35. Vectorize the text content in the summary index layer, structured information layer, and main text block layer respectively, and store the corresponding vector results in a vector database or vector index structure; Suppose the text embedding model is... The vector representations of the summary index, structured information, and main text blocks are as follows:

[0148] In the formula, Summary Index The vector representation of , Representing structured information The vector representation of , Indicates the main text block Vector representation of;

[0149] S35. Establish the relationships between the three-tiered knowledge base so that the summary index, structured information, and main text blocks can all be tracked by document number, chapter number, or text block number; for example, the summary index. Structured information and the collection of text blocks All are related to document number This association supports the identification of the source of the question-and-answer results and the tracing of evidence.

[0150] S4. Enhanced question-and-answer generation based on a three-layer knowledge base using multi-layered retrieval:

[0151] S41. Natural Language Problems in Obtaining User Input The query is then rewritten and semantically expanded to obtain the rewritten query statement. :

[0152] In the formula, This refers to the rewritten and expanded query statement. This indicates the query rewriting process;

[0153] For example, when a user enters "What methods did this paper use?", the system can rewrite it based on the current document context as "Query the research methods, model structure, core technical routes and main innovations of the current paper";

[0154] S42. Regarding the rewritten query statement Vectorize the candidate texts in the three-layer knowledge base; let the candidate texts be... The text embedding model is Then the vector representation of the query and candidate text is:

[0155] In the formula, e_ represents the vector representation of the rewritten query. Indicates candidate text Vector representation of;

[0156] S43, Calculate and rewrite the query With candidate text The semantic similarity between them is calculated using the following formula:

[0157] In the formula, Represents the vector dot product. Represents the vector norm;

[0158] S44. Perform document-level candidate recall in the summary index layer to obtain the candidate result set of the summary index layer. The summary index layer is used to quickly narrow down the range of documents related to user questions;

[0159] S45. Perform key academic element matching in the structured information layer to obtain a candidate result set for the structured information layer. The structured information layer is used to match research questions, research methods, datasets, innovative points, and performance indicators, etc.

[0160] S46. Perform fine-grained evidence recall at the text block level to obtain the candidate result set at the text block level. The main text block layer is used to recall method details, experimental setup, result analysis, and conclusion descriptions that are directly related to user questions;

[0161] S47. Merge the candidate recall results from the summary index layer, structured information layer, and main text block layer to obtain an initial candidate evidence set:

[0162] In the formula, This represents the set of candidate results for the summary index layer. This represents the set of candidate results for the structured information layer. This represents the set of candidate results for the main text block layer. This represents the initial set of candidate evidence retrieved jointly by the three-layer knowledge base.

[0163] S48. Reorder the initial candidate evidence set according to relevance; let the candidate evidence be... ∈ The reordering score is expressed as:

[0164] In the formula, Represents the initial set of candidate evidence. The first in One candidate piece of evidence, Indicate candidate evidence With query statement The relevance score, This indicates the process of reordering candidate evidence.

[0165] S49. Furthermore, to facilitate comparison of the relevance between different candidate pieces of evidence, the re-ranking scores can be normalized. Represented as:

[0166] In the formula, Indicate candidate evidence The normalized relevance score, where the denominator represents the initial set of candidate evidence. The sum of the reordering score index values ​​of all candidate evidence;

[0167] S410. Select the top M candidate pieces of evidence with the highest scores based on the normalized scores to form the final context set:

[0168] In the formula, This represents the final set of highly relevant candidate evidence.

[0169] S5. Generate research and study Q&A results and output evidence source information:

[0170] S51, The user's original question Final context set and information on the source of evidence The joint organization generates prompt content. :

[0171] In the formula, This indicates the prompt content for inputting the generative model. This indicates the process of text concatenation and formatting. This indicates the source information corresponding to the final context set;

[0172] S52, will generate prompt content Input the model to generate the final answer:

[0173] In the formula, Represents a generative model. This indicates the research and study Q&A results generated by the system;

[0174] S53. Incorporate evidence constraints and source tracing mechanisms into the answer generation process to link the generated results to corresponding documents, chapters, structured fields, or text blocks; for example, when a user asks "What are the innovative points of a certain paper?", the system prioritizes retrieving information from the structured information layer. The field recalls relevant content and generates an answer by combining the method description and result analysis in the main text block layer, while also returning the corresponding document name, chapter name, and text block number.

[0175] S54. Return the results of the study tour Q&A and the source information of the evidence to the user.

[0176] As a further functional extension of S6, the intelligent research and study question-and-answer system can generate academic profile results based on the structured information layer and document metadata; in a specific embodiment, the system can count the number of documents corresponding to each research direction, let the first... The document collection corresponding to each research direction is Then the proportion of this research direction Represented as:

[0177] In the formula, Indicates belonging to the first A collection of documents in each research area. This indicates the number of documents corresponding to this research direction, where D represents the document collection of the research team. This indicates the total number of documents submitted by the research team. Indicates the first The percentage of documents in each research area.

[0178] Furthermore, the academic profile module can organize the distribution of research directions, changes in research hotspots, keyword statistics, and author-related information in the form of charts, statistical results, and text descriptions, allowing users to view the distribution of team research fields and research trends;

[0179] Furthermore, the academic inheritance module can construct the scope of academic inheritance knowledge based on process documents, code materials, experimental records, and experience summary materials, enabling the system to provide Q&A support around project takeover, environment configuration, code execution, experiment reproduction, and handling of common problems;

[0180] When a user asks a question related to academic succession, the system prioritizes retrieving candidate evidence from process documents, code materials, and experience materials, and combines this with a generative model to output a project-practice-oriented answer. For example, when a user enters "How do I configure the environment for this project?", the system prioritizes retrieving environment configuration instructions, README files, and summaries of historical issues to generate an answer that includes dependency installation, running commands, common errors, and solutions.

[0181] S6. Build and maintain an intelligent research and study Q&A system:

[0182] S61. Construct an intelligent research and study Q&A system, the system including a user management module, a research and study Q&A module, a knowledge base module, an academic profile module, an academic inheritance module, and a system maintenance module;

[0183] S62. The User Management module is used to complete user registration and login, identity authentication, role identification, and access control; the Research and Study Q&A module is used to receive user questions, invoke the multi-layered search enhancement generation process, and return Q&A results; the Knowledge Base module is used to display document summaries, structured information, and main text evidence content; the Academic Profile module is used to display research directions, research trends, and keyword statistics; the Academic Heritage module is used to provide Q&A services around process documents, code materials, and experience materials; and the System Maintenance module is used to support the import of research materials, knowledge base updates, log recording, and system operation and maintenance.

[0184] S63. During system operation, record user questions, system answers, sources of candidate evidence, and user feedback information, and incorporate the reviewed feedback into the knowledge base update process;

[0185] S64. When new papers, patents, code materials, or process documents are added, the system re-executes S2 to S3 to preprocess the new materials and add them to the knowledge base. When users report inaccurate answers or insufficient evidence, the system adjusts the relevant candidate evidence, structured information, and prompt word templates to continuously improve the question-and-answer effect.

[0186] As an embodiment of the present invention, the present invention also provides an intelligent research and study question-and-answer system based on a three-layer knowledge base and multi-layer retrieval enhancement generation. The system is used to implement the above method and includes:

[0187] The data acquisition module is used to acquire multi-source research data from within the research team, including papers, patents, code data, project process documents, experimental records, environment configuration instructions, experience summary materials, and preset question and answer data.

[0188] The data preprocessing module is used to perform text extraction, OCR recognition, content cleaning, chapter recognition, and intelligent block segmentation on the acquired multi-source scientific research data to obtain standardized text results. ;

[0189] A three-tiered knowledge base construction module is used to build upon normalized text results. Construct a summary index layer, a structured information layer, and a main text block layer, and generate summary indexes for each layer. Structured information and the collection of text blocks ;

[0190] The vector retrieval module is used to retrieve user questions. Rewrite query The candidate texts in the three-layer knowledge base are vectorized and candidate evidence is recalled based on semantic similarity.

[0191] The reordering module is used to recall candidate evidence sets from the summary index layer, structured information layer, and main text block layer. The relevance is reordered, and candidate evidence with higher scores is selected to form the final context set. ;

[0192] The answer generation module is used to generate the user's original question. Final context set Information on the source of evidence The organization generates prompt content. The results of the research and study Q&A were generated by inputting the generative model. ;

[0193] The evidence tracing module is used to output the research and study Q&A results. The corresponding document name, chapter name, structured fields, text block number, and original data path are one or more source information.

[0194] The academic profiling module is used to generate profile results such as research direction distribution, research trends, keyword statistics, and author information based on the structured information layer and document metadata.

[0195] The academic succession module provides Q&A services related to project handover, environment configuration, code execution, and experiment reproduction, focusing on code materials, process documents, experimental records, and experience summary materials.

[0196] The knowledge update module is used to re-perform preprocessing, knowledge extraction, vectorized indexing, and knowledge base update operations on relevant data after adding new research data or receiving user feedback.

[0197] In this embodiment, users can input natural language questions at the system front end, such as "What is the core innovation of a certain paper?", "How to configure the operating environment for this project?", and "What are the main research directions of the research group in recent years?". After receiving the question, the system first queries and rewrites the question, then recalls candidate evidence in the summary index layer, structured information layer, and main text block layer, and finally filters the final context set through the re-sorting module. Subsequently, the answer generation module generates answers based on user questions. Final context set Information on the source of evidence Generate answer The system then returns the answer and its source to the user. In this way, the system can support data retrieval, research Q&A, academic profiling, and experience transfer in multi-source data scenarios within research teams, while improving the accuracy, relevance, and traceability of the Q&A results.

[0198] The foregoing description illustrates and describes several preferred embodiments of the present invention. However, as mentioned above, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments. Any alterations and changes made by those skilled in the art based on the foregoing without departing from the spirit and scope of the present invention should fall within the protection scope of the claims of the present invention.

Claims

1. A method for intelligent research and learning question answering based on a three-layer knowledge base and multi-layer retrieval enhancement, characterized in that, Includes the following steps: S1. Obtain multi-source data from the research team, including one or more of the following: papers, patents, code data, experimental records, process documents, experience summaries, and pre-set question and answer data; S2. Preprocess the multi-source data to obtain standardized text data. The preprocessing includes text extraction, OCR recognition, content cleaning, chapter recognition, and text segmentation. S3. Construct a three-layer knowledge base based on the standardized text data. The three-layer knowledge base includes a summary index layer, a structured information layer, and a main text block layer, and establish corresponding retrieval indexes for each layer. S4. Receive research questions input by the user, perform multi-layer retrieval enhancement based on the three-layer knowledge base to generate questions and answers, and obtain the final context set; S5. Generate research and study Q&A results based on the user's original question, the final context set, and the evidence source information, and output the evidence source information corresponding to the research and study Q&A results; S6. Generate academic profile results based on the structured information layer and document metadata, provide academic inheritance Q&A support based on process documents, code materials, experimental records and experience summary materials, and update the knowledge base content and search index after adding new scientific research materials or receiving user feedback.

2. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, is characterized in that, The step S1 of obtaining multi-source data from the research team specifically includes: obtaining existing journal articles, conference papers, patent documents, project codes, experimental instructions, process records, environment configuration documents, and experience summary materials from the research team; classifying and archiving research data from different sources and in different formats according to data type, upload time, and project or research direction; and establishing basic metadata records for the research data, which include data number, data name, data type, source path, uploader, upload time, research group, and processing status.

3. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, is characterized in that, The preprocessing of the multi-source data in step S2 specifically includes: extracting page text, titles, paragraph content, and basic layout information from text-based PDF documents; performing OCR recognition on scanned PDFs, image documents, or screenshots; extracting text from README files, code comments, configuration instructions, and operating instructions in code documents; cleaning the extracted text by removing headers, footers, page numbers, duplicate blank lines, abnormal spaces, residual references, and unreasonable line breaks; identifying chapter information such as abstracts, introductions, related work, methods, experiments, results analysis, and conclusions based on title numbers, chapter keywords, and paragraph structure; and, based on chapter identification, dividing long documents into blocks according to chapter boundaries, paragraph semantics, and text length, so that the resulting text blocks retain chapter attributes, document numbers, chapter names, text content, text block order, and source location, thereby reducing semantic breaks caused by fixed-length segmentation.

4. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, characterized in that, The construction of the three-layer knowledge base in step S3 specifically includes: constructing a summary index layer, which performs document-level summarization on the preprocessed research materials to generate summary information reflecting the document's theme, research direction, main content, and core contributions; constructing a structured information layer, which extracts one or more structured academic elements from the research materials, such as research questions, research methods, datasets, innovative points, experimental indicators, and performance results; and constructing a main text block layer, which stores the main text content after chapter recognition and main text block processing as fine-grained evidence text.

5. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, characterized in that, The establishment of the corresponding retrieval index in step S3 specifically includes: vectorizing the text content in the summary index layer, structured information layer, and main text block layer respectively, and storing the corresponding vector results in a vector database or vector index structure; at the same time, establishing the association between the summary index, structured information, and main text blocks so that they can be tracked by document number, chapter number, or text block number.

6. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, characterized in that, Step S4, which involves multi-layered retrieval enhancement and question-answer generation based on the three-layered knowledge base, specifically includes: normalizing the text of the research question input by the user, rewriting the query, and semantically expanding it to obtain a rewritten query; vectorizing the rewritten query and candidate texts and calculating the semantic similarity between them; recalling candidate evidence in the summary index layer, the structured information layer, and the main text block layer respectively to obtain candidate result sets for the summary index layer, the structured information layer, and the main text block layer; merging and deduplicating the three candidate result sets to obtain an initial candidate evidence set.

7. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 6, is characterized in that, The specific steps in step S4 to obtain the final context set include: reordering the candidate evidence in the initial candidate evidence set according to their relevance to obtain a relevance score between each candidate evidence and the rewritten query; normalizing the relevance scores; and selecting the top M candidate evidence with the highest scores based on the normalized scores to form the final context set.

8. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, characterized in that, Step S5, which generates research and study Q&A results and outputs evidence source information, specifically includes: organizing the user's original question, the final context set, and the evidence source information into generated prompt content; inputting the generated prompt content into the answer generation model to generate research and study Q&A results; adding evidence constraints and source tracing mechanisms during the answer generation process to associate the generated results with corresponding documents, chapters, structured fields, or text blocks; and returning the research and study Q&A results and evidence source information to the user.

9. The intelligent research and study question-and-answer method based on a three-layer knowledge base and multi-layer retrieval enhancement as described in claim 1, characterized in that, Step S6 specifically includes: generating academic profile results based on the structured information layer and document metadata, wherein the academic profile results include one or more of the following: research direction distribution, research trends, keyword statistics, and author information; constructing the scope of academic inheritance knowledge based on process documents, code materials, experimental records, and experience summary materials, and providing Q&A support around project takeover, environment configuration, code execution, experiment reproduction, and handling of common problems; and re-performing preprocessing, knowledge extraction, vectorized indexing, and knowledge base update operations on the relevant materials after adding new research materials or receiving user feedback.

10. An intelligent research and study question-and-answer system based on a three-layer knowledge base and multi-layer retrieval enhancement generation, implementing the method as described in any one of claims 1 to 9, characterized in that, include: The data acquisition module is used to acquire papers, patents, code materials, project process documents, experimental records, environment configuration instructions, experience summary materials, and preset Q&A data within the research team. The data preprocessing module is used to perform text extraction, OCR recognition, content cleaning, chapter recognition, and text segmentation on the acquired multi-source data to obtain standardized text data. The three-layer knowledge base construction module is used to build a summary index layer, a structured information layer, and a main text block layer based on standardized text data, and to establish corresponding retrieval indexes for each layer. The multi-layer retrieval module is used to vectorize user questions, rewritten queries, and candidate texts in the three-layer knowledge base, and to recall candidate evidence based on semantic similarity. The reordering module is used to reorder the candidate evidence sets recalled from the summary index layer, structured information layer and main text block layer based on relevance, and select the candidate evidence with higher scores to form the final context set. The answer generation module is used to organize the user's original question, the final context set, and the evidence source information into generated prompts, and input them into the answer generation model to generate research and study Q&A results; The evidence tracing module is used to output one or more source information from the document name, chapter name, structured fields, text block number and original data path corresponding to the research and study Q&A results; The academic profiling module is used to generate research direction distribution, research trends, keyword statistics, and author information based on the structured information layer and document metadata. The academic succession module provides Q&A services related to project handover, environment configuration, code execution, and experiment reproduction, focusing on code materials, process documents, experimental records, and experience summaries. The knowledge update module is used to re-perform preprocessing, knowledge extraction, vectorized indexing, and knowledge base update operations on relevant data after adding new research data or receiving user feedback.

Images