A RAG intelligent retrieval question and answer system and method based on enhanced metadata

Abstract

The application discloses an RAG intelligent retrieval question and answer system and method based on enhanced metadata, relates to the technical field of information processing and intelligent retrieval, and obtains unified knowledge data after preprocessing of multi-source heterogeneous knowledge data, extracts structured metadata from the unified knowledge data based on the difference in text form; the literature text in the structured metadata is vectorized in combination with embedding of a knowledge graph to obtain document representation, and then the document representation, the structured metadata and an enhanced keyword set are output as an enhanced metadata object; a display relationship between different enhanced metadata objects is marked, and a knowledge database is constructed; restrictive conditions and a question intention are extracted from a user question, mixed retrieval is performed from the knowledge database based on the restrictive conditions and the question intention, a candidate literature semantic set is output, and then a structured statistical visual report and a structured answer are output, so that precise, interpretable and multifunctional intelligent knowledge service is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information processing and intelligent retrieval technology, and in particular to a RAG intelligent retrieval question and answer system and method based on enhanced metadata. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the continuous development of subject development, knowledge explosion and digitization, knowledge resources such as scientific research literature, patent technology and degree papers show characteristics such as rapid growth in quantity, diversification in source and diversification in content. How to effectively organize and intelligently utilize a large amount of knowledge resources has become an important technical direction in the field of academic research and knowledge service.

[0004] In the prior art, academic literature retrieval systems are generally based on information retrieval principles, and through indexing construction and feature extraction of literature content, fast retrieval and relevance sorting of target literature are realized. Such systems generally take keywords or subject headings as the core, and through the establishment of inverted indexes, the calculation of term frequency weights and the scoring of relevance, user queries and literature content are matched to realize directional retrieval and result sorting of information. With the development of retrieval technology, some systems gradually introduce semantic analysis and natural language processing methods to understand and expand query intent, in order to improve the accuracy of retrieval results and expand the coverage.

[0005] On this basis, intelligent question and answer technology has gradually become an important development direction of literature retrieval and knowledge service. This technology is based on natural language processing and realizes automatic answering to specific questions through semantic analysis, knowledge association and content generation of query text. In recent years, retrieval augmented generation (RAG) technology, as a technology framework combining information retrieval and generation model, has been widely used in knowledge question and answer, information retrieval and academic literature analysis. This technology introduces external knowledge sources before generating answers to enhance the knowledge support ability of language models, thereby improving the relevance and reliability of question and answer results.

[0006] Existing knowledge question and answer systems are generally based on semantic analysis and vectorization of text content. Such systems usually process documents or corpora by segmenting, convert text content into high-dimensional semantic vectors, and use vector similarity calculation to realize information retrieval and matching. With the development of text embedding models and vector database technology, question and answer systems based on semantic vector retrieval have been widely used in general knowledge services, product information retrieval and academic literature analysis.

[0007] In the literature-based knowledge Q&A scenario, the system takes the full text or abstract content of the literature as the main retrieval object, calculates the semantic relevance between the user query and the literature text through the embedding model, to determine the candidate literature or generate the Q&A result. Some systems also combine keyword matching, statistical learning, and weighted ranking methods to comprehensively evaluate and optimize the retrieval results, thereby improving the accuracy and context relevance of the Q&A output.

[0008] Overall, the existing technology has formed a general implementation framework based on semantic vector retrieval in the field of literature knowledge Q&A, and through the combination of external knowledge bases and language generation models, it provides effective technical support for multi-domain knowledge services. However, there are still some technical limitations in knowledge organization and Q&A applications for specific disciplines:

[0009] Existing academic literature retrieval systems mostly use subject headings or author-provided keywords as the main retrieval basis. Due to differences in research fields, language expression habits, and cognitive backgrounds among different authors, even for the same research topic, different keywords may be used to describe it, leading to the widespread existence of synonyms and polysemy. At the same time, in keyword selection, some authors select representative methods as keywords, while others select research fields as keywords. This inconsistency in keyword indexing reduces the retrieval system's ability to uniformly express literature topics and affects the recall rate and accuracy of the retrieval results.

[0010] RAG-based knowledge Q&A systems usually use unstructured text content (such as the main text and abstract) as the main information source, and the utilization of structured metadata of literature (including author, affiliated institution, publication time, research direction, and keywords) is limited. Since the retrieval process mainly relies on vector space models for semantic similarity matching, the system lacks precise identification and constraints on literature attributes (such as publication time and source) in result determination, and is easily affected by semantic drift and context bias, leading to deficiencies in domain-specificity and knowledge accuracy of the generated results.

[0011] In addition, the knowledge base relied on by the knowledge Q&A system is relatively single, usually only covering text corpus of uniform format, lacking the ability to integrate and associate multiple sources of knowledge. For knowledge resources of different sources (such as journal papers, conference papers, degree papers, patent information, and ancient book data), existing systems are not perfect in data collection, format analysis, and semantic unification, making it difficult to achieve comprehensive utilization of cross-source knowledge. At the same time, there are still significant obstacles in understanding and retrieving multi-language literature, and the cross-language semantic mapping capability is insufficient, limiting the application effect of the system in international academic knowledge Q&A scenarios.

[0012] Further, the prior art also has deficiencies in user interaction and knowledge presentation. When the user has a cross-disciplinary background, it is often difficult to accurately understand the terminology and professional expressions in the field, affecting the efficiency of information acquisition. For the case where the search results contain multiple articles, the user needs to read each article to obtain effective knowledge, which is time-consuming and has low conversion rate. At the same time, the existing system lacks intuitive display methods such as tabularization and visualization, and it is difficult to provide the user with a overall understanding of the research theme, method distribution or time evolution, and the interaction and interpretability need to be improved.

[0013] In summary, the existing literature retrieval and knowledge question answering system has deficiencies in keyword indexing uniformity, structured metadata utilization, multi-source and multi-language knowledge integration, and user interaction experience, making it difficult to efficiently, accurately and intuitively acquire knowledge in a specific discipline. SUMMARY

[0014] To overcome the problems of information integration difficulty, search accuracy, depth deficiency and single question and answer mode of the RAG system in processing field knowledge in the prior art, the present application provides a RAG intelligent retrieval question and answer system and method based on enhanced metadata, which realizes accurate, interpretable and multifunctional intelligent knowledge service by constructing a deep structured knowledge database and using a hybrid retrieval and dual-mode response mechanism.

[0015] To achieve the above object, one or more embodiments of the present application provide the following technical solutions:

[0016] In a first aspect, the present application provides a RAG intelligent retrieval question and answer system based on enhanced metadata, comprising:

[0017] A multi-source data acquisition module for acquiring multi-source heterogeneous knowledge data and preprocessing the same to obtain unified knowledge data;

[0018] A metadata construction module for extracting structured metadata from the unified knowledge data based on different text forms;

[0019] An enhanced metadata construction module for vectorizing the literature text in the structured metadata in combination with the embedding of the knowledge graph to obtain document representation, and then outputting the document representation, structured metadata and enhanced keyword set as an enhanced metadata object; the literature text includes at least one of an abstract and a body text;

[0020] A knowledge database construction module for annotating and storing the display relationship between different enhanced metadata objects to construct a knowledge database;

[0021] The dual retrieval module is configured to receive a user question, extract a restrictive condition and a question intention from the user question, perform mixed retrieval from a knowledge database based on the restrictive condition and the question intention, and output a candidate literature semantic set;

[0022] The dual-mode reply module is configured to output a structured statistical visualization report and a structured answer based on the candidate literature semantic set after semantic refinement.

[0023] In a further technical solution, the enhanced metadata construction module comprises a keyword enhancement extraction module, a knowledge graph construction module, and a knowledge graph embedded text vectorization module;

[0024] The keyword enhancement extraction module is configured to extract enhanced keywords from structured metadata using a large language model; the enhanced keywords include theme keywords, subject classifications, and method keywords.

[0025] The knowledge graph construction module is configured to establish a multi-level subject node structure according to the subject classifications, embed the theme keywords and the method keywords into the multi-level subject node structure, and obtain a knowledge graph.

[0026] The knowledge graph embedded text vectorization module is configured to vectorize literature texts in structured metadata in combination with embedding of the knowledge graph, and obtain document representations.

[0027] In a further technical solution, embedding the theme keywords and the method keywords into the multi-level subject node structure specifically comprises:

[0028] When the theme keywords have semantic intersections among multiple subjects, a semantic similarity between the theme keywords and the subject nodes is calculated, and a theme-subject relationship edge is constructed with the correlation strength as a weight.

[0029] Based on the enhanced keywords extracted for each literature, a co-occurrence frequency of the theme keywords and the method keywords is calculated, the frequency is normalized, a threshold is set, and a theme-method relationship edge is constructed.

[0030] In a further technical solution, the knowledge graph embedded text vectorization specifically comprises:

[0031] The literature texts are divided into multiple sentences, and knowledge entity recognition is performed on each sentence to obtain a candidate entity set.

[0032] Each candidate entity is mapped to a node in the knowledge graph to obtain a sentence-level target node.

[0033] Based on the sentence-level target node, a sentence-level local subgraph and a node vector matrix are constructed.

[0034] The node information of sentences and sentence-level local subgraphs in the literature text is fused and input into the encoder. The sentence-level enhanced representation is obtained by using the joint mechanism of semantic attention and knowledge attention.

[0035] The sentence-level enhanced representations are aggregated to obtain the document representation.

[0036] A further technical solution involves mapping each candidate entity to nodes in the knowledge graph, resulting in sentence-level target nodes, specifically:

[0037] The semantic vectors of each candidate entity are matched with the semantic vectors of nodes in the knowledge graph. If they match, the corresponding node is taken as the sentence-level target node.

[0038] If they are inconsistent, the cosine similarity between the semantic vector of the candidate entity and the semantic vector of the node in the knowledge graph is calculated. The nodes are sorted from largest to smallest according to the cosine similarity, and the first set number of nodes are selected as candidate nodes. The comprehensive score of the candidate nodes is calculated based on the candidate entities, and the node with the highest comprehensive score is selected as the sentence-level target node.

[0039] A further technical solution involves calculating a comprehensive score to determine the sentence-level target node. The comprehensive score is expressed as follows:

[0040]

[0041] in, This represents the overall score. Indicates candidate entities, Nodes representing a knowledge graph This represents the semantic similarity weight coefficient. Represents the cosine similarity between candidate entities and nodes. This indicates the positional weight of an entity within a sentence. This indicates the frequency of an entity in the full text of the document. Indicates the prior importance of nodes. , , These represent the adjustment coefficients for position weight, word frequency weight, and node prior weight, respectively.

[0042] A further technical solution is that the hybrid retrieval specifically involves: performing a structured query in the knowledge database based on restrictive conditions, and using metadata fields to filter out a precise subset of knowledge;

[0043] The intent of the question is vectorized to obtain an intent vector. Based on the intent vector, a vector retrieval operation is performed in the precise knowledge subset to output a semantic set of candidate documents.

[0044] Secondly, the present invention provides a RAG intelligent retrieval and question answering method based on enhanced metadata, including:

[0045] Acquire multi-source heterogeneous knowledge data and preprocess it to obtain unified knowledge data;

[0046] Based on the different text formats, structured metadata is extracted from unified knowledge data;

[0047] By embedding knowledge graphs, the document text in structured metadata is vectorized to obtain a document representation. Then, the document representation, structured metadata, and enhanced keyword set are output as an enhanced metadata object. The document text includes at least one of abstract and main text.

[0048] By annotating and storing the display relationships between different enhanced metadata objects, a knowledge database is constructed.

[0049] The system receives user questions, extracts restrictive conditions and question intent from them, performs a hybrid search from the knowledge database based on the restrictive conditions and question intent, and outputs a semantic set of candidate documents.

[0050] Based on the semantically refined candidate document semantic set, a structured statistical visualization report and structured responses are output.

[0051] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a RAG intelligent retrieval and question-answering method based on enhanced metadata as described in the second aspect.

[0052] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the RAG intelligent retrieval and question answering method based on enhanced metadata as described in the second aspect.

[0053] The above one or more technical solutions have the following beneficial effects:

[0054] This invention integrates heterogeneous and multilingual knowledge data into structured metadata through multi-source knowledge fusion. It then uses a large language model to extract various enhanced keywords from this structured metadata, thereby constructing a knowledge graph. This knowledge graph is injected into the vectorization process of documents during the knowledge database construction, ensuring that the semantic representation of documents not only reflects linguistic similarities but also possesses semantic constraints related to disciplines and research methods. This effectively solves the problems of synonymous expressions and polysemous ambiguity, ensuring consistency in semantic recall. Furthermore, it proposes a hybrid retrieval strategy of prioritizing structured queries and refining vector retrieval to improve retrieval accuracy.

[0055] This invention introduces a knowledge graph-enhanced vectorization mechanism to achieve joint modeling of semantic representation, knowledge structure, and subject hierarchy, significantly improving the system's understanding of complex semantics and the accuracy and interpretability of cross-domain retrieval. This method innovatively utilizes LLM to extract enhanced knowledge elements of method keywords, topic keywords, and subject classifications, giving the knowledge database deep semantic annotation capabilities and resolving the inconsistency of original knowledge keywords, laying a solid foundation for high-quality question answering. Furthermore, it introduces structural supervision of knowledge graph nodes and neighborhood relationships during text encoding, ensuring that the semantic representation of documents not only reflects linguistic similarities but also possesses semantic constraints of subject hierarchy and research methods, effectively resolving synonymous heteronyms and polysemous ambiguities, and ensuring consistent semantic recall. By jointly optimizing linguistic loss and knowledge structure loss, the embedded vectors exhibit interpretable clustering distribution in high-dimensional space, allowing documents from the same discipline or method to naturally aggregate, while interdisciplinary research resides in a transitional zone, achieving semantically traceable and hierarchical knowledge organization. The resulting enhanced metadata includes semantic embeddings and constructs a structured knowledge network through explicit relation annotations. This enables the system to use semantic similarity and knowledge dependency structure for reasoning in question-answering generation and semantic retrieval, significantly improving the accuracy of knowledge retrieval, the interpretability of retrieval results, and the level of intelligence in responding to complex questions.

[0056] This invention fundamentally resolves the contradiction between retrieval accuracy and semantic depth. The proposed hybrid retrieval strategy of "structured query first, vector retrieval refinement" first uses metadata for precise filtering, effectively avoiding the ambiguity problem of pure vector retrieval, and then performs semantic refinement within the precise range, thereby achieving both extremely high retrieval accuracy and in-depth semantic understanding capabilities.

[0057] This invention provides multifunctional and multidimensional knowledge service capabilities. Its innovative dual-mode question-answering mechanism not only provides generative answers like traditional systems, but also offers macro-level statistical insights and visual analysis based on precisely selected results. This simultaneously meets users' combined needs for in-depth content understanding and grasp of macro trends. In particular, for users across professional fields, the system can integrate, translate, and summarize selected professional literature into easily understandable language and reviews through LLM (Local Language Management), enabling users to quickly grasp the core issues, mainstream methods, and overall trends in the field without extensive reading of original literature. This significantly lowers the barrier to knowledge acquisition and expands the system's application scenarios.

[0058] This invention possesses powerful knowledge discovery and domain adaptability. Because all documents are deconstructed into unified structured knowledge elements, the system's underlying layer possesses strong knowledge association and aggregation capabilities, easily enabling cross-document knowledge context analysis. For example, it can quickly track the "evolutionary trends of different research methods under a specific research topic," which is difficult for traditional systems processing unstructured text. Simultaneously, this knowledge element-based data paradigm endows the system with excellent domain adaptability. By customizing the types of knowledge elements to be extracted, the system can quickly adapt to the professional knowledge systems of different disciplines, constructing a highly accurate vertical domain knowledge question-answering system. Attached Figure Description

[0059] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0060] Figure 1 This is a flowchart of a RAG intelligent retrieval and question-answering system based on enhanced metadata, according to an embodiment of the present invention.

[0061] Figure 2 This is a flowchart of a RAG intelligent retrieval and question answering method based on enhanced metadata, according to an embodiment of the present invention. Detailed Implementation

[0062] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0063] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0064] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment discloses a RAG intelligent retrieval and question-answering system based on enhanced metadata, including:

[0067] The multi-source data acquisition module is used to acquire heterogeneous knowledge data from multiple sources and preprocess it to obtain unified knowledge data.

[0068] In this embodiment, multi-source heterogeneous knowledge data is obtained through various means such as web crawling, downloading from existing databases (such as Web of Science, CNKI, Scopus, etc.), and collecting and uploading photos of ancient books and documents. The data includes patents, conference papers, dissertations, journal articles, and research grant projects. The formats include PDF, Word, and citation formats, and the languages ​​cover multiple languages ​​from the knowledge sources.

[0069] Preprocessing involves unified parsing and semantic transformation of multi-source heterogeneous knowledge data to obtain unified knowledge data. This is achieved through Python parsing libraries and translation API modules. Specifically, firstly, Python parsing libraries (such as BeautifulSoup, pdfminer, docx, and json parsing libraries) are used to perform structured parsing of document files in different formats (including PDF, Word, HTML, JSON, TXT, etc.) to extract text information. Then, translation APIs (such as Baidu Translate, Youdao Translate, or DeepL interfaces) are called to automatically translate the text content from different languages ​​into Chinese, preserving the original document versions for subsequent comparison and verification.

[0070] In some implementations, knowledge data acquisition is not limited to web scraping, database downloading, and uploading ancient texts, but also includes real-time data acquisition via API interfaces (such as academic database APIs, open data platforms, etc.) and manual data entry. The formats of multi-source heterogeneous knowledge data are not limited to PDF, Word, and citation formats, but may also include HTML web pages, XML, JSON, plain text files, image OCR-recognized text, audio-transcribed text, etc. The source language of knowledge is not limited to multiple languages; only a single language (such as Chinese or English) may be processed, and then standardized through subsequent steps; alternatively, multi-language parallel processing may be used directly during acquisition, without translation. In some implementations, document format conversion is not limited to using Python parsing libraries and translation APIs, but may also use parsing tools in other programming languages ​​(such as Java, C#), open-source software (such as Apache Tika for format extraction), and cloud services (such as Google Cloud Document AI) for text extraction and standardization. Language translation may not rely on external APIs, but may use localized machine translation models (such as Transformer-based offline translation models), or only translate key metadata while retaining the original language in the main text.

[0071] The metadata building module is used to extract structured metadata from unified knowledge data based on different text formats.

[0072] In this embodiment, the unified knowledge data contains various text formats, such as text saved in a standard format, unstructured text, text containing special or complex metadata, and complex text. Appropriate extraction methods are used for different text formats to extract structured metadata.

[0073] For text already saved in standard formats (such as BibTeX, EndNote, RIS, XML, etc.), the metadata construction module directly reads field content using a Python parsing library and extracts and standardizes the corresponding information based on field names (such as author, title, year, source, abstract, etc.). This method is suitable for well-structured literature data with fixed fields and high extraction accuracy. For unstructured text converted from PDFs or web page text, the metadata construction module extracts information such as author, institution, time, original keywords, and text through regular expression rule matching and entity recognition (such as pre-trained models like BERT and ERNIE). This method is suitable for literature without a fixed format and can automatically identify and classify the main metadata fields. For documents containing special or complex metadata (such as patent search time), the metadata construction module fine-tunes the pre-trained model by building a targeted sample set, and then extracts data using the fine-tuned pre-trained model to improve extraction accuracy. For complex documents with multiple locations, authors, or missing fields, after initial extraction, the metadata construction module calls the online retrieval module to send query requests to online academic databases (such as CrossRef, CNKI, Google Scholar, etc.) based on existing or extracted keywords such as title, author, or DOI (Digital Object Unique Identifier). The returned results are then compared using field similarity matching algorithms (such as Levenshtein distance or TF-IDF similarity) to automatically correct or complete abnormal fields, thereby significantly improving the accuracy and completeness of metadata extraction.

[0074] When there is a DOI for a complex document, the online retrieval module directly accesses the document's homepage to crawl publicly available information. When there is no DOI, under the premise of complying with the legality of crawling, it simulates manual querying by designing URL requests, loading key information such as the title, receiving the returned JSON or HTML content, and parsing it (common Python parsing libraries include lxml, requests, etc.). For non-public databases, login cookies for personal or institutional accounts can be configured to ensure the crawler runs while preventing the information from being used for commercial purposes.

[0075] For the parsed results (i.e., the parsed JSON or HTML content), it is still necessary to extract the missing fields such as publication location and time using regular expressions or entity extraction. (Since the layout of web pages in the same database is consistent, the returned content has strong regularity and can be extracted using XPath or CSS without additional training. If extraction fails, then consider entity extraction based on NLP.) Field similarity matching is performed using existing fields. For example, if the original fields are (title, author, and the missing field is publication location), the title and author fields of the returned results are matched for similarity. If they are highly similar, it is considered that the returned results are from the same paper, and the location information is added to the missing field.

[0076] In some implementations, the extraction of basic knowledge metadata is not limited to authors, knowledge types, etc. Metadata fields can be added or removed, such as adding DOI (Digital Object Identifier), citation count, impact factor, etc., or only extracting the title and abstract as core metadata.

[0077] The enhanced metadata construction module is used to vectorize the document text in structured metadata by embedding knowledge graphs, obtaining document representations. The document representations, structured metadata, and enhanced keyword sets are then output as enhanced metadata objects. The document text includes at least one of abstract and main text. The enhanced metadata construction module includes a keyword enhancement extraction module, a knowledge graph construction module, and a text vectorization module embedded with the knowledge graph. The knowledge graph plays a supervisory role in the metadata vectorization process, promoting knowledge integration.

[0078] In this embodiment, the keyword enhancement extraction module is used to extract enhanced keywords from structured metadata using a large language model.

[0079] The keyword enhancement and extraction module generates a basic knowledge metadata dataset based on structured metadata, which includes, but is not limited to: author, knowledge type (such as journal article, conference paper, monograph, report, etc.), publication time, publisher or source, original keywords, abstract, and main text.

[0080] Based on the basic knowledge metadata dataset, the keyword enhancement extraction module utilizes a large language model (LLM) and prompt word engineering to perform deep semantic analysis on structured metadata (standardized knowledge text, such as abstracts and main text) to extract three types of enhanced keywords: topic keywords, method keywords, and subject classifications. This standardizes the keywords and solves the problem of inconsistent keyword indexing.

[0081] Specifically, firstly, based on authoritative databases in relevant fields, a set of professional terms for three types of keywords is constructed and then manually entered into the system. Taking one or more professional fields as the core, such as the field of new energy vehicle research or the field of new energy vehicle sales forecasting, a set of professional terms is constructed through parallel searches of authoritative databases in the field. The subject-specific terminology set mainly uses existing subject classification systems, such as the *Chinese Library Classification*, and relevant subjects are manually searched. Subject keywords and methodological keywords need to be manually constructed based on field dictionaries or terminologies.

[0082] Then, by constructing prompt words, the system interacts with the LLM (Limited Learning Model). The constructed prompt words are sent to the LLM, which processes the knowledge text (at least one of the abstract and main text) based on the prompt words and generates a structured response, outputting subject keywords, subject classifications, and methodological keywords. This allows for the extraction of categorized keywords from the knowledge text. The prompt words are built into the intelligent retrieval and question-answering system. During the prompt word construction process, keywords from multiple documents are first annotated by multiple experts in the relevant field. The results are then compared with those of a large model, and the prompt words are continuously improved until an expert-approved result is obtained. Simultaneously, changes to the subject area reveal the universality of the prompt words.

[0083] When constructing the prompt, the prompt should include the following: (1) Clear identity (inform the field and purpose of the LLM literature, and what role it plays in the extraction process (e.g., professor in XX field)); (2) Keyword standardization (ensure consistent keyword expression, for example, unify synonyms into a single expression, and prioritize the use of more commonly used or standardized academic terms); (3) Keyword redundancy removal (avoid extracting keywords with similar or repeated meanings); (4) Return format (return 3-5 keywords for each category, sorted by importance, with keywords separated by commas); (5) Classification dimensions (subject keywords: reflecting the main research direction or topic of the literature, subject area: specific to the first or second-level subject classification, method keywords: the technology, methods or tools used in the research. If there is insufficient information for a certain category of keywords, please mark it as "none". If possible, you can also provide examples).

[0084] To avoid instability caused by inconsistent results from multiple "questions" from different LLMs, each document will be "questioned" twice. To prevent a single LLM from being insufficient in understanding textual information, the system selects five types of LLMs released by different vendors (such as OpenAI, Anthropic, Baidu, Alibaba, and iFlytek) to ask questions, and assigns scores to the returned keyword results to obtain a comprehensive score. For each document, the three highest-scoring keywords from each category are returned. Simultaneously, the returned keywords are segmented and fuzzy matched against a vocabulary set of technical terms. If a similar keyword is successfully matched, the term set is used for modification; otherwise, the original keywords are returned and added to the vocabulary set.

[0085] Furthermore, the keyword scoring logic follows: (1) Whenever the same keyword appears repeatedly in multiple model results, the basic score is accumulated according to the number of occurrences; (2) The score is weighted and corrected by combining the model's own confidence (returned by the model API or determined by the output quality assessment) to finally form a comprehensive score.

[0086] Furthermore, the fuzzy matching process is as follows: semantic similarity calculation: input keywords and terms into a pre-trained word vector model (such as Word2Vec, BERT or SimCSE) to obtain the corresponding semantic vectors; calculate the cosine similarity between the two, and if the similarity value is higher than a set threshold (0.8 in this embodiment), they are determined to be similar in meaning.

[0087] In some implementations, different large language models (LLMs) are used for semantic parsing, such as replacing the OpenAIGPT series with other LLMs (e.g., BERT, T5, ChatGLM, etc.), or using domain-specific pre-trained models (e.g., BioBERT for the biomedical field). In some implementations, cue word engineering may not employ complex designs, but instead uses fixed templates or rule-based methods to extract keywords, such as rule-based regular expression matching or traditional NLP tools (e.g., NLTK, spaCy) for entity recognition and classification. In some implementations, enhanced keywords are not limited to topic, method, and subject classifications, but may include other types, such as technical terms, entity names (e.g., personal names, place names), research questions, conclusions, etc., or custom keyword types based on the domain (e.g., disease names in the medical field, compounds in the chemical field). In some implementations, enhanced metadata construction may not rely on LLMs, but instead uses knowledge graph technology to construct structured knowledge through entity links and relationship extraction, such as mapping document content to nodes in existing knowledge graphs (e.g., DBpedia, Wikidata).

[0088] The knowledge graph construction module is used to establish a multi-level subject node structure based on subject classification, and embed the topic keywords and method keywords into this multi-level subject node structure to obtain the knowledge graph.

[0089] After the enhanced keyword extraction and standardization of all documents (multi-source heterogeneous knowledge data is transformed and completed into metadata in a unified format), the knowledge graph construction module organizes the three types of enhanced keywords and constructs the knowledge graph based on them. ,in, It represents the set of all nodes in a knowledge graph (including subject nodes, topic keyword nodes, method keyword nodes, etc.). This represents the set of all relational edges between nodes (including topic-subject relationships, method-topic relationships, etc.). The knowledge graph construction module first establishes a multi-level subject node structure based on authoritative subject classification standards (such as the Chinese Library Classification or the ACM Computing Classification System). This structure consists of three levels: first-level subjects, second-level subjects, and third-level subjects. Subject nodes at each level are connected by "subordinate relationship" edges to form a traceable subject hierarchy system. On this basis, the knowledge graph construction module embeds topic keywords and method keywords into the subject hierarchy system to enrich the semantics and horizontal associations of the nodes.

[0090] Furthermore, embedding thematic keywords and methodological keywords into the subject-specific hierarchy is specifically as follows:

[0091] (1) When the topic keywords have semantic overlap across multiple disciplines, the system uses a sentence embedding model (SimCSE or Sentence-BERT model) to calculate the semantic similarity between the topic keywords and discipline nodes, and constructs "topic-discipline" relationship edges with the association strength as the weight. First, the discipline definition and topic keywords are encoded into vectors using a model (such as the SimCSE model). Then, the cosine similarity between the discipline definition vector and the topic keyword vector is calculated. If the similarity is higher than a set threshold, a relationship edge is established in the knowledge graph; otherwise, no relationship edge is established. For each discipline node, a "topic-discipline" relationship edge is added.

[0092] (2) Calculate the co-occurrence frequency of subject keywords and method keywords by extracting enhanced keywords (three categories, 1-3 keywords in each category) from each document. If a document contains both subject keyword A and method keyword B, it is considered that the two co-occur once in the document. Calculate all co-occurrence times to obtain the frequency. Normalize the frequency to remove the bias of high-frequency words (such as empirical analysis and case study). Construct subject-method relationship edges by setting a threshold. That is, construct relationship edges for subject and method combinations that are higher than the threshold.

[0093] In a high-dimensional vector space, texts from the same discipline or using similar methods automatically form cluster structures. When texts involve cross-disciplinary content, their embedding positions are located in the transition zone between related disciplinary clusters, thus forming an interpretable distribution pattern at the visualization level. This enhances the disciplinary traceability and semantic interpretability of the embedding representation, enabling the question-answering system to perform precise recall based on the above distribution (knowledge graph) during the retrieval stage. This structured vector return makes the response more accurate in the subsequent generation stage.

[0094] The knowledge graph embedding text vectorization module is used to vectorize the document text in structured metadata by combining the knowledge graph embedding, thus obtaining a document representation.

[0095] After the knowledge graph is constructed, the embedded text vectorization module performs knowledge entity recognition and knowledge context construction for each document. Specifically, the embedded text vectorization module uses an entity extraction model to identify research methods, theoretical concepts, or subject terms appearing in the abstract or main text (document text) of the structured metadata, and retrieves their corresponding nodes and adjacent nodes in the knowledge graph. The embedded text vectorization module thus generates a local knowledge context (local knowledge subgraph) for the document, which is a subgraph structure composed of the target node and its first-order neighbor nodes. ,in

[0096]

[0097] in, Indicates the first Local knowledge subgraphs corresponding to each document; This represents knowledge entity nodes identified in the text; Represents the target node The set of first-order neighbor nodes; This represents the set of nodes in a local knowledge subgraph. This represents the set of edges between nodes.

[0098] When semantically vectorizing the document text, the most relevant target node in the knowledge graph is located first based on the content of the document text, and the local knowledge subgraph of the target node is extracted. The node is embedded as an external structural signal input to the encoding model, so that the semantic embedding reflects the meaning of the text and retains the knowledge structure constraints.

[0099] In other words, the information from the constructed knowledge graph is embedded into the semantic vectorization of the document text. The specific steps are as follows:

[0100] (1) Divide the document text into multiple sentences, perform knowledge entity recognition on each sentence, and obtain a candidate entity set.

[0101] Knowledge entities are identified from the document text using entity extraction models (such as BERT-NER, ERNIE-NER, etc.). First, the document text is divided into multiple sentence sequences according to periods, semicolons, or semantic boundaries, as follows:

[0102]

[0103] in, Represents a set of sentence sequences. Indicates the first A sequence of sentences, Indicates the first A sequence of sentences, Indicates the number of sentences.

[0104] For a specific sentence Extract the candidate entity set ,in, Sentence The number of candidate entities identified in the process. Sentence The Middle One candidate entity, .

[0105] Calculate the semantic vector for each candidate entity. (Using the same encoder, such as SimCSE or BERT). Each node in the knowledge graph Its semantic vector has been pre-stored. (Generated and cached by subject definition text or node description during the knowledge graph construction phase).

[0106] (2) Map each candidate entity to a node in the knowledge graph to obtain sentence-level target nodes. For a sentence Each candidate entity The process involves retrieving the set of nodes that are closest to the target node in the knowledge graph. The matching process is as follows:

[0107] 1) Perform consistency matching between the semantic vector of each candidate entity and the semantic vector of the node in the knowledge graph. If they match, the corresponding node is taken as the sentence-level target node.

[0108] 2) If they are inconsistent, calculate the cosine similarity between the semantic vector of the candidate entity and the semantic vector of the node in the knowledge graph. Sort the nodes from largest to smallest according to the cosine similarity and select the first set number of nodes as candidate nodes. Calculate the comprehensive score of the candidate nodes based on the candidate entities and select the node with the highest comprehensive score as the sentence-level target node.

[0109] Calculate cosine similarity :

[0110]

[0111] in, Indicates candidate entities semantic vectors, Represents a node The semantic vector. For all Obtain similarity scores, sort nodes according to their similarity scores from highest to lowest, and select Top-K candidate nodes.

[0112] Taking into account similarity and the weight of entity position in the sentence (Title / Abstract above the middle of the text), frequency of entity occurrences throughout the text and the importance of node priors Calculate the comprehensive score of the candidate nodes. This is used to determine which candidate node is selected as the target node for the sentence. The target node represents the knowledge meaning of the sentence, that is, the knowledge representation of the sentence.

[0113]

[0114] in, This represents the overall score. Indicates candidate entities, Nodes representing a knowledge graph This represents the semantic similarity weight coefficient. This represents the cosine similarity between the candidate entity and the node. This indicates the positional weight of an entity within a sentence. This indicates the frequency of an entity in the full text of the document. Indicates the prior importance of nodes. , , These represent the adjustment coefficients for position weight, word frequency weight, and node prior weight, respectively.

[0115] In this embodiment, the prior importance of nodes This prior weight is used to characterize the fundamental importance of nodes within an external knowledge system. Its value is derived from two types of information: first, the hierarchical structure determined by the subject classification system (e.g., first-level, second-level, and third-level disciplines), with higher-level nodes assigned greater prior weights; and second, the structural importance calculated based on the node's topological characteristics in the knowledge graph (e.g., node degree, PageRank, and other centrality metrics). These two types of prior features are normalized and then incorporated into the model to improve the global consistency and structural interpretability of node representations.

[0116] Furthermore, to characterize the fundamental importance of nodes within the external knowledge system, this embodiment uses each node... Constructing Prior Importance Weights It is obtained by linearly fusing the subject-level weights and the graph structure centrality weights:

[0117]

[0118] in, For balancing parameters, As a hierarchical weighting based on the subject system, These are the centrality weights based on the graph structure.

[0119]

[0120] in, Represents a node The subject level (e.g., first-level discipline = 3, second-level discipline = 2, third-level discipline = 1) is assigned, and the whole is normalized. Represents a node Subject level.

[0121]

[0122] in, For nodes The topological structural features are analyzed, and weights are ensured to be comparable through normalization. For nodes The topological structure characteristics. The node that achieves the highest score. (Or, if the scores are close, retain multiple nodes) as sentence-level target nodes. The parameters in this embodiment are set to... , , , .

[0123] It should be noted that the above process is a read-only retrieval and does not apply to the knowledge graph. Add or delete any node or edge.

[0124] (3) Based on the sentence-level target node, construct the sentence-level local subgraph and node vector matrix. . Target node at the sentence level Find the set of its first-order neighbors centered on the target. Constructing sentence-level local subgraphs:

[0125]

[0126] And arrange the node vectors into a matrix:

[0127]

[0128] Each row is the vector representation of the corresponding node. This is the vector representation of the sentence-level target node; Sentence-level target node of Each first-order neighbor node.

[0129] As a structural supervision signal, it is fed into the encoding model along with the sentence input.

[0130] (4) The node information of sentences and sentence-level local subgraphs in the literature text is fused and input into the Transformer encoder. The sentence-level enhanced representation is obtained by using the joint mechanism of semantic attention and knowledge attention (sentence-level joint input and attention fusion).

[0131] Get Sentence token sequence ,in, It is a sentence The number of tokens, Sentence The Middle Each token. (The sentence is incomplete.) The token sequence and the node sequence of the sentence-level local subgraph The input can be composed of splicing or structured connections:

[0132]

[0133] in, This represents the combined sequence of inputs to the Transformer encoder.

[0134] Semantic attention is computed in the Transformer self-attention layer. With knowledge attention (Knowledge attention can be designed based on the graph topology between nodes or the similarity of node vectors), and according to hyperparameters. Fusion:

[0135]

[0136] in, Represents a node For nodes The final attention weight.

[0137] Subsequently, a sentence-level enhancement representation was obtained. :

[0138]

[0139] in, The vector of input items (token vector or node vector). This is a trainable mapping matrix. If a sentence corresponds to multiple target nodes, multiple mappings can be constructed in parallel. And it is included in the attention calculation.

[0140] (5) The sentence-level enhanced representations are aggregated to obtain the document representation.

[0141] For all sentence-level enhancement representations The document representation is obtained by aggregation. Common methods include weighted averaging or recoding (e.g.: Pooling weights can be set based on sentence importance, topic weights within the sentence, or summary position.

[0142] (6) Training objectives and structural supervision. Joint loss is used for the Transformer encoder. :

[0143]

[0144]

[0145] in, The loss of semantic preservation at the language level can be represented by the squared difference between sentence-level vectors and reference language model vectors or MLM cross-entropy. The parameter represents the parameters used to balance semantic and structural constraints; in this embodiment, the range is set to... ; Representing structural constraint loss, for sentence-level local subgraph node vectors Forced local consistency.

[0146] After joint optimization (i.e., knowledge graph injection (embedding) during metadata vectorization), the document representation output by the model has both semantic expressive power and knowledge hierarchy constraints. It presents an interpretable cluster distribution in high-dimensional space, that is, literature of the same discipline or method is clustered in the nearest neighbor region, and interdisciplinary research is located in the transition zone between related discipline clusters, thereby significantly improving the knowledge recall accuracy and traceability of the generated content of the question answering system.

[0147] For each document, the system ultimately outputs a uniformly formatted enhanced metadata object, which includes structured metadata, an enhanced keyword set, and a document representation.

[0148] The knowledge database construction module is used to annotate and store the display relationships between different enhanced metadata objects, thereby constructing the knowledge database.

[0149] By using relation extraction and manual annotation, explicit relationships between different knowledge objects are annotated based on the extracted metadata fields and document content, thereby enhancing knowledge integration.

[0150] Enhanced metadata objects, also known as knowledge objects, include, but are not limited to, documents, patents, and grants. When storing multiple knowledge objects, some of these objects may have sources beyond documents, such as grant information. For example, document A may be funded by grant B. This relationship cannot be expressed through semantic understanding (i.e., vectorization). Therefore, an explicit relationship needs to be established and stored in the database. By establishing an explicit relationship, document A and grant B are integrated. In subsequent retrieval processes, restricted searches can be performed, such as searching for "papers published with grant B".

[0151] Relationships include, but are not limited to, the following types: (1) "funding relationship" (e.g., "XXX paper" is funded by "National Natural Science Foundation of China Project (No. XXXXXX)"); (2) "citation relationship" (e.g., "XXX paper" cites "YYY literature"). By adding fields to the metadata for identification, these enhanced metadata objects are finally stored in the database, completing the construction of the knowledge database.

[0152] In some implementations, the knowledge database is not limited to relational databases (such as MySQL and PostgreSQL), and can be stored using NoSQL databases (such as MongoDB), graph databases (such as Neo4j), or vector databases (such as Pinecone and Weaviate) to optimize retrieval performance. In some implementations, the structured knowledge base may not be built using unified storage, but rather distributed storage or a multi-level index structure, for example, storing metadata and full-text content separately, or using inverted indexes to accelerate queries.

[0153] The dual retrieval module receives user questions, extracts restrictive conditions and question intent from them, performs a hybrid retrieval from the knowledge database based on the restrictive conditions and question intent, and outputs a semantic set of candidate documents.

[0154] In this embodiment, the prompt word template optimization module is used to iteratively optimize the composite prompt word template based on a preset dataset to obtain an improved prompt word template.

[0155] The prompt word template optimization module uses a composite prompt word template during parsing, which includes identity settings, question decomposition, output constraints, and a computable structured template. For example:

[0156] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0157] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0158] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0159] To mitigate the risk of missing restrictive conditions in LLM (Limited Language Learning), this invention further introduces a text parsing feedback learning mechanism. The system constructs a "question-parsing result" dataset (a pre-defined dataset), storing each user's question and manually extracted restrictive conditions in a structured manner, i.e., question and answer, forming the following sample format (containing the question and the extracted restrictive conditions):

[0160] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0161] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0162] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0163] }

[0164] The prompt word template optimization module acquires a sample of questions and parsing results from a preset dataset. It then uses LLM (Limited Language Modeling) to identify the questions in the samples, obtaining restrictive conditions. By analyzing the differences between the restrictive conditions output by the LLM and the parsing results, a multi-round iterative optimization strategy is employed to automatically adjust the composite prompt word template, resulting in an improved version. In practical application, the system uses the latest improved prompt template to parse user questions, extracting comprehensive and accurate restrictive conditions. Specifically, the system leverages the semantic alignment capabilities of LLM to compare the differences between the model output and manual corrections, generating an improved prompt template.

[0165] Furthermore, based on a pre-defined dataset, the LLM algorithm performs result evaluation and optimization to achieve automated improvement of the prompt words. One iteration process is as follows:

[0166] Obtain a sample from the "Question-Analysis Results" dataset, which includes the user's question sentence C and the actual constraints marked by the human annotation, such as the actual constraints being [a,b];

[0167] The initial prompt word P1 is obtained based on the compound prompt word template. The initial prompt word P1 is used to send a request to the LLM. The LLM processes the question sentence C and obtains the restrictive conditions extracted by the LLM, namely the first type of restrictive conditions.

[0168] The initial prompt word P1 is:

[0169] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0170] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0171] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0172] If the only restrictive condition extracted by the LLM is 'a', while the actual restrictive conditions are 'a' and 'b', then write a prompt word P2 to let the LLM determine whether the first type of restrictive condition is consistent with the actual restrictive condition.

[0173] The prompt word P2 is:

[0174] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}}

[0175] If there is a discrepancy, use prompt word P3 to guide the LLM to optimize the initial prompt word P1;

[0176] The prompt word P3 is:

[0177] {"query_id": "Q202510260001", "user_query": "XXXXXX", "constraints": {"field": "XXX", "year_range": "XXX"}} 1 {"query_id ​ 1 ​

[0178] The optimized prompt is P4.

[0179] When the data samples accumulate to a certain scale (e.g., more than 2,000), the system can use lightweight fine-tuning (e.g., LoRA or Adapter) to enhance the local LLM training, focusing on optimizing intent recognition and semantic abstraction capabilities, without having to retrain the entire model.

[0180] The question extraction module receives user-submitted questions and automatically identifies the restrictive conditions and question intent within them. When a user inputs a question, they can input the restrictive conditions and question intent separately into the large language model, or they can input them together. When inputting together, the input is split using a specific delimiter. The large language model then uses regular expressions to segment the input and execute the restrictive conditions and question intent separately.

[0181] The so-called restrictive conditions refer to the elements that constrain the scope of the search, such as the research object, time interval, document type or research method, etc.; the core question intent refers to the semantic center of the user's question, that is, the focus of the required knowledge, such as "the source of error in the prediction of new energy vehicle sales", "the applicable boundary of the grey model", "the performance comparison of a certain method in a specific scenario", etc.

[0182] After extracting the question intent and restrictive conditions, the question extraction module performs semantic expansion and reduction strategies based on the keywords in the restrictive conditions to obtain search terms. Specifically, when the system detects keywords with semantic ambiguity or many synonyms in the user's question (e.g., "grey model," "grey system prediction," "GM(1,1)"), it uses a built-in terminology vocabulary and semantic similarity models (such as BERT or SimCSE embedding vector calculation) to perform polysemy correction and synonym merging. If the semantic similarity is lower than a set threshold (e.g., 0.75), the system adds synonyms to the search term set to improve recall. Conversely, when the search scope is too broad (e.g., when keywords have cross-domain ambiguity), the system performs keyword reduction strategies, retaining only highly relevant words to improve accuracy. This expansion and reduction process generates a candidate set using LLM, and the final search term set is determined after semantic similarity calculation and domain weight correction.

[0183] The hybrid retrieval module is used to perform hybrid retrieval from the knowledge database based on restrictive conditions and question intent, and output a semantic set of candidate documents.

[0184] The hybrid search module performs structured queries in the knowledge database based on restrictive conditions, using metadata fields (including author, institution, year, research direction, keywords, etc.) to filter out precise knowledge subsets. For example, when the restrictive conditions are "2020-2024, grey prediction model, new energy vehicle field", the system uses SQL statements or graph database query syntax (such as Cypher) to locate the set of documents that meet the conditions and form a precise knowledge subset.

[0185] After obtaining the precise knowledge subset, the hybrid retrieval module vectorizes the question intent (e.g., by generating a vector representation using BERT or OpenAI Embedding API) to obtain the intent vector, and performs vector retrieval operations based on the intent vector in the precise knowledge subset, calculating the semantic similarity between the abstract and the text embedding vector (document representation) of each document.

[0186] During the retrieval process, the system employs a multi-level filtering strategy to improve semantic matching accuracy: the first layer performs a coarse screening based on vector similarity, and the second layer performs a secondary screening based on keyword matching and topic consistency verification. The final set of high-similarity results constitutes the candidate document semantic set.

[0187] In some implementations, user query parsing is not limited to using LLM; traditional NLP techniques (such as dependency parsing and intent recognition models) or rule engines (such as regular expression-based pattern matching) can be employed to extract restrictive conditions and core question intent. In some implementations, keyword expansion may not involve polysemy or polysemous word checks, but instead utilizes thesaurus (such as WordNet), domain terminology databases, or user query history for automatic expansion. In some implementations, structured queries are not limited to precise filtering based on metadata, but may include fuzzy queries, range queries (such as time ranges), or Boolean logic queries (AND / OR / NOT). In some implementations, vector retrieval may not be limited to the summary and main text in the metadata, but may directly vectorize the entire text content, or use multimodal vectors (combining text, images, etc.). In some implementations, semantic refinement may not use vector retrieval, but instead employs other semantic similarity calculation methods (such as cosine similarity, Jaccard similarity), or graph-based diffusion algorithms. In some implementations, vectorization techniques are not limited to LLM-based embeddings; other embedding models (such as Sentence-BERT, Doc2Vec) or traditional TF-IDF weighted vectors can be used. In some implementations, hybrid retrieval is not limited to two retrieval methods; a third retrieval layer can be introduced, such as retrieval based on citation networks, personalized retrieval based on user feedback, or real-time popularity retrieval.

[0188] The dual-mode response module is used to output structured statistical visualization reports and structured responses based on the semantically refined candidate document semantic set.

[0189] After obtaining the semantic set of candidate documents, the system performs semantic refinement processing to obtain a semantically refined semantic set of candidate documents. This process is completed by LLM, which combines the user's question with the semantic set of candidate documents to generate an aggregated knowledge answer. Specifically, this includes: 1. Aggregating and compressing the content of the candidate document abstracts, removing duplicate and redundant information, and outputting an aggregated abstract; 2. Retaining semantic segments highly relevant to the question's intent to form a refined abstract; 3. LLM generating a structured answer from the refined abstract based on the prompt template.

[0190] To further improve the interpretability and user experience of the question-and-answer results, after semantic retrieval and refinement, the system adopts a dual-mode output mechanism for the output results. This involves the parallel generation of structured statistical visualization reports and semantically generated text answers, achieving collaborative output of "fact-level knowledge display" and "semantic-level knowledge explanation." Specifically, based on the candidate document semantic set obtained through a hybrid retrieval strategy (combining structured retrieval and vector retrieval), the system first calls the Large Language Model (LLM) and prompt word engineering module to automatically identify the statistical dimensions that can be generated, and then executes a visualization data generation task in the background. During this process, the system automatically calculates the following indicators (including but not limited to) based on the structured information fields in the knowledge metadata (including author, institution, publication year, subject classification, research method, etc.): 1. Document quantity statistics: Calculate the total number of documents that meet the retrieval conditions and their distribution in different years or source databases; 2. Subject distribution statistics: Based on the subject keyword field in the enhanced metadata, calculate the number and proportion of documents in different subject areas, and construct a subject-level pie chart or sunburst chart; 3. Time trend analysis: Calculate the time series change trend of the number of documents based on the publication time field, and draw a line graph or area chart. After statistics, a collection of literature that meets the restrictive conditions is returned (each knowledge is returned via JSON). Then, the information in the JSON format is extracted by a Python program, and fields that need to be visualized, such as time and subject, are statistically analyzed. The system automatically generates corresponding visualization charts based on these data and displays them using components such as ECharts or Matplotlib, forming a structured statistical visualization report.

[0191] While generating the visualization report, the system combines the semantically refined candidate literature semantic set with the user's core question intent, submitting both as input to the LLM for text generation to obtain a structured answer. This stage employs cue word engineering to organize the answer information and the language of the response text. When constructing cue words, the following logic should be followed to guide the model in generating highly interpretable answers: 1. Retain key conclusions; 2. Explain the source of the conclusions; 3. Clearly define the structured output format; 4. If there are research disagreements, please point out the different viewpoints.

[0192] After the answer is output, the system allows users to click on specific charts or answers in the report interface to provide feedback. The feedback results are automatically entered into the prompt word optimization loop, forming a closed-loop improvement system.

[0193] In some implementations, statistical information is not limited to the number of documents or subject distribution, but may include other dimensions such as keyword co-occurrence networks, research hotspot maps, and author collaboration networks. In some implementations, visualization chart generation may not rely on an LLM (Library Management Model), but can be automatically generated based on search results using dedicated visualization libraries (such as D3.js or Matplotlib) or commercial BI tools (such as Tableau). In some implementations, statistical information may not be visualized, but presented as text summaries, such as listing the top-N authors or describing trends. In some implementations, text answer generation is not limited to using an LLM, but can employ template-based methods (such as predefined sentence combinations based on search results) or rule-based generators. In some implementations, answer generation may not be based on the user's core question intent, but can be directly based on the retrieved information for summarization or translation. In some implementations, bimodal question answering is not limited to statistical and generative models, but can include a third model, such as providing recommended reading lists, related question-and-answer pairs, or audio summaries. In some implementations, visualized search result reports and text answers may not be integrated into a single interface, but sent separately (such as via email or API return), or dynamically displayed based on user selection. In some implementations, the output format is not limited to graphics and text, but may include voice output, interactive dashboards, or downloadable data files (such as CSV or JSON). In some implementations, module functionality can be implemented using a microservice architecture, with each module deployed as an independent service communicating via APIs, rather than a monolithic architecture. In some implementations, the use of LLM may not be a full-fledged dependency, but only in critical steps (such as enhanced metadata construction or question-answer generation), while other steps use lightweight models or traditional algorithms. In some implementations, the system may not be limited to domain-specific question answering, but may be applicable to general knowledge question answering, customer service systems, educational support, or enterprise knowledge management. In some implementations, the system may not process multi-source knowledge, but rather optimize for a single data source (such as an internal document repository). In some implementations, the method may exclude all steps, for example, omitting the knowledge standardization step and directly processing raw data, or omitting enhanced metadata construction and using only basic metadata for retrieval.

[0194] Example 2

[0195] like ​ As shown, this embodiment discloses a RAG intelligent retrieval and question-answering method based on enhanced metadata, characterized by including:

[0196] Acquire multi-source heterogeneous knowledge data and preprocess it to obtain unified knowledge data;

[0197] Based on the different text formats, structured metadata is extracted from unified knowledge data;

[0198] By embedding knowledge graphs, the document text in structured metadata is vectorized to obtain a document representation. Then, the document representation, structured metadata, and enhanced keyword set are output as an enhanced metadata object. The document text includes at least one of abstract and main text.

[0199] By annotating and storing the display relationships between different enhanced metadata objects, a knowledge database is constructed.

[0200] The system receives user questions, extracts restrictive conditions and question intent from them, performs a hybrid search from the knowledge database based on the restrictive conditions and question intent, and outputs a semantic set of candidate documents.

[0201] Based on the semantically refined candidate document semantic set, a structured statistical visualization report and structured responses are output.

[0202] Example 3

[0203] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 2.

[0204] Example 4

[0205] The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 2.

[0206] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0207] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0208] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0209] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A RAG intelligent retrieval and question-answering system based on enhanced metadata, characterized in that, include: The multi-source data acquisition module is used to acquire heterogeneous knowledge data from multiple sources and preprocess it to obtain unified knowledge data. The metadata building module is used to extract structured metadata from unified knowledge data based on different text formats; the metadata includes author, knowledge type, publication time, publisher or source, original keywords, abstract, and main text; An enhanced metadata construction module is used to vectorize the document text in structured metadata by embedding knowledge graphs to obtain document representations, and then output the document representations, structured metadata, and enhanced keyword sets as enhanced metadata objects; the document text includes at least one of abstract and main text. When vectorizing the document text in structured metadata, sentence-level target nodes are determined by calculating a comprehensive score, which is expressed as: in, This represents the overall score. Indicates candidate entities, Nodes representing a knowledge graph This represents the semantic similarity weight coefficient. This represents the cosine similarity between the candidate entity and the node. This indicates the positional weight of an entity within a sentence. This indicates the frequency of an entity in the full text of the document. Indicates the prior importance of nodes. , , These represent the adjustment coefficients for position weight, word frequency weight, and node prior weight, respectively. The enhanced metadata construction module includes a keyword enhancement extraction module, a knowledge graph construction module, and a text vectorization module for knowledge graph embedding; The keyword enhancement extraction module is used to extract enhanced keywords from structured metadata using a large language model; enhanced keywords include subject keywords, subject classifications, and methodological keywords. The knowledge graph construction module is used to establish a multi-level subject node structure based on subject classification, and to embed topic keywords and method keywords into this multi-level subject node structure to obtain a knowledge graph. The knowledge graph embedding text vectorization module is used to vectorize the document text in the structured metadata by combining the knowledge graph embedding to obtain the document representation; Embedding thematic keywords and methodological keywords into a multi-level subject node structure is specifically as follows: When the topic keywords have semantic overlap across multiple disciplines, calculate the semantic similarity between the topic keywords and discipline nodes, and construct topic-discipline relationship edges with the association strength as the weight. Based on the enhanced keywords extracted from each document, the co-occurrence frequency of topic keywords and method keywords is calculated, the frequency is normalized, and a topic-method relationship edge is constructed by setting a threshold. The knowledge database construction module is used to annotate and store the display relationships between different enhanced metadata objects to build a knowledge database; using relation extraction and manual annotation, the display relationships between different enhanced metadata objects are annotated based on the extracted metadata fields and document content. A dual retrieval module receives user questions, extracts restrictive conditions and question intent from them, performs a hybrid retrieval from a knowledge database based on the restrictive conditions and question intent, and outputs a semantic set of candidate documents. It also includes a prompt word template optimization module to improve prompt words and obtain comprehensive and accurate restrictive conditions. This module acquires a sample of question-analysis results from a preset dataset, identifies the questions in the sample using LLM (Language Modeling), obtains restrictive conditions, analyzes the differences between the restrictive conditions output by LLM and the analysis results, and automatically adjusts the composite prompt word template using a multi-round iterative optimization strategy to obtain an improved prompt template. When constructing suggestion words, the keywords of multiple documents were first annotated by multiple domain experts, then compared with the annotation results of a large model, and the suggestion words were continuously improved until the results were approved by experts. The hybrid retrieval specifically involves: performing a structured query in the knowledge database based on restrictive conditions, and using metadata fields to filter out a precise subset of knowledge; The intent of the question is vectorized to obtain an intent vector. Based on the intent vector, a vector retrieval operation is performed in the precise knowledge subset to output a semantic set of candidate documents. The dual-mode response module is used to output structured statistical visualization reports and structured responses based on the semantically refined candidate document semantic set.

2. The RAG intelligent retrieval and question-answering system based on enhanced metadata as described in claim 1, characterized in that, By embedding knowledge graphs, the document text in structured metadata is vectorized, specifically as follows: The document text is divided into multiple sentences, and knowledge entity recognition is performed on each sentence to obtain a candidate entity set; Each candidate entity is mapped to a node in the knowledge graph to obtain sentence-level target nodes; Based on sentence-level target nodes, construct sentence-level local subgraphs and node vector matrices; The node information of sentences and sentence-level local subgraphs in the literature text is fused and input into the encoder. The sentence-level enhanced representation is obtained by using the joint mechanism of semantic attention and knowledge attention. The sentence-level enhanced representations are aggregated to obtain the document representation.

3. The RAG intelligent retrieval and question-answering system based on enhanced metadata as described in claim 2, characterized in that, Mapping each candidate entity to a node in the knowledge graph yields the sentence-level target nodes as follows: The semantic vectors of each candidate entity are matched with the semantic vectors of nodes in the knowledge graph. If they match, the corresponding node is taken as the sentence-level target node. If they are inconsistent, the cosine similarity between the semantic vector of the candidate entity and the semantic vector of the node in the knowledge graph is calculated. The nodes are sorted from largest to smallest according to the cosine similarity, and the first set number of nodes are selected as candidate nodes. The comprehensive score of the candidate nodes is calculated based on the candidate entities, and the node with the highest comprehensive score is selected as the sentence-level target node.

4. A RAG intelligent retrieval and question-answering method based on enhanced metadata, characterized in that, include: Acquire multi-source heterogeneous knowledge data and preprocess it to obtain unified knowledge data; Based on the different text formats, structured metadata is extracted from the unified knowledge data; the metadata includes author, knowledge type, publication time, publisher or source, original keywords, abstract, and main text. By embedding knowledge graphs, the document text in structured metadata is vectorized to obtain a document representation. The document representation, structured metadata, and enhanced keyword set are then output as an enhanced metadata object. The document text includes at least one of abstract and main text. When vectorizing the document text in structured metadata, sentence-level target nodes are determined by calculating a comprehensive score, which is expressed as: in, This represents the overall score. Indicates candidate entities, Nodes representing a knowledge graph This represents the semantic similarity weight coefficient. This represents the cosine similarity between the candidate entity and the node. This indicates the positional weight of an entity within a sentence. This indicates the frequency of an entity in the full text of the document. Indicates the prior importance of nodes. , , These represent the adjustment coefficients for position weight, word frequency weight, and node prior weight, respectively. Enhanced keywords were extracted from structured metadata using a large language model; these enhanced keywords included subject keywords, subject classifications, and methodological keywords. A multi-level subject node structure is established based on subject classification. Thematic keywords and method keywords are embedded into this multi-level subject node structure to obtain a knowledge graph. By embedding knowledge graphs, the document text in structured metadata is vectorized to obtain document representations; Embedding thematic keywords and methodological keywords into a multi-level subject node structure is specifically as follows: When the topic keywords have semantic overlap across multiple disciplines, calculate the semantic similarity between the topic keywords and discipline nodes, and construct topic-discipline relationship edges with the association strength as the weight. Based on the enhanced keywords extracted from each document, the co-occurrence frequency of topic keywords and method keywords is calculated, the frequency is normalized, and a topic-method relationship edge is constructed by setting a threshold. The display relationships between different enhanced metadata objects are labeled and stored to construct a knowledge database; using relation extraction and manual annotation, the display relationships between different enhanced metadata objects are labeled based on the extracted metadata fields and document content. The system receives user questions, extracts restrictive conditions and question intent from them, performs a hybrid search from the knowledge database based on the restrictive conditions and question intent, and outputs a semantic set of candidate documents. It then extracts restrictive conditions from user questions using a large language model to improve prompt words and obtain comprehensive and accurate restrictive conditions. A sample of question-analysis results from a pre-defined dataset is obtained, and LLM is used to identify questions in the sample to obtain restrictive conditions. By analyzing the differences between the restrictive conditions output by LLM and the analysis results, a multi-round iterative optimization strategy is adopted to automatically adjust the composite prompt word template, resulting in an improved prompt template. When constructing suggestion words, the keywords of multiple documents were first annotated by multiple domain experts, then compared with the annotation results of a large model, and the suggestion words were continuously improved until the results were approved by experts. The hybrid retrieval specifically involves: performing a structured query in the knowledge database based on restrictive conditions, and using metadata fields to filter out a precise subset of knowledge; The intent of the question is vectorized to obtain an intent vector. Based on the intent vector, a vector retrieval operation is performed in the precise knowledge subset to output a semantic set of candidate documents. Based on the semantically refined candidate document semantic set, a structured statistical visualization report and structured responses are output.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the RAG intelligent retrieval and question answering method based on enhanced metadata as described in claim 4.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the RAG intelligent retrieval and question answering method based on enhanced metadata as described in claim 4.

Images