A knowledge base intelligent question answering method based on multi-granularity slicing and context extension

Abstract

This invention discloses a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion. The method includes: based on the query text, semantically retrieving matching clauses in a vector database and retrieving corresponding content in a relational database, fusing them to obtain an initial retrieval result; selecting several clauses from the initial retrieval result that are most relevant to the query text, expanding their data, and integrating them to obtain the final context; concatenating the final context with a prompt word template and inputting it into a large language model to generate the final answer. This invention proposes a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion. Through multi-path retrieval and relying on parent block information to complete the dynamic context expansion of adjacent text segments, it can improve the accuracy of intelligent question answering answers from both the retrieval matching and information completion dimensions. It can solve the problem of low accuracy of generated answers caused by insufficient precision and inaccurate expansion in traditional knowledge base intelligent question answering technology due to the single retrieval method.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, and in particular to a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models have been widely used in the field of knowledge-based question answering, becoming an important support for realizing intelligent interactive question answering. However, in practical applications, large language models are prone to providing answers that are inconsistent with the facts, affecting the actual effectiveness of question answering. Retrieval-enhanced generation technology, with its core idea of ​​introducing external knowledge bases, has become a key technical means to alleviate this problem and improve the factual accuracy of question answering. Among the existing knowledge base intelligent question answering technologies, the mainstream solutions are based on the basic RAG process to realize document segmentation, vectorization, and retrieval generation; some solutions also improve reasoning ability by constructing structured knowledge graphs, or optimize retrieval relevance by relying on probability models and word distance; long documents are mostly processed by vectorizing the segments after segmentation with fixed length or punctuation, and most of them rely on single vector semantic retrieval, while non-text elements are mostly simply recognized as text for processing.

[0003] However, the basic RAG process has not been deeply optimized for core challenges such as long document processing, retrieval strategies, and non-text information parsing, still relying on simple document segmentation and a single retrieval mode. The construction and maintenance of knowledge graph solutions are costly and time-consuming, making them difficult to adapt to the massive and dynamically updated unstructured documents required by enterprises. Various retrieval optimization methods can only improve retrieval accuracy in specific scenarios and cannot solve global architectural problems such as document segmentation and contextual coherence. Furthermore, unreasonable segmentation of long documents can easily lead to semantic fragmentation or missing context; single-vector semantic retrieval can easily create semantic gaps due to differences in terminology; and coarse processing of non-text elements can cause the loss of key data. These problems, combined, further exacerbate retrieval bias, ultimately making it difficult to guarantee the factual accuracy of question-and-answer answers. Summary of the Invention

[0004] This invention provides a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion, in order to solve the problem that traditional knowledge base intelligent question answering technology suffers from low accuracy in generated answers due to insufficient precision of a single retrieval method and inaccurate expansion.

[0005] To achieve the above objectives, this application provides a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion, including: Get the query text entered by the user; Based on the query text, semantic retrieval and clause matching are performed in a vector database, and keyword retrieval and corresponding content are performed in a relational database using a keyword inverted index. The two retrieval results are then combined to obtain the initial retrieval results. Obtain several clauses from the initial search results that are most relevant to the query text. Locate the corresponding parent block in the relational database based on the parent block information corresponding to the several clauses. Expand the several clauses into adjacent text segments based on the location results and integrate them to obtain the final context. The final context is concatenated with the preset prompt word template and then input into the large language model to generate the final answer.

[0006] This invention employs a dual-path retrieval fusion strategy, balancing semantic relevance and keyword accuracy. Semantic retrieval from a vector database deeply mines the semantic connections between the query text and clauses, avoiding the literal limitations of keyword retrieval. The inverted keyword index from a relational database achieves precise matching between keywords and content, compensating for potential ambiguities in semantic retrieval. The fusion of these two approaches effectively improves the accuracy of the initial retrieval results, solving the problem of insufficient precision in single-path retrieval. First, the clause with the highest relevance to the query text is selected. Then, the corresponding parent block is located based on the parent block information. Adjacent text fragments are expanded based on the parent block, ensuring that the expanded text fragments are highly relevant to the core clause. This avoids the introduction of irrelevant information in traditional expansion methods, providing accurate and complete contextual support for the large language model. Finally, the final context is concatenated with a preset prompt word template and input into the large language model. This template standardizes the model's generation logic and anchors the answer direction, ensuring that the generated content closely adheres to the context and fits the core needs of the knowledge base. This reduces retrieval bias, chaotic expansion, and inaccurate answers caused by unconstrained model generation from the source, ultimately achieving a simultaneous improvement in retrieval accuracy, expansion precision, and answer accuracy.

[0007] Compared to existing technologies, this invention achieves multi-granularity precise matching by integrating semantic retrieval from vector databases and keyword inverted indexes from relational databases. It also leverages parent block information to dynamically expand the context of adjacent text segments, thereby compensating for the accuracy limitations of single retrieval methods and the inaccuracy of context expansion. This improves the accuracy of intelligent question-answering answers from both the retrieval matching and information completion dimensions. Therefore, it can solve the problem of low accuracy in generating answers caused by insufficient precision and inaccurate expansion in traditional knowledge base intelligent question-answering technologies.

[0008] As a preferred embodiment, the method for obtaining the vector database includes: Acquire knowledge documents in the target domain; Text extraction is performed on the knowledge document, and the tables in the knowledge document are identified and converted into structured data and processed as independent parent blocks to obtain the basic text of the document and the structured parent blocks of the tables respectively. The document's basic text is divided into several parent blocks according to preset logical units. Within the parent blocks and the table-structured parent blocks, clauses are segmented according to semantic boundaries. While retaining the original parent block data, the segmented clause data is generated, and the clause data is bound to the corresponding parent block identifier and its position offset within the parent block. The clause data is vectorized based on the embedding model to generate high-dimensional vector data of the clauses. The high-dimensional vector data of the clause, carrying the parent block identifier and the position offset, is stored in the initial vector database to obtain the vector database.

[0009] This preferred solution transforms tables in knowledge documents into structured parent blocks and processes them independently, avoiding the fragmentation and misinterpretation of table information due to text extraction, thus ensuring the integrity of structured knowledge. The multi-granularity processing approach, dividing parent blocks according to preset logical units and segmenting clauses according to semantic boundaries, preserves the overall relevance of knowledge while achieving fine-grained semantic segmentation. Furthermore, it binds parent block identifiers and positional offsets to clauses, providing precise positioning for subsequent contextual expansion. Vectorization based on an embedding model ensures accurate semantic representation of clauses. After being stored in a vector database, it significantly improves the matching accuracy and efficiency of subsequent semantic retrieval, laying a data foundation for the accuracy of the entire intelligent question-answering method and effectively avoiding retrieval bias or missing context issues caused by single-granularity slicing.

[0010] As a preferred embodiment, the method for obtaining the relational database includes: From the knowledge document, extract several target keywords for each of the several parent blocks and the table structured parent blocks, and construct a corresponding keyword inverted index for the several target keywords; wherein, the keyword inverted index consists of keywords, parent block identifiers and clause positions; The clause data, the parent block identifier, the target keywords, the table structured parent block, and the keyword inverted index are stored in the initial relational database to obtain the relational database.

[0011] This optimized solution constructs a relational database that complements vector databases, further enhancing the knowledge base's retrieval capabilities. By extracting target keywords for each parent block and building an inverted index, it achieves precise association between keywords and their parent blocks and clause positions, significantly improving the speed and targeting of keyword retrieval and resolving the issues of low retrieval efficiency and ambiguous positioning inherent in traditional relational databases. Integrating and storing clause data, parent block identifiers, target keywords, structured table data, and inverted indexes enables structured knowledge management, facilitating rapid retrieval of various data types while ensuring data relevance and integrity. This storage method allows subsequent searches to quickly locate relevant content using keywords and easily obtain the parent block to which the clause belongs and related information, providing efficient data support for contextual expansion and search result fusion, effectively improving the coherence and reliability of the entire question-and-answer process.

[0012] As a preferred embodiment, the corresponding parent block is located in the relational database based on the parent block information corresponding to the clauses. Based on the location result, adjacent text segments are expanded from the clauses and integrated to obtain the final context. Specifically: Using the parent block identifier bound to the aforementioned clauses as the retrieval basis, a match is performed in the relational database to obtain the parent block data to which the aforementioned clauses belong; Using the position offset of the clauses bound in their parent block as the positioning standard, and according to preset rules, extract a preset number of adjacent text segments before and after each clause from the acquired parent block data, and integrate them to obtain combined text data composed of the clauses and their corresponding adjacent text segments. Based on the query text and the parent block identifier bound to the clauses, the corresponding first table structured parent block is retrieved in the relational database; The combined text data and the first table structured parent block are integrated to obtain the final context.

[0013] This preferred solution accurately locates the parent block to which a clause belongs by using parent block identifiers, ensuring the accuracy of context expansion and avoiding semantic deviations caused by positioning errors. Using positional offset as a standard, adjacent text fragments before and after the clause are extracted according to preset rules, supplementing the clause's contextual information and allowing isolated clauses to form complete semantic units, reducing answer bias caused by semantic fragmentation. Simultaneously, integrating the table-structured parent block takes into account both textual and structured information, avoiding omissions of table knowledge. This ensures that the final context includes both textual semantics and structured data, providing more sufficient and comprehensive semantic support for the large language model, significantly improving the accuracy, completeness, and logical consistency of the final answer.

[0014] As a preferred solution, the query text input by the user is obtained, specifically: Obtain the initial query text input by the user; Based on the large language model, terms that do not meet the preset standardization requirements are identified and extracted from the initial query text to obtain non-standardized terms; Based on the aforementioned non-standardized terms, semantic vector retrieval is used to match several standard terms with the highest similarity from a preset domain terminology dictionary as mapping results to obtain synonymous extended terms; The initial query text and the synonymous extended terms are integrated, and the integration result is semantically fused based on a large language model to obtain the query text.

[0015] This optimized solution identifies and extracts non-standardized terms using a large language model, accurately capturing non-standard expressions in user queries and avoiding missed content due to terminology differences. Semantic vector retrieval based on a domain-specific terminology dictionary matches the most similar standard terms as synonymous expansion terms, achieving standardized terminology mapping, enriching the semantic dimension of the query text, and ensuring that the expanded query does not deviate from the user's initial intent. Semantic fusion integrates the initial query and synonymous expansion terms, ensuring that the query text retains the user's core needs while possessing standardized and comprehensive semantic features, significantly improving the recall and precision of subsequent searches, ensuring that search results accurately match the user's actual query needs, and reducing question-and-answer errors caused by non-standardized queries.

[0016] As a preferred approach, several clauses with the highest relevance to the query text are obtained from the initial search results, specifically: Calculate the cosine similarity between the query text and each clause in the initial search results to obtain the relevance score corresponding to each clause; Based on the relevance score, each clause in the initial search result is sorted in descending order, and the clauses with the highest relevance scores are obtained from the descending sort results.

[0017] This optimized approach filters the most relevant clauses by quantifying relevance scores, effectively improving the quality of initial search results and providing a high-quality foundation for subsequent contextual expansion. Using cosine similarity to calculate the relevance between the query text and each clause achieves precise quantification of semantic relevance, avoiding filtering biases caused by subjective judgment and making relevance assessment more objective and scientific. Sort by relevance score in descending order to quickly filter out the most relevant clauses to the query text, eliminating irrelevant or low-relevance clauses, reducing redundant information from interfering with subsequent processes, and improving the efficiency and targeting of contextual expansion. This filtering method ensures that the subsequent expanded context revolves around the core user query, avoiding contextual redundancy and semantic deviation caused by introducing irrelevant clauses, providing high-quality material support for the large language model to generate accurate answers, and further improving the accuracy of the entire intelligent question answering method.

[0018] As a preferred approach, based on the query text, semantic retrieval and clause matching are performed in a vector database, and keyword retrieval and corresponding content are performed in a relational database using a keyword inverted index. The results from these two retrieval methods are then combined to obtain the initial retrieval results, specifically: Based on the query text, semantic retrieval matching clauses are performed in the vector database to obtain vector semantic retrieval results; Based on the query text, keyword retrieval and matching of corresponding content are performed using the keyword inverted index in the relational database to obtain keyword inverted retrieval results; The vector semantic search results and the keyword inverted search results are weighted and fused to obtain the initial search results.

[0019] This optimized solution employs a two-pronged retrieval approach to obtain initial search results, effectively overcoming the limitations of single-method retrieval and improving the comprehensiveness and accuracy of the initial results. Vector semantic retrieval captures the deep semantic relationships between query text and clauses, solving the problem that keyword retrieval cannot handle semantically similar but different keywords, and avoiding missed detections due to missing keywords. Keyword inverted index retrieval quickly locates content related to core keywords, improving retrieval efficiency and compensating for the potential issues of insufficient recall and slow retrieval speed in vector retrieval. By weightedly fusing the results from both approaches, semantic relevance and retrieval efficiency are balanced, ensuring both accuracy and efficiency, effectively avoiding missed detections, false positives, or retrieval delays caused by single-method retrieval. This fusion approach provides comprehensive and accurate initial data for subsequent selection of the most relevant clauses and expansion of context, providing crucial support for the efficient and accurate operation of the entire intelligent question-answering process.

[0020] This application also provides a knowledge base intelligent question answering system based on multi-granularity slicing and context expansion, including a data acquisition module, a hybrid retrieval module, a semantic expansion module and a data output module; The data acquisition module is used to acquire the query text input by the user. The hybrid retrieval module is used to perform semantic retrieval and clause matching in a vector database based on the query text, and to perform keyword retrieval and matching of corresponding content in a relational database using a keyword inverted index, and to merge the two retrieval results to obtain the initial retrieval result; The semantic expansion module is used to obtain several clauses in the initial search results that are most relevant to the query text, locate the corresponding parent block in the relational database according to the parent block information corresponding to the several clauses, expand the several clauses into adjacent text fragments according to the location results, and integrate them to obtain the final context. The data output module is used to concatenate the final context with the preset prompt word template and input it into the large language model to generate the final answer.

[0021] This application also provides a storage medium storing a computer program, which is invoked and executed by a computer to implement the knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described above.

[0022] This application also provides a computer program product, including a computer program or instructions, which, when executed by a communication device, implements the knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described above. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a knowledge base intelligent question answering system based on multi-granularity slicing and context expansion provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the description of this application, it should be understood that the term "first" is used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "several" means two or more.

[0026] The knowledge base intelligent question answering method based on multi-granularity slicing and context expansion provided in this application aims to achieve high-precision and high-completeness intelligent question answering of unstructured long documents and their tabular information without the need to build a knowledge graph through innovative document processing, retrieval fusion and context construction mechanisms, effectively improving the accuracy and practicality of the question answering system.

[0027] Example 1: Please see Figure 1The embodiments of this application provide a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion, including S1~S4, and the specific implementation steps are as follows: S1. Obtain the query text entered by the user.

[0028] Step S1 in this embodiment of the application is specifically as follows: Get the initial query text input by the user; Based on the Large Language Model (LLM), keywords or phrases that do not meet the preset standardization requirements are identified and extracted from the initial query text to obtain non-standardized terms, such as colloquial expressions like "error code".

[0029] Based on non-standardized terminology, this system uses semantic vector retrieval to match the most similar standard terms from a pre-defined domain terminology dictionary as mapping results, thus obtaining synonymous extended terms. The "domain terminology dictionary" includes standardized term entries, along with their corresponding synonyms, abbreviations, colloquial expressions, and other related content. The core of this system is to build a basic framework based on industry norms, technical standards, and officially published professional terminology systems within the target domain. Simultaneously, it systematically supplements and verifies this framework by incorporating high-frequency professional expressions, common industry abbreviations, synonyms in different scenarios, and the precise correspondence between colloquial non-standard and standardized terms, ultimately forming a structured domain terminology mapping and extension system.

[0030] The system integrates the initial query text and synonymous extended terms, and performs semantic fusion on the integrated results based on a large language model to generate enhanced query data that is both professional and semantically complete, which is then used as the query text in subsequent retrieval processes. For example, if the original query is "how to solve error codes", after combining the standard term "error code" and the extended term "fault code" obtained through mapping, the large language model will output the enhanced query: "Please provide solutions for error codes (including similar fault codes)".

[0031] In practical applications, when a user asks "How to resolve device error code E105?", the system first retrieves the domain terminology dictionary, maps the "error code" in the query to the standard term "error code", and simultaneously completes the corresponding synonym expansion operation.

[0032] In this embodiment, S1 uses a large language model to identify and extract non-standardized terms, accurately capturing irregular expressions in user queries and avoiding missed detections of relevant content due to terminology differences. Semantic vector retrieval is performed based on a domain terminology dictionary, matching the most similar standard terms as synonymous expansion terms, achieving standardized terminology mapping, enriching the semantic dimension of the query text, and ensuring that the expanded query does not deviate from the user's initial intent. Semantic fusion integrates the initial query and synonymous expansion terms, ensuring that the query text retains the user's core needs while possessing standardized and comprehensive semantic features, significantly improving the recall and precision of subsequent searches, ensuring that search results accurately match the user's actual query needs, and reducing question-and-answer errors caused by non-standardized queries.

[0033] S2. Based on the query text, perform semantic retrieval and clause matching in the vector database, and perform keyword retrieval and content matching in the relational database using the keyword inverted index. Combine the two retrieval results to obtain the initial retrieval results.

[0034] Step S2 in this embodiment includes S2.1 to S2.3, specifically as follows: S2.1 Establish a vector database, the specific implementation method is as follows: This invention involves acquiring knowledge documents in the target domain. These documents are enterprise-level professional knowledge carriers tailored to a specific industry or business area, primarily in unstructured formats. They encompass various document types, including technical manuals, technical standard documents, and product failure case libraries, and also include non-text elements such as tables and charts that carry key information. This represents a centralized presentation of professional knowledge, business rules, and solutions within the domain. It is important to note that all knowledge documents related to the target domain, as well as the data extracted from them, require prior formal consent and legal authorization from the relevant organizations. Complete and compliant authorization procedures are necessary, and there is no unauthorized acquisition or use of related knowledge data.

[0035] Text extraction is performed on the knowledge document to obtain the basic text. Simultaneously, table areas within the knowledge document are accurately identified, converted into JSON-formatted structured parent blocks while fully preserving the table headers, rows, and columns. This structured table is then treated as an independent parent block. Specifically, cell detection and text recognition are performed sequentially on the detected table areas, and then the topological relationships between cells are reconstructed using an algorithm. The final output contains... 、 、 The format of HTML tags, or a JSON object with caption, headers, and rows keys, is used as a whole as a text string to participate in subsequent parent block construction and vectorization operations.

[0036] A two-level parent-child slicing strategy is adopted. First, the basic text of the document is divided into several parent blocks according to complete logical units by combining rules and semantic models. Then, for each parent block and the table-structured parent block, clauses are segmented within them according to semantic boundaries and sentence coherence. While retaining the original parent block data, the segmented clause data is generated, and the clause data is bound to the corresponding parent block identifier (ID) and its position offset within the parent block. Here, "dividing the basic text of the document into several parent blocks according to complete logical units" means combining the inherent structure of the document, such as chapter and section titles, with the semantic model to divide the text content of the basic text of the document into content modules with independent and complete knowledge semantics. Each module is a parent block, ensuring that a single parent block can carry complete information of a certain core theme / category without breaking the coherent knowledge logic. The "semantic model" used for parent block segmentation here specifically refers to a natural language processing semantic understanding model adapted to enterprise-level knowledge document processing. It can analyze the theme relevance and content coherence of the text from a semantic level, supplementing and calibrating the segmentation method based on the inherent structure of the document, thereby ensuring the logical integrity of the parent block segmentation. "Parent block identifier" is a unique identifier assigned to each parent block that is divided into complete logical units. This identifier can be a number or a combination of codes. It is used to accurately associate clauses with their parent blocks and is also the core indexing basis for locating the corresponding parent block through clauses and retrieving the complete content of the parent block from the relational database. "Offset within the parent block" is the specific position information of each clause within its parent block, such as paragraph number, sentence number, or character offset value. It is used to accurately locate the specific position of the clause within the parent block, providing a positional basis for extracting adjacent text segments before and after the clause and implementing dynamic context expansion.

[0037] Embedding models such as BGE-M3 are used to vectorize all clauses in the clause data, generating corresponding high-dimensional clause vector data, and a vector index is built based on the high-dimensional clause vector data; The high-dimensional vector data of the clauses, carrying the parent block identifier and position offset, along with the constructed vector index, are stored in the initial vector database to obtain the vector database. Among them, Milvus is preferred as the initial vector database; Milvus is an open-source, high-performance, and highly scalable vector database developed by Zilliz, designed specifically for the storage, indexing, and similarity search of large-scale high-dimensional vector data, and can run efficiently in various software and hardware environments.

[0038] In this embodiment S2.1, by converting tables in the knowledge document into structured parent blocks and processing them independently, the fragmentation and misinterpretation of table information due to text extraction are avoided, thus ensuring the integrity of structured knowledge. The multi-granularity processing method, which divides parent blocks according to preset logical units and segments clauses according to semantic boundaries, preserves the overall relevance of knowledge while achieving refined semantic segmentation. Simultaneously, it binds parent block identifiers and position offsets to clauses, providing precise positioning basis for subsequent context expansion. Vectorization processing based on the embedding model enables accurate representation of clause semantics. After being stored in the vector database, it significantly improves the matching accuracy and efficiency of subsequent semantic retrieval, laying a data foundation for the accuracy of the entire intelligent question-answering method and effectively avoiding retrieval bias or context loss problems caused by single-granularity slicing.

[0039] S2.2 Establish a relational database, the specific implementation method is as follows: Using a large language model, several target keywords are extracted from each parent block in the aforementioned knowledge document and the table-structured parent block, and a summary is generated. A corresponding keyword inverted index is constructed for the target keywords. The "keyword inverted index" consists of keywords, parent block identifiers, and clause positions. The "several target keywords" should cover the main information content (core theme or entity) of the parent block and be able to effectively distinguish the content of different parent blocks. The number of keywords is preferably 10 to 20. The clause data, several parent blocks, parent block identifiers, several target keywords, structured parent blocks in a table, keyword inverted index, and summary are stored in an initial relational database to obtain the relational database. The initial relational database can be selected based on the size of the knowledge base, the technology stack, and the use case. For production environments, open-source databases such as MySQL are preferred; for large enterprise commercial scenarios, Oracle can be used; and for smaller scenarios, SQLite can be used. All databases must meet the core requirements of structured / semi-structured data storage, index building, and relational queries.

[0040] The relational database constructed in S2.2 of this embodiment complements the vector database, further enhancing the retrieval capabilities of the knowledge base. By extracting target keywords for each parent block and constructing an inverted index for those keywords, a precise association between keywords and parent blocks / clause positions is achieved, significantly improving the speed and targeting of keyword retrieval and solving the problems of low retrieval efficiency and ambiguous positioning in traditional relational databases. Integrating and storing clause data, parent block identifiers, target keywords, structured table data, and inverted indexes enables structured knowledge management, facilitating rapid retrieval of various data types while ensuring data relevance and integrity. This storage method allows for rapid location of relevant content via keywords during subsequent retrievals and convenient access to the parent block to which the clause belongs and related information, providing efficient data support for contextual expansion and retrieval result fusion, effectively improving the coherence and reliability of the entire question-and-answer process.

[0041] S2.3 Based on the query text, semantic retrieval is carried out in the vector database using vector indexes. By vectorizing the query text and the clause text in the vector database respectively and calculating the cosine similarity, relevant clauses are matched to obtain vector semantic retrieval results. The score range of the cosine similarity calculation is 0-1 points, and the higher the score, the more similar the two are semantically. Based on the query text, keyword retrieval is performed in a relational database using the keyword inverted index to match relevant content and obtain keyword inverted retrieval results; Following a pre-configured weighting strategy corresponding to the knowledge base type, the system weights and merges the vector semantic search results and the keyword inverted index search results, and then performs result merging and deduplication to obtain the initial search results. Furthermore, this weighting strategy can be flexibly configured according to the business attributes of the knowledge base; for example, the semantic search weight can be configured to 0.7 and the keyword search weight to 0.3. In practical application scenarios, such as for the query requirement of "resolving device error code E105", the system will search for semantically similar clauses in the vector database and retrieve parent blocks containing core keywords such as "E105" and "error" in the relational database. The two search results are then merged and deduplicated according to the preset weights to form the corresponding initial search results.

[0042] This embodiment, S2.3, employs a two-way retrieval fusion approach to obtain initial search results, effectively overcoming the limitations of a single retrieval method and improving the comprehensiveness and accuracy of the initial search results. Vector semantic retrieval can capture the deep semantic relationships between query text and clauses, solving the problem that keyword retrieval cannot handle semantically similar but different keywords, and avoiding missed detections due to missing keywords. Keyword inverted index retrieval can quickly locate content related to core keywords, improving retrieval efficiency and compensating for the potential issues of insufficient recall and slow retrieval speed in vector retrieval. By weightedly fusing the two retrieval results, both semantic relevance and retrieval efficiency are considered, ensuring both the accuracy and efficiency of the search results, effectively avoiding the problems of missed detections, false detections, or retrieval delays caused by a single retrieval method. This fusion method provides comprehensive and accurate initial data for subsequent selection of the most relevant clauses and expansion of context, providing key support for the efficient and accurate operation of the entire intelligent question-answering process.

[0043] It should be noted that the division of parent blocks is primarily based on the inherent structural markers of the document, such as heading markers like ## and §. Alternatively, a trained classifier can be used to identify logical paragraphs in the document to achieve a reasonable division. Clause segmentation integrates punctuation and semantic segmentation models to ensure that each clause can express a relatively complete sub-viewpoint and guarantee the semantic independence of fine-grained slices; In the index association process, the corresponding parent block ID of each clause vector needs to be stored, and the position offset of the clause within its parent block needs to be recorded to achieve accurate association and tracing between clauses and parent blocks; The weight configuration for hybrid retrieval is implemented through the "Knowledge Base Type - Retrieval Strategy" configuration file maintained by the system. The weight ratio of semantic retrieval and keyword retrieval can be flexibly adjusted according to the characteristics of the knowledge base type. Specifically, for the technical standard document library, to emphasize the need for semantic understanding, a semantic weight of 0.8 and a keyword weight of 0.2 can be configured; for the product fault case library, to enhance the accurate matching of error codes, device models, and other information, a semantic weight of 0.4 and a keyword weight of 0.6 can be configured.

[0044] S3. Obtain the clauses with the highest relevance to the query text from the initial search results. Locate the corresponding parent block in the relational database based on the parent block information of the clauses. Expand the adjacent text segments of the clauses based on the location results and integrate them to obtain the final context.

[0045] Step S3 in this embodiment includes S3.1 to S3.2, specifically as follows: S3.1. Vectorize the query text and each clause in the initial search results. Then, based on the vectorization results, use a re-ranking model to calculate the cosine similarity between the query text and each clause to obtain the relevance score for each clause. Based on the relevance score, sort each clause in the initial search results in descending order and obtain the clauses with the highest relevance scores from the descending sort results.

[0046] Among them, the "re-ranking model" is a machine learning model in the fields of information retrieval and natural language processing that, after the initial recall / coarse ranking of candidate text sets, uses a deep learning model to deeply evaluate the matching degree between the query and each text and re-rank them to significantly improve the relevance and accuracy of the final result.

[0047] In this embodiment, S3.1 filters the most relevant clauses by quantifying relevance scores, effectively improving the quality of the initial search results and providing a high-quality foundation for subsequent context expansion. Cosine similarity is used to calculate the relevance between the query text and each clause, achieving precise quantification of semantic relevance and avoiding filtering biases caused by subjective judgment, making the relevance assessment more objective and scientific. Sorting by relevance score in descending order quickly filters out the clauses most relevant to the query text, eliminating irrelevant or low-relevance clauses, reducing redundant information from interfering with subsequent processes, and improving the efficiency and targeting of context expansion. This filtering method ensures that the subsequent expanded context revolves around the core user query, avoiding contextual redundancy and semantic deviation caused by introducing irrelevant clauses, providing high-quality material support for the large language model to generate accurate answers, and further improving the accuracy of the entire intelligent question answering method.

[0048] S3.2. Using the parent block identifier bound to several clauses as the retrieval basis, match in the relational database to obtain the parent block data to which several clauses belong; Using the position offset of several clauses bound in their parent block as the positioning standard, and according to preset rules, extract a preset number of adjacent text segments before and after each clause from the acquired parent block data, and integrate the several clauses with their respective matched adjacent text segments to obtain combined text data; Based on the query text and the parent block identifier bound to several clauses, the corresponding first table structured parent block is retrieved in the relational database; By integrating and combining the text data with the first table structured parent block, the final context is obtained.

[0049] In this embodiment, S3.2 accurately locates the parent block to which the clause belongs by identifying the parent block, ensuring the accuracy of context expansion and avoiding semantic deviation caused by positioning errors. Using the positional offset as a standard, adjacent text fragments before and after the clause are extracted according to preset rules, supplementing the clause's contextual information and allowing isolated clauses to form complete semantic units, reducing answer deviations caused by semantic fragmentation. Simultaneously, integrating the table-structured parent block takes into account both textual and structured information, avoiding omissions of table knowledge. This ensures that the final context includes both textual semantics and structured data, providing more sufficient and comprehensive semantic support for the large language model, significantly improving the accuracy, completeness, and logic of the final answer.

[0050] S4. After concatenating the final context with the preset prompt word template, input it into the large language model to generate the final answer.

[0051] Step S4 in this embodiment is specifically as follows: The assembled final context is combined with a pre-set prompt word template. This combined result is then input into a large language model, such as the QWEN series, via an API interface. The model then generates the corresponding initial answer based on the context information and the specific requirements of the prompt words. The "prompt word template" is a standardized instruction text pre-set to meet the business requirements of knowledge base intelligent question answering. It serves as a fixed paradigm to constrain and guide the large language model in generating answers. It clearly defines the format, expression logic, and answering requirements of the answers generated by the model based on the knowledge base context, ensuring that the model's output answers conform to enterprise-level knowledge question answering standards and actual usage needs.

[0052] The initial answers generated by the large language model are standardized and formatted, and the knowledge source parent block information corresponding to the answers is accurately labeled. Finally, the formatted answers with source annotations are fed back to the user.

[0053] Overall, this embodiment has the following beneficial effects: This invention employs a dual-path retrieval fusion strategy, which balances semantic relevance and keyword accuracy. Semantic retrieval from a vector database deeply mines the semantic connections between the query text and clauses, avoiding the literal limitations of keyword retrieval. The inverted keyword index from a relational database achieves precise matching between keywords and content, compensating for potential ambiguities in semantic retrieval. The fusion of these two results effectively improves the accuracy of the initial retrieval results, solving the problem of insufficient precision in single-path retrieval. First, the clause with the highest relevance to the query text is selected. Then, the corresponding parent block is located based on the parent block information. Adjacent text fragments are expanded based on the parent block, ensuring that the expanded text fragments are highly relevant to the core clauses. This avoids the introduction of irrelevant information in traditional expansion methods, providing accurate and complete contextual support for the large language model. Finally, the final context is concatenated with a preset prompt word template and input into the large language model. This template standardizes the model's generation logic and anchors the answer direction, ensuring that the generated content closely adheres to the context and fits the core needs of the knowledge base. This reduces retrieval bias, messy expansion, and inaccurate answers caused by unconstrained model generation from the source, ultimately achieving a simultaneous improvement in retrieval accuracy, expansion precision, and answer accuracy. In summary, this invention employs a two-level parent-child slicing strategy combined with a dynamic context expansion mechanism. Fine-grained clauses ensure retrieval accuracy, while complete parent blocks guarantee contextual integrity, fundamentally solving the problem of fragmented information in long documents. This provides ample and coherent reasoning for large models, significantly improving the accuracy of long document question answering. Simultaneously, it innovatively designs a configurable hybrid retrieval strategy, organically integrating the generalization ability of semantic retrieval with the precise matching ability of keyword retrieval. Weights can be flexibly adjusted according to domain characteristics, effectively bridging the semantic gap between user speech and professional text, significantly improving the overall recall rate and scenario adaptability of complex and diverse queries. Furthermore, the solution innovatively extracts and indexes tables as independent semantic units in a structured manner, fully preserving their row and column logical relationships, achieving accurate processing and question answering of non-textual information, filling a gap in existing technologies. Furthermore, the solution abandons the traditional knowledge graph construction and maintenance process, creating a lightweight technology system that significantly reduces implementation costs and shortens the implementation cycle. The core modules are all designed to be configurable, and combined with an automated testing platform, they form a continuously iterative operational loop, giving the system strong practicality and operability, and enabling it to better adapt to the massive and rapidly changing unstructured knowledge document application needs of enterprises.

[0054] Example 2: Please see Figure 2 The embodiments of this application provide a knowledge base intelligent question answering system based on multi-granularity slicing and context expansion, including a data acquisition module 10, a hybrid retrieval module 20, a semantic expansion module 30 and a data output module 40; The data acquisition module 10 is used to acquire the query text input by the user. The hybrid retrieval module 20 is used to perform semantic retrieval and clause matching in a vector database based on the query text, and keyword retrieval and matching of corresponding content in a relational database through keyword inverted index, and to merge the two retrieval results to obtain the initial retrieval results; The semantic extension module 30 is used to obtain several clauses that are most relevant to the query text in the initial search results, locate the corresponding parent block in the relational database according to the parent block information of the several clauses, expand the adjacent text fragments of the several clauses according to the location results, and integrate them to obtain the final context. The data output module 40 is used to concatenate the final context with the preset prompt word template and input it into the large language model to generate the final answer.

[0055] It should be noted that the technical concept of this second embodiment is completely consistent with that of the first embodiment. The two maintain a high degree of synergy at the technical logic level. The specific technical details can be referred to the relevant description of the first embodiment, which will not be repeated here.

[0056] Example 3: This application provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the aforementioned intelligent question-answering method for a knowledge base based on multi-granularity slicing and context expansion when it is running. The knowledge base intelligent question-answering method based on multi-granularity slicing and context expansion, when implemented as a software functional unit and used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0057] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video optical disc; or it can be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both types of storage media.

[0058] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A knowledge base intelligent question answering method based on multi-granularity slicing and context expansion, characterized in that, include: Get the query text entered by the user; Based on the query text, semantic retrieval and clause matching are performed in a vector database, and keyword retrieval and corresponding content are performed in a relational database using a keyword inverted index. The two retrieval results are then combined to obtain the initial retrieval results. Obtain several clauses from the initial search results that are most relevant to the query text. Locate the corresponding parent block in the relational database based on the parent block information corresponding to the several clauses. Expand the several clauses into adjacent text segments based on the location results and integrate them to obtain the final context. The final context is concatenated with the preset prompt word template and then input into the large language model to generate the final answer.

2. The knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described in claim 1, characterized in that, The methods for obtaining the vector database include: Acquire knowledge documents in the target domain; Text extraction is performed on the knowledge document, and the tables in the knowledge document are identified and converted into structured data and processed as independent parent blocks to obtain the basic text of the document and the structured parent blocks of the tables respectively. The document's basic text is divided into several parent blocks according to preset logical units. Within the parent blocks and the table-structured parent blocks, clauses are segmented according to semantic boundaries. While retaining the original parent block data, the segmented clause data is generated, and the clause data is bound to the corresponding parent block identifier and its position offset within the parent block. The clause data is vectorized based on the embedding model to generate high-dimensional vector data of the clauses. The high-dimensional vector data of the clause, carrying the parent block identifier and the position offset, is stored in the initial vector database to obtain the vector database.

3. The knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described in claim 2, characterized in that, The methods for obtaining the relational database include: From the knowledge document, extract several target keywords for each of the several parent blocks and the table structured parent blocks, and construct a corresponding keyword inverted index for the several target keywords; wherein, the keyword inverted index consists of keywords, parent block identifiers and clause positions; The clause data, the parent block identifier, the target keywords, the table structured parent block, and the keyword inverted index are stored in the initial relational database to obtain the relational database.

4. The knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described in claim 3, characterized in that, Based on the parent block information corresponding to the aforementioned clauses, the corresponding parent block is located in the relational database. Based on the location result, adjacent text segments are expanded from the aforementioned clauses, and the resulting context is integrated to obtain the final context. Specifically: Using the parent block identifier bound to the aforementioned clauses as the retrieval basis, a match is performed in the relational database to obtain the parent block data to which the aforementioned clauses belong; Using the position offset of the clauses bound in their parent block as the positioning standard, and according to preset rules, extract a preset number of adjacent text segments before and after each clause from the acquired parent block data, and integrate them to obtain combined text data composed of the clauses and their corresponding adjacent text segments. Based on the query text and the parent block identifier bound to the clauses, the corresponding first table structured parent block is retrieved in the relational database; The combined text data and the first table structured parent block are integrated to obtain the final context.

5. The intelligent question-answering method for a knowledge base based on multi-granularity slicing and context expansion as described in claim 1, characterized in that, Obtain the query text entered by the user, specifically: Obtain the initial query text input by the user; Based on the large language model, terms that do not meet the preset standardization requirements are identified and extracted from the initial query text to obtain non-standardized terms; Based on the aforementioned non-standardized terms, semantic vector retrieval is used to match several standard terms with the highest similarity from a preset domain terminology dictionary as mapping results to obtain synonymous extended terms; The initial query text and the synonymous extended terms are integrated, and the integration result is semantically fused based on a large language model to obtain the query text.

6. The intelligent question-answering method for a knowledge base based on multi-granularity slicing and context expansion as described in claim 1, characterized in that, Specifically, the following steps are taken: First, obtain the clauses from the initial search results that are most relevant to the query text. Calculate the cosine similarity between the query text and each clause in the initial search results to obtain the relevance score corresponding to each clause; Based on the relevance score, each clause in the initial search result is sorted in descending order, and the clauses with the highest relevance scores are obtained from the descending sort results.

7. The intelligent question-answering method for a knowledge base based on multi-granularity slicing and context expansion as described in claim 1, characterized in that, Based on the query text, semantic retrieval and clause matching are performed in a vector database, and keyword retrieval and corresponding content are performed in a relational database using a keyword inverted index. The results from both retrieval methods are then combined to obtain the initial retrieval results, specifically: Based on the query text, semantic retrieval matching clauses are performed in the vector database to obtain vector semantic retrieval results; Based on the query text, keyword retrieval and matching of corresponding content are performed using the keyword inverted index in the relational database to obtain keyword inverted retrieval results; The vector semantic search results and the keyword inverted search results are weighted and fused to obtain the initial search results.

8. A knowledge base intelligent question-answering system based on multi-granularity slicing and context expansion, characterized in that, It includes a data acquisition module, a hybrid retrieval module, a semantic expansion module, and a data output module; The data acquisition module is used to acquire the query text input by the user. The hybrid retrieval module is used to perform semantic retrieval and clause matching in a vector database based on the query text, and to perform keyword retrieval and matching of corresponding content in a relational database using a keyword inverted index, and to merge the two retrieval results to obtain the initial retrieval result; The semantic expansion module is used to obtain several clauses in the initial search results that are most relevant to the query text, locate the corresponding parent block in the relational database according to the parent block information corresponding to the several clauses, expand the several clauses into adjacent text fragments according to the location results, and integrate them to obtain the final context. The data output module is used to concatenate the final context with the preset prompt word template and input it into the large language model to generate the final answer.

9. A storage medium, characterized in that, The storage medium stores a computer program, which is called and executed by a computer to implement a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the communication device, they implement a knowledge base intelligent question answering method based on multi-granularity slicing and context expansion as described in any one of claims 1 to 7.

Images