Knowledge text retrieval method and device, storage medium and computer equipment

Abstract

According to the knowledge text retrieval method and device, the storage medium and the computer equipment, when knowledge text retrieval is carried out, the retrieval statement and the target knowledge document corresponding to the retrieval statement are obtained firstly, the document structure tree of the target knowledge document is determined, the document structure tree is of a tree structure constructed by utilizing the directory structure of the target knowledge document, and the retrieval statement is stored in the document structure tree; according to the method, a target knowledge document is taken as a component of the target knowledge document, an index structure between each index node in the component of the target knowledge document and the target knowledge document is recorded, and leaf nodes in the index nodes are connected through pointers to form a linked list structure. The corresponding index node can be dynamically retrieved based on the association relationship between the index structure and the linked list structure of the tree structure, and the context content corresponding to the retrieval statement in the target knowledge document is intercepted by adopting different modes based on the type of the index node. And omission of unrelated contents of other chapters and key contents of the same chapter is avoided, so that the retrieval precision and quality are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a knowledge text retrieval method, device, storage medium and computer equipment. Background Art

[0002] With the continuous advancement of large language model technology, enterprise-level intelligent customer service systems are becoming more and more widely used. These systems rely on the powerful language organization capabilities of large language models to provide efficient customer service. However, the corpus used in the training process of large language models often cannot cover all possible questions that users may ask, especially those involving enterprise-specific business scenarios and proprietary knowledge.

[0003] To make up for this deficiency, RAG (Retrieval Augmented Generation) technology came into being. It enhances the ability of large language models to understand and answer user questions by introducing external prompt information. However, it is obviously unrealistic to input all of the company's private knowledge data into a large language model indiscriminately, which will not only increase the computing cost, but also reduce the pertinence of the answer. Therefore, it is necessary to retrieve and extract the key information points most relevant to the user's question from the huge knowledge base, and then provide them to the large language model.

[0004] The current knowledge text retrieval method mainly divides long texts according to the number of words or tokens to form multiple text fragments, and indexes these fragments into the search database. When a user asks a question, the system retrieves several text fragments related to the user's question from the database, and then inputs them into a large language model to assist in answering. However, the text segmentation method in this method is not accurate in the segmentation position, resulting in information fragmentation, making it easy for the retrieval results to miss the key content containing the answer or be mixed with irrelevant content, resulting in low retrieval accuracy and quality. Summary of the invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the text segmentation method in the prior art leads to information fragmentation, which makes the search results easily miss the key content containing the answer or are mixed with irrelevant content, resulting in the technical defects of low search accuracy and quality.

[0006] The present application provides a knowledge text retrieval method, the method comprising:

[0007] Obtain a search statement and a target knowledge document corresponding to the search statement, and determine a document structure tree of the target knowledge document; the document structure tree is constructed according to the directory structure of the target knowledge document, and includes a plurality of index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure;

[0008] Using keyword indexing technology and vector indexing technology to index the document structure tree to obtain an index node corresponding to the search statement;

[0009] When it is determined that the index node is a branch node, determining the chapter title and chapter length corresponding to the branch node in the target knowledge document, and recalling the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text;

[0010] When it is determined that the index node is a leaf node, the paragraph text corresponding to the leaf node in the target document is determined, and the paragraph text is contextually expanded according to the linked list structure and the preset recall length, and auxiliary knowledge text is generated according to the expansion result.

[0011] Optionally, the process of constructing the document structure tree includes:

[0012] Acquire a target knowledge document, and perform structural analysis on the target knowledge document to obtain a directory structure; the directory structure includes a document title, a chapter title, and a paragraph body;

[0013] Based on the directory structure, a document structure tree of the target knowledge document is constructed with the document title as the root node, the chapter title as the branch node, and the paragraph body as the leaf node;

[0014] For each node in the document structure tree, a keyword index and a vector index of the node are constructed based on the target knowledge document to form an index node;

[0015] According to the logical order of the text in the target knowledge document, pointers are established between the leaf nodes in the document structure tree after the index is constructed to connect them and form a linked list structure.

[0016] Optionally, constructing a keyword index and a vector index of the node based on the target knowledge document to form an index node includes:

[0017] Determine the text content corresponding to the node from the target knowledge document, extract index keywords of the text content, and generate an index vector of the text content;

[0018] An index structure is established between the index keyword, the index vector and the text content respectively, and the index structure is added to the node to form an index node.

[0019] Optionally, the step of establishing pointers between leaf nodes in the document structure tree after indexing according to the text logical order in the target knowledge document to connect them and form a linked list structure includes:

[0020] Sort each leaf node according to the logical order of the text in the target knowledge document to obtain a sorting result;

[0021] Determine in sequence whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document;

[0022] If yes, a pointer is established between the two adjacent leaf nodes to connect them;

[0023] If not, no connection is made between the two adjacent leaf nodes.

[0024] Optionally, the step of using a keyword indexing technique and a vector indexing technique to index an index node corresponding to the search statement from the document structure tree includes:

[0025] Extracting search keywords of the search statement and generating a search vector of the search statement;

[0026] The BM25 algorithm is used to calculate the search keyword in the document structure tree to obtain a first index result, and the HNSW algorithm is used to calculate the search vector to obtain a second index result;

[0027] An index node corresponding to the search statement is determined according to the first index result and the second index result.

[0028] Optionally, the database pre-stores a target knowledge document and a chapter summary corresponding to each chapter title in the target knowledge document; the chapter summary is obtained by extracting the chapter text under the corresponding chapter title;

[0029] The chapter text or chapter summary under the chapter title is recalled from the database based on the length of the chapter as auxiliary knowledge text, including:

[0030] Determine whether the length of the chapter exceeds a preset output length;

[0031] If so, the chapter summary corresponding to the chapter title is recalled from the database as auxiliary knowledge text;

[0032] If not, the total text of the chapter corresponding to the chapter title is recalled from the database as auxiliary knowledge text.

[0033] Optionally, the context expansion of the paragraph body according to the linked list structure and the preset recall length, and generating auxiliary knowledge text according to the expansion result, includes:

[0034] Determine the branch node to which the leaf node belongs and the chapter title corresponding to the branch node, and judge whether the length of the chapter corresponding to the chapter title exceeds the preset output length;

[0035] If so, recall the leaf node and the paragraph texts corresponding to the leaf nodes above and below the leaf node from the database in sequence according to the pointer sequence in the linked list structure, until the recalled paragraph texts reach the preset output length, and use the recalled paragraph texts as auxiliary knowledge texts;

[0036] If not, the chapter text under the chapter title is recalled from the database as auxiliary knowledge text.

[0037] The present application also provides a knowledge text retrieval device, comprising:

[0038] An information acquisition module is used to acquire a search statement and a target knowledge document corresponding to the search statement, and determine a document structure tree of the target knowledge document; the document structure tree is constructed according to the directory structure of the target knowledge document, and includes a plurality of index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure;

[0039] A node indexing module, used to obtain an index node corresponding to the search statement from the document structure tree by using a keyword indexing technique and a vector indexing technique;

[0040] A first text generation module is used for determining, when the index node is determined to be a branch node, a chapter title and a chapter length corresponding to the branch node in the target knowledge document, so as to recall the chapter text or chapter summary under the chapter title from a database based on the chapter length as an auxiliary knowledge text;

[0041] The second text generation module is used to determine the paragraph text corresponding to the leaf node in the target document when it is determined that the index node is a leaf node, and to contextually expand the paragraph text according to the linked list structure and the preset recall length, and to generate auxiliary knowledge text according to the expansion result.

[0042] The present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the knowledge text retrieval method described in any one of the above embodiments.

[0043] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0044] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the knowledge text retrieval method described in any one of the above embodiments are performed.

[0045] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0046] The knowledge text retrieval method, device, storage medium and computer equipment provided by the present application can first obtain the search statement and its corresponding target knowledge document, and determine the document structure tree of the target knowledge document when performing knowledge text retrieval. The document structure tree here is a tree structure constructed using the directory structure of the target knowledge document, and records the index structure between each index node in its components and the target knowledge document, wherein the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure. Therefore, when searching, the present application can dynamically intercept the context content corresponding to the search statement in the target knowledge document based on the association relationship between the index structure of the tree structure and the linked list structure, thereby improving the accuracy and quality of the retrieval. For example, keyword indexing technology and vector indexing technology can be used to index the index nodes corresponding to the search statements from the document structure tree, so as to improve the semantic relevance of the search results through two-dimensional indexing technology, thereby improving the search accuracy; among them, if the index node is a branch node, the chapter title and chapter length corresponding to the branch node in the target knowledge document can be determined, so as to recall the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text, so as to avoid returning irrelevant content of other chapters; if the index node is a leaf node, the paragraph text corresponding to the leaf node in the target document can be determined, and the paragraph text can be contextually expanded according to the linked list structure and the preset recall length, and auxiliary knowledge text can be generated according to the expansion result to avoid missing key content. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 A flowchart of a knowledge text retrieval method provided in an embodiment of the present application;

[0049] Figure 2 A logical diagram of a chapter summary generation storage process provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a document structure tree provided in an embodiment of the present application;

[0051] Figure 4 A logical diagram of a node indexing process provided in an embodiment of the present application;

[0052] Figure 5 A logical schematic diagram of a branch node recall process provided in an embodiment of the present application;

[0053] Figure 6 A logical schematic diagram of a leaf node recall process provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the structure of a knowledge text retrieval device provided in an embodiment of the present application;

[0055] Figure 8 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] The current knowledge text retrieval method mainly divides long texts according to the number of words or tokens to form multiple text fragments, and indexes these fragments into the search database. When a user asks a question, the system retrieves several text fragments related to the user's question from the database, and then inputs them into a large language model to assist in answering. However, the text segmentation method in this method is not accurate in the segmentation position, resulting in information fragmentation, making it easy for the retrieval results to miss the key content containing the answer or be mixed with irrelevant content, resulting in low retrieval accuracy and quality.

[0058] Based on this, this application proposes the following technical solutions, see below for details:

[0059] In one embodiment, Figure 1 As shown, Figure 1 A flowchart of a knowledge text retrieval method provided in an embodiment of the present application; the present application provides a knowledge text retrieval method, which specifically includes the following:

[0060] S110: Obtain the search statement and the target knowledge document corresponding to the search statement, and determine the document structure tree of the target knowledge document; the document structure tree is constructed according to the directory structure of the target knowledge document, including multiple index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure.

[0061] In this step, when the user needs to obtain professional knowledge text related to the search statement as text assistance, the user can first enter the search statement in the search interface of the computer device, as well as the target knowledge document corresponding to the search statement, that is, the document file that records the enterprise's private professional knowledge content corresponding to the search background of the search statement. After obtaining the target knowledge document, the computer device can also determine the document structure tree corresponding to the target knowledge document, and then can index the auxiliary knowledge text corresponding to the search statement from the target knowledge document based on the document structure tree.

[0062] It can be understood that the document structure tree here is a tree-like framework with parent-child and association relationships built based on the document directory structure. It divides the content into multiple levels of index nodes from the whole to the details according to the logical organization of the document, such as the root node at the highest level, the branch nodes at the middle level, and the leaf nodes at the lowest level; the document structure tree can reflect the logical hierarchy of the document through the index nodes at each level. For example, the branch nodes can be used to represent the chapter content in the document, including the chapter title and the chapter text under the chapter title, and the leaf nodes extending from the branch nodes are used to represent the specific content items or fine-grained knowledge points in the chapter text. Therefore, the computer device can efficiently organize and index the document content in the document through the document structure tree, which is convenient for rapid retrieval and positioning of the target text.

[0063] It should be noted that, unlike the traditional document structure tree, the leaf nodes in the document structure tree of the present application can be connected by pointers to form a linked list structure, so as to enhance the logical relationship between the underlying nodes, that is, the leaf nodes, through the pointers in the linked list structure to achieve contextual association. In this way, after the computer device locates the corresponding text content through a leaf node, it can also determine and smoothly access its contextual content through the tree index and linked list structure of the document structure tree, thereby avoiding the omission of key content and / or the interspersion of irrelevant content due to the fragmentation of the text content corresponding to the located leaf node.

[0064] S120: Using keyword indexing technology and vector indexing technology to index the document structure tree to obtain an index node corresponding to the search statement.

[0065] In this step, after determining the document structure tree of the target knowledge document through step S110, the computer device can use keyword indexing technology and vector indexing technology to index the index nodes corresponding to the search statement from the document structure tree, so as to improve the semantic relevance of the index results through these two dimensional indexing technologies, thereby improving the accuracy of subsequent retrieval.

[0066] Among them, keyword indexing technology and vector indexing technology are two different information retrieval technologies, which process documents and search statements in different ways to improve search efficiency and semantic relevance. To elaborate, keyword indexing technology refers to a direct matching technology based on words, which can quickly find document content containing certain keywords by building an inverted index; vector indexing technology refers to a retrieval method based on semantic matching, which matches by converting document content and search statements into high-dimensional vectors and calculating the similarity between vectors.

[0067] It is understandable that keyword indexing technology relies on direct matching of explicit keywords in document content. It can quickly locate knowledge points or text fragments containing keywords. It is an efficient and basic retrieval method. Especially in scenarios where the search statements are clear or the keywords are clear, it can show extremely high performance, but it cannot identify semantically similar content but expressed differently, and it is difficult to handle context or complex semantic relationships. Vector indexing technology can convert document content and search statements into high-dimensional semantic vectors, and use semantic similarity calculations to match content. Even if the text content is different or the expression is diversified, it can capture semantic associations. However, when it faces longer texts, the quality of its corresponding vector will decrease, resulting in inaccurate semantic relevance calculations. Therefore, the present application can utilize the complementarity of the characteristics of keyword indexing technology and vector indexing technology to enhance the semantic relevance of index results from different levels, thereby improving the accuracy of subsequent retrieval.

[0068] S130: When the index node is determined to be a branch node, the chapter title and chapter length corresponding to the branch node in the target knowledge document are determined, so as to recall the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text.

[0069] In this step, after indexing the index node in step S120, the computer device can determine the node type of the index node. If the index node is a branch node, the computer device can determine the chapter title corresponding to the branch node in the target knowledge document, and the chapter length corresponding to the chapter text under the chapter title, and then can recall the chapter text or chapter summary of the chapter title from the database based on the chapter length as auxiliary knowledge text, thereby avoiding returning irrelevant content of other chapters.

[0070] It is understandable that the chapter text refers to the complete content under the chapter title, including detailed information, background, analysis, argument or narrative, which is usually unfolded in a logical order and covers all details and supporting information in the chapter; while the chapter summary refers to a highly summarized and refined expression of the chapter text, aiming to extract the core points, main ideas or key information of the chapter, and present the overall ideas and logical framework of the chapter in a concise form. Therefore, the thematic logic of the chapter summary is consistent with the chapter text, but the number of words in the chapter summary is much lower than the number of words in the chapter text. Based on this, the present application can control the output length of the auxiliary knowledge text through the recall of the chapter text or chapter summary to avoid the auxiliary knowledge text being too long and affecting the application performance of the large language model.

[0071] Specifically, when the index node is determined to be a branch node, the computer device can further determine the chapter title and chapter length corresponding to the branch node in the target knowledge document, so as to clearly define the chapter position and content range corresponding to the branch node in the target knowledge document, thereby avoiding the mistaken inclusion of irrelevant content of other chapters in the recall results. The computer device can then determine whether it is necessary to recall the chapter title directly from the database or recall the chapter summary based on the chapter length corresponding to the chapter text, so as to meet the needs of rapid understanding of the subsequent large prediction model. In this way, the computer device can flexibly adjust the granularity of the recalled content, return a more comprehensive chapter text or a more refined chapter summary according to the model requirements, and provide more targeted support for knowledge text assistance in different scenarios.

[0072] Furthermore, the present application can pre-extract and store the chapter summaries of the chapter texts under each chapter title in the target knowledge document. Figure 2 As shown, Figure 2 A logical diagram of a chapter summary generation storage process provided in an embodiment of the present application; Figure 2 In the example, for each chapter title, the computer device can first obtain the chapter text containing multiple paragraphs under the chapter title, and then input the entire chapter text into the LLM (Large Language Model) to extract and summarize the key points, and output it to form a chapter summary; after extracting the chapter summary of each chapter text, the computer device can also establish an index relationship between the chapter summary and the corresponding chapter title, and store it in the database.

[0073] S140: When the index node is determined to be a node, the paragraph text corresponding to the leaf node in the target document is determined, and the paragraph text is contextually expanded according to the linked list structure and the preset recall length, and the auxiliary knowledge text is generated according to the expansion result.

[0074] In this step, after obtaining the index node corresponding to the search statement in step S120, the computer device can determine the node type of the index node. If the index node is a leaf node, the computer device can determine the paragraph text corresponding to the leaf node in the target knowledge document, and perform context expansion on the paragraph text according to the linked list structure and the preset recall length, and then generate auxiliary knowledge text according to the expansion result to avoid missing key content.

[0075] Among them, the preset recall length refers to the output length range set by the computer device to assist in the generation of knowledge text. The preset recall length can be flexibly set according to the model requirements and defined in the form of the number of paragraphs, the number of sentences or the number of text characters, etc., and is not restricted here.

[0076] Specifically, if the index node is a leaf node, it indicates that it is a specific paragraph text in a chapter in the target knowledge document. Therefore, in order to ensure the contextual coherence of the auxiliary knowledge text finally generated, the computer device can lock the paragraph text while further combining the linked list structure in the document structure tree, and expand the contextual content of the paragraph according to the preset recall length, by searching for nodes adjacent to the current leaf node in the linked list structure and retrieving the corresponding paragraph text in turn, forming a contextually coherent content set as the auxiliary knowledge text, thereby avoiding the omission of key content.

[0077] It is understandable that the role of context expansion here is to enhance semantic integrity, especially in some cases, a single paragraph text may rely on the context before and after to accurately express its meaning. For example, the technical concept in a paragraph text may be defined in the previous paragraph text, or its application scenario is detailed in the next paragraph text. At this time, through the association of the linked list structure, the computer device can accurately trace or look forward to the relevant content, ensuring that the recalled auxiliary knowledge text can provide more comprehensive and coherent information, thereby improving the practicality and accuracy of the retrieval results, and providing technical support for multi-level association retrieval in complex knowledge scenarios.

[0078] In the above embodiment, when performing knowledge text retrieval, the search statement and its corresponding target knowledge document can be obtained first, and the document structure tree of the target knowledge document can be determined. The document structure tree here is a tree structure constructed using the directory structure of the target knowledge document, and records the index structure between each index node in its components and the target knowledge document, wherein the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure. Therefore, when searching, the present application can dynamically intercept the context content corresponding to the search statement in the target knowledge document based on the association relationship between the index structure of the tree structure and the linked list structure, thereby improving the accuracy and quality of the retrieval. For example, keyword indexing technology and vector indexing technology can be used to index the index nodes corresponding to the search statements from the document structure tree, so as to improve the semantic relevance of the search results through two-dimensional indexing technology, thereby improving the search accuracy; among them, if the index node is a branch node, the chapter title and chapter length corresponding to the branch node in the target knowledge document can be determined, so as to recall the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text, so as to avoid returning irrelevant content of other chapters; if the index node is a leaf node, the paragraph text corresponding to the leaf node in the target document can be determined, and the paragraph text can be contextually expanded according to the linked list structure and the preset recall length, and auxiliary knowledge text can be generated according to the expansion result to avoid missing key content.

[0079] In one embodiment, the process of constructing the document structure tree in step S110 may include:

[0080] S111: Obtain a target knowledge document, and perform structural analysis on the target knowledge document to obtain a directory structure; the directory structure includes a document title, a chapter title, and a paragraph body.

[0081] S112: Based on the directory structure, a document structure tree of the target knowledge document is constructed with the document title as the root node, the chapter title as the branch node, and the paragraph body as the leaf node.

[0082] S113: For each node in the document structure tree, a keyword index and a vector index of the node are constructed based on the target knowledge document to form an index node.

[0083] S114: Establish pointers between leaf nodes in the document structure tree after indexing according to the logical order of the text in the target knowledge document to connect them and form a linked list structure.

[0084] In this embodiment, after the computer device obtains the target knowledge document, it can first perform structural analysis on the target knowledge document to obtain a directory structure. The directory structure is a multi-level structure obtained by combining the document title, chapter title and paragraph text. Therefore, the computer device can construct a document structure tree of the target knowledge document based on the directory structure, with the document title as the root node, the chapter title as the branch node, and the paragraph text as the leaf node; then the computer device can construct the corresponding keyword index and vector index in each node based on the target knowledge document, and then form an index node; in addition, the computer device can also establish pointers between each leaf node in the document structure tree after the index is constructed according to the logical order of the text in the target knowledge document to connect them, so as to form a linked list structure.

[0085] It should be noted that the document title of the present application is the file identifier of the target knowledge document, which is used to distinguish different target knowledge documents. The target knowledge document consists of multiple chapters, and each chapter corresponds to a chapter title. For any chapter, it consists of multiple paragraphs, and each paragraph can include at least one paragraph body. The paragraph body here has one and only one period, which is a sentence ending with the period.

[0086] It is understandable that when the computer device is parsing the target knowledge document, the directory structure of the target knowledge document can be clearly divided into multiple levels, including the document title, each chapter title and the paragraph text under each chapter title. This hierarchical structure can organize the document content into a tree system to form a document structure tree. Figure 3 As shown, Figure 3 A schematic diagram of the structure of a document structure tree provided in an embodiment of the present application; Figure 3 In the document structure tree of the example, the document title is the root node, representing the basic framework of the target knowledge document; each chapter title is a branch node, representing each chapter of the target knowledge document, guiding the content classification in the document; and each paragraph body is a leaf node, carrying specific knowledge points, used to provide detailed descriptions and explanations. Therefore, based on this hierarchical document structure tree, the present application can accurately locate the knowledge points in the target knowledge document during the retrieval process, thereby improving the retrieval efficiency and accuracy.

[0087] Specifically, after constructing the document structure tree, the computer device can construct a corresponding keyword index and vector index in each node of the document structure tree to form an index vector. Among them, the keyword index is mainly created by analyzing the text content corresponding to each node in the target knowledge document and extracting its keywords, while the vector index is mainly created by converting the text content corresponding to each node in the target knowledge document into a high-dimensional vector. Therefore, in the process of node indexing, the computer device can quickly index the index node corresponding to the search statement through the keyword index and the vector index, and then use the index node to quickly locate the text content in the target knowledge document, thereby improving the search efficiency and accuracy.

[0088] In addition, in order to enhance the relevance between each leaf node in the document structure tree, the computer device can establish pointers between each leaf node in the document structure tree according to the logical order of the text in the target knowledge document to connect them and form a linked list structure. Therefore, in this linked list structure, each leaf node not only contains the paragraph text, but also forms a close connection with the previous and next related paragraph texts. In one of the leaf nodes, it can be located to the level where it is in the target knowledge document, such as the chapter where it is located, and the paragraph text in the chapter, through the tree index of the document structure tree, and then through the pointer in the linked list structure, it can not only point to the leaf node corresponding to the previous paragraph text, but also point to the leaf node corresponding to the next paragraph text, so as to ensure the continuity and logic of the content.

[0089] In one embodiment, the process of constructing the keyword index and the vector index of the node based on the target knowledge document to form the index node in step S113 may include:

[0090] S1131: Determine the text content corresponding to the node from the target knowledge document, extract index keywords of the text content, and generate an index vector of the text content.

[0091] S1132: Establish index structures between index keywords, index vectors and text content respectively, and add the index structures to the node to form an index node.

[0092] In this embodiment, when the computer device constructs the index of each node in the document structure tree, for each node, the computer device can first determine the text content corresponding to the node from the target knowledge document, extract the index keywords of the text content, and generate an index vector of the text content. Then, an index structure between the index keywords, index vectors and text content can be established respectively, and the index structure can be added to the node to form an index node.

[0093] It can be understood that for each node, the computer device can extract the text content corresponding to the node from the target knowledge document. When the node type is different, the corresponding text content length and complexity are also different. For example, the text content corresponding to the leaf node is the paragraph text, and the text content corresponding to the branch node is the chapter text, that is, the sum of the paragraph texts corresponding to all leaf nodes extending from the branch node.

[0094] Specifically, index keywords refer to words or phrases that are highly representative and semantically relevant extracted from text content. These keywords can reflect the theme and core idea of ​​the text content. Here, the computer device can use natural language processing technology, such as word frequency analysis, TF-IDF (term frequency-inverse document frequency) and other methods to extract keywords that can effectively represent the text content from the text content. The index vector refers to the form of converting text content into a high-dimensional vector representation. Usually, word embedding technology, such as Word2Vec, GloVe, or deep learning methods, such as BERT, can be used to convert text content into corresponding vector representations to capture semantic information in the text content. After obtaining the index keywords and index vectors, the computer device can also associate the index keywords, index vectors and the text content corresponding to the node, that is, organize these information through data structures, such as hash tables, inverted indexes, B-trees and other methods, generate index structures and store them in nodes to form index nodes, so that in the subsequent retrieval process, the text content corresponding to the retrieval statement can be quickly found through the index structures of each index node, thereby improving retrieval efficiency and accuracy.

[0095] In one embodiment, the process of establishing pointers between leaf nodes in the document structure tree after indexing to form a linked list structure in step S114 according to the logical order of the text in the target knowledge document may include:

[0096] S1141: Sort each leaf node according to the logical order of the text in the target knowledge document to obtain a sorting result.

[0097] S1142: Determine in turn whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document.

[0098] S1143: If yes, a pointer is established between two adjacent leaf nodes to connect them.

[0099] S1144: If not, no connection is made between two adjacent leaf nodes.

[0100] In this embodiment, when constructing a linked list structure, the computer device can first sort each leaf node according to the logical order of the text in the target knowledge document to obtain a sorting result, and then judge in turn whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document. If they belong to the same paragraph, the computer device can establish a pointer between the two adjacent leaf nodes to connect them; if they do not belong to the same paragraph, the computer device can not connect the two adjacent leaf nodes. Therefore, through this linked list structure, the present application can ensure the continuity and logic between each leaf node in the document structure tree, and then provide a contextual expansion basis for the paragraph text.

[0101] It can be understood that the sorting process of leaf nodes is based on the natural order of each paragraph sentence in the target knowledge document, which ensures that the arrangement order of each leaf node in the linked list structure is consistent with the arrangement of the original content in the target knowledge document, and is used to reflect the logical structure and hierarchical relationship of the target knowledge document.

[0102] Specifically, after obtaining the sorting result, the computer device can check the sorted leaf nodes in turn and determine whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph. Here, the comparison can be performed through paragraph identifiers or other text features. If the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph, the computer device can establish a pointer between them to connect them, ensure that the two leaf nodes are directly connected, and then form a continuous node chain in the linked list structure to maintain the logical coherence of the text content. If the paragraph texts corresponding to two adjacent leaf nodes do not belong to the same paragraph, the computer device can not connect between the two leaf nodes to avoid mixing the contents of different paragraphs.

[0103] Of course, in a target knowledge document with concise content, the present application may also simply not establish a pointer between two adjacent leaf nodes representing different chapters to indicate the boundary between the two chapters.

[0104] In one embodiment, the process of using the keyword indexing technology and the vector indexing technology to index the document structure tree to obtain the index node corresponding to the search statement in step S120 may include:

[0105] S121: extracting search keywords of the search statement and generating a search vector of the search statement.

[0106] S122: Using the BM25 algorithm to calculate the search keywords in the document structure tree to obtain a first index result, and using the HNSW algorithm to calculate the search vector to obtain a second index result.

[0107] S123: Determine the index node corresponding to the search statement according to the first index result and the second index result.

[0108] In this embodiment, when the computer device indexes the search statement, it can first extract the search keywords of the search statement and generate the search vector of the search statement. The process of obtaining the search keywords and the search vector here is consistent with the process of obtaining the index keywords and the index vector, and will not be repeated here. After obtaining the search keywords and the search vector, the computer device can use the BM25 algorithm to calculate the search keywords in the document structure tree to obtain the first index result, and use the HNSW algorithm to calculate the search vector to obtain the second index result, and then determine the index node corresponding to the search statement according to the first index result and the second index result.

[0109] Among them, the BM25 algorithm is a text retrieval algorithm based on a probability model. It is an extension and improvement of the F-IDF (term frequency-inverse document frequency) model. It is used to calculate the relevance score between the retrieval keyword and each text content, and sort the text content according to the score to output the most relevant text content. The HNSW algorithm refers to an efficient approximate nearest neighbor search algorithm, which is widely used in vector space retrieval, especially in large-scale, high-dimensional data sets. It provides a good balance between speed and accuracy.

[0110] Specifically, Figure 4 As shown, Figure 4 A logical diagram of a node indexing process provided in an embodiment of the present application; Figure 4 In the above, the computer device can perform index calculations on the index structure stored in each index node based on the search keyword and the search vector. Therefore, the first index result calculated based on the BM25 algorithm includes the first correlation score between each index node and the search keyword, while the second index result calculated based on the HNSW algorithm includes the second correlation score between each index node and the search vector; after calculating the first index result and the second index result, the computer device can determine the first adjustment coefficient of the search keyword and the second adjustment coefficient of the search vector respectively, and then use the first adjustment coefficient and the second adjustment coefficient to perform weighted summation on the correlation scores of each index node to obtain the matching degree of each index node. Then, the computer device can sort the index nodes according to the matching degrees to obtain the sorting results, and use the index node with the highest matching degree in the sorting results as the index node corresponding to the search statement.

[0111] Of course, in addition to the BM25 algorithm and the HNSW algorithm, other algorithms that can be used to calculate the relevance scores of search keywords or search vectors, such as the cosine similarity algorithm, can be used as technical means for the present application to determine the index node corresponding to the search statement, and are not limited here.

[0112] In one embodiment, in step S130, the target knowledge document and the chapter summaries corresponding to the chapter titles in the target knowledge document are pre-stored in the database; the chapter summaries are extracted from the chapter texts under the corresponding chapter titles. The process of recalling the chapter texts or chapter summaries under the chapter titles from the database based on the length of the chapters as auxiliary knowledge texts may include:

[0113] S131: Determine whether the chapter length exceeds the preset output length.

[0114] S132: If yes, then the chapter summary corresponding to the chapter title is retrieved from the database as auxiliary knowledge text.

[0115] S133: If not, the chapter text corresponding to the chapter title is recalled from the database as auxiliary knowledge text.

[0116] In this embodiment, since the target knowledge document and the chapter summaries corresponding to the chapter titles in the target knowledge document are pre-stored in the database of this application, and the chapter summaries are extracted based on the chapter texts under the corresponding chapter titles. Therefore, the subject logic of the chapter summary is consistent with the chapter text, but the number of words in the chapter summary is much lower than the number of words in the chapter text. Based on this, when the index node is a branch node, the computing device can determine whether the length of the chapter corresponding to the branch node exceeds the preset output length; if so, the chapter summary corresponding to the chapter title is recalled from the database as an auxiliary knowledge text; if not, the chapter text corresponding to the chapter title is recalled from the database as an auxiliary knowledge text.

[0117] It is understandable that the preset output length is the number of characters or paragraphs set according to the actual needs of the large prediction model. It is used to control the degree of simplicity of the returned text to avoid returning too much redundant or irrelevant content, and to ensure the efficiency and pertinence of the search results. Therefore, when the length of a chapter does not exceed the preset output length, it means that the chapter text corresponding to the branch node is relatively short, so the computer device can directly recall the complete chapter text from the database; on the contrary, when the length of a chapter exceeds the preset output length, it means that the chapter text corresponding to the branch node is relatively long, so the computer device can recall the chapter summary of the chapter text from the database, which can effectively reduce information overload while maintaining the simplicity and efficiency of the auxiliary knowledge text.

[0118] Indicatively, Figure 5 As shown, Figure 5 A logical schematic diagram of a branch node recall process provided in an embodiment of the present application; wherein, Figure 5 (a) represents the process of recalling the chapter text. Since the chapter text corresponding to the branch node is obtained by combining the paragraph texts of each leaf node under the branch node, the computer device can first hit each leaf node under the branch node, and recall the paragraph texts in each leaf node in turn to obtain the chapter text; Figure 5 (b) represents the recall process of the chapter summary. Since the chapter summaries of the main text of each chapter are pre-stored in the database, and these chapters are obtained by extracting and summarizing the key points using LLM, the computer device can directly hit the branch node and directly recall the chapter summary corresponding to the branch node.

[0119] In one embodiment, the process of performing context expansion on the paragraph body according to the linked list structure and the preset recall length in step S140 and generating auxiliary knowledge text according to the expansion result may include:

[0120] S141: Determine the branch node to which the leaf node belongs and the chapter title corresponding to the branch node, and determine whether the length of the chapter corresponding to the chapter title exceeds the preset output length.

[0121] S142: If so, recall the leaf nodes and the paragraph texts corresponding to the leaf nodes above and below the leaf nodes from the database in sequence according to the pointer sequence in the linked list structure until the recalled paragraph texts reach the preset output length, and use the recalled paragraph texts as auxiliary knowledge texts.

[0122] S143: If not, the chapter text under the chapter title is recalled from the database as auxiliary knowledge text.

[0123] In this embodiment, when it is determined that the index node is a leaf node, the computer device can first determine the branch node to which the leaf node belongs and the chapter title corresponding to the branch node, and determine whether the length of the chapter corresponding to the chapter title exceeds the preset output length; if so, the computer device can recall the leaf node and the paragraph text corresponding to the leaf node above and below the leaf node from the database in sequence according to the pointer order in the linked list structure, until the recalled paragraph text reaches the preset output length, and use the recalled paragraph text as the auxiliary knowledge text; if not, the computer device can directly recall the chapter text under the chapter title from the database as the auxiliary knowledge text. In this way, the present application can avoid information overload while ensuring the integrity and coherence of the auxiliary knowledge text, thereby affecting the performance of the large language model.

[0124] It is understandable that in order to ensure that the auxiliary knowledge text is both concise and complete, when the length of a chapter exceeds the preset output length, the computer device can analyze the contextual association between leaf nodes through a linked list structure, and then recall the paragraph text corresponding to the current leaf node and the leaf nodes adjacent to it from the database in sequence according to the pointer sequence in the linked list, so that the adjacent paragraph texts can be recalled according to the sequence between the leaf nodes, and these paragraph texts can be merged together in a logical order until the recalled paragraph text reaches or approaches the preset output length, so that the recalled paragraph text is logically coherent in sequence, and the context is fully preserved, avoiding information fragmentation or incoherence. When the length of a chapter does not exceed the preset output length, it means that the corresponding chapter text is relatively short and will not cause information overload or redundancy. At this time, the computer device will directly recall the chapter text related to the chapter title from the database as the auxiliary knowledge text.

[0125] Indicatively, Figure 6 As shown, Figure 6 A logical schematic diagram of a leaf node recall process provided in an embodiment of the present application; wherein, Figure 6 (a) represents the process of recalling the main body of a chapter. After the computer device hits a leaf node representing the main body of a paragraph, if the length of the chapter corresponding to the branch node representing the chapter title to which the leaf node belongs does not exceed the preset output length, the computer device can recall the main body of the paragraphs in each leaf node under the branch node in turn to obtain the main body of the chapter. Figure 6 (b) represents the process of expanding the paragraph text. After the computer device hits the leaf node representing the paragraph text, if the length of the chapter corresponding to the branch node representing the chapter title to which the leaf node belongs exceeds the preset output length, the computer device can recall the paragraph text corresponding to the leaf node according to the pointer in the linked list structure, and expand the adjacent leaf nodes in the forward and backward directions synchronously with the leaf node as the center, and recall and merge the paragraph text corresponding to the expanded leaf nodes at the same time, until the merged paragraph text reaches or approaches the preset output length, and uses it as the auxiliary knowledge text.

[0126] The knowledge text retrieval device provided in an embodiment of the present application is described below. The knowledge text retrieval device described below and the knowledge text retrieval method described above can be referenced to each other.

[0127] In one embodiment, Figure 7 As shown, Figure 7 A schematic diagram of the structure of a knowledge text retrieval device provided in an embodiment of the present application; the present application also provides a knowledge text retrieval device, including an information acquisition module 210, a node indexing module 220, a first text generation module 230 and a second text generation module 240, specifically including the following:

[0128] The information acquisition module 210 is used to obtain the search statement and the target knowledge document corresponding to the search statement, and determine the document structure tree of the target knowledge document; the document structure tree is constructed according to the directory structure of the target knowledge document, including multiple index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure.

[0129] The node indexing module 220 is used to obtain the index nodes corresponding to the search statement from the document structure tree by using the keyword indexing technology and the vector indexing technology.

[0130] The first text generation module 230 is used to determine the chapter title and chapter length corresponding to the branch node in the target knowledge document when the index node is determined to be a branch node, so as to recall the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text.

[0131] The second text generation module 240 is used to determine the paragraph text corresponding to the leaf node in the target document when the index node is determined to be a leaf node, and to contextually expand the paragraph text according to the linked list structure and the preset recall length, and to generate auxiliary knowledge text according to the expansion result.

[0132] In the above embodiment, when performing knowledge text retrieval, the search statement and its corresponding target knowledge document can be obtained first, and the document structure tree of the target knowledge document can be determined. The document structure tree here is a tree structure constructed using the directory structure of the target knowledge document, and records the index structure between each index node in its components and the target knowledge document, wherein the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure. Therefore, when searching, the present application can dynamically intercept the context content corresponding to the search statement in the target knowledge document based on the association relationship between the index structure of the tree structure and the linked list structure, thereby improving the accuracy and quality of the retrieval. For example, keyword indexing technology and vector indexing technology can be used to index the index nodes corresponding to the search statements from the document structure tree, so as to improve the semantic relevance of the search results through two-dimensional indexing technology, thereby improving the search accuracy; among them, if the index node is a branch node, the chapter title and chapter length corresponding to the branch node in the target knowledge document can be determined, so as to recall the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text, so as to avoid returning irrelevant content of other chapters; if the index node is a leaf node, the paragraph text corresponding to the leaf node in the target document can be determined, and the paragraph text can be contextually expanded according to the linked list structure and the preset recall length, and auxiliary knowledge text can be generated according to the expansion result to avoid missing key content.

[0133] In one embodiment, the information acquisition module 210 may include:

[0134] The structure analysis submodule is used to obtain the target knowledge document and perform structure analysis on the target knowledge document to obtain a directory structure; the directory structure includes a document title, a chapter title, and a paragraph body.

[0135] The node construction submodule is used to construct a document structure tree of the target knowledge document based on the directory structure, with the document title as the root node, the chapter title as the branch node, and the paragraph body as the leaf node.

[0136] The index building submodule is used to build a keyword index and a vector index of each node in the document structure tree based on the target knowledge document to form an index node.

[0137] The linked list generation submodule is used to establish pointers between the leaf nodes in the document structure tree after indexing according to the logical order of the text in the target knowledge document to connect them and form a linked list structure.

[0138] In one embodiment, the index building submodule may include:

[0139] The index information generating unit is used to determine the text content corresponding to the node from the target knowledge document, extract the index keywords of the text content, and generate the index vector of the text content.

[0140] The index structure establishment unit is used to establish the index structure between the index keyword, the index vector and the text content respectively, and add the index structure to the node to form an index node.

[0141] In one embodiment, the linked list generation submodule may include:

[0142] The node sorting unit is used to sort each leaf node according to the logical order of the text in the target knowledge document to obtain a sorting result.

[0143] The paragraph determination unit is used to determine in sequence whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document.

[0144] The first establishing unit is used to establish a pointer between two adjacent leaf nodes to connect them when the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document.

[0145] The second establishing unit is used for not connecting two adjacent leaf nodes when the paragraph texts corresponding to two adjacent leaf nodes do not belong to the same paragraph in the target knowledge document.

[0146] In one embodiment, the node index module 220 may include:

[0147] The search information generation submodule is used to extract search keywords of the search statement and generate search vectors of the search statement.

[0148] The information indexing submodule is used to calculate the search keywords in the document structure tree using the BM25 algorithm to obtain the first index result, and to calculate the search vector using the HNSW algorithm to obtain the second index result.

[0149] The node determination submodule is used to determine the index node corresponding to the search statement according to the first index result and the second index result.

[0150] In one embodiment, the first text generation module 230 pre-stores the target knowledge document and the chapter summaries corresponding to the chapter titles in the target knowledge document in the database; the chapter summaries are obtained by extracting the chapter texts under the corresponding chapter titles. The first text generation module 230 may also include:

[0151] The first length judgment submodule is used to judge whether the chapter length exceeds the preset output length.

[0152] The first recall submodule is used to recall the chapter text corresponding to the chapter title from the database as auxiliary knowledge text when the chapter length exceeds the preset output length.

[0153] The second recall submodule is used to recall the chapter summary corresponding to the chapter title from the database as auxiliary knowledge text when the chapter length does not exceed the preset output length.

[0154] In one embodiment, the second text generation module 240 may include:

[0155] The second length judgment submodule is used to determine the branch node to which the leaf node belongs and the chapter title corresponding to the branch node, and to judge whether the length of the chapter corresponding to the chapter title exceeds the preset output length.

[0156] The third recall submodule is used to recall the leaf nodes and the paragraph texts corresponding to the leaf nodes above and below the leaf nodes from the database in sequence according to the pointer order in the linked list structure when the chapter length corresponding to the chapter title exceeds the preset output length, until the recalled paragraph text reaches the preset output length, and use the recalled paragraph text as the auxiliary knowledge text.

[0157] The fourth recall submodule is used to recall the chapter text under the chapter title from the database as auxiliary knowledge text when the chapter length corresponding to the chapter title does not exceed the preset output length.

[0158] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the knowledge text retrieval method as described in any of the above embodiments.

[0159] In one embodiment, the present application also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the knowledge text retrieval method as described in any one of the above embodiments.

[0160] Indicatively, Figure 8 As shown, Figure 8 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 8 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301 for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the knowledge text retrieval method of any of the above embodiments.

[0161] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0162] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0164] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0165] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A knowledge text retrieval method, characterized in that: The method comprises: Obtain a search statement and a target knowledge document corresponding to the search statement, and determine a document structure tree of the target knowledge document; the document structure tree is constructed according to the directory structure of the target knowledge document, and includes a plurality of index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure; Using keyword indexing technology and vector indexing technology to index the document structure tree to obtain an index node corresponding to the search statement; When it is determined that the index node is a branch node, determining the chapter title and chapter length corresponding to the branch node in the target knowledge document, and recalling the chapter text or chapter summary under the chapter title from the database based on the chapter length as auxiliary knowledge text; When it is determined that the index node is a leaf node, the paragraph text corresponding to the leaf node in the target document is determined, and the paragraph text is contextually expanded according to the linked list structure and the preset recall length, and auxiliary knowledge text is generated according to the expansion result.

2. The knowledge text retrieval method according to claim 1, characterized in that: The construction process of the document structure tree includes: Acquire a target knowledge document, and perform structural analysis on the target knowledge document to obtain a directory structure; the directory structure includes a document title, a chapter title, and a paragraph body; Based on the directory structure, a document structure tree of the target knowledge document is constructed with the document title as the root node, the chapter title as the branch node, and the paragraph body as the leaf node; For each node in the document structure tree, construct a keyword index and a vector index of the node based on the target knowledge document to form an index node; According to the logical order of the text in the target knowledge document, pointers are established between the leaf nodes in the document structure tree after the index is constructed to connect them and form a linked list structure.

3. The knowledge text retrieval method according to claim 2, characterized in that: The step of constructing a keyword index and a vector index of the node based on the target knowledge document to form an index node includes: Determine the text content corresponding to the node from the target knowledge document, extract index keywords of the text content, and generate an index vector of the text content; An index structure is established between the index keyword, the index vector and the text content respectively, and the index structure is added to the node to form an index node.

4. The knowledge text retrieval method according to claim 2, characterized in that: The step of establishing pointers between leaf nodes in the document structure tree after indexing according to the text logical order in the target knowledge document to connect them and form a linked list structure includes: Sort each leaf node according to the logical order of the text in the target knowledge document to obtain a sorting result; Determine in sequence whether the paragraph texts corresponding to two adjacent leaf nodes belong to the same paragraph in the target knowledge document; If yes, a pointer is established between the two adjacent leaf nodes to connect them; If not, no connection is made between the two adjacent leaf nodes.

5. The knowledge text retrieval method according to claim 1, characterized in that: The method of using keyword indexing technology and vector indexing technology to index the document structure tree to obtain an index node corresponding to the search statement includes: Extracting search keywords of the search statement and generating a search vector of the search statement; The BM25 algorithm is used to calculate the search keyword in the document structure tree to obtain a first index result, and the HNSW algorithm is used to calculate the search vector to obtain a second index result; An index node corresponding to the search statement is determined according to the first index result and the second index result.

6. The knowledge text retrieval method according to claim 1, characterized in that: The database pre-stores a target knowledge document and a chapter summary corresponding to each chapter title in the target knowledge document; the chapter summary is obtained by extracting the chapter text under the corresponding chapter title; The chapter text or chapter summary under the chapter title is recalled from the database based on the length of the chapter as auxiliary knowledge text, including: Determine whether the length of the chapter exceeds a preset output length; If so, the chapter summary corresponding to the chapter title is recalled from the database as auxiliary knowledge text; If not, the chapter text corresponding to the chapter title is recalled from the database as auxiliary knowledge text.

7. The knowledge text retrieval method according to claim 1, characterized in that: The process of performing context expansion on the paragraph text according to the linked list structure and the preset recall length, and generating auxiliary knowledge text according to the expansion result, includes: Determine the branch node to which the leaf node belongs and the chapter title corresponding to the branch node, and judge whether the length of the chapter corresponding to the chapter title exceeds the preset output length; If so, recall the leaf node and the paragraph texts corresponding to the leaf nodes above and below the leaf node from the database in sequence according to the pointer sequence in the linked list structure, until the recalled paragraph texts reach the preset output length, and use the recalled paragraph texts as auxiliary knowledge texts; If not, the chapter text under the chapter title is recalled from the database as auxiliary knowledge text.

8. A knowledge text retrieval device, characterized in that: include: An information acquisition module, used to acquire a search statement and a target knowledge document corresponding to the search statement, and determine a document structure tree of the target knowledge document; The document structure tree is constructed according to the directory structure of the target knowledge document, and includes a plurality of index nodes; the index nodes include branch nodes and leaf nodes, and each leaf node is connected by a pointer to form a linked list structure; A node indexing module, used to obtain an index node corresponding to the search statement from the document structure tree by using a keyword indexing technique and a vector indexing technique; A first text generation module is used for determining, when the index node is determined to be a branch node, a chapter title and a chapter length corresponding to the branch node in the target knowledge document, so as to recall the chapter text or chapter summary under the chapter title from a database based on the chapter length as an auxiliary knowledge text; The second text generation module is used to determine the paragraph text corresponding to the leaf node in the target document when it is determined that the index node is a leaf node, and to contextually expand the paragraph text according to the linked list structure and the preset recall length, and to generate auxiliary knowledge text according to the expansion result.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the knowledge text retrieval method as described in any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the knowledge text retrieval method according to any one of claims 1 to 7 are performed.

Images