Knowledge base construction method for large language model, retrieval method and related device

Through the knowledge base construction method for large language models, chapter blocks are divided according to the knowledge document type and metadata is stored, which solves the knowledge integrity problem in knowledge base construction, and improves the answer accuracy and semantic integrity of large language models.

CN118916441APending Publication Date: 2024-11-08BEIJING SUPERMAP SOFTWARE CO LTD

Patent Information

Application Number
CN202410905546.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When constructing existing large language model knowledge bases, the existing large language model knowledge base construction methods often lead to the destruction of the completeness of knowledge, resulting in the lack of semantics of the search results, which in turn affects the accuracy of the model's answers.

Method used

Using a knowledge base construction method for large language models, the knowledge document is divided into chapter blocks by setting storage rules according to the type of knowledge document, including chapter titles and text, and the spliced ​​text is stored as metadata in the vector database to ensure the integrity of knowledge. , and add additional information when searching to improve semantic accuracy.

Benefits of technology

It effectively improves the completeness of knowledge in the knowledge base and the accuracy of search results, ensuring the completeness and relevance of large language model answers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916441A_ABST
    Figure CN118916441A_ABST
Patent Text Reader

Abstract

The invention discloses a big language model-oriented knowledge base construction method, retrieval method and related devices.The knowledge base construction method comprises the following steps: firstly, obtaining a plurality of knowledge documents; then, based on the storage rules corresponding to the types of the multiple knowledge documents, the multiple knowledge documents are stored in a vector database, and a knowledge base is obtained; wherein the storage rule at least comprises the following steps: if the type of the knowledge document is a first type containing chapter information, dividing the knowledge document into a plurality of chapter blocks based on a chapter title, and the chapter blocks comprise the chapter title and a chapter text; in one chapter block, a spliced text is obtained based on a chapter title and a plurality of single-sentence texts obtained by dividing a chapter text according to sentences; and storing the spliced text as knowledge, storing the chapter text as supplementary information as metadata of all knowledge in the chapter blocks to which the chapter text belongs, and storing the chapter blocks into a vector database one by one. Therefore, the integrity of the knowledge document cannot be damaged by dividing the knowledge document according to sentences, and the integrity of the knowledge stored in the knowledge base is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a knowledge base construction method, a retrieval method and related devices for large language models. Background Art

[0002] With the development of artificial intelligence, the application of large language models has become more and more extensive. When a large language model needs to be applied to a specific field, it is usually enhanced through knowledge base retrieval, thereby solving the problem of insufficient knowledge reserves of large language models in general fields when facing vertical fields at a low cost.

[0003] At present, in the process of knowledge base construction, the text in the knowledge document is usually cut into single-sentence texts by sentence or line order, and the single-sentence texts are stored as knowledge in the vector database, and the knowledge base construction is completed. However, such a knowledge base construction method often leads to the destruction of the integrity of knowledge. When searching based on the knowledge base constructed in this way, the results obtained will have semantic omissions, which will lead to incomplete and inaccurate answers from the large language model after the search results are input into the large language model.

[0004] Therefore, how to improve the integrity of knowledge in the knowledge base becomes a problem that needs to be solved. Summary of the invention

[0005] Based on the above problems, the present application provides a knowledge base construction method, a retrieval method and related devices for a large language model, which can improve the integrity of the knowledge in the constructed knowledge base.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, the present application provides a method for constructing a knowledge base for a large language model, the method comprising:

[0008] Obtain multiple knowledge documents;

[0009] Based on the storage rules corresponding to the types of multiple knowledge documents, the multiple knowledge documents are stored in a vector database to obtain a knowledge base; the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a spliced ​​text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the spliced ​​text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

[0010] Optionally, the concatenated text is obtained based on the chapter titles and the multiple single-sentence texts obtained by dividing the chapter text into sentences, including:

[0011] Based on the hierarchy of the chapter title, concatenate the chapter title with all its parent chapter titles to obtain a joint title;

[0012] Divide the chapter text into sentences to obtain multiple single-sentence texts;

[0013] The joint title and the single sentence text are concatenated to obtain a concatenated text.

[0014] Optionally, before dividing the knowledge document into a plurality of chapter blocks based on the chapter titles, the method further includes:

[0015] Based on the hierarchy of the chapter titles in the knowledge document, the chapter titles are set to a title style corresponding to the hierarchy; and the chapter texts in the knowledge document are set to a preset text style.

[0016] Optionally, after dividing the knowledge document into a plurality of chapter blocks based on the chapter titles, the method further comprises:

[0017] If there is an untitled chapter block with an empty chapter title, the chapter text is divided into sentences to obtain multiple single-sentence texts; the single-sentence texts are stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the chapter block, and the untitled chapter blocks are stored one by one in the vector database.

[0018] Optionally, the storage rule further includes:

[0019] If the type of the knowledge document is the second type in the form of question-answer pairs, the questions in the question-answer pairs are stored as knowledge, and the answers are stored as supplementary information as metadata of the knowledge, and each question-answer pair is stored in the vector database.

[0020] Optionally, the storage rule further includes:

[0021] If the type of the knowledge document is the third type that does not contain chapter information and is not in the form of a question-answer pair, the knowledge document is divided by sentence to obtain multiple texts; the texts are stored as knowledge, and each text is stored in a vector database.

[0022] In a second aspect, the present application provides a knowledge base retrieval method for a large language model, the method being used to search in a knowledge base constructed by the knowledge base construction method for a large language model described in any implementation of the first aspect, the method comprising:

[0023] Receive questions to be retrieved;

[0024] Based on the question to be searched, calling the search function of the vector database, searching from the knowledge base to obtain the first k pieces of knowledge with the highest similarity to the question to be searched; wherein k is a positive integer;

[0025] The supplementary information existing in the metadata of the retrieved knowledge is added to the reference information list; the combination of the reference information list and the question to be retrieved is the prompt information of the large language model.

[0026] Optionally, after calling the search function of the vector database based on the question to be searched and retrieving the first k pieces of knowledge with the highest similarity to the question to be searched from the knowledge base, the method further comprises:

[0027] If there is no supplementary information in the metadata of the retrieved current knowledge, multiple pieces of associated knowledge whose entry time is later than that of the current knowledge are retrieved from the knowledge base in sequence based on the order of storage; the total length of the current knowledge and the associated knowledge is: the maximum value that can be achieved without being greater than the preset maximum length value of a single piece of reference information;

[0028] A text block obtained by concatenating the current knowledge and the associated knowledge is added to a reference information list.

[0029] In a third aspect, the present application provides a knowledge base construction device for a large language model, the device comprising: an acquisition module and a storage module;

[0030] The acquisition module is used to acquire multiple knowledge documents;

[0031] The storage module is used to store the multiple knowledge documents in a vector database based on the storage rules corresponding to the types of the multiple knowledge documents to obtain a knowledge base; the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a spliced ​​text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the spliced ​​text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

[0032] In a fourth aspect, the present application provides a knowledge base retrieval device for a large language model, the device being used to search in a knowledge base constructed by the knowledge base construction method for a large language model described in any one of the embodiments of the first aspect, the device comprising: a receiving module, a retrieval module, and an adding module;

[0033] The receiving module is used to receive the question to be retrieved;

[0034] The retrieval module is used to call the retrieval function of the vector database based on the question to be retrieved, and retrieve the first k pieces of knowledge with the highest similarity to the question to be retrieved from the knowledge base; wherein k is a positive integer;

[0035] The adding module is used to add the retrieved knowledge and the supplementary information in the metadata of the knowledge to the reference information list; the combination of the reference information list and the question to be retrieved is the prompt information of the large language model.

[0036] Compared with the prior art, this application has the following beneficial effects:

[0037] The embodiment of the present application provides a knowledge base construction method for a large language model, in which, first, a plurality of knowledge documents are obtained; then, based on the storage rules corresponding to the types of the plurality of knowledge documents, the plurality of knowledge documents are stored in a vector database to obtain a knowledge base; wherein the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into a plurality of chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a plurality of single-sentence texts obtained by dividing the chapter text by sentence are obtained to obtain a concatenated text; the concatenated text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in a vector database. Thus, different storage rules are set for different types of documents, especially for knowledge documents containing chapter information, through the storage rules provided by the embodiment of the present application, the entire text of the chapter block is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, no matter which knowledge is retrieved, the entire text of the corresponding chapter block can be obtained, and the integrity of the sentence or paragraph will not be destroyed due to the division of the knowledge document by sentence or line, thereby ensuring the integrity of the knowledge stored in the knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a method for constructing a knowledge base for a large language model provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a knowledge document containing chapter information provided in an embodiment of the present application;

[0041] Figure 3A flow chart of another method for constructing a knowledge base for a large language model provided in an embodiment of the present application;

[0042] Figure 4 A flow chart of a knowledge base retrieval method for a large language model provided in an embodiment of the present application;

[0043] Figure 5 A flow chart of another knowledge base retrieval method for a large language model provided in an embodiment of the present application;

[0044] Figure 6 A reference information provided for an embodiment of the present application includes a comparison chart of answers generated by a large language model for a document in the form of a question-answer pair;

[0045] Figure 7 A reference information provided for an embodiment of the present application includes a large language model generated answer comparison chart of a chapter information document;

[0046] Figure 8 A schematic diagram of a knowledge base construction device for a large language model provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of a knowledge base retrieval device for a large language model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The knowledge base construction method, retrieval method and related devices for a large language model provided in the present application can be used in the field of large language models. The above is only an example and does not limit the application field of the knowledge base construction method, retrieval method and related devices for a large language model provided in the present application.

[0049] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0050] In the embodiments of the present application, the words "as an example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "as an example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the words "as an example" or "for example" is intended to present the relevant concepts in a specific way.

[0051] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] See also Figure 1 , which is a flow chart of a method for constructing a knowledge base for a large language model provided in an embodiment of the present application, the method comprising:

[0054] S101: Acquire multiple knowledge documents.

[0055] As an example, a knowledge document may be of the first type that contains chapter information and exists in the form of a long text; it may also be of the second type that is in the form of a question-answer pair and exists in the form of a table, for example, the table may have two columns and multiple rows, where each row corresponds to a question-answer pair, with one column for the question and one column for the answer; it may also be of the third type that does not contain chapter information and is not in the form of a question-answer pair and exists in the form of a text document.

[0056] S102: Based on storage rules corresponding to the types of the multiple knowledge documents, the multiple knowledge documents are stored in a vector database to obtain a knowledge base.

[0057] Specifically, different storage rules can be designed in advance according to different types of knowledge documents. In the process of building a database, the knowledge documents are stored in the vector database according to the storage rules corresponding to their types.

[0058] As an example, the storage rules include at least: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, wherein the chapter block includes the chapter title and the chapter text; in a chapter block, a concatenated text is obtained based on the chapter title and multiple single-sentence texts obtained by dividing the chapter text by sentence; the concatenated text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

[0059] Alternatively, see Figure 2 , which is a schematic diagram of a knowledge document containing chapter information provided by an embodiment of the present application. Before dividing the chapter blocks, you can first set the chapter titles to the title style corresponding to the hierarchy based on the hierarchy of the chapter titles in the knowledge document; set the chapter text in the knowledge document to the preset text style. For example, you can set the first-level title to the "Title 1" style, the second-level title to the "Title 2" style, and the third-level title to the "Title 3" style. Figure 2 As shown, the knowledge document includes three chapter titles: a first-level title "Welcome", a second-level title "1.1 Product Introduction" and a third-level title "1.1.1 Function Introduction".

[0060] Specifically, the knowledge document may be divided into a plurality of chapter blocks based on the chapter titles, and each chapter block may be composed of a chapter title and a chapter text.

[0061] Then, for each chapter block, based on the hierarchy of the chapter title, the chapter title is spliced ​​with all its parent chapter titles to obtain a joint title (T); for example, for the second-level title "1.1 Product Introduction", it can be spliced ​​with the first-level title "Welcome" through a short horizontal line, so as to obtain a joint title (T) "Welcome-1.1 Product Introduction"; for the third-level title "1.1.1 Function Introduction", it can be spliced ​​with the first-level title "Welcome" and the second-level title "1.1 Product Introduction" through a short horizontal line, in order from high to low levels, to obtain a joint title (T) "Welcome-1.1 Product Introduction-1.1.1 Function Introduction"; and for the first-level title "Welcome" without a parent title, the text of the chapter title can be directly assigned to the joint title (T), that is, the joint title (T) is "Welcome".

[0062] Next, the chapter text is divided into sentences to obtain multiple single-sentence texts. For example, the chapter text can be divided based on symbols such as periods and / or line breaks.

[0063] Then, the joint title (T) and the single sentence text are concatenated to obtain a concatenated text. For example, the joint title (T) and the single sentence text can be concatenated by a colon and a line break, thereby obtaining a concatenated text such as "joint title (T): (line break) single sentence text". Therefore, when searching based on the knowledge base constructed by this method, no matter whether the question to be searched is related to the chapter title or the chapter text, the relevant knowledge can be retrieved, which can improve the accuracy of the knowledge base search.

[0064] Finally, the concatenated text is stored as knowledge, the chapter text is stored as supplementary information "parent" as metadata of all knowledge in the chapter block, and each chapter block is stored in the vector database.

[0065] Optionally, if the knowledge document containing chapter information starts with a body style, the chapter title in the first chapter block obtained by division is a null value, and the chapter block is an untitled chapter block. For untitled chapter blocks, the chapter body can be divided by sentence to obtain multiple single-sentence texts; the single-sentence texts are stored as knowledge, and the chapter body is stored as supplementary information "parent" as metadata of all knowledge in the chapter block, and the untitled chapter blocks are stored in the vector database one by one.

[0066] In the embodiment of the present application, first, a plurality of knowledge documents are obtained; then, based on the storage rules corresponding to the types of the plurality of knowledge documents, the plurality of knowledge documents are stored in a vector database to obtain a knowledge base; wherein the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into a plurality of chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a plurality of single-sentence texts obtained by dividing the chapter text by sentence are obtained based on the chapter title; the concatenated text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database. Thus, different storage rules are set for different types of documents, especially for knowledge documents containing chapter information, through the storage rules provided by the embodiment of the present application, the entire text of the chapter block is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, no matter which knowledge is retrieved, the entire text of the corresponding chapter block can be obtained, and the integrity of the sentence or paragraph will not be destroyed due to the division of the knowledge document by sentence or line, thereby ensuring the integrity of the knowledge stored in the knowledge base.

[0067] See also Figure 3 , which is a flow chart of another method for constructing a knowledge base for a large language model provided in an embodiment of the present application, the method comprising:

[0068] S301: Start by acquiring multiple knowledge documents.

[0069] As an example, a knowledge document may be of the first type containing chapter information, in the form of a long text; or of the second type in the form of a question-answer pair, in the form of a table, for example, the table may have two columns and multiple rows, where each row corresponds to a question-answer pair, one column for the question and one column for the answer; or of the third type not containing chapter information and not in the form of a question-answer pair, in the form of a text document. As an example, the first type may be a long document in the format of .doc, .docx or .pdf; the second type may be a document in the format of .xls or .xlsx; and the third type may be a document in the format of .txt.

[0070] S302: Based on storage rules corresponding to the types of the multiple knowledge documents, the multiple knowledge documents are stored in a vector database to obtain a knowledge base.

[0071] Specifically, different storage rules can be designed in advance according to different types of knowledge documents. In the process of building a database, the knowledge documents are stored in the vector database according to the storage rules corresponding to their types.

[0072] As an example, the storage rules may include: (1) If the type of the knowledge document is the first type that contains chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, wherein the chapter block includes the chapter title and the chapter text; in a chapter block, a concatenated text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the concatenated text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in a vector database. (2) If the type of the knowledge document is the second type in the form of a question-answer pair, the question in the question-answer pair is stored as knowledge, and the answer is stored as supplementary information as metadata of the knowledge, and each question-answer pair is stored in a vector database. (3) If the type of the knowledge document is the third type that does not contain chapter information and is not in the form of a question-answer pair, the knowledge document is divided by sentence to obtain multiple texts; the texts are stored as knowledge, and each text is stored in a vector database.

[0073] Specifically, for a document containing chapter information, the knowledge document can be first divided into multiple chapter blocks based on the chapter titles; then each chapter block is traversed sequentially, and the following steps are performed for each chapter block: ① Based on the hierarchy of the chapter titles, the chapter title is spliced ​​with all its parent chapter titles to obtain a joint title (T), wherein the chapter titles can be spliced ​​in order from high to low hierarchy; ② The chapter body (i.e., the body text of the chapter block) is cut by sentence to obtain multiple single-sentence texts; ③ Each single-sentence text is traversed sequentially, and each single-sentence text is spliced ​​with the joint title (T) to obtain multiple spliced ​​texts; ④ The multiple spliced ​​texts are stored as knowledge respectively and stored in a vector database; ⑤ The body text is stored as supplementary information "parent" as metadata of all knowledge in the corresponding chapter block.

[0074] For a knowledge document in the form of a question-answer pair, taking a question-answer pair text located in the same row as an example, the question-answer pair data can be read in row order until all the question-answer pairs in the knowledge document are traversed; for each question-answer pair, the following steps are performed: ① The question in the question-answer pair is stored as knowledge and stored in the vector database; ② The answer in the question-answer pair is stored as supplementary information "answer" as metadata of the knowledge. Therefore, for a knowledge document in the form of a question-answer pair, the answer in the question-answer pair is stored as supplementary information as metadata of the corresponding knowledge. Therefore, when searching the knowledge base, the complete answer text corresponding to the retrieved question can be extracted, and the integrity of the answer will not be destroyed due to dividing the knowledge document by sentence or row, thereby ensuring the integrity of the knowledge stored in the knowledge base.

[0075] For other text documents that do not contain chapter information and are not in the form of question-answer pairs, the knowledge document can be divided by sentence to obtain multiple texts; each text is read sequentially and stored as knowledge until all texts are stored in the vector database. Therefore, storage rules are also provided for other text documents that do not contain chapter information and are not in the form of question-answer pairs, so that various types of knowledge documents can be smoothly stored in the knowledge base.

[0076] See also Figure 4 , which is a flow chart of a knowledge base retrieval method for a large language model provided in an embodiment of the present application, the method is used to search in a knowledge base constructed by the above-mentioned knowledge base construction method for a large language model, and the method includes:

[0077] S401: Receive questions to be searched.

[0078] Specifically, the question to be retrieved may be a question submitted by a user.

[0079] S402: Based on the question to be searched, call the search function of the vector database to retrieve the first k pieces of knowledge with the highest similarity to the question to be searched from the knowledge base; wherein k is a positive integer.

[0080] As an example, k can be a preset number, and its value can be flexibly set according to actual needs. For example, k can be set to 5, and when calling the retrieval function of the vector database and searching from the knowledge base, the similarity between the retrieval results and the question to be searched can be sorted from high to low, and then the knowledge with the top 5 similarities can be extracted.

[0081] S403: Add the supplementary information existing in the metadata of the retrieved knowledge to the reference information list.

[0082] Specifically, if there is supplementary information "parent" or "answer" in the metadata of the retrieved knowledge, the supplementary information "parent" or "answer" is taken out and added to the reference information list, where the combination of the reference information list and the question to be retrieved will serve as prompt information for the large language model.

[0083] Therefore, when there is supplementary information in the metadata of the retrieved knowledge, the complete supplementary information in the metadata can be extracted and provided to the large language model as reference information, so that the reference information provided to the large language model is semantically free of redundancy or omission, thereby improving the semantic accuracy of the reference information provided to the large language model, and further improving the relevance and completeness of the answers generated by the large language model.

[0084] See also Figure 5, which is a flow chart of another knowledge base retrieval method for a large language model provided in an embodiment of the present application, the method is used to search in a knowledge base constructed by the above-mentioned knowledge base construction method for a large language model, and the method includes:

[0085] S501: Set the number of retrieved knowledge items k and the maximum length L of a single reference information to be returned.

[0086] Specifically, k and L are both positive integers, and can be flexibly set before searching according to actual needs. Optionally, it is not necessary to set the values ​​of k and L before each search. After setting the values ​​of k and L once, before resetting the values ​​of k and L, the set values ​​of k and L can be used as default values ​​during the search process.

[0087] S502: Receive questions to be searched.

[0088] Specifically, the question to be retrieved may be a question submitted by a user.

[0089] S503: Based on the question to be searched, call the search function of the vector database to retrieve the first k pieces of knowledge with the highest similarity to the question to be searched from the knowledge base; wherein k is a positive integer.

[0090] S504: Traverse the k pieces of knowledge retrieved, and determine whether there is supplementary information in the metadata of the current knowledge. If so, execute step S505; if not, execute steps S506 to S507.

[0091] S505: Add the supplementary information to the reference information list.

[0092] Specifically, if the metadata of the retrieved knowledge contains supplementary information "parent" or "answer", the supplementary information "parent" or "answer" is retrieved and added to the reference information list.

[0093] S506: taking out a plurality of related knowledge pieces whose entry time is later than the current knowledge from the knowledge base in sequence based on the storage order.

[0094] Specifically, the total length of the current knowledge and the associated knowledge is: the maximum value that can be reached without exceeding the preset maximum length of a single reference information. For example, the preset maximum length of a single reference information is 500 characters. After 5 associated knowledge are taken out, the total length of the current knowledge and the associated knowledge is 450 characters. However, the length of the sixth associated knowledge is greater than 50 characters. Then, after taking out 5 associated knowledge, no further extraction will be performed. In the embodiment of the present application, in the process of storing the knowledge for which supplementary information does not exist in the metadata into the database, the multiple texts in the knowledge document are stored in sequence. Therefore, for the knowledge for which supplementary information does not exist in the metadata, the associated knowledge that is entered later than the current knowledge is extracted in sequence based on the storage order. The current knowledge and the associated knowledge are concatenated to obtain reference information with higher integrity. In addition, when extracting, the number of knowledge items is used as the unit, rather than extracting the context according to a fixed number of words, so that the semantics of the reference information can be ensured to be relatively complete.

[0095] S507: Add the text block obtained by concatenating the current knowledge and the associated knowledge to the reference information list.

[0096] As an example, after traversing k pieces of knowledge, a reference information list with k pieces of reference information will be obtained. By combining the search question and the k pieces of reference information, the prompt information of the large language model can be obtained. The prompt information is input into the large language model, and the large language model can generate a highly relevant and complete answer to the search question.

[0097] See also Figure 6 , which is a comparison diagram of answers generated by a large language model of a document in the form of a question-answer pair provided by an embodiment of the present application, Figure 6 The knowledge base in (a) is constructed by dividing knowledge by sentences. Figure 6 The knowledge base in (b) is constructed in the knowledge base construction method provided in this application, that is, the questions in the question-answer pair are stored as knowledge, and the answers are stored as supplementary information as metadata of the knowledge. Figure 6 In (a), the reference information retrieved from the knowledge base is incomplete and truncated, resulting in the lack of key points in the answer generated by the large language model. Figure 6 In (b), the reference information obtained through knowledge base retrieval is more complete, and the answer generated by the large language model is more comprehensive and complete.

[0098] See also Figure 7 , which is a comparison diagram of answers generated by a large language model of a document including chapter information provided by an embodiment of the present application, Figure 7 The knowledge base in (a) is constructed by dividing knowledge by sentences. Figure 7The knowledge base in (b) is constructed in the knowledge base construction method provided in this application, that is, the concatenated text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block. Figure 7 In (a), the reference information obtained through knowledge base retrieval is not comprehensive, while in Figure 7 In (b), the chapter title can also be retrieved, the retrieval effect is improved, the reference information obtained is more complete, and the answer generated by the large language model is more comprehensive and complete.

[0099] See also Figure 8 , this figure is a schematic diagram of a knowledge base construction device for a large language model provided in an embodiment of the present application, and the device includes: an acquisition module 801 and a storage module 802.

[0100] The acquisition module 801 is used to acquire multiple knowledge documents.

[0101] The storage module 802 is used to store multiple knowledge documents into a vector database based on the storage rules corresponding to the types of the multiple knowledge documents to obtain a knowledge base; the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, then the knowledge document is divided into multiple chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a spliced ​​text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the spliced ​​text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

[0102] Therefore, different storage rules are set for different types of documents. In particular, for knowledge documents containing chapter information, through the storage rules provided in the embodiments of the present application, the entire text of the chapter block is stored as supplementary information as metadata of all knowledge in the corresponding chapter block. No matter which knowledge is retrieved, the entire text of the corresponding chapter block can be obtained. The integrity of sentences or paragraphs will not be destroyed due to dividing the knowledge document by sentences or lines, thereby ensuring the integrity of the knowledge stored in the knowledge base.

[0103] Optionally, the storage module 802 includes a title splicing unit, a text division unit and a text splicing unit; wherein the title splicing unit is used to splice the chapter title with all its parent chapter titles based on the hierarchy of the chapter titles to obtain a joint title; the text division unit is used to divide the chapter text into sentences to obtain multiple single-sentence texts; the text splicing unit is used to splice the joint title with the single-sentence text to obtain a spliced ​​text.

[0104] Optionally, the storage module 802 further includes: a style setting unit, which is used to set the chapter title to a title style corresponding to the hierarchy based on the hierarchy of the chapter title in the knowledge document; and to set the chapter text in the knowledge document to a preset text style.

[0105] Optionally, the storage module 802 also includes: an untitled chapter block storage unit, which is used to divide the chapter text by sentences to obtain multiple single-sentence texts when there are untitled chapter blocks with empty chapter titles; store the single-sentence texts as knowledge and the chapter text as supplementary information as metadata of all knowledge in the chapter block, and store the untitled chapter blocks one by one into the vector database.

[0106] Optionally, the storage module 802 is also used to, when the type of the knowledge document is the second type in the form of question-answer pairs, store the questions in the question-answer pairs as knowledge, and the answers as supplementary information as metadata of the knowledge, and store each question-answer pair in the vector database.

[0107] Optionally, the storage module 802 is also used to, when the type of the knowledge document is the third type that does not contain chapter information and is not in the form of a question-and-answer pair, divide the knowledge document by sentences to obtain multiple texts; store the texts as knowledge, and store each text in a vector database.

[0108] See also Fig. 9 , this figure is a schematic diagram of a knowledge base retrieval device for a large language model provided in an embodiment of the present application, the device is used to search in a knowledge base constructed by the above-mentioned knowledge base construction method for a large language model, and the device includes: a receiving module 901, a retrieval module 902 and an adding module 903.

[0109] The receiving module 901 is used to receive questions to be searched.

[0110] The retrieval module 902 is used to call the retrieval function of the vector database based on the question to be retrieved, and retrieve the first k pieces of knowledge with the highest similarity to the question to be retrieved from the knowledge base; wherein k is a positive integer.

[0111] The adding module 903 is used to add the retrieved knowledge and the supplementary information in the metadata of the knowledge to the reference information list; the combination of the reference information list and the question to be retrieved is the prompt information of the large language model.

[0112] Therefore, when there is supplementary information in the metadata of the retrieved knowledge, the complete supplementary information in the metadata can be extracted and provided to the large language model as reference information, so that the reference information provided to the large language model is semantically free of redundancy or omission, thereby improving the semantic accuracy of the reference information provided to the large language model, and further improving the relevance and completeness of the answers generated by the large language model.

[0113] Optionally, the adding module 903 is also used for: when there is no supplementary information in the metadata of the retrieved current knowledge, taking out a plurality of related knowledge whose entry time is later than the current knowledge from the knowledge base in sequence based on the storage order; the total length of the current knowledge and the related knowledge is: the maximum value that can be reached without exceeding the preset maximum length value of a single reference information; and adding the text block obtained by splicing the current knowledge and the related knowledge to the reference information list.

[0114] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the unit described as a separate component may or may not be physically separated, and the component prompted as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A knowledge base construction method for a large language model, characterized in that: The method comprises: Acquire multiple knowledge documents; Based on the storage rules corresponding to the types of multiple knowledge documents, the multiple knowledge documents are stored in a vector database to obtain a knowledge base; the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a spliced ​​text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the spliced ​​text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

2. The knowledge base construction method according to claim 1, characterized in that: The multiple single-sentence texts obtained based on the chapter titles and the sentence-based division of the chapter text to obtain the spliced ​​text include: Based on the hierarchy of the chapter title, concatenate the chapter title with all its parent chapter titles to obtain a joint title; Divide the chapter text into sentences to obtain multiple single-sentence texts; The joint title and the single sentence text are concatenated to obtain a concatenated text.

3. The knowledge base construction method according to claim 1, characterized in that: Before dividing the knowledge document into a plurality of chapter blocks based on the chapter titles, the method further includes: Based on the hierarchy of the chapter titles in the knowledge document, the chapter titles are set to a title style corresponding to the hierarchy; and the chapter texts in the knowledge document are set to a preset text style.

4. The knowledge base construction method according to claim 1, characterized in that: After dividing the knowledge document into a plurality of chapter blocks based on the chapter titles, the method further includes: If there is an untitled chapter block with an empty chapter title, the chapter text is divided into sentences to obtain multiple single-sentence texts; the single-sentence texts are stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the chapter block, and the untitled chapter blocks are stored one by one in the vector database.

5. The knowledge base construction method according to claim 1, characterized in that: The storage rules also include: If the type of the knowledge document is the second type in the form of question-answer pairs, the questions in the question-answer pairs are stored as knowledge, and the answers are stored as supplementary information as metadata of the knowledge, and each question-answer pair is stored in the vector database.

6. The knowledge base construction method according to claim 1, characterized in that: The storage rules also include: If the type of the knowledge document is the third type that does not contain chapter information and is not in the form of a question-answer pair, the knowledge document is divided by sentence to obtain multiple texts; the texts are stored as knowledge, and each text is stored in a vector database.

7. A knowledge base retrieval method for a large language model, characterized in that: The method is used to search in a knowledge base constructed by the knowledge base construction method for a large language model according to any one of claims 1 to 6, and the method comprises: Receive questions to be retrieved; Based on the question to be searched, calling the search function of the vector database, searching from the knowledge base to obtain the first k pieces of knowledge with the highest similarity to the question to be searched; wherein k is a positive integer; The supplementary information existing in the metadata of the retrieved knowledge is added to the reference information list; the combination of the reference information list and the question to be retrieved is the prompt information of the large language model.

8. The knowledge base retrieval method according to claim 7, characterized in that: After calling the search function of the vector database based on the problem to be searched and retrieving the first k pieces of knowledge with the highest similarity to the problem to be searched from the knowledge base, the method further includes: If there is no supplementary information in the metadata of the retrieved current knowledge, multiple pieces of associated knowledge whose entry time is later than that of the current knowledge are retrieved from the knowledge base in sequence based on the order of storage; the total length of the current knowledge and the associated knowledge is: the maximum value that can be achieved without being greater than the preset maximum length value of a single piece of reference information; A text block obtained by concatenating the current knowledge and the associated knowledge is added to a reference information list.

9. A knowledge base construction device for a large language model, characterized in that: The device comprises: an acquisition module and a storage module; The acquisition module is used to acquire multiple knowledge documents; The storage module is used to store the multiple knowledge documents in a vector database based on the storage rules corresponding to the types of the multiple knowledge documents to obtain a knowledge base; the storage rules at least include: if the type of the knowledge document is the first type containing chapter information, the knowledge document is divided into multiple chapter blocks based on the chapter title, and the chapter block includes the chapter title and the chapter text; in a chapter block, a spliced ​​text is obtained based on the chapter title and the multiple single-sentence texts obtained by dividing the chapter text by sentence; the spliced ​​text is stored as knowledge, and the chapter text is stored as supplementary information as metadata of all knowledge in the corresponding chapter block, and each chapter block is stored in the vector database.

10. A knowledge base retrieval device for a large language model, characterized in that: The device is used to search in a knowledge base constructed by the knowledge base construction method for a large language model according to any one of claims 1 to 6, and the device comprises: a receiving module, a retrieval module and an adding module; The receiving module is used to receive the question to be retrieved; The retrieval module is used to call the retrieval function of the vector database based on the question to be retrieved, and retrieve the first k pieces of knowledge with the highest similarity to the question to be retrieved from the knowledge base; wherein k is a positive integer; The adding module is used to add the retrieved knowledge and the supplementary information in the metadata of the knowledge to the reference information list; the combination of the reference information list and the question to be retrieved is the prompt information of the large language model.

Citation Information

Patent Citations

  • Multi-modal multi-scale multi-recall large language model retrieval enhancement generation method

    CN118296120A

Cited By

  • Content generation method and device, computing equipment and medium

    CN120353901A

  • Knowledge base construction and retrieval method and system based on multi-source text in building field

    CN121681810A

  • Product business document generation method and apparatus based on large language model, and related device

    WO2026144355A1