Retrieval Augmentation and Generation Methods and Systems for Knowledge Management

Abstract

This invention provides a retrieval augmentation and generation method for knowledge management, comprising: a processing unit inputting a first prompt word into a first large-scale language model, outputting a question description and a keyword set; the processing unit embedding the question description and keyword set into vectors and performing a retrieval in a vectorized knowledge database; the processing unit inputting the question description, keyword set, and various knowledge content fragments into a ranking model, outputting a list of retrieval results; and the processing unit inputting a structured framework into a second large-scale language model, outputting a response or at least a dynamic prompt. Thus, by understanding, filtering, combining, and structuring through the first large-scale language model, the ranking model, and the second large-scale language model, the large-scale language model is effectively utilized while protecting data confidentiality.

Description

Technical Field

[0001] This invention relates to a knowledge management system, and more particularly to a retrieval, augmentation, and generation method and system for knowledge management. Prior Technology

[0002] In recent years, large language models (LLMs) have demonstrated powerful natural language generation and understanding capabilities and have been widely applied in various scenarios. However, the knowledge of large language models is time-sensitive, unable to provide real-time information or incorporate the latest changes into the generated content; furthermore, due to the limited data used to train the models, large language models may fabricate answers out of thin air, generating content that is actually incorrect; additionally, large language models lack background knowledge in specific domains, and for highly specialized or niche questions, they may generate inaccurate or even completely wrong responses; finally, the update and maintenance costs of large language models are high, which is generally unaffordable for most enterprises.

[0003] To address the aforementioned issues, Retrieval-Augmented Generation (RAG) technology has emerged. Through a pre-established knowledge base, it provides large language models with real-time, complete, or domain-specific data, generating more accurate, reliable, and immediate responses. In particular, it provides professional information, internal documents, and data for specific knowledge domains, allowing large language models to be applied in specific professional fields or within enterprises.

[0004] However, for applications requiring high confidentiality, such as internal corporate technical documents, using external large-scale language models poses the risk of leaking confidential information as training material, while building local large-scale language models is prohibitively expensive. Therefore, resolving this dilemma is a pressing issue in current artificial intelligence technology. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a retrieval augmentation generation method and system for knowledge management. After segmenting and decontextualizing the original data, it provides the data to at least two large language models for generation according to the instructions provided by us. This ensures that a single external large language model cannot know the full content of the original data, thereby protecting the confidentiality of the original data and saving the cost of creating large language models ourselves.

[0006] To achieve the aforementioned objectives, this invention provides a retrieval, augmentation, and generation method for knowledge management, comprising the following steps: a processing unit inputs a first prompt word into a first large-scale language model, outputting a question description and a keyword set, wherein the first prompt word instructs the first large-scale language model to understand and augment a user input by referring to a historical record, thereby obtaining the question description and the keyword set, the historical record being stored in a user database, and the historical record including a dialogue summary record and a browsing record; the processing unit embeds the question description and the keyword set into a vector, and performs a retrieval in a vectorized knowledge database, obtaining multiple retrieval results, each of which contains a fragment of knowledge content; the processing unit then embeds the question description, the keyword set, and each fragment of knowledge content into a vectorized knowledge database. The processing unit inputs a content fragment into a ranking model and outputs a list of search results. The ranking model compares the similarity of the question description, the keyword set, and each knowledge content fragment, assigns a weight value to each search result, and filters and ranks the search results according to the weight value to obtain the list of search results. The processing unit inputs a structured framework into a second large-scale language model and outputs a response content or at least a dynamic prompt. The structured framework includes a second prompt word and a response instruction. The second prompt word includes the user input, the history, the question description, the keyword set, and the list of search results, and is aggregated according to the instructions of the structured framework. The response content or the dynamic prompt is obtained according to the word count, detail, style, or method indicated by the response instruction.

[0007] The present invention further provides a retrieval, augmentation, and generation system for knowledge management, comprising a memory for storing one or more computer programs containing multiple instructions; and a processing unit for executing the instructions to perform the following operations: the processing unit inputs a first prompt word into a first large-scale language model and outputs a question description and a keyword set, wherein the first prompt word instructs the first large-scale language model to understand and augment a user input with reference to a historical record to obtain the question description and the keyword set, the historical record being stored in a user database and including a dialogue summary record and a browsing record; the processing unit embeds the question description and the keyword set into a vector and performs a retrieval in a vectorized knowledge database to obtain multiple retrieval results, each retrieval result containing a fragment of knowledge content; the processing unit... The problem description, keyword set, and each knowledge content fragment are input into a ranking model, which outputs a list of search results. The ranking model compares the similarity of the problem description, keyword set, and each knowledge content fragment, assigns a weight value to each search result, and filters and ranks the search results according to the weight value to obtain the list of search results. The processing unit inputs a structured framework into a second large-scale language model and outputs a response content or at least a dynamic prompt. The structured framework includes a second prompt word and a response instruction. The second prompt word includes the user input, the history, the problem description, the keyword set, and the list of search results, and is aggregated according to the instructions of the structured framework. The response content or the dynamic prompt is obtained according to the word count, detail, style, or method indicated by the response instruction.

[0008] Through the above steps, the retrieval augmentation generation method and system for knowledge management provided by this invention pre-stores various types of text data that have been segmented, indexed, and converted into vectors in a vectorized knowledge database. Then, through a first large-scale language model, a ranking model, and a second large-scale language model, it further understands, filters, combines, and structures the data to independently and accurately generate response content or dynamic prompts. It effectively uses at least two large-scale language models to complete instructions, and through the process of splitting and recombining, as well as the application of different large-scale language models, it protects confidential data. Simple Explanation of the Diagram

[0009] Figure 1 is a block diagram of a retrieval, augmentation, and generation system for knowledge management according to an embodiment of the present invention; Figure 2 is a flowchart of a retrieval, augmentation, and generation method for knowledge management according to an embodiment of the present invention; Figure 3 is a schematic diagram of the knowledge management retrieval, augmentation, and generation method in one embodiment of the present invention; Figure 4 is a flowchart of a retrieval, augmentation, and generation method for knowledge management according to another embodiment of the present invention; and Figure 5 is a schematic diagram of the knowledge management retrieval and augmentation generation method in another embodiment of the present invention. Implementation

[0010] The following will describe a retrieval, augmentation, and generation method and system for knowledge management according to an embodiment of the present invention, with reference to the relevant figures, wherein the same elements will be described with the same reference numerals.

[0011] Referring to Figure 1, the retrieval, augmentation, and generation system 10 for knowledge management includes a memory 11, a processing unit 12, and a user interface 13. The memory 11 is electrically connected to the processing unit 12, and the processing unit 12 generates the user interface 13. The memory 11 can be a non-transitory computer-readable medium, such as read-only memory, flash memory, hard disk, optical disk, USB flash drive, network database, or other accessible media, used to store one or more computer programs 14 containing multiple instructions. The processing unit 12 can be a central processing unit or a microprocessor. The user interface 13 allows users to operate the computer programs 14 stored in the memory 11 through the processing unit 12. It can be further operated using a keyboard, mouse, touchpad, or a touchpad or similar device connected to a mobile electronic device (such as a mobile phone or tablet), which are not shown here and are not limited thereto.

[0012] Please also refer to Figures 1 and 2. When memory 11 stores one or more computer programs 14 containing multiple instructions, processing unit 12 executes the instructions stored in memory 11. When processing unit 12 executes the instructions, it performs the following disclosed retrieval, augmentation, and generation method 20 for knowledge management, which includes steps S10 to S40, detailed below.

[0013] Referring to Figures 2 and 3, in step S10, the processing unit 12 inputs a first prompt word 30 into a first large-scale language model 31 and outputs a question description 32 and a keyword set 33. The first prompt word 30 instructs the first large-scale language model 31 to understand and amplify a user input 35 by referring to a history record 34 to obtain the question description 32 and the keyword set 33. The history record 34 is stored in a user database 36 and includes a dialogue summary record 341 and a browsing record 342. Furthermore, the first prompt 30 utilizes the dialogue summary record 341 and browsing record 342 in the history record 34 to understand the content that the user has browsed, read, and interacted with the system in the past. This content includes, but is not limited to, various sources such as books, papers, and multimedia, to understand the user's intentions, purposes, or causal relationships within the content input by the user 35. The dialogue summary record 341 is a record of the complete dialogue between the user and the system, or a simplified or extracted version, which serves as the system's short-term or long-term memory, allowing the system to recall past interactions with the user and maintain the continuity of the dialogue. Moreover, the browsing record 342 records what content the user has browsed or read, specifically which specific sources and paragraphs have been browsed or read, in order to avoid recommending duplicate information in subsequent question-and-answer sessions or to conduct more in-depth exploration based on the read content. By understanding the content of user input, the first large-scale language model 31 can enrich and expand the content of user input 35, further supplementing the user with additional information to express the user's intent more completely and clearly, so as to obtain a more accurate and detailed question description 32. From this question description 32, or in combination with user input 35, a keyword set 33 containing multiple keywords is extracted as an important basis for subsequent retrieval.

[0014] In another preferred embodiment, as shown in FIG5, step S10 further includes: the first prompt word 30 instructs the first large language model 31 to refer to an intent classification example set, identify the intent of the user input 35, and output a user intent code 351. The intent classification example set includes categories selected from one or more of the following groups: malicious attacks, system performance testing, product introduction, and frequently asked questions, but is not limited thereto. Specifically, in the intent classification example set, at least one classification example is provided for each of the categories such as malicious attacks, system performance testing, product introduction, and frequently asked questions, allowing the first large language model 31 to perform in-context learning based on the intent classification example set, thereby determining which category the user input 35's intent belongs to, and providing the user intent code 351. Through such identification and classification steps, user input can be analyzed and used as a reference for subsequent optimization. For example, it can help understand the most frequently asked questions by users and build security defense mechanisms accordingly. For example, it can identify malicious attacks such as prompt injection or malicious garbled input. By identifying attacks and malicious misleading in user input targeting large language models, it is possible to rewrite these malicious user inputs into normal content that meets service requirements, prevent system crashes, and avoid the generation of inappropriate content.

[0015] In another preferred embodiment, as shown in Figures 4 and 5, step S10 further includes step S11, whereby the processing unit 12, upon identifying the problem description as a common problem, outputs a preset response content 37. Furthermore, after obtaining the problem description 32 through step S10, before proceeding to step S20, the processor 12 will first identify the problem description 32. When the problem description 32 is a common problem, it outputs a preset response content 37 and simultaneously stops proceeding to subsequent steps S20, S30, and S40. This bypasses the intermediate retrieval, sorting, and structuring processes, directly providing a preset answer to save overall operational resources and time.

[0016] In step S20, as shown in Figures 2 and 3, after the processing unit 12 embeds the problem description 32 and the keyword set 33 into a vector, it performs a search in a vectorized knowledge database 40 to obtain multiple search results 41. Each search result 41 contains a knowledge content fragment 42, and each knowledge content fragment 42 has a knowledge content fragment identification code 43. The knowledge content fragment identification code 43 corresponds to a specific text source, which is used to identify the source of the knowledge content fragment 42 in the vectorized knowledge database 40. It can also be encoded using various encoding rules. For example, the knowledge content fragment identification code 43 can include a text source identification code, paragraph level, sequence relationship, etc., to add more relevant information about the knowledge content fragment 42. Furthermore, the vectorized knowledge database 40 already stores data that has been pre-segmented, indexed, and converted into vectors from various types of text. Depending on the type of text, the vectorized knowledge database 40 is selected from one or more of the following database groups: book database, thesis database, or multimedia database. In one embodiment, it may also be a database of other specific data, such as a document database, contract database, report database, etc., but it is not limited to this. For example, in the book database, the knowledge content fragment identification code 43 may include the ISBN code of the book.

[0017] As shown in Figures 2 and 3, in step S30, processing unit 12 inputs the problem description 32, keyword set 33, and various knowledge content fragments 42 into a ranking model 50, and outputs a search result list 51. The ranking model 50 compares the similarity of the problem description 32, keyword set 33, and various knowledge content fragments 42, assigns a weight value to each search result 41, and then sorts the search results 41 from largest to smallest according to the weight value. Finally, according to a preset filtering parameter, a specific number of search results 41 are selected to obtain the search result list 51. For example, if step S20 obtains 50 search results 41, and then sorts them from largest to smallest according to the weight value, with the preset filtering parameter being 10, the top 10 search results 41 by weight value are retained and output as the search result list 51. It should be noted that filtering each search result 41 is an important verification mechanism and quality control procedure to filter out seemingly relevant but actually invalid content, ensuring that the information provided to subsequent steps is high-quality and useful search results. Through S30, a filtered, weighted, and sorted list of search results 51 is output. Each knowledge content fragment 42 in the search results list 51 can be traced and reconciled with its source and sequence through the knowledge content fragment identification code 43. For example, the index includes the correspondence between text identification codes and knowledge content fragment identification codes, thereby identifying the source and sequence.

[0018] As shown in Figures 2 and 3, in step S40, the processing unit 12 inputs a structured framework 60 (context engineering) into a second large-scale language model 61 and outputs a response content 62 or at least a dynamic prompt 63. The structured framework 60 includes a second prompt word 601 and a response instruction 602 (system prompt / system). The second prompt 601 includes user input 35, history 34, question description 32, keyword set 33, and search result list 51, and is compiled according to the instructions of the structured framework 60. It generates response content 62 or dynamic prompts 63 according to the word count, level of detail, style, or method indicated by the response instruction 602. The dynamic prompts 63 instruct the second large language model 61 to further produce at least one topic for in-depth thinking or exploration, providing the user with interactive prompts to continue the conversation. For example, it may suggest that the user read specific content of a specific text or ask deeper questions about the current conversation or topic. It may also recommend other works by the author of the text, other specific texts on similar topics, or even guide the user to explore different texts. It may also turn irrelevant questions into dynamic prompts, such as referring to those search results 41 that were not output as search result list 51 in step S30, expanding from these unused knowledge content fragments 42, and recommending extended topics or content that the user may be interested in. Furthermore, the structured framework 60 is a task specification specifically provided to the second large language model 61 for final inference. The second prompt word 601 dynamically concatenates basic information such as user input 35, history 34, question description 32, keyword set 33, and search result list 51. The specific instructions in the response instruction 602 include defining the response language, the topic to be generated, whether an index table needs to be generated, and the response method for different users, but are not limited to these.By combining the second prompt 601 and the response instruction 602, all necessary information is integrated into the structured framework 60. This way, the second large-scale language model 61 does not need to understand the previous retrieval, background understanding, and data processing processes. It only needs to read this complete and structured framework 60 to independently and accurately generate response content 62 or dynamic prompts 63. This achieves de-contextualization of data input, avoiding the use of non-local commercial large-scale language models, while also providing the source data content to the commercial large-scale language model for learning and training, thereby protecting confidential information. Furthermore, it allows for the distribution of tasks from various processes to two or more entities. The system can operate using large language models that are the same or different, local or non-local, achieving a high degree of modularity and flexibility. For example, two different non-local large language models can be used as the first large language model 31 and the second large language model 61, thus splitting steps S10 and S40; alternatively, one local large language model can be used as the first large language model 31, and the other can be a non-local large language model as the second large language model 61; or two different local large language models can be used as the first large language model 31 and the second large language model 61. Users can configure this according to their own considerations of cost and computing power. It should be noted that in step S40, the response content 62 or the dynamic prompt 63 can be produced selectively, or both can be produced simultaneously, depending on the system's default response.

[0019] In a preferred embodiment, the processing unit 12 is a single processor or may include multiple processors. When the processing unit 12 is multiple processors, these processors may be located in the same device or in different devices. When these devices are located in different locations, the retrieval, augmentation, and generation method for knowledge management disclosed in this invention can be implemented remotely or in the cloud. When at least one step, sub-step, or computer program described above is executed by processors located in devices in different locations, the retrieval, augmentation, and generation method for knowledge management disclosed in this invention can be implemented by multiple people. Therefore, the process implemented by multiple people in this invention can be executed at different times and locations. In other words, the retrieval, augmentation, and generation method for knowledge management disclosed in this invention is not limited to simultaneous, same-location, same-device, or same-person operation.

[0020] In summary, the retrieval, augmentation, and generation method and system for knowledge management provided by this invention employs at least two large language models to process data in stages, thereby decontextualizing the data input. This avoids providing the original data content to a single external commercial large language model for learning and training, effectively protecting confidential information. Simultaneously, users can configure the model themselves based on cost and computing power, helping to save the expense of building their own large language models. Furthermore, the first large language model understands and augments the user input, generating a question description and keyword set. Then, a ranking model filters out a high-quality list of search results. Finally, the second large language model only needs to read the integrated structured framework to independently and accurately generate response content or dynamic prompts, effectively utilizing multiple large language models to complete complex instructions and generating responses accurately and efficiently. Ultimately, the distributed processing using at least two large language models improves computational efficiency. Moreover, when the system identifies the question description as a common problem, it can directly output preset response content and stop subsequent retrieval, ranking, and structuring processes, significantly saving overall operational resources and time.

[0021] It must be noted that the detailed description above, in conjunction with the drawings, is merely one embodiment provided to illustrate the technical content and features of the present invention. Any simple modifications, substitutions, or reductions in components made by those with ordinary knowledge in the field of the present invention, after understanding the technical content and features of the present invention, without departing from the spirit of the present invention, should fall within the scope of the patent application disclosed in the present invention.

[0022] 10: Retrieval, Augmentation, and Generation Systems for Knowledge Management 11: Memory 12: Processing Unit 13: User Interface 14: Computer Programs 20: Retrieval Augmentation Generation Methods for Knowledge Management 30: First prompt word 31: The First Large-Scale Language Model 32: Problem Description 33: Keyword Set 34: Historical Records 341: Dialogue Summary Record 342: Browsing History 35: User Input 351: User Intent Code 36: User Database 37: Preset response content 40: Vectorized Knowledge Base 41: Search Results 42: Knowledge Content Fragments 43: Knowledge Content Fragment Identification Code 50: Sorting Model 51: List of Search Results 60: Structured Framework 601: Second prompt word 602: Response Command 61: The Second Large-Scale Language Model 62: Response Content 63: Dynamic prompts S10, S11, S20, S30, S40: Steps

Claims

1. A retrieval augmentation generation method for knowledge management, comprising the following steps: (a) A processing unit inputs a first prompt word into a first large-scale language model and outputs a question description and a keyword set, wherein the first prompt word instructs the first large-scale language model to understand and augment a user input with reference to a historical record in order to obtain the question description and the keyword set, the historical record being stored in a user database and including a dialogue summary record and a browsing record; (b) The processing unit embeds the question description and the keyword set into a vector and performs a retrieval in a vectorized knowledge database to obtain multiple retrieval results, each of which includes a knowledge content fragment; (c) The processing unit inputs the problem description, the keyword set, and each knowledge content fragment into a ranking model and outputs a list of search results. The ranking model compares the similarity of the problem description, the keyword set, and each knowledge content fragment, assigns a weight value to each search result, and filters and ranks the search results according to the weight value to obtain the list of search results. (d) The processing unit inputs a structured framework into a second large-scale language model and outputs a response content or at least a dynamic prompt. The structured framework includes a second prompt word and a response instruction. The second prompt word includes the user input, the history, the problem description, the keyword set, and the list of search results and is aggregated according to the instructions of the structured framework. The response content or the dynamic prompt is obtained according to the word count, detail, style, or method indicated by the response instruction.

2. As described in claim 1, in step (a), the first prompt word further instructs the first large language model to refer to an intent classification example set, identify the intent of the user input, and output a user intent code.

3. The method as described in claim 2, wherein the category items of the intent classification example set are selected from one or more of the following groups: malicious attacks, system performance testing, product introduction, and frequently asked questions.

4. The method described in request 3 further includes step (a1), in which the processing unit, when identifying the problem as a common problem, outputs a preset response.

5. The method as described in claim 1, in step (b), each knowledge content fragment includes a knowledge content fragment identification code, which corresponds to a specific text source to identify the source of the knowledge content fragment in the vectorized knowledge database.

6. The method as described in claim 1, in step (b), the vectorized knowledge database is selected from one or more of the following database groups: book database, paper database or multimedia database.

7. As described in Request 1, in step (d), the dynamic prompt instructs the second large language model reference that the search results not output as part of the search results list in step (c) be expanded upon the unused knowledge content fragments to recommend extended topics or content that may be of interest to the user.

8. A retrieval, augmentation, and generation system for knowledge management, comprising: A memory for storing one or more computer programs containing a plurality of instructions; a processing unit for executing the instructions to perform the following operations: (a) the processing unit inputs a first prompt word into a first large-scale language model and outputs a question description and a keyword set, wherein the first prompt word instructs the first large-scale language model to understand and amplify a user input by referring to a history record to obtain the question description and the keyword set, the history record being stored in a user database and containing a dialogue summary record and a browsing record; (b) the processing unit embeds the question description and the keyword set into a vector and performs a search in a vectorized knowledge database to obtain a plurality of search results, each of the search results containing a knowledge content fragment; (c) The processing unit inputs the problem description, the keyword set, and each knowledge content fragment into a ranking model and outputs a list of search results. The ranking model compares the similarity of the problem description, the keyword set, and each knowledge content fragment, assigns a weight value to each search result, and filters and ranks the search results according to the weight value to obtain the list of search results. (d) The processing unit inputs a structured framework into a second large-scale language model and outputs a response content or at least a dynamic prompt. The structured framework includes a second prompt word and a response instruction. The second prompt word includes the user input, the history, the problem description, the keyword set, and the list of search results and is aggregated according to the instructions of the structured framework. The response content or the dynamic prompt is obtained according to the word count, detail, style, or method indicated by the response instruction.

9. The system as described in claim 8, wherein in step (a), the first prompt word further instructs the first large language model to refer to an intent classification example set, identify the intent input by the user, and output a user intent code.

10. The system as described in claim 9, wherein the category items of the intent classification example set are selected from one or more of the following groups: malicious attacks, system performance testing, product introduction, and frequently asked questions.

11. The system as described in claim 10 further includes step (a1), in which the processing unit, upon identifying the problem as a common problem, outputs a preset response.

12. The system as described in claim 8, in step (b), each knowledge content fragment further includes a knowledge content fragment identification code, which corresponds to a specific text source for identifying the source of the knowledge content fragment in the vectorized knowledge database.

13. The system as described in claim 8, in step (b), the vectorized knowledge database is selected from one or more of the following database groups: book database, paper database or multimedia database.

14. The system as described in claim 8, in step (d), the dynamic prompt instructs the second large language model reference that the search results not output as the search results list in step (c) be expanded upon the unused knowledge content fragments to recommend extended topics or content that may be of interest to the user.

Images