Knowledge management system and method for constructing large voice model

Abstract

The embodiment of the invention provides a method for constructing a large language model knowledge management system. The method comprises a knowledge construction step, a knowledge retrieval step and an answer production step. According to the scheme, the artificial intelligence large language model technology can be utilized. External knowledge is obtained from multiple data sources, a search technology and a cue word function of a large language model are combined, RAG proposes a problem to the large model, knowledge obtained by searching the problem in the multiple data sources is used as a background context, and information of the problem and the background context is integrated into cue words of the large language model. And therefore, the large language model can make a final accurate answer.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a knowledge management system and method for building a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, especially large language model technology, the potential of knowledge management needs to be further explored. Large language models (such as ChatGPT, Qwen, Gemini, Gemma, etc.) have good natural language understanding capabilities. They can not only process and analyze large amounts of text data, but also generate high-quality summaries, answer complex queries, and even promote automated decision-making. These capabilities help to greatly improve the efficiency and intelligence level of knowledge management. However, when generating the final answer, the large language model will have hallucinations and generate inappropriate content due to insufficient knowledge in its own professional field, untimely knowledge updates, and inability to obtain industry data. This has become the biggest obstacle to application promotion in industry fields that require accurate, professional, and compliant content. Retrieval-augmented generation (RAG) came into being and has become the core technology for solving this problem in the current industry. Summary of the invention

[0003] In order to improve the efficiency of vehicle scheduling, the present application provides an intelligent vehicle scheduling method based on multi-factor dynamic optimization.

[0004] In order to achieve the above-mentioned purpose, the embodiment of the present application proposes a method for constructing a large language model knowledge management system, including: a knowledge construction step, a knowledge retrieval step, and an answer production step;

[0005] Knowledge construction steps: used to build knowledge of the large language model knowledge management system, including: knowledge data preprocessing, indexing and knowledge storage, mainly to uniformly process the internal database, knowledge graph, documents, external Web knowledge and constructed QA pairs of the enterprise, and store them in a unified form of the enterprise knowledge base to complete the processing and construction of enterprise knowledge data; specifically include:

[0006] Step 1.1: Document knowledge can be transformed from original document data to knowledge in the knowledge base only after preprocessing, document segmentation, and vector embedding. First, the irregular documents are preprocessed to form well-structured documents. Second, they are segmented according to paragraph recursive segmentation, title segmentation, line segmentation, fixed delimiter segmentation, title segmentation, and semantic segmentation. While meeting the limit of the slice size, the semantics of each slice is relatively complete. Finally, by adding the chapter or title information of the slice to the head of each slice content, the slice content can better maintain the semantic information in the original document.

[0007] Step 1.2: The knowledge graph organizes knowledge into a network structure to represent it, connecting various entities and the relationships between them, providing an intuitive structured representation of knowledge. The knowledge graph construction process includes: data preprocessing, entity recognition, relationship extraction, attribute extraction, knowledge integration and storage; since the big voice model has a strong semantic understanding ability, the big voice model can be used to perform entity recognition and relationship extraction through preset prompts to form triples for storage;

[0008] Step 1.3, database knowledge is the knowledge stored in traditional relational databases, analytical databases and other databases; through relational database theory or data warehouse theory, the database data accumulated by the enterprise is used to establish relevant databases and tables. In the enterprise knowledge management system, this part of data needs to be included in the knowledge management system so that users can easily use the existing database knowledge;

[0009] The knowledge retrieval step is used to implement the knowledge retrieval process of the large language model knowledge management system, including: basic retrieval process and complex retrieval process. Among them, the typical complex retrieval process includes pre-retrieval processing, knowledge retrieval, post-retrieval processing, and answer generation; specifically, it includes:

[0010] Step 2.1, retrieval pre-processing refers to a series of technologies and methods for pre-processing user queries, aiming to optimize queries to improve retrieval efficiency and accuracy. Pre-processing methods use multiple query expansion, decomposition of subqueries, term replacement, completion history, reference ablation, hypothetical answers, StepBack prompts, query rewriting, query routing, etc. Through pre-processing, the knowledge retrieval system can more effectively understand the user's query intention, optimize the query to adapt to the complex retrieval environment, and thus provide more accurate and relevant retrieval results, which include:

[0011] Step 2.1.1, decomposing subqueries is to split a complex original query into several smaller and easier to handle parts, each of which represents an information-independent sub-problem. To achieve this goal, a multi-query retriever can be used. This retriever uses a large speech model to automatically generate multiple queries for a given user input from multiple dimensions, and then automatically executes the prompt optimization process. For each generated subquery, the multi-query retriever will retrieve a set of related documents, and finally take a union operation on the documents retrieved by all subqueries, thereby constructing a wider set of potentially relevant documents. Decomposing subqueries can break through the limitations of vector distance-based retrieval methods, thereby obtaining a richer and more diverse set of retrieval results;

[0012] Step 2.1.2, Codon resolution is used to process user queries containing codons to improve the accuracy of the retrieval system for user queries. Few-shot Prompt is used in combination with the Think-Action-Observe (CoT) strategy to perform codon resolution. By integrating some common codon resolution scenarios as Few-shot examples into the Prompt of the large speech model and combining the CoT method, the large speech model can analyze and handle more complex codon resolution problems.

[0013] Step 2.1.3, query rewriting: By optimizing and adjusting the user's initial query at the language level, the retrieval efficiency is enhanced and the accuracy of the results is improved. For queries that are vague, unclear or overly complex in structure, query rewriting can more accurately grasp the user's information needs and return more relevant retrieval results. At the same time, in order to improve the accuracy of the retrieval system, query rewriting relies on the powerful capabilities of the big voice model. By using carefully designed prompt words, the big voice model can effectively rewrite the user's query. In order to further improve the query rewriting effect, an auxiliary model rewriter is introduced, which is responsible for adjusting the user query to better adapt it to the processing requirements of the fixed retriever and the big voice model. The auxiliary model rewriter can be obtained by collecting domain data for supervised pre-training or fine-tuning to meet the rewriting needs in actual application scenarios.

[0014] Step 2.2: Knowledge retrieval uses a variety of retrieval strategies to enhance the depth of retrieval and improve the accuracy of retrieval results. Figure 5 As shown, hybrid retrieval usually rewrites user query questions to generate one or more queries. After passing through the query routing module, these query questions are distributed to different retrieval method processes. After multiple retrieval methods such as database query, vector retrieval, QA retrieval, knowledge graph retrieval, plug-in retrieval, and keyword retrieval, each retrieval method will output TopK retrieval results. Due to the different scoring standards of the retrieval results generated by different retrieval methods, it is necessary to introduce a new re-ranking algorithm to combine and re-rank these TopK results to obtain the final TopK, and discard other candidate retrieval results. The final selected TopK will be handed over to the big model together with the user query questions as context, so that the big voice model can generate answers for the user's questions based on the context content; specifically including:

[0015] Step 2.2.1, Knowledge Graph Retrieval Using the semantic understanding ability of the big voice model and the prompt word engineering, a big voice model can be used to better perform knowledge retrieval on multiple knowledge graphs. Text2Cypher translates the user's question into a Cypher statement that the graph database can recognize, and then calls the graph database interface to execute the generated Cypher statement to obtain the execution result, and then generates the final answer through the big voice model. GraphRAG uses the contextual knowledge in the knowledge graph to process user queries by constructing a sub-graph. First, entities are extracted from the query content entered by the user, and then a sub-graph of entities related to the query is constructed to establish the context. Finally, the sub-graph information is sent to the big model as context together with the user query to give an accurate answer;

[0016] Step 2.2.2, Text2SQL, is a process for converting natural language (NL) queries into SQL query language executable in relational databases. Users can make query requests in natural language without writing SQL statements, thus reducing the complexity of interacting with the database. By introducing the big model, the entire Text2SQL process can be accelerated and the accuracy rate can be increased from the original 60% to 80%. Text2SQL for database data retrieval mainly includes the following steps: first, use the Schema filter to filter the Schema related to the user input, then give the filtered Schema list and the question to the big model, use the big model to generate and execute SQL statements, and finally use the big model to analyze and summarize the SQL execution results;

[0017] Step 2.2.3, plug-in API retrieval is a way for the big voice model to interact with the outside world. It is the process of calling external services or functions through API, mainly function calling. Use API filters to filter user queries input by users through natural language, extract TopK candidate-related APIs, and send these API definitions and user queries to the big model for processing. For the big voice model that supports the Function Call function, it will return information such as the name and parameters of the function call. The big model can better send requests and receive responses through API descriptions, and combine queries to more accurately call related APIs for retrieval;

[0018] Step 2.3, post-retrieval processing, is responsible for further optimization and adjustment of the retrieval results to improve the performance of the retrieval system and the quality of the retrieval results. The core activities of this stage include but are not limited to filtering, compressing and re-ranking the retrieval results in order to refine and organize a set of final results. These results will then be submitted to the large speech model to generate response answers; specifically, they include:

[0019] Step 2.3.1, context compression is obtained through RAG. The context length often reaches thousands of tokens. When the results obtained in the retrieval step are large and exceed the context length of the large model, the context needs to be compressed to remove redundant information and query-irrelevant noise while maintaining semantics, thereby providing more effective context information for the large speech model. The LongLLMLingua method is used for context compression. First, a small encoder model (usually a bidirectional encoder such as BERT) is used to encode the long input into a shorter vector representation, and then the encoded vector is input into a large speech model together with the query (the large speech model decoder can recognize the information encoded by the small encoder) to generate the final output. LongLLMLingua uses the aligned and trained small model to detect and remove unimportant tokens in the context, and converts them into a form that is difficult for humans to understand but easy for the large speech model to understand, effectively improving system performance;

[0020] Step 2.3.2, the rearrangement operation is to adjust the relevance score of the search results obtained in the search stage again, and adopt a more sophisticated sorting algorithm to achieve the rearrangement of the search results. On the one hand, the cross encoder is introduced to design an efficient scoring model. For a given query, the cross encoder encodes and scores all the search results, and then arranges them in descending order by score. The one with the highest score is the most relevant search result. On the other hand, bge-rerank is used as a rearranger, and the bge-embedding model is used for document embedding, which has achieved good results. The optimization of the rearrangement link helps to improve the relevance and quality of the context, thereby providing more reliable semantic support for the final answer generation;

[0021] The answer production process of the large language model knowledge management system is used for users to use the enterprise knowledge base to retrieve knowledge after the knowledge base is built; the knowledge retrieval process will obtain the contextual information of the knowledge content related to the user's question. Finally, based on the contextual content obtained in the knowledge retrieval process, the large language model generates the final answer.

[0022] At the same time, the embodiment of the present application also proposes a large language model knowledge management system for implementing the above method, including: an infrastructure module, a large model capability module, a knowledge storage module, a knowledge service module and a business application module; wherein,

[0023] The infrastructure module is the underlying foundation for building a large language model knowledge management system, including the computing, storage, and network resources required to run the system, especially the GPU resources needed for model deployment and reasoning. The deployment method can be based on public cloud services, on the enterprise's internal private cloud, or on hybrid cloud services.

[0024] The large model capability module includes a variety of pre-trained general large language model knowledge models, which are used to understand and generate natural language and are the intelligent core of the system. The large model capability includes not only the exclusive large language model trained or fine-tuned for a specific field, but also the embedding model and rearrangement model used in the knowledge construction and knowledge retrieval process, as well as the prompt word engineering using these models. By calling the computing resources of the infrastructure layer, it provides large model services for other layers of the entire system.

[0025] The knowledge storage module is responsible for storing and managing the knowledge assets of the enterprise and mainly provides knowledge storage services. The knowledge storage module includes a database system for storing structured data, a distributed storage system for storing unstructured data such as documents, pictures, audio and video, a graph database for storing knowledge graph data, an external plug-in subsystem for accessing external knowledge search engine APIs obtained through external application programming interfaces (APIs), and a vector database for storing vector data generated based on large model embedding technology;

[0026] The knowledge service module consists of three units: knowledge construction unit, knowledge retrieval unit, and knowledge management unit:

[0027] The knowledge construction unit is used to pre-process knowledge data from various sources, then import it into the system and store it using the storage component of the knowledge storage module. Common sources of knowledge data include unstructured document data, structured database data, question-and-answer data, knowledge graph data, and external API plug-in data;

[0028] The knowledge retrieval unit is used to realize the process of obtaining knowledge answers based on user questions. The first step of retrieval requires understanding and rewriting the user questions, followed by retrieval in various ways. The data obtained by various retrieval methods will also undergo a re-ranking process, and the retrieval results will be generated after the large model finally understands them.

[0029] The knowledge management unit is used to uniformly encapsulate and manage system capabilities, provide knowledge service capabilities to the business layer, and encapsulate a unified knowledge open interface, knowledge retrieval capability interface, and knowledge question-and-answer capability interface for use by upper-layer businesses;

[0030] The business application module is used to carry out knowledge services provided by knowledge services for typical scenario business applications built on the large language model knowledge management system. This layer provides intelligent customer service that provides services in a question-and-answer format, product consulting assistants for pre-sales personnel or customers, knowledge retrieval and knowledge question-and-answer applications for enterprises, operation and maintenance service assistants for complex systems and scenarios, and a new generation of search engines based on large models.

[0031] The beneficial effects of the above technical solution of the present application are as follows: The solution of the embodiment of the present application can utilize the artificial intelligence large language model technology. By acquiring external knowledge from multiple data sources, combining search technology and the prompt word function of the large language model, RAG asks questions to the large model, and uses the knowledge obtained by searching the multiple data sources as the background context, and integrates the question and background context information into the prompt words of the large language model, so that the large language model can make the final accurate answer. The technical solution of the embodiment of the present application has at least one of the following advantages:

[0032] (1) Build a knowledge management system based on the RAG architecture to provide efficient knowledge retrieval and utilization capabilities for industry intelligent operation and maintenance.

[0033] (2) By combining deep mining and integration of these heterogeneous knowledge sources, we propose open system architectures, processes, and methods to form a comprehensive professional knowledge base and ensure data security.

[0034] (3) Adopt an effective embedding and Rerank combination model to further improve the retrieval efficiency and accuracy of RAG technology.

[0035] (4) Introduce a compliance review mechanism for the final answer to improve the efficiency of knowledge retrieval and the quality of generated answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of constructing a large language model knowledge management system according to an embodiment of the present application;

[0037] Figure 2 A schematic diagram of constructing a large language model knowledge management system according to an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the knowledge construction process of the large language model knowledge management system according to an embodiment of the present application;

[0039] Figure 4 This is a schematic diagram of the knowledge retrieval process of implementing a large language model knowledge management system according to an embodiment of the present application;

[0040] Figure 5 A schematic diagram of hybrid retrieval according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] To make the technical problems, technical solutions and advantages to be solved by the present application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the examples in the present application and the features in the examples can be combined with each other, and unless explicitly required, separate components and functions are optional, and the order of operation can be changed. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. The scope of the embodiments of the present invention includes the entire scope of the claims, and all available equivalents of the claims. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with examples.

[0042] According to an embodiment of the present invention, a large language model knowledge management system and method are provided; Figure 1 As shown, the construction of a large language model knowledge management system mainly includes: infrastructure module, large model capability module, knowledge storage module, knowledge service module and business application module; among them,

[0043] The infrastructure module is the underlying foundation for building a large language model knowledge management system, including the computing, storage, and network resources required to run the system, especially the graphics processing unit (GPU) resources required for model deployment and reasoning. The deployment method can be based on public cloud services, on the enterprise's internal private cloud, or on hybrid cloud services.

[0044] The big model capability module, including a variety of pre-trained general large language model knowledge models, is used to understand and generate natural language and is the intelligent core of the system. The big model capability includes not only exclusive large language models trained or fine-tuned for specific fields to adapt to specific fields, but also embedding models and rearrangement models used in the process of knowledge construction and knowledge retrieval, as well as prompt word engineering using these models. By calling the computing resources of the infrastructure layer, big model services are provided for other layers of the entire system.

[0045] The knowledge storage module is responsible for storing and managing the knowledge assets of the enterprise, and mainly provides knowledge storage services. The knowledge storage module includes a database system for storing structured data, a distributed storage system for storing unstructured data such as documents, pictures, audio and video, a graph database for storing knowledge graph data, an external plug-in subsystem for accessing external knowledge search engine APIs obtained through external application programming interfaces (APIs), and a vector database for storing vector data generated based on large model embedding technology.

[0046] The knowledge service module consists of three units: knowledge construction unit, knowledge retrieval unit, and knowledge management unit:

[0047] The knowledge construction unit is used to pre-process knowledge data from various sources, then import it into the system and store it using the storage component of the knowledge storage module. Common sources of knowledge data include unstructured document data, structured database data, question-and-answer (QA) data, knowledge graph data, and external API plug-in data;

[0048] The knowledge retrieval unit is used to realize the process of obtaining knowledge answers based on user questions. The first step of retrieval is to understand and rewrite the user's questions, and then use multiple methods to retrieve. The data obtained by multiple retrieval methods will also go through a re-ranking process, and the retrieval results will be generated after the big model finally understands them;

[0049] The knowledge management unit is used to uniformly encapsulate and manage system capabilities, provide knowledge service capabilities to the business layer, and encapsulate a unified knowledge open interface, knowledge retrieval capability interface, and knowledge question and answer capability interface for use by upper-level businesses.

[0050] The business application module is used to carry out typical scenario business applications built on the large language model knowledge management system. Through the knowledge services provided by the knowledge service, this layer provides intelligent customer service that provides services in the form of questions and answers, product consulting assistants for market pre-sales personnel or customers, knowledge retrieval and knowledge Q&A applications for enterprises (especially library book retrieval, archives archive retrieval, legal and regulatory article retrieval, intellectual property patent retrieval), operation and maintenance service assistants for complex systems and scenarios, and new generation search engines based on large models.

[0051] like Figure 2 As shown, a large language model knowledge management system can be constructed by the following method: (1) knowledge construction step, (2) knowledge retrieval step, (3) answer production step;

[0052] 1. Knowledge construction step; it is used for the knowledge construction process of the large language model knowledge management system, the process is as follows Figure 3 As shown in the figure, it includes: knowledge data preprocessing, indexing and knowledge storage, mainly to uniformly process the internal database, knowledge graph, documents, external Web knowledge and constructed QA pairs of the enterprise, and store them in a unified form of the enterprise knowledge base to complete the processing and construction of enterprise knowledge data; specifically includes:

[0053] 1.1 Document knowledge can only be transformed from original document data to knowledge in the knowledge base after preprocessing, document segmentation, and vector embedding; first, the irregular documents are normalized and preprocessed to form well-structured documents; second, they are segmented according to paragraph recursive segmentation, title segmentation, line segmentation, fixed delimiter segmentation, title segmentation, and semantic segmentation, while meeting the limit on the size of the slice, ensuring that the semantics of each slice are relatively complete; finally, by adding the chapter or title information of the slice to the header of each slice content, the slice content can better maintain the semantic information in the original document.

[0054] 1.2 The knowledge graph represents knowledge by organizing it into a network structure graph, connecting various entities and the relationships between them, and providing an intuitive structured representation of knowledge. The knowledge graph construction process includes: data preprocessing, entity recognition, relationship extraction, attribute extraction, knowledge integration and storage; because the big voice model has a strong semantic understanding ability, the big voice model can be used to perform entity recognition and relationship extraction through preset prompts to form triples for storage.

[0055] 1.3 Database knowledge is the knowledge stored in traditional relational databases, analytical databases and other databases. In the process of informatization construction, enterprises generally accumulate a large amount of database data. Through relational database theory or data warehouse theory, enterprises have established relevant databases and tables. In the enterprise knowledge management system, it is not necessary to repeat the construction of this part of knowledge data, but it is necessary to incorporate this part of data into the knowledge management system so that users can easily use the existing database knowledge.

[0056] 2. Knowledge retrieval step, used to implement the knowledge retrieval process of the large language model knowledge management system. The process is as follows Figure 4 As shown in the figure, it includes: basic retrieval process and complex retrieval process. Among them, the typical complex retrieval process includes pre-retrieval processing, knowledge retrieval, post-retrieval processing, answer generation and other steps. Each retrieval step involves many technical details, and the basic retrieval process often omits the pre-retrieval processing and post-processing links. By adopting a variety of pre- and post-processing and hybrid knowledge retrieval methods, the accuracy of RAG is improved.

[0057] 2.1 Pre-retrieval processing refers to a series of technologies and methods for pre-processing user queries, aiming to optimize queries to improve retrieval efficiency and accuracy. Pre-processing methods use multiple query expansion, decomposition of subqueries, term replacement, completion history, reference ablation, hypothetical answers, StepBack prompts, query rewriting, query routing, etc. Through pre-processing, the knowledge retrieval system can more effectively understand the user's query intent, optimize the query to adapt to the complex retrieval environment, and thus provide more accurate and relevant retrieval results.

[0058] 2.1.1 Decomposing subqueries is to split a complex original query into several smaller and easier to handle parts, each of which represents an information-independent sub-problem. To achieve this goal, a multi-query retriever can be used. With the help of a large speech model, the retriever automatically generates multiple queries for a given user input from multiple dimensions, and then automatically executes the prompt optimization process. For each subquery generated, the multi-query retriever retrieves a set of related documents, and finally takes a union operation on the documents retrieved by all subqueries to build a wider set of potentially relevant documents. Decomposing subqueries can break through the limitations of vector distance-based retrieval methods, thereby obtaining a richer and more diverse set of retrieval results.

[0059] 2.1.2 Co-reference resolution is used to handle situations where user queries contain pronouns (such as "it", "this company", etc.), which helps improve the accuracy of the retrieval system for user queries. Co-reference resolution is performed using the Fewshot Prompt and the Think-Action-Observe (CoT) strategy. By integrating some common co-reference resolution scenarios as Few-shot examples into the Prompt of the Big Voice Model and combining it with the CoT method, the Big Voice Model can analyze and handle more complex co-reference resolution problems.

[0060] 2.1.3 Query Rewriting By optimizing and adjusting the user's initial query at the language level, the retrieval efficiency and the accuracy of the results are enhanced. For queries that are vague, unclear, or overly complex in structure, query rewriting can more accurately grasp the user's information needs and return more relevant search results. At the same time, in order to improve the accuracy of the retrieval system, query rewriting relies on the powerful capabilities of the big voice model, using carefully designed prompt words to enable the big voice model to effectively rewrite the user's query. In order to further improve the query rewriting effect, an auxiliary model "rewriter" is introduced, which is specifically responsible for adjusting user queries to better adapt them to the processing requirements of fixed retrievers and big voice models. The auxiliary model rewriter can be obtained by collecting domain data for supervised pre-training or fine-tuning to meet the rewriting needs in actual application scenarios.

[0061] 2.2 Knowledge retrieval uses a variety of retrieval strategies to enhance the depth of retrieval and improve the accuracy of retrieval results. Figure 5As shown, hybrid retrieval usually rewrites user query questions to generate one or more queries. After passing through the query routing module, these query questions are distributed to different retrieval method processes. After multiple retrieval methods such as database query, vector retrieval, QA retrieval, knowledge graph retrieval, plug-in retrieval, keyword retrieval, etc., each retrieval method will output the TopK retrieval results. Since the retrieval results generated by different retrieval methods have different scoring criteria, they cannot be simply combined and sorted. It is necessary to introduce a new rearrangement algorithm to combine and re-sort these TopK results to obtain the final TopK and discard other candidate retrieval results. The topK that is finally selected will be handed over to the big model together with the user query question as the context, so that the big voice model can generate answers for the user's questions based on the context content.

[0062] 2.2.1 Knowledge Graph Retrieval Using the semantic understanding ability of the big voice model and the prompt word engineering, a big voice model can be used to better perform knowledge retrieval on multiple knowledge graphs. Text2Cypher translates user questions into Cypher statements that the graph database can recognize, and then calls the graph database interface to execute the generated Cypher statement to obtain the execution result, and then generates the final answer through the big voice model capability. GraphRAG uses the contextual knowledge in the knowledge graph to process user queries by constructing a sub-graph. First, entities are extracted from the query content entered by the user, and then a sub-graph of entities related to the query is constructed to establish context. Finally, the sub-graph information is sent to the big model as context together with the user query to give an accurate answer.

[0063] 2.2.2Text2SQL, also known as NL2SQL, is the process of converting natural language (NL) queries into SQL query language that can be executed in relational databases. Users can make query requests in natural language without having to write SQL statements, thereby reducing the complexity of interacting with the database. By introducing a big model, the entire Text2SQL process can be accelerated and the accuracy rate can be increased from the original 60% to 80%. Text2SQL database data retrieval mainly includes the following steps: first, use the Schema filter to filter the Schema related to the user input, then give the filtered Schema list and the question to the big model, use the big model to generate and execute SQL statements, and finally use the big model to analyze and summarize the SQL execution results.

[0064] 2.2.3 Plug-in API retrieval is a way for the big voice model to interact with the outside world. It is the process of calling external services or functions through APIs, mainly based on function calls. Use API filters to filter user queries entered by users through natural language, extract the TopK candidate-related APIs, and send these API definitions and user queries to the big model for processing. For big voice models that support the Function Call function, it will return information such as the name and parameters of the function call. The big model can better send requests and receive responses through API descriptions, and combine queries to more accurately call relevant APIs for retrieval.

[0065] 2.3 Post-processing is responsible for further optimization and adjustment of the retrieval results to improve the performance of the retrieval system and the quality of the retrieval results. The core activities of this stage include but are not limited to operations such as screening, compression and re-ranking of the retrieval results. The purpose is to refine and organize a set of final results. These results will then be submitted to the large speech model to generate response answers.

[0066] 2.3.1 Context Compression The context length obtained by RAG often reaches thousands of tokens. When the results obtained in the retrieval step are large and exceed the context length of the large model, the context needs to be compressed to remove redundant information and query-irrelevant noise while maintaining semantics, thereby providing more effective context information for the large speech model. The LongLLMLingua method is used for context compression. First, a small encoder model (usually a bidirectional encoder such as BERT) is used to encode the long input into a shorter vector representation, and then the encoded vector is input into a large speech model together with the query (the large speech model decoder can recognize the information encoded by the small encoder) to generate the final output. LongLLMLingua uses an aligned and trained small model to detect and remove unimportant tokens in the context, and converts them into a form that is difficult for humans to understand but easy for the large speech model to understand, effectively improving system performance.

[0067] 2.3.2 The rearrangement operation is to adjust the relevance scores of the retrieval results obtained in the retrieval stage again, and adopt a more sophisticated sorting algorithm to achieve the rearrangement of the retrieval results. On the one hand, the cross encoder is introduced to design an efficient scoring model. For a given query, the cross encoder encodes and scores all the retrieval results, and then arranges them in descending order by score. The one with the highest score is the most relevant retrieval result. On the other hand, bge-rerank is used as a rearranger, and the bge-embedding model is used for document embedding, which has achieved good results. The optimization of the rearrangement link helps to improve the relevance and quality of the context, thereby providing more reliable semantic support for the final answer generation.

[0068] Among them, (3) the answer production process of the large language model knowledge management system includes: after the knowledge base is built, the user uses the enterprise knowledge base to retrieve knowledge. The knowledge retrieval process will obtain contextual information about the knowledge content related to the user's question. Finally, based on the contextual content obtained in the knowledge retrieval process, the large language model generates the final answer.

[0069] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage medium such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0070] A person skilled in the art may understand that the various numerical numbers such as first and second involved in the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application, and also indicate the order of precedence.

[0071] At least one in the present application can also be described as one or more, and a plurality can be two, three, four or more, which is not limited in the present application. In the embodiments of the present application, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no order of precedence or size between the technical features described by the "first", "second", "third", "A", "B", "C" and "D".

[0072] The corresponding relationships shown in each table in the present application can be configured or predefined. The values ​​of the information in each table are only examples and can be configured as other values, which are not limited by the present application. When configuring the corresponding relationship between the information and each parameter, it is not necessarily required to configure all the corresponding relationships illustrated in each table. For example, in the table in the present application, the corresponding relationships shown in some rows may not be configured. For another example, appropriate deformation adjustments can be made based on the above table, such as splitting, merging, etc. The names of the parameters shown in the titles of the above tables can also use other names that can be understood by the communication device, and the values ​​or representations of the parameters can also be other values ​​or representations that can be understood by the communication device. When implementing the above tables, other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables or hash tables.

[0073] The predefined in the present application may be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

[0074] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0076] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for constructing a large speech model knowledge management system, comprising: knowledge construction step, knowledge retrieval step, answer production step; Knowledge building steps; It is used to build the knowledge of large language model knowledge management system, including: knowledge data preprocessing, indexing and knowledge storage, mainly to uniformly process the internal database, knowledge graph, documents, external Web knowledge and constructed QA pairs of the enterprise, and store them in a unified form of enterprise knowledge base to complete the processing and construction of enterprise knowledge data; specifically including: Step 1.1: Document knowledge can be transformed from original document data to knowledge in the knowledge base only after preprocessing, document segmentation, and vector embedding. First, the irregular documents are preprocessed to form well-structured documents. Second, they are segmented according to paragraph recursive segmentation, title segmentation, line segmentation, fixed delimiter segmentation, title segmentation, and semantic segmentation. While meeting the limit of the slice size, the semantics of each slice is relatively complete. Finally, by adding the chapter or title information of the slice to the head of each slice content, the slice content can better maintain the semantic information in the original document. Step 1.2: The knowledge graph organizes knowledge into a network structure to represent it, connecting various entities and the relationships between them, providing an intuitive structured representation of knowledge. The knowledge graph construction process includes: data preprocessing, entity recognition, relationship extraction, attribute extraction, knowledge integration and storage; since the big voice model has a strong semantic understanding ability, the big voice model can be used to perform entity recognition and relationship extraction through preset prompts to form triples for storage; Step 1.3, database knowledge is the knowledge stored in traditional relational databases, analytical databases and other databases; through relational database theory or data warehouse theory, the database data accumulated by the enterprise is used to establish relevant databases and tables. In the enterprise knowledge management system, this part of data needs to be included in the knowledge management system so that users can easily use the existing database knowledge; The knowledge retrieval step is used to implement the knowledge retrieval process of the large language model knowledge management system, including: basic retrieval process and complex retrieval process. Among them, the typical complex retrieval process includes pre-retrieval processing, knowledge retrieval, post-retrieval processing, and answer generation; specifically, it includes: Step 2.1, retrieval pre-processing refers to a series of technologies and methods for pre-processing user queries, aiming to optimize queries to improve retrieval efficiency and accuracy. Pre-processing methods use multiple query expansion, decomposition of subqueries, term replacement, completion history, reference ablation, hypothetical answers, StepBack prompts, query rewriting, query routing, etc. Through pre-processing, the knowledge retrieval system can more effectively understand the user's query intention, optimize the query to adapt to the complex retrieval environment, and thus provide more accurate and relevant retrieval results, which include: Step 2.1.1, decomposing subqueries is to split a complex original query into several smaller and easier to handle parts, each of which represents an information-independent sub-problem. To achieve this goal, a multi-query retriever can be used. This retriever uses a large speech model to automatically generate multiple queries for a given user input from multiple dimensions, and then automatically executes the prompt optimization process. For each generated subquery, the multi-query retriever will retrieve a set of related documents, and finally take a union operation on the documents retrieved by all subqueries, thereby constructing a wider set of potentially relevant documents. Decomposing subqueries can break through the limitations of vector distance-based retrieval methods, thereby obtaining a richer and more diverse set of retrieval results; Step 2.1.2, Co-reference resolution is used to process user queries containing co-reference words to improve the accuracy of the retrieval system for user queries. Few-shot Prompt is used in combination with the think-action-observe CoT strategy to resolve co-reference. By integrating some common co-reference resolution scenarios as Few-shot examples into the Prompt of the large speech model and combining the CoT method, the large speech model can analyze and handle more complex co-reference resolution problems. Step 2.1.3, query rewriting: By optimizing and adjusting the user's initial query at the language level, the retrieval efficiency is enhanced and the accuracy of the results is improved. For queries that are vague, unclear or overly complex in structure, query rewriting can more accurately grasp the user's information needs and return more relevant retrieval results. At the same time, in order to improve the accuracy of the retrieval system, query rewriting relies on the powerful capabilities of the big voice model. By using carefully designed prompt words, the big voice model can effectively rewrite the user's query. In order to further improve the query rewriting effect, an auxiliary model rewriter is introduced, which is responsible for adjusting the user query to better adapt it to the processing requirements of the fixed retriever and the big voice model. The auxiliary model rewriter can be obtained by collecting domain data for supervised pre-training or fine-tuning to meet the rewriting needs in actual application scenarios. Step 2.2, knowledge retrieval uses a variety of retrieval strategies to enhance the depth of retrieval and improve the accuracy of retrieval results. As shown in Figure 5, hybrid retrieval usually rewrites user query questions to generate one or more queries. After passing through the query routing module, these query questions are distributed to different retrieval method processes. After multiple retrieval methods such as database query, vector retrieval, QA retrieval, knowledge graph retrieval, plug-in retrieval, keyword retrieval, etc., each retrieval method will output TopK retrieval results. Due to the different scoring standards of the retrieval results generated by different retrieval methods, it is necessary to introduce a new rearrangement algorithm to combine and re-rank these TopK results to obtain the final TopK, and discard other candidate retrieval results. The final selected TopK will be handed over to the big model together with the user query questions as context, so that the big voice model can generate answers for the user's questions based on the context content; specifically including: Step 2.2.1, knowledge graph retrieval uses the semantic understanding ability of the big voice model and the prompt word engineering. Through a big voice model, knowledge retrieval can be performed on multiple knowledge graphs. Text2Cypher translates the user's question into a Cypher statement that the graph database can recognize, and then calls the graph database interface to execute the generated Cypher statement to obtain the execution result, and then generates the final answer through the big voice model. GraphRAG uses the contextual knowledge in the knowledge graph to process user queries by constructing a sub-graph. First, entities are extracted from the query content entered by the user, and then a sub-graph of entities related to the query is constructed to establish the context. Finally, the sub-graph information is sent to the big model as context together with the user query to give an accurate answer. Step 2.2.2, Text2SQL, is a process for converting natural language NL queries into SQL query languages ​​that can be executed in relational databases. Users can make query requests in natural language without writing SQL statements, thus reducing the complexity of interacting with the database. By introducing a big model, the entire Text2SQL process can be accelerated and the accuracy rate can be increased from the original 60% to 80%. Text2SQL for database data retrieval mainly includes the following steps: first, use the Schema filter to filter the Schema related to the user input, then give the filtered Schema list and the question to the big model, use the big model to generate and execute SQL statements, and finally use the big model to analyze and summarize the SQL execution results; Step 2.2.3, plug-in API retrieval is a way for the big voice model to interact with the outside world. It is the process of calling external services or functions through API, mainly function calling Function Calling. Use API filters to filter user queries input by users through natural language, extract TopK candidate related APIs, and send these API definitions and user queries to the big model for processing. For the big voice model that supports the Function Call function, it will return the name and parameters of the function call and other information. The big model can better send requests and receive responses through API descriptions, and combine queries to more accurately call related APIs for retrieval; Step 2.3, post-retrieval processing, is responsible for further optimization and adjustment of the retrieval results to improve the performance of the retrieval system and the quality of the retrieval results. The core activities of this stage include but are not limited to filtering, compressing and re-ranking the retrieval results in order to refine and organize a set of final results. These results will then be submitted to the large speech model to generate response answers; specifically, they include: Step 2.3.1, context compression is obtained through RAG. The context length often reaches thousands of tokens. When the results obtained in the retrieval step are large and exceed the context length of the large model, the context needs to be compressed to remove redundant information and query-irrelevant noise while maintaining semantics, thereby providing more effective context information for the large speech model. The LongLLMLingua method is used for context compression. First, a small encoder model is used to encode the long input into a shorter vector representation, and then the encoded vector is input into a large speech model together with the query. The large speech model decoder can recognize the information encoded by the small encoder and generate the final output. LongLLMLingua uses the aligned and trained small model to detect and remove unimportant tokens in the context, and converts them into a form that is difficult for humans to understand but easy for the large speech model to understand, effectively improving the system performance; Step 2.3.2, the rearrangement operation is to adjust the relevance score of the search results obtained in the search stage again, and adopt a more sophisticated sorting algorithm to achieve the rearrangement of the search results. On the one hand, the cross encoder is introduced to design an efficient scoring model. For a given query, the cross encoder encodes and scores all the search results, and then arranges them in descending order by score. The one with the highest score is the most relevant search result. On the other hand, bge-rerank is used as a rearranger, and the bge-embedding model is used for document embedding, which has achieved good results. The optimization of the rearrangement link helps to improve the relevance and quality of the context, thereby providing more reliable semantic support for the final answer generation; The answer production process of the large language model knowledge management system is used for users to use the enterprise knowledge base to retrieve knowledge after the knowledge base is built; the knowledge retrieval process will obtain the contextual information of the knowledge content related to the user's question. Finally, based on the contextual content obtained in the knowledge retrieval process, the large language model generates the final answer.

2. A large speech model knowledge management system for implementing the method according to claim 1, comprising: Infrastructure module, large model capability module, knowledge storage module, knowledge service module and business application module; among them, The infrastructure module is the underlying foundation for building a large language model knowledge management system, including the computing, storage, and network resources required to run the system, especially the GPU resources needed for model deployment and reasoning. The deployment method can be based on public cloud services, on the enterprise's internal private cloud, or on hybrid cloud services. The large model capability module includes a variety of pre-trained general large language model knowledge models, which are used to understand and generate natural language and are the intelligent core of the system. The large model capability includes not only the exclusive large language model trained or fine-tuned for a specific field, but also the embedding model and rearrangement model used in the knowledge construction and knowledge retrieval process, as well as the prompt word engineering using these models. By calling the computing resources of the infrastructure layer, it provides large model services for other layers of the entire system. The knowledge storage module is responsible for storing and managing the knowledge assets of the enterprise and mainly provides knowledge storage services. The knowledge storage module includes a database system for storing structured data, a distributed storage system for storing unstructured data such as documents, pictures, audio and video, a graph database for storing knowledge graph data, an external plug-in subsystem for accessing external knowledge search engine APIs obtained through external application programming interfaces (APIs), and a vector database for storing vector data generated based on large model embedding technology; The knowledge service module consists of three units: knowledge construction unit, knowledge retrieval unit, and knowledge management unit: The knowledge construction unit is used to pre-process knowledge data from various sources, then import it into the system and store it using the storage component of the knowledge storage module. Common sources of knowledge data include unstructured document data, structured database data, question-and-answer data, knowledge graph data, and external API plug-in data; The knowledge retrieval unit is used to realize the process of obtaining knowledge answers based on user questions. The first step of retrieval requires understanding and rewriting the user questions, followed by retrieval in various ways. The data obtained by various retrieval methods will also undergo a re-ranking process, and the retrieval results will be generated after the large model finally understands them. The knowledge management unit is used to uniformly encapsulate and manage system capabilities, provide knowledge service capabilities to the business layer, and encapsulate a unified knowledge open interface, knowledge retrieval capability interface, and knowledge question-and-answer capability interface for use by upper-layer businesses; The business application module is used to carry out knowledge services provided by knowledge services for typical scenario business applications built on the large language model knowledge management system. This layer provides intelligent customer service that provides services in a question-and-answer format, product consulting assistants for pre-sales personnel or customers, knowledge retrieval and knowledge question-and-answer applications for enterprises, operation and maintenance service assistants for complex systems and scenarios, and a new generation of search engines based on large models.

Images