Customized question answering system and method based on large language model

Abstract

The invention relates to the technical field of artificial intelligence, discloses a customized question-answering system and method based on a large language model, and aims to solve the problems that an existing question-answering system is poor in accuracy and lacks permission adaptation ability. The question and answer platform is linked with a local large language model, an online large language model, a knowledge graph, a business knowledge base, an authority management module, a multi-modal data processing module and a feedback module; the enterprise communication tool receives the multi-modal question data sent by the user, and receives and displays a question and answer result; the multi-modal data processing module converts multi-modal problem data into corresponding text information, the authority management module obtains user authority information and adjusts access content and editing authority of a knowledge graph and a business knowledge base of a user, and the knowledge graph and the business knowledge base generate corresponding retrieval results. And the large language model generates a question-answer result according to the retrieval result. According to the method, the accuracy of professional knowledge questions and answers is improved, and permission adaptation is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to a customized question-answering system and method based on a large language model. Background Art

[0002] A question-answering system based on a large language model (LLM) is an artificial intelligence system constructed based on deep learning technology, which can understand, generate natural language texts, and perform complex language tasks.

[0003] With the development of enterprises, cross-departmental collaboration and knowledge management have become increasingly complex. Traditional question-answering systems mostly rely on fixed rules or simple retrieval technologies, and cannot adapt to the rapidly changing business needs of enterprises. Moreover, it is difficult to understand and process the proprietary knowledge and terms of each business group in the enterprise, resulting in poor question-answering accuracy and difficulty in meeting the specific needs of different business groups. In addition, enterprise data is usually managed hierarchically (such as departments, roles, projects, etc.). Traditional question-answering systems mostly rely on predefined static rules (such as keywords, blacklists, etc.) and lack dynamic permission adaptation capabilities, making it difficult to cope with dynamic organizational structures or temporary permission requirements, and prone to risks of unauthorized access and enterprise information leakage. Summary of the Invention

[0004] The present invention aims to solve the problems of poor accuracy and lack of permission adaptation capabilities in existing question-answering systems, and proposes a customized question-answering system and method based on a large language model.

[0005] The technical solutions adopted by the present invention to solve the above technical problems are as follows:

[0006] In a first aspect, the present invention provides a customized question-answering system based on a large language model. The system includes a question-answering platform and an enterprise communication tool. The question-answering platform is linked with a local large language model, an online large language model, a knowledge graph, a business knowledge base, a permission management module, a multi-modal data processing module, and a feedback module based on the LangChain framework;

[0007] The Xinference platform and the OneAPI platform are introduced into the LangChain framework. The Xinference platform is used to manage the local large language model, and the OneAPI platform is used to manage the online large language model accessed through the API. The LangChain framework switches between the local large language model and the online large language model according to the question-answering requirements;

[0008] The enterprise communication tool is used to receive the multi-modal question data and feedback data sent by the user, send the multi-modal question data and feedback data to the question-answering platform, and receive and display the question-answering results sent by the question-answering platform;

[0009] The multimodal data processing module is used to convert the multimodal problem data into corresponding text information;

[0010] The permission management module is used to obtain the corresponding user permission information through the enterprise communication tool, and adjust the access content and editing permissions of the knowledge graph and business knowledge base of the corresponding user according to the user permission information;

[0011] The knowledge graph is constructed based on enterprise element data, and the business knowledge base is constructed based on enterprise knowledge data. The knowledge graph and business knowledge base are used to generate corresponding retrieval results according to the text information and user permission information;

[0012] The local large language model and the online large language model are used to generate natural language Q&A results according to the retrieval results of the knowledge graph and business knowledge base, and send the Q&A results to the enterprise communication tool;

[0013] The feedback module is used to update and optimize the knowledge graph and business knowledge base according to the feedback data.

[0014] Further, the enterprise communication tool is DingTalk, WeCom, Zhizhi, J2L3x or Feishu.

[0015] Further, the multimodal data processing module includes an image recognition unit and a speech recognition unit. The image recognition unit is used to convert the image data in the multimodal problem data into corresponding text information based on OCR technology, and the speech recognition unit is used to convert the speech data in the multimodal problem data into corresponding text information based on the ASR model.

[0016] Further, the enterprise element data includes entity nodes and entity relationships. The entity nodes at least include the products, departments, documents and technical keywords of the enterprise, and the entity relationships are obtained by analyzing the internal structure, business process, product hierarchy and customer information of the enterprise.

[0017] Further, the feedback module is also used to record the Q&A time and Q&A text corresponding to the user, perform statistics on the records, optimize the corresponding entries in the business knowledge base according to the statistical results, and adjust the weights of the corresponding entities and entity relationships in the knowledge graph according to the statistical results.

[0018] In a second aspect, the present invention provides a customized Q&A method based on a large language model, which is applied to the customized Q&A system based on a large language model as described in the first aspect. The method includes:

[0019] The enterprise communication tool receives the multimodal problem data sent by the user and sends the multimodal problem data to the Q&A platform;

[0020] The multimodal data processing module converts the multimodal problem data into corresponding text information;

[0021] The permission management module obtains the corresponding user permission information through the enterprise communication tool, and adjusts the access content and editing permissions of the knowledge graph and business knowledge base for the corresponding user according to the user permission information;

[0022] Retrieve in the knowledge graph and business knowledge base respectively according to the text information and user permission information, and generate corresponding retrieval results;

[0023] Select a local large language model or an online large language model according to the Q&A requirements. The local large language model and the online large language model generate natural language Q&A results based on the retrieval results of the knowledge graph and business knowledge base, and send the Q&A results to the enterprise communication tool;

[0024] After the enterprise communication tool receives and displays the Q&A results sent by the Q&A platform, it receives the feedback data sent by the user and sends the feedback data to the Q&A platform;

[0025] The feedback module updates and optimizes the knowledge graph and business knowledge base according to the feedback data.

[0026] Further, the enterprise communication tool is DingTalk, WeCom, Zhizhi, J2L3x or Feishu.

[0027] Further, the multimodal data processing module converts the multimodal problem data into corresponding text information, including:

[0028] The image recognition unit converts the image data in the multimodal problem data into corresponding text information based on OCR technology, and the speech recognition unit converts the speech data in the multimodal problem data into corresponding text information based on the ASR model.

[0029] Further, the enterprise element data includes entity nodes and entity relationships. The entity nodes at least include the products, departments, documents and technical keywords of the enterprise, and the entity relationships are obtained by analyzing the internal structure, business processes, product hierarchies and customer information of the enterprise.

[0030] Further, the method further includes:

[0031] The feedback module records the Q&A time and Q&A text corresponding to the user, statistically analyzes the records, optimizes the corresponding entries in the business knowledge base according to the statistical results, and adjusts the weights of the corresponding entities and entity relationships in the knowledge graph according to the statistical results.

[0032] The beneficial effects of the present invention are as follows: The customized Q&A system and method based on the large language model provided by the present invention generate Q&A results based on the retrieval results of the enterprise customized knowledge graph and business knowledge base by constructing the enterprise customized knowledge graph and business knowledge base, enabling employees to quickly and accurately obtain the required business knowledge, improving the accuracy of professional knowledge Q&A, and significantly reducing the time cost of information search and processing; by connecting the Q&A platform to the enterprise communication tool and using the enterprise communication tool to obtain the user permissions of the corresponding users, the access and editing permissions of the enterprise users to the knowledge graph and business knowledge base can be dynamically adjusted and adapted, thus avoiding the risks of unauthorized access and information leakage; by collecting the feedback data of the users on the Q&A results and optimizing the knowledge graph and business knowledge base according to the feedback data, the Q&A accuracy is further improved. Description of the Drawings

[0033] Figure 1 Schematic structural diagram of the customized Q&A system based on the large language model provided for the embodiment;

[0034] Figure 2 Schematic structural diagram of the customized Q&A method based on the large language model provided for the embodiment. Detailed Embodiments

[0035] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the present embodiment will be clearly and completely described below in conjunction with the accompanying drawings in the present embodiment.

[0036] In some processes described in the specification of the present invention and the above-mentioned drawings, there are multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations are only used to distinguish each different operation, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0037] To provide customized Q&A services for different business groups within an enterprise, improve the efficiency and accuracy of Q&A, and achieve permission adaptation to avoid risks of unauthorized access and information leakage, the technical solution of this application is proposed. In the present invention, a customized Q&A system based on a large language model includes: a Q&A platform and an enterprise communication tool. The Q&A platform is linked with a local large language model, an online large language model, a knowledge graph, a business knowledge base, a permission management module, a multi-modal data processing module, and a feedback module based on the LangChain framework; the Xinference platform and the OneAPI platform are introduced in the LangChain framework. The Xinference platform is used to manage the local large language model, and the OneAPI platform is used to manage the online large language model accessed through the API. The LangChain framework switches between the local large language model and the online large language model according to the Q&A requirements; the enterprise communication tool is used to receive multi-modal question data and feedback data sent by the user, send the multi-modal question data and feedback data to the Q&A platform, and receive and display the Q&A results sent by the Q&A platform; the multi-modal data processing module is used to convert the multi-modal question data into corresponding text information; the permission management module is used to obtain the corresponding user permission information through the enterprise communication tool, and adjust the access content and editing permissions of the knowledge graph and business knowledge base of the corresponding user according to the user permission information; the knowledge graph is constructed based on enterprise element data, and the business knowledge base is constructed based on enterprise knowledge data. The knowledge graph and the business knowledge base are used to generate corresponding retrieval results according to the text information and the user permission information; the local large language model and the online large language model are used to generate natural language Q&A results according to the retrieval results of the knowledge graph and the business knowledge base, and send the Q&A results to the enterprise communication tool; the feedback module is used to update and optimize the knowledge graph and the business knowledge base according to the feedback data.

[0038] Specifically, the present invention manages local large language models and online large language models by building a LangChain framework and introducing the Xinference platform and the OneAPI platform, and can then switch between local large language models and online large language models according to the question-and-answer requirements to ensure low latency and high performance; by connecting the question-and-answer platform to enterprise communication tools, enterprise users can use the enterprise communication tools to make question-and-answer requests and obtain the question-and-answer results of the question-and-answer platform, improving the question-and-answer efficiency. Moreover, during the question-and-answer process, the question-and-answer platform can use the enterprise communication tools to obtain the user permissions of the corresponding users, and then can dynamically adjust and adapt the access and editing permissions of enterprise users to the knowledge graph and business knowledge base, thus avoiding the risks of unauthorized access and information leakage; by constructing an enterprise-customized knowledge graph and business knowledge base and generating question-and-answer results based on the retrieval results of the enterprise-customized knowledge graph and business knowledge base, enterprise employees can quickly and accurately obtain the required business knowledge, improving the accuracy of professional knowledge question-and-answer and significantly reducing the time cost of information search and processing; by collecting the feedback data of users on the question-and-answer results and optimizing the knowledge graph and business knowledge base according to the feedback data, the question-and-answer accuracy is further improved.

[0039] Next, the technical solutions in this embodiment will be clearly and completely described with reference to the accompanying drawings in this embodiment. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0040] Figure 1 The structure diagram of a customized question-and-answer system based on a large language model is shown. Please refer to Figure 1 , the system includes: a question-and-answer platform and enterprise communication tools. The question-and-answer platform is linked with a local large language model, an online large language model, a knowledge graph, a business knowledge base, a permission management module, a multi-modal data processing module, and a feedback module based on the LangChain framework.

[0041] In this embodiment, the Xinference platform and the OneAPI platform are introduced into the LangChain framework. The Xinference platform is used to manage local large language models, and the OneAPI platform is used to manage online large language models accessed through APIs. The LangChain framework switches between local large language models and online large language models according to the question-and-answer requirements.

[0042] It can be understood that LangChain is an open-source framework that simplifies the development of applications based on large language models through modular design and standardized interfaces, connecting models, data sources, and toolchains, thereby enabling the construction of complex AI applications and supporting interactions with multiple external systems (such as enterprise communication tools, databases, etc.) and large language models. Xinference is a high-performance distributed inference framework for generative AI scenarios, and OneAPI is a unified programming model launched by Intel that simplifies the development process across heterogeneous hardware (such as CPUs, GPUs, FPGAs, AI accelerators), improving computing efficiency and reducing multi-architecture programming complexity.

[0043] In this embodiment, the Xinference platform manages locally deployed large language models, and the OneAPI platform manages online large language models accessed through APIs. In practical applications, the Q&A platform dynamically selects a local large language model or an online large language model according to the Q&A requirements or business scenarios to generate corresponding Q&A results, realizing the dynamic switching of large language models according to actual needs, thereby improving the Q&A efficiency and accuracy.

[0044] In this embodiment, the enterprise communication tool can be, but is not limited to, DingTalk, WeCom, Zhizhi, J2L3x, or Feishu. The enterprise communication tool is used to receive the multi-modal question data and feedback data sent by the user, send the multi-modal question data and feedback data to the Q&A platform, and receive and display the Q&A results sent by the Q&A platform.

[0045] It can be understood that the enterprise communication tool interacts directly with enterprise users. Enterprise users use the enterprise communication tool to send Q&A requests to the Q&A platform. The Q&A request can be multi-modal question data, that is, a Q&A request containing data such as images, voices, and texts. After the Q&A platform generates corresponding Q&A results according to the Q&A request, it is received and displayed to enterprise users through the enterprise communication tool. Enterprise users can also send feedback data to the Q&A platform through the enterprise communication tool according to their satisfaction with the Q&A results.

[0046] In this embodiment, the multi-modal data processing module is used to convert the multi-modal question data into corresponding text information. The multi-modal data processing module includes an image recognition unit and a voice recognition unit. The image recognition unit is used to convert the image data in the multi-modal question data into corresponding text information based on OCR technology, and the voice recognition unit is used to convert the voice data in the multi-modal question data into corresponding text information based on the ASR model.

[0047] It can be understood that to ensure interoperability and data consistency among modules, a unified data exchange format and API specification can be defined. The multi-modal data processing module automatically detects the content type of the problem data and performs corresponding data conversion. In practical applications, for images containing text information, the Q&A platform supports users to upload them in various ways (such as drag-and-drop, file selection, etc.). After the Q&A platform receives the image data, it applies corresponding image preprocessing steps as needed (such as cropping, rotation correction, etc.), and efficiently and accurately converts the printed or handwritten text in the image into a machine-readable text format based on OCR technology; the Q&A platform also supports enterprise users to ask questions directly through the microphone. After the Q&A platform receives the voice data, it applies corresponding voice preprocessing steps as needed (such as noise reduction, volume adjustment, etc.), and quickly completes the conversion from voice to text locally based on the ASR model while maintaining high accuracy.

[0048] In this embodiment, the permission management module is used to obtain the corresponding user permission information through the enterprise communication tool, and adjust the access content and editing permissions of the knowledge graph and business knowledge base of the corresponding user according to the user permission information.

[0049] It can be understood that enterprise data is usually managed in layers (such as departments, roles, projects, etc.). Different users using the enterprise communication tool have different identities in the enterprise and also have different access and editing permissions for enterprise data. Based on this, in this embodiment, different access and editing permissions for the knowledge graph and business knowledge base are configured for different enterprise users through the permission management module. When enterprise users conduct Q&A through the enterprise communication tool, the permission management module obtains the corresponding user permission information through the enterprise communication tool, and then controls the access content and editing permissions of the knowledge graph and business knowledge base of the corresponding user. For example, DingTalk is connected to the Q&A platform, and enterprise users use DingTalk for Q&A. The permission management module obtains the identity information of the current DingTalk user (such as role, department, permission level, etc.). Ordinary users can query the knowledge related to their role, and only administrators have the permission to edit the business knowledge base, avoiding the chaos of business knowledge base information. For example, users in the sales department are allowed to access product materials, and users in the technical department are allowed to access technical documents and maintenance processes. In this way, enterprise users can only access and edit enterprise data within their own permission scope, thus avoiding unauthorized access and enterprise information leakage.

[0050] In this embodiment, the knowledge graph is constructed based on enterprise element data, the business knowledge base is constructed based on enterprise knowledge data, and the knowledge graph and business knowledge base are used to generate corresponding retrieval results according to the text information and user permission information.

[0051] In practical applications, the methods for constructing a knowledge graph include: extracting entity nodes from the enterprise's products, departments, documents, and technical keywords, obtaining entity relationships by analyzing the enterprise's internal structure, business processes, product hierarchies, and customer information, clarifying each entity and the association relationships between them, so as to construct a knowledge graph including entity nodes and entity relationships. In order to express the association strength or trust degree between different entities, indicators such as the number of user searches, interaction frequencies, and the number of associated documents can be used to calculate the weights of the edges. If an entity appears together with multiple other entities in similar business scenarios multiple times, a higher-weight connection will be formed in the knowledge graph. Finally, by converting the above knowledge graph into a graph data structure and combining the real business needs reflected by user feedback, a graph neural network is used for training. The model can learn the importance of entity nodes, the semantic information of relationships, and potential associations in the graph structure, not only can more accurately discover weak associations or implicit associations, but also can provide richer semantic understanding for the subsequent question-and-answer system.

[0052] In practical applications, the methods for constructing a business knowledge base include: according to the requirements and characteristics of each business group of the enterprise, collecting and organizing the enterprise knowledge data of different business groups, standardizing and organizing these enterprise knowledge data, using vectorization technologies such as FAISS (Fast Approximate Nearest Neighbors Search) to vectorize the documents, and customizing an exclusive business knowledge base, including enterprise knowledge data such as frequently asked questions, company internal documents, process manuals, etc., and regularly updating the enterprise knowledge data to keep up with business development. The accuracy and richness of the business knowledge base can be enhanced by combining real-time user feedback, automatically capturing the search behaviors and feedback of enterprise users, dynamically updating the entries of the business knowledge base, introducing intelligent knowledge completion technology, recording the questions that the current business knowledge base cannot answer during the daily Q&A process of users, creating process to-dos on the workbench using enterprise communication tools, and the administrator is responsible for regularly sorting out these questions and updating them to the business knowledge base.

[0053] In this embodiment, the local large language model and the online large language model are used to generate natural language Q&A results according to the retrieval results of the knowledge graph and the business knowledge base, and send the Q&A results to the enterprise communication tool.

[0054] It can be understood that the local language large model is deployed on local devices and does not rely on network connections. Users can fully control the model operation and data flow, and use the local language large model to generate Q&A results, which have the characteristics of high efficiency and data security. The online language large model is a third-party cloud service accessed through APIs (such as OpenAI, Google). It relies on network connections, and the service provider maintains model updates. The cloud computing power is strong, supporting complex reasoning tasks. Using the online language large model to generate Q&A results can improve the accuracy of Q&A. In practical applications, the local large language model or the online large language model can be dynamically selected according to the Q&A requirements or business scenarios to generate corresponding Q&A results, thus improving the flexibility of Q&A.

[0055] The large language model is a deep learning model trained based on a large amount of text data, capable of generating natural language text and understanding semantics. The local large language model and the online large language model in this embodiment can combine the context of the conversation, the retrieval results of the knowledge graph and the business knowledge base to generate professional natural language Q&A results.

[0056] In this embodiment, the feedback module is used to update and optimize the knowledge graph and the business knowledge base according to the feedback data.

[0057] Specifically, in order to ensure the continuous update and optimization of the knowledge base, this embodiment establishes a feedback mechanism. After presenting the Q&A results to enterprise users, enterprise users can directly send corresponding feedback data to the Q&A platform. For example, users feedback "incomplete", "information does not match my scenario", etc. In addition, after generating the Q&A results, the Q&A platform can also provide active feedback channels, such as adding evaluation buttons like "Are you satisfied with the current answer?", "Do you need additional information?", as well as an open-text feedback box. Enterprise users can give more specific opinions or problem descriptions through these channels, facilitating the system to accurately locate the deficiencies or imperfections in the knowledge graph and the business knowledge base. The Q&A system updates and optimizes the knowledge graph and the business knowledge base according to the analysis of the feedback data.

[0058] To ensure the validity of the feedback data, before the feedback data enters the subsequent analysis stage, it needs to be cleaned and preprocessed, including removing duplicate, irrelevant or obvious noise data, and uniformly formatting structured and unstructured data to facilitate subsequent data mining and model training. After analyzing the user feedback data, if some knowledge points are incorrect or expired, the system will trigger an automated or semi-automated correction process. Specifically, it can be reviewed by the administrator or professional personnel in combination with the feedback opinions. If it is confirmed that the content needs to be updated after review, it will be written into the latest version of the knowledge base; at the same time, if enterprise users often search for some uncollected or insufficiently answered content, the system can automatically generate a requirements report to guide experts to supplement knowledge or generate new entries; in addition, the business knowledge base has a version management function to mark and note each update. When there are content disputes or knowledge inconsistencies, it can quickly backtrack to the historical version for comparison to accurately locate the root cause of the problem.

[0059] In this embodiment, the feedback module is further configured to record the Q&A time and Q&A text corresponding to the user, count the records, optimize the corresponding entries in the business knowledge base according to the statistical results, and adjust the weights of the corresponding entities and entity relationships in the knowledge graph according to the statistical results.

[0060] Specifically, during the processes of enterprise users asking questions, reading answers, asking follow-up questions, and exiting, the system will record the interactions in detail, including timestamps, conversation texts, operation paths, etc. Combining the permission settings and information security policies of the business system, the data will be encrypted or anonymized and then stored in the data warehouse. By statistically analyzing the collected interaction data, the main demands of the user group at different time periods can be found, such as whether the attention to a certain product or process has increased sharply, or whether the demand for a certain type of question has been continuously rising. Based on this result, the corresponding entries in the business knowledge base can be optimized accordingly, and the weights of relevant entities and relationships in the knowledge graph can be enhanced. Newly emerging business requirements or newly added professional concepts can be added to the knowledge graph in an automatic or semi-automatic manner. The system continuously iterates or trains the GNN according to the demand changes revealed by user feedback, making the graph structure more complete and the association relationships more refined, so as to provide more accurate semantic support for subsequent Q&A.

[0061] Through the above mechanisms and processes, an efficient knowledge management and intelligent Q&A closed-loop can be formed within the enterprise. The system can not only improve the Q&A effect in the short term, but also continuously update the business knowledge base and knowledge graph to adapt to the dynamic changes of business development and user needs, and ultimately achieve a higher degree of intelligence and automation.

[0062] Based on the above customized Q&A system based on the large language model, this embodiment also proposes a customized Q&A method based on the large language model. Please refer to Figure 2, the method includes the following steps:

[0063] Step 1: The enterprise communication tool receives the multi-modal question data sent by the user and sends the multi-modal question data to the Q&A platform;

[0064] Step 2: The multi-modal data processing module converts the multi-modal question data into corresponding text information;

[0065] Step 3: The permission management module obtains the corresponding user permission information through the enterprise communication tool and adjusts the access content and editing permissions of the knowledge graph and business knowledge base for the corresponding user according to the user permission information;

[0066] Step 4: Retrieve in the knowledge graph and business knowledge base respectively according to the text information and user permission information to generate corresponding retrieval results;

[0067] Step 5: Select a local large language model or an online large language model according to the Q&A requirements. The local large language model and the online large language model generate natural language Q&A results according to the retrieval results of the knowledge graph and business knowledge base, and send the Q&A results to the enterprise communication tool;

[0068] Step 6: After the enterprise communication tool receives and displays the Q&A results sent by the Q&A platform, it receives the feedback data sent by the user and sends the feedback data to the Q&A platform;

[0069] Step 7: The feedback module updates and optimizes the knowledge graph and business knowledge base according to the feedback data.

[0070] Specifically, based on the above customized Q&A system based on large language models, in practical applications, enterprise users can initiate Q&A requests to the Q&A platform through enterprise communication tools. The Q&A requests can be multi-modal question data, that is, Q&A requests containing data such as images, voices, and texts. Before initiating the Q&A request, enterprise users can manually select a local large language model or an online large language model for Q&A, or automatically and dynamically switch the large language model according to actual business needs. After the Q&A platform receives the multi-modal question data, it calls the multi-modal data processing module to convert the multi-modal question data into corresponding text information, and at the same time calls the permission management model to obtain the user permission information of the enterprise user from the enterprise communication tool to determine the access and editing permissions of the enterprise user's knowledge graph and business knowledge base. Then, based on the converted text information, it queries and matches in the knowledge graph and business knowledge base to obtain retrieval results, and calls the corresponding local large language model or online large language model to generate natural language Q&A results according to the Q&A context and retrieval results, forming a smooth and accurate answer. Finally, the Q&A results are sent to the enterprise communication tool, and the enterprise communication tool displays them to the enterprise user. The enterprise user can send feedback data of the Q&A results to the Q&A platform so that the Q&A platform can call the feedback module to update and optimize the knowledge graph and business knowledge base according to the feedback data.

[0071] In summary, the customized Q&A system and method based on large language models provided in this embodiment construct an enterprise-customized knowledge graph and business knowledge base, and generate Q&A results based on the retrieval results of the enterprise-customized knowledge graph and business knowledge base, enabling employees to quickly and accurately obtain the required business knowledge, improving the accuracy of Q&A, and significantly reducing the time cost of information search and processing; by connecting the Q&A platform to enterprise communication tools and using the enterprise communication tools to obtain the user permissions of the corresponding users, the access and editing permissions of the enterprise users to the knowledge graph and business knowledge base can be dynamically adjusted and adapted, thus avoiding the risks of unauthorized access and information leakage; by collecting the feedback data of users on the Q&A results and optimizing the knowledge graph and business knowledge base according to the feedback data, the accuracy of Q&A is further improved; by dynamically selecting a local large language model or an online large language model to generate corresponding Q&A results, the flexibility of Q&A is improved; through the conversion of multi-modal question data, the system supports the input of multi-modal data such as pictures and voices, enhancing the flexibility of data retrieval and the user experience.

Claims

1. A customized question-answering system based on a large language model, characterized in that The system includes a Q&A platform and an enterprise communication tool. The Q&A platform is linked with a local large language model, an online large language model, a knowledge graph, a business knowledge base, a permission management module, a multimodal data processing module, and a feedback module based on the LangChain framework; The Xinference platform and the OneAPI platform are introduced into the LangChain framework. The Xinference platform is used to manage the local large language model, and the OneAPI platform is used to manage the online large language model accessed through the API. The LangChain framework switches between the local large language model and the online large language model according to the Q&A requirements; The enterprise communication tool is used to receive the multimodal question data and feedback data sent by the user, send the multimodal question data and feedback data to the Q&A platform, and receive and display the Q&A results sent by the Q&A platform; The multimodal data processing module is used to convert the multimodal question data into corresponding text information; The permission management module is used to obtain the corresponding user permission information through the enterprise communication tool, and adjust the access content and editing permissions of the knowledge graph and business knowledge base of the corresponding user according to the user permission information; The knowledge graph is constructed based on enterprise element data, and the business knowledge base is constructed based on enterprise knowledge data. The knowledge graph and business knowledge base are used to generate corresponding retrieval results according to the text information and user permission information; The local large language model and the online large language model are used to generate natural language Q&A results according to the retrieval results of the knowledge graph and business knowledge base, and send the Q&A results to the enterprise communication tool; The feedback module is used to update and optimize the knowledge graph and business knowledge base according to the feedback data.

2. The customized question-answering system based on the large language model according to claim 1, wherein The enterprise communication tool is DingTalk, WeCom, Zhizhi, J2L3x or Feishu.

3. The customized question-answering system based on the large language model according to claim 1, wherein The multimodal data processing module includes an image recognition unit and a voice recognition unit. The image recognition unit is used to convert the image data in the multimodal question data into corresponding text information based on OCR technology, and the voice recognition unit is used to convert the voice data in the multimodal question data into corresponding text information based on the ASR model.

4. The customized question-answering system based on the large language model according to claim 1, wherein, The enterprise element data includes entity nodes and entity relationships. The entity nodes at least include the products, departments, documents and technical keywords of the enterprise. The entity relationships are obtained by analyzing the internal structure, business processes, product hierarchies and customer information of the enterprise.

5. The customized question-answering system based on the large language model according to claim 1, characterized in that, The feedback module is also used to record the Q&A time and Q&A text corresponding to the user, statistically analyze the records, optimize the corresponding entries in the business knowledge base according to the statistical results, and adjust the weights of the corresponding entities and entity relationships in the knowledge graph according to the statistical results.

6. A customized question-answering method based on a large language model, characterized in that, Applied to the customized Q&A system based on the large language model according to any one of claims 1 to 5, the method includes: The enterprise communication tool receives the multimodal question data sent by the user and sends the multimodal question data to the Q&A platform; The multimodal data processing module converts the multimodal question data into corresponding text information; The permission management module obtains the corresponding user permission information through the enterprise communication tool, and adjusts the access content and editing permissions of the corresponding user to the knowledge graph and business knowledge base according to the user permission information; Retrieve respectively in the knowledge graph and business knowledge base according to the text information and user permission information, and generate corresponding retrieval results; Select a local large language model or an online large language model according to the question and answer requirements. The local large language model and the online large language model generate natural language question and answer results based on the retrieval results of the knowledge graph and business knowledge base, and send the question and answer results to the enterprise communication tool; After the enterprise communication tool receives and displays the question and answer results sent by the question and answer platform, it receives the feedback data sent by the user and sends the feedback data to the question and answer platform; The feedback module updates and optimizes the knowledge graph and business knowledge base according to the feedback data.

7. The customized question-answering method based on a large language model according to claim 6, wherein The enterprise communication tool is DingTalk, WeCom, Zhizhi, J2L3x or Feishu.

8. The customized question-answering method based on a large language model according to claim 6, wherein The multimodal data processing module converts the multimodal question data into corresponding text information, including: The image recognition unit converts the image data in the multimodal question data into corresponding text information based on the OCR technology, and the speech recognition unit converts the speech data in the multimodal question data into corresponding text information based on the ASR model.

9. The customized Q&A method based on a large language model according to claim 6, wherein The enterprise element data includes entity nodes and entity relationships. The entity nodes at least include the products, departments, documents and technical keywords of the enterprise, and the entity relationships are obtained by analyzing the internal structure, business process, product hierarchy and customer information of the enterprise.

10. The customized question-answering method based on a large language model according to claim 6, characterized in that, The method further includes: The feedback module records the corresponding question and answer time and question and answer text of the user, statistically analyzes the records, optimizes the corresponding entries in the business knowledge base according to the statistical results, and adjusts the weights of the corresponding entities and entity relationships in the knowledge graph according to the statistical results.

Images