Intelligent question answering method, device and equipment based on combination of large model and knowledge graph

Abstract

The invention relates to the technical field of intelligent questioning and answering, in particular to an intelligent questioning and answering method, device and equipment based on combination of a large model and a knowledge graph. The method comprises the steps that user input is received, and whether the user input is a document question and answer is judged; if the user input is the document question and answer, calling an RAG module and an LLM module to judge whether the user input is related to the uploaded document; if the user inputs a non-document question and answer, or the user input is unrelated to the uploaded document, calling an LLM module to judge whether the user input is related to the legal field; if the user input is related to the legal field, calling a KBQA module and an LLM module to generate a retrieval result input by the user in a preset graph database of the legal field; and based on the retrieval result and the user input, calling the LLM module to generate a reply result input by the user. According to the method and the device, accurate reply aiming at user input can be realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent question answering technology, and in particular to an intelligent question answering method, device and equipment based on the combination of a large model and a knowledge graph. Background Art

[0002] In my country, with the continuous advancement of the rule of law, people's legal awareness is increasing. In traditional consultation, people consult legal professionals, who often need to face a large amount of repetitive knowledge consultation, which not only leads to low work efficiency of lawyers and other legal workers, but also makes users spend a lot of time waiting for answers. At the same time, due to the uneven professional level of legal practitioners, the answers given by different lawyers may be different, and it is difficult for users to distinguish the authenticity, which undoubtedly increases legal risks. With the development of the Internet, more and more people are beginning to seek legal help through the Internet.

[0003] KG-RAG is a typical method that combines knowledge graphs with retrieval-augmented generation (RAG) technology. This method converts nodes and relationships in the knowledge graph into vectorized representations, and then uses semantic retrieval technology to locate the nodes and relationship information most relevant to the question from the knowledge graph, and uses these retrieval results as input conditions for the generation model to generate answers. This combination effectively utilizes the structured information of the knowledge graph, provides a richer semantic background for the generation process, and greatly improves the accuracy and logical consistency of the question-answering system. However, because the embedded representation of the knowledge graph is usually generated statically, the system cannot update the dynamically changing knowledge base content in real time, which makes it less adaptable in fast-updating scenarios. In addition, KG-RAG has limited multi-hop reasoning capabilities, and its performance is insufficient when faced with complex semantic associations or problems that require deep logical reasoning, which affects its application effect in high-complexity question-answering scenarios.

[0004] GraphRAG directly retrieves the subgraph structure related to the question from the knowledge graph, including related entities and the semantic relationships between them, and then embeds these subgraph information into the generation model to generate answers. GraphRAG highlights the structured characteristics of the knowledge graph and can use the graph structure to complete a certain degree of complex semantic reasoning and logical relationship analysis, making it suitable for scenarios that require multi-hop reasoning. However, this method faces the problem of low retrieval efficiency when dealing with large-scale knowledge graphs, especially when the knowledge graph is large and the query demand is frequent, the performance bottleneck is particularly obvious. The response speed is slow, and GraphRAG has weak support for dynamic expansion and limited ability to quickly integrate new entities or relationships into the knowledge graph. In addition, the generated answers still have certain deficiencies in semantic consistency and logical coherence, which affects its actual application effect on complex problems.

[0005] It can be seen that when dealing with legal issues, the question-answering systems or search engines of related technologies can often only provide keyword-based search results rather than accurate legal answers. It is not only time-consuming and laborious for users to search for answers among thousands of search results, but it is also difficult to ensure the accuracy and authority of the information obtained. In addition, many legal issues are somewhat professional, and it is difficult for ordinary users to judge the correctness of the answers. Summary of the invention

[0006] In order to solve the problem that the answers obtained by consulting legal issues on the Internet in the prior art are not accurate, the present application provides an intelligent question-answering method, device and equipment based on the combination of a large model and a knowledge graph.

[0007] In the first aspect, the present application provides an intelligent question-answering method based on a combination of a large model and a knowledge graph, which adopts the following technical solutions: An intelligent question-answering method based on a combination of a large model and a knowledge graph, comprising: Receive user input and determine whether the user input is a document question and answer; If the user input is a document question and answer, the RAG module and the LLM module are called to determine whether the user input is related to the uploaded document; If the user input is not a document question or answer, or the user input is not related to the uploaded document, the LLM module is called to determine whether the user input is related to the legal field; If the user input is related to the legal field, calling the KBQA module and the LLM module to generate a search result of the user input in the preset graph database of the legal field; Based on the search result and the user input, the LLM module is called to generate a reply result of the user input.

[0008] By adopting the above technical solution, using modular design and multi-level judgment mechanism, it is determined whether the user input is a document question and answer, and the RAG module and LLM module are dynamically called to process document-related queries to ensure efficient use of document content; if the input is not related to the document, it is further determined whether it involves the legal field, and the KBQA module and LLM module are called to generate accurate search results in combination with the legal knowledge graph; based on the search results and user input, the LLM module generates natural language responses. This hierarchical processing mechanism not only improves the pertinence of questions and answers, but also realizes intelligent question and answer in multiple scenarios by combining the capabilities of documents, legal knowledge graphs and large language models, enhancing the practicality and user experience of the system, so that accurate responses to user inputs are finally obtained.

[0009] In a preferred example, the present application may be further configured as follows: the determining whether the user input is a document question and answer includes: Determine whether the user input and the uploaded document are received simultaneously in the current round; If the user input and the uploaded document are received at the same time, determining that the user input is a document question and answer; If the current round only receives the user input, it is determined whether the uploaded document was received in the previous round or the round before the previous round; if the uploaded document was received in the previous round or the round before the previous round, it is determined that the user input is a document question and answer.

[0010] By adopting the above technical solution and utilizing the dynamic judgment mechanism of multiple rounds of interaction, it is possible to effectively identify whether the user input is a document question and answer, thereby improving the flexibility and accuracy of the system. It not only supports users to flexibly upload documents and ask questions in multiple rounds of conversations, but also avoids misjudgment caused by the separation of document uploading and asking questions, thereby ensuring the continuity and efficiency of the document question and answer scenarios.

[0011] In a preferred example, the present application may be further configured as follows: the calling of the RAG module and the LLM module to determine whether the user input is related to the uploaded document includes: Using the RAG module to slice and vectorize the uploaded document to generate a vector database; Vectorizing the user input using the RAG module, and retrieving the information block set with the highest correlation with the vectorized user input in the vector database; sorting the information blocks in the information block set by using a re-sorting algorithm to obtain a correlation retrieval result; The relevance search result and the user input are combined into a prompt word, the user input, the uploaded document and the prompt word are input into the LLM module, and the LLM module outputs a determination result as to whether the user input is related to the uploaded document; wherein, if the user input is related to the uploaded document, the LLM module simultaneously outputs a reply result for the user input.

[0012] By adopting the above technical solution and utilizing the collaborative work of the RAG module and the LLM module, it is possible to efficiently determine whether the user input is relevant to the uploaded document and generate accurate replies. This not only enables the rapid location of content in the document that is related to the user input, but also significantly improves the accuracy of judgment and replies through reordering and the semantic understanding capabilities of LLM, thus achieving efficient and intelligent processing of document questions and answers.

[0013] In a preferred example, the present application may be further configured as follows: the calling of the KBQA module and the LLM module to generate the search result of the user input in the preset graph database in the legal field includes: Calling the LLM module to perform label recognition on the user input to obtain an intent label and an entity label; Calling the KBQA module to link the entity tag with the entity in the preset entity knowledge base to obtain a standard entity representation of each entity tag in the preset entity knowledge base; The search result is obtained by searching the preset graph database based on the intention label and the standard entity expression.

[0014] By adopting the above technical solutions and utilizing the collaborative work of the KBQA module and the LLM module, efficient retrieval and accurate response to legal questions and answers are achieved. The LLM module identifies the intent and entity labels of user input to clarify user needs; the KBQA module links the entity labels with the standard entities in the preset entity knowledge base to ensure the accuracy and consistency of the entity representation; based on the intent labels and standard entity representations, relevant results are retrieved in the graph database, which can not only accurately understand the user's intent, but also quickly locate relevant information in the legal field through the combination of the knowledge base and the graph database.

[0015] In a preferred example, the present application may be further configured as follows: the search result obtained by searching the preset graph database based on the intention label and the standard entity expression includes: Retrieving a preset query statement template corresponding to each intent tag and each standard entity expression, wherein the preset query statement template includes an entity slot; The called preset query statement templates are spliced ​​to obtain a spliced ​​query statement; Filling the intent label and the standard entity expression into the entity slot of the concatenated query statement to form a graph database query statement; The graph database query statement is passed into the preset graph database to obtain the search result.

[0016] By adopting the above technical solution, the corresponding preset query template is called according to the intent label and standard entity expression to ensure the structuring of the query statement; multiple templates are spliced ​​and the entity slots are filled to generate a complete graph database query statement; the query statement is passed into the graph database for retrieval, which can not only flexibly adapt to different intents and entity combinations, but also ensure the accuracy and completeness of the query statement through the preset template and entity filling mechanism, thereby significantly improving the retrieval efficiency and quality.

[0017] In a preferred example, the present application may be further configured as follows: calling the LLM module to perform label recognition on the user input to obtain an intent label and an entity label, including: Using a prompt word tool to construct a prompt word instruction, the prompt word instruction including: examples of generating intent labels and entity labels based on user input samples and task instructions; The user input and the prompt word instruction are input into the LLM module, and the intention label and entity label of the user input output by the LLM module are received.

[0018] By adopting the above technical solution, the prompt word tool is used to construct prompt word instructions containing examples and task instructions, and the rules and format of label recognition are clarified; user input and prompt word instructions are input into the LLM module to guide the model to accurately understand the task and generate intent labels and entity labels. It can not only quickly adapt to different scenarios through a few sample examples, but also improve the accuracy and consistency of label recognition through structured instructions, significantly enhancing the system's semantic understanding ability.

[0019] In a preferred example, the present application can be further configured as follows: the method further includes: Build a preset entity knowledge base; The preset entity knowledge base includes multiple categories of entities, each of which corresponds to intent labels. Each category of entity includes multiple attributes, and each of which corresponds to entity labels.

[0020] By adopting the above technical solution, a preset entity knowledge base is constructed, which provides structured domain knowledge support for the system. It can help the system quickly identify the intentions and entities in user input, and accurately locate relevant information in the knowledge base through the association between entities and attributes.

[0021] In a preferred example, the present application can be further configured as follows: the method further includes: If the user input is not related to the legal field, the LLM module is called to generate a response result of the user input.

[0022] By adopting the above technical solution, when the user input is not related to the legal field, the system directly calls the LLM module to generate a response, using the powerful semantic understanding and generation capabilities of the large language model to provide accurate and natural answers. This not only avoids question-and-answer failures caused by field restrictions, but also covers a wider range of user needs through the general knowledge base of LLM.

[0023] In the second aspect, the present application provides an intelligent question-answering device based on a combination of a large model and a knowledge graph, which adopts the following technical solutions: An intelligent question-answering device based on a combination of a large model and a knowledge graph, comprising: An input receiving module, used to receive user input and determine whether the user input is a document question and answer; A first calling module, for calling the RAG module and the LLM module to determine whether the user input is related to the uploaded document if the user input is a document question and answer; A second calling module is used to call the LLM module to determine whether the user input is related to the legal field if the user input is not a document question or the user input is not related to the uploaded document; A retrieval module, configured to call the KBQA module and the LLM module to generate a retrieval result of the user input in a preset graph database in the legal field if the user input is related to the legal field; A reply module is used to call the LLM module to generate a reply result of the user input based on the search result and the user input.

[0024] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: one or more processors; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the intelligent question answering method based on the combination of a large model and a knowledge graph as described in any one of the first aspects.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the intelligent question-answering method based on the combination of a large model and a knowledge graph as described in any one of the first aspects.

[0026] In a fifth aspect, the present application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program. When the computer program is executed by a processor, it implements the intelligent question answering method based on the combination of a large model and a knowledge graph as described in any one of the first aspects.

[0027] In summary, this application includes the following beneficial technical effects: This application uses modular design and multi-level judgment mechanism to determine whether the user input is a document question and answer, and dynamically calls the RAG module and LLM module to process document-related queries to ensure efficient use of document content; if the input is not related to the document, it further determines whether it involves the legal field, calls the KBQA module and LLM module to combine the legal knowledge graph to generate accurate search results; based on the search results and user input, the LLM module generates natural language responses. This hierarchical processing mechanism not only improves the pertinence of questions and answers, but also realizes intelligent question and answer in multiple scenarios by combining the capabilities of documents, legal knowledge graphs and large language models, enhancing the practicality and user experience of the system, so that accurate responses to user input are finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a system architecture diagram provided by an embodiment of the present application; Figure 2 It is a module composition diagram provided in the embodiment of the present application; Figure 3 It is a flowchart of an intelligent question-answering method based on a combination of a large model and a knowledge graph provided in an embodiment of the present application; Figure 4 It is a flowchart of an intelligent question-answering method based on a combination of a large model and a knowledge graph provided in another embodiment of the present application; Figure 5 is an example of the content of the preset entity knowledge base provided in the embodiment of the present application; Figure 6 It is a structural diagram of an intelligent question-answering device based on a combination of a large model and a knowledge graph provided in an embodiment of the present application; Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following is combined with Figure 1 -Attached Figure 7 This application is described in further detail.

[0030] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0033] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0034] The intelligent question answering method based on the combination of a large model and a knowledge graph provided in the embodiment of the present application can be applied to an intelligent question answering system based on the combination of a large model and a knowledge graph, such as Figure 1 As shown, it shows a system architecture diagram provided in an embodiment of the present application, the system includes: a basic layer, a data layer, a business layer and an application layer.

[0035] The basic layer provides computing resources, storage resources, network resources and security resources. Computing resources provide powerful computing power to support the various computing needs of the system, including GPU, CPU, cloud server, etc. Storage resources include sufficient storage space to store a large amount of industry knowledge base data, document data, user data and model files, etc. Network resources include stable network connections to ensure real-time transmission and processing of data. Security resources take a variety of security measures to protect the security of the system and data, and prevent unauthorized access and data leakage.

[0036] The data layer is mainly responsible for cleaning, converting and extracting knowledge from raw data. It converts unstructured text data such as judicial documents, typical cases, and case source reports into structured data and stores them in the graph database to form a legal industry knowledge base. It stores documents uploaded by users in the document knowledge base and uses text embedding and other technologies to vectorize the text and store it in the vector database for subsequent knowledge retrieval and question-and-answer applications.

[0037] The business layer is the central nervous system of the system, responsible for coordinating the interaction and data flow between various sub-modules. It calls the corresponding service interface to obtain the required information resources according to the user's query intention.

[0038] The application layer is the interface for users to interact with the system. The main functions include: document upload, document Q&A, knowledge base Q&A and intelligent Q&A. Document upload is responsible for receiving various documents uploaded by users, such as judgment documents, typical cases, etc. Document Q&A provides users with Q&A services based on document content, helping users to quickly obtain important information in documents, saving time and improving efficiency for users. Knowledge base Q&A answers users' questions based on the information in the legal industry knowledge base, providing legal workers with efficient and professional consulting services and case handling basis, and improving case handling efficiency. Intelligent Q&A uses the powerful intent understanding and emergence capabilities of large models to provide accurate answers to various knowledge in general fields and improve user experience.

[0039] The embodiment of the present application provides an intelligent question answering method based on a combination of a large model and a knowledge graph, which is based on three modules, including: LLM (Large Language Model) module, RAG (Retrieval-Augmented Generation) module and KBQA (Knowledge Base Question Answering) module. Figure 2 , which shows the module composition diagram provided in the embodiment of the present application.

[0040] The LLM module refers to a model that uses deep learning algorithms and is trained through massive amounts of text, can recognize human language, perform language tasks, and has a large number of parameters. LLM compresses training text information and has generalization capabilities. Unlike databases and search engines, LLM can creatively generate text content that has never appeared in history. The LLM module is the intelligent core of the entire system. It can simulate the human thinking process and continuously improve its own understanding and expression capabilities by learning a large amount of text corpus.

[0041] The function of the LLM module is mainly used to understand and process the user's natural language input.

[0042] The working principle of the LLM module includes: using deep neural networks, especially the Transformer architecture, to capture the context and semantics of language. It acquires language understanding ability through unsupervised learning on a large-scale corpus and generates response content based on probability. This embodiment can adopt Zhipu Qingyan's GLM-4-9B-chat model and use legal field data such as judicial documents for fine-tuning to enhance the large model's ability to understand and make decisions on legal field knowledge.

[0043] The processing flow of the LLM module includes: receiving natural language input, analyzing and understanding questions, and generating responses based on the semantics and context of the questions.

[0044] The main responsibility of the RAG module is to capture document fragments related to the current problem from the (big model) external database and provide them to the big model for reference to generate a well-founded response. RAG technology is currently an important direction for the implementation of LLM applications. The main application scenarios are enterprise customer service systems and structured display of search results, providing users with explainable evidence.

[0045] The functions of the RAG module include: combining retrieval technology and LLM models, using information in external document libraries to enhance the quality of generative LLM answers, and make up for the timeliness, knowledge limitations, and illusions of LLM. Retrieval refers to searching external knowledge bases, and enhanced generation refers to sending the retrieved knowledge to the large language model to optimize the generation results of the large model, so that the large model can generate more accurate and more context-appropriate answers while effectively reducing the possibility of generating misleading information.

[0046] The working principle of the RAG module includes: finding paragraphs related to user questions from the document library through retrieval technology, and then providing these paragraphs to the LLM to generate more accurate and rich answers.

[0047] The processing flow of the RAG module includes: data storage, relevance retrieval, and LLM output generation. Data storage divides the documents uploaded by users into text blocks, uses embedding technology to vectorize and encode the text blocks, and stores them in the vector database. Relevance retrieval also vectorizes the questions raised by users, performs similarity retrieval in the vector database, and obtains N information blocks related to the user's questions. LLM output generation includes combining the retrieved information blocks with user input into prompt word instructions and inputting them into LLM, allowing LLM to output answers or execute user instructions.

[0048] The KBQA module focuses on obtaining precise factual answers directly from a structured domain knowledge base and is suitable for simple questions that can be directly mapped to existing knowledge.

[0049] The function of the KBQA module includes extracting information from a structured legal domain knowledge base to answer questions. It is suitable for questions that can be answered clearly by querying the knowledge base.

[0050] The working principle of the KBQA module includes: using intent recognition, entity recognition and other technologies to convert the user's natural language questions into query statements that can be executed on the knowledge base, and then executing the query and parsing the results to generate answers. Traditional intent recognition and entity recognition require a large amount of question corpus, and the intent and entity are labeled separately, and multiple intent recognition models and entity recognition models are trained using deep learning algorithms. This system uses the powerful understanding ability of LLM, and through the prompt word engineering, LLM performs intent recognition and entity recognition, which greatly saves the labeling cost while ensuring quality.

[0051] The processing flow of the KBQA module includes: building an entity knowledge base, intent recognition and entity recognition, entity linking, slot filling, graph database retrieval, and LLM response generation.

[0052] The present application embodiment provides an intelligent question-answering method based on a combination of a large model and a knowledge graph, such as Figure 3 As shown, the method provided in the embodiment of the present application is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. The method includes steps S301-S305, wherein: S301, receiving user input, and determining whether the user input is a document question and answer.

[0053] See also Figure 4 , which shows a flow chart of an intelligent question-and-answer method based on a combination of a large model and a knowledge graph provided by another embodiment of the present application. Each time a user uploads content, it is regarded as a round of query, and the content input by the user in the current round is received, and it is determined whether the current round receives both the user input (the user's query question query) and the uploaded document. If the current round receives both the user input and the uploaded document, it is determined that the user input is a document question and answer. If the current round only receives user input, it is determined whether the previous round was a document question and answer. If the previous round was a document question and answer, it is determined that the user input is a document question and answer. If the user has not uploaded a document, the current round only receives user input, and the previous round was not a document question and answer, it is determined that the user input is not a document question and answer.

[0054] S302: If the user input is a document question and answer, the RAG module and the LLM module are called to determine whether the user input is related to the uploaded document.

[0055] When the user input is a document question and answer, it enters the RAG module, which cuts the uploaded document into blocks and vectorizes it, and stores the obtained vectorized information blocks as a vector database; it also vectorizes the user input, and performs relevance search on the vectorized user input in the vector database, and obtains the N information blocks most relevant to the user input as the relevance search results.

[0056] The relevance search results are combined into prompt words, and the user input, uploaded document and prompt words are input into the LLM module. The LLM module determines whether the user input is related to the uploaded document based on the received content.

[0057] S303. If the user input is not a document question or the user input is not related to the uploaded document, the LLM module is called to determine whether the user input is related to the legal field.

[0058] If the LLM module determines that the user input is related to the uploaded document, it will further generate a reply result corresponding to the user input.

[0059] If the LLM module determines that the user input is not related to the uploaded document, or the user input is a non-document question, the LLM module is called to determine whether the user input is related to the content in the legal field through a pre-set programming procedure.

[0060] If the user input is not related to the legal field, the LLM module intelligently generates a response result corresponding to the user input based on knowledge and understanding ability.

[0061] S304: If the user input is related to the legal field, the KBQA module and the LLM module are called to generate a search result of the user input in the preset graph database of the legal field.

[0062] If the user input is related to the legal field, call the LLM module to identify the label of the user input and obtain the intent label and entity label. Then, call the KBQA module to link the entity label with the entity in the preset entity knowledge base, call the preset query statement template through the intent label and entity link result, and splice to obtain the spliced ​​query statement, fill the intent label and entity link result into the entity slot in the spliced ​​query statement to obtain the graph database query statement. Convert unstructured text data such as judicial documents, typical cases, and case source reports into structured data and store them in the graph database to form a legal industry knowledge base, namely the preset graph database. Retrieve the search results corresponding to the graph database query statement in the preset graph database.

[0063] S305: Based on the search results and user input, call the LLM module to generate a reply result of the user input.

[0064] Specifically, the search results are used as background knowledge, and the search results, user input and execution instructions are combined into prompt words and input into the LLM module, and the LLM module outputs the reply result.

[0065] This embodiment significantly improves the accuracy and flexibility of the intelligent question-answering system through modular design and multi-level judgment mechanism. First, the system determines whether the user input is a document question and answer, and dynamically calls the RAG module and the LLM module to process document-related queries to ensure efficient use of the document content; if the input is not related to the document, it further determines whether it involves the legal field, calls the KBQA module and the LLM module to generate accurate retrieval results in combination with the legal knowledge graph; finally, based on the retrieval results and user input, the LLM module generates natural language replies. This hierarchical processing mechanism not only improves the pertinence of questions and answers, but also realizes intelligent question and answer in multiple scenarios by combining the capabilities of documents, legal knowledge graphs and large language models, enhancing the practicality and user experience of the system, so that accurate replies to user inputs are finally obtained.

[0066] The KBQA module combines the knowledge graph with the large model (LLM). When constructing the knowledge graph, the fine-tuned LLM is used to extract knowledge from unstructured text data such as judicial documents, extract complex entities and relationships, and form a preset graph database, which greatly saves annotation and training costs. In the question-answering stage, LLM is used to simultaneously perform intent recognition and entity recognition. Compared with the traditional KBQA model that requires training intent recognition models and entity recognition models separately, it saves a lot of annotation and training costs and solves the problems of difficulty in collecting training corpus, time-consuming and labor-intensive problems. Compared with KG-RAG in the existing technical solution, LLM is used to perform entity recognition from user queries and then find the corresponding nodes in the knowledge graph. The understanding of intent and the positioning accuracy of the knowledge graph subgraph will be slightly worse, and questions outside the domain knowledge base cannot be answered. Through modular design and intelligent update mechanism, manual intervention and development costs are reduced, and the overall resource utilization efficiency of the system is improved.

[0067] A possible implementation of the embodiment of the present application is to determine whether the user input is a document question and answer, including: Determine whether the current round receives both user input and uploaded documents; If both user input and uploaded documents are received at the same time, the user input is determined to be a document question and answer; If only user input is received in the current round, it is determined whether an uploaded document was received in the previous round or the round before the previous round. If an uploaded document was received in the previous round or the round before the previous round, it is determined that the user input is a document question and answer.

[0068] Specifically, after a user uploads a document, multiple rounds of questions may be asked about the document content, but the document will not be uploaded in every round of questions. Therefore, when it is determined that only user input is received in the current round, it is further determined whether the uploaded document was received in the previous round or the round before the previous round.

[0069] The embodiment of the present application utilizes a dynamic judgment mechanism of multiple rounds of interaction to effectively identify whether the user input is a document question and answer, thereby improving the flexibility and accuracy of the system. It not only supports users to flexibly upload documents and ask questions in multiple rounds of conversations, but also avoids misjudgment caused by the separation of document uploading and asking questions, thereby ensuring the continuity and efficiency of the document question and answer scenario.

[0070] A possible implementation of the embodiment of the present application is to call the RAG module and the LLM module to determine whether the user input is related to the uploaded document, including: Use the RAG module to slice and vectorize the uploaded document to generate a vector database; The RAG module is used to vectorize the user input, and the information block set with the highest correlation with the vectorized user input is retrieved from the vector database; Using a reordering algorithm to sort the information blocks in the information block set, and obtaining a correlation retrieval result; The relevance search results and user input are combined into a prompt word, and the user input, uploaded document and prompt word are input into the LLM module to obtain the judgment result of whether the user input is related to the uploaded document output by the LLM module; among them, if the user input is related to the uploaded document, the LLM module also outputs a reply result for the user input.

[0071] In this embodiment, the uploaded document is sliced ​​and vectorized using the RAG module to obtain multiple information blocks in vector form, and the obtained multiple information blocks are stored as a vector database. Vectorization is to map the text into vectors so that semantically similar words are close in distance in the vector space, thereby capturing the similarity and relationship between objects.

[0072] After obtaining the vector database and the vectorized user input, the similarity between the user input and each information block is obtained by using semantic search and keyword search in the vector database. For any information block, the similarity between the user input obtained by semantic search and the information block is used as the first similarity, and the similarity between the user input obtained by keyword search and the information block is used as the second similarity. The first similarity and the second similarity are weighted and summed according to the pre-set weights of the semantic search and keyword search, and the result is used as the similarity between the information block and the user input. Referring to the above method, the similarity between each information block in the vector database and the user input is determined.

[0073] The N information blocks with the highest similarity to the user input are determined from the vector database as the information block set, and the number of N can be set in advance according to actual needs, and is not limited in this embodiment. Then, the obtained N information blocks are processed using a reordering algorithm, and the reordered N information blocks are output as the relevance search result.

[0074] The embodiment of the present application utilizes the collaborative work of the RAG module and the LLM module to efficiently determine whether the user input is related to the uploaded document and generate accurate replies. It can not only quickly locate the content related to the user input in the document, but also significantly improve the accuracy of judgment and reply through reordering and the semantic understanding ability of LLM, thereby realizing efficient and intelligent processing of document questions and answers.

[0075] A possible implementation of the embodiment of the present application is to call the KBQA module and the LLM module to generate a search result of the user input in the preset graph database in the legal field, including: Call the LLM module to perform label recognition on user input to obtain intent labels and entity labels; Call the KBQA module to link the entity label with the entity in the preset entity knowledge base to obtain the standard entity representation of each entity label in the preset entity knowledge base; The search results are obtained by searching in the preset graph database based on the intent labels and standard entity expressions.

[0076] In this embodiment, an entity knowledge base is pre-built as a preset entity knowledge base, and the fine-tuned LLM is used to structure text data such as judicial documents and typical cases to form seven categories of entities, each of which has corresponding attributes. Among them, the intent label is determined according to the structure of the preset entity knowledge base, the seven categories of entities are used as intent labels, and the attributes under each category of entities are used as entity labels. Figure 5 An example of the content of the preset entity knowledge base provided in the embodiment of the present application is shown, and the seven categories of entities include: typical cases, reports, judicial documents, legal organs, legal personnel, persons involved in the case, and laws and regulations.

[0077] For each extracted entity tag, similarity matching is performed with the preset entity knowledge base, and the entity that best matches the entity tag is found from the preset entity knowledge base as the standard entity representation of the entity tag, thereby avoiding different expressions of the same thing affecting the search results. For example, the entity tag "Beijing" in the user input question "Typical cases in Beijing" has a standard entity representation of "Beijing" in the preset entity knowledge base.

[0078] The embodiment of the present application utilizes the collaborative work of the KBQA module and the LLM module to achieve efficient retrieval and accurate response to questions and answers in the legal field. The LLM module identifies the intent and entity labels of user input to clarify user needs; the KBQA module links the entity labels with the standard entities in the preset entity knowledge base to ensure the accuracy and consistency of the entity representation; based on the intent labels and standard entity representations, relevant results are retrieved in the graph database, which can not only accurately understand the user's intent, but also quickly locate relevant information in the legal field through the combination of the knowledge base and the graph database.

[0079] A possible implementation of the embodiment of the present application is to obtain a search result by searching in a preset graph database based on the intent label and the standard entity representation, including: Retrieving a preset query statement template corresponding to each intent tag and each standard entity expression, wherein the preset query statement template includes an entity slot; The called preset query statement templates are spliced ​​to obtain a spliced ​​query statement; Fill the intent label and standard entity representation into the entity slot of the concatenated query statement to form a graph database query statement; Pass the graph database query statement into the preset graph database to obtain the retrieval results.

[0080] In this embodiment, a preset query statement template is defined for each intent tag and entity tag in advance, and the preset query statement template includes an entity slot, which is used to fill in a specific entity value. The called preset query statement template is automatically spliced ​​into a spliced ​​query statement in a logical order, and then the entity slot is filled to obtain a completed graph database query statement.

[0081] Call the retrieval interface of the preset graph database, pass in the graph database query statement, and retrieve matching results in the preset graph database.

[0082] The embodiment of the present application retrieves the corresponding preset query template according to the intent label and the standard entity expression to ensure the structuring of the query statement; splices multiple templates and fills the entity slots to generate a complete graph database query statement; and passes the query statement into the graph database for retrieval, which can not only flexibly adapt to different intents and entity combinations, but also ensure the accuracy and completeness of the query statement through the preset template and entity filling mechanism, thereby significantly improving the retrieval efficiency and quality.

[0083] A possible implementation of the embodiment of the present application is to call the LLM module to perform label recognition on the user input to obtain the intent label and the entity label, including: Use the prompt word tool to build prompt word instructions, which include: examples of generating intent labels and entity labels based on user input samples and task instructions; The user input and prompt word instructions are input into the LLM module, and the intention label and entity label of the user input output by the LLM module are received.

[0084] Specifically, a prompt word instruction prompt is designed using the few-sample prompt and thought chain technology. Examples and corresponding task instructions are provided in the prompt. The examples show how to extract intent labels and entity labels from user input. The examples include user input samples, intent labels, and entity labels. The user output and the designed prompt are input to the LLM module together. The LLM module analyzes the user input according to the examples and instructions in the prompt, and identifies the intent labels and entity labels from them. The LLM module returns the identified intent labels and entity labels in JSON format.

[0085] The embodiment of the present application utilizes a prompt word tool to construct prompt word instructions including examples and task instructions, and clarifies the rules and format of label recognition; user input and prompt word instructions are input into the LLM module to guide the model to accurately understand the task and generate intent labels and entity labels. It can not only quickly adapt to different scenarios through a small number of sample examples, but also improve the accuracy and consistency of label recognition through structured instructions, significantly enhancing the system's semantic understanding ability.

[0086] In a possible implementation manner of the embodiment of the present application, the method further includes: Build a preset entity knowledge base; Among them, the preset entity knowledge base includes multiple categories of entities, the multiple categories of entities correspond to intent labels, each category of entities includes multiple attributes, and the multiple attributes correspond to entity labels.

[0087] The embodiment of the present application provides structured domain knowledge support for the system by constructing a preset entity knowledge base, which can help the system quickly identify the intentions and entities in the user input, and can also accurately locate relevant information in the knowledge base through the association between entities and attributes.

[0088] In a possible implementation manner of the embodiment of the present application, the method further includes: If the user input is not related to the legal field, the LLM module is called to generate a response result of the user input.

[0089] Specifically, when the user input is not related to the legal field, the LLM module conducts intelligent question and answer based on its own knowledge and understanding ability and outputs the response result.

[0090] In the embodiment of the present application, when the user input is not related to the legal field, the system directly calls the LLM module to generate a reply, using the powerful semantic understanding and generation capabilities of the large language model to provide accurate and natural answers, which not only avoids question and answer failures caused by field restrictions, but also covers a wider range of user needs through the general knowledge base of LLM.

[0091] The embodiment of this application innovatively combines knowledge graph and retrieval-augmented generation (RAG) technology to improve the system's ability to understand complex semantics by building structured knowledge representation and efficient retrieval mechanisms, and achieves the accuracy and fluency of natural language generation through generative models. The core is to use the legal field knowledge graph for semantic expansion and deep association reasoning, and through the collaborative work of retrieval and generation, the question-and-answer system can cope with legal industry scenarios. At the same time, it can build a temporary knowledge base for documents in any field uploaded by users to conduct document question-and-answer and general knowledge question-and-answer, which is more flexible and improves the user experience.

[0092] In the existing technical solutions, Graph RAG changes the document segmentation part in the traditional RAG to structure the document content into triples and construct a graph. There is no relatively professional vertical knowledge graph. The graph must be rebuilt every time according to the uploaded documents, and the retrieval and response efficiency is very low. The present invention is based on the construction of a domain knowledge base, and utilizes the structured characteristics of the knowledge graph and an efficient retrieval algorithm to significantly shorten the system response time, meet the large-scale knowledge base question and answer requirements, and is suitable for high real-time scenarios.

[0093] In the KBQA module of the embodiment of the present application, the slot filling method is transformed from designing a complete set of query templates for each type of intent to pre-setting template fragments for each type of intent and entity based on the characteristics of the preset entity knowledge base ontology structure. When the LLM recognizes the user's intent and entity, it is directly generated by splicing and filling, which greatly saves the amount of code and computing resources and avoids the reuse of template fragments.

[0094] The preset graph database combined with RAG does not simply structure documents into a knowledge graph as an external knowledge base, but has its own legal domain knowledge base, and has the ability to upload documents to expand external knowledge for question and answer. It combines the knowledge graph domain knowledge, instantly uploaded document knowledge, and LLM general knowledge question and answer capabilities, significantly improving the accuracy of question and answer and user experience. The legal domain knowledge graph effectively avoids the problem of false or irrelevant content in traditional generative LLM question and answer, and the answer content is more accurate and has a causal chain, and the source of the answer can be traced.

[0095] The embodiments of this application can enhance the professionalism and pertinence of the system, and provide in-depth customized support for the field of ecological public interest litigation. The system contains a wealth of legal knowledge and case libraries, enabling the system to provide more professional and specific legal advice. The large model fine-tuned with a large amount of judicial document data provides a deeper understanding of field-related issues.

[0096] In addition, the intelligent question-answering method based on the combination of a large model and a knowledge graph provided in the embodiment of the present application can also support the semantic understanding and optimization processing of scene data by constructing a domain knowledge graph and using a large language model, replacing the global optimization solution that fully relies on a large model. This solution uses the structured information of the knowledge graph to enhance semantic associations, and realizes the reasoning and generation of scene data through a large language model, which is suitable for optimization needs oriented to technical problems. Although it may be slightly inferior to the original solution in terms of deep reasoning ability and dynamic adaptability of scenes, it can still achieve the invention goal of optimizing the experimental scene by replacing some method steps and optimizing details, and provide a feasible technical path with lower resource requirements.

[0097] The above embodiment introduces an intelligent question-answering method based on the combination of a big model and a knowledge graph from the perspective of the method flow. The following embodiment introduces an intelligent question-answering device based on the combination of a big model and a knowledge graph from the perspective of a virtual module or a virtual unit. For details, please see the following embodiment.

[0098] The present application embodiment provides an intelligent question-answering device based on a combination of a large model and a knowledge graph, such as Figure 6 As shown, the device may include: An input receiving module 601 is used to receive user input and determine whether the user input is a document question and answer; The first calling module 602 is used to call the RAG module and the LLM module to determine whether the user input is related to the uploaded document if the user input is a document question and answer; The second calling module 603 is used to call the LLM module to determine whether the user input is related to the legal field if the user input is not a document question or the user input is not related to the uploaded document; A search module 604 is used to call the KBQA module and the LLM module to generate a search result of the user input in a preset graph database in the legal field if the user input is related to the legal field; The reply module 605 is used to call the LLM module to generate a reply result of the user input based on the search result and the user input.

[0099] An electronic device is provided in an embodiment of the present application, such as Figure 7 As shown, Figure 7 The electronic device 700 shown includes: a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, such as through a bus 702. Optionally, the electronic device 700 may also include a transceiver 704. It should be noted that in actual applications, the transceiver 704 is not limited to one, and the structure of the electronic device 700 does not constitute a limitation on the embodiments of the present application.

[0100] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0101] The bus 702 may include a path to transmit information between the above components. The bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 702 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.

[0102] The memory 703 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0103] The memory 703 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 701. The processor 701 is used to execute the application code stored in the memory 703 to implement the contents shown in the embodiment of the intelligent question-answering method based on the combination of a large model and a knowledge graph.

[0104] Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0105] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer can execute the contents shown in the embodiment of the aforementioned intelligent question-answering method based on the combination of a large model and a knowledge graph.

[0106] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0107] An embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the contents shown in the aforementioned embodiment of the intelligent question-answering method based on the combination of a large model and a knowledge graph are implemented.

[0108] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An intelligent question answering method based on a combination of a large model and a knowledge graph, characterized in that: include: Receive user input and determine whether the user input is a document question and answer; If the user input is a document question and answer, the RAG module and the LLM module are called to determine whether the user input is related to the uploaded document; If the user input is not a document question or answer, or the user input is not related to the uploaded document, the LLM module is called to determine whether the user input is related to the legal field; If the user input is related to the legal field, calling the KBQA module and the LLM module to generate a search result of the user input in the preset graph database of the legal field; Based on the search result and the user input, the LLM module is called to generate a reply result of the user input.

2. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 1 is characterized in that: The determining whether the user input is a document question and answer comprises: Determine whether the user input and the uploaded document are received simultaneously in the current round; If the user input and the uploaded document are received at the same time, determining that the user input is a document question and answer; If the current round only receives the user input, it is determined whether the uploaded document was received in the previous round or the round before the previous round; if the uploaded document was received in the previous round or the round before the previous round, it is determined that the user input is a document question and answer.

3. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 1 is characterized in that: The calling of the RAG module and the LLM module to determine whether the user input is related to the uploaded document includes: Using the RAG module to slice and vectorize the uploaded document to generate a vector database; Vectorizing the user input using the RAG module, and retrieving the information block set with the highest correlation with the vectorized user input in the vector database; sorting the information blocks in the information block set by using a re-sorting algorithm to obtain a correlation retrieval result; The relevance search result and the user input are combined into a prompt word, the user input, the uploaded document and the prompt word are input into the LLM module, and the LLM module outputs a determination result as to whether the user input is related to the uploaded document; wherein, if the user input is related to the uploaded document, the LLM module simultaneously outputs a reply result for the user input.

4. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 1 is characterized in that: The calling of the KBQA module and the LLM module to generate the search result of the user input in the preset graph database in the legal field includes: Calling the LLM module to perform label recognition on the user input to obtain an intent label and an entity label; Calling the KBQA module to link the entity tag with the entity in the preset entity knowledge base to obtain a standard entity representation of each entity tag in the preset entity knowledge base; The search result is obtained by searching the preset graph database based on the intention label and the standard entity expression.

5. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 4 is characterized in that: The searching in the preset graph database based on the intention label and the standard entity expression to obtain the search result includes: Retrieving a preset query statement template corresponding to each intent tag and each standard entity expression, wherein the preset query statement template includes an entity slot; The called preset query statement templates are spliced ​​to obtain a spliced ​​query statement; Filling the intent label and the standard entity expression into the entity slot of the concatenated query statement to form a graph database query statement; The graph database query statement is passed into the preset graph database to obtain the search result.

6. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 4 is characterized in that: The calling of the LLM module to perform label recognition on the user input to obtain an intent label and an entity label includes: Using a prompt word tool to construct a prompt word instruction, the prompt word instruction including: examples of generating intent labels and entity labels based on user input samples and task instructions; The user input and the prompt word instruction are input into the LLM module, and the intention label and entity label of the user input output by the LLM module are received.

7. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 1, characterized in that: The method further comprises: Build a preset entity knowledge base; The preset entity knowledge base includes multiple categories of entities, the multiple categories of entities correspond to intention labels, each category of entities includes multiple attributes, and the multiple attributes correspond to entity labels.

8. The intelligent question-answering method based on the combination of a large model and a knowledge graph according to claim 1, characterized in that: The method further comprises: If the user input is not related to the legal field, the LLM module is called to generate a response result of the user input.

9. An intelligent question-answering device based on a combination of a large model and a knowledge graph, characterized in that: include: An input receiving module, used to receive user input and determine whether the user input is a document question and answer; A first calling module, for calling the RAG module and the LLM module to determine whether the user input is related to the uploaded document if the user input is a document question and answer; A second calling module is used to call the LLM module to determine whether the user input is related to the legal field if the user input is not a document question or the user input is not related to the uploaded document; A retrieval module, configured to call the KBQA module and the LLM module to generate a retrieval result of the user input in a preset graph database in the legal field if the user input is related to the legal field; A reply module is used to call the LLM module to generate a reply result of the user input based on the search result and the user input.

10. An electronic device, characterized in that: include: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the intelligent question answering method based on the combination of a large model and a knowledge graph as described in any one of claims 1-8.

Images