MoE model-based legal knowledge graph question and answer method and system

Abstract

The invention discloses a legal knowledge graph question-answering method and system based on a MoE model, and the method comprises the steps: building a legal knowledge graph: collecting different legal knowledge, carrying out the extraction processing, forming the legal knowledge graph, and providing data support for a question-answering process; constructing a legal knowledge big language model based on the MoE model; the legal knowledge graph is called, the sub-questions are processed through a plurality of intelligent agents, a final result is generated in a cooperation mode, and the legal knowledge graph question-answering function is achieved. Through the knowledge graph and the architecture based on the MoE model, intelligent agents for processing different information are established, work division and cooperation are achieved, response is accurate, the efficiency of legal intelligent question answering is improved, and the illusion problem is reduced.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a legal knowledge graph question-answering method and system based on a MoE model. Background Art

[0002] Large language models have performed well in various NLP tasks (such as question answering, machine translation, and text generation), but they have also been criticized for a series of drawbacks, such as lack of flexibility, hallucinations, and poor interpretability. Knowledge graphs store a large number of facts in the form of triples. They are a structured and deterministic way of representing knowledge, with advantages such as specific domain knowledge, accuracy, and interpretability. In recent years, the paradigm of using knowledge graphs to enhance large language models has attracted increasing attention from researchers and practitioners. Knowledge graphs can not only be incorporated into the pre-training and reasoning stages of LLMs to provide external knowledge, but can also be used to analyze LLMs and provide interpretability. When large language models are applied to the legal field, they face greater challenges, such as the logic of knowledge reasoning, accuracy, and legal timeliness. In legal large models, whether in the pre-training stage or the reasoning stage, there are few related studies on incorporating knowledge graphs into large language model training or reasoning, and there are many problems. Application No. 202311787452.4 proposes a trademark law intelligent question-answering method that uses a knowledge graph to enhance the reasoning of a large language model, which combines the trademark law knowledge graph with the trademark law large language model to improve the logic, accuracy, and legal timeliness of the generated response content. However, in this patent, all questions are reasoned and analyzed using the same large language model. For a wider range of universal legal knowledge, how to effectively identify problems, extract key information, analyze and process different types of language structures, and generate highly accurate professional content in the legal field is a major challenge currently faced.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present application provides a legal knowledge graph question-answering method and system based on the MoE model, which aims to solve the problem of hallucination in large language models in the legal field in the prior art, low processing efficiency, and inability to adapt to intelligent question-answering of professional issues in the legal field, resulting in low credibility of the results.

[0005] This application is implemented through the following technical solutions: The first aspect of the present application is to provide a legal knowledge graph question answering method based on the MoE model, comprising the following steps: Constructing a legal knowledge graph: collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; Constructing a large language model: Integrate the MoE model into the large language model of legal knowledge, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form the large language model of legal knowledge based on the MoE model; Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

[0006] Specifically, the several intelligent agents include: a legal assistant intelligent agent, a legal search intelligent agent, a senior lawyer intelligent agent and a legal editing intelligent agent; the intelligent agents use a unified standardized communication language and exchange information by sending and receiving messages.

[0007] Specifically, the construction of the legal knowledge graph includes: Multi-source data integration: extracting legal knowledge from a variety of different legal data sources and unifying them into a primary knowledge graph; Natural language processing: Use NLP algorithms to clean, segment, identify entities, and extract relationships from the information in the primary knowledge graph to ensure the quality of the information; The language relationship between the entities is established, the rule engine is defined, manual annotation and verification are performed, and fine-tuning training is performed based on the MoE model to form the legal knowledge graph.

[0008] Specifically, the fine-tuning training includes: Based on existing data sets, optimize legal question-answer pairs through reverse data synthesis; The fine-tuning training is performed using a parameter efficient fine-tuning method.

[0009] Specifically, the construction of a large language model includes: Specifically, the MoE module consisting of a word segmenter and several sub-models is set to realize task allocation through feature extraction and feature processing, including: In the word segmenter, the legal reminder words are converted into a number of subwords through word embedding processing, and then the semantic similarity and contextual relationship between the subwords are extracted using a multi-head attention mechanism to form feature data; Based on the MoE framework, the feature data is evaluated based on a routing network to obtain the correlation between the feature data and the sub-model, and the task allocation is implemented according to the correlation.

[0010] Specifically, based on the MoE framework, the feature data is evaluated based on a routing network to obtain the correlation between the feature data and the sub-model, and the task allocation is implemented according to the correlation, including: Setting the routing network to a sparse model; On the feedforward network layer, a number of sub-models are correspondingly set; The routing network determines the correlation between the feature data and the sub-model, determines the processing order according to the correlation, and decides which feature data are sent to which sub-models; one feature data is sent to one or more sub-models, and each sub-model processes the feature data with the highest matching degree.

[0011] Specifically, the processing of legal issues and providing intelligent question-answering functions include: The legal assistant agent identifies and analyzes the legal questions raised by the user, asks the user questions based on the predefined entity set, supplements the nodes of the legal knowledge graph according to the user's answers, and sends the analysis results to the legal search agent; The legal search agent extracts and analyzes legal information from the database and sends it to the legal editing agent and the senior lawyer agent; The senior lawyer agent generates a legal opinion based on the legal information. The legal editing agent combines the legal opinion and the legal information, automatically generates consulting information according to a template, and sends it to the user, thereby completing the intelligent question-and-answer function.

[0012] The second aspect of the present application is to provide a legal knowledge graph question-answering system based on the MoE model, including: The knowledge graph module is used to integrate multi-source data of legal knowledge, extract and process it, form a legal knowledge graph, and provide data support for legal question-and-answer services; the knowledge graph module includes: a data integration unit, a data processing unit, and a legal knowledge graph construction unit; A model training module performs word embedding processing on the prompt words to extract feature data, and processes the feature data through a routing network allocation sub-model based on the MoE framework; The legal question-and-answer service module, based on a multi-agent protocol framework, decomposes the process, constructs a number of intelligent agents, calls the database of the legal knowledge graph, processes different legal question-and-answer services respectively, and generates result information to feed back to the user.

[0013] Specifically, the several intelligent agents include: a legal assistant intelligent agent, a legal search intelligent agent, a senior lawyer intelligent agent and a legal editing intelligent agent; the intelligent agents use a unified standardized communication language and exchange information by sending and receiving messages.

[0014] The third aspect of the present application is to provide a storage medium, which is a non-transitory computer-readable storage medium, storing a program of a legal knowledge graph question-answering method based on a MoE model, and when the program is executed by a processor, the following steps are implemented: Constructing a legal knowledge graph: collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; Constructing a large language model: Integrate the MoE model into the large language model of legal knowledge, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form the large language model of legal knowledge based on the MoE model; Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

[0015] Beneficial effects: Through high-quality knowledge graphs and MoE model-based architecture, intelligent agents are established to process different information, divide the work and collaborate, respond accurately, improve the efficiency of legal intelligent question answering, and reduce hallucination problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 The present invention is a flowchart of a method for solving the large language model hallucination problem according to an embodiment of the present application.

[0018] Figure 2 A flowchart for constructing a legal knowledge graph for an embodiment of the present application.

[0019] Figure 3 A structural diagram of the legal knowledge graph constructed for the embodiment of the present application.

[0020] Figure 4 This is a structural diagram of a large language model based on the MoE model of an embodiment of the present application.

[0021] Figure 5 This is a structural diagram of a multi-agent collaboration framework according to an embodiment of the present application.

[0022] Figure 6A structural diagram of a system for solving the large language model hallucination problem according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the implementation methods of this application to clearly and completely describe the technical solutions in the implementation methods of this application. Obviously, the described implementation methods are only part of the implementation methods of this application, not all of the implementation methods. Based on the implementation methods in this application, all other implementation methods obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] It should be understood that the terms used in this specification of the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. As used in the specification of the embodiments of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0026] Large language models have performed well in various NLP tasks (such as question answering, machine translation, and text generation), but they have also been criticized for a series of drawbacks, such as lack of flexibility, hallucinations, and poor interpretability. Knowledge graphs store a large number of facts in the form of triples. They are a structured and deterministic way of representing knowledge, with advantages such as specific domain knowledge, accuracy, and interpretability. In recent years, the paradigm of using knowledge graphs to enhance large language models has attracted increasing attention from researchers and practitioners. Knowledge graphs can not only be incorporated into the pre-training and reasoning stages of LLMs to provide external knowledge, but can also be used to analyze LLMs and provide interpretability. When large language models are applied to the legal field, they face greater challenges, such as the logic of knowledge reasoning, accuracy, and legal timeliness. In legal large models, whether in the pre-training stage or the reasoning stage, there are few related studies on incorporating knowledge graphs into large language model training or reasoning, and there are many problems.

[0027] A trademark law intelligent question-answering method with knowledge graph enhanced large language model reasoning, application number 202311787452.4, proposes to combine the trademark law knowledge graph with the trademark law large language model to improve the logic, accuracy and legal timeliness of the generated response content. However, in this patent, all questions are reasoned and analyzed using the same large language model. For the wider and more universal legal knowledge, how to effectively identify problems, extract key information, analyze and process different types of language structures, and generate highly accurate professional content in the legal field is a major challenge currently faced. Although this patent adds the combination of knowledge graphs compared to other existing legal large language models to ensure that the content generated each time for the same question remains semantically similar, thereby improving stability. However, when it is expanded to a wider range of legal issues with larger data volumes, more parameters, and more knowledge graphs involved, the patent cannot guarantee the accuracy of the results.

[0028] Mixture of Experts (MoE) is a machine learning architecture that processes different parts or aspects of input data by assigning tasks to multiple "expert" networks. Each expert is an independent sub-model, usually a neural network, and they work together to complete a specific task. The MoE architecture aims to improve the efficiency and performance of the model, especially in large language models. How to use the MoE model, combined with the legal knowledge graph, to solve the poor accuracy of the results and the prominent hallucination problem in legal knowledge intelligent question answering is a problem that this application needs to solve.

[0029] In view of this, the first aspect of this application proposes a legal knowledge graph question answering method based on the MoE model. Figure 1 As shown, it is a flowchart of a legal knowledge graph question-answering method based on the MoE model of the present application, which includes the following steps: S1. Constructing a legal knowledge graph: Collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; S2. Building a large language model: Integrate the MoE model into the legal knowledge large language model, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form the legal knowledge large language model based on the MoE model; S3. Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

[0030] As a powerful knowledge representation method, knowledge graph can play an important role in the field of intelligent question answering. Knowledge graph includes features such as entities, attributes and relationships. It models real concepts and their connections in the form of nodes (entities) and edges (relationships). It not only stores factual information, but also captures the complex relationships between entities through semantic networks, thus providing a structured way to understand a large amount of heterogeneous data. In this application, the legal knowledge graph is first constructed, such as Figure 2 As shown, step S1 includes the following steps: S11. Multi-source data integration: extracting the legal knowledge from a variety of different legal data sources and unifying them into a primary knowledge graph; S12, Natural Language Processing: Use NLP algorithm to clean, segment, identify entities and extract relations from the information in the primary knowledge graph to ensure the quality of the information; S13. Establish semantic relationships between the entities, define a rule engine, manually annotate and verify, and perform fine-tuning training based on the MoE model to form the legal knowledge graph.

[0031] like Figure 3 As shown in the figure, it is a schematic diagram of the construction of the legal knowledge graph. First, a variety of different legal data sources are integrated, such as regulations, cases, academic literature, etc., and legal service data can also be included. After integration, information is first extracted from them and unified into a coherent primary knowledge graph, and then preprocessed, including denoising and deleting duplicate values. After that, natural language processing (NLP) technology is used to clean, segment, identify entities and extract relationships from the information in the preprocessed primary knowledge graph, and artificial standards and verification are performed through the intervention of human experts to improve the accuracy and logic of the information, thereby improving the quality of the information. After that, fine-tuning training is performed to generate a legal knowledge graph and establish a connection with the question-answering service module and the legal service agent.

[0032] Furthermore, complex semantic relationships between entities are established, including but not limited to causal relationships, reference relationships, time sequence, etc.

[0033] Specifically, fine-tuning training includes: optimizing legal question-answer pairs through reverse data synthesis based on existing data sets; and using parameter-efficient fine-tuning methods to perform the fine-tuning training. Specifically, in the first step, based on existing data sets, including forensic document data sets and criminal law data sets, the method of optimizing legal question-answer pairs through reverse data synthesis is used to gradually infer and synthesize 30,000 samples. In the second step, a parameter-efficient fine-tuning (PEFT) method called KaSA (Knowledge-aware Singular-value Adaptation) is used for fine-tuning. This method dynamically activates the parameter knowledge weights related to legal tasks by adjusting knowledge-aware singular values, and re-parameterizes legal task-specific knowledge through SVD truncation, so as to controllably, seamlessly, and with low latency fine-tune large models.

[0034] Furthermore, the legal knowledge graph supports incremental learning and online updates, enabling the model to continuously adapt to new changes in laws and regulations over time, thereby ensuring timeliness.

[0035] Furthermore, in step S2, a MoE module consisting of a word segmenter and several sub-models is set to implement task allocation through feature extraction and feature processing, specifically including: S21, in the word segmenter, the legal reminder word is converted into a plurality of subwords through word embedding processing, and then the semantic similarity and contextual relationship between the subwords are extracted using a multi-head attention mechanism to form feature data; S22, based on the MoE framework, evaluating the feature data based on the routing network, obtaining the correlation between the feature data and the sub-model, and implementing the task allocation according to the correlation; Specifically, the legal reminder words are converted into sub-words through feature extraction, and the multi-head attention mechanism is used to extract the semantic similarities and contextual relationships between the sub-words.

[0036] The Multi-Head Self-Attention Mechanism is one of the core components of the Transformer model, which allows the model to focus on different parts of the input when processing sequence data and can capture long-distance dependencies between elements in the sequence. This mechanism greatly improves the performance of the model on NLP tasks. Figure 4Shown is a structural diagram of a large language model based on the MoE model of the present application. Based on the MoE model, the present application establishes a tokenizer and a MoE module. In the tokenizer, the words in the vocabulary are mapped to dense vectors in a continuous vector space through word embedding technology, and the vocabulary includes legal reminder words in text format. In other words, through word embedding, the legal reminder words in text format are converted into multiple interrelated sub-words. Furthermore, the multi-head attention mechanism is used to analyze the relationship and importance of the sub-words, so as to extract text features. Optionally, the word embedding technology adopts any one of Word2Vec, GloVe, FastText, and ELMo.

[0037] Specifically, based on the MoE framework, the feature data is evaluated based on a routing network to obtain the correlation between the feature data and the sub-model, and the task allocation is implemented according to the correlation, including: Setting the routing network to a sparse model; On the feedforward network layer, a number of sub-models are correspondingly set; The routing network determines the correlation between the feature data and the sub-model, determines the processing order according to the correlation, and decides which feature data are sent to which sub-models; one feature data is sent to one or more sub-models, and each sub-model processes the feature data with the highest matching degree.

[0038] The generated text features are input into the routing network of the MoE module. The routing network is a key component of the MoE model. In this application, the routing network adopts a linear network, and its output is activated by the softmax function to generate a probability distribution. This probability distribution determines which expert should process a given text feature. Specifically, the routing network uses a sparse model. On the feedforward network layer, a number of "experts" are set up correspondingly. Each sub-model corresponds to an "expert". The routing network determines the correlation between the feature data and the corresponding "expert" of the sub-model, and determines the processing order according to the correlation, and determines which tokens are sent to which "experts"; one token is sent to one or more "experts", and each "expert" processes several tokens with the highest matching degree; the parameter learning of the routing network is pre-trained simultaneously with the entire MoE model. Specifically, the token corresponds to the feature data, that is, to the subwords generated by the word segmenter. Each subword corresponds to a feature data and also corresponds to a token. The token is used for subsequent "expert" semantic analysis.

[0039] Specifically, in the routing network, the correlation between the feature data and the corresponding expert of the sub-model is determined by the gating function, the processing order is determined according to the correlation, and a group of experts are dynamically selected to process the input data; the MoE architecture is usually a sparse activation (sparse MoE) setting, which only selects the top-k experts with the highest probability for activation, and only some experts are used for calculation in each forward propagation. Each sub-model corresponds to an expert. The results output by the activated experts will be weighted and summed according to their corresponding probability weights, and their decision-making strategies will be dynamically adjusted according to the performance of the model on different tasks to adapt to different legal task requirements and form the final model output.

[0040] Further, S3, processing legal issues and providing intelligent question-answering function: calling the legal knowledge graph, using the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, processing the sub-issues respectively through a number of intelligent agents, and collaboratively generating the final results, thereby realizing the legal knowledge graph question-answering function, including: S31, the legal assistant agent identifies and analyzes the legal issues raised by the user, supplements the nodes of the legal knowledge graph, and sends the analysis results to the legal search agent; S32, the legal search agent extracts and analyzes legal information from the database and sends it to the legal editing agent and the senior lawyer agent; S33. The senior lawyer agent generates a legal opinion based on the legal information. The legal editing agent combines the legal opinion and the legal information, automatically generates consulting information according to a template, and sends it to the user, thereby completing the intelligent question-and-answer function.

[0041] like Figure 5As shown, it is a structural diagram of four agents handling legal issues based on a multi-agent collaboration framework. In order to improve the efficiency and accuracy of the entire solution, this application adopts a multi-agent protocol framework. By building a multi-agent collaboration framework, we decompose the legal service process into multiple specialized steps, each of which is executed by a specific agent (such as a legal assistant agent, a legal search agent, a legal editing agent, and a senior lawyer agent). Agents can be software programs, robots, or any other form of automated systems. Different agents have different capabilities and play different roles. Different agents use a unified standardized communication language, such as FIPA ACL (Foundation for Intelligent Physical Agents Agent Communication Language), XML, JSON, etc., to send and receive messages to exchange information. Through the limitation of grammatical compatibility and semantic consistency, different agents have the same understanding of the same terminology, thereby improving the uniformity between agent models, further reducing the hallucination problem, and improving the accuracy of the results.

[0042] Specifically, the user consultation starts the process, the legal assistant agent identifies and analyzes the legal issues raised by the user, asks the user questions based on a predefined entity set, supplements the nodes of the legal knowledge graph according to the user's answers, and sends the analysis results to the legal search agent; in addition, after receiving the legal questions, the user can also be asked questions about unclear information, thereby guiding the user to ask more targeted questions.

[0043] Specifically, the legal search agent extracts and analyzes relevant legal information from the database, including legal entities, relationships between legal entities, and substantive cases. The legal database is linked to the knowledge graph. The search results are sent to the legal editor agent and the senior lawyer agent.

[0044] The senior lawyer agent generates a legal opinion based on the legal information. The legal editing agent combines the legal opinion and the legal information, automatically generates consulting information according to a template, and sends it to the user, thereby completing the intelligent question-and-answer function.

[0045] The entire system achieves efficient and accurate responses to legal consultation or legal questions and answers through effective collaboration between various agents. The main responsibilities of the legal assistant agent are to conduct preliminary interactions with users, collect key case information, and guide users to provide complete case details. It selects the corresponding predefined entity set based on the user's input and builds a knowledge graph based on it. If some nodes in the knowledge graph do not have enough information, the legal assistant agent will ask questions to obtain more information until a complete consultation knowledge graph is formed, providing a basis for subsequent processing.

[0046] The legal search agent is responsible for retrieving the latest legal provisions and judicial interpretations based on the information collected by the legal assistant agent and the knowledge graph it constructs. It uses the Retrieval Augmentation Generation (RAG) technology to combine relevant nodes in the knowledge graph to form keyword pairs, and then calls the retrieval system to collect relevant information from the Internet and domain databases. The retrieved documents are processed in parallel, and the relevance of each document is evaluated through a large language model, and finally a self-evaluated and optimized answer is output.

[0047] The Legal Editor Agent is responsible for converting the information provided by the Legal Assistant Agent and the Legal Research Agent into a formal legal document. It summarizes opinions based on previous information and drafts a formal legal document, such as a contract, to meet user needs. The Legal Editor Agent's text library contains various document templates into which the output of the Legal Assistant Agent and the Legal Research Agent can be inserted to generate the final output document.

[0048] The Advanced Lawyer Agent is responsible for providing comprehensive legal advice based on collected information and relevant legal frameworks. It uses the user's consulting questions and auxiliary information as input to provide consulting suggestions through a large language model. The goal of this stage is to come up with as many answers as possible, covering a wide range of scenarios. At the same time, it is also responsible for identifying potential risks and resolving them.

[0049] This application provides an efficient and accurate legal intelligent question-answering method by constructing a high-quality legal knowledge graph, training a large language model based on the MoE structure, and establishing multiple intelligent agents that handle different businesses. It solves the shortcomings of existing legal intelligent question-answering methods, reduces hallucinations, and improves timeliness and accuracy.

[0050] Example 1 User consultation specific questions Q-1: I would like to consult about divorce. We have a 6-year-old child and I would like to obtain custody of the child. We have a stable source of income, my monthly income is about A yuan, and my spouse's monthly income is about B yuan. We would like to divide the jointly owned house, which was purchased jointly after marriage. There is no domestic physical conflict or infidelity between us.

[0051] Agent 1 (legal assistant): Based on the user's question, the large model starts asking questions based on the predefined entity set "marriage issues" to fill in the nodes of the knowledge graph. For example, it asks about the children's situation, the financial situation of both parties, and the willingness to divide the property. Provide the age of the children during the complete divorce, the income of both parties, and the willingness to divide the property.

[0052] Agent 2 (Legal Search): Based on the information collected in step 1, the big model retrieves relevant legal provisions and precedents. For example, it searches for relevant legal provisions on child custody and property division. Based on the similarity, the relevant legal provisions retrieved include the provisions on marital property division in the XXX Civil Code and the judicial interpretation on child custody.

[0053] Agent 3 (Senior Lawyer): Based on the steps of Agent 2 and Agent 1, it provides legal advice based on the information provided by the user and the retrieved legal provisions. For example, regarding the ownership of child custody, how to divide property, and how to draft a divorce agreement. The output advice may be as follows: Since the user has a stable income and the income is higher than that of the spouse, it may be beneficial to fight for child custody; the house purchased jointly after marriage should be divided equally; it is recommended to negotiate a divorce agreement with the spouse to reduce litigation time and costs.

[0054] Agent 4 (Legal Editor): Based on the opinions of the above agents, draft formal legal documents, such as a draft divorce agreement. The final legal consultation report includes: an overview of key consultation issues, detailed opinions on child custody, property division, domestic physical conflict, infidelity and divorce agreements, and references to relevant cases and legal provisions.

[0055] In Example 1, Agent 1 effectively supplements the nodes of the legal knowledge graph by reversely querying the user information, thereby facilitating Agent 2 to call the relevant database more comprehensively. Agent 3 combines the result information of Agents 1 and 2 to give as many suggestions as possible, and Agent 4 finally outputs the consulting report based on the template and the aforementioned result information. Through the division of labor and cooperation among the four agents, the service process is refined, the user's problems and needs are efficiently and accurately identified, and the final legal consulting report is accurately generated. Compared with the simple question-and-answer dialogue of the prior art, this application is efficient, comprehensive, and accurate.

[0056] The second aspect of the present application is to provide a legal knowledge graph question-answering system based on the MoE model, corresponding to the above-mentioned legal knowledge graph question-answering method based on the MoE model. Figure 6 As shown, the legal knowledge graph question-answering system 10 based on the MoE model of the present application includes: The knowledge graph module 11 is used to collect different legal knowledge, extract and process to form the legal knowledge graph, and provide data support for the question-answering process; the knowledge graph module includes: a data integration unit, a data processing unit, and a legal knowledge graph construction unit; The MoE model module 12 integrates the MoE model into the legal knowledge language model, sets a MoE module consisting of a word segmenter and several sub-models, realizes task allocation through feature extraction and feature processing, and then performs pre-training to form a legal knowledge language model based on the MoE model; The legal question-and-answer service module 13 processes legal issues and provides intelligent question-and-answer functions: calling the legal knowledge graph, using the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and using a number of intelligent agents to process the sub-issues separately, collaboratively generate the final results, and realize the legal knowledge graph question-and-answer function.

[0057] The model training module 12 includes a feature extraction unit 121 and an expert assignment unit 122; the feature extraction unit 121 converts the prompt word into subwords, and uses the multi-head attention mechanism to extract the semantic similarity and contextual relationship between the subwords, and the extracted feature data is input into the expert assignment unit 122; The expert allocation unit 122 is based on the MoE framework and includes a routing network and a plurality of sub-models; the routing network evaluates the relevance between the feature data and each of the sub-models and processes the feature data according to the relevance ranking.

[0058] Specifically, the third aspect of the present application is to provide a storage medium, which is a non-transitory computer-readable storage medium, storing a program of a legal knowledge graph question-answering method based on the MoE model, and when the program is executed by a processor, the following steps are implemented: S1. Constructing a legal knowledge graph: Collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; S2. Building a large language model: Integrate the MoE model into the legal knowledge large language model, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form the legal knowledge large language model based on the MoE model; S3. Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

[0059] In summary, the present application provides a legal knowledge graph question-answering method and system based on the MoE model, the method comprising: constructing a legal knowledge graph: collecting different legal knowledge, extracting and processing to form the legal knowledge graph, and providing data support for the question-answering process; constructing a large language model: based on the legal knowledge large language model of the MoE model, embedding the prompt words to extract feature data, and assigning sub-models to process the feature data through a routing network; processing legal issues and providing intelligent question-answering functions: calling the legal knowledge graph, using the legal knowledge large language model based on the MoE model to identify and split the legal issues, forming sub-issues, processing the sub-issues separately through a number of intelligent agents, and collaboratively generating the final results to realize the legal knowledge graph question-answering function. Through the knowledge graph and the architecture based on the MoE model, intelligent agents that process different information are established, with division of labor and collaboration, accurate responses, improved efficiency of legal intelligent question-answering, and reduced illusion problems.

[0060] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A legal knowledge graph question answering method based on the MoE model, characterized in that: include: Constructing a legal knowledge graph: collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; Constructing a large language model: Integrate the MoE model into the legal knowledge large language model, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form a legal knowledge large language model based on the MoE model; Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

2. According to claim 1, a legal knowledge graph question-answering method based on the MoE model is characterized in that: The several intelligent agents include: legal assistant agent, legal search agent, senior lawyer agent and legal editing agent; the intelligent agents use a unified standardized communication language and exchange information by sending and receiving messages.

3. According to the MoE model-based legal knowledge graph question-answering method of claim 1, it is characterized in that: The construction of the legal knowledge graph includes: Multi-source data integration: extracting legal knowledge from a variety of different legal data sources and unifying them into a primary knowledge graph; Natural language processing: Use NLP algorithms to clean, segment, identify entities, and extract relationships from the information in the primary knowledge graph to ensure the quality of the information; The language relationship between the entities is established, the rule engine is defined, manual annotation and verification are performed, and fine-tuning training is performed based on the MoE model to form the legal knowledge graph.

4. According to claim 3, a legal knowledge graph question-answering method based on the MoE model is characterized in that: The fine-tuning training includes: Based on existing data sets, optimize legal question-answer pairs through reverse data synthesis; The fine-tuning training is performed using a parameter efficient fine-tuning method.

5. According to claim 1, a legal knowledge graph question-answering method based on the MoE model is characterized in that: The MoE module consisting of a word segmenter and several sub-models is set to realize task allocation through feature extraction and feature processing, including: In the word segmenter, the legal reminder words are converted into a number of subwords through word embedding processing, and then the semantic similarity and contextual relationship between the subwords are extracted using a multi-head attention mechanism to form feature data; Based on the MoE framework, the feature data is evaluated based on a routing network to obtain the correlation between the feature data and the sub-model, and the task allocation is implemented according to the correlation.

6. According to claim 5, a legal knowledge graph question-answering method based on the MoE model is characterized in that: Based on the MoE framework, the feature data is evaluated based on a routing network to obtain the correlation between the feature data and the sub-model, and the task allocation is implemented according to the correlation, including: Setting the routing network to a sparse model; On the feedforward network layer, a number of sub-models are correspondingly set; The routing network determines the correlation between the feature data and the sub-model, determines the processing order according to the correlation, and decides which feature data are sent to which sub-models; one feature data is sent to one or more sub-models, and each sub-model processes the feature data with the highest matching degree.

7. According to claim 2, a legal knowledge graph question-answering method based on the MoE model is characterized in that: The said processing of legal issues and providing intelligent question-answering functions include: The legal assistant agent identifies and analyzes legal questions raised by users, asks users questions based on predefined entity sets, supplements nodes of the legal knowledge graph based on user answers, and sends analysis results to the legal search agent; The legal search agent extracts and analyzes legal information from the database and sends it to the legal editing agent and the senior lawyer agent; The senior lawyer agent generates a legal opinion based on the legal information. The legal editing agent combines the legal opinion and the legal information, automatically generates consulting information according to a template, and sends it to the user, thereby completing the intelligent question-and-answer function.

8. A legal knowledge graph question-answering system based on the MoE model, characterized in that: include: The knowledge graph module is used to collect different legal knowledge, extract and process it to form the legal knowledge graph, and provide data support for the question-answering process; The MoE model module integrates the MoE model into the legal knowledge language model, sets up a MoE module consisting of a word segmenter and several sub-models, implements task allocation through feature extraction and feature processing, and then performs pre-training to form a legal knowledge language model based on the MoE model; The legal question-and-answer service module handles legal issues and provides intelligent question-and-answer functions: calling the legal knowledge graph, using the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and using a number of intelligent agents to handle the sub-issues separately, collaboratively generate the final results, and realize the legal knowledge graph question-and-answer function.

9. A legal knowledge graph question-answering system based on the MoE model according to claim 8, characterized in that: The several intelligent agents include: legal assistant agent, legal search agent, senior lawyer agent and legal editing agent; the intelligent agents use a unified standardized communication language and exchange information by sending and receiving messages.

10. A storage medium, characterized in that: The storage medium is a non-transitory computer-readable storage medium, storing a program of a legal knowledge graph question-answering method based on the MoE model, and the program implements the following steps when executed by a processor: Constructing a legal knowledge graph: collecting different legal knowledge, extracting and processing it to form the legal knowledge graph, and providing data support for the question-answering process; Constructing a large language model: Integrate the MoE model into the large language model of legal knowledge, set up a MoE module consisting of a word segmenter and several sub-models, implement task allocation through feature extraction and feature processing, and then perform pre-training to form the large language model of legal knowledge based on the MoE model; Process legal issues and provide intelligent question-and-answer functions: call the legal knowledge graph, use the legal knowledge large language model based on the MoE model to identify and split the legal issues into sub-issues, and use several intelligent agents to process the sub-issues separately, collaborate to generate the final results, and realize the legal knowledge graph question-and-answer function.

Images