Chinese legal consultation framework based on large-model multi-agent cooperation

Abstract

The invention discloses a Chinese legal consultation framework based on large model multi-agent collaboration, which comprises a core large model, the core large model is constructed based on ChatGLM-3-6b, a Transform architecture is adopted, the core large model carries out LoRA fine tuning through knowledge intensive dialogue data and real lawyer consultation multi-round dialogue data, a multi-agent system is arranged in the core large model, and the core large model is connected with the multi-agent system. The multi-agent system comprises a receptionist agent, a lawyer agent, a secretary agent and a boss agent, and a real legal consultation scene is simulated through cooperation among the receptionist agent, the lawyer agent, the secretary agent and the boss agent. The method has the beneficial effects that efficient legal consultation service is provided; the professionality and accuracy of consultation are improved; the user experience is improved; the cost is reduced; the working efficiency is improved; a practice learning platform is provided; knowledge updating is promoted; the judicial efficiency is improved; the model performance and adaptability are improved; and the expandability and the expandability are realized.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and legal consulting, and in particular to a Chinese legal consulting framework based on large-model multi-agent collaboration. Background Art

[0002] In today's society, with the continuous improvement of people's legal awareness and the increasing complexity and diversity of legal affairs, the demand for legal consulting services has shown a continuous growth trend. The traditional legal consulting model mainly relies on lawyers in law firms to communicate with clients face-to-face or by phone, which has many limitations. On the one hand, the distribution of resources for traditional legal consulting services is extremely uneven, mainly concentrated in economically developed areas and large cities, while in remote areas or small and medium-sized towns, people often find it difficult to obtain high-quality legal consulting services, resulting in a large number of potential legal needs not being met. On the other hand, the cost of hiring professional lawyers is high, which is a considerable financial burden for many individuals and small and medium-sized enterprises, especially for some simple legal issues, paying high lawyer fees may make them discouraged.

[0003] At the same time, artificial intelligence technology has achieved rapid development in recent decades, bringing opportunities for innovation in many fields. Natural language processing technology, as an important branch of artificial intelligence, has demonstrated its powerful capabilities. In this context, large language models (LLMs) came into being and have gradually been applied in many fields, including the field of legal consulting. Large language models can process and analyze natural languages ​​by learning and training massive text data, and can provide users with text information processing, knowledge question and answer services to a certain extent. In the field of Chinese legal consulting, many large language models have begun to emerge. They try to use natural language processing technology to provide users with legal question answers, legal document generation and other services, bringing new hope to alleviate the pain points of traditional legal consulting.

[0004] However, although the big language model has shown great development potential at the technical level, it still faces many urgent problems when it is actually applied to Chinese legal consultation. First of all, from the perspective of model design, most of the existing Chinese legal big language models are based on a simple single model and a single-point interaction mode with users. This model is quite different from the real legal consultation scenario. In the actual legal consultation process, it involves the collaborative work of multiple professional roles. For example, a complete legal consultation process usually includes a receptionist responsible for preliminary communication and information collection with the user, a lawyer providing detailed and professional legal opinions to the user with his professional knowledge, a secretary responsible for organizing the documents in the consultation process and providing follow-up services, and even a manager (such as the boss) may be involved to supervise the quality and effect of the entire consultation service. However, the current legal big language model does not effectively integrate these multi-role collaboration mechanisms into it, so that users cannot experience the process and service quality similar to the legal consultation services provided by real law firms when using it, which seriously restricts the value and practicality of these models in actual legal consultation.

[0005] Secondly, for users, since most users lack professional legal knowledge, they often find it difficult to express their questions clearly and accurately in professional language when asking legal questions to the large language model. The questions they ask are often vague and lack key details. For example, users may simply ask broad questions such as "I have encountered a contract dispute, what should I do?" "I want a divorce, how should I deal with it?" In this case, the large language model, without further information supplementation and refinement, can easily generate answers that do not meet the actual needs of users, and even so-called "hallucination answers", that is, the generated answers may deviate from the actual provisions of the law and the actual situation of the user, which not only cannot help users solve problems, but may mislead users to make wrong legal decisions, bringing potential risks and losses to users.

[0006] Furthermore, from the perspective of training data for large language models, the quality of data currently used to train Chinese legal large language models varies. Although a large amount of legal question-and-answer data is used in the training process, most of this data comes from public legal question-and-answer platforms, and the information on these platforms has great uncertainty. The questions and answers may come from users and providers of different levels, and some data are less professional, and may even contain wrong information or inaccurate statements. Since training data is the basis for large language models to learn and generate answers, low-quality data will seriously affect the ability of large language models to follow instructions and the accuracy of their answers. For example, when users ask some complex legal questions, the model may give answers that violate legal norms due to the influence of training data, or when faced with some challenging questions, the model shows a low level of confidence, and the quality and reliability of the answers given are greatly reduced.

[0007] In addition, the existing large language models also show obvious deficiencies in multi-round dialogue capabilities. In actual legal consultation, the communication between users and lawyers is often a gradual and in-depth process, which requires multiple rounds of dialogue to continuously refine and clarify user needs and ultimately develop appropriate solutions. However, when training, the existing legal large language models focus more on single-round dialogue scenarios and have poor processing capabilities for contextual information in multi-round dialogues. This results in the model generating answers that are out of touch with the previous context or even lose key information during multiple rounds of consultation, which greatly affects the user's consultation experience and fails to meet the user's needs for consultation on complex legal issues.

[0008] In summary, although large language models have shown certain application prospects in the field of legal consulting, due to problems in many aspects, such as model design, handling of user questions, data quality, and multi-round dialogue capabilities, their actual application effects are far from meeting the expectations of users and the market for high-quality, efficient, and low-cost legal consulting services, which also provides a broad space and urgent needs for further technological innovation and improvement. In this context, how to develop a legal consulting system that can be closer to actual legal consulting scenarios, can handle user fuzzy questions, has high-quality data support, and has strong multi-round dialogue capabilities has become an important research direction in the current field of artificial intelligence and legal consulting.

[0009] In the field of legal artificial intelligence, early legal question-answering systems were mainly divided into several categories, including retrieval-based question-answering systems, knowledge-based question-answering systems, and machine reading comprehension-based question-answering systems. Retrieval-based question-answering systems mainly rely on pre-defined question-answer pairs, which can provide some help when facing some simple and common questions, but their limitations are prominent for complex and ambiguous user questions because they cannot flexibly handle information beyond the predefined scope. Knowledge-based question-answering systems require the construction of knowledge graphs, but the scope of knowledge graphs is usually limited. It is difficult to update and maintain the constantly updated legal provisions and various new cases, so they seem to be unable to cope with dynamic legal information. Although the question-answering system based on machine reading comprehension can extract information from text, it has major defects in generating comprehensive and personalized answers, and it is difficult to meet users' needs for detailed legal advice. Although some high-quality Chinese legal datasets have emerged in recent years, such as the Chinese Judicial Machine Reading Comprehension Dataset CJRC and the Chinese Judicial Examination Evaluation Dataset JEC-QA, these datasets have played a certain role in promoting the development of Chinese legal question-answering technology, but they still cannot completely solve the above-mentioned problems, especially in multi-round dialogue and user experience. There is still much room for improvement.

[0010] With the development of large language models, some projects have begun to try to apply them to the field of legal question answering and have achieved certain results. For example, LawGPT has continued to pre-train and fine-tune the Chinese-based large model Chinese-LLaMA-7B using public legal documents and judicial examination data, which has improved its understanding and processing capabilities of legal content to a certain extent; the LexiLaw project uses Hualvwang's question and answer data, judicial examination data, and question and answer data with legal basis to fine-tune ChatGLM-6B, and uses Freeze, Lora, and P-Tuning-V2 to optimize the training process; LawGPT_zh and lawyer-llama have also enhanced the application capabilities of large language models in legal scenarios to a certain extent by fine-tuning ChatGLM-6B and Chinese-LLaMA-13 ​​respectively. However, these attempts still do not completely solve the many challenges currently faced by large language models for Chinese legal consultation, and the market and users are still looking forward to the emergence of more complete and practical solutions.

[0011] Based on the above-mentioned industry status and technological development background, the present invention is committed to providing a new technical path to optimize and improve the performance of the Chinese legal large language model through the construction and application of a multi-agent collaborative framework and high-quality data sets, so as to better meet the legal consulting needs of users, fill the shortcomings of current technology in this field, and provide users with better-quality, more professional, and more practical legal consulting services. Summary of the invention

[0012] In order to solve the above problems, especially to address the deficiencies in the prior art, the present invention provides a Chinese legal consultation framework based on large-model multi-agent collaboration that can solve the above problems.

[0013] To achieve the above purpose, the present invention adopts the following technical means:

[0014] A Chinese legal consultation framework based on large-model multi-agent collaboration, including a core large model, which is built on ChatGLM-3-6b and adopts the Transformer architecture. The core large model is fine-tuned by LoRA through knowledge-intensive dialogue data and multi-round dialogue data of real lawyer consultation to obtain a multi-round dialogue core large model with legal background

[0015] Among them, U 1:T represents the sequence of questions asked by the user from the first round to the Tth round of dialogue, R 1:T Represents the answer sequence returned by the model, and s0 represents the initial dialogue state;

[0016] The core large model is provided with a multi-agent system, which includes a receptionist agent, a lawyer agent, a secretary agent and a boss agent, and simulates a real legal consultation scenario through the collaboration among the receptionist agent, the lawyer agent, the secretary agent and the boss agent;

[0017] The receptionist agent is responsible for classifying user questions and assigning lawyers;

[0018] The lawyer agent provides professional legal advice;

[0019] The secretary agent generates a consulting report;

[0020] The boss agent performs supervision and evaluation.

[0021] A further solution of the present invention is that the construction of the knowledge-intensive dialogue data adopts a Self-Instruct strategy, specifically:

[0022] First, we screened out high-quality data from the existing legal dialogue data and eliminated data of poor quality;

[0023] Second, in order to enhance the model’s knowledge application of key legal concepts and terms, we constructed conversation data on the interpretation of legal concepts and terms;

[0024] Third, generate judicial interpretation and legal judgment dialogue data to improve the model's ability to understand important crimes and laws;

[0025] The real lawyer consultation multi-round dialogue data is constructed based on real lawyer consultation multi-round dialogues.

[0026] A further solution of the present invention is that the universal conversation data Alpaca-GPT4 is added in the LoRA fine-tuning process, and the universal conversation data Alpaca-GPT4 contains 52,000 universal Chinese conversations.

[0027] A further solution of the present invention is that the LoRA fine-tuning decomposes the weight matrix of the core large model into the product of two low-rank matrices, thereby reducing the number of parameters and reducing the computational complexity. The fine-tuning process of the LoRA fine-tuning is shown in formula (1):

[0028]

[0029] Among them, θ is the initial parameter of ChatGLM-3-6b, θ Legal It is the legal large model LLM obtained after LoRA fine-tuning Legal Parameters, represents the nth training sample, and LoRA(·) represents LoRA fine-tuning.

[0030] A further solution of the present invention is that the receptionist agent classifies user questions based on the ChatGLM-3-6b model fine-tuned by LoRA, by comparing the trained Chinese version of the RoBERTa model.

[0031] A further solution of the present invention is that the lawyer agent uses role enhancement technology to generate more professional answers based on the ChatGLM-3-6b model fine-tuned by LoRA; at the same time, based on the clarification tree, a legal clarification tree algorithm is proposed to deal with rough and vague legal questions raised by users.

[0032] A further solution of the present invention is that the secretary agent generates an expected consulting report through In-Context Learning based on the ChatGLM-3-6b model fine-tuned by LoRA.

[0033] A further solution of the present invention is that the boss agent is a Reward Model.

[0034] Beneficial effects of the present invention:

[0035] 1. The present invention provides efficient legal consulting services:

[0036] Traditional legal consulting services are often limited by the lawyer's time and location. Users may have to wait a long time for the lawyer's response or fail to obtain high-quality services. This framework can provide users with answers to legal questions in real time through multi-agent collaboration and the powerful capabilities of the core big model. After the user raises a legal question, the receptionist agent quickly assigns it to the appropriate lawyer agent. The lawyer agent can quickly generate professional legal advice through the powerful text processing capabilities and role enhancement technology of the core big model, greatly shortening the user's waiting time and improving the efficiency of users in obtaining legal advice.

[0037] 2. The present invention improves the professionalism and accuracy of consultation:

[0038] The core large model uses high-quality knowledge-intensive dialogue data and multi-round dialogue data of real lawyer consultations to fine-tune LoRA, so that when answering user questions, the lawyer agent can provide users with more accurate and legally compliant professional advice based on rich legal knowledge and professional legal information. At the same time, the legal clarification tree algorithm helps users clarify ambiguous issues and ensures that the legal advice users receive is targeted and operational.

[0039] 3. The present invention improves user experience:

[0040] Multi-agent collaboration simulates the workflow of a real law firm, allowing users to experience a more comprehensive service experience. Multi-round dialogue support allows users to communicate repeatedly with lawyer agents, gradually solve complex legal problems, and make the consultation process smoother. The consultation report generated by the secretary agent provides users with a clear consultation record, making it convenient for users to review the entire consultation process and results, and increasing users' satisfaction with the consultation service. Moreover, since the system can quickly respond to and handle various user issues, users can get high-quality services regardless of whether they are in cities or remote areas, overcoming the problem of uneven distribution of traditional legal consultation resources and improving users' accessibility and experience of legal consultation services.

[0041] 4. The present invention reduces costs:

[0042] For enterprises, integrating this framework into the corporate legal system can provide enterprises with automated legal consulting services to handle legal issues in various aspects such as corporate law, intellectual property, contract review, etc. Compared with hiring full-time lawyers or long-term legal consultants, using this framework can significantly reduce legal costs, especially for small and medium-sized enterprises or startups, which can obtain basic legal support at a lower cost.

[0043] 5. The present invention improves work efficiency:

[0044] Enterprises can use this system to quickly handle a large number of daily legal affairs, such as handling employee labor disputes, contract disputes, etc., and improve the efficiency of corporate legal work. At the same time, for complex legal affairs, the system can also provide valuable reference and assistance for internal legal personnel, helping them to formulate legal strategies more quickly, reduce the time and energy of handling legal affairs, and enable corporate legal personnel to devote more energy to more complex and strategic legal affairs.

[0045] 6. The present invention provides a practical learning platform:

[0046] The framework can provide law students and practitioners with simulated legal consultation scenarios to help them exercise their ability to apply legal knowledge. Through multiple rounds of dialogue and handling of various legal issues, students can practice how to communicate with clients, how to analyze legal issues, and how to apply legal provisions in a simulated environment, thereby improving their practical skills.

[0047] 7. The present invention promotes knowledge updating:

[0048] Since the core model is trained and updated through a large amount of legal data, the use of this system can enable legal practitioners to keep abreast of the latest legal developments and legal knowledge, helping them to continuously update their knowledge system and avoid mistakes in actual work due to outdated knowledge.

[0049] 8. The present invention improves judicial efficiency:

[0050] The framework can provide judges and lawyers with support such as legal text retrieval, case analysis, and legal advice generation. Judges can use the system to assist in analyzing cases and find relevant legal texts and similar cases. Lawyers can obtain different perspectives and ideas on complex legal issues and improve the efficiency of case handling. At the same time, the legal advice generated by the system complies with the latest legal provisions, which helps to ensure the consistency and authority of the judiciary and promote the efficient operation of the judicial system.

[0051] 9. The present invention improves model performance and adaptability:

[0052] Through the LoRA fine-tuning strategy, the performance of the core large model in the legal consulting scenario is optimized, reducing the demand for computing resources while maintaining high reasoning accuracy. By adding general dialogue data Alpaca-GPT4 for training, the generalization ability of the model is improved, so that the core large model can show good performance when facing various types of user input, enhancing the adaptability of the system. Moreover, through multi-agent collaboration and algorithm optimization strategies, the system can be flexibly adjusted according to different user needs and usage scenarios, improving the overall performance of the system.

[0053] 10. Scalability and expansibility of the present invention:

[0054] The framework has good scalability and can expand its coverage of legal fields by introducing more high-quality legal data sets, from the current main fields of civil law and criminal law to more fields such as international law, environmental law, and financial law, to meet a wider range of legal consulting needs. At the same time, the system can be expanded to multi-language support to serve users in more countries and regions. The combination of knowledge graph technology can deepen the understanding of the relationship between legal provisions and enhance the depth and breadth of consultation. The combination of blockchain technology can ensure the transparency of the consultation process and data security, giving the system broad development prospects and strong expansion capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Legal consulting fields and statistical information graphs predefined for the present invention;

[0056] Figure 2 The winning rate graph of LawLuo of the present invention compared with the baseline model;

[0057] Figure 3 Generate a graph of the change in response quality for the model of the present invention;

[0058] Figure 4 This is a diagram of the ablation experiment results of the present invention;

[0059] Figure 5 The clarification tree generated by ToLD of the present invention and the result diagram after the user marks Yes / No. DETAILED DESCRIPTION

[0060] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example

[0062] A Chinese legal consultation framework based on large-scale multi-agent collaboration, including a core large model. The core large model is built on ChatGLM-3-6b and adopts the Transformer architecture. The core large model is fine-tuned by LoRA through knowledge-intensive dialogue data and multi-round dialogue data of real lawyer consultation to obtain a multi-round dialogue core large model with legal background.

[0063] Among them, U 1:T represents the sequence of questions asked by the user from the first round to the Tth round of dialogue, R 1:T Represents the answer sequence returned by the model, and s0 represents the initial dialogue state;

[0064] The core model has a multi-agent system, which includes receptionist agent, lawyer agent, secretary agent and boss agent. The multi-agent system simulates the real legal consultation scenario through the collaboration among receptionist agent, lawyer agent, secretary agent and boss agent.

[0065] The receptionist agent is responsible for assigning lawyers in the corresponding fields to visiting users based on the questions they raise;

[0066] The lawyer agent conducts multiple rounds of LQA with the user in order to solve the legal problems he faces;

[0067] The secretary agent is responsible for organizing the conversation between the user and the lawyer, forming a consulting report and submitting it to the user and the boss;

[0068] The boss agent is responsible for scoring lawyers based on consulting reports and user evaluations.

[0069] To ensure the diversity and professionalism of the data, the construction of knowledge-intensive dialogue data adopts a Self-Instruct strategy, specifically: first, high-quality parts are screened out from the existing legal dialogue data, and poor-quality data are eliminated; second, in order to enhance the model's knowledge application of key legal concepts and terms, interpretation dialogue data of legal concepts and terms are constructed; third, judicial interpretation and legal judgment dialogue data are generated to enhance the model's ability to understand important crimes and laws.

[0070] The real lawyer consultation multi-round dialogue data is constructed based on the real lawyer consultation multi-round dialogue. Different from the traditional single-round dialogue data, the real lawyer consultation multi-round dialogue data simulates the multi-round interaction scenarios in actual legal consultation, ensuring the consistency and practicality of the data.

[0071] During the fine-tuning process of LoRA, the general conversation data Alpaca-GPT4 was added, which contains 52,000 general Chinese conversations. The use of this mixed data not only enhances the generalization ability of the model, but also ensures its stable performance in legal consulting scenarios.

[0072] LoRA fine-tuning decomposes the weight matrix of the core large model into the product of two low-rank matrices, thereby reducing the number of parameters and the computational complexity. The fine-tuning process of LoRA fine-tuning is shown in formula (1):

[0073]

[0074] Among them, θ is the initial parameter of ChatGLM-3-6b, θ Legal It is the legal large model LLM obtained after LoRA fine-tuningLegal Parameters, represents the nth training sample, and LoRA(·) represents LoRA fine-tuning.

[0075] The receptionist agent is based on the ChatGLM-3-6b model fine-tuned by LoRA, and classifies user questions by comparing the trained Chinese version of the RoBERTa model to ensure that users can be assigned to the most appropriate lawyer agent. The classification process is shown in formula (2):

[0076]

[0077] in It is a set of predefined consulting fields. According to the online legal consulting field classification, it contains 16 fields, such as Figure 1 Among them, 15 are common consulting areas, while other uncommon consulting areas are grouped into the category of "others".

[0078] We use the small language model SML, which is the Chinese version of RoBERTa trained by comparison, as the Sec agent. To this end, we first crawled 35,060 legal questions with domain labels from the Internet. The number of questions in each domain is as follows: Figure 1 shown.

[0079] Then, we take the current question as the anchor, and take questions from the same field as positive examples, questions from different fields as negative examples, and use contrastive loss to optimize the semantic distance between samples. The loss function is shown in formula (3):

[0080]

[0081] in, represents the embedded representation of the i-th anchor sample, represents the representation of positive samples from the same field as the anchor, and represents the embedded representation of negative samples from different fields than the anchor. In addition, α is the hyperparameter margin in the contrastive loss, and ||·|‖ represents the Euclidean distance. We use Lawformer as the embedding model for representation.

[0082] The lawyer agent uses the ChatGLM-3-6b model fine-tuned by LoRA and uses role enhancement technology to generate more professional answers. For consulting field d, lawyers who are good at this field are denoted as Lawyer d , Lawyer d ←RE d (LLM Legal ). Among them, RE d (·) indicates a hint for character enhancement.

[0083] At the same time, based on the Tree of Clarifications (ToC), the Tree of Legal Clarifications (ToLC) algorithm was proposed to deal with rough and ambiguous legal questions raised by users. Compared with ToC, ToLC has the following two key improvements:

[0084] Case Library Retrieval: ToLC searches for the top-k similar cases from past case libraries to guide LLM to generate legal requirements that need clarification, rather than searching for articles from Wikipedia like ToC. This improvement enables the model to generate legal clarification questions more accurately.

[0085] Active user labeling: ToLC prunes irrelevant nodes by allowing users to actively label nodes with Yes / No, rather than relying on LLM self-verification. This design is based on a core idea: no language model or external knowledge can understand the user's true situation better than the user himself. Therefore, the task of the lawyer agent is to guide the user to clarify the legal facts, rather than arbitrarily guessing the legal facts involved by the user.

[0086] The secretary agent generates an expected consulting report based on the ChatGLM-3-6b model fine-tuned by LoRA through In-Context Learning. The generation process is shown in formula (4):

[0087]

[0088] Where r represents the consulting report generated by the secretary agent. We use ChatGLM-3-6b as the secretary agent and use In-Context Learning to guide the model to generate consulting reports that meet expectations.

[0089] The boss agent is the Reward Model (RM). We first train a binary evaluation RM. Where o represents the output of the lawyer or secretary agent, and y represents RM's evaluation of o, including two categories: "better" and "worse". The training goal of RM is to minimize the following loss function:

[0090]

[0091] where y i is the true label of the i-th sample, which takes the value of 0 or 1, representing "worse" and "better" respectively. is the RM prediction of the i-th output o iThe probability of being "better".

[0092] Working principle:

[0093] The working principle of this Chinese legal consulting framework based on big model multi-agent collaboration mainly involves the coordinated operation of the core big model, multi-agent system and algorithm optimization strategy. The following is its detailed workflow:

[0094] 1. User-initiated consultation:

[0095] The user asks the system a legal question, which is input into the system in the form of natural language. For example, the user inputs a question like "I was injured in a traffic accident, how can I get compensation?"

[0096] 2. Preliminary processing of the receptionist agent:

[0097] The receptionist agent starts working, using the Chinese version of the RoBERTa model trained by comparison to process the user's questions.

[0098] The receptionist agent takes the user's question as input and calls the Chinese version of the RoBERTa model to perform semantic analysis and classification. The model compares the user's question with the 16 predefined consulting fields through contrastive learning, and classifies the user's question into the appropriate field based on semantic similarity. For example, for the above-mentioned traffic accident compensation question, the receptionist agent may classify it into the "traffic accident law" field.

[0099] 3. Professional services of lawyer agents:

[0100] The lawyer agent starts multiple rounds of legal consultation dialogues with the user based on the receptionist agent's assignment.

[0101] The lawyer agent is based on the fine-tuned ChatGLM-3-6b model, which has been fine-tuned by LoRA using knowledge-intensive conversation data, multi-round conversation data of real lawyer consultations, and general conversation data Alpaca-GPT4. It has a deep understanding of legal knowledge and the ability to handle multi-round conversations.

[0102] First, the lawyer agent uses role enhancement technology to provide users with more targeted legal advice. It will add corresponding role information to the ChatGLM-3-6b model to make the model's answers more in line with the professional characteristics of the field. For the above-mentioned traffic accident compensation case, the lawyer agent may add the role information of "traffic accident lawyer" to make the model more professional when answering.

[0103] When the user's question is vague, the lawyer agent uses the Legal Clarification Tree (ToLC) algorithm. The algorithm searches for the top-k similar cases from the past case library, inputs this information into the core model, and guides the core model to generate legal requirements that require clarification by the user. For example, for the user's traffic accident compensation question, ToLC may retrieve similar accident cases, and then the core model will generate clarification questions such as "Please tell me the specific circumstances of the accident, including whether there were traffic lights, whether there was drunk driving, the extent of your injuries, etc." based on these cases to guide the user to supplement key information.

[0104] The lawyer agent has multiple rounds of conversations with the user. During this process, the core model continuously generates new responses based on each round of responses from the user and previous information, such as "Based on the information you provided, you can first collect evidence such as medical expense invoices and accident liability determination letters. If the other party is fully responsible, you can file a claim for compensation from the other party's insurance company, including medical expenses, loss of work time, etc. The specific amount of compensation can be calculated based on relevant legal provisions and your actual losses..."

[0105] 4. Information collation of secretary agents:

[0106] The secretary agent starts working after multiple rounds of conversations between the user and the lawyer agent are completed.

[0107] The secretary agent takes the multi-round conversation information between the user and the lawyer agent as input, and generates a consulting report using the ChatGLM-3-6b model and In-Context Learning technology. It will integrate the user's questions, the lawyer's suggestions, the user's clarification information and other content according to the pre-constructed standard consulting report template to form a complete consulting report. For example, the generated report may contain content such as "The user is consulting about compensation after being injured in a traffic accident. The user had a traffic accident on [specific date] and the injury was [specific description]. The lawyer suggested that the user collect relevant evidence and file a claim for compensation with the other party's insurance company..."

[0108] 5. Supervision and evaluation of boss agents:

[0109] The boss agent acts as a Reward Model (RM) to supervise and evaluate the outputs of the lawyer agent and the secretary agent.

[0110] The boss agent takes the legal advice of the lawyer agent and the consulting report of the secretary agent as input, evaluates these outputs according to its internal evaluation criteria, and optimizes the entire system based on the evaluation results. If the evaluation results show that the response of the lawyer agent or the report of the secretary agent is of low quality, the system will adjust the strategy based on the feedback of the boss agent, prompting the lawyer agent and the secretary agent to improve the subsequent services and make the system output more in line with professional standards.

[0111] 6. Basic support and algorithm optimization of core large models:

[0112] The core large model plays a basic supporting role in the whole process.

[0113] The core large model continuously optimizes its performance through the Transformer architecture and LoRA fine-tuning, providing powerful text understanding and generation capabilities in various aspects such as processing user questions, generating legal advice, and compiling reports.

[0114] LoRA fine-tuning improves reasoning accuracy while reducing computing resources by decomposing the weight matrix of the core large model into the product of two low-rank matrices, ensuring that the core large model can process legal information quickly and accurately.

[0115] The knowledge-intensive conversation data used in the core big model's training process ensures the learning of legal terminology, legal interpretations, judgment conversations and other information. The multi-round conversation data of real lawyer consultations provides multi-round conversation experience, while the general conversation data Alpaca-GPT4 enhances the generalization ability of the model, enabling the core big model to perform well under various legal issues and user inputs.

[0116] 7. Continuous optimization and iteration of the system:

[0117] By continuously receiving user inquiries, the system can continue to collect new data and further train and optimize the core large model.

[0118] As users interact with the system, data such as new consulting questions, lawyers’ responses, and secretaries’ reports can be used to update and optimize the core model and each intelligent agent. At the same time, the supervision and evaluation of the boss intelligent agent can continuously adjust the performance of the system, allowing the entire system to evolve continuously during continuous use and provide better legal consulting services.

[0119] Testing and verification solutions

[0120] In order to comprehensively evaluate the performance of the Chinese legal consultation framework based on large-model multi-agent collaboration, a test plan with three parts: single-round question test, multi-round dialogue test and ablation experiment was designed.

[0121] Single round test

[0122] In the single-round question test, we adopted the Pairwise Evaluation method and used three evaluation criteria: Lawyer-like language, usefulness of legal advice, and accuracy of legal knowledge to compare LawLuo with baseline models (including ChatGLM-3-6b, LawGPT, LawyerLLaMa, and GPT-4). The test results show that LawLuo has significant advantages over all baseline models. In particular, a win rate of 72% was achieved on ChatGLM-3-6b, which proves the effectiveness of the instruction fine-tuning process. In addition, LawLuo also showed significant advantages in the comparison with GPT-4, further verifying its ability to generate high-quality legal advice. The specific results are as follows: Figure 2 shown.

[0123] Multi-round dialogue test

[0124] In the multi-round dialogue test, we used GPT-4o to score the answers generated in each round of dialogue. The scoring criteria were the same as the single-round question test, with a score range of 1-10. The test results showed that as the number of dialogue rounds increased, the quality of the answers generated by LawLuo remained at a high level. This result is mainly attributed to the use of multi-round dialogue data from real law firms for instruction fine-tuning, while other legal LLMs only used single-round dialogue data, resulting in poor multi-round dialogue performance. The specific results are as follows: Figure 3 shown.

[0125] Ablation experiment

[0126] In the ablation experiment, we verified the contribution of each part by removing different components such as the receptionist agent, the role enhancement module, and the ToLD module. The test results show that after removing the receptionist agent and the role enhancement module, LawLuo's win rate on ChatGPT decreased by 3%, indicating the importance of giving legal LLM roles in different fields. After removing the ToLD module, the model effect dropped significantly, indicating that in legal Q&A, clarifying users' rough and vague questions is crucial to generating high-quality answers. In addition, the boss agent further improved the performance of the model by optimizing the responses generated by lawyers. The specific results are as follows: Figure 4 shown.

[0127] ToLD Case

[0128] To further verify the contribution of ToLD, we conducted an analysis through a specific case. Take "I want to divorce, what should I do?" as an example. Figure 5 shown.

[0129] Application scenarios and expansion directions

[0130] Application Scenario

[0131] First, the LawLuo framework can provide users with real-time and accurate legal consulting services to help solve daily legal problems, such as contract disputes, labor rights protection, marriage and family issues, etc. Through multi-agent collaboration and fine-tuning of high-quality data sets, the framework can quickly respond to users' legal needs and improve consulting efficiency.

[0132] Second, the framework can be integrated into the corporate legal system to provide companies with automated legal consulting services, helping to deal with legal issues in the fields of corporate law, intellectual property, contract review, etc., and reducing corporate legal costs. Through role enhancement technology, the framework can provide more targeted advice on legal issues in different fields, ensuring the professionalism and accuracy of the consultation.

[0133] Third, the LawLuo framework can provide students and legal practitioners with simulated legal consultation scenarios to help them improve their ability to apply legal knowledge. Through multi-round dialogue support and the Legal Clarification Tree (ToLD) algorithm, the framework can gradually solve complex legal problems and improve users' legal practice capabilities.

[0134] Fourth, the framework can provide judges and lawyers with support such as legal text retrieval, case analysis, and legal advice generation, thus improving judicial efficiency. By integrating with authoritative legal databases, the framework ensures that the generated legal advice complies with the latest legal provisions, thus improving the authority and consistency of the judicial system.

[0135] Expansion direction

[0136] The coverage of the legal field can be expanded. Currently, the LawLuo framework mainly covers common legal fields, such as civil law and criminal law. In the future, support for international law, environmental law, financial law and other fields can be increased to meet a wider range of legal consulting needs. By introducing more high-quality legal data sets, the framework can further enhance its professionalism and accuracy in different legal fields.

[0137] The current framework mainly supports Chinese legal consultation. In the future, it can be expanded to legal consultation services in other languages ​​by introducing multilingual support, thus improving its international application capabilities. This will enable the framework to provide high-quality legal consultation services to users in more countries and regions.

[0138] It can be combined with knowledge graph technology to help the framework better understand the relationship between legal provisions and enhance the depth and breadth of consultation.

[0139] By exploring the integration with blockchain technology, the framework can ensure the transparency and data security of the legal consultation process and enhance users’ trust in the system. Blockchain technology can be used to record the consultation process and data, ensuring the non-tamperability and traceability of the consultation.

[0140] The examples given in the present invention are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here, and the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A Chinese legal consultation framework based on large-scale multi-agent collaboration, characterized by: Contains a core large model, which is built on ChatGLM-3-6b and adopts the Transformer architecture. The core large model is fine-tuned by LoRA through knowledge-intensive dialogue data and multi-round dialogue data of real lawyer consultation to obtain a multi-round dialogue core large model with legal background Among them, U 1:T represents the sequence of questions asked by the user from the first round to the Tth round of dialogue, R 1:T Represents the answer sequence returned by the model, and s0 represents the initial dialogue state; The core large model is provided with a multi-agent system, which includes a receptionist agent, a lawyer agent, a secretary agent and a boss agent, and simulates a real legal consultation scenario through the collaboration among the receptionist agent, the lawyer agent, the secretary agent and the boss agent; The receptionist agent is responsible for classifying user questions and assigning lawyers; The lawyer agent provides professional legal advice; The secretary agent generates a consulting report; The boss agent performs supervision and evaluation.

2. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The construction of the knowledge-intensive dialogue data adopts the Self-Instruct strategy, specifically: First, we screened out high-quality data from the existing legal dialogue data and eliminated data of poor quality; Second, in order to enhance the model’s knowledge application of key legal concepts and terms, we constructed conversation data on the interpretation of legal concepts and terms; Third, generate judicial interpretation and legal judgment dialogue data to improve the model's ability to understand important crimes and laws; The real lawyer consultation multi-round dialogue data is constructed based on real lawyer consultation multi-round dialogues.

3. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: During the LoRA fine-tuning process, the general conversation data Alpaca-GPT4 was added, and the general conversation data Alpaca-GPT4 contained 52,000 general Chinese conversations.

4. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The LoRA fine-tuning decomposes the weight matrix of the core large model into the product of two low-rank matrices, thereby reducing the number of parameters and reducing the computational complexity. The fine-tuning process of the LoRA fine-tuning is shown in formula (1): Among them, θ is the initial parameter of ChatGLM-3-6b, θ Legal It is the legal large model LLM obtained after LoRA fine-tuning Legal Parameters, represents the nth training sample, and LoRA(·) represents LoRA fine-tuning.

5. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The receptionist agent classifies user questions based on the ChatGLM-3-6b model fine-tuned by LoRA and the Chinese version of the RoBERTa model trained by comparison.

6. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The lawyer agent is based on the ChatGLM-3-6b model fine-tuned by LoRA and uses role enhancement technology to generate more professional answers. At the same time, based on the clarification tree, a legal clarification tree algorithm is proposed to deal with rough and ambiguous legal questions raised by users.

7. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The secretary agent generates an expected consulting report through In-Context Learning based on the ChatGLM-3-6b model fine-tuned by LoRA.

8. According to claim 1, a Chinese legal consultation framework based on large model multi-agent collaboration is characterized in that: The boss agent is a Reward Model.

Images