A legal issue clarification method based on large language model

Abstract

The present invention discloses a method for clarifying legal issues based on a large language model. The present invention first formulates a legal issue clarity evaluation standard based on judicial practice experience, focuses on the core elements of legal requirements and legal claims, and quantifies the clarity of legal issues by evaluating key indicators of completeness, specificity, and clarity; secondly, it constructs a multi-source legal knowledge base, covering multi-source legal knowledge such as laws and regulations, trial elements, judicial cases, historical legal questions and answers, to ensure the comprehensiveness and authority of civil, criminal, administrative and other legal knowledge; then, through multi-library collaboration, it generates a hierarchical and systematic list of detection clarity elements, outputs the legal issue clarity verification results, and formulates a clarification result text evaluation standard, uses a large language model to simulate the full-process dialogue technology of legal questions and answers, and dynamically generates high-quality clarification result texts. The present invention significantly improves the case coverage, accuracy and interpretability of legal issue clarifications.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a legal issue clarification method based on multi-source knowledge base collaboration and large language model driving. Background Art

[0002] Large language model technology is developing rapidly, and its application in the legal field is becoming increasingly in-depth. Although large language models have the advantages of flexibility and wide coverage when generating clarification prompts for legal issues, they still have obvious shortcomings when dealing with complex legal issues. In particular, when clarification prompts for complex issues involving multiple parties, multiple legal fields, and legal gaps, large language models not only lag behind in legal knowledge such as laws, regulations, and trial elements, but also lack a deep understanding of legal knowledge. The clarification prompts generated have problems such as false laws, false case facts, and low accuracy, making it difficult to ensure accuracy and comprehensiveness. Traditional methods mainly rely on manually sorted rules or templates to generate clarification prompts. Although they can deal with common and simple legal issues, they also have problems such as narrow case coverage, low accuracy, and a single source of legal knowledge, making it difficult to meet complex and changing actual needs. Summary of the Invention

[0003] In response to the problems of narrow coverage, low accuracy, single source of legal knowledge, large language model hallucination, and poor interpretability of clarification prompts in the field of legal issues in existing technologies, the present invention aims to provide a legal issue clarification method based on the collaboration of multi-source knowledge bases and driven by a large language model. The method first formulates a legal issue clarity assessment standard based on judicial practice experience, focusing on the core elements of legal requirements and legal claims, and quantifies the clarity of legal issues by evaluating key indicators such as completeness, specificity, and clarity; secondly, it constructs a multi-source legal knowledge base, covering multi-source legal knowledge such as laws and regulations, trial elements, judicial cases, and historical legal questions and answers, to ensure the comprehensiveness and authority of civil, criminal, and administrative legal knowledge. Through the collaboration of multiple databases, a hierarchical and systematic list of clarity detection elements is generated, the clarity verification results of legal issues are output, and a clarification result text evaluation standard is formulated. The large language model is used to simulate the full-process dialogue technology of legal questions and answers, and high-quality clarification result text is dynamically generated. Through this method, we can effectively solve the problems of narrow case coverage, low accuracy, large language model hallucination, and poor interpretability of traditional methods, and significantly improve the coverage of cases for clarification of legal issues, the accuracy and interpretability of complex legal issues, thereby improving the coverage, accuracy and interpretability of cases for clarification of legal issues.

[0004] To address the above issues, the present invention provides a legal issue clarification method based on the collaboration of multiple knowledge bases and driven by a large language model. The main steps include:

[0005] S1: Develop a Criteria for Assessing Legal Issue Clarity: To ensure efficient and accurate clarification of legal issues, develop a criterion for assessing legal issue clarity. This criterion focuses on two core elements of legal issues: the legally required facts and the legal claims. Building upon these two core elements, key indicators for completeness, specificity, and clarity are constructed.

[0006] S2: Constructing a Multi-Source Legal Knowledge Base: This involves collecting and organizing legal key points, identifying legal facts and legal responsibilities, building a legal knowledge association system, detecting and resolving conflicts in related legal knowledge, vectorizing knowledge, and updating it. This multi-source legal knowledge base includes m vector knowledge bases, including legal regulations, trial elements, case law, and historical legal questions and answers.

[0007] S3: Receiving and processing user legal questions: Based on prompt engineering and large language model fine-tuning technology, from the perspective of judicial practice, the legal facts and legal claims of the legal questions q input by the user are identified, and the identified legal facts and legal claims are used to construct a key information set. Based on the identified legal facts and legal claims, query statements are generated to match multi-source legal knowledge bases such as laws and regulations, trial elements, similar cases, and historical legal questions and answers.

[0008] S4: Collaborative generation of a list of elements for detecting the clarity of legal issues by using multiple databases: Based on the query statements generated in step S3, collaborate with legal knowledge bases such as laws and regulations, trial elements, judicial cases, and historical legal questions and answers to generate a hierarchical and systematic list of elements for detecting the clarity of legal issues.

[0009] S5: Outputting Legal Question Clarity Verification Results: Based on the Legal Question Clarity Assessment Criteria, the process compares and analyzes the key information in the user's question against the Clarity Detection Factors List and outputs the Legal Question Clarity Verification Results. Based on a set threshold, the process determines whether the Legal Question Clarity Assessment Criteria are met. If the assessment score P meets the set threshold, the process outputs "Verification Passed, No Clarification Required." Otherwise, the process outputs "Verification Failed, Clarification Required," and proceeds to step S6.

[0010] S6: Multi-database collaborative generation of preliminary clarification result texts: Adopting a hierarchical and progressive strategy, formulate clarification prompt generation rules, construct clarification result text generation prompt instructions, and rely on a large language model to output multiple preliminary clarification result texts of multi-database collaboration.

[0011] S7: Evaluate and filter the preliminary clarification texts to generate the final clarification results: Develop evaluation criteria for clarification texts applicable to the judicial field, focusing on evaluating the clarification texts' alignment with the user's legal issues and legal knowledge, including laws and regulations, trial elements, and adjudication rules, as well as their applicability and explanatory power in specific civil, criminal, and administrative cases. Based on these clarification text evaluation criteria, a large language model will be used to simulate the full legal question-and-answer dialogue process, combined with the experience of legal experts, to screen clarification texts with judicial applicability and generate the final clarification texts.

[0012] Furthermore, in step S1, the main content is to develop clear evaluation criteria for legal issues, which specifically includes the following key sub-steps:

[0013] S11: Establishment of the clarity evaluation standard for legal requirements facts: The clarity evaluation standard for legal requirements facts refers to ensuring the completeness, specificity and clarity of the case facts in the user's legal issues by selecting the legal requirements fact completeness index, the legal requirements fact specificity index and the legal requirements fact clarity index. The completeness of legal requirements facts means that the user's legal issues should include legal requirements facts such as the subject qualifications, the nature of the case, and the impact of rights and obligations; the specificity of legal requirements facts means that the description of legal requirements facts such as the subject qualifications, the nature of the case, and the impact of rights and obligations should be specific and avoid the use of vague expressions; the clarity of legal requirements facts means using accurate and standardized language to describe legal requirements facts to avoid ambiguity or misunderstanding. The calculation formula for the clarity evaluation score of legal requirements facts is:

[0014]

[0015] in, They are the scores for the factual completeness, specificity and clarity of legal requirements, with a score range of 0-5 and an integer.

[0016] S12: Establishing the evaluation standard for the clarity of legal claims: The evaluation standard for the clarity of legal responsibilities is intended to ensure the completeness, specificity, and clarity of the legal claims in user legal issues. The completeness of legal claims means that the user's legal issues should include legal claims such as stopping infringement, removing obstacles, eliminating dangers, returning property, restoring the original state, compensating for losses, and apologizing; the specificity of legal claims refers to the description of legal claims, which should be specific and avoid using vague expressions; the clarity of legal claims refers to the use of accurate and standardized language to describe the expected legal effects to avoid ambiguity or misunderstanding. The calculation formula for the clarity of legal claims evaluation score is:

[0017]

[0018] in, They are the scores for the completeness, specificity and clarity of the legal claim, with a score range of 0-5 and an integer.

[0019] S13: The assessment criteria for clarity of legal issues are scored as follows:

[0020] P=(P a +P b ) / 2.

[0021] Furthermore, the main task of the S2 step is to build a multi-source legal knowledge base to provide highly relevant legal knowledge for clarifying legal issues. It specifically includes the following key steps:

[0022] S21: Collect and organize legal knowledge data: Collect comprehensive and up-to-date data, covering multi-source legal knowledge data such as laws and regulations, trial elements, judicial cases, historical legal questions and answers, etc. Classify and organize the collected legal knowledge data, and divide them according to dimensions such as the formulating authority, release date, effective date, and timeliness to ensure the comprehensiveness, representativeness, and timeliness of the data. Update the data regularly to ensure the continuous updating of the multi-source legal knowledge base. Taking the legal knowledge base as an example, this knowledge base includes different types of laws and regulations such as legal legislative interpretations, judicial interpretations, administrative regulations, departmental regulations, local regulations, autonomous regulations, local government regulations, local judicial documents, local normative documents, legislative materials of the National People's Congress, China's administrative legislative materials, judicial materials of the two high courts, and local judicial materials.

[0023] S22: Extract and generate summary of key legal points: Based on the key legal points summary prompt instruction, using large language model technology, for each piece of legal knowledge in the legal knowledge base, trial element knowledge base, adjudication case knowledge base, and historical legal question and answer knowledge base, the generated content includes the applicable legal problem domain, case type, applicable object, core viewpoint, legal conclusion, etc., which is used to describe the characteristics and application scenarios of different legal knowledge, so as to quickly retrieve and match the user's legal issues. Among them, for different types of legal knowledge, the key legal points summary prompt instruction contains information such as: legal knowledge content (legal content, trial elements, etc.) and key legal points summary prompt corpus.

[0024] S23: Identify legal requirements and legal liabilities: Based on the legal points and legal requirements and legal liabilities prompt instructions generated in step S22, use large language model technology to realize the identification of legal requirements (such as subject qualifications, case nature, rights and obligations, etc.) and legal liabilities (such as breach of contract liability, tort liability, etc.) of legal knowledge, and generate legal requirements and legal liabilities combinations.

[0025] Among them, the instructions on legal requirements, facts and legal responsibilities include legal knowledge content (legal content, trial elements, etc.), legal key points content and legal requirements, facts and legal responsibilities reminder corpus.

[0026] S24: Constructing a Legal Knowledge Association System: Using a clustering algorithm, we cluster the attribute values within each type of legal requirement and legal liability, grouping the attributes to explore potential association patterns, thereby establishing associations between legal knowledge and linking similar legal knowledge related to the same legal requirement or legal liability. When constructing the legal knowledge association system, we use the legal knowledge base as the core foundation, systematically integrating and linking legal knowledge related to the same legal requirement or legal liability from laws and regulations, trial elements, case law, and historical legal questions and answers, thereby forming a complete legal knowledge association system.

[0027] S25: Detecting and Resolving Conflicts in Related Legal Knowledge: First, develop rules for detecting and resolving conflicts in related legal knowledge. Conflict types include pros and cons, scope of superior and subordinate legal entities, deadlines, scope of liability, conditions, legal requirements, and whether legal liability conflicts. Rules for resolving conflicts in related legal knowledge include, but are not limited to, the following: Timeliness priority: The latest legal knowledge takes precedence over older legal knowledge. Authority priority: The authority of the enacting authority is used as the basis. For example, laws and regulations issued by the Supreme People's Court take precedence over judicial documents issued by provinces and cities. Second, based on the related legal knowledge conflict detection and resolution prompt instructions, large language model technology is used to detect conflicts in specific legal knowledge within the legal knowledge association system and output a filtered list of legal knowledge. The related legal knowledge conflict detection and resolution prompt instructions include legal knowledge type, legal knowledge publication date, legal knowledge enacting authority, legal requirements, legal liability, conflict type, resolution rules, and a corpus of related legal knowledge conflict detection and resolution prompts. Finally, develop a solution that combines automatic resolution with manual sampling and review. For clear and direct legal content conflicts, such as when historical legal questions and answers directly contradict legal provisions, the legal knowledge base will automatically be used as the basis for correction or deletion of the conflicting legal knowledge content. For complex conflicts (such as those involving disputes over legal interpretation or application), they will be submitted to legal experts for manual review and resolution.

[0028] S26: Core element combination. The legal key points summary, legal essential facts, and legal responsibilities generated in the legal knowledge base are spliced together, and their sparse vectors and dense vectors are generated and imported into the corresponding knowledge base.

[0029] S27: Importing a vector library: Storing the text, relationships, and vectors from the legal and regulatory collection, trial element collection, adjudication rule collection, and historical legal Q&A into the vector library to construct an external legal vector knowledge base. Optionally, the vector library can be Milvus, Elasticsearch, or other libraries.

[0030] Furthermore, the main task of the S3 step is to receive and process legal issues, and analyze and process the legal issues provided by users, which specifically includes the following key steps:

[0031] S31: Preprocessing of user legal questions: In legal question scenarios, the question content is usually a mixture of multiple case facts, including not only substantive content such as the identity of the parties, the legal relationship of the case, the nature of the case, and the legal claims, but also redundant information and noise such as irrelevant life facts and the personal emotions of the parties. Therefore, the legal question q input by the user must be cleaned and standardized, including removing irrelevant characters, correcting spelling errors, and word segmentation to generate a standardized question q′.

[0032] S32: Semantic classification and key information extraction: Based on the semantic classification and key information extraction prompts, the large language model technology is used to classify and semantically analyze the q′ generated in step S31, determine the legal field to which it belongs, identify and extract the background information, legal facts and legal claims in the legal issue, and construct the key information set C = {c1,…,c l}, where l represents the number of key information extracted. Legal requirements include, but are not limited to, the qualifications of the parties involved, the nature of the case, and facts affecting rights and obligations. Legal demands include, but are not limited to, one or more requirements such as cessation of infringement, removal of obstruction, elimination of danger, return of property, restoration of the original state, compensation for losses, and apology. The extracted background information, legal requirements, and legal demands are combined to form the query statement for the knowledge base.

[0033] The semantic classification and key information extraction prompt instructions include standardized questions q′ and semantic classification and key information extraction prompt corpus.

[0034] Furthermore, the main task of the S4 step is to collaboratively generate a list of factors for detecting the clarity of legal issues by combining multiple databases. This list is generated based on the legal issues input by the user and combines multiple legal knowledge bases. The specific steps include the following:

[0035] S41: Multi-database parallel search: In the multi-source legal vector knowledge base, a hybrid search matching and parallel search method is used to search the query statement to obtain relevant legal knowledge for the user question.

[0036] S42: Integrate and cross-validate search results: Integrate and cross-validate search results from multiple legal knowledge bases to ensure consistency and accuracy. Search results from multiple legal knowledge bases include, but are not limited to, the qualifications of the parties involved, the nature of the law, and the legal requirements and liabilities affecting legal rights and obligations. Results from the legal and regulatory database, the trial elements database, the adjudication case database, and the legal Q&A database are cross-validated and supplemented.

[0037] S43: Generate a list of factors for detecting clarity of legal issues: Based on the above steps S41-S42, a hierarchical and systematic list of factors for detecting clarity of legal issues is generated in combination with the user's legal issues and corresponding background information. where k i represents the number of elements corresponding to the i-th knowledge base, Indicates that the kth legal knowledge base is retrieved from the i-th knowledge base of the multi-source legal knowledge base. i Elements, m is the total number of knowledge bases in the multi-source legal knowledge base. The detection list includes detection elements such as the subject qualifications of the parties, the legal nature of the case, and the legal requirements, facts, and legal responsibilities that affect legal rights and obligations.

[0038] Furthermore, the main task of step S5 is to compare and analyze the key information C in the user's question with the clarity checklist D based on the legal question clarity assessment standard, and output the legal question clarity verification result, which specifically includes the following key steps:

[0039] S51: Evaluate the clarity of the user's question and output missing elements. Based on the legal question clarity assessment criteria, construct clarity assessment prompts. Utilizing a large language model, output scores for the completeness, specificity, and clarity of the legal requirements and claims. Output a list of missing legal requirements and claims, denoted as D′. Output a clarity score based on the calculation formula from step S1.

[0040] The clarity assessment prompt instruction includes the legal question clarity assessment criteria, key information C in the user question, a clarity detection checklist D and the assessment prompt corpus.

[0041] S52: Set a threshold and output a conclusion: If the clarity score exceeds the threshold, output "verification passed, no clarification required". Otherwise, output "verification failed, clarification required", and proceed to step S6.

[0042] Furthermore, the main task of step S6 is to use a multi-library collaborative strategy to clarify the user's question and generate a preliminary clarification result text, which specifically includes the following key steps:

[0043] S61: Formulate rules for generating clarification prompts: When generating clarification prompts, since D′ contains different types of elements such as the legal and regulatory database, the trial element database, the case database, and the historical legal question and answer database, the elements corresponding to the legal and regulatory database in D′ are prioritized to generate clarification prompts and explanations. Then, based on the score of the legal question clarity assessment standard, it is determined whether clarification is still needed. If so, additional clarification prompts and explanations are generated based on the corresponding elements in D′, such as the trial element database, the case database, and the historical legal question database. The explanations of the clarification prompts should be clear and accurate, and help users understand the legal basis and reasoning process of the results. Finally, all clarification prompts and the user's original question are combined to output K clarification result texts.

[0044] S62: Generate clarification result text based on multi-library collaboration: According to the established rules for generating clarification prompts, construct prompt instructions for generating clarification result text, and rely on the large language model to achieve multi-library collaborative clarification result text generation.

[0045] The prompt instruction for generating the clarification result text includes the user's legal question, the missing element list D′, the clarification prompt generation rule, and the prompt corpus for generating the clarification result text.

[0046] Furthermore, step S7 mainly involves evaluating and filtering the initial clarification result text, using a large language model to simulate the full-process legal question-answering dialogue technology, and generating the final version of the clarification result text. This specifically includes the following key sub-steps:

[0047] S71: Simulating User Answers: According to step S62, each candidate clarification result text regarding the legal question input by the user is obtained. For each of the above initial clarification result texts, the large language model simulates the user's possible subsequent answers.

[0048] S72: Predicting the dialogue path for legal issues: Based on the dialogue completed in step S71, the large language model predicts the dialogue path for subsequent legal issues and generates detailed answers that comply with laws, regulations, judicial interpretations, and adjudication rules, ensuring that the dialogue logic is rigorous and has judicial applicability.

[0049] S73: Develop evaluation criteria for clarification result texts generated by evaluation filtering: Develop evaluation criteria for clarification result texts applicable to the judicial field, focusing on assessing the alignment of the initial clarification result text with the user's legal question (q) and legal knowledge, including laws, regulations, trial elements, and adjudication rules, as found in a multi-source legal knowledge base. This evaluation criteria also includes five core indicators: completeness of the clarification result text, clarity of the clarification result text, consistency of the clarification result text, logic of the clarification result text, and practicality of the clarification result text. Completeness of the clarification result text refers to whether it effectively supplements the missing legal elements, facts, or legal claims in the user's legal question; clarity of the clarification result text refers to accurately resolving ambiguity or ambiguity regarding legal concepts, application of provisions, or factual findings in the user's question; consistency of the clarification result text refers to its close alignment with relevant laws, regulations, judicial interpretations, and adjudication rules; logic of the clarification result text refers to its formation of a logically rigorous analytical chain based on legal reasoning methods; and practicality of the clarification result text refers to whether it provides substantial assistance to the user in understanding the legal issue, clarifying rights and obligations, and taking legal action.

[0050] S74: Evaluate the quality score of the clarification result text: Based on the five core indicators of the clarification result text evaluation criteria (completeness, clarity, fit, logic, and practicality), combined with the answer generated in step S72, based on the clarification result text evaluation criteria scoring prompt instructions, using large language model technology, output the five scores corresponding to each candidate clarification result text and calculate the final score:

[0051] R=(s1+s2+s3+s4+s5) / (5×5)

[0052] Among them, s1, s2, s3, s4, and s5 represent the completeness of the clarification result text, the clarity of the clarification result text, the consistency of the clarification result text, the logic of the clarification result text, and the practicality of the clarification result text, respectively, and the value range is 0 to 5.

[0053] S75: Filter the preliminary clarification result text and generate the final clarification result text: Based on the experience of legal experts, set a threshold for the quality score of the clarification result text, and filter it to ensure that only the clarification result text with judicial applicability value and meeting the five core indicators is selected.

[0054] S76: Output high-quality clarification result text: Output the final screened clarification result text to the user to ensure that it can effectively guide the user to clarify legal issues and provide substantial assistance for subsequent legal consultation or actions.

[0055] The present invention also provides a server, characterized in that it includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing the above method.

[0056] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above method when executed by a processor.

[0057] The advantages of the present invention are as follows:

[0058] The present invention provides a legal issue clarification method based on the collaboration of multi-source knowledge bases and driven by a large language model, which has the following advantages: first, based on judicial practice experience, a rigorous legal issue clarity assessment standard is formulated, the legal requirements and facts and the core elements of legal claims in legal issues are clarified, key indicators of completeness, specificity and clarity are proposed, the clarity of legal issues is quantitatively evaluated, a clear basis is provided for the clarification of legal issues, and standardization and operability are enhanced; second, an efficient and reliable multi-source legal knowledge base conflict detection and resolution mechanism is constructed, with priority given to the legal and regulatory knowledge base, combining automatic resolution with manual review to ensure the authority and consistency of legal knowledge, and improve accuracy and reliability; third, the multi-source legal knowledge base (covering laws and regulations, trial elements, judicial cases and historical legal questions and answers) is combined with a large language model to automatically and efficiently generate a hierarchical and systematic clarification prompt detection element list, which not only solves the problems of narrow coverage and few legal issue clarity detection points caused by the traditional method relying on manual combing, but also solves the problems of illusion, low accuracy and time lag when generating clarification prompts by the large language model, significantly improving the comprehensiveness and accuracy of legal issue clarification. Fourth, a hierarchical and progressive prompt generation strategy is adopted, with preliminary prompts generated based on the legal and regulatory knowledge base as a priority, and supplemented by other knowledge bases to ensure the accuracy and practicality of the preliminary clarification result text. Fifth, standards for evaluating and filtering clarification result texts are formulated, and large language models are used to simulate the full-process dialogue technology of legal question-and-answering, to scientifically evaluate the comprehensiveness and effectiveness of the clarification result texts. In summary, the present invention has significant advantages in legal issue clarity evaluation standards, legal knowledge base conflict resolution, legal knowledge base collaboration, legal issue clarity detection list generation, clarification prompt generation, and clarification result text evaluation, which comprehensively improves the accuracy, coverage, and practicality of legal issue clarification. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flowchart for specific implementation.

[0060] Figure 2 A flowchart for generating a list of elements for legal issue clarity detection for multi-database collaboration. DETAILED DESCRIPTION

[0061] To further illustrate the technical solution of the present invention, the present invention is described in further detail below with reference to the accompanying drawings. It is apparent that the embodiments described herein are only a portion of the embodiments of this application, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0062] The specific implementation flow chart of the present invention is shown in the attached figure. Figure 1 As shown, a legal issue clarification method based on the collaboration of multi-source knowledge bases and driven by a large language model is described in detail:

[0063] S1: Develop a legal issue clarity assessment standard: To achieve efficient and accurate clarification of legal issues, develop a legal issue clarity assessment standard. This assessment standard focuses on two core elements of legal issues: legal facts and legal claims. Based on these two core elements, key indicators of completeness, specificity, and clarity are constructed, including the following key steps:

[0064] (1) Formulate the evaluation standard of the clarity of legal requirements: The evaluation standard of the clarity of legal requirements refers to ensuring the completeness, specificity and clarity of the case facts in the user's legal issues by selecting the completeness index of legal requirements, the specificity index of legal requirements and the clarity index of legal requirements. The completeness of legal requirements means that the legal issues of users should include legal requirements such as the qualifications of the subject, the nature of the case, and the impact of rights and obligations; the specificity of legal requirements means that the description of legal requirements such as the qualifications of the subject, the nature of the case, and the impact of rights and obligations should be specific and avoid the use of vague expressions; the clarity of legal requirements means that accurate and standardized language should be used to describe legal requirements to avoid ambiguity or misunderstanding. The calculation formula for the clarity evaluation score of legal requirements is:

[0065]

[0066] in, They are the scores for the factual completeness, specificity and clarity of legal requirements, with a score range of 0-5 and an integer.

[0067] (2) Establishing a standard for assessing the clarity of legal claims: The standard for assessing the clarity of legal responsibilities aims to ensure the completeness, specificity, and clarity of the legal claims in user legal issues. The completeness of legal claims means that user legal issues should include legal claims such as stopping infringement, removing obstacles, eliminating dangers, returning property, restoring the original state, compensating for losses, and apologizing; the specificity of legal claims refers to the description of legal claims, which should be specific and avoid using vague expressions; the clarity of legal claims refers to the use of accurate and standardized language to describe the expected legal effects to avoid ambiguity or misunderstanding. The formula for calculating the score for the clarity of legal claims assessment is:

[0068]

[0069] in, They are the scores for the completeness, specificity and clarity of the legal claim, with a score range of 0-5 and an integer.

[0070] (3) The evaluation criteria for clarity of legal issues are:

[0071] P=(P a +P b ) / 2

[0072] S2: Constructing a multi-source legal knowledge base: This involves collecting and organizing legal key points, identifying legal facts and legal responsibilities, building a legal knowledge association system, detecting and resolving conflicts in related legal knowledge, vectorizing knowledge, and updating it. The multi-source legal knowledge base mentioned above includes four vector knowledge bases: laws and regulations, trial elements, case law, and historical legal questions and answers. Specifically, it includes the following key steps:

[0073] (1) Collect and organize legal knowledge data: Collect comprehensive and up-to-date data, covering multiple sources of legal knowledge data, including laws and regulations, trial elements, adjudication cases, historical legal questions and answers, etc. Categorize and organize the collected legal knowledge data, dividing them by dimensions such as formulating authority, release date, effective date, and timeliness to ensure the comprehensiveness, representativeness, and timeliness of the data. Update the data regularly to ensure the continuous updating of the multi-source legal knowledge database.

[0074] (2) Extract and generate summary of key legal points: Based on the summary prompt instructions of key legal points, see Table 1, the large language model technology is used to generate the applicable legal problem field, case type, applicable object, core viewpoint, legal conclusion and other contents for each piece of legal knowledge in the legal knowledge base, trial element knowledge base, judgment case knowledge base, and historical legal question and answer knowledge base. These contents are used to describe the characteristics and application scenarios of different legal knowledge, so as to quickly retrieve and match the user's legal problems.

[0075] Table 1: Summary of legal points and instructions

[0076]

[0077]

[0078]

[0079] (3) Identification of legal essential facts and legal responsibilities: Based on the generated legal points and legal essential facts and legal responsibility prompt instructions (see Table 2), the large language model technology is used to realize the identification of legal essential facts of legal knowledge (such as subject qualifications, case nature, rights and obligations, etc.) and legal responsibilities (such as breach of contract liability, tort liability, etc.), and generate legal essential facts combinations and legal responsibility combinations.

[0080] Table 2: Legal requirements and legal liability instructions

[0081]

[0082]

[0083] (4) Constructing a legal knowledge association system: Using a clustering algorithm, cluster the attribute values of each type of legal essential facts and legal responsibilities, realize attribute grouping, and explore potential association patterns, thereby establishing association relationships between legal knowledge and associating similar legal knowledge related to the same legal essential facts or legal responsibilities. When constructing a legal knowledge association system, the legal knowledge base of laws and regulations is used as the core basis, and the legal knowledge related to the same legal essential facts or legal responsibilities in laws and regulations, trial elements, judicial cases, and historical legal questions and answers is systematically integrated and associated, thus forming a complete legal knowledge association system.

[0084] (5) Detection and resolution of conflict points in related legal knowledge: First, formulate rules for detecting and resolving conflict points in related legal knowledge. The types of conflict in related legal knowledge include dimensions such as positive and negative views, scope of superior and inferior legal subjects, time limit, scope of liability, conditions, legal essential facts, and whether legal liability conflicts; the rules for resolving conflict points in related legal knowledge include but are not limited to the following: Timeliness priority: the latest legal knowledge takes precedence over old legal knowledge. Authority priority: based on the authority of the enacting authority, for example, the laws and regulations issued by the Supreme People's Court are superior to the judicial documents issued by various provinces and cities. Secondly, based on the detection and resolution prompt instructions for related legal knowledge conflicts, see Table 3, use large language model technology to detect conflict points in related legal knowledge and output a list of filtered legal knowledge. Among them, the detection and resolution prompt instructions for related legal knowledge conflicts include legal knowledge type, legal knowledge release date, legal knowledge enacting authority, legal essential facts, legal liability, conflict type, resolution rules, and related legal knowledge conflict point detection and resolution prompt corpus. Finally, formulate a solution that combines automatic resolution with manual sampling review. For clear and direct legal content conflicts, such as when historical legal questions and answers directly contradict legal provisions, the legal knowledge base will automatically be used as the basis for correction or deletion of the conflicting legal knowledge content. For complex conflicts (such as those involving disputes over legal interpretation or application), they will be submitted to legal experts for manual review and resolution.

[0085] Table 3: Detection and resolution instructions for conflict points in related legal knowledge

[0086]

[0087]

[0088] (6) Construct four types of knowledge bases. Based on the above steps, the multi-source legal knowledge base is constructed, and the main components of each knowledge base are:

[0089] The components of the legal and regulatory knowledge base include the enacting authority, promulgation date, implementation date, timeliness, effectiveness level, subject classification, legal issue field, case type, applicable objects, core viewpoints, legal conclusions, legal essential facts, legal liability, etc.

[0090] Components of the trial elements knowledge base. The trial elements knowledge base includes the formulating agency, promulgation date, legal issue field, case type, applicable objects, legal essential facts, legal liability, etc.

[0091] The components of the adjudication rules knowledge base include the source of the published case, case number, trial procedure, adjudication results, legal issue areas, case type, applicable objects, core viewpoints, legal essential facts, legal responsibilities, etc.

[0092] The components of the historical legal Q&A database include the source of the legal Q&A, publication date, author, legal issue area, case type, applicable object, core viewpoint, legal conclusion, legal essential facts, legal liability, etc.

[0093] (7) Core element combination. The legal key points summary, legal essential facts and legal responsibilities generated in the legal knowledge base are spliced together, and their sparse vectors and dense vectors are generated and imported into the corresponding knowledge base.

[0094] (8) Importing the vector library: storing the text content, association relationships and vectors of legal and regulatory knowledge, trial element knowledge, adjudication rule knowledge, and historical legal questions and answers into the milvus vector library to build an external legal vector knowledge base.

[0095] S3: Receiving and processing user legal questions: Based on prompt engineering and large language model fine-tuning technology, from the perspective of judicial practice, the legal facts and legal claims of the legal question q entered by the user are identified. The identified legal facts and legal claims are used to construct a key information set. Based on the identified legal facts and legal claims, query statements are generated based on the legal knowledge base, such as matching laws and regulations, trial elements, similar cases, and historical legal questions and answers. The specific steps include the following:

[0096] (1) Preprocessing of user legal questions: In legal question scenarios, the question content is usually a mixture of multiple case facts, including not only substantive content such as the identity of the parties, the legal relationship of the case, the nature of the case, and the legal claims, but also redundant information and noise such as irrelevant life facts and the personal emotions of the parties. Therefore, the legal question q input by the user needs to be cleaned and standardized, including removing irrelevant characters, correcting spelling errors, and segmenting words to generate a standardized question q′.

[0097] (2) Semantic classification and key information extraction: Based on the semantic classification and key information extraction prompt instructions (see Table 4), the large language model technology is used to classify and semantically analyze the q′ generated in the above steps to determine the legal field to which it belongs, identify and extract the background information, legal requirements and legal claims in the legal issues, and construct the key information set C = {c1,…,c l}, where l represents the number of key information extracted. Legal requirements include, but are not limited to, the qualifications of the parties involved, the nature of the case, and facts affecting rights and obligations. Legal demands include, but are not limited to, one or more requirements such as cessation of infringement, removal of obstruction, elimination of danger, return of property, restoration of the original state, compensation for losses, and apology. The extracted background information, legal requirements, and legal demands are combined to form the query statement for the knowledge base.

[0098] Table 4: Semantic classification and key information extraction prompt instructions

[0099]

[0100]

[0101] S4: Collaborative generation of a legal issue clarity checklist using multiple databases: Based on the query statements generated in the above steps, we collaborate with legal knowledge bases such as laws and regulations, trial elements, case law, and historical legal questions and answers to generate a hierarchical and systematic list of legal issue clarity checklists. This includes the following key steps:

[0102] (1) Multi-database parallel search: In the multi-source legal knowledge base, a hybrid search matching and parallel search method is used to search the search query statement to obtain the relevant legal knowledge of the user's question.

[0103] (2) Integration and cross-validation of search results: The search results of multiple legal knowledge bases are integrated and cross-validated to ensure the consistency and accuracy of the information. The search results of multiple legal knowledge bases include but are not limited to the qualifications of the parties, the nature of the law, and the legal facts and legal responsibilities that affect legal rights and obligations. The results of the legal and regulatory database, the trial elements knowledge base, the judicial case knowledge base, and the legal question and answer knowledge base are mutually verified and supplemented.

[0104] (3) Generate a list of factors for detecting the clarity of legal issues: Based on the above steps, a hierarchical and systematic list of factors for detecting the clarity of legal issues is generated in combination with the user's legal issues and corresponding background information. where k i Represents the number of elements corresponding to the i-th knowledge base. The detection list includes detection elements such as the subject qualifications of the parties, the legal nature of the case, and the legal requirements, facts, and legal responsibilities that affect legal rights and obligations.

[0105] S5: Output legal question clarity verification results: Based on the legal question clarity assessment criteria, compare and analyze the key information C in the user's question with the clarity detection factor list D, and output the legal question clarity verification results. Based on the set threshold, determine whether the legal question clarity assessment criteria are met. If so, output "Verification passed, no clarification required." Otherwise, output "Verification failed, clarification required," and proceed to step S6. Specifically, the following key steps are included:

[0106] (1) Evaluate the clarity of the user's question and output missing elements. Based on the legal question clarity evaluation standard, construct a clarity evaluation prompt instruction (see Table 5). Using the large language model, output the completeness, specificity, and clarity scores of the legal requirements and legal claims, and output a list of missing legal requirements and legal claims, denoted as D′. Based on the legal question clarity calculation formula, output the clarity score.

[0107] Table 5: Clarity Assessment Prompt Instructions

[0108]

[0109]

[0110] (2) Set a threshold and output a conclusion: When the clarity score exceeds the threshold, output “verification passed, no clarification required”. Otherwise, output “verification failed, clarification required” and proceed to step S6.

[0111] S6: Multi-database collaborative generation of preliminary clarification result texts: Using a hierarchical and progressive strategy, we formulate clarification prompt generation rules, build preliminary clarification result text prompt instructions, and rely on a large language model to generate multiple preliminary clarification result texts from multi-database collaboration. This includes the following key steps:

[0112] (1) Formulate rules for generating clarification prompts: When generating clarification prompts, since D′ contains different types of elements such as the legal and regulatory database, trial element database, judicial case database, and historical legal question and answer database, the elements corresponding to the legal and regulatory knowledge database in D′ are used first to generate clarification prompts and explanations. Then, based on the legal question clarity assessment standard, it is judged whether clarification is still needed. If so, additional clarification prompts and explanations are generated based on the elements corresponding to the trial element, judicial case database, historical legal question and answer database, and other knowledge databases in D′. The explanations of the clarification prompts should be clear and accurate, and be able to help users understand the legal basis and reasoning process of the results. Finally, all clarification prompts and the user's original questions are combined to output K clarification result texts.

[0113] (2) Generate the initial clarification result text based on multi-database collaboration: According to the established rules for generating clarification prompts, construct prompt instructions for the initial clarification result text, see Table 6, and rely on the large language model to achieve multi-database collaborative clarification result text generation.

[0114] Table 6: Prompt instructions for clarifying result text

[0115]

[0116]

[0117]

[0118] S7: Evaluate and filter the initial clarification result text to generate the final version of the clarification result text: Develop clarification prompt evaluation standards applicable to the judicial field, focusing on evaluating the consistency of the initial clarification results with the user's legal issues and legal knowledge such as laws and regulations, trial elements and judgment rules, as well as their applicability and explanatory power in specific civil, criminal, administrative and other types of cases. According to the above evaluation standards for the initial clarification result text, use a large language model to simulate the full-process dialogue technology of legal question-and-answering, combined with the experience of legal experts, screen clarification result texts with judicial applicability value, and generate the final version of the clarification result text. Specifically, it includes the following key sub-steps:

[0119] (1) Simulating User Answers: Based on the above steps, each candidate clarification result text regarding the legal question input by the user is obtained. For each of the above initial clarification result texts, the large language model simulates the user's possible subsequent answers.

[0120] (2) Predicting the dialogue path of legal issues: Based on the dialogue completed in the above steps, the large language model predicts the dialogue path of subsequent legal issues and generates detailed answers that comply with laws, regulations, judicial interpretations and adjudication rules, ensuring that the dialogue logic is rigorous and has judicial applicability value.

[0121] (3) Formulate evaluation criteria for clarification result texts: Formulate evaluation criteria for clarification result texts applicable to the judicial field, focusing on evaluating the consistency of the clarification result texts with the user's legal questions and legal knowledge such as laws, regulations, trial elements and adjudication rules in the multi-source legal knowledge base, as well as their applicability and explanatory power in specific civil, criminal, administrative and other cases. This evaluation standard focuses on five core indicators, namely the completeness of the clarification result text, the clarity of the clarification result text, the consistency of the clarification result text, the logic of the clarification result text, and the practicality of the clarification result text. Among them, the completeness of the clarification result text refers to whether it effectively supplements the missing legal elements, facts or legal claims in the user's legal questions; the clarity of the clarification result text refers to accurately eliminating the ambiguity or ambiguity in the legal concepts, application of provisions or fact determination involved in the user's questions; the consistency of the clarification result text refers to closely matching relevant laws, regulations, judicial interpretations and judicial adjudication rules; the logic of the clarification result text refers to forming a logically rigorous analysis chain based on legal reasoning methods; and the practicality of the clarification result text refers to whether it provides substantial help to users in understanding legal issues, clarifying rights and obligations, and taking legal actions.

[0122] (4) Evaluate the quality score of the clarification result text: Based on the five core indicators of the clarification result text evaluation criteria (completeness, clarity, fit, logic, and practicality), combined with the answers generated in the above steps, and based on the scoring prompt instructions of the clarification result text evaluation criteria, see Table 7, use the large language model technology to output the five scores corresponding to each candidate clarification result text and calculate the final score:

[0123] R=(s1+s2+s3+s4+s5) / (5×5)

[0124] Among them, s1, s2, s3, s4, and s5 represent the completeness of the clarification result text, the clarity of the clarification result text, the consistency of the clarification result text, the logic of the clarification result text, and the practicality of the clarification result text, respectively.

[0125] Table 7: Clarification of the results of the text assessment criteria scoring instructions

[0126]

[0127]

[0128] (5) Filter the preliminary clarification result text and generate the final clarification result text: Based on the experience of legal experts, set a threshold for the quality score of the clarification result text, and filter it to ensure that only clarification prompts with judicial applicability and meeting the five core indicators are selected.

[0129] (6) Output high-quality clarification result text: The final screened clarification result text is output to the user to ensure that it can effectively guide the user to clarify legal issues and provide substantial assistance for subsequent legal consultation or action.

[0130] The present invention also provides a server, characterized in that it includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing the above method.

[0131] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above method when executed by a processor.

[0132] It should be noted that the above description is merely an embodiment of the present invention and the accompanying drawings, which are intended to provide a better understanding of the present invention and are not intended to limit the present invention. It will be apparent to those skilled in the art that various embodiments of the present invention may be employed. Various substitutions, variations, and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the disclosed contents of the preferred embodiments, and the scope of protection claimed in the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for clarifying legal issues based on a large language model, comprising the following steps: 1) Select indicators related to the legal facts and core elements of legal claims in legal issues to construct a legal issue clarity assessment standard, which is used to calculate the legal issue clarity assessment standard score; 2) Constructing a multi-source legal knowledge base, which includes a legal and regulatory vector knowledge base, a trial element vector knowledge base, a judicial case vector knowledge base, and a historical legal question-and-answer vector knowledge base; 3) Identifying the legal facts and legal claims of the legal question q input by the user based on the large language model, and generating a query statement for querying the multi-source legal knowledge base based on the identified legal facts and legal claims; 4) querying the multi-source legal knowledge base based on the query statement, and generating a list of legal issue clarity detection factors based on the query results; 5) Based on the legal issue clarity assessment criteria, compare and analyze the legal essential facts and legal claims identified from the legal issue q with the clarity detection element list, and output an assessment result of the legal issue clarity; If the evaluation standard score P is greater than the set threshold, the verification is passed and no clarification is required; otherwise, proceed to step 6); 6) Constructing a clarification result text generation prompt instruction according to the set clarification prompt generation rules and inputting it into the large language model to output several preliminary clarification result texts; 7) Based on the established clarification result text evaluation criteria, a large language model is used to simulate the full-process legal question-and-answer dialogue technology and, combined with the experience of legal experts, clarification result texts with judicial applicability are screened from the preliminary clarification result texts to generate the final version of the clarification result text.

2. The method according to claim 1, characterized in that The legal issue clarity assessment criteria include the legal element fact clarity assessment criteria and the legal claim clarity assessment criteria. The method for calculating the assessment criteria score P is: 11) Based on the selected legal requirement fact completeness index, legal requirement fact specificity index and legal requirement fact clarity index, formulate legal requirement fact clarity assessment standards to calculate the legal requirement fact clarity assessment score in, The scores are respectively the completeness index of legal requirements facts, the specificity index of legal requirements facts and the clarity index of legal requirements facts. The value range of is 0~N; 12) Based on the selected legal claim completeness indicators, legal claim specificity indicators, and legal claim clarity indicators, formulate legal claim clarity assessment criteria to calculate the legal claim clarity assessment score. in, The scores of the legal claim completeness index, legal claim specificity index and legal claim clarity index are respectively. The value range of is 0~M; 13) Calculate the score of the legal clarity assessment criteria P = (P a +P b ) / 2.

3. The method according to claim 1, characterized in that The method for constructing the multi-source legal knowledge base is: 21) Collect legal knowledge data covering laws and regulations, trial elements, adjudication cases, and historical legal questions and answers; categorize and organize the collected legal knowledge data; divide it by enacting authority, promulgation date, effective date, and timeliness; and generate a knowledge base of laws and regulations, a knowledge base of trial elements, a knowledge base of adjudication cases, and a knowledge base of historical legal questions and answers; 22) Based on the legal points summary prompt instruction, a large language model is used to generate the legal points of each legal knowledge data in the legal and regulatory knowledge base, trial element knowledge base, adjudication case knowledge base, and historical legal question and answer knowledge base; 23) Based on the legal points, legal facts and legal liability prompt instructions, use the large language model to identify the legal facts and legal liabilities in each piece of legal knowledge data; 24) Linking legal knowledge related to the same legal elements, facts, or legal responsibilities in laws and regulations, trial elements, adjudication cases, and historical legal questions and answers to form a legal knowledge linkage system; 25) Based on the detection and resolution prompts of associated legal knowledge conflicts, a large language model is used to detect and correct conflicts in the associated legal knowledge system; 26) Splicing the legal points, legal facts and legal responsibilities of each piece of legal knowledge data, and generating sparse vectors and dense vectors of the corresponding legal knowledge data; 27) Each law and regulation in the laws and regulations knowledge base and its sparse vectors, dense vectors, and the relationship between laws and regulations are stored in the vector library to obtain the laws and regulations vector knowledge base; each trial element in the trial element knowledge base and its sparse vectors, dense vectors, and the relationship between trial elements are stored in the vector library to obtain the trial element vector knowledge base; each adjudication case in the adjudication case knowledge base and its sparse vectors, dense vectors, and the relationship between adjudication cases are stored in the vector library to obtain the adjudication case vector knowledge base; each historical legal question and answer in the historical legal question and answer knowledge base and its sparse vectors, dense vectors, and the relationship between historical legal questions and answers are stored in the vector library to obtain the historical legal question and answer vector knowledge base.

4. The method according to claim 1, 2 or 3, characterized in that: The method for generating a query statement for querying the multi-source legal knowledge base is: 31) Clean and standardize the legal text of the legal question q input by the user to generate a standardized question q′; 32) Based on semantic classification and key information extraction prompt instructions, a large language model is used to classify and semantically analyze the standardized question q′, determine the legal field to which it belongs, identify and extract background information, legal facts and legal claims in the legal issue as key information, and obtain a key information set C; the extracted background information, legal facts and legal claims are spliced as a query statement for querying the multi-source legal knowledge base.

5. The method according to claim 4, characterized in that The method for generating the above-mentioned list of elements for detecting clarity of legal issues is as follows: 41) searching the multi-source legal knowledge base according to the query statement to obtain relevant legal knowledge; 42) Integrate and cross-validate search results to ensure consistency and accuracy of information; 43) Combine the user's legal issues and corresponding background information to generate a hierarchical and systematic list of legal issue clarity detection factors in, Indicates that the kth legal knowledge base is retrieved from the i-th knowledge base of the multi-source legal knowledge base. i elements, and m is the total number of knowledge bases in the multi-source legal knowledge base.

6. The method according to claim 5, characterized in that The method to obtain the evaluation score P for the clarity of legal issues is: 51) Based on the legal issue clarity assessment standard, a clarity assessment prompt instruction is constructed and input into the large language model to obtain the legal fact clarity assessment score P a and Legal Claim Clarity Assessment Score P b , and output the missing legal facts and legal claims list D′; 52) Calculate the score of the legal clarity assessment standard P = (P a +P b ) / 2.

7. The method according to claim 6, characterized in that The method for obtaining the preliminary clarification result text is: 61) Generate clarification prompts and explanations based on the legal and regulatory elements in the missing legal facts and legal claims list D′, then determine whether clarification is still needed based on the legal question clarity assessment standard. If so, generate supplementary clarification prompts and explanations based on the corresponding elements of the trial elements, judicial cases, and historical legal questions and answers in the missing legal facts and legal claims list D′; combine all clarification prompts and legal questions q, and output K clarification result texts; 62) Construct prompt instructions for generating clarification result texts according to the established rules for generating clarification prompts, and obtain a number of the aforementioned preliminary clarification result texts based on the large language model.

8. The method according to claim 1, characterized in that The method to generate the final version of the clarified results text is: 71) For each preliminary clarification result text, use the large language model to simulate the user's answer; 72) Based on the dialogue in step 71), the large language model is used to predict the subsequent dialogue path of legal issues and generate answers that comply with laws, regulations, judicial interpretations, and adjudication rules; 73) Develop evaluation criteria for clarification texts applicable to the judicial field, to evaluate the consistency of the preliminary clarification text with the legal question q, the laws and regulations, trial elements, and adjudication rules in the multi-source legal knowledge base, including criteria for completeness (s1), clarity (s2), consistency (s3), logic (s4), and practicality (s5). 74) Based on the evaluation results of step 73) and the answer generated in step 72), and based on the clarification result text evaluation standard scoring prompt instruction, the large language model technology is used to calculate the clarification result text completeness s1, the clarification result text clarity s2, the clarification result text fit s3, the clarification result text logic s4, and the clarification result text practicality s5 of the preliminary clarification result text, and the score of the preliminary clarification result text is calculated as R = (s1 + s2 + s3 + s4 + s5) / (5 × Q); the value range of s1, s2, s3, s4, and s5 is 0 to Q; 75) The quality threshold of the clarification result text is set based on the experience of legal experts, and the preliminary clarification result text with a score R greater than the quality threshold of the clarification result text is retained as the final version of the clarification result text.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images