Legal problem clarification method based on large language model driving

By adopting multi-source knowledge base synergy and large language model-driven methods in the field of legal issue clarification, the problems of narrow coverage, low accuracy and poor interpretability of legal issue clarification prompts in the prior art are solved, and more efficient and accurate clarification of legal issues are achieved.

CN120144703AActive Publication Date: 2025-06-13中国司法大数据研究院有限公司
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510177728.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-02-18
Publication Date
2025-06-13
Estimated Expiration
2045-02-18

AI Technical Summary

Technical Problem

The clarification prompts of the prior art in the field of legal issues are caused by narrow coverage, low accuracy, single source of legal knowledge, and large language models. There are problems of hallucination, low accuracy and poor interpretability.

Method used

A legal problem clarification method is adopted based on the coordination of multi-source knowledge bases and large language models, and a multi-source legal knowledge base is built by formulating legal problem clarity evaluation standards, including laws and regulations, trial elements, judgment cases and historical legal Q&A. A large language model is used to simulate the full-process dialogue technology of legal Q&A, and dynamically generate high-quality clarification result texts.

Benefits of technology

It significantly improves the case coverage of legal issues and the accuracy and interpretability of complex legal issues, ensuring the accuracy and comprehensiveness of clarification results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120144703A_ABST
    Figure CN120144703A_ABST
Patent Text Reader

Abstract

The invention discloses a legal problem clarification method based on large language model driving. The method comprises the following steps: firstly, formulating a legal problem definition evaluation standard based on judicial practice experience, focusing on legal essential facts and legal appeal core elements, and quantifying legal problem definition by evaluating integrity, specificity and definition key indexes; secondly, a multi-source legal knowledge base is constructed, multi-source legal knowledge such as laws and regulations, judgment elements, judgment cases and historical legal questions and answers are covered, and comprehensiveness and authority of the legal knowledge such as civil affairs, criminal affairs and administrative affairs are ensured; then, a hierarchical and systematic detection definition element list is generated through multi-library collaboration, a legal question definition verification result is output, a clarification result text evaluation standard is formulated, a large language model is used for simulating a legal question and answer whole-process dialogue technology, and a high-quality clarification result text is dynamically generated. According to the method, the action coverage, the accuracy and the interpretability of law problem clarification are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular, to a method for clarifying legal issues based on the collaboration of multi-source knowledge bases and the drive of large language models. Background Art

[0002] With the rapid development of large language model technology, its application in the legal field has gradually deepened. Although large language models have the advantages of flexibility and wide coverage when generating clarification prompts for legal issues, there are still obvious deficiencies in dealing with complex legal issues. Especially when generating clarification prompts for complex issues such as cases involving multiple parties, multiple legal fields, and legal blank scenarios, large language models not only have the lag of legal knowledge such as laws and regulations and trial elements, but also lack a deep understanding of legal knowledge. The generated clarification prompts have problems such as false laws, false case facts, and low accuracy, making it difficult to ensure accuracy and comprehensiveness. Traditional methods mainly rely on manually sorted rules or templates to generate clarification prompts. Although they can handle common simple legal issues, they also have problems such as narrow case coverage, low accuracy, and single source of legal knowledge, and are difficult to meet the actual needs of complex and changeable situations. Summary of the Invention

[0003] Aiming at the problems of narrow case coverage, low accuracy, single source of legal knowledge, large language model hallucinations, and poor interpretability of the existing technology in the field of legal issue clarification prompts, the present invention aims to provide a method for clarifying legal issues based on the collaboration of multi-source knowledge bases and the drive of large language models. This method first formulates an evaluation standard for the clarity of legal issues based on judicial practice experience, focuses on the key elements of legal requirements facts and legal claims, and quantifies the clarity of legal issues by evaluating key indicators such as integrity, specificity, and clarity. Secondly, a multi-source legal knowledge base is constructed, covering multi-source legal knowledge such as laws and regulations, trial elements, judgment cases, and historical legal Q&A, ensuring the comprehensiveness and authority of legal knowledge in civil, criminal, administrative, etc. Through the collaboration of multiple knowledge bases, a hierarchical and systematic checklist of clarity detection elements is generated, the verification result of the clarity of legal issues is output, and an evaluation standard for the clarification result text is formulated. The full process dialogue technology of legal Q&A is simulated using a large language model to dynamically generate a high-quality clarification result text. Through this method, the problems of narrow case coverage, low accuracy of traditional methods, as well as large language model hallucinations and poor interpretability can be effectively solved, significantly improving the case coverage, accuracy, and interpretability of complex legal issues in legal issue clarification, thereby improving the case coverage, accuracy, and interpretability of legal issue clarification.

[0004] Based on the above problems, the present invention provides a method for clarifying legal issues based on the collaboration of multi-source knowledge bases and the drive of large language models, and the main steps include:

[0005] S1: Establish an evaluation standard for the clarity of legal issues: To achieve the efficient and accurate clarification of legal issues, an evaluation standard for the clarity of legal issues is formulated. This evaluation standard focuses on two core elements of legal issues: the factual elements of legal requirements and legal claims. Based on these two core elements, key indicators of integrity, specificity, and clarity are constructed.

[0006] S2: Construct a multi-source legal knowledge base: Through steps such as collection and collation, summary of legal key points, identification of factual elements of legal requirements and legal responsibilities, construction of a legal knowledge association system, detection and resolution of conflicting points in related legal knowledge, knowledge vectorization, and updating, a multi-source legal knowledge base is constructed. The above-mentioned multi-source legal knowledge base contains m vector knowledge bases such as laws and regulations, trial elements, adjudication cases, and historical legal Q&A.

[0007] S3: Receive and process users' legal issues: Based on prompt engineering and large language model fine-tuning techniques, from the perspective of judicial practice, identify the factual elements of legal requirements and legal claims in the legal issue q input by the user, construct a key information set from the identified factual elements of legal requirements and legal claims, and generate query statements that match multi-source legal knowledge bases such as laws and regulations, trial elements, similar cases, and historical legal Q&A according to the identified factual elements of legal requirements and legal claims.

[0008] S4: Collaboratively generate a list of clarity detection elements for legal issues based on multiple knowledge bases: Based on the query statements generated in step S3, collaborate with legal knowledge bases such as laws and regulations, trial elements, adjudication cases, and historical legal Q&A to generate a hierarchical and systematic list of clarity detection elements for legal issues.

[0009] S5: Output the verification result of the clarity of legal issues: Based on the evaluation standard for the clarity of legal issues, compare and analyze the key information in the user's question with the list of clarity detection elements, and output the verification result of the clarity of legal issues. Based on the set threshold, judge whether the evaluation standard for the clarity of legal issues is met. If the evaluation standard score P meets the set threshold, output "Verification passed, no clarification needed". Otherwise, output "Verification failed, clarification needed" and enter step S6.

[0010] S6: Collaboratively generate a preliminary clarification result text based on multiple knowledge bases: Adopt a hierarchical and progressive strategy, formulate rules for generating clarification prompts, construct prompt instructions for generating clarification result texts, and rely on large language models to output multiple preliminary clarification result texts collaboratively generated based on multiple knowledge bases.

[0011] S7: Evaluate and filter the preliminary clarification result text and generate the final version of the clarification result: formulate evaluation standards for clarification result texts applicable to the judicial field, focusing on evaluating the consistency of the clarification result text with the user's legal issues and legal knowledge such as laws and regulations, trial elements and judgment rules, as well as its applicability and explanatory power in specific civil, criminal, administrative and other types of cases. According to the above clarification result text evaluation standards, use the large language model to simulate the full-process dialogue technology of legal question and answer, combine the experience of legal experts, screen the clarification result texts with judicial applicability value, and generate the final version of the clarification result text.

[0012] Furthermore, in step S1, the main content is to formulate clear evaluation criteria for legal issues, which specifically includes the following key sub-steps:

[0013] S11: Establishment of the clarity assessment standard for legal requirements facts: The clarity assessment standard for legal requirements facts refers to ensuring the completeness, specificity and clarity of case facts in user legal issues by selecting the completeness index of legal requirements facts, the specificity index of legal requirements facts and the clarity index of legal requirements facts. The completeness of legal requirements facts means that user legal issues should include legal requirements facts such as subject qualifications, case nature, and impact of rights and obligations; the specificity of legal requirements facts means that the description of legal requirements facts such as subject qualifications, case nature, and impact of rights and obligations should be specific and avoid the use of vague expressions; the clarity of legal requirements facts means using accurate and standardized language to describe legal requirements facts to avoid ambiguity or misunderstanding. The calculation formula for the clarity assessment score of legal requirements facts is:

[0014]

[0015] in, They are the scores for the factual completeness, specificity and clarity of legal requirements, with a score range of 0-5 and an integer.

[0016] S12: Establish a standard for assessing the clarity of legal claims: The standard for assessing the clarity of legal responsibilities is designed to ensure the completeness, specificity, and clarity of the legal claims in user legal issues. The completeness of legal claims means that user legal issues should include legal claims such as stopping infringement, removing obstacles, eliminating dangers, returning property, restoring the original state, compensating for losses, and apologizing; the specificity of legal claims refers to the description of legal claims, which should be specific and avoid using vague expressions; the clarity of legal claims refers to the use of accurate and standardized language to describe the expected legal effects to avoid ambiguity or misunderstanding. The formula for calculating the score for the clarity of legal claims assessment is:

[0017]

[0018] in, They are the scores for the integrity, specificity, and clarity indicators of legal claims respectively, with the score range being 0 - 5 points and the scores being integers.

[0019] S13: The score for the assessment criterion of clear legal issues is:

[0020] P = (P a + P b ) / 2.

[0021] Furthermore, the main task of the S2 step is to construct a multi-source legal knowledge base to provide highly relevant legal knowledge for clarifying legal issues, which specifically includes the following key steps:

[0022] S21: Collect and organize legal knowledge data: Collect comprehensive and up-to-date data, covering multi-source legal knowledge data such as laws and regulations, trial elements, judgment cases, and historical legal Q&A. Classify and organize the collected legal knowledge data, dividing it according to dimensions such as the formulating authority, release date, effective date, and timeliness to ensure the comprehensiveness, representativeness, and timeliness of the data. Regularly update the data to ensure the continuous update of the multi-source legal knowledge base. Taking the laws and regulations knowledge base as an example, this knowledge base includes different types of laws and regulations such as legal legislative interpretations, judicial interpretations, administrative regulations, departmental rules, local regulations, autonomous regulations, local government rules, local judicial documents, local regulatory documents, national people's congress legislative materials, Chinese administrative legislative materials, supreme court and supreme procuratorate judicial materials, and local judicial materials.

[0023] S22: Refine and generate legal key point summaries: Based on the legal key point summary prompt instructions, using large language model technology, generate content such as the applicable legal issue area, case type, applicable object, core view, and legal conclusion for each piece of legal knowledge in the laws and regulations knowledge base, trial element knowledge base, judgment case knowledge base, and historical legal Q&A knowledge base, to describe the characteristics and application scenarios of different legal knowledge for quick retrieval and matching of users' legal issues. Among them, for different types of legal knowledge, the information included in the legal key point summary prompt instructions is: legal knowledge content (such as legal article content, trial elements, etc.) and legal key point summary prompt corpus.

[0024] S23: Identify legal element facts and legal liabilities: Based on the legal key points generated in the S22 step and the legal element facts and legal liability prompt instructions, using large language model technology, identify the legal element facts (such as subject qualification, case nature, impact on rights and obligations, etc.) and legal liabilities (such as liability for breach of contract, tort liability, etc.) of legal knowledge, and generate combinations of legal element facts and combinations of legal liabilities.

[0025] Among them, the instructions on legal requirements, facts and legal responsibilities include legal knowledge content (legal content, trial elements, etc.), legal key points content and legal requirements, facts and legal responsibilities reminder corpus.

[0026] S24: Constructing a legal knowledge association system: Clustering algorithms are used to cluster the attribute values ​​in each type of legal elements and legal responsibilities, realize attribute grouping, and explore potential association patterns, thereby establishing associations between legal knowledge and associating similar legal knowledge for the same legal elements or legal responsibilities. When constructing a legal knowledge association system, the legal knowledge base is used as the core basis to systematically integrate and associate legal knowledge involving the same legal elements or legal responsibilities in laws and regulations, trial elements, judicial cases, and historical legal questions and answers, so as to form a complete legal knowledge association system.

[0027] S25: Detection and resolution of conflict points in related legal knowledge: First, formulate rules for detecting and resolving conflict points in related legal knowledge. The types of conflict in related legal knowledge include dimensions such as positive and negative views, scope of subjects of superior and subordinate laws, deadlines, scope of liability, conditions, legal essential facts, and whether legal liability conflicts; the rules for resolving conflict points in related legal knowledge include but are not limited to the following: Timeliness priority: the latest legal knowledge takes precedence over old legal knowledge. Authority priority: based on the authority of the enacting authority, for example, the laws and regulations issued by the Supreme People's Court are superior to the judicial documents issued by various provinces and cities. Secondly, based on the detection and resolution prompt instructions for related legal knowledge conflicts, the large language model technology is used to detect the conflict points of specific legal knowledge in the legal knowledge association system, and the filtered legal knowledge list is output. Among them, the detection and resolution prompt instructions for related legal knowledge conflicts include legal knowledge type, legal knowledge release date, legal knowledge enacting authority, legal essential facts, legal liability, conflict type, resolution rules, and detection and resolution prompt corpus for related legal knowledge conflicts. Finally, formulate a solution that combines automatic resolution with manual sampling review. For clear and direct conflicts in legal content, such as direct contradictions between historical legal questions and answers and legal provisions, the legal knowledge base will be automatically used to correct or delete the conflicting legal knowledge content. For complex conflicts (such as disputes involving legal interpretation or application), they will be submitted to legal experts for manual review and resolution.

[0028] S26: Core element combination. The legal key point summary, legal essential facts and legal responsibilities generated in the legal knowledge base are spliced, and their sparse vectors and dense vectors are generated and imported into the corresponding knowledge base.

[0029] S27: Import the vector library: Store the text content, association relationships, and vectors in the legal regulations set, trial element set, adjudication rule set, historical legal Q&A into the vector library to build an external legal vector knowledge base. Optionally, the vector library can be Milvus, Elasticsearch, etc.

[0030] Furthermore, the main task of step S3 is to receive and process legal questions, analyze and process the legal questions provided by users, which specifically includes the following key steps:

[0031] S31: Preprocess the user's legal question: In the legal question scenario, the question content usually mixes multiple case facts, including not only substantial content such as the identities of the parties, legal relationships of the case, nature of the case, and legal claims, but also redundant information and noise such as irrelevant life facts and the personal emotions of the parties. Therefore, it is necessary to perform legal text cleaning and standardization processing on the legal question q input by the user, including removing irrelevant characters, correcting spelling mistakes, word segmentation, etc., to generate a standardized question q'.

[0032] S32: Semantic classification and key information extraction: Based on the semantic classification and key information extraction prompt instructions, using large language model technology, classify and semantically analyze q' generated in step S31 to determine the legal field to which it belongs, identify and extract the background information, legal element facts, and legal claims in the legal question, and construct a key information set C = {c 1 , …, c l}, where l represents the number of key information extracted. Among them, legal element facts include, but are not limited to: content such as subject qualifications, nature of the case, facts affecting rights and obligations, etc.; legal claims include, but are not limited to: one or more of stopping infringement, removing obstacles, eliminating dangers, returning property, restoring the original state, compensating for losses, making an apology, etc. Concatenate the extracted background information, legal element facts, and legal claims as the query statement for the knowledge base.

[0033] Among them, the semantic classification and key information extraction prompt instructions include the standardized question q' and the semantic classification and key information extraction prompt corpus.

[0034] Furthermore, the main task of step S4 is to generate a legal question clarity detection element list through multi-library collaboration, combine multi-source legal knowledge bases, and generate a legal question clarity detection element list according to the legal questions input by users, which specifically includes the following key steps:

[0035] S41: Multi-library parallel retrieval: In the multi-source legal vector knowledge base described above, adopt a hybrid retrieval matching and parallel retrieval method to retrieve the query statement and obtain the relevant legal knowledge of the user's question.

[0036] S42: Integrate and cross-validate the retrieval results: Integrate the retrieval results of the multi-source legal knowledge bases and ensure the consistency and accuracy of the information through cross-validation. The retrieval results of the multi-source legal knowledge bases include, but are not limited to, the subject qualifications of the parties, the legal nature, and the legal element facts and legal liabilities that affect legal rights and obligations. The results of the laws and regulations library, the trial element knowledge library, the adjudicated case knowledge library, and the legal Q&A knowledge library are mutually verified and supplemented.

[0037] S43: Generate a checklist of legal issue clarity detection elements: According to the above steps S41 - S42, combined with the user's legal issue and corresponding background information, etc., generate a hierarchical and systematic checklist of legal issue clarity detection elements. where k i represents the number of elements corresponding to the i-th knowledge base, represents retrieving the k i -th element from the i-th knowledge base of the multi-source legal knowledge bases, and m is the total number of knowledge bases in the multi-source legal knowledge bases. This detection checklist includes detection elements such as the subject qualifications of the parties, the legal nature of the case, and the legal element facts and legal liabilities that affect legal rights and obligations.

[0038] Furthermore, the main task of the S5 step is to, based on the legal issue clarity evaluation criteria, compare and analyze the key information C in the user's question with the clarity detection checklist D and output the legal issue clarity verification result, which specifically includes the following key steps:

[0039] S51: Evaluate the clarity of the user's question and output missing elements. Based on the legal issue clarity evaluation criteria, construct a clarity evaluation prompt instruction, and use the large language model to respectively output the integrity, specificity, and clarity scores of the legal element facts and legal claims, and output the list of missing legal element facts and legal claims, denoted as D'. Based on the calculation formula in the S1 step, output the clarity score.

[0040] The clarity evaluation prompt instruction includes the legal issue clarity evaluation criteria, the key information C in the user's question, the clarity detection checklist D, and the evaluation prompt corpus.

[0041] S52: Set a threshold and output a conclusion: When the clarity score exceeds the threshold, output "Verification passed, no clarification needed". Otherwise, output "Verification failed, clarification needed" and enter step S6.

[0042] Furthermore, the main task of the S6 step is to adopt a multi-library collaborative strategy to clarify the user's question and generate a preliminary clarification result text, which specifically includes the following key steps:

[0043] S61: Formulate rules for generating clarification prompts: When generating clarification prompts, since D' contains different types of elements such as a legal regulations library, a trial elements library, a referee case library, and a historical legal Q&A library, the elements corresponding to the legal regulations library in D' are preferentially used to generate clarification prompts and explanations. Then, based on the score of the legal issue clarity evaluation criterion, it is judged whether further clarification is still required. If so, supplementary clarification prompts and explanations are generated based on the elements corresponding to the trial elements library, the referee case library, the historical legal Q&A library, etc. in D'. The explanations of the clarification prompts should be clear, accurate, and able to help users understand the legal basis and reasoning process of the results. Finally, all the clarification prompts and the user's original question are combined to output K clarification result texts.

[0044] S62: Generate clarification result texts based on multi-library collaboration: According to the formulated rules for generating clarification prompts, construct prompt instructions for generating clarification result texts, and rely on large language models to achieve the generation of clarification result texts through multi-library collaboration.

[0045] The prompt instructions for generating clarification result texts include the user's legal question, the missing element list D', the rules for generating clarification prompts, and the prompt corpus for generating clarification result texts.

[0046] Furthermore, in step S7, the main content is to evaluate and filter the initial clarification result texts, and use large language models to simulate the full-process dialogue technology of legal Q&A to generate the final version of the clarification result texts, which specifically includes the following key sub-steps:

[0047] S71: Simulate user responses: According to step S62, obtain each candidate clarification result text for the user's input legal question. For each of the above initial clarification result texts, the large language model simulates the possible subsequent responses of the user.

[0048] S72: Predict the legal question dialogue path: Based on the dialogue completed in step S71, the large language model predicts the subsequent legal question dialogue path and generates a detailed answer that complies with laws and regulations, judicial interpretations, and referee rules, ensuring that the dialogue logic is rigorous and has judicial application value.

[0049] S73: Establish evaluation criteria for the clarified result text generated by evaluation and filtering: Establish evaluation criteria for the clarified result text applicable to the judicial field, focusing on evaluating the degree of fit between the initial clarified result text and the user's legal question q and legal knowledge such as laws and regulations, trial elements, and adjudication rules in the multi-source legal knowledge base, as well as its applicability and interpretability in specific civil, criminal, administrative, and other cases. This evaluation criteria focuses on five core indicators, namely the integrity of the clarified result text, the clarity of the clarified result text, the fit of the clarified result text, the logic of the clarified result text, and the practicality of the clarified result text. Among them, the integrity of the clarified result text means effectively supplementing the missing legal element facts or legal claims in the user's legal question; the clarity of the clarified result text: means accurately eliminating the ambiguity or equivocality of legal concepts, article applications, or fact findings involved in the user's question; the fit of the clarified result text: closely fits relevant laws and regulations, judicial interpretations, and judicial adjudication rules; the logic of the clarified result text: forms a logically rigorous analysis chain based on legal reasoning methods; the practicality of the clarified result text: whether it provides substantial help for the user to understand legal issues, clarify rights and obligations, and take legal actions.

[0050] S74: Evaluate the quality score of the clarified result text: According to the five core indicators (integrity, clarity, fit, logic, practicality) of the clarified result text evaluation criteria, combined with the answers generated in step S72, based on the scoring prompt instructions of the clarified result text evaluation criteria, use large language model technology to output the five scores corresponding to each candidate clarified result text, and calculate the final score:

[0051] R=(s 1 +s 2 +s 3 +s 4 +s 5 ) / (5×5)

[0052] Where s 1 、s 2 、s 3 、s 4 、s 5 respectively represent the integrity of the clarified result text, the clarity of the clarified result text, the fit of the clarified result text, the logic of the clarified result text, and the practicality of the clarified result text, and the value range is 0 to 5.

[0053] S75: Filter the preliminary clarified result text to generate the final clarified result text: Combining the experience of legal experts, set a threshold for the quality score of the clarified result text, screen and filter to ensure that only the clarified result text with judicial application value and meeting the five core indicators is selected.

[0054] S76: Output high-quality clarified result text: Output the finally selected clarified result text to the user to ensure that it can effectively guide the user to clarify legal issues and provide substantial assistance for subsequent legal consultations or actions.

[0055] The present invention also provides a server, which is characterized by comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is configured to be executed by the processor, and the computer program includes instructions for executing the above method.

[0056] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and is characterized in that when the computer program is executed by a processor, the above method is implemented.

[0057] The advantages of the present invention are as follows:

[0058] The present invention provides a legal issue clarification method based on multi-source knowledge base collaboration and large language model drive, which has the following advantages: First, a rigorous legal issue clarity evaluation standard is formulated based on judicial practice experience, the legal element facts and the core elements of legal claims in legal issues are clarified, and the key indicators of integrity, specificity, and clarity are proposed to quantitatively evaluate the clarity of legal issues, providing a clear basis for legal issue clarification and enhancing the normativity and operability; Second, an efficient and reliable multi-source legal knowledge base conflict detection and resolution mechanism is constructed, giving priority to the legal regulations knowledge base, combined with automatic resolution and manual review, to ensure the authority and consistency of legal knowledge and improve the accuracy and reliability; Third, the multi-source legal knowledge base (covering laws and regulations, trial elements, judgment cases, and historical legal Q&A) is combined with the large language model to automatically and efficiently generate a hierarchical and systematic list of clarification prompt detection elements, which not only solves the problems of narrow coverage and few legal issue clarity detection points caused by traditional methods relying on manual sorting, but also solves the problems of hallucination, low accuracy, and time lag in the generation of clarification prompts by the large language model, significantly improving the comprehensiveness and accuracy of legal issue clarification. Fourth, a hierarchical progressive prompt generation strategy is adopted, generating preliminary prompts based on the legal regulations knowledge base first and supplementing with other knowledge bases to ensure the accuracy and practicality of the preliminary clarified result text. Fifth, a standard for evaluating and filtering the clarified result text is formulated, and the large language model is used to simulate the full-process dialogue technology of legal Q&A to scientifically evaluate the comprehensiveness and effectiveness of the clarified result text. In summary, the present invention has significant advantages in aspects such as legal issue clarity evaluation standards, legal knowledge base conflict resolution, legal knowledge base collaboration, legal issue clarity detection list generation, clarification prompt generation, and clarified result text evaluation, comprehensively improving the accuracy, coverage, and practicality of legal issue clarification. Description of the Drawings

[0059] Figure 1 It is a specific implementation flowchart.

[0060] Figure 2 It is a flowchart for generating a list of clarity detection elements for legal issues through multi-database collaboration. Specific implementation manners

[0061] To further elaborate on the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] The specific implementation flowchart of the present invention is as shown in the appendix Figure 1 and a method for clarifying legal issues based on multi-source knowledge base collaboration and large language model drive will be described in detail as follows:

[0063] S1: Establish an evaluation standard for the clarity of legal issues: In order to achieve efficient and accurate clarification of legal issues, an evaluation standard for the clarity of legal issues is established. This evaluation standard focuses on two core elements of legal issues: legal element facts and legal claims. Based on these two core elements, key indicators of integrity, specificity, and clarity are constructed, which specifically include the following key steps:

[0064] (1) Establish an evaluation standard for the clarity of legal element facts: The evaluation standard for the clarity of legal element facts refers to ensuring the integrity, specificity, and clarity of the case facts in the user's legal issues by selecting indicators of integrity, specificity, and clarity of legal element facts. The integrity of legal element facts means that the legal element facts such as the subject qualification, case nature, and impact of rights and obligations should be included in the user's legal issues; the specificity of legal element facts means that the description of legal element facts such as subject qualification, case nature, and impact of rights and obligations should be specific, avoiding the use of ambiguous expressions such as etc.; the clarity of legal element facts means that accurate and standardized language is used to describe legal element facts to avoid ambiguity or misunderstanding. The calculation formula for the evaluation score of the clarity of legal element facts is:

[0065]

[0066] Wherein, are respectively the scores of the integrity, specificity, and clarity indicators of legal element facts, and the score range is 0-5 points, and the score is an integer.

[0067] (2) Establish clear evaluation criteria for legal claims: The evaluation criteria for the clarity of legal liability aim to ensure the integrity, specificity, and clarity of legal claims in users' legal issues. The integrity of legal claims means that legal claims in users' legal issues should include legal claims such as cessation of infringement, removal of obstacles, elimination of dangers, return of property, restoration of the original state, compensation for losses, and apology. The specificity of legal claims refers to the description of legal claims, which should be specific and avoid using vague expressions. The clarity of legal claims means using accurate and standardized language to describe the expected legal effects to avoid ambiguity or misunderstanding. The calculation formula for the clear evaluation score of legal claims is:

[0068]

[0069] Among them, are the scores of the integrity, specificity, and clarity indicators of legal claims respectively, with the score range of 0 - 5 points and the scores being integers.

[0070] (3) The evaluation score for clear legal issues is:

[0071] P = (P a + P b ) / 2

[0072] S2: Construct a multi-source legal knowledge base: Through steps such as collection and collation, summary of legal key points, identification of legal element facts and legal liabilities, construction of a legal knowledge association system, detection and resolution of associated legal knowledge conflict points, knowledge vectorization, and updating, construct a multi-source legal knowledge base. The above-mentioned multi-source legal knowledge base contains four vector knowledge bases: laws and regulations, trial elements, judgment cases, and historical legal Q&A. Specifically, it includes the following key steps:

[0073] (1) Collect and organize legal knowledge data: Collect comprehensive and up-to-date data covering multi-source legal knowledge data such as laws and regulations, trial elements, judgment cases, and historical legal Q&A. Classify and organize the collected legal knowledge data, and divide it according to dimensions such as the formulating authority, release date, effective date, and timeliness to ensure the comprehensiveness, representativeness, and timeliness of the data. Regularly update the data to ensure the continuous update of the multi-source legal knowledge base.

[0074] (2) Refine and generate summaries of legal key points: Based on the prompt instructions for summarizing legal key points, as shown in Table 1, use large language model technology to generate content such as the applicable legal issue area, case type, applicable object, core view, and legal conclusion for each piece of legal knowledge in the laws and regulations knowledge base, trial elements knowledge base, judgment cases knowledge base, and historical legal Q&A knowledge base, so as to describe the characteristics and application scenarios of different legal knowledge for quick retrieval and matching of users' legal issues.

[0075] Table 1: Prompt Instructions for Summarizing Legal Key Points

[0076]

[0077]

[0078]

[0079] (3) Identification of legal facts and legal liabilities: Based on the generated legal points and legal facts and legal liabilities prompt instructions (see Table 2), the large language model technology is used to identify the legal facts of legal knowledge (such as subject qualifications, case nature, rights and obligations, etc.) and legal liabilities (such as breach of contract liability, tort liability, etc.), and generate legal facts combinations and legal liabilities combinations.

[0080] Table 2: Legal requirements and legal liability instructions

[0081]

[0082]

[0083] (4) Constructing a legal knowledge association system: clustering algorithms are used to cluster the attribute values ​​in each type of legal elements and legal responsibilities, grouping the attributes, and mining potential association patterns, thereby establishing associations between legal knowledge and associating similar legal knowledge on the same legal elements or legal responsibilities. When constructing a legal knowledge association system, the legal knowledge base is used as the core basis to systematically integrate and associate legal knowledge on the same legal elements or legal responsibilities in laws and regulations, trial elements, judicial cases, and historical legal questions and answers, thereby forming a complete legal knowledge association system.

[0084] (5) Detect and resolve conflict points of related legal knowledge: First, formulate rules for detecting and resolving conflict points of related legal knowledge. The types of conflicts in related legal knowledge include dimensions such as positive and negative views, the scope of the subject of upper and lower laws, time limits, scope of liability, conditions, legal element facts, and whether there are conflicts in legal liability; the rules for resolving conflict points of related legal knowledge include, but are not limited to, the following: Priority of timeliness: The latest legal knowledge takes precedence over the old legal knowledge. Priority of authority: Based on the authority of the formulating agency, for example, the laws and regulations issued by the Supreme People's Court are superior to the judicial documents issued by each province and city. Second, based on the prompt instructions for detecting and resolving conflict points of related legal knowledge, as shown in Table 3, use large language model technology to detect the conflict points in related legal knowledge and output a list of filtered legal knowledge. Among them, the prompt instructions for detecting and resolving conflict points of related legal knowledge include legal knowledge type, legal knowledge release date, legal knowledge formulating agency, legal element facts, legal liability, conflict type, resolution rules, and the corpus of prompt words for detecting and resolving conflict points of related legal knowledge. Finally, formulate a resolution method that combines automatic resolution and manual sampling review. For clear and direct conflicts in legal content, such as historical legal questions directly contradicting the provisions of laws and regulations, automatically use the legal knowledge base as the standard to correct or delete the conflicting legal knowledge content. For complex conflicts (such as those involving legal interpretation or application disputes), submit them to legal experts for manual review and resolution.

[0085] Table 3: Prompt Instructions for Detecting and Resolving Conflict Points of Related Legal Knowledge

[0086]

[0087]

[0088] (6) Build four types of knowledge bases. Based on the above steps, build a multi-source legal knowledge base. The main components of each knowledge base are as follows:

[0089] Components of the laws and regulations knowledge base. The laws and regulations knowledge base includes the formulating agency, announcement date, implementation date, timeliness, validity level, subject classification, legal issue area, case type, applicable object, core view, legal conclusion, legal element facts, legal liability, etc.;

[0090] Components of the trial element knowledge base. The trial element knowledge base includes the formulating agency, announcement date, legal issue area, case type, applicable object, legal element facts, legal liability, etc.;

[0091] Components of the adjudication rule knowledge base. The adjudication rules include the source of the published case, case number, trial procedure, adjudication result, legal issue area, case type, applicable object, core view, legal element facts, legal liability, etc.;

[0092] Components of the historical legal Q&A database. It includes legal Q&A sources, publication dates, authors, legal issue areas, case types, applicable objects, core viewpoints, legal conclusions, legal essential facts, legal responsibilities, etc.

[0093] (7) Core element combination. Concatenate the legal key point summaries, legal essential facts, and legal responsibilities generated in the legal knowledge base, generate their sparse vectors and dense vectors, and import them into the corresponding knowledge base.

[0094] (8) Import into the vector database: Store the text content, association relationships, and vectors of laws and regulations knowledge, trial element knowledge, adjudication rule knowledge, and historical legal Q&As into the Milvus vector database to build an external legal vector knowledge base.

[0095] S3: Receive and process users' legal questions: Based on prompt engineering and large language model fine-tuning techniques, from the perspective of judicial practice, identify the legal essential facts and legal claims in the legal question q input by the user, construct a key information set, and generate legal association knowledge base query statements such as matching laws and regulations, trial elements, similar cases, and historical legal Q&As according to the identified legal essential facts and legal claims. It specifically includes the following key steps:

[0096] (1) Preprocessing of users' legal questions: In the legal question scenario, the question content usually mixes multiple case facts, including not only substantial content such as the identities of the parties, legal relationships of the case, nature of the case, and legal claims, but also redundant information and noise such as unimportant life facts and personal emotions of the parties. Therefore, it is necessary to perform legal text cleaning and standardization processing on the legal question q input by the user, including removing irrelevant characters, correcting spelling mistakes, word segmentation, etc., to generate a standardized question q'.

[0097] (2) Semantic classification and key information extraction: Based on the semantic classification and key information extraction prompt instructions, as shown in Table 4, use large language model technology to classify and semantically analyze q' generated in the above step, determine the legal field to which it belongs, identify and extract the background information, legal essential facts, and legal claims in the legal question, and construct a key information set C = {c 1 , …, c l}, where l represents the number of key information extracted. Among them, legal essential facts include, but are not limited to, content such as subject qualifications, nature of the case, and facts affecting rights and obligations; legal claims include, but are not limited to, one or more of stopping infringement, removing obstacles, eliminating dangers, returning property, restoring the original state, compensating for losses, and making an apology. Concatenate the extracted background information, legal essential facts, and legal claims as the query statement for the knowledge base.

[0098] Table 4: Semantic Classification and Key Information Extraction Hint Instructions

[0099]

[0100]

[0101] S4: Multi-library Collaborative Generation of Legal Issue Clarity Detection List: Based on the query statements generated in the above steps, collaborate with legal knowledge bases such as laws and regulations, trial elements, adjudication cases, and historical legal Q&A to generate a hierarchical and systematic legal issue clarity detection element list. Specifically, it includes the following key steps:

[0102] (1) Multi-library Parallel Retrieval: In the multi-source legal knowledge bases, adopt a hybrid retrieval matching and parallel retrieval method to retrieve the retrieval query statements and obtain the relevant legal knowledge of the user's question.

[0103] (2) Integrate and Cross-validate Retrieval Results: Integrate the retrieval results of the multi-source legal knowledge bases and ensure the consistency and accuracy of the information through cross-validation. The retrieval results of the multi-source legal knowledge bases include, but are not limited to, the subject qualifications of the parties, the legal nature, and the legal element facts and legal responsibilities that affect legal rights and obligations. The results of the laws and regulations library, the trial element knowledge library, the adjudication case knowledge library, and the legal Q&A knowledge library are mutually verified and supplemented.

[0104] (3) Generate a Legal Issue Clarity Detection Element List: According to the above steps, combine the user's legal question and the corresponding background information and other content to generate a hierarchical and systematic legal issue clarity detection element list, where k i represents the number of elements corresponding to the i-th knowledge base. This detection list includes detection elements such as the subject qualifications of the parties, the legal nature of the case, and the legal element facts and legal responsibilities that affect legal rights and obligations.

[0105] S5: Output the Legal Issue Clarity Verification Result: Based on the legal issue clarity evaluation criteria, compare and analyze the key information C in the user's question with the clarity detection element list D, and output the legal issue clarity verification result. Based on the set threshold, judge whether the legal issue clarity evaluation criteria are met. If so, output "Verification Passed, No Clarification Required". Otherwise, output "Verification Failed, Clarification Needed" and enter step S6. Specifically, it includes the following key steps:

[0106] (1) Evaluate the clarity of the user's question and output the missing elements. Based on the legal question clarity evaluation criteria, construct the clarity evaluation prompt instructions, as shown in Table 5. Utilize the large language model to respectively output the integrity, specificity, and clarity scores of the legal element facts and legal claims, and output the list of missing legal element facts and legal claims, denoted as D'. Based on the legal question clarity calculation formula, output the clarity score.

[0107] Table 5: Clarity Evaluation Prompt Instructions

[0108]

[0109]

[0110] (2) Set a threshold and output a conclusion: When the clarity score exceeds the threshold, output "Verification passed, no clarification required". Otherwise, output "Verification failed, clarification needed" and proceed to step S6.

[0111] S6: Generate a preliminary clarification result text through multi-library collaboration: Adopt a hierarchical and progressive strategy, formulate clarification prompt generation rules, construct preliminary clarification result text prompt instructions, and rely on the large language model to generate multiple preliminary clarification result texts through multi-library collaboration. Specifically, it includes the following key steps:

[0112] (1) Formulate clarification prompt generation rules: When generating clarification prompts, since D' contains elements of different types such as legal regulations libraries, trial element libraries, judgment case libraries, and historical legal Q&A libraries, prioritize using the elements corresponding to the legal regulations knowledge base in D' to generate clarification prompts and explanations. Then, based on the legal question clarity evaluation criteria, judge whether further clarification is still required. If so, generate supplementary clarification prompts and explanations based on the elements corresponding to the trial elements, judgment cases, historical legal Q&A, etc. in D'. The explanations of the clarification prompts should be clear, accurate, and able to help users understand the legal basis and reasoning process of the results. Finally, combine all the clarification prompts and the user's original question to output K clarification result texts.

[0113] (2) Generate an initial clarification result text based on multi-library collaboration: According to the formulated clarification prompt generation rules, construct the prompt instructions for the initial clarification result text, as shown in Table 6, and rely on the large language model to achieve the generation of the clarification result text through multi-library collaboration.

[0114] Table 6: Prompt Instructions for Clarification Result Text

[0115]

[0116]

[0117]

[0118] S7: Evaluate the initially clarified result text and generate the final clarified result text: Develop evaluation criteria for clarification prompts applicable to the judicial field, focusing on evaluating the degree of fit between the initially clarified result and the user's legal issues, legal knowledge such as laws and regulations, trial elements, and adjudication rules, as well as its applicability and interpretability in different types of cases such as specific civil, criminal, and administrative cases. According to the above evaluation criteria for the initially clarified result text, use large language models to simulate the full process of legal Q&A dialogue technology, combined with the experience of legal experts, to screen the clarified result text with judicial application value and generate the final clarified result text. This specifically includes the following key sub-steps:

[0119] (1) Simulate user responses: According to the above steps, obtain each candidate clarified result text for the user's input legal question. For each of the above initially clarified result texts, the large language model simulates possible subsequent responses from the user.

[0120] (2) Predict the legal question dialogue path: Based on the dialogue completed in the above steps, the large language model predicts the subsequent legal question dialogue path and generates a detailed answer that complies with laws and regulations, judicial interpretations, and adjudication rules, ensuring that the dialogue logic is rigorous and has judicial application value.

[0121] (3) Develop evaluation criteria for the clarified result text: Develop evaluation criteria for the clarified result text applicable to the judicial field, focusing on evaluating the degree of fit between the clarified result text and the user's legal question q and legal knowledge such as laws and regulations, trial elements, and adjudication rules in the multi-source legal knowledge base, as well as its applicability and interpretability in different cases such as specific civil, criminal, and administrative cases. This evaluation criteria focuses on five core indicators, namely the integrity of the clarified result text, the clarity of the clarified result text, the fit of the clarified result text, the logic of the clarified result text, and the practicality of the clarified result text. Among them, the integrity of the clarified result text means effectively supplementing the missing legal element facts or legal claims in the user's legal question; the clarity of the clarified result text: means accurately eliminating the ambiguity or equivocality of legal concepts, article applications, or fact findings involved in the user's question; the fit of the clarified result text: closely conforms to relevant laws and regulations, judicial interpretations, and judicial adjudication rules; the logic of the clarified result text: forms a logically rigorous analysis chain based on legal reasoning methods; the practicality of the clarified result text: whether it provides substantial help for the user to understand legal issues, clarify rights and obligations, and take legal actions.

[0122] (4) Evaluate the quality score of the clarified result text: According to the five core indicators (integrity, clarity, fitness, logic, and practicality) of the clarified result text evaluation criteria, combined with the answers generated in the above steps, based on the scoring prompt instructions of the clarified result text evaluation criteria, as shown in Table 7, using large language model technology, output the five scores corresponding to each candidate clarified result text, and calculate the final score:

[0123] R = (s 1 + s 2 + s 3 + s 4 + s 5 ) / (5 × 5)

[0124] Where s 1 , s 2 , s 3 , s 4 , s 5 respectively represent the integrity of the clarified result text, the clarity of the clarified result text, the fitness of the clarified result text, the logic of the clarified result text, and the practicality of the clarified result text.

[0125] Table 7: Scoring Prompt Instructions for Clarified Result Text Evaluation Criteria

[0126]

[0127]

[0128] (5) Filter the preliminary clarified result text to generate the final clarified result text: Combining the experience of legal experts, set a threshold for the quality score of the clarified result text, and perform screening and filtering to ensure that only the clarification prompts with judicial application value and meeting the five core indicators are selected.

[0129] (6) Output the high-quality clarified result text: Output the finally selected clarified result text to the user to ensure that it can effectively guide the user to clarify legal issues and provide substantial assistance for subsequent legal consultations or actions.

[0130] The present invention also provides a server, which is characterized by including a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing the above method.

[0131] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and is characterized in that the computer program realizes the above method when executed by a processor.

[0132] It should be noted that the above description is only the embodiments and drawings of the present invention, and its purpose is to better understand the content of the present invention, rather than to limit the present invention. For those skilled in the art, the present invention can be implemented in various ways. Without departing from the spirit and scope of the present invention and the appended claims, various substitutions, changes and modifications are possible. Therefore, the present invention should not be limited to the content disclosed in the best embodiments, and the scope of protection claimed by the present invention shall be defined by the scope defined in the claims.

Claims

1. A method for clarifying legal issues based on a large language model, the steps of which include: 1) Select indicators related to the legal facts and core elements of legal claims in legal issues, construct a legal issue clarity assessment standard, and use it to calculate the legal issue clarity assessment standard score; 2) constructing a multi-source legal knowledge base, which includes a legal and regulatory vector knowledge base, a trial element vector knowledge base, a judicial case vector knowledge base, and a historical legal question-and-answer vector knowledge base; 3) Identifying the legal facts and legal claims of the legal question q input by the user based on the large language model, and generating a query statement for querying the multi-source legal knowledge base based on the identified legal facts and legal claims; 4) querying the multi-source legal knowledge base based on the query statement, and generating a list of legal issue clarity detection elements according to the query results; 5) Based on the legal issue clarity assessment standard, compare and analyze the legal essential facts, legal claims and clarity detection element list identified from the legal issue q, and output the legal issue clarity assessment result; If the evaluation standard score P is greater than the set threshold, the verification is passed without clarification; otherwise, proceed to step 6); 6) constructing a clarification result text generation prompt instruction according to the set clarification prompt generation rule and inputting it into the large language model to output a number of preliminary clarification result texts; 7) Based on the set evaluation criteria for clarification result texts, a large language model is used to simulate the full-process dialogue technology of legal question-and-answering and combined with the experience of legal experts, clarification result texts with judicial applicability are screened from the preliminary clarification result texts to generate the final version of the clarification result text.

2. The method according to claim 1, characterized in that The legal issue clarity assessment criteria include the legal requirement fact clarity assessment criteria and the legal claim clarity assessment criteria. The method for calculating the assessment criteria score P is: 11) Based on the selected legal requirement fact completeness index, legal requirement fact specificity index and legal requirement fact clarity index, formulate legal requirement fact clarity assessment criteria to calculate the legal requirement fact clarity assessment score in, The scores are the completeness index of legal requirements, the specificity index of legal requirements and the clarity index of legal requirements. The value range of is 0~N; 12) Based on the selected legal claim completeness index, legal claim specificity index and legal claim clarity index, formulate a legal claim clarity assessment standard to calculate the legal claim clarity assessment score in, They are the scores of the legal claim completeness index, the legal claim specificity index, and the legal claim clarity index. The value range of is 0~M; 13) Calculate the score of the legal issue clarity assessment standard P = (P a +P b ) / 2.

3. The method according to claim 1, characterized in that The method for constructing the multi-source legal knowledge base is: 21) Collect legal knowledge data covering laws and regulations, trial elements, judicial cases, and historical legal questions and answers, classify and organize the collected legal knowledge data, divide them by the dimensions of formulating authority, release date, effective date, and timeliness, and generate a legal and regulatory knowledge base, a trial element knowledge base, a judicial case knowledge base, and a historical legal question and answer knowledge base; 22) Based on the summary prompt instructions of legal points, use the large language model to generate the legal points of each legal knowledge data in the legal and regulatory knowledge base, trial element knowledge base, judicial case knowledge base, and historical legal question and answer knowledge base; 23) Based on the legal points, legal essential facts and legal liability prompt instructions, use the large language model to identify the legal essential facts and legal liability in each piece of legal knowledge data; 24) Associate the legal knowledge related to the same legal elements, facts or legal responsibilities in laws and regulations, trial elements, judicial cases and historical legal questions and answers to form a legal knowledge association system; 25) Based on the detection and resolution prompt instructions of the associated legal knowledge conflict points, a large language model is used to detect the conflict points in the legal knowledge association system and correct them; 26) Splicing the legal points, legal essential facts and legal responsibilities of each piece of legal knowledge data, and generating sparse vectors and dense vectors of the corresponding legal knowledge data; 27) Each law and regulation in the legal and regulatory knowledge base and its sparse vector, dense vector, and the relationship between laws and regulations are stored in the vector library to obtain a legal and regulatory vector knowledge base; each trial element in the trial element knowledge base and its sparse vector, dense vector, and the relationship between trial elements are stored in the vector library to obtain a trial element vector knowledge base; each adjudication case in the adjudication case knowledge base and its sparse vector, dense vector, and the relationship between adjudication cases are stored in the vector library to obtain a adjudication case vector knowledge base; each historical legal question and answer in the historical legal question and answer knowledge base and its sparse vector, dense vector, and the relationship between historical legal questions and answers are stored in the vector library to obtain a historical legal question and answer vector knowledge base.

4. The method according to claim 1, 2 or 3, characterized in that: The method for generating a query statement for querying the multi-source legal knowledge base is: 31) Perform legal text cleaning and standardization on the legal question q input by the user to generate a standardized question q′; 32) Based on semantic classification and key information extraction prompt instructions, a large language model is used to classify and semantically analyze the standardized question q′, determine the legal field to which it belongs, identify and extract background information, legal facts and legal claims in the legal issue as key information, and obtain a key information set C; the extracted background information, legal facts and legal claims are spliced ​​as a query statement for querying the multi-source legal knowledge base.

5. The method according to claim 4, characterized in that The method for generating the above-mentioned list of elements for detecting clarity of legal issues is as follows: 41) searching the multi-source legal knowledge base according to the query statement to obtain relevant legal knowledge; 42) Integrate and cross-validate search results to ensure consistency and accuracy of information; 43) Combine the user’s legal issues and corresponding background information to generate a hierarchical and systematic list of legal issue clarity detection factors in, Indicates that the kth legal knowledge base is retrieved from the i-th knowledge base of the multi-source legal knowledge base i elements, and m is the total number of knowledge bases in the multi-source legal knowledge base.

6. The method according to claim 5, characterized in that The method to obtain the evaluation criteria score P for legal clarity is: 51) Based on the legal issue clarity assessment standard, a clarity assessment prompt instruction is constructed and input into the large language model to obtain the legal requirement fact clarity assessment score P a and Legal Claim Clarity Assessment Score P b , and output the missing legal requirements, facts and legal claims list D′; 52) Calculate the score of the legal issue clarity assessment standard P = (P a +P b ) / 2.

7. The method according to claim 6, characterized in that The method for obtaining the preliminary clarification result text is: 61) Generate clarification prompts and explanations based on the legal and regulatory elements in the missing legal requirements, facts and legal claims list D′, and then judge whether clarification is still needed based on the legal issue clarity assessment standard. If so, generate supplementary clarification prompts and explanations based on the elements corresponding to the trial elements, judicial cases, and historical legal questions and answers in the missing legal requirements, facts and legal claims list D′; combine all clarification prompts and legal questions q, and output K clarification result texts. 62) Construct prompt instructions for generating clarification result texts according to the formulated rules for generating clarification prompts, and obtain a number of preliminary clarification result texts based on the large language model.

8. The method according to claim 1, characterized in that The method to generate the final version of the clarified results text is: 71) For each preliminary clarification result text, use a large language model to simulate the user to answer; 72) Based on the dialogue in step 71), a large language model is used to predict the subsequent dialogue path of legal issues and generate answers that comply with laws, regulations, judicial interpretations and adjudication rules; 73) Formulate evaluation criteria for clarification result texts applicable to the judicial field, which are used to evaluate the consistency of the preliminary clarification result text with the legal issue q, the laws and regulations, trial elements and adjudication rules in the multi-source legal knowledge base, including the completeness of the clarification result text s1, the clarity of the clarification result text s2, the consistency of the clarification result text s3, the logic of the clarification result text s4, and the practicality of the clarification result text s5; 74) According to the evaluation result of step 73) and in combination with the answer generated in step 72), based on the clarification result text evaluation standard scoring prompt instruction, the large language model technology is used to calculate the clarification result text completeness s1, the clarification result text clarity s2, the clarification result text fit s3, the clarification result text logic s4, and the clarification result text practicality s5 of the preliminary clarification result text, and the score of the preliminary clarification result text is calculated to be R=(s1+s2+s3+s4+s5) / (5×Q); the value ranges of s1, s2, s3, s4, and s5 are all 0 to Q; 75) The quality threshold of the clarification result text is set in combination with the experience of legal experts, and the preliminary clarification result text with a score R greater than the quality threshold of the clarification result text is retained as the final version of the clarification result text.

9. A server, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • Question and answer pair generation method and system based on large language model

    CN118332086A

  • AI intelligent question and answer implementation method and system based on digital intelligent platform

    CN118761470A

  • Legal file element extraction method and system based on large language model and RAG

    CN118885594A

  • Knowledge question and answer accuracy improving method based on semantic elements

    CN119202209A

  • Drug knowledge quiz method, apparatus, electronic device and medium

    US20240420587A1