A Generative Multi-hop Legal Rule Chain Mining and Evaluation Method

Abstract

This invention discloses a generative multi-hop legal rule chain mining and evaluation method. It employs a dual-path mining approach to generate a set Z of candidate potential legal rule chains. Specifically, it includes generating a first set Z1 of potential legal rule chains based on statistical chain-like rule mining and a second set Z2 of potential legal rule chains based on a legal ontology logical reasoning model using a large language model. All potential legal rule chains are evaluated from two dimensions: a legal logic rationality vote using the large language model and a fact-oriented fact-reasoning chain credibility vote using the large language model. Finally, it performs legal rule chain rationality arbitration, filtering and selecting highly credible legal rule chains from the potential legal rule chain set Z. This invention enhances the semantic understanding of entities and relationships in legal knowledge graphs, improves the interpretability of pre-trained generative language models, reduces training costs, and achieves a multi-hop legal rule chain mining technique with high robustness and rationality.

Description

Technical Field

[0001] This invention belongs to the fields of computer-related artificial intelligence and natural language processing technology, and in particular relates to a method for automatically generating comprehensive legislative opinion reports for long-term, cross-domain applications. Background Technology

[0002] Building legal knowledge graphs is a complex task, aiming to create a broad network of relationships between entities and concepts to support various intelligent applications. However, the construction process often encounters problems such as missing relationships and sparse connections. This means that the associations between some entities may not be fully identified or recorded, resulting in gaps or incompleteness in the knowledge graph's representation of complex real-world interactions. This sparsity not only affects the coverage and accuracy of the knowledge graph but also poses challenges to graph-based querying and data mining. To address these issues, researchers and developers have adopted various strategies, including automatically extracting relationships from large amounts of unstructured data using text mining techniques, inferring implicit relationships using statistical and machine learning methods, and compensating for missing connections through human intervention and expert knowledge. Nevertheless, missing relationships and sparse connections remain among the significant challenges in the field of knowledge graph construction, requiring continuous effort and innovation to overcome.

[0003] Currently, chain rule mining of knowledge graphs can employ traditional association rule mining algorithms such as Apriori and FP-Growth to statistically analyze the frequency of occurrence between entities and relations. More recently, methods like RNNLogic have emerged, combining neural networks and symbolic reasoning. Other approaches transform this problem into a multi-step knowledge graph link prediction problem, mapping entities and relations to vector representations in a low-dimensional space. Representative models include GNN, GCN, TransE, DistMult, and ComplEx, which predict potential relationships between entities by learning vector representations of entities and relations.

[0004] The shortcomings of existing technologies, derived through causal reasoning, include:

[0005] 1) Problem description of traditional association rule mining algorithms (such as Apriori and FP-Growth):

[0006] Data sparsity: In large-scale knowledge graphs, there may be a large number of entities and relations, but the number of times each combination of entity and relation may occur is relatively small, making it difficult to effectively mine frequent itemsets.

[0007] High-dimensional problems: When the number of entities and relationship types is large, traditional algorithms need to explore a huge space of potential association rules, resulting in high computational complexity.

[0008] Lack of semantic understanding: These algorithms rely solely on statistical frequency, ignoring the potential semantic connections between entities and relationships.

[0009] 2) Problem description of hybrid methods combining neural networks and symbolic reasoning (such as RNNLogic):

[0010] Interpretation issues: Although the neural network combines the powerful fitting ability of neural networks with the rule-expressing ability of symbolic logic, the black-box nature of the neural network part may make the final inference rules difficult to interpret.

[0011] High training cost: Neural networks require a large amount of data and computing resources for training, making them unfriendly to environments with limited resources.

[0012] 3) Problem description of multi-step knowledge graph link prediction methods (such as GNN, GCN, TransE, etc.):

[0013] Overfitting risk: These models are prone to overfitting on small or highly homogeneous datasets and may not be able to generalize well to complex and ever-changing real-world environments.

[0014] Robustness issues: It is sensitive to noisy data and outliers, which may lead to inaccurate learned vector representations.

[0015] The relevant terms and their explanations in this invention are as follows:

[0016] [Chain Rules in Knowledge Graphs] A knowledge graph (KG) consists of a set of triples G={(h,r,t)|h,t∊E,r∊R}, where h represents the head entity, t represents the target entity, r represents the relation type, E is the entity set, and R is the relation set. Logical rules are usually expressed as first-order logic formulas, for example... or The left side of the implied symbol "→" is called the body or premise of the rule, and the right side is the conclusion. Logical rules are usually restricted to being closed, forming a chain. Chained rules are common logical rules in KG, such as symmetry rules, inverse rules, compound rules, hierarchical rules, and intersection rules. These rules play an important role in KG reasoning. For chained rules, consecutive relations share an intermediate entity (e.g., y), and the head and tail entities of the rule head and body are the same. Chained rules are presented in conjunction form, and the length of each rule is defined by the number of atoms (relations) in its body. For a length of... The chain rules are as follows:

[0017] ;

[0018] [Statistical Chain Rule Mining] Statistical chain rule mining analyzes the frequency of occurrence of entities and relationships, and summarizes reasonable chain rules based on the probability of rule occurrence. These rules are then used to reason and fill in missing links in knowledge graphs. For example, in knowledge graphs in the healthcare field, by analyzing common patterns between symptoms, treatments, and outcomes, rules can be generated to help doctors quickly diagnose and treat diseases.

[0019] [Legal Ontology] A legal ontology is a structured framework specifically designed to represent legal information and knowledge. It employs the concept of ontology to model entities, concepts, attributes, and the various relationships between these elements within the legal field. By defining a shared and universal vocabulary, a legal ontology facilitates information exchange and understanding between different systems, applications, and users, thereby improving the efficiency and accuracy of legal information retrieval, management, and automated processing. A legal ontology can encompass a wide range of legal concepts, such as legal provisions, cases, legal principles, legal acts, parties, etc., as well as the relationships between these concepts, such as "applicable," "violated," or "cited." Furthermore, a legal ontology can express legal rules and logic, supporting reasoning and analysis of legal knowledge.

[0020] [Think Chain Hint] Thinking chain is a technique that allows large models to explicitly output intermediate steps during the reasoning process, thereby improving their reasoning ability. This concept draws inspiration from how humans solve complex problems, breaking them down into a series of reasoning steps in natural language until a final conclusion is reached. Specifically, the goal of thinking chain is to allow large models to explicitly output the intermediate reasoning steps before providing the final answer, thus enhancing their conceptual understanding and reasoning abilities. This is significant for solving tasks requiring precise reasoning, such as mathematical problems and common-sense reasoning. Through thinking chain, large models can more closely resemble the human thought process when solving problems, thereby improving their intelligence level.

[0021] Generative language models are deep learning models that are trained on massive amounts of unlabeled or labeled text to learn natural language processing tasks. These models learn general representations of language in a pre-training phase, making them applicable to a wide range of natural language processing tasks without requiring extensive labeled data. GPT3 and ChatGPT are representative examples of generative language models that aim to simulate human natural language processing capabilities by using neural network models for text generation and processing. By using these models, users can engage in more natural and realistic conversations with computers and perform natural language processing tasks such as speech recognition, text summarization, machine translation, text classification, computer code generation, and screenwriting. Summary of the Invention

[0022] The purpose of this invention is to propose a generative multi-hop legal rule chain mining and evaluation method. By comprehensively evaluating the potential legal rule chain set generated by dual-path mining, combining the rationality of legal logic and the credibility of legal fact reasoning chain, a credible mining result is obtained by filtering and selecting highly credible legal rule chains from the potential legal rule chain set Z.

[0023] This invention is achieved using the following technical solution:

[0024] This invention provides a generative multi-hop legal rule chain mining and evaluation method, including:

[0025] Step 1: Generate potentially valid legal rules using two paths: statistical chain rule mining and a legal ontology logic inference engine based on a large language model, to generate a candidate set of potential legal rule chains Z. The set of potential legal rule chains Z further includes a set of potential legal rule chains generated by statistical chain rule mining (Z1) and a set of potential legal rule chains generated by a legal ontology logic inference engine based on a large language model (Z2).

[0026] Step 2: Evaluate the reasonableness of the potential chain-like legal rule set Z generated by the dual-path mining in Step 1. Specifically, evaluate all potential legal rule chains from two dimensions, including:

[0027] i. Use the legal logic rationality voting of the large language model to evaluate the legal logic rationality of the generated set of potential legal rule chains. This includes using a legal ontology prompter to integrate the legal ontology semantic knowledge of each legal rule chain into the prompts of the large language model, and through reasoning with the large language model, to complete the identification of the rationality of the legal rule chain at the level of legal common sense, and to score the logical rationality confidence score as the result of the logical rationality voting.

[0028] ii. Using a large language model, a credibility assessment of the generated set of potential legal rule chains is conducted on the credibility of the fact-based reasoning chains. This includes using a fact chain prompter to calculate the credibility confidence score of the legal fact reasoning chains, constructing knowledge graph retrieval statements by mining incomplete legal rule chains as described above, searching for triplet fact data that meet the conditions in the knowledge graph and performing random sampling, and using a large language model to reason and calculate the credibility confidence score of each legal fact reasoning chain, generating the corresponding credibility confidence score of the legal fact reasoning chain as the voting result of the legal logic rationality.

[0029] iii. Combine the confidence scores for legal logic rationality and legal factual reasoning chain credibility to generate a weighted confidence assessment score for each potential legal rule chain;

[0030] Step 3: Conduct legal rule chain rationality arbitration. Specifically, the legal logic rationality assessment results of the legal rule chain obtained in Step 2 and the credibility assessment results of the factual reasoning chain of the legal rule chain obtained in Step 3 are weighted and synthesized to filter and select highly credible legal rule chains from the potential legal rule chain set Z.

[0031] In some implementations, the process of generating a set of potential legal rule chains Z1 based on statistical chain rule mining further includes:

[0032] A statistical legal rule mining model is constructed, taking the set of potential legal rule chains Z as input to the model, and selecting any chain rule {r1, r2, ..., r...} from the set of potential legal rule chains Z. l Let} be used as the legal rule mining chain. The legal rule mining query operation is represented as q1. The result of the query operation q1 obtained through mining is the equivalent rule one (target relation) as {r1,r2,...,r}. l}→r a1 As the output of the mining model, the probability distribution p of the potential legal rule chain set Z1 generated by the statistical chain rule mining in this step is calculated based on the model's output. g,e (A|G,q), the expression is as follows:

[0033]

[0034] Where, p g To generate the probability distribution of potential legal rule chains through statistical chain-based legal rule mining, p e As the evaluation result of the potential legal rule chain, A is the equivalent legal rule r. a The evaluation results are given, where G represents the legal knowledge graph and Z represents the set of potential legal rule chains.

[0035] In some implementations, the process of generating the potential legal rule chain set Z2 based on the legal ontology logical reasoning model of the large language model further includes:

[0036] A legal ontology logical reasoning model is constructed to obtain a legal ontology logical reasoning model based on a large language model. This model takes the legal knowledge ontology compiled by experts as its input and selects any chain rule {r1, r2, ..., r...} from the potential legal rule chain set Z. l Let} be used as the legal rule mining chain. The legal rule reasoning query operation is represented as q2. The result of query operation q2 obtained through reasoning is the equivalent rule two (target relation) as {r1,r2,...,r}. l}→r a2As the second output of the model, the probability distribution of the potential legal rule chain set Z2 generated by the legal ontology logical reasoning model based on the large language model in this step is calculated based on the second output of the reasoning model. The expression is as follows:

[0037]

[0038] in, Let T1(O) be the probability of the large language model, and T1(O) be the function that converts the legal knowledge ontology O compiled by experts into LLM hints. For all legal rules preceding the i-th order legal rule in Z, p gLM The goal is to generate a probability distribution of possible legal rule chains by mining statistical chain rules.

[0039] In some implementations, the legal ontology logical reasoning model based on a large language model specifically includes obtaining hints from previously compiled legal knowledge ontology by experts, then using the hints to enrich a general large language model, and finally defining the reasoning ability of the enriched large language model in handling legal issues.

[0040] In some implementations, the legal rule chain set Z is obtained by taking the union of legal rule chain set one Z1 and legal rule chain set two Z2, as shown in the following expression:

[0041] Z = Z1∪Z2.

[0042] In some implementations, the thought chain process in the legal ontology prompter is represented as follows:

[0043]

[0044]

[0045]

[0046] in, To interpret a given chain rule relation R and the chain rule... And equivalent rule r a1 r a2 Given assessment result A, the total probability of confidence score C is given. The probability of evaluating chain rules and their interpretations.

[0047] In some implementations, the fact chain prompter generates Cypher query statements for the knowledge graph from the mined multi-hop legal rule chain, randomly samples the query results from the graph database, and enables the fact chain prompter to traverse these sampled records and construct appropriate fact evaluation prompts.

[0048] In some implementations, a weighted confidence score is given for each potential chain of legal rules. The expression is as follows:

[0049]

[0050] In the formula, , These are the weight parameters, and they have... , The credibility confidence score for the legal fact reasoning chain. The score represents the confidence level in the rationality of the legal logic.

[0051] Compared with the prior art, the advantages of the present invention are mainly reflected in the following aspects:

[0052] 1) Enhanced semantic understanding: Traditional algorithms usually ignore the potential semantic connections between entities and relations, relying only on frequency statistics. However, the "legal ontology prompter" and "fact chain prompter" of this invention can evaluate the legal logic and the rationality of the fact chain, thereby deepening the semantic understanding of entities and relations in the legal knowledge graph.

[0053] 2) Improved interpretability and reduced training cost: Compared with the hybrid method of neural network and symbolic reasoning, the generative multi-hop chain rule mining method of this invention provides better interpretability through explicit logic and fact chain evaluation, while the use of pre-trained generative language model also reduces training cost.

[0054] 3) Enhance the generalization ability and robustness of the model: In view of the problems of overfitting and sensitivity to noise in multi-step knowledge graph link prediction methods, the method of this invention can effectively improve the generalization ability and robustness of the model through a dual evaluation strategy and a legal rule rationality arbiter, ensuring that the generated legal rules are logically rigorous and consistent with real legal facts.

[0055] 4) A multi-hop legal rule chain mining technology with high robustness and rationality has been implemented. Attached Figure Description

[0056] Figure 1 This is an overall flowchart of a generative multi-hop legal rule chain mining and evaluation method according to the present invention;

[0057] Figure 2 This is a detailed flowchart of step 1;

[0058] Figure 3 This is a detailed flowchart of step 2;

[0059] Figure 4This is a framework diagram of a specific embodiment of the generative multi-hop legal rule chain mining and evaluation algorithm of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly described below with reference to the accompanying drawings.

[0061] like Figure 1 As shown, the generative multi-hop legal rule chain mining and evaluation method proposed in this invention consists of four steps:

[0062] Step 1: Generate a set of candidate potential legal rule chains. The specific operations of this step include generating possible legal rules using two paths: statistical chain rule mining and legal ontology logic inference based on large language model. Then, the large language model LLM is used to generate a set of candidate potential legal rule chains from these possible legal rules.

[0063] Formal definitions include: a legal knowledge graph represented by G, a legal knowledge ontology compiled by experts represented by O, a set of potential legal rule chains represented by Z, and arbitrary chained legal rules represented by {r1, r2, ..., r...}. l}, legal rule mining query operation is represented as q1, and legal rule reasoning query operation is represented as q2; such as Figure 2 As shown, the following detailed processes are further included:

[0064] Step 1.1: Construct a statistical legal rule mining model. This model generates a set of potential legal rule chains, Z1. Specifically, the statistical legal rule mining employs probabilistic rule mining algorithms such as RNNLogic, RLogic, and NCRL. The set of potential legal rule chains Z is used as input to the model. Any chain rule {r1, r2, ..., r...} is selected from the set of potential legal rule chains Z. l Let {r1, r2, ..., r} be used as the legal rule mining chain. The legal rule mining query operation is represented as q1. The result of the query operation q1 obtained through mining is the equivalent rule one, which is {r1, r2, ..., r}. l}→r a1 As the output of the mining model, the probability distribution p of the potential legal rule chain set Z1 generated by the statistical chain rule mining in this step is calculated based on the model's output. g,e (A|G,q), the expression is as follows:

[0065] Where, p g To generate the probability distribution of potential legal rule chains through statistical chain-based legal rule mining, pe As the evaluation result of the potential legal rule chain, A is the equivalent legal rule r. a The evaluation results;

[0066] Step 1.2: Construct a legal ontology logical reasoning model. This involves generating a potential legal rule chain set Z2 based on a large language model. This includes obtaining the legal ontology logical reasoning model based on a large language model, where the model takes expert-analyzed legal knowledge ontology as input.

[0067] Specifically, the process involves obtaining hints from the legal knowledge ontology previously compiled by experts, then using these hints to enrich a general large-scale language model. The reasoning ability of this enriched large-scale language model in handling legal issues is then defined as a legal ontology logical reasoner based on the large-scale language model. This involves selecting any chain rule {r1, r2, ..., r...} from the potential legal rule chain set Z. l} is used as the legal rule mining chain. The legal rule reasoning query operation is represented as q2. The result of the query operation q2 obtained through reasoning is the equivalent rule two, which is {r1,r2,...,r}. l}→r a2 As the second output of the model, the probability distribution of the potential legal rule chain set Z2 generated by the legal ontology logical reasoning model based on the large language model in this step is calculated based on the second output of the reasoning model. The expression is as follows:

[0068]

[0069] in, Let T1(O) be the probability of the large language model, and T1(O) be the function that converts the legal knowledge ontology O compiled by experts into LLM hints. For all legal rules preceding the i-th order legal rule in Z, p gLM To generate the probability distribution of possible legal rule chains by mining statistical chain rules;

[0070] Among them, equivalence rules are rules defined as logically equivalent. For example, after multiple steps of reasoning, a single step of reasoning is equivalent. If there is a logical reasoning relationship between title 1 and title 2, for example: individual 1 is title 1 of individual 2, and individual 2 is title 1 of individual 3, then individual 1 is title 2 of individual 3. The equivalence rule is that title 2 of title 2 is title 3. Formal expression: r1 is title 2, r2 is title 3, and the equivalence rule is: {r1,r1}->r2. Equivalence rules are also the target relationships in logical reasoning.

[0071] Step 1.3: Take the union of legal rule chain set one Z1 and legal rule chain set two Z2 to obtain the legal rule chain set Z:

[0072] Z = Z1∪Z2

[0073] Step 2 involves evaluating the reasonableness of the potential chain-like legal rule set Z generated by the dual-path mining in Step 1. Specifically, all output legal rule chains are evaluated in two dimensions, such as... Figure 3 As shown, the following detailed processes are further included:

[0074] Step 2.1: The legal logic rationality of the generated set of potential legal rule chains is evaluated using a large language model's legal logic rationality voting system. Specifically, this involves using a legal ontology prompter to integrate the legal ontology semantic knowledge of each legal rule chain into the prompts of the large language model. Through reasoning with the large language model, the rationality of the potential legal rule chains at the level of legal common sense is identified, and a logical rationality confidence score is assigned as the result of the logical rationality voting. The legal ontology prompter is used to generate prompts based on the semantic information of the legal ontology (CLCF).

[0075] Specifically, before processing the latent legal rule chain data using a large language model, each first-order inference logical relation (such as the previous statement that individual 1 is the title 1 of individual 2) is decomposed and semantic information about the domain and scope of the relation is extracted from the legal ontology. This semantic information is written in the form of triples (legal entity subject, legal entity predicate, legal entity object), for example, "[entity: subject value][link: predicate value][entity: object value]", and then the ontology triples corresponding to these relations are concatenated.

[0076] To illustrate the process, let's take the chain rule "LawChanges, LawChanges, PromulgateDecree → PromulgateDecree" as an example: The legal meaning of the LawChanges relation is "Law A is an amendment to Law B," written as [entity:LawA][link:LawChanges][entity:LawB]; the legal meaning of the PromulgateDecree relation is "The legislative body of the law is a certain organ," written as [entity:Law][link:PromulgateDecree][entity:State Organs]. This chain rule can be rewritten in a format that includes ontology information:

[0077] [entity:LawA][link:lawChanges][entity:LawB] ^

[0078] [entity:LawB][link:lawChanges][entity:LawC] ^

[0079] [entity:LawC][link:PromulgateDecree][entity:State0rgamsD] →

[0080] [entity:LawA][link:PromulgateDecree][entity:State0rgamsD].

[0081] In the legal ontology prompter, the large language model is used to analyze and explain the rationality of this logical reasoning chain through the thinking chain method. The prompt is divided into two steps: a) Based on legal common sense, understand the interconnection between triples, analyze the relationship between triples and logical reasoning, follow the rules of logical reasoning such as transitivity, and similar case judgments and causal relationships in legal reasoning, to construct a logical chain that shows how to reason from one triple to the next, including the legal rules or principles used in the reasoning.

[0082] The following are the instructions for the large language model in step a), instruction 1: "As a legal expert, explain the input triple rule chain and its reasoning from a legal perspective, providing a step-by-step explanation. The explanation should be as concise and detailed as possible; an example of a rule chain is as follows: [Entity:LawA][Link:LawChanges][Entity:LawB]: indicates that law A underwent changes, resulting in the emergence of law B; [Entity:LawB][Link:LawChanges][Entity:LawC]: indicates that law B, after undergoing some changes, became law C; [Entity:LawC][Link:PromulgateDecree][Entity:StateOrgansD]: indicates that law C was promulgated through the decree-issuing agency D; [Entity:LawA][Link:PromulgateDecree][Entity:StateOrgansD]: indicates that law A was ultimately promulgated by the decree-issuing agency D."

[0083] T1 = LLM (Instruction 1, triple chain rule to be evaluated)

[0084] (b) Assess the legitimacy of the legal rule chain by evaluating its reasonableness within legal common sense and the existing legal system, and whether all inferences are based on legal grounds. Calculate a confidence score for the reasonableness of legal logic, providing a score that reflects the logical strength of the legal rule chain and takes into account the uncertainties of real-world law, given the rigor of the legal rule chain and the reasonableness of legal logic.

[0085] The result of concatenating step a) is used as the input instruction for the large language model in step b), instruction 2: "As a legal expert, [the triple chain rule to be evaluated, interpreted as {T1}], evaluate the rationality of the reasoning chain based on legal logic rationality; the rule chain provides a confidence level between 0 and 1."

[0086] T2 = LLM (Instruction 2)

[0087] This process is repeated multiple times, and the arithmetic mean of the legal logic rationality confidence scores is taken as the final legal logic rationality vote result.

[0088] The thought process representation in the legal ontology prompter is shown in the following formula:

[0089]

[0090]

[0091]

[0092] in, To interpret a given chain rule relation R and the chain rule... And equivalent rule r a1 r a2 Given assessment result A, the total probability of confidence score C is given. The probability of evaluating chain rules and their interpretations;

[0093] The legal ontology prompter is repeatedly executed to perform the process of analyzing and interpreting the logical reasoning chain using the large language model and the thought chain method to determine its rationality. The legal logical rationality confidence score is recorded, and the arithmetic mean is taken as the final legal logical rationality confidence score. As shown in the following formula:

[0094]

[0095] Step 2.2: The credibility of the generated set of potential legal rule chains is evaluated using a fact-oriented fact-reasoning chain credibility vote based on a large language model. Specifically, this includes calculating the credibility confidence score of the legal fact-reasoning chain using a fact chain prompter; obtaining incomplete potential legal rule chains through the aforementioned mining process within a sparsely linked legal knowledge graph; constructing knowledge graph retrieval statements; searching for triplet fact data that meet the conditions in the knowledge graph and performing random sampling; and using a large language model to reason and calculate the credibility confidence score of each legal fact-reasoning chain, generating the corresponding legal fact-reasoning chain credibility confidence score as the legal logic rationality voting result.

[0096] Specifically, based on the mined multi-hop legal rule chain, a Cypher query statement for the knowledge graph is generated to query the graph database. The results are then randomly sampled to produce K records. If there are fewer than K results, the complete dataset is retained. Then, a fact chain prompter is enabled to traverse these sampled records and construct appropriate fact evaluation prompts.

[0097] For example, consider the query with the chain rule "LawChanges, LawChanges, PromulgateDecree → PromulgateDecree", which produces the following sampled facts: "[Law Change 1][link:LawChanges][Law Change 2]ˆ[Law Change 2][link:LawChanges][Law Change 3]ˆ[Law Change 3][link:PromulgateDecree][Law Enactment] → [Law Change 4][link:PromulgateDecree][Law Enactment]".

[0098] The fact chain prompter follows a three-step thought chain to determine the validity of facts: (1) The fact chain interpretation requires a clear definition and explanation of each element, its respective legal provisions, and their interrelationships; (2) The fact inference validity assessment involves analyzing the likelihood of factual conclusions drawn based on the previously elucidated chain; (3) The reliability of the inference is quantified by assigning a value from 0 to 1 after the inference validity assessment to complete the confidence level calculation. The formalized thought chain process of fact assessment is consistent with the formula of the legal ontology prompter, except that the input data is facts. The credibility confidence score of the legal fact reasoning chain is obtained by taking the arithmetic mean of the confidence scores of K fact chains. As shown in the following formula:

[0099]

[0100] Step 2.3 involves combining the confidence scores for legal logic rationality and legal factual reasoning chain credibility. Specifically, this includes weighted fusion of the confidence scores from the previous two steps. By weighting and integrating the confidence scores for legal logic rationality and legal factual reasoning chain credibility, a weighted confidence assessment score is generated for each potential legal rule chain. As shown in the following formula:

[0101]

[0102] In the formula, , These are the weight parameters, and they have... ,

[0103] The weighted confidence score AR is compared with a predefined reliability identification threshold L1 to filter out legal rule chains that are higher than the threshold, thereby constructing a set of reliable legal rule chains. Thus, the generative multi-hop legal rule chain mining and evaluation method finally achieves the extraction of reliable multi-hop legal reasoning chain rules from the legal knowledge graph.

[0104] Specifically, the credibility assessment of the fact-based legal fact reasoning chain further includes: using a fact chain prompter to convert candidate potential legal rule chains into graph query statements, randomly extracting matching triple chains from the legal knowledge graph, and using the fact chain prompter to generate fact assessment prompts to assess and score the credibility of the fact reasoning chain;

[0105] Step 3 involves conducting a reasonableness arbitration of the legal rule chain. Specifically, for example, a reasonableness arbitrator is constructed to perform a weighted synthesis of the legal logic reasonableness assessment of the legal rule chain obtained in Step 2 and the credibility assessment of the factual reasoning chain of the legal rule chain obtained in Step 3, and to filter out highly credible legal rule chains from the candidate rule set.

[0106] like Figure 4 As shown in the figure, the overall framework of a specific embodiment of the generative multi-hop legal rule chain mining and evaluation algorithm of the present invention is illustrated. The labels ①-④ in the figure correspond to the above steps.

[0107] In summary, the key points of this invention can be summarized as follows:

[0108] (1) Generative multi-hop chain rule mining: This method utilizes advanced generative language models and statistical methods to extract multi-hop chain legal rules from incomplete legal knowledge graphs. It can discover implicit and complex association rules, filling in the missing connections between entities in existing legal knowledge graphs.

[0109] (2) A novel technical framework for automatic mining and evaluation of legal rule chains was constructed, which includes four parts: "candidate chain rule generator", "legal ontology prompter", "fact chain prompter" and "legal rule rationality arbitrator". The unique design of the functional settings and interaction of these four components is the key to achieving high-quality legal rule mining.

[0110] (3) Dual-path mining to generate a set of potential legal rule chains as candidates: The dual-path approach of “chain rule generator based on generative language model” and “rule mining algorithm based on statistical method” helps to improve the quality and coverage of rule generation.

[0111] (4) Dual evaluation strategy: Combining the evaluation of legal logic by the "legal ontology prompter" and the evaluation of the rationality of the fact chain by the "fact chain prompter", a comprehensive rule verification mechanism is provided, which significantly improves the accuracy and reliability of rule mining.

[0112] (5) Legal Rule Reasonableness Arbitrator: This component comprehensively considers the assessment results of legal logic and factual reasonableness, and conducts the final rule reasonableness arbitration to ensure that the generated legal rules are logically rigorous and consistent with the actual legal facts.

[0113] This invention is not limited to the technical solutions described above. Any modifications and variations made by those skilled in the art without departing from the spirit and scope of this invention shall fall within the scope defined by the appended claims.

Claims

1. A generative multi-hop legal rule chain mining and evaluation method, characterized in that, Comprise: Step 1, two paths of statistical-based chain rule mining and large language model-based legal ontology logical inference are used to generate possible legal rules to generate a candidate set of potential legal rule chains Z, which further comprises a set of potential legal rule chains Z1 generated by statistical-based chain rule mining and a set of potential legal rule chains Z2 generated by a large language model-based legal ontology logical inference model; the large language model-based legal ontology logical inference model specifically comprises obtaining prompts from a previously expert-combed legal knowledge ontology, enriching a general large language model using the prompts, and then defining the reasoning ability of the enriched large language model in handling legal problems; Step 2, the set of potential legal rule chains Z generated by the double-path mining in step 1 is reasonably evaluated, specifically, all potential legal rule chains are evaluated from two dimensions, specifically including: i. The legal logic reasonableness voting of the large language model is used to evaluate the legal logic reasonableness of the generated set of potential legal rule chains, including integrating the legal ontology semantic knowledge of each legal rule chain into the prompts of the large language model using a legal ontology prompter, and completing the identification of the reasonableness of the legal rule chain at the level of legal common sense through reasoning with the large language model, and performing a logic reasonableness confidence score scoring as a logic reasonableness voting result; ii. The fact-oriented fact reasoning chain credibility voting of the large language model is used to evaluate the legal fact reasoning chain credibility of the generated set of potential legal rule chains, including calculating the legal fact reasoning chain credibility confidence score using a fact chain prompter, constructing a knowledge graph retrieval sentence by mining incomplete legal rule chains, searching for triple fact data meeting the conditions in the knowledge graph and performing random sampling, and reasoning and calculating the credibility confidence score of each legal fact reasoning chain using the large language model to produce the corresponding legal fact reasoning chain credibility confidence score as the legal logic reasonableness voting result; in the fact chain prompter, the Cypher query statement of the knowledge graph is generated from the multi-hop legal rule chains, the query results of the graph database are randomly sampled, and the fact chain prompter is enabled to traverse these sampled records and construct appropriate fact evaluation prompts; iii. The legal logic reasonableness confidence score and the legal fact reasoning chain credibility confidence score are integrated to generate a weighted confidence evaluation score of each potential legal rule chain; Step 3, legal rule chain reasonableness arbitration, specifically, the legal logic reasonableness evaluation of the legal rule chain obtained in step 2 and the fact reasoning chain credibility evaluation result of the legal rule chain obtained in step 2 are weighted and integrated, and a highly credible legal rule chain is filtered and selected from the set of potential legal rule chains Z.

2. The generative multi-hop legal rule chain mining and evaluation method of claim 1, wherein, The union of the set of potential legal rule chains Z1 and the set of potential legal rule chains Z2 is taken to obtain the set of potential legal rule chains Z, and the expression is as follows: Z=Z1∪Z2.

3. The generative multi-hop legal rule chain mining and evaluation method of claim 1, wherein, a weighted confidence assessment score for each potential legal rule chain The expression is as follows: ； wherein , are weight parameters, and have , is a legal fact reasoning chain plausibility confidence score, is a legal logic soundness confidence score.

Images