Legal decision prediction method and device based on large language model and logic enhancement

Abstract

The invention provides a law decision prediction method and device based on a large language model and logic enhancement, which can realize adaptive adjustment of law decision rules and improve the accuracy of decision prediction, and comprises the following steps: generating decision rules in the form of a first-order logic language by adopting an LLM model in combination with a law article original text and a historical case, describing key elements in legal provisions and historical cases through first-order logic symbols to form a consequent of a judgment rule, and taking a judgment label as a consequent of the judgment rule; calculating statement similarity of the cases based on a BGE vector model, and screening historical cases with high similarity to construct a confusion-prone case verification set; based on the confusion-prone case verification set, using a confusion perception comparative learning method to optimize a judgment rule; and for a to-be-predicted target case, generating a candidate tag of the target case based on a pre-trained domain small model, judging whether the target case accords with the candidate tag through the LLM model in combination with the thinking chain and the optimized judgment rule, and outputting a predicted judgment result.

Description

Technical Field

[0001] The present invention relates to the fields of large language model reasoning and legal judgment prediction, and specifically relates to a legal judgment prediction method and device based on a large language model and logic enhancement. Background Art

[0002] Legal Judgment Prediction (LJP) aims to automatically predict judgment results based on case fact descriptions, such as applicable legal articles, charged crimes, and prison terms.

[0003] Since legal judgment is a task in the legal field, professional knowledge in the legal field needs to be used in the process. The mainstream directions of existing legal judgment prediction methods that combine professional knowledge mainly include using model-based domain knowledge modeling and using retrieval methods to retrieve historical cases or domain knowledge. Model-based knowledge modeling focuses on the extraction of knowledge and the semantic association between specific words in case texts and knowledge. The retrieval method uses semantic similarity to retrieve historical cases or domain knowledge bases to improve the accuracy of the model in judgment prediction. The above two methods both associate and extract case texts and legal knowledge texts based on the semantic features on the surface of the text, and neither delves into the logical-level knowledge application. When dealing with complex legal cases, legal judgment is a rigorous and logical reasoning process, and only focusing on semantic similarity is difficult to meet the strict requirements of legal judgment prediction for accuracy and logic.

[0004] In order to improve the accuracy of judgment prediction, researchers have used the logical experience of adjudicating cases to improve the reasoning ability of the model and assist in legal judgment prediction. Existing technologies also use fixed logical symbol rules or classical legal judgment syllogisms to assist the reasoning process of the model, no longer limited to only considering the semantic similarity of cases, legal knowledge, and the current case. However, the logical symbol rules and classical legal judgment syllogisms used in the above methods are fixed logical rules and cannot dynamically adjust judgment rules according to complex and changeable cases. Fixed logical rules lack the ability to handle complex situations. Summary of the Invention

[0005] In view of the above problems, the present invention provides a legal judgment prediction method and device based on a large language model and logic enhancement, which can realize the adaptive adjustment of legal judgment rules and improve the accuracy of judgment prediction.

[0006] The technical solution is as follows: A legal judgment prediction method based on a large language model and logic enhancement includes the following steps:

[0007] Step 1: Combine the original legal provisions and historical cases, and use the LLM model generated based on the large language model to generate judgment rules in the form of first-order logic language. Describe the key elements in the legal provisions and historical cases through first-order logic symbols to form the antecedent of the judgment rule, and use the judgment label as the consequent of the judgment rule;

[0008] Step 2: Calculate the sentence similarity of historical cases based on the BGE vector model, and screen historical cases with high similarity to construct a verification set of easily confused cases; Based on the verification set of easily confused cases, use the confusion-aware contrast learning method to optimize the judgment rule;

[0009] Step 3: For the target case to be predicted, generate candidate labels for the target case based on the pre-trained domain small model Bert. The candidate labels include the legal provisions, charges, and sentences of the judgment of the target case predicted by the domain small model. Combine the chain of thought and the optimized judgment rule, and use the LLM model to judge whether the target case conforms to the candidate label, and output the predicted judgment result of the target case based on the matching candidate judgment label.

[0010] Furthermore, in Step 1, use the LLM model generated based on the large language model to generate judgment rules in the form of first-order logic language, and the specific implementation is as follows:

[0011] Provide the original legal provisions and historical cases;

[0012] Randomly select multiple cases from historical cases with the same judgment label, and extract the key elements of the cases. The key elements include the category of the criminal individual, time, place, criminal behavior, objective consequences, and subjective psychology;

[0013] Use first-order logic symbols including quantifier symbols, individual variable symbols, and predicate symbols to define the key elements, and use logical connectives to combine the quantifier symbols, individual variable symbols, and predicate symbols to form the antecedent of the judgment rule. The individual variable symbols are used to describe the subject and event elements in the case, the predicate symbols are used to describe the specific characteristics of criminal behavior, and the quantifier symbols are used to describe the universality or particularity of the judgment rule;

[0014] Use the judgment label as the consequent of the judgment rule. The judgment label includes legal provisions, charges, and sentences. Combine the antecedent and the consequent to form a judgment rule of legal provisions, charges, and sentences in first-order logic language.

[0015] Furthermore, in Step 2, the specific implementation of constructing the verification set of easily confused cases is as follows:

[0016] For the judgment rules obtained in step 1, taking one of the judgment labels as the target label, the historical case facts corresponding to the target label form a positive example set, and the case facts of other label categories form other case fact sets. The BGE model is used to convert each case fact in different sets into an embedding vector of a fixed dimension;

[0017] The dot product is used to calculate the similarity scores of the embedding vectors in the two sets, forming a similarity matrix. The similarity scores are used to reflect the proximity of the embedding vectors of the cases in the two sets in the vector space;

[0018] Each row of the similarity matrix is sorted in descending order to obtain the similarity ranking result of the cases of other categories and the cases of the target category;

[0019] According to the similarity ranking result, the case facts with high confusion are screened to form a negative example set, and the negative example set and the positive example set are constructed into a verification set of easily confused cases.

[0020] Furthermore, in step 2, based on the verification set of easily confused cases, the confusion-aware contrastive learning method is used to optimize the judgment rules, and the specific implementation is as follows:

[0021] For the rule optimization tree constructed for the judgment rules, the rule optimization tree takes the judgment rules as nodes, the parent-child relationship between the judgment rules as edges, and the accuracy rate of the judgment rules as the weight of the nodes;

[0022] The verification set of easily confused cases is used to evaluate each node in the rule optimization tree, and the node with the highest accuracy rate in the current rule optimization tree is selected;

[0023] The judgment rule corresponding to the selected node is optimized using confusion-aware contrastive learning, the optimization direction of the judgment rule is summarized and evaluated based on historical data, and a new judgment rule is generated;

[0024] The generated new judgment rule is added as a child node of the selected node to the rule optimization tree,

[0025] The verification set of easily confused cases is repeatedly used to evaluate each node in the rule optimization tree until a node with an accuracy rate reaching the threshold or the number of splits of the node reaches the agreed number of times.

[0026] Furthermore, the accuracy rate of the judgment rule is specifically evaluated as follows:

[0027] The case facts in the positive and negative example sets are put into the question to construct a binary single-choice question, and it is judged whether the case facts conform to the judgment rule;

[0028] The output selection result of the question is analyzed by the LLM model of the large model, and the ability of the judgment rule to distinguish easily confused cases is evaluated according to the accuracy rate of the selection result. The accuracy rate is calculated by the following formula:

[0029]

[0030] Among them, W(n) is the accuracy rate, TP is the number of positive examples correctly judged, TN is the number of negative examples correctly judged; FP is the number of positive examples wrongly judged, and FN is the number of negative examples wrongly judged.

[0031] Furthermore, the decision rules corresponding to the selected nodes are optimized using confusion-aware contrastive learning, and the specific implementation is as follows:

[0032] Construct contrastive samples based on the evaluation process of judging whether the case facts conform to the decision rules. Among them, the correct reasoning processes are formed by the problem-solving processes that are correctly predicted as positive and negative examples according to the evaluation process, and the wrong reasoning processes are formed by the problem-solving processes that are wrongly predicted as positive and negative examples according to the evaluation process;

[0033] Use the LLM model to compare the correct reasoning process and the wrong reasoning process, and summarize the advantages worth maintaining and the directions for improvement of the current decision rules.

[0034] Use the LLM model with prompt engineering to optimize the decision rules according to the summarized improvement directions to obtain new decision rules.

[0035] Furthermore, in step 3, for the target case to be predicted, if the text length of the case facts of the target case is greater than the set value, a case summary is formed through the LLM model, and the case summary includes the category of the criminal individual, time, place, criminal behavior, objective consequences, and subjective psychology.

[0036] Furthermore, in step 3, candidate labels for the target case are generated based on a pre-trained domain-specific small model, and the specific implementation is as follows:

[0037] Use the pre-trained small model Bert to screen the top N candidate labels, and based on the optimized decision rules obtained in step 2 through the LLM model, reason and judge whether the target case conforms to the candidate labels. If none of the N candidate labels conform, then traverse the decision rules corresponding to other candidate labels in random order.

[0038] By adopting the method of the present invention, the LLM model is used to generate judgment rules in the form of first-order logic based on legal knowledge and historical case summaries. By using the first-order logic symbolic language, a formal language that can express complex logical reasoning can be used to better describe the judgment rules. In the present invention, easily confused positive and negative cases are used as questions, combined with the tree structure of the rule optimization tree, and easily confused cases are used to simulate judgments to summarize the experience of correct and incorrect reasoning, and the judgment rules of the first-order logic are iteratively optimized in the direction of rule optimization, and finally the generated judgment rules can better distinguish easily confused cases. The LLM model uses the optimized first-order logic judgment rules combined with the judgment results of the thinking chain reasoning case, which can dig out the deep logical relationships contained in the legal texts and historical cases, realize the adaptive adjustment of the legal judgment rules, assist the LLM model to complete the logically rigorous legal judgment tasks, and improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A legal decision prediction method based on a large language model and logic enhancement in an embodiment;

[0040] Figure 2 is a schematic block diagram of a legal decision prediction method based on a large language model and logic enhancement in an embodiment;

[0041] Figure 3 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] See Figure 1 , a legal judgment prediction method based on a large language model and logic enhancement of the present invention comprises the following steps:

[0043] Step 1: Combine the original text of the law and historical cases, use the LLM model generated by the LLM model based on the large language model to generate judgment rules in the form of first-order logic language, and the generated judgment rules describe the key elements in the historical cases through first-order logic symbols to form the antecedent of the judgment rules, and use the judgment label as the consequent of the judgment rules. In the embodiment, the key elements include the category of criminal individuals, time, place, criminal behavior, objective consequences, and subjective psychology, and the judgment labels include legal provisions, charges, and sentences;

[0044] Step 2: Calculate the sentence similarity of historical cases based on the BGE vector model, select historical cases with high similarity to build a verification set of easily confused cases; Based on the verification set of easily confused cases, use the confusion-aware contrastive learning method to optimize the judgment rules;

[0045] Step 3: For the target case to be predicted, generate candidate labels for the target case based on the pre-trained domain small model. The candidate labels include the legal provisions, charges, and sentences of the judgment of the target case predicted by the pre-trained domain small model. Combining the chain of thought and judgment rules, the LLM model of the large language model determines whether the target case conforms to the candidate labels, and outputs the predicted judgment result of the target case based on the matching candidate judgment labels.

[0046] Existing mainstream methods of using model to model domain knowledge and retrieve historical cases or domain knowledge only perform association and extraction based on the semantic features on the surface of the text, without delving into the knowledge application at the logical level. When dealing with complex legal cases, only focusing on semantic similarity is difficult to meet the strict requirements of legal judgment prediction for accuracy and logic. And the existing methods of using fixed logical symbol rules or classical legal judgment syllogism to assist model reasoning, due to the fixed logical rules, cannot dynamically adjust the reasoning rules according to complex and changeable cases, lacking the ability to handle complex situations.

[0047] By adopting the method of this invention, the deep logical relationships contained in legal texts and historical cases can be mined to assist the LLM in completing the task of legally judgment with rigorous logic. First, the LLM model summarizes and generates judgment rules in the form of first-order logic based on legal knowledge and historical cases. Then, use confusing positive and negative cases as questions, and optimize the judgment rules of first-order logic based on confusion-aware contrast learning in combination with a tree structure. Finally, the LLM model uses the optimized first-order logic judgment rules to infer the judgment result of the case in combination with the chain of thought.

[0048] Specifically, in an embodiment of the present invention, in step 1, an LLM model generated based on a large language model is adopted to generate judgment rules in the form of first-order logic language, and the specific implementation is as follows:

[0049] Provide the original legal provisions and historical cases. The legal text is used as the basis for judging whether a certain law is violated. Summarizing the judgment rules in combination with the original legal provisions and historical cases can ensure that the generated judgment rules do not violate the original legal provisions;

[0050] Randomly select multiple cases from historical cases with the same judgment labels, and extract the key elements of the cases. The key elements include the category of the criminal individual, time, place, criminal behavior, objective consequences, and subjective psychology;

[0051] Define the key elements using first-order logic symbols including quantifier symbols, individual variable symbols, and predicate symbols. Combine the quantifier symbols, individual variable symbols, and predicate symbols using logical connectives to form the antecedent of the judgment rule. The individual variable symbols are used to describe the subjects and event elements in the case, the predicate symbols are used to describe the specific characteristics of criminal acts, and the quantifier symbols are used to describe the generality or particularity of the judgment rule. For example, the universal quantifier ANY and the existential quantifier EXIST are used to describe the generality and particularity of the judgment rule.

[0052] Use the judgment label as the consequent of the judgment rule. The judgment label includes the legal article, charge, and sentence. Combine the antecedent and the consequent to form the judgment rule for the legal article, charge, and sentence in the first-order logic language.

[0053] In one embodiment of the present invention, when generating the inference rule for the sentence, the sentencing standards stipulated in different legal articles are different. Therefore, in the method of the embodiment, 5 historical cases violating the same type of legal articles and sentenced to the same type of sentences are extracted to assist in generating the sentence judgment rule, and the sentencing standards involved in the legal article text are combined as auxiliary knowledge for the sentence judgment rule to ensure that the generated rule meets the legal requirements. The number of historical cases can also be more.

[0054] In step 1 of the embodiment, using the first-order logic (FOL) language and combining the prompt engineering method, automatically summarize and generate the legal judgment rules for the legal article, charge, and sentence, which can significantly reduce the intervention of human experts and greatly improve the rule construction efficiency. Different from the existing method based only on the surface semantic features of the text, the present invention delves into the logical level. By generating the judgment rule in the form of the first-order logic language, it can achieve a deeper logical analysis of the case, making the judgment rule more in line with the requirements of the rigorous and logical reasoning process of legal judgment. The existing method using fixed logical rules cannot handle complex and changeable cases, while the judgment rule generated by the method in the embodiment can be dynamically generated according to different legal article texts and historical cases, improving the ability of the LLM model to handle complex situations and greatly enhancing the accuracy and logic of legal judgment prediction.

[0055] In step 2 of the embodiment, first calculate the sentence similarity of historical cases based on the BGE vector model, screen historical cases with high similarity to construct a verification set of confusing cases, and optimize the judgment rules for each type of judgment label one by one. Use the rule optimization tree to assist in the update of the rules. Each tree represents the update process of the judgment rule corresponding to a type of label, and the nodes on the tree represent the generated first-order logic rules. Step 2 can be divided into 2 sub-steps:

[0056] 201: For the judgment rules obtained in step 1, taking one of the judgment labels as the target label target, filter the historical case facts corresponding to the target label target to form a positive example set S_postive, and form other case fact sets S_other = {s_1, s_2, s_3,...} for the historical case facts of other label categories. s_1, s_2, s_3 are other case facts in the set. Use the BGE model to convert each case fact in different sets into an embedded vector of a fixed dimension;

[0057] Calculate the similarity scores of the embedded vectors in the two sets using the dot product to form a similarity matrix A. A(i, j) represents the similarity score between case S_postive(i) in the positive example set S_postiv and case S_other(j) in the other case fact set S_other. The dot product value indicates the distance between the two sentences in the semantic space. The higher the dot product value, the higher the semantic similarity. The similarity score is used to reflect the proximity of the embedded vectors of the cases in the two sets in the vector space;

[0058] Sort each row of the similarity matrix in descending order to obtain the similarity ranking result of other category cases and the target category cases;

[0059] Filter the case facts with high confusion degree according to the similarity ranking result to form a negative example set S_negative ∈ S_other, and construct the negative example set S_negative and the positive example set S_postiv into a verification set of easily confused cases. The verification set of easily confused cases is expressed as V_target = S_negative ∪ S_postive.

[0060] In step 202, based on the verification set of easily confused cases, use the confusion-aware contrast learning method to optimize the judgment rules, and the specific implementation is as follows:

[0061] a: Construct a rule optimization tree T_label = (N, E, W) for the judgment rules, where N represents nodes, E represents edges, and W represents weights. The node N is different versions of the judgment rules in the optimization process. The edges represent the parent-child relationship between the rules, and the weight w represents the value of the node. In implementation, take the accuracy of the judgment rules;

[0062] b: Use the verification set of easily confused cases to evaluate each node in the rule optimization tree, and select the node n* with the highest accuracy in the current rule optimization tree. The node n* corresponds to the judgment rule R*, indicating that the judgment rule R* has the highest judgment accuracy in the verification set of easily confused cases;

[0063] c: Optimize the decision rule R* corresponding to the selected node n* using confusion-aware contrastive learning, summarize the evaluation history to obtain the optimization direction of the decision rule, and generate a new decision rule; add the generated new decision rule as the child node of the selected node to the rule optimization tree;

[0064] d: Repeatedly evaluate each node in the rule optimization tree using the verification set of confusing cases until a node with an accuracy rate reaching the threshold or the number of splits of the node reaches the agreed number of times. The number of splits of the node is the number of times of generating child nodes.

[0065] In an embodiment of the present invention, in step b of step 202, the accuracy rate of the decision rule can be evaluated as follows:

[0066] Put the case facts in the positive example set and negative example set of the target label into the question, construct a binary single-choice question, and judge whether the case facts conform to the decision rule;

[0067] Analyze the question through the LLM model of the large model. The LLM model will use the decision rule to analyze the case facts, judge whether the case facts in the positive example set and negative example set put into the question conform to the decision rule, and output the selection result, that is, the case facts conform to the decision rule or do not conform to the decision rule;

[0068] Then, the ability of the decision rule to distinguish confusing cases can be evaluated according to the accuracy rate of the selection result. In the embodiment, the accuracy rate is calculated by the following formula:

[0069]

[0070] Where, W(n) is the accuracy rate, TP is the number of cases correctly judged as positive examples, TN is the number of cases correctly judged as negative examples; FP is the number of cases wrongly judged as positive examples, FN is the number of cases wrongly judged as negative examples.

[0071] Select the node with the largest weight in the tree according to the node weights in the tree That is, the rule R* with the highest accuracy rate in the verification set.

[0072] In step c of step 202 of the embodiment, use confusion-aware contrastive learning to optimize the decision rule corresponding to the selected node, and the specific implementation is as follows:

[0073] Establish contrast samples according to the evaluation process of judging whether the case facts conform to the decision rule. Among them, the correct reasoning records are formed according to the problem-solving processes correctly predicted as positive examples and negative examples in the evaluation process, and the wrong reasoning records are formed according to the problem-solving processes wrongly predicted as positive examples and negative examples in the evaluation process;

[0074] Use the LLM model to compare the correct reasoning records and incorrect reasoning records, summarize the advantages worth maintaining and the directions for improvement of the current judgment rules.

[0075] Use the LLM model to use prompt engineering, optimize the judgment rules according to the summarized improvement directions, obtain new judgment rules, and add the obtained new judgment rule R’ as a child node of R* to the rule optimization tree.

[0076] In step d of step 202 of the embodiment, repeatedly use the verification set of confusing cases to evaluate each node in the rule optimization tree until a node with an accuracy rate reaching the threshold or the number of splits of the node reaches the agreed number of times.

[0077] In step 2, calculate the sentence similarity of the judgment rules based on the BGE vector model, screen and construct a verification set of confusing cases, and use the confusion-aware contrastive learning method to optimize the judgment rules. By executing step 2, the judgment rules can be made more accurate and rigorous, and the judgment ability of the LLM model in the face of similar but slightly different cases can be improved.

[0078] In step 3 of an embodiment, for the target case to be predicted, content summarization can be performed first to avoid the details of the case facts being too detailed and affecting the judgment. Specifically, if the text length of the case facts of the target case is greater than the set value, a case summary is formed through the LLM model, and the case summary includes the crime time, crime location, criminal behavior, criminal consequences, and subjective psychology. The set value given in the embodiment is 5000 characters, and this set value can be adjusted.

[0079] Step 3 of the embodiment is specifically executed as follows:

[0080] For the target case to be predicted, generate candidate labels for the target case based on the pre-trained domain small model Bert. The candidate labels include the legal provisions, charges, and sentences predicted by the pre-trained domain small model for the judgment of the target case.

[0081] Screen the top N candidate labels, and the candidate labels include the legal provisions, charges, and sentences predicted by the pre-trained small model Bert.

[0082] Combine the chain of thought and the judgment rules, and use the LLM model of the large language model to judge whether the target case conforms to the candidate labels. Based on the matching candidate judgment labels, output the predicted judgment result of the target case. The candidate judgment includes the legal provisions, charges, and sentences of the case judgment. The chain of thought in the embodiment refers to indicating in the prompt words to let the LLM model think carefully step by step.

[0083] If none of the N candidate tags match, traverse the decision rules corresponding to other candidate tags in random order. N is a positive integer. In the embodiment, N is taken as 10, and N can also be other numbers. When steps 1, 2, and 3 of the present invention are executed relying on the LLM model, they can all be output by the LLM model according to the specific requirements of the steps of the method by setting the prompt word template.

[0084] In step 3 of the embodiment, the LLM model uses the decision rule corresponding to the tag and combines the chain-of-thought reasoning to predict the judgment result of the target case plot, which can analyze the case more comprehensively and deeply, and improve the accuracy and logic of legal judgment prediction.

[0085] Figure 2 The schematic block diagram of the legal judgment prediction method based on the large language model and logic enhancement in the embodiment is shown. The part with a blue background in the upper left corner of the figure is the decision rule generation part in step 1; the part with a gray background in the figure is the decision rule optimization in step 2; the part with a green background in the figure is to complete the legal rule pre-judgment task using the optimized decision rule in step 3.

[0086] The method provided in the embodiment of the present invention can dynamically optimize the legal judgment rule, aiming to achieve the adaptive adjustment of the rule. In the solution, the first-order logic symbolic language, a formal language capable of expressing complex logical reasoning, is used to describe the judgment rule; combined with the tree splitting process and contrast learning, the confusing cases are used to simulate the judgment to summarize the experience of correct and incorrect reasoning, and generate the rule optimization direction for iterative optimization; finally, the generated judgment rule can better distinguish the confusing cases, so as to assist the LLM model to make more accurate legal judgments.

[0087] Experimental verification:

[0088] After experimental demonstration, the method of the present invention has achieved the optimal effect in two publicly available legal judgment prediction data sets. Here, RLJP represents the legal judgment prediction method based on the large language model and logic enhancement of the present invention.

[0089] As shown in Tables 1 and 2, the performance comparison results of RLJP and various baseline models on the CAIL2018 and cjo22 datasets are respectively presented. These results are the average values obtained from five test rounds. The experimental results show that RLJP has achieved the best performance in all metrics, verifying the enhancement effect of the first-order logic decision rule on LJP (Legal Judgment Prediction, LJP). Specifically, compared with the sub-optimal model, the average improvements of Acc and Ma-F of RLJP on the CAIL2018 and CJ022 datasets are 1.43% and 14.98% respectively. These experimental results fully verify the effectiveness and superiority of the method of the present invention in LJP, indicating its significant advantages in legal case judgments.

[0090] Table 1

[0091]

[0092] Table 1: Experiment results of LJP in CALL2018 dataset. “Bold” indicates optimal results, and “ underline ” indicates sub-optimal reesults. The experimental results represent the average values obtained from five test rounds.

[0093] Table 2

[0094]

[0095] Table 2: Experiment results of LJP in CJO22 dataset. “Bold” indicates optimal results. and “ underline ” indicates sub-optimal results. The experimental results represent the average values obtained from five test rounds.

[0096] Moreover, according to the length of the case facts, the present invention extracts the top 5% of the data in each dataset to detect the effect of the RLJP method in long and logically complex cases, which are called CAIL2018_Long and CJO22_Long. The experimental results in Tables 3 and 4 show that the RLJP method proposed by the present invention performs well in judging complex facts. The present invention can conclude that using the first-order logic judgment rule can effectively capture the key elements in the case facts, reduce the interference of redundant information, and focus on the key facts decisive for judgment prediction. In contrast, PLJP, which reorganizes case facts using fixed three elements, may ignore some important details. The first-order logic judgment rule helps the model better understand the logical structure and legal terms in complex facts, can focus on important logical details, and reduce misjudgment caused by overly long text.

[0097] Table 3: Experimental Results of RLJP and PLJP Methods in CAIL2018_Long Dataset

[0098]

[0099] Table 4: Experimental Results of RLJP and PLJP Methods in CJ022_Long Dataset

[0100]

[0101] In an embodiment of the present invention, there is also provided a legal judgment prediction system based on a large language model and logic enhancement, including:

[0102] An initialization logic judgment rule module, which is configured to combine the original text of the law and historical cases, and use the LLM model generated based on the large language model to generate judgment rules in the form of first-order logic language. The antecedent of the judgment rule is constituted by describing the key elements in the legal provisions and historical cases through first-order logic symbols, and the judgment label is used as the consequent of the judgment rule;

[0103] An optimized judgment rule module, which is configured to calculate the sentence similarity of historical cases based on the BGE vector model, screen historical cases with high similarity to construct a verification set of confusing cases; and optimize the judgment rule based on the verification set of confusing cases using a confusion-aware contrast learning method;

[0104] The judgment assessment module is configured to generate candidate labels for the target case to be predicted based on the pre-trained domain small model Bert. The candidate labels include the legal provisions, charges, and sentences of the judgment of the target case predicted by the domain small model. Combining the chain of thought and the optimized judgment rules, the LLM model is used to determine whether the target case conforms to the candidate labels, and the predicted judgment result of the target case is output based on the conforming candidate judgment labels.

[0105] In an embodiment of the present invention, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the legal judgment prediction method based on the large language model and logic enhancement as described above is implemented.

[0106] The computer device may be a terminal, and its internal structure diagram may be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the legal judgment prediction method based on the large language model and logic enhancement. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball, or touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0107] The memory may be, but is not limited to, a random access memory (Random Access Memory, abbreviated as RAM), a read-only memory (Read Only Memory, abbreviated as ROM), a programmable read-only memory (Programmable Read-Only Memory, abbreviated as PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory, abbreviated as EPROM), an electrically erasable programmable read-only memory (Electric Erasable Programmable Read-Only Memory, abbreviated as EEPROM), etc. Among them, the memory is used to store the program, and the processor executes the program after receiving the execution instruction.

[0108] A processor may be an integrated circuit chip with the ability to process signals. The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. The processor may also be other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0109] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0110] In an embodiment of the present invention, there is also provided a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the legal judgment prediction method based on a large language model and logic enhancement as described above.

[0111] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a computer device, or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0112] Embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, computer devices, or computer program products according to embodiments of the present invention. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in the flowchart and / or block diagram.

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in the flowchart.

[0114] In an embodiment of the present invention, a computer program product is also provided, including a computer program / instructions, which, when executed by a processor, implement the steps of the above method.

[0115] In actual application processes, the above computer program product includes, but is not limited to: smart phones, desktop computers, laptop computers, tablet computers, host computers, and server platforms, etc., and no specific limitations are made here.

[0116] The above has introduced in detail the application of the legal judgment prediction method, system, computer device, computer-readable storage medium, and computer program product provided by the present invention based on large language models and logic enhancement. Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A legal judgment prediction method based on large language models and logic enhancement, characterized in that Including the following steps: Step 1: Combine the original legal provisions and historical cases, and use the LLM model generated based on the large language model to generate judgment rules in the form of first-order logic language. The antecedent of the judgment rule is formed by describing the key elements in the legal provisions and historical cases through first-order logic symbols, and the judgment label is used as the consequent of the judgment rule; Step 2: Calculate the sentence similarity of historical cases based on the BGE vector model, and screen historical cases with high similarity to construct a verification set of easily confused cases; Based on the verification set of easily confused cases, use the confusion-aware contrast learning method to optimize the judgment rule; Step 3: For the target case to be predicted, generate candidate labels for the target case based on the pre-trained domain small model Bert. The candidate labels include the legal provisions, charges, and sentences of the judgment of the target case predicted by the domain small model. Combine the chain of thought and the optimized judgment rule, and use the LLM model to determine whether the target case conforms to the candidate label, and output the predicted judgment result of the target case based on the matching candidate judgment label.

2. The legal judgment prediction method based on large language models and logic enhancement according to claim 1, wherein In Step 1, use the LLM model generated based on the large language model to generate judgment rules in the form of first-order logic language, and the specific implementation is as follows: Provide the original legal provisions and historical cases; Randomly select multiple cases from historical cases with the same judgment label, and extract the key elements of the cases. The key elements include the category of the criminal individual, time, place, criminal behavior, objective consequences, and subjective psychology; Use first-order logic symbols including quantifier symbols, individual variable symbols, and predicate symbols to define the key elements, and use logical connectives to combine the quantifier symbols, individual variable symbols, and predicate symbols to form the antecedent of the judgment rule. The individual variable symbol is used to describe the subject and event elements in the case, the predicate symbol is used to describe the specific characteristics of the criminal behavior, and the quantifier symbol is used to describe the universality or particularity of the judgment rule; Use the judgment label as the consequent of the judgment rule. The judgment label includes legal provisions, charges, and sentences. Combine the antecedent and the consequent to form a judgment rule of legal provisions, charges, and sentences in first-order logic language.

3. The legal judgment prediction method based on large language models and logic enhancement according to claim 1, wherein In Step 2, the specific implementation of constructing the verification set of easily confused cases is as follows: For the judgment rule obtained in Step 1, use one of the judgment labels as the target label, and the historical case facts corresponding to the target label form a positive example set, and the case facts of other label categories form other case fact sets. Use the BGE model to convert each case fact in different sets into an embedding vector of a fixed dimension; Use the dot product to calculate the similarity score of the embedding vectors in the two sets to form a similarity matrix. The similarity score is used to reflect the closeness of the embedding vectors of the cases in the two sets in the vector space; Sort each row of the similarity matrix in descending order to obtain the similarity ranking result of other category cases and the target category cases; According to the similarity ranking result, screen the case facts with high confusion degree to form a negative example set, and construct the negative example set and the positive example set into a verification set of easily confused cases.

4. The legal judgment prediction method based on large language models and logic enhancement according to claim 3, wherein In Step 2, based on the verification set of easily confused cases, use the confusion-aware contrast learning method to optimize the judgment rule, and the specific implementation is as follows: A rule optimization tree constructed for judgment rules, where the rule optimization tree takes judgment rules as nodes, the parent-child relationship between judgment rules as edges, and the accuracy rate of judgment rules as the weight of nodes; Use the verification set of confusing cases to evaluate each node in the rule optimization tree, and select the node with the highest accuracy rate in the current rule optimization tree; Use confusion-aware contrastive learning to optimize the judgment rules corresponding to the selected nodes, summarize the evaluation history to obtain the optimization direction of the judgment rules, and generate new judgment rules; Add the generated new judgment rules as the child nodes of the selected nodes to the rule optimization tree, Repeat using the verification set of confusing cases to evaluate each node in the rule optimization tree until a node with an accuracy rate reaching the threshold or the number of splits of the node reaches the agreed number of times.

5. The legal judgment prediction method based on large language models and logic enhancement according to claim 4, characterized in that The accuracy rate of the judgment rules is specifically evaluated as follows: Put the case facts in the positive and negative example sets into the questions to construct a binary multiple-choice question, and judge whether the case facts conform to the judgment rules; Analyze the output selection results of the questions through the LLM model of the large model, and evaluate the ability of the judgment rules to distinguish confusing cases according to the accuracy rate of the selection results. The accuracy rate is calculated by the following formula: Among them, W(n) is the accuracy rate, TP is the number of cases correctly judged as positive examples, TN is the number of cases correctly judged as negative examples; FP is the number of cases wrongly judged as positive examples, and FN is the number of cases wrongly judged as negative examples.

6. The legal judgment prediction method based on large language models and logic enhancement according to claim 5, characterized in that Use confusion-aware contrastive learning to optimize the judgment rules corresponding to the selected nodes, and the specific implementation is as follows: Establish contrast samples according to the evaluation process of judging whether the case facts conform to the judgment rules. Among them, the correct reasoning process is formed by the problem-solving process that is correctly predicted as positive and negative examples according to the evaluation process, and the wrong reasoning process is formed by the problem-solving process that is wrongly predicted as positive and negative examples according to the evaluation process; Use the LLM model to compare the correct reasoning process and the wrong reasoning process, and summarize the advantages worth maintaining and the improvement directions of the current judgment rules, Use the LLM model to use prompt engineering to optimize the judgment rules according to the summarized improvement directions to obtain new judgment rules.

7. The legal judgment prediction method based on large language models and logic enhancement according to claim 1, characterized in that, In step 3, for the target case to be predicted, if the text length of the case facts of the target case is greater than the set value, a case summary is formed through the LLM model, and the case summary includes the category of criminal individuals, time, place, criminal behavior, objective consequences, and subjective psychology.

8. The legal judgment prediction method based on large language models and logic enhancement according to claim 1, characterized in that In step 3, generate candidate labels for the target case based on the pre-trained domain small model, and the specific implementation is as follows: Use the pre-trained small model Bert to screen the top N candidate labels, and based on the optimized judgment rules obtained in step 2 through the LLM model, reason and judge whether the target case conforms to the candidate labels. If none of the N candidate labels conform, traverse the judgment rules corresponding to other candidate labels in random order.

9. A legal judgment prediction system based on large language models and logic enhancement, characterized in that, Include: Initialize the logical judgment rule module, which is configured to combine the original text of the law and historical cases, adopt the LLM model generated based on the large language model, generate judgment rules in the form of first-order logic language, and describe the key elements in the legal provisions and historical cases through first-order logic symbols to form the antecedent of the judgment rule, and use the judgment label as the consequent of the judgment rule; Optimize the judgment rule module, which is configured to calculate the sentence similarity of historical cases based on the BGE vector model, and screen historical cases with high similarity to construct a verification set of easily confused cases; Based on the verification set of easily confused cases, use the confusion-aware contrast learning method to optimize the judgment rule; Judgment assessment module, which is configured to, for the target case to be predicted, generate candidate labels for the target case based on the pre-trained domain small model Bert. The candidate labels include the legal provisions, charges, and sentences of the judgment of the target case predicted by the domain small model. Combine the chain of thought and the optimized judgment rule, and use the LLM model to judge whether the target case conforms to the candidate label, and output the predicted judgment result of the target case based on the matching candidate judgment label.

10. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the legal judgment prediction method based on the large language model and logic enhancement as described in claim 1.

11. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by the processor, it implements the legal judgment prediction method based on the large language model and logic enhancement as described in claim 1.

12. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the method described in claim 1 are implemented.

Images