Hierarchical reply reasoning method and system for deep legal element extraction

Abstract

The invention relates to a hierarchical reply reasoning method and system for deep legal element extraction, and belongs to the technical field of natural language processing and deep learning in legal artificial intelligence, and the method comprises the steps: converting an element extraction task into a question and answer task, and enabling a question list to be reconstructed into a hierarchical question tree according to subordinate logic; performing depth-first traversal on the hierarchical question tree through multiple rounds of question and answer dialogues, and sequentially answering the questions according to the response to the father node; wherein in the question answering process, a repeated answering reasoning method is adopted, namely, after an initial answer is obtained, the initial answer is improved according to question explanation, and a final answer is output. According to the method, an element extraction task is converted into a question and answer task, so that the problems of internal logic deficiency of legal elements, deficiency of legal knowledge, weak LLM instruction tracking capability and the like are solved by utilizing strong knowledge and reasoning capability of LLM and through hierarchical repeated answer reasoning, and more accurate and powerful support is provided for legal practice.

Description

Technical Field

[0001] The present invention relates to a hierarchical re-answer reasoning method and system for deep legal element extraction, belonging to the technical field of natural language processing and deep learning in legal artificial intelligence. Background Art

[0002] Legal elements refer to key factual details in judicial documents that may affect judicial decisions, such as specific components of determining guilt in criminal cases and key facts in civil cases. In essence, extracting legal elements involves structuring and compressing effective information in a case, which is an important way to improve the efficiency of human-machine collaborative case processing. At the same time, legal elements can effectively improve the performance and interpretability of models in downstream applications, such as legal judgment prediction and case retrieval.

[0003] At present, legal element extraction tasks can be mainly divided into two categories according to methods. One method draws on the success of feature extraction in general fields, and its elements are extracted from legal documents based on annotations pre-defined by experts or according to rules. For example, the Legal Element Extraction dataset provides a comprehensive criminal element extraction dataset with 159 legal element labels. Another method regards legal elements as classification labels, mapping paragraphs or sentences in documents to relevant element labels. For example, the 2019 China Law Research Cup Judicial Artificial Intelligence Challenge (CAIL-2019) element recognition dataset aims to identify legal elements in given fact descriptions, with 20 labels for each type of case (divorce, labor, and private lending cases). However, both methods have their shortcomings. The former directly extracts elements from documents, which is only a structural summary of the case, and it is difficult to directly apply these elements to assist judicial decisions. The latter basically determines the elements of case matching, but cannot conduct a comprehensive and in-depth analysis of the case. In fact, existing research mostly focuses on the extraction of shallow legal elements, lacking the ability to understand deep semantics and mine complex relationships.

[0004] In order to further improve the accuracy and efficiency of legal element extraction, it is particularly important to improve the reasoning ability of the large language model (LLM). Existing LLM reasoning enhancement methods can be roughly divided into two categories. One category is represented by the chain of thought (CoT), which reduces the complexity of reasoning by decomposing the task into several steps and solving them in sequence. The CoT-based method takes advantage of the consistency of the model's reasoning output and effectively improves the performance of tasks such as mathematical and logical reasoning. However, in the absence of knowledge or when using weak LLM, this method performs poorly. The other category aims to enable the model to correct its own output through the three-stage paradigm of "response-feedback-improvement", that is, self-correction. According to the correction time of feedback, self-correction is divided into two categories: post-correction and generation-time correction. Post-correction occurs after the response is generated and is more flexible. It can be corrected using a variety of methods after generation; while generation-time correction gradually improves the intermediate reasoning steps by providing feedback. The key to self-correction lies in how to provide effective feedback. Generally speaking, feedback that uses additional information is better than feedback that relies solely on the model's own capabilities. Compared with stronger LLMs, the self-correction effect of weak LLMs tends to be less prominent because weak LLMs sometimes misinterpret feedback and improvement instructions.

[0005] In summary, the existing legal element extraction methods fail to accurately and efficiently extract elements that are close to legal clauses and contain hierarchical logical relationships, making it difficult to intelligently assist judges in making decisions. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a hierarchical re-answer reasoning method and system for deep legal element extraction, aiming to perform deep legal element extraction through hierarchical re-answer reasoning, focusing on the information in the case that directly contributes to the judge's decision, and extracting legal elements with logic and more standardization.

[0007] Terminology explanation:

[0008] 1. Re-answer reasoning: The re-answer reasoning method mentioned in the present invention specifically refers to packaging the question and the first answer into a set of dialogues after the system completes the first answer to the user's question, which serves as a reference for the system's second answer. The system refers to the dialogue and additional question explanations and answers the question again. Specific examples can be found in the attached Figure 2 .

[0009] 2. Qwen2.5-14B model: The Qwen2.5-14B model is a large-scale pre-trained language model developed by Alibaba Cloud, which is further optimized and extended based on the Tongyi Qianwen series of models. The model has 14 billion parameters and is designed to provide powerful natural language understanding and generation capabilities, especially optimized for multimodal and long text processing.

[0010] 3. Qwen-turbo model: The Qwen-turbo model is the commercial version of the Tongyi Qianwen model. Compared with the open source Qwen2.5-14B model, the commercial version of the Qwen-turbo model has the latest capabilities and improvements and is the fastest and lowest cost model in the Tongyi Qianwen series.

[0011] 4. Zero-shot prompt: A prompt is an input text used to guide the model to generate a specific type of output. The specific content varies depending on the target task. A zero-shot prompt means that no additional examples or training data are provided to the model during the prompt writing process.

[0012] The present invention adopts the following technical solution:

[0013] A hierarchical re-answer reasoning method for deep legal element extraction includes the following steps;

[0014] (1) Convert the feature extraction task into a question answering (QA) task, so that the question list is reconstructed into a hierarchical question tree based on the subordinate logic;

[0015] (2) Through multiple rounds of question-and-answer dialogues, the hierarchical question tree is traversed in depth first, and questions are answered in sequence according to the responses to the parent nodes. The process of answering questions adopts the repeated answer reasoning method, that is, after obtaining the initial answer, the initial answer is improved according to the explanation of the question and the final answer is output.

[0016] Preferably, in step (1), the question-answering task is in a yes / no question-answering format, with one element corresponding to one question;

[0017] A hierarchical question tree is established by using the internal logic between the element questions. The hierarchical question tree includes top-level nodes and question nodes. The top-level node (level 0) is the question category, such as "litigation request", "borrower", etc. Each question node represents a separate element question. In the present invention, the parent node is the premise question of the child node, and the child node is a further refinement and expansion of the parent node question. For example, the parent node question "whether to claim repayment of interest" and its child node question "whether to explain the interest calculation base, period and calculation process".

[0018] Different from the traditional method of directly extracting elements from legal documents, the present invention aims to extract elements that are closer to legal clauses and have internal logic such as classification and hierarchical relationships. It is crucial to convert this task into a yes / no QA format, thereby utilizing the knowledge and reasoning ability of LLM to answer each question related to the element.

[0019] Preferably, the method for constructing the hierarchical problem tree is:

[0020] For the t-element problem {q1,q2,…,qt}, divided into s categories according to the subordinate relationship Each category is further divided into several layers. In this case, the element problem {q1,q2,…,q t} is rewritten as where q i represents the i-th problem in the first level (level 1) problem, q ij represents the jth problem in the second level (level 2) under the i-th problem in the first level, q ijk Represents question q ij The kth question in the third level (level 3) below.

[0021] Preferably, the implementation process in step (2) is:

[0022] Iterate through the question categories;

[0023] For a given question category, traverse the first level (level 1), input the first level questions into the re-answer reasoner, and get the corresponding answers;

[0024] Take the first-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the first-level questions, that is, enter the QA dialogue of the first-level questions into the history dialogue list as a reference for the second-level (level 2) questions. Specifically, when the answer to the first-level question is "no", the answer to the second-level question is directly output as "no". When the answer to the first-level question is "yes", input the history dialogue list and the second-level (level 2) question into the re-answer reasoner to obtain the corresponding answer.

[0025] Take the second-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the second-level questions, that is, enter the QA dialogue of the second-level questions into the history dialogue list as a reference for the third-level (level 3) questions. Specifically, when the answer to the second-level question is "no", the answer to the third-level question is directly output as "no". When the answer to the second-level question is "yes", the history dialogue list and the third-level (level 3) question are input into the re-answer reasoner to obtain the corresponding answer.

[0026] And so on, until all levels of all hierarchical problem trees are traversed.

[0027] In order to make full use of the inherent logic between the element questions, the present invention does not directly answer the above question list, but constructs it into a hierarchical question tree. By integrating the multi-round QA format of LLM, the hierarchical question tree is traversed in depth first, and each question is answered in sequence according to the response to the parent node. The hierarchical QA structure is the core of the present invention to efficiently and accurately complete the element extraction.

[0028] Preferably, for a given problem category c H , generate the first level (level 1) question q through the re-answer reasoner i The answer is i , expressed as:

[0029]

[0030] in, represents the re-answer reasoner, d represents the legal document, Represents the text connection function;

[0031] The answer r i After post-processing, when r i = "Yes", the QA dialogue [q i ,r i ]Add to the history conversation list pre1;

[0032] For a given level 1 problem q i , traverse the second level problem, when r i = "No", the second level (level 2) question directly outputs "No", when r i = "Yes", the second level (level 2) question q ij and pre1 are input into the re-answer reasoner to obtain q ij The answer is ij :

[0033]

[0034] Where pre1 = [q i ,r i ];

[0035] The answer r ij After post-processing, when r ij = "Yes", the QA dialogue [q ij ,r ij ] Added to the history conversation list pre2;

[0036] For a given level 2 problem q ij , traverse the second level (level 3) problem, when r ij= "No", the second level (level 3) question directly outputs "No", when r ij = "Yes", the third level (level 3) question q ijk and pre2 are input into the re-answer reasoner to obtain q ijk The answer is ijk :

[0037]

[0038] Where pre2 = [q i ,r i ,q ij ,r ij ];

[0039] When the number of levels in the hierarchical problem tree is greater than three, the same analogy can be applied.

[0040] Preferably, the re-answer reasoner includes an LLM reasoner and an LLM re-answerer, and the process of generating the answer by the re-answer reasoning method is:

[0041] First, a question in a certain layer, or a question in a certain layer and the historical dialogue list of its corresponding parent node are input into the LLM reasoner to obtain the initial answer;

[0042] A more powerful LLM and manual correction are used to generate a question explanation for each factor question. The initial answer and question explanation are input into the LLM repeater, and the answer is re-reasoned and generated as the final answer output.

[0043] In order to further improve the model reasoning ability, the present invention adopts re-answer reasoning to improve the answer as a key step in legal element extraction. Different from the previous self-correction method, the present invention integrates the feedback and improvement stages into a unified program, namely the re-answer reasoning module, thereby reducing the difficulty of following weak LLM instructions and improving the accuracy and efficiency of answers.

[0044] Preferably, the LLM reasoner and the LLM repeater both use the Qwen2.5-14B model, and the LLM with stronger performance uses the Qwen-turbo model. The present invention uses the LLM with stronger performance to generate question explanations to assist the LLM with weaker performance to complete tasks, thereby reducing the difficulty of following the weak LLM instructions and effectively improving the accuracy of the LLM answers.

[0045] Preferably, for a given question q, an initial answer R is obtained using an LLM reasoner init , expressed as:

[0046]

[0047] in, represents the LLM reasoner, p init represents a zero-sample prompt word, which is used to guide the LLM reasoner to generate an initial answer. The specific content is "You are a senior legal practitioner. Answer the user's question based on the case content." || represents a connection relationship. d represents a legal document. pre3 represents the QA dialogue of the parent node question. c represents the question category. Represents the text connection function;

[0048] [q,R init ] Add to the historical conversation pre′;

[0049] Input pre′ and q into the LLM repeater to get the final answer R final ,express:

[0050]

[0051] in, represents the LLM repeater, e represents the corresponding question explanation, pre′=[q,R init ], p final Represents a zero-sample prompt word, which is used to guide the LLM repeater to follow e to re-answer q to generate the final answer. The specific content is "The following is a set of dialogues. A asks B questions about the case, and B answers. Please strictly follow the additional question explanation definition and re-answer A's question, outputting the reason first and then the answer."

[0052] Compared with extracting only shallow elements from legal documents, the present invention focuses on deep legal element extraction, focusing on which information in the case directly contributes to the judge's decision. In order to achieve this goal, the present invention converts it into a question-answering task in the form of yes / no, and uses LLM to efficiently output answers, greatly reducing dependence on manual work. When performing question-answering tasks, different from the method of decomposing complex questions to construct a question tree for general text question answering tasks, the present invention constructs a hierarchical question tree for the binary characteristics of yes / no questions and answers, in which the parent question is a prerequisite for the judgment of the child question. When the answer to the parent node question is negative, the negative answer to the child node question can be directly obtained, saving time. When the answer to the parent node question is positive, it can also assist the answer to the child node question, ensuring that the reasoning process is more coherent and accurate. In addition, when using LLM to answer questions, the present invention proposes a new method of re-answering reasoning. After the initial answer is obtained by using LLM for the first time, the initial answer is further optimized through the re-answering link. This method is different from the usual self-correction method, without lengthy steps and prompt words, reducing the difficulty of following weak LLM instructions, and effectively improving the accuracy of LLM answers.

[0053] A hierarchical re-answer reasoning system for deep legal element extraction, used to implement the above-mentioned hierarchical re-answer reasoning method for deep legal element extraction, comprising:

[0054] The hierarchical question tree construction module is configured to: transform the feature extraction task into a question answering (QA) task, so that the question list is reconstructed into a hierarchical question tree according to the subordinate logic;

[0055] The question-answering module is configured to: perform a depth-first traversal of the hierarchical question tree through multiple rounds of question-answering dialogues, answer questions in order according to the responses to the parent nodes, and obtain answers;

[0056] The re-answer reasoning module is configured to: implement the process of answering questions, that is, after obtaining the initial answer, improve the initial answer according to the question explanation and output the final answer.

[0057] For any details not provided in the present invention, please refer to the prior art.

[0058] The beneficial effects of the present invention are:

[0059] The present invention proposes a hierarchical re-answer reasoning method and system for deep legal element extraction, which provides in-depth case information for trial. This method transforms the element extraction task into a QA task to utilize the powerful knowledge and reasoning ability of LLM. Through hierarchical re-answer reasoning, it improves the problems of lack of internal logic of legal elements, lack of legal knowledge and weak instruction tracking ability of LLM. Through deep legal element extraction, not only the efficiency of legal services is improved, but also judges are assisted to make more accurate judgments, providing more precise and powerful support for legal practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0061] Figure 1 A flowchart of the hierarchical re-answer reasoning method for deep legal element extraction of the present invention;

[0062] Figure 2 This is an example of the re-answer reasoning mentioned in the present invention. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings in the implementation of this specification, but are not limited to this. Anything not fully described in the present invention shall be based on the conventional technology in the art.

[0064] Example 1

[0065] A hierarchical re-answer reasoning method for deep legal element extraction includes the following steps;

[0066] (1) Convert the feature extraction task into a question answering (QA) task, so that the question list is reconstructed into a hierarchical question tree based on the subordinate logic;

[0067] (2) Through multiple rounds of question-and-answer dialogues, the hierarchical question tree is traversed in depth first, and questions are answered in sequence according to the responses to the parent nodes. The process of answering questions adopts the repeated answer reasoning method, that is, after obtaining the initial answer, the initial answer is improved according to the explanation of the question and the final answer is output.

[0068] In step (1), the question-answering task is in a yes / no question-answering format, with one element corresponding to one question;

[0069] A hierarchical question tree is established by using the internal logic between the element questions. The hierarchical question tree includes top-level nodes and question nodes. The top-level node (level 0) is the question category, such as "litigation request", "borrower", etc. Each question node represents a separate element question. In the present invention, the parent node is the premise question of the child node, and the child node is a further refinement and expansion of the parent node question. For example, the parent node question "whether to claim repayment of interest" and its child node question "whether to explain the interest calculation base, period and calculation process".

[0070] Different from the traditional method of directly extracting elements from legal documents, the present invention aims to extract elements that are closer to legal clauses and have internal logic such as classification and hierarchical relationships. It is crucial to convert this task into a yes / no QA format, thereby utilizing the knowledge and reasoning ability of LLM to answer each question related to the element.

[0071] Furthermore, the construction method of the hierarchical problem tree is:

[0072] For the t-element problem {q1,q2,…,q t}, divided into s categories according to the subordinate relationship Each category is further divided into several layers. In this case, the element problem {q1,q2,…,q t} is rewritten as where q i represents the i-th problem in the first level (level 1) problem, q ij represents the jth problem in the second level (level 2) under the i-th problem in the first level, q ijk Represents question q ij The kth question in the third level (level 3) below.

[0073] The implementation process in step (2) is:

[0074] Iterate through the question categories;

[0075] For a given question category, traverse the first level (level 1), input the first level questions into the re-answer reasoner, and get the corresponding answers;

[0076] Take the first-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the first-level questions, that is, enter the QA dialogue of the first-level questions into the history dialogue list as a reference for the second-level (level 2) questions. Specifically, when the answer to the first-level question is "no", the answer to the second-level question is directly output as "no". When the answer to the first-level question is "yes", input the history dialogue list and the second-level (level 2) question into the re-answer reasoner to obtain the corresponding answer.

[0077] Take the second-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the second-level questions, that is, enter the QA dialogue of the second-level questions into the history dialogue list as a reference for the third-level (level 3) questions. Specifically, when the answer to the second-level question is "no", the answer to the third-level question is directly output as "no". When the answer to the second-level question is "yes", the history dialogue list and the third-level (level 3) question are input into the re-answer reasoner to obtain the corresponding answer.

[0078] And so on, until all levels of all hierarchical problem trees are traversed.

[0079] In order to make full use of the inherent logic between the element questions, the present invention does not directly answer the above question list, but constructs it into a hierarchical question tree. By integrating the multi-round QA format of LLM, the hierarchical question tree is traversed in depth first, and each question is answered in sequence according to the response to the parent node. The hierarchical QA structure is the core of the present invention to efficiently and accurately complete the element extraction.

[0080] Furthermore, for a given problem category c H , generate the first level (level 1) question q through the re-answer reasoner i The answer is i , expressed as:

[0081]

[0082] in, represents the re-answer reasoner, d represents the legal document, and specific examples can be seen Figure 2 , Represents the text connection function;

[0083] The answer r i After post-processing, when ri = "Yes", the QA dialogue [q i ,r i ]Add to the history conversation list pre1;

[0084] For a given level 1 problem q i , traverse the second level problem, when r i = "No", the second level (level 2) question directly outputs "No", when r i = "Yes", the second level (level 2) question q ij and pre1 are input into the re-answer reasoner to obtain q ij The answer is ij :

[0085]

[0086] Where pre1 = [q i ,r i ];

[0087] The answer r ij After post-processing, when r ij = "Yes", the QA dialogue [q ij ,r ij ]Added to the history conversation list pre2;

[0088] For a given level 2 problem q ij , traverse the second level (level 3) problem, when r ij = "No", the second level (level 3) question directly outputs "No", when r ij = "Yes", the third level (level 3) question q ijk and pre2 are input into the re-answer reasoner to obtain q ijk The answer is ijk :

[0089]

[0090] Where pre2 = [q i ,r i ,q ij ,r ij ];

[0091] When the number of levels in the hierarchical problem tree is greater than three, the same analogy can be applied.

[0092] The above-mentioned re-answer reasoner is a machine that gives answers to input content (legal documents, historical dialogues, question categories, questions), such as Figure 1 The hierarchical problem section.

[0093] The re-answer reasoner includes the LLM reasoner and the LLM re-answerer. The process of generating answers through the re-answer reasoning method is as follows:

[0094] First, a question in a certain layer, or a question in a certain layer and the historical dialogue list of its corresponding parent node are input into the LLM reasoner to obtain the initial answer;

[0095] A more powerful LLM and manual correction are used to generate a question explanation for each factor question. The initial answer and question explanation are input into the LLM repeater, and the answer is re-reasoned and generated as the final answer output.

[0096] In order to further improve the model reasoning ability, the present invention adopts re-answer reasoning to improve the answer as a key step in legal element extraction. Different from the previous self-correction method, the present invention integrates the feedback and improvement stages into a unified program, namely the re-answer reasoning module, thereby reducing the difficulty of following weak LLM instructions and improving the accuracy and efficiency of answers.

[0097] In this embodiment, the LLM reasoner and the LLM repeater both use the Qwen2.5-14B model, and the LLM with stronger performance uses the Qwen-turbo model. The present invention uses the LLM with stronger performance to generate question explanations to assist the LLM with weaker performance to complete tasks, thereby reducing the difficulty of following the weak LLM instructions and effectively improving the accuracy of the LLM answers.

[0098] Furthermore, for a given question q, the LLM reasoner is used to obtain the initial answer R init , expressed as:

[0099]

[0100] in, represents the LLM reasoner, p init represents a zero-sample prompt word, which is used to guide the LLM reasoner to generate an initial answer. The specific content is "You are a senior legal practitioner. Answer the user's question based on the case content." || represents a connection relationship. d represents a legal document. pre3 represents the QA dialogue of the parent node question. c represents the question category. Represents the text connection function;

[0101] [q,R init ] Add to the historical conversation pre′;

[0102] Input pre′ and q into the LLM repeater to get the final answer R final ,express:

[0103]

[0104] in, represents the LLM repeater, e represents the corresponding question explanation, pre′=[q,R init ], p final Represents a zero-sample prompt word, which is used to guide the LLM repeater to follow e to re-answer q to generate the final answer. The specific content is "The following is a set of dialogues. A asks B questions about the case, and B answers. Please strictly follow the additional question explanation definition and re-answer A's question, outputting the reason first and then the answer."

[0105] Compared with extracting only shallow elements from legal documents, the present invention focuses on deep legal element extraction, focusing on which information in the case directly contributes to the judge's decision. In order to achieve this goal, the present invention converts it into a question-answering task in the form of yes / no, and uses LLM to efficiently output answers, greatly reducing dependence on manual work. When performing question-answering tasks, different from the method of decomposing complex questions to construct a question tree for general text question answering tasks, the present invention constructs a hierarchical question tree for the binary characteristics of yes / no questions and answers, in which the parent question is a prerequisite for the judgment of the child question. When the answer to the parent node question is negative, the negative answer to the child node question can be directly obtained, saving time. When the answer to the parent node question is positive, it can also assist the answer to the child node question, ensuring that the reasoning process is more coherent and accurate. In addition, when using LLM to answer questions, the present invention proposes a new method of re-answering reasoning. After the initial answer is obtained by using LLM for the first time, the initial answer is further optimized through the re-answering link. This method is different from the usual self-correction method, without lengthy steps and prompt words, reducing the difficulty of following weak LLM instructions, and effectively improving the accuracy of LLM answers.

[0106] Example 2

[0107] A hierarchical re-answer reasoning method for deep legal element extraction, such as Figure 1 As shown, the specific implementation process is:

[0108] (1) Convert the feature extraction task into a question answering (QA) task, so that the question list is reconstructed into a hierarchical question tree based on the subordinate logic;

[0109] Convert the feature extraction task into a yes / no question answering task, where one feature corresponds to one question;

[0110] Using the internal logic between the element problems, a hierarchical problem tree is established. The top node is the problem category, and one, two or three layers of problems can be set under it, such as Figure 1 shown.

[0111] (2) Through multiple rounds of question-answering dialogues, the hierarchical question tree is traversed in depth first, and questions are answered in order according to the responses to the parent nodes. The process of answering questions adopts the re-answer reasoning method, that is, after obtaining the initial answer, the initial answer is improved according to the explanation of the question and the final answer is output;

[0112] For a given legal document, traverse the question categories, traverse the level 1 under each question category, input the legal document, question category, and level 1 question into the LLM reasoner to get the initial answer, then input the legal document, initial question-answer pair, and question explanation into the LLM repeater to get the final answer, which is output as the answer to the level 1 question;

[0113] Take the level 1 questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, then end. Figure 1 Level 1 in: Whether the borrower attends the first trial in court, no child nodes, end; if there are corresponding child nodes, such as Figure 1 Level 1: Is the borrower married? The answer to level 1 is post-processed, that is, the QA dialogue of the first level question is entered into the historical dialogue list as a reference for the level 2 question.

[0114] Traverse level 2 under the above level 1 question. When the answer to the level 1 question is "no", the answer to the level 2 question is directly output as "no". When the answer to the level 1 question is "yes", the legal document, level 1 historical dialogue, question category, and level 2 question are input into the LLM reasoner and output as the initial answer to the level 2 question. Then, the legal document, initial answer, and question explanation are input into the LLM repeater to obtain the final answer, which is output as the answer to the level 2 question.

[0115] Similarly, take the level 2 questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are child nodes, post-process the answers to the level 2 questions and add the QA with the answer "yes" to the history conversation list as a reference for level 3.

[0116] Traverse level 3 under the above level 2 question. When the answer to the level 2 question is "no", the level 3 question directly outputs "no". When the answer to the level 2 question is "yes", the legal document, the historical dialogue of level 1+level 2, the question category, and the level 3 question are input into the LLM reasoner to obtain the initial answer. Then the legal document, the initial answer, and the question explanation are input into the LLM repeater to obtain the final answer, which is output as the answer to the level 3 question.

[0117] Example 3

[0118] A hierarchical re-answer reasoning system for deep legal element extraction, used to implement the above-mentioned hierarchical re-answer reasoning method for deep legal element extraction, comprising:

[0119] The hierarchical question tree construction module is configured to: transform the feature extraction task into a question answering (QA) task, so that the question list is reconstructed into a hierarchical question tree according to the subordinate logic;

[0120] The question-answering module is configured to: perform a depth-first traversal of the hierarchical question tree through multiple rounds of question-answering dialogues, answer questions in order according to the responses to the parent nodes, and obtain answers;

[0121] The re-answer reasoning module is configured to: implement the process of answering questions, that is, after obtaining the initial answer, improve the initial answer according to the question explanation and output the final answer.

[0122] For any details not provided in the present invention, please refer to the prior art.

[0123] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hierarchical re-answer reasoning method for deep legal element extraction, characterized in that: The method comprises the following steps: (1) Convert the feature extraction task into a question-answering task, so that the question list is reconstructed into a hierarchical question tree based on the subordinate logic; (2) Through multiple rounds of question-and-answer dialogues, the hierarchical question tree is traversed in depth first, and questions are answered in sequence according to the responses to the parent nodes. The process of answering questions adopts the repeated answer reasoning method, that is, after obtaining the initial answer, the initial answer is improved according to the explanation of the question and the final answer is output.

2. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 1 is characterized in that: In step (1), the question-answering task is in a yes / no question-answering format, with one element corresponding to one question; A hierarchical problem tree is established by using the internal logic between element problems. The hierarchical problem tree includes top-level nodes and problem nodes. The top-level nodes are problem categories, and each problem node represents a separate element problem.

3. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 2 is characterized in that: The construction method of the hierarchical problem tree is: For the t-element problem {q1,q2,…,q t }, divided into s categories according to the subordinate relationship Each category is further divided into several layers. In this case, the element problem {q1,q2,…,q t } is rewritten as where q i represents the i-th problem in the first level, q ij represents the jth problem in the second layer under the i-th problem in the first layer, q ijk Represents question q ij The kth question in the third layer below.

4. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 3 is characterized in that: The implementation process in step (2) is: Iterate through the question categories; For a given question category, traverse the first layer, input the first layer questions into the re-answer reasoner, and get the corresponding answers; Take the first-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the first-level questions, that is, enter the QA dialogue of the first-level questions into the historical dialogue list as a reference for the second-level questions. Specifically, when the answer to the first-level questions is "no", the answer to the second-level questions is directly output as "no". When the answer to the first-level questions is "yes", the historical dialogue list and the second-level questions are input into the re-answer reasoner to obtain the corresponding answer. Take the second-level questions as parent nodes and determine whether there are corresponding child nodes. If there are no corresponding child nodes, end the process. If there are corresponding child nodes, post-process the answers to the second-level questions, that is, enter the QA dialogue of the second-level questions into the historical dialogue list as a reference for the third-level questions. Specifically, when the answer to the second-level question is "no", the answer to the third-level question is directly output as "no". When the answer to the second-level question is "yes", input the historical dialogue list and the third-level question into the re-answer reasoner to obtain the corresponding answer. And so on, until all levels of all hierarchical problem trees are traversed.

5. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 4 is characterized in that: For a given problem category c H , the first-level question q is generated by the re-answer reasoner i The answer is i , expressed as: in, represents the re-answer reasoner, d represents the legal document, Represents the text connection function; The answer r i After post-processing, when r i = "Yes", the QA dialogue [q i ,r i ]Add to the history conversation list pre1; For a given first-level problem q i , traverse the second level problem, when r i = "No", the second level question directly outputs "No", when r i = "Yes", the second level question q ij and pre1 are input into the re-answer reasoner to obtain q ij The answer is ij : Where pre1 = [q i ,r i ]; The answer r ij After post-processing, when r ij = "Yes", the QA dialogue [q ij ,r ij ] Added to the history conversation list pre2; For a given second-level problem q ij , traverse the second level problem, when r ij = "No", the second level question directly outputs "No", when r ij = "Yes", the third level question q ijk and pre2 are input into the re-answer reasoner to obtain q ijk The answer is ijk : Where pre2 = [q i ,r i ,q ij ,r ij ]; When the number of levels in the hierarchical problem tree is greater than three, the same applies.

6. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 5 is characterized in that: The re-answer reasoner includes the LLM reasoner and the LLM re-answerer. The process of generating answers through the re-answer reasoning method is as follows: First, a question in a certain layer, or a question in a certain layer and the historical dialogue list of its corresponding parent node are input into the LLM reasoner to obtain the initial answer; A more powerful LLM and manual correction are used to generate a question explanation for each factor question. The initial answer and question explanation are input into the LLM repeater, and the answer is re-reasoned and generated as the final answer output.

7. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 6 is characterized in that: The LLM reasoner and LLM repeater both use the Qwen2.5-14B model; The LLM with stronger performance adopts the Qwen-turbo model.

8. The hierarchical re-answer reasoning method for deep legal element extraction according to claim 7 is characterized in that: For a given question q, use the LLM reasoner to get the initial answer R init , expressed as: in, represents the LLM reasoner, p init represents a zero-sample prompt word, which is used to guide the LLM reasoner to generate an initial answer, || represents a connection relationship, d represents a legal document, pre3 represents the QA dialogue of the parent node question, c represents the question category, Represents the text connection function; [q,R init ] Add to the historical conversation pre′; Input pre′ and q into the LLM repeater to get the final answer R final ,express: in, represents the LLM repeater, e represents the corresponding question explanation, pre′=[q,R init ], p final Represents a zero-shot prompt word used to guide the LLM repeater to follow e to re-answer q to generate the final answer.

9. A hierarchical re-answer reasoning system for deep legal element extraction, used to implement the hierarchical re-answer reasoning method for deep legal element extraction described in any one of claims 1 to 8, characterized in that: include: The hierarchical question tree construction module is configured to: transform the feature extraction task into a question answering task, so that the question list is reconstructed into a hierarchical question tree according to the subordinate logic; The question-answering module is configured to: perform a depth-first traversal of the hierarchical question tree through multiple rounds of question-answering dialogues, answer questions in order according to the responses to the parent nodes, and obtain answers; The re-answer reasoning module is configured to: implement the process of answering questions, that is, after obtaining the initial answer, improve the initial answer according to the question explanation and output the final answer.

Images