Slow-thinking-mode legal question answering method and device based on judge visual angle

Abstract

The invention discloses a slow-thinking mode legal question answering method and device based on a judge visual angle, and belongs to the technical field of artificial intelligence. The method comprises the following steps: constructing a multi-dimensional legal vector knowledge base; understanding the user question, and generating a plurality of retrieval query statements; retrieving the retrieval query statement in the multi-dimensional legal vector knowledge base to obtain related legal knowledge of the user question; screening the related legal knowledge to construct a candidate knowledge base, and evaluating whether the legal knowledge in the candidate knowledge base is enough to fully answer the user question; and under the condition that the legal knowledge in the candidate knowledge base is sufficient to fully answer the user question, generating an answer to the user question based on the legal knowledge in the candidate knowledge base. According to the method, the accuracy, reliability, robustness and comprehensiveness of the legal question-answering system can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a slow-thinking mode legal question-answering method and device based on a judge's perspective. Background Art

[0002] With the advancement of model algorithms, the development of scalable basic model architectures, and the abundance of high-quality data sets, large language model technology has made remarkable achievements. However, large language model systems still face challenges such as lagging knowledge updates, insufficient coverage of long-tail knowledge, data leakage risks, and high training and reasoning costs. Especially in specific fields, due to the lack of representative knowledge, they still face problems such as hallucinations. To solve these problems, retrieval-enhanced generation technology came into being. By introducing information retrieval mechanisms, relevant objects are retrieved from existing data warehouses to enhance the results generated by large language models, thereby improving accuracy and robustness.

[0003] However, the current legal question-answering system based on RAG technology is mostly limited to a single search, and the retrieved knowledge often cannot fully or partially cover all knowledge points, and there may be internal contradictions between the knowledge, resulting in a significant decrease in the performance of answer accuracy. To solve this problem, the present invention proposes a slow thinking mode legal question-answering method based on the judge's perspective. The method first starts from the judge's perspective, understands the questions input by the user, generates a search query statement that is easy to match the current knowledge base, and then relies on a lightweight large model to use the constructed multi-dimensional knowledge screening and reflector in the legal field to achieve accurate screening of highly relevant legal knowledge, and uses the reflector to evaluate whether this knowledge is sufficient to fully answer the user's questions. If the evaluation result is sufficient, the user's questions are answered using the large model based on the retrieved knowledge; if the evaluation result is insufficient, a new search statement is automatically generated, and a secondary search is performed, and so on, until the retrieved knowledge can answer the user's question or reaches the set reflection number limit. Using this slow thinking mode based on the judge's perspective, the accuracy of legal questions and answers can be significantly improved. Summary of the invention

[0004] In view of the problems that the existing RAG technology in legal question and answering has incomplete retrieval content and internal contradictions and insufficiency in the retrieved knowledge, the present invention aims to provide a slow thinking mode legal question and answering method and device based on the judge's perspective, so as to improve the accuracy, reliability, robustness and comprehensiveness of the legal question and answer system.

[0005] To achieve the above objectives, the technical solution of the present invention includes the following contents.

[0006] A slow-thinking legal question-answering method based on a judge's perspective, the method comprising:

[0007] Construct a multi-dimensional legal vector knowledge base;

[0008] Understand user questions and generate several search query statements;

[0009] Retrieving the search query statement in the multi-dimensional legal vector knowledge base to obtain legal knowledge related to the user's question;

[0010] Screening the relevant legal knowledge to construct a candidate knowledge base, and evaluating whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question;

[0011] In the case that the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question, an answer to the user's question is generated based on the legal knowledge in the candidate knowledge base.

[0012] Furthermore, the construction of a multi-dimensional legal vector knowledge base includes:

[0013] Collecting original data and cleaning the original data based on timeliness, accuracy, consistency and effectiveness; wherein the original data includes: legal questions and answers, laws and regulations, and case data;

[0014] The cleaned raw data is deeply analyzed and refined to form a legal question-answering data set A = {(t i ,a i ,g i ,s i )|i=1,…,n A}、Law and regulation set B={(l i ,d i ,c i ,e i ,k i )|i=1,…,n B} and case data set C = {(y i ,f i ,b i ,p i ,z i )|i=1,…,n C}; where t i represents the question content corresponding to the i-th legal question-answering data, a i represents the answer corresponding to the i-th legal question and answer data, g i represents the question summary corresponding to the i-th legal question-answering data, s i represents the data source corresponding to the i-th legal question-and-answer data, n A represents the data size of the legal question-answering dataset A, l i Indicates the name of the law or regulation corresponding to the i-th article, d i represents the directory corresponding to the i-th legal article, c irepresents the content of the legal article corresponding to the i-th legal article, e i represents the legal interpretation corresponding to the i-th legal article, k i represents the key element corresponding to the i-th legal article, n B represents the data size of the legal and regulatory set B, y i represents the cause of the i-th case data, f i represents the review and verification of the i-th case data, b i The Court believes that p i The judgment of the i-th case data is as follows, z i represents the case summary of the i-th case data, n C Indicates the data size of the case data set C;

[0015] Generate question content i , key elements k i and case summary i The sparse vectors and dense vectors are obtained and stored in the legal question and answer data set A, the legal and regulatory data set B and the case data set C respectively, and then imported into the corresponding legal question and answer vector knowledge base, the legal and regulatory vector knowledge base and the case data vector knowledge base.

[0016] Furthermore, the user's question is understood and several search query statements are generated, including:

[0017] When searching the legal question-answering vector knowledge base, if it is a single-round dialogue, the user's question is used as the search query statement of the legal question-answering vector knowledge base; if it is a multi-round dialogue, the user's question is rewritten with pronouns based on the historical dialogue and using the reference resolution strategy, and the rewritten question is used as the search query statement of the legal question-answering vector knowledge base;

[0018] When searching the legal and regulatory vector knowledge base, we use question-based legal key element prompt instructions and large language models to clarify the legal field to which the question belongs, extract the core legal facts, legal scenarios, legal subjects and other legal key elements in the user's question, and splice these legal key elements as the search statement for the legal and regulatory knowledge base;

[0019] When searching the case data vector knowledge base, a case summary prompt instruction is generated based on the prompt corpus of the user question and the case summary, and after using the large language model to generate the case summary corresponding to the user question, the case summary corresponding to the user question is used as the retrieval query statement of the case data vector knowledge base.

[0020] Furthermore, the relevant legal knowledge is screened, including:

[0021] Define the relevance assessment criteria based on the legal analysis of similar cases, detailed explanation of laws and regulations, and determination of the identity of the problem purpose;

[0022] Based on the relevance evaluation standard, a relevance evaluation prompt instruction is constructed; wherein the relevance evaluation prompt instruction requires the output of a highly relevant knowledge list, and the information in the relevance evaluation prompt instruction includes: similar case knowledge, legal and regulatory knowledge, legal question and answer knowledge, user questions, and relevance evaluation prompt corpus;

[0023] For user question samples and knowledge blocks recalled based on the user question samples, a highly relevant knowledge list is output with the help of a large language model and relevance evaluation prompt instructions;

[0024] Training a knowledge screening model based on the highly relevant knowledge list;

[0025] The trained knowledge screening model is used to screen the relevant legal knowledge.

[0026] Furthermore, the legal knowledge in the candidate knowledge base is evaluated to determine whether it is sufficient to fully answer the user's question, including:

[0027] Define the evaluation criteria for knowledge reflection based on the legal analysis of similar cases, detailed explanation of laws and regulations, and determination of the identity of the purpose of the problem;

[0028] Based on the knowledge reflection evaluation standard, the legal knowledge in the candidate knowledge base is embedded in the reflection evaluation prompt instruction, and based on the big model, it is evaluated whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question; wherein, the information in the reflection evaluation prompt instruction includes: similar case knowledge, legal and regulatory knowledge, legal question and answer knowledge, user questions and reflection evaluation prompt corpus.

[0029] Furthermore, generating an answer to the user's question based on the legal knowledge in the candidate knowledge base includes:

[0030] Based on the reciprocal fusion sorting algorithm, the legal knowledge in the candidate knowledge base is deduplicated and re-sorted to generate the final knowledge list;

[0031] User questions and the final knowledge list are embedded into the instructions for generating answer prompts, and a large model is used to generate natural language answers to user questions.

[0032] Furthermore, the legal knowledge in the candidate knowledge base is deduplicated and reordered according to the inverse fusion sorting algorithm to generate a final knowledge list, including:

[0033] De-duplicate the legal knowledge in the candidate knowledge base;

[0034] For different types of knowledge, select the knowledge content to be input into the big model;

[0035] The inverse fusion sorting method is used to calculate the similarity between the knowledge content to be input into the big model and the user's question, and the deduplicated legal knowledge is sorted based on the similarity to obtain a final knowledge list; wherein the legal knowledge in the final knowledge list is integrated according to type.

[0036] Furthermore, the method further comprises:

[0037] In the case that the legal knowledge in the candidate knowledge base is not sufficient to fully answer the user's question, a new search query statement is generated based on the user's question and the deficiencies of the legal knowledge in the candidate knowledge base;

[0038] Based on the new search query statement, the search of the search query statement in the multi-dimensional legal vector knowledge base is re-executed to obtain relevant legal knowledge of the user question.

[0039] A slow-thinking mode legal question-and-answer device based on a judge's perspective, the device comprising:

[0040] Knowledge base construction module, used to construct a multi-dimensional legal vector knowledge base;

[0041] The statement generation module is used to understand the user's questions and generate several search query statements;

[0042] A knowledge retrieval model, used to retrieve the retrieval query statement in the multi-dimensional legal vector knowledge base to obtain relevant legal knowledge of the user's question;

[0043] A knowledge evaluation module, used for screening the relevant legal knowledge to construct a candidate knowledge base, and evaluating whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question;

[0044] The answer generation module is used to generate an answer to the user's question based on the legal knowledge in the candidate knowledge base when the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question.

[0045] An electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the slow thinking mode legal question and answer method based on the judge's perspective as described in any one of the above items is implemented.

[0046] Compared with the prior art, the present invention has at least the following beneficial effects.

[0047] In the problem understanding stage, the present invention generates easy-to-search query statements for different knowledge bases from the perspective of judges for the questions input by users, thereby improving the recall accuracy rate; the method also relies on multiple dimensions such as the legal analysis of similar cases, the detailed explanation of laws and regulations, and the determination of the identity of the purpose of the question, to construct a multi-dimensional knowledge screening and reflector in the legal field, and screens the recalled knowledge based on the knowledge screener to solve the problems of internal contradictions and irrelevant knowledge between knowledge; based on the reflector, it is judged whether the knowledge fully answers the user's question and generates search statements adapted to different dimensions for iterative query, so as to solve the problems of incomplete search content caused by the single search of the current legal question-and-answer system based on RAG technology, thereby constructing a legal question-and-answer system with a slow thinking mode. It can be seen that the introduction of the slow thinking mode based on the perspective of judges into the legal question-and-answer system based on RAG can realize multi-angle knowledge retrieval, screening, and reflection, and realize the extraction of efficient and sufficient legal knowledge content, which significantly improves the accuracy of legal questions and answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flowchart of the slow thinking model legal question-and-answer method based on the judge's perspective.

[0049] Figure 2 Flowchart of building modules for legal knowledge vector library. DETAILED DESCRIPTION

[0050] To further illustrate the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] The slow thinking mode legal question-answering method based on the judge's perspective of the present invention is as follows: Figure 1 As shown, the steps include the following steps S1 to S8.

[0052] Step S1: Construct a multi-dimensional legal vector knowledge base.

[0053] The present invention constructs a multi-dimensional legal vector knowledge base through the steps of collection, cleaning, organization, knowledge processing, vectorization, and updating. In this embodiment, the legal vector knowledge base includes three vector knowledge bases, namely, legal questions and answers, laws and regulations, and case data. Figure 2 The specific steps include the following:

[0054] Step S11: Knowledge collection and cleaning.

[0055] First, we collect original data such as legal questions and answers, laws and regulations, and case data from various authoritative legal databases, official websites, and other channels (such as the Supreme People's Court, China Legal Service Network, and other official websites). Case data includes practical cases and judicial documents. Then, we strictly screen and clean these materials from the perspectives of timeliness, accuracy, consistency, and effectiveness, remove duplicate, erroneous, and outdated information, and ensure the accuracy and timeliness of the knowledge base.

[0056] Step S12: Knowledge processing.

[0057] The present invention conducts in-depth analysis and refinement of the cleansed knowledge to form knowledge units with legal expertise.

[0058] In terms of legal question-answering data processing, for millions of legal question-answering datasets A = {(t i ,a i ,s i )|i=1,…,n A}, where t i 、a i and i Respectively represent the question content, answer and data source corresponding to the i-th data, n A The size of the legal question-answering dataset A is represented by the following two legal information processing strategies: (1) Extraction of legal key points and conclusions: Based on the large language model technology, according to the preset answer summary prompt instructions, see Table 1, the answers in the legal question-answering pair are deeply analyzed to accurately extract the legal key points and concluding statements. This process aims to eliminate knowledge redundancy and ensure that the extracted information is direct, accurate and focused on the core legal points. (2) Accurate extraction and matching of laws and regulations: Using efficient pattern matching technology, such as the extraction rules in Table 2 and the fuzzy query statements in Table 3, the names, specific items and content summaries of the laws and regulations mentioned in the answer text are identified. Subsequently, the fuzzy matching algorithm is used to retrieve and extract the complete legal and regulatory content that matches the identification results from the authoritative legal and regulatory database. This step ensures the accuracy and completeness of the extracted legal and regulatory names, items and content information. The content processed by the above two strategies is integrated into a new field, called answer summary g, thereby improving the practical value and accuracy of the legal question-answering pair. The final constructed legal question-answering dataset is represented as: A = {(t i ,a i ,g i ,s i )|i=1,…,n A The information contained in the answer summary prompt instruction includes: the answer content of the legal question and answer, and the prompt corpus of the answer summary.

[0059] Table 1: Answer summary prompt instructions

[0060]

[0061] Table 2 Rules for extracting legal provisions

[0062]

[0063] Table 3 Fuzzy query statements

[0064]

[0065] In terms of legal and regulatory processing, the following processing process is mainly adopted for the text content of laws and regulations. First, according to the established rules, the legal provisions are disassembled one by one to construct a legal and regulatory set B = {(l i ,d i ,c i )|i=1,…,n B}, where l i d i and c i Respectively represent the name of the law or regulation corresponding to the i-th law, the directory, and the content of the law, n B represents the total size of the collection, which contains more than 1 million currently effective laws and regulations. Then, based on the collected interpretations of regulations and other contents, the relationship matching technology is used to achieve the association matching between the interpretations of regulations and legal provisions. A new variable e is added to the legal provisions collection to represent the interpretations of legal provisions: B = {(l i ,d i ,c i ,e i )|i=1,…,n B Finally, by using the large language model technology, by setting the key elements of laws and regulations prompt instructions, see Table 4, the key elements such as legal scenarios and core components involved in the interpretation of regulations and legal provisions are accurately extracted and recorded as k. The final set of laws and regulations is B = {(l i ,d i ,c i ,e i ,k i )|i=1,…,n B}.

[0066] Table 4: Key element prompt instructions

[0067]

[0068] In terms of case data processing, the following processing is mainly adopted for the document contents of more than 2 million civil, criminal and administrative cases collected: First, the existing document segmentation technology is used to accurately extract the main components of the case, including but not limited to the case name, trial and investigation section, this court's opinion section, and the judgment section; then, the large language model is used to summarize and summarize the factual details in the trial and investigation section by adopting the case summary prompt instructions as described in Table 5 to extract the core plot of the case; finally, based on the above extracted and summarized information, a structured case data set is constructed, C = {(y i ,f i ,b i ,p i ,z i )|i=1,…,n C}, where y i ,f i ,b i ,p i ,z i They represent the cause of action, findings during trial, opinion of this court, judgment as follows, and case summary in the ith case, n C Indicates the total size of the collection.

[0069] Table 5: Case summary prompt instructions

[0070]

[0071] Step S13: Knowledge vectorization.

[0072] In order to efficiently retrieve knowledge information that matches the user's specific needs from the knowledge base, for the multiple knowledge sets constructed in step S12, the question content contained in set A, the key elements generated in set B, and the case summary summarized in set C are converted into two forms of vector representations: sparse vectors and dense vectors through the vector embedding model, and stored in sets A, B, and C respectively. These vectors can capture the semantic features and contextual relationships of legal knowledge, providing a basis for subsequent information retrieval and autonomous reflection. Optionally, industry-leading embedding models such as bge-m3 and bce are used as the core algorithms for vectorization processing.

[0073] Step S14: Import the vector library.

[0074] The text content and vectors in the legal question and answer collection, the legal regulations collection, and the case data collection are stored in the vector library to realize the construction of the plug-in legal vector knowledge base. Optionally, the vector library can be milvus, elasticsearch, etc.

[0075] Step S2: Relying on the lightweight large model, a multi-dimensional knowledge screening and reflector in the legal field is constructed to achieve accurate screening of highly relevant legal knowledge and effective iterative generation of new search query statements.

[0076] (1) Synthesize training data based on large language models.

[0077] 1. Define the relevance assessment criteria based on the three dimensions of legal analysis of similar cases, detailed interpretation of laws and regulations, and determination of the identity of the purpose of the problem. In the dimension of legal analysis of similar cases, the main focus is on whether the case facts, dispute focus, legal subjects are consistent, and whether the reasoning and legal basis of the case are useful; in the dimension of detailed interpretation of laws and regulations, the validity of the clause is analyzed based on the interpretation of the legal provisions and the content of the clauses; in the dimension of determining the identity of the purpose of the problem, the main focus is on whether the legal relationship, legal subject, legal behavior, legal consequences, and purpose of the problem are consistent in the problem statement.

[0078] 2. Based on the relevance evaluation criteria, construct relevance evaluation prompt instructions, as shown in Table 6:

[0079] Table 6: Relevance evaluation prompt instructions

[0080]

[0081]

[0082] 3. The user's questions, recalled cases, laws and regulations, and question-and-answer knowledge are numbered and substituted into the relevance assessment prompt instruction. With the help of the large language model, a list of highly relevant knowledge is output. Highly relevant knowledge is marked as "relevant"; otherwise, it is marked as "irrelevant", thereby generating training data.

[0083] (2) Knowledge screening model training. For the dataset generated by the above steps, add the knowledge category at the beginning of each set of data, use the bge-m3 embedding model to encode the user's questions and knowledge respectively and concatenate them as input, input them into the LSTM model, add a fully connected layer to the LSTM model, and map the hidden state to two categories (i.e., relevant and irrelevant). Then, use the Sigmoid function to map the output to between 0 and 1, and set a threshold of 0.5 to determine which category the output belongs to.

[0084] (3) Construction of a multi-dimensional knowledge reflector based on a lightweight large language model.

[0085] 1. Define the knowledge reflection assessment standards based on the three dimensions of legal analysis of similar cases, detailed explanation of laws and regulations, and determination of the identity of the purpose of the question. In the dimension of legal analysis of similar cases, the focus is on whether the reasoning and legal basis of all the cases provided are sufficient to solve the user's problem; in the dimension of detailed explanation of laws and regulations, based on all the interpretations of the laws and regulations and the content of the clauses, analyze whether the clauses can sufficiently support answering the user's questions; in the dimension of determining the identity of the purpose of the question, the focus is on whether the content of the answer summary is sufficient to answer the user's question.

[0086] 2. Based on the knowledge reflection evaluation criteria, construct reflection evaluation prompt instructions, as shown in Table 7. Clearly specify the output requirements in the reflection evaluation prompt instructions: if the knowledge is sufficient, return "terminate search", otherwise, based on the reflection results, generate a re-search query statement for the corresponding dimension.

[0087] Table 7: Reflective Assessment Prompt Instructions

[0088]

[0089] 3. Based on the lightweight large language model, a multidimensional knowledge reflector is constructed using reflective evaluation prompts.

[0090] Step S3: For the questions input by the user, from a legal perspective, a multi-dimensional and multi-granular understanding is performed on the questions input by the user, and search query statements that are easy to match legal questions and answers, laws and regulations, and case knowledge bases are generated.

[0091] The generation of search query statements includes the following three aspects:

[0092] (1) Matching the legal question-answering knowledge base. For a single-round dialogue, the user's original question q is directly used as the search query statement; for a multi-round dialogue, based on the historical dialogue, the reference resolution strategy is adopted to rewrite the user's question q with pronouns to generate the rewritten question q ′ , as a retrieval query statement for legal issues.

[0093] (2) Matching the direction of laws and regulations. Using the question-type legal key element prompt instructions shown in Table 8, the large language model is used to clarify the legal field to which the question belongs, extract the core legal facts, legal scenarios, legal subjects and other legal key elements in the user's question, and splice these legal key elements as the search statement of the legal knowledge base.

[0094] Table 8: Question-based instructions for key legal elements

[0095]

[0096] (3) Matching the case knowledge base direction. Using the case summary prompt instruction and the large language model, a case description consistent with the case summary template is generated as a search query statement for the case knowledge base.

[0097] Finally, according to the knowledge characteristics of different knowledge bases, their own unique retrieval query statements are generated, and finally a comprehensive query statement is constructed, Q = {q1,…,q3}.

[0098] Step S4: Based on the search query statement, recall the TopK knowledge from each vector knowledge base to build a candidate knowledge base.

[0099] (1) Input the search query Q into the vector representation model to generate sparse and dense vectors respectively, represented as in They represent the sparse vector and dense vector corresponding to the i-th query statement respectively.

[0100] (2) Information retrieval system based on V Q , a hybrid retrieval strategy is adopted to recall the Topk corresponding similar questions and answers, similar cases, relevant laws, relevant legal points and other knowledge combinations from each legal knowledge base, denoted as Z = {Z1, ..., Z3}, where Z i ={z i,1 ,…,z i,k}, Z i Represents the knowledge list obtained from the i-th knowledge base.

[0101] Step S5: Evaluate the candidate knowledge base.

[0102] Based on the multi-dimensional knowledge filter, legal knowledge that is highly relevant to the query is screened out and stored in the candidate knowledge base P (the knowledge in the P base will serve as the basis for generating the final legal opinion or answer). Subsequently, the reflector will use the reflection mechanism to evaluate whether this knowledge is sufficient to fully answer the user's question based on the screened knowledge and the user's specific questions. If the evaluation result is insufficient, a new search statement will be automatically generated, Q new = {q 1,new ,…,q 3,new}; If the evaluation result is sufficient, the search process is terminated and proceeds to the subsequent step S7.

[0103] Step S6: Search iteration and terminate.

[0104] According to the search statement generated in step S5, Q new = {q 1,new ,…,q 3,new}, repeat steps S4 and S5 until the preset reflection number limit 3 is reached.

[0105] S7: Candidate knowledge verification.

[0106] Based on the Reciprocal Ranking Fusion (RRF) technology, the knowledge contained in the candidate knowledge base P is deduplicated and reordered. The specific steps are as follows:

[0107] (1) De-duplicate the knowledge in the candidate knowledge base P to avoid inputting duplicate knowledge into the large model;

[0108] (2) For different types of knowledge, select the knowledge content to be input into the big model. For example, for a list of legal question-and-answer knowledge, select the question description and answer summary as knowledge: “Question: t i \nLegal points: g i "; For the list of legal knowledge, select the legal content as knowledge: "c i ”.

[0109] (3) Using the Reciprocal Ranking Fusion (RRF) method, calculate the similarity between each piece of knowledge selected in step S72 and the user's question, sort the selected knowledge, and finally generate an optimized knowledge list: Z ′ ={z ′ 1,…,z ′ n}.

[0110] (4) Optimized knowledge list Z ′ Integrate by category to form structured knowledge content to facilitate subsequent answer generation.

[0111] Step S8: Generate a response.

[0112] Based on the finalized knowledge list K ′ , and use the prompt instructions and heavyweight model for generating answers as shown in Table 9 to generate answers to user questions.

[0113] Table 9: Prompt instructions for generating answers

[0114]

[0115] It should be noted that the above is only the embodiment and the accompanying drawings of the present invention, and its purpose is to better understand the content of the present invention, and is not used to limit the present invention. For those skilled in the art, the present invention can be implemented in various ways. Various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the best embodiment, and the scope of the present invention to be protected shall be subject to the scope defined in the claims.

Claims

1. A slow thinking mode legal question-answering method based on the judge's perspective, characterized by: The method comprises: Construct a multi-dimensional legal vector knowledge base; Understand user questions and generate several search query statements; Retrieving the search query statement in the multi-dimensional legal vector knowledge base to obtain legal knowledge related to the user's question; Screening the relevant legal knowledge to construct a candidate knowledge base, and evaluating whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question; In the case that the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question, an answer to the user's question is generated based on the legal knowledge in the candidate knowledge base.

2. The method according to claim 1, characterized in that The construction of a multi-dimensional legal vector knowledge base includes: Collecting original data and cleaning the original data based on timeliness, accuracy, consistency and effectiveness; wherein the original data includes: legal questions and answers, laws and regulations, and case data; The cleaned raw data is deeply analyzed and refined to form a legal question-answering data set A = {(t i ,a i ,g i ,s i )|i=1,…,n A }、Law and regulation set B={(l i ,d i ,c i ,e i ,k i )|i=1,…,n B } and case data set C = {(y i ,f i ,b i ,p i ,z i )|i=1,…,n C }; where t i represents the question content corresponding to the i-th legal question-answering data, a i represents the answer corresponding to the i-th legal question and answer data, g i represents the question summary corresponding to the i-th legal question-answering data, s i represents the data source corresponding to the i-th legal question-and-answer data, n A represents the data size of the legal question-answering dataset A, l i Indicates the name of the law or regulation corresponding to the i-th article, d i represents the directory corresponding to the i-th legal article, c i represents the content of the legal article corresponding to the i-th legal article, e i represents the legal interpretation corresponding to the i-th legal article, k i represents the key element corresponding to the i-th legal article, n B represents the data size of the legal and regulatory set B, y i Indicates the cause of the i-th case data, f i represents the review and verification of the i-th case data, b i The Court believes that p i The judgment of the i-th case data is as follows, z i represents the case summary of the i-th case data, n C Indicates the data size of the case data set C; Generate question content i , key elements k i and case summary i The sparse vectors and dense vectors are obtained and stored in the legal question and answer data set A, the legal and regulatory set B, and the case data set C respectively, and then imported into the corresponding legal question and answer vector knowledge base, the legal and regulatory vector knowledge base, and the case data vector knowledge base.

3. The method according to claim 2, characterized in that The user's question is understood and several search query statements are generated, including: When searching the legal question-answering vector knowledge base, if it is a single-round dialogue, the user's question is used as the search query statement of the legal question-answering vector knowledge base; if it is a multi-round dialogue, the user's question is rewritten with pronouns based on the historical dialogue and using the reference resolution strategy, and the rewritten question is used as the search query statement of the legal question-answering vector knowledge base; When searching the legal and regulatory vector knowledge base, we use question-based legal key element prompt instructions and large language models to clarify the legal field to which the question belongs, extract the core legal facts, legal scenarios, legal subjects and other legal key elements in the user's question, and splice these legal key elements as the search statement for the legal and regulatory knowledge base; When searching the case data vector knowledge base, a case summary prompt instruction is generated based on the prompt corpus of the user question and the case summary, and after using the large language model to generate the case summary corresponding to the user question, the case summary corresponding to the user question is used as the retrieval query statement of the case data vector knowledge base.

4. The method according to claim 1, characterized in that: Screening of relevant legal knowledge, including: Define the relevance assessment criteria based on the legal analysis of similar cases, detailed explanation of laws and regulations, and determination of the identity of the problem purpose; Based on the relevance evaluation standard, a relevance evaluation prompt instruction is constructed; wherein the relevance evaluation prompt instruction requires the output of a highly relevant knowledge list, and the information in the relevance evaluation prompt instruction includes: similar case knowledge, legal and regulatory knowledge, legal question and answer knowledge, user questions, and relevance evaluation prompt corpus; For user question samples and knowledge blocks recalled based on the user question samples, a highly relevant knowledge list is output with the help of a large language model and relevance evaluation prompt instructions; Training a knowledge screening model based on the highly relevant knowledge list; The trained knowledge screening model is used to screen the relevant legal knowledge.

5. The method according to claim 1, characterized in that Evaluate whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question, including: Define the evaluation criteria for knowledge reflection based on the legal analysis of similar cases, detailed explanation of laws and regulations, and determination of the identity of the purpose of the problem; Based on the knowledge reflection evaluation standard, the legal knowledge in the candidate knowledge base is embedded in the reflection evaluation prompt instruction, and based on the big model, it is evaluated whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question; wherein, the information in the reflection evaluation prompt instruction includes: similar case knowledge, legal and regulatory knowledge, legal question and answer knowledge, user questions and reflection evaluation prompt corpus.

6. The method according to claim 1, characterized in that The generating of an answer to the user's question based on the legal knowledge in the candidate knowledge base includes: Based on the reciprocal fusion sorting algorithm, the legal knowledge in the candidate knowledge base is deduplicated and re-sorted to generate the final knowledge list; User questions and the final knowledge list are embedded into the instructions for generating answer prompts, and a large model is used to generate natural language answers to user questions.

7. The method according to claim 6, characterized in that According to the inverse fusion sorting algorithm, the legal knowledge in the candidate knowledge base is deduplicated and re-sorted to generate a final knowledge list, including: De-duplicate the legal knowledge in the candidate knowledge base; For different types of knowledge, select the knowledge content to be input into the big model; The inverse fusion sorting method is used to calculate the similarity between the knowledge content to be input into the big model and the user's question, and the deduplicated legal knowledge is sorted based on the similarity to obtain a final knowledge list; wherein the legal knowledge in the final knowledge list is integrated according to type.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In the case that the legal knowledge in the candidate knowledge base is not sufficient to fully answer the user's question, a new search query statement is generated based on the user's question and the deficiencies of the legal knowledge in the candidate knowledge base; Based on the new search query statement, the search of the search query statement in the multi-dimensional legal vector knowledge base is re-executed to obtain relevant legal knowledge of the user question.

9. A slow-thinking legal question-answering device based on the judge's perspective, characterized in that: The device comprises: Knowledge base construction module, used to construct a multi-dimensional legal vector knowledge base; The statement generation module is used to understand the user's questions and generate several search query statements; A knowledge retrieval model, used to retrieve the retrieval query statement in the multi-dimensional legal vector knowledge base to obtain relevant legal knowledge of the user's question; A knowledge evaluation module, used for screening the relevant legal knowledge to construct a candidate knowledge base, and evaluating whether the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question; The answer generation module is used to generate an answer to the user's question based on the legal knowledge in the candidate knowledge base when the legal knowledge in the candidate knowledge base is sufficient to fully answer the user's question.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the slow thinking mode legal question-and-answer method based on the judge's perspective as described in any one of claims 1 to 8 is implemented.

Images