A method for automatically generating legal advice based on search enhancement

Abstract

The application discloses a kind of legal advice automatic generation method based on search enhancement, including legal advice dataset construction step, legal clause search step, case fact splitting step, case fact coding step and legal advice automatic generation step.Legal advice dataset construction is to collect legal advice data and legal clause data and carry out cleaning, legal clause search step is to search out the legal clause relevant to case fact, case fact splitting step is to split case fact into fragments for model processing, case fact coding step is to code the split case fact one by one and obtain case fact representation, legal advice automatic generation step is to generate legal advice according to case fact representation.The present application can handle complex case facts under limited hardware resources, and through the combination with the searcher, the fluency and logic of the generated legal advice can be effectively improved, and the update of legal clauses can be flexibly responded.

Description

Technical Field

[0001] The present invention relates to a method for automatically generating legal opinions based on retrieval enhancement, and belongs to the field of natural language processing. Background Art

[0002] In recent years, the combination of law and natural language processing technology has spawned many legal intelligence products, such as intelligent legal consulting systems and automatic legal opinion generation systems. They have greatly reduced the cost of obtaining legal services and are of great significance to the promotion of smart judicial construction. Current research on legal opinion generation can be divided into two aspects: (1) Criminal legal opinion generation: predicting the crimes involved in the defendant based on the description of the case facts, and making reasonable explanations based on legal provisions such as the criminal law. (2) Civil legal opinion generation: responding to the claims of the plaintiff and the defendant based on the description of the case facts, and giving reasonable explanations based on legal provisions such as the civil law. Current methods often use a generative model based on the Transformer structure to process the case facts and generate legal opinions with high fluency. In addition, additional subtasks such as crime prediction or legal clause prediction are added to improve the logic of the generated court opinions. Large language models have achieved excellent performance in many natural language processing tasks, such as LLaMA. Therefore, many methods directly apply large language models to the legal opinion generation task and have also achieved good performance.

[0003] However, the current method still has certain defects:

[0004] Unable to handle complex case facts. Current generative models all use a Transformer structure, and the input length of these models has a quadratic relationship with the model's computational complexity. In real-world scenarios, case facts are relatively complex, manifested in lengthy descriptions. Therefore, a larger maximum input length during model training places higher demands on the hardware. Current methods limit the maximum input length to 1024 tokens, making the model unable to handle complex case facts.

[0005] Inability to flexibly adapt to legal updates. Previous methods incorporate additional legal knowledge by designing a multi-label classification subtask for legal clause prediction, thereby improving the logic of generated court opinions. Legal clauses undergo a process of enactment, amendment, and repeal. Previous methods require retraining to better adapt to legal clause updates, which increases model training and deployment costs.

[0006] To solve the above problems, we designed a retrieval-enhanced automatic generation method for legal opinions.

[0007] Glossary:

[0008] Transformer layer: The Transformer is an encoder-decoder model. In the BERT pre-trained model used in this paper, the Transformer layer refers to the Transformer encoder, which primarily consists of a multi-head attention mechanism, a feedforward neural network, residual connections, and a normalization layer.

[0009] Token: When processing text, the pre-trained model first uses the model's tokenizer to split the text into a sequence of tokens. It then uses a vocabulary to find the number of each token, thereby converting the text into a vector for model processing. When BERT processes input text, it adds a special token, "[CLS]," to the beginning. During pre-training, BERT randomly masks some tokens, replacing them with "[MASK]" tokens. This masked language model training enhances BERT's representational capabilities.

[0010] LoRA: An efficient parameter fine-tuning solution, commonly used for fine-tuning large language models, which can effectively reduce graphics memory requirements without introducing inference costs.

[0011] MIPS: Full name, Maximum Inner Product Search. Given a vector query and a vector set X (of the same dimension), find some vectors in the vector set X that have a large dot product with the query.

[0012] Faiss: Full name, Facebook AI Similarity Search, provides data compression methods and implements vector fast retrieval functions.

[0013] BM25: BM25 is a classic algorithm used in the information indexing field to calculate the similarity score between queries and documents. It takes into account the relevance between each word in the query and the document, the relevance between words in the query and the query, and the word frequency information.

[0014] Lawformer: A pre-trained language model for long Chinese legal documents. Summary of the Invention

[0015] The present invention designs a method for automatically generating legal opinions based on retrieval enhancement, which can handle complex case facts under limited hardware resources. By using a legal clause retriever, it can flexibly respond to the update of legal clauses and improve the fluency and logic of the generated legal opinions.

[0016] A method for automatically generating legal opinions based on retrieval enhancement includes the following steps:

[0017] Step 1: Construct a legal clause search engine, which includes a case fact encoder. fact and legal terms encoder law ;

[0018] Step 2: Constructing a legal clause knowledge base and a legal opinion dataset. The legal opinion dataset includes time information and legal clause information in the legal opinions;

[0019] Step 3: Train the case fact encoder using the legal terms knowledge base and legal opinion dataset fact and legal terms encoder law Get a trained legal terms search engine;

[0020] Step 4: Training the legal opinion generation model: Input the case facts for which legal opinions need to be generated into the legal opinion generation model. The legal opinion generation model first divides the Chinese sentences into n sentences according to the punctuation marks at the end of the sentences, and then divides the n sentences into s segments on an even basis.

[0021] Then, t tokens are concatenated before each segment, and the segments are encoded using the BERT pre-trained model. When processing the first segment, t "[MASK]"s are used to initialize the concatenated portion. Subsequent segments are initialized using the concatenated portion of the previous segment, i.e., the representation corresponding to the t tokens. Finally, the representation corresponding to the concatenated portion of the last concatenated segment is taken as the representation of the case facts.

[0022] The obtained case facts are then projected into the word embedding space of the large language model LLaMA and the obtained case facts are input into LLaMA to generate a legal opinion;

[0023] Calculate the training loss of the legal opinion generator based on the legal opinions generated by LLaMA and make After convergence, the trained legal opinion generation model is obtained;

[0024] Step 5. Generate legal opinions: Input the facts of the case to be handled into the trained legal clause retriever to retrieve the legal clauses related to the facts of the case to be handled, and then input the retrieved legal clauses and the facts of the case to be handled into the trained legal opinion generation model to obtain the final legal opinion.

[0025] For further improvement, the specific steps of step 2 are as follows:

[0026] In step 2, the method for constructing the legal terms knowledge base and legal opinion dataset is as follows:

[0027] We crawled the judgment documents from the China Judgments Online website and extracted the legal opinions. We extracted the time information and legal clause information from the legal opinions, and finally constructed a legal opinion dataset. We crawled the legal clauses from the national laws and regulations database and constructed a legal clause knowledge base.

[0028] Further improvement, in step 2, the legal opinion dataset includes a legal clause knowledge base L = {L1, L2, ..., L n}, case facts F=[f1,f2,...,f s ], legal provisions related to the facts of the case Among them, n + is the number of legal provisions related to the facts of the case, select L + As a positive sample of the case facts F; the set of legal clauses that are irrelevant to the case facts is LL + ;Use BM25 in LL + Search for the legal clause that is most similar to the case facts as the negative sample of the case facts F, denoted as n - is the number of legal clauses that are irrelevant to the facts of the case; s represents the number of tokens in the facts of the case.

[0029] Further improvement, in step 3, the training loss of the legal term retriever As shown below:

[0030]

[0031] Among them, sim() is used to calculate the similarity between the facts of the case and the legal provisions. Here, sim(F,L)=Encoder fact (F) T Encoder law (L), T represents the matrix transpose, F represents the facts of the case, Encoder fact () indicates the case facts encoder code, Encoder law () indicates the legal clause encoder code; represents the jth legal clause related to the facts of the case, represents the i-th legal clause related to the facts of the case, represents the kth legal clause that is irrelevant to the facts of the case; + represents a positive sample, and - represents a negative sample;

[0032] When training a legal term retriever, Convergence to optimize the case fact encoder fact and legal terms encoder law .

[0033] Further improvement, in step 3, the case facts encoder Encoder fact and legal terms encoder law The data processing flow is as follows:

[0034] Case Facts Encoder fact The facts of the case F=[f1,f2,...,f s ] is encoded to obtain the representation of the case facts F in, is an s×d dimensional matrix, where s is the length of the legal text, d is the dimension of the hidden layer state in the encoder, and f s Indicates the sth token in the legal text, Indicates the representation of the sth token in the case facts; take The token "[CLS]" is used as a representation of the facts of the case. The specific calculation process is shown in formula (1);

[0035] Legal Terms Encoder law For legal terms L=[l1,l2,...,l m ] is encoded to obtain the representation of the legal clause L in, It is an m×d dimensional matrix, where m is the length of the legal clause; The "[CLS]" token is used as a legal term. The calculation process is shown in formula (2);

[0036]

[0037]

[0038] Among them, the case facts encoder Encoder fact and legal terms encoder law Both are pre-trained Lawformer models, but they do not share weights. [0,:] represents taking out the first element in a two-dimensional vector, that is, taking out the representation corresponding to the "[CLS]" token; H F The facts of the case indicate that H L Indicates that the legal terms indicate that Table The representation of the 0th position in , Indicates taking The representation of the 0th position in , lawformer() represents the Lawformer model.

[0039] Further improvement, in step 4, the case facts are divided into n sentences, namely F = [seg1, seg2, ..., seg n ], then divide n sentences into s segments, that is, F = [f1,f2,...,f s ] is divided into multiple segments.

[0040] Further improvement, the case fact encoder Encoder fact The steps for coding the case facts are as follows:

[0041] Concatenate t "[MASK]" tokens before the first segment, and then encode it using the BERT pre-trained model. Specifically, given a segment in the case facts F Concatenate the fragment seg1 with the "[MASK]" token to obtain Then input it into the pre-trained model BERT to get Representation The specific steps are shown in formulas (4) and (5):

[0042]

[0043]

[0044] BERT() represents the pre-trained model BERT, which takes the representation corresponding to t “[MASK]” tokens as the representation of the current segment, that is, the representation of segment seg1 is mask t Indicates the tth mask, [:t,:] means taking the representation of the first t tokens, Indicates the s1th token in the 1st segment;

[0045] When encoding other segments, the representation of the previous segment is used to initialize the embedding of t tokens. Specifically, given the i-th segment seg i Representation Will With the i+1th segment seg i+1 Embed Splicing to obtain The calculation process is shown in formula (6), the concatenated embedding Input into the pre-trained model BERT and take the representation of the first t tokens to get The calculation process is shown in formula (7). and Perform residual connection to get seg i+1 Fragment Representation And so on, until all the fragments are encoded, the representation of the first t tokens of the last fragment is selected as the representation of the entire case facts H F , the calculation steps are shown in formula (8).

[0046]

[0047]

[0048]

[0049]

[0050] Among them, || represents the concatenation operation, EmbeddingLayer() is the embedding layer in the pre-trained BERT, and TransformerLayers() is the 12-layer Transformers in the pre-trained model BERT.

[0051] As a further improvement, LoRA, an efficient parameter fine-tuning method, is used to train the BERT model.

[0052] For further improvement, the steps in step 4 are as follows:

[0053] Use the adapter to align the case fact representation obtained by the pre-trained model BERT with the large language model LLaMA: Align the case fact representation H F Projected into the word embedding space of the LLaMA model to obtain the case fact embedding The specific calculation steps are shown in formula (9).

[0054]

[0055] Among them, Relu() is the activation function, is the weight matrix, b is the bias, and the hidden layer state dimension of the large language model LLaMA is d llm ;Adatpter() is the BERT model;

[0056] Embed the facts of the case Input into the large language model LLaMA to obtain the generated legal opinion, denoted as Training loss for the legal opinion generator The calculation process is shown in formula (10).

[0057]

[0058] Where T is the length of the actual legal opinion, Represents the legal opinion given the facts F of the case and the previous t steps To predict the probability of the token at step t.

[0059] Advantages of this invention: By splitting longer case facts into multiple segments and using the pre-trained BERT model to sequentially process the split segments and compress them into fixed-length virtual token embeddings, it can handle more complex case facts with limited hardware resources. By using a retriever to obtain legal terms and case fact representations, and using MIPS to retrieve the most relevant legal terms, this method can flexibly and effectively respond to updates to legal terms. Finally, the case facts and retrieved legal terms are input into the large language model LLaMA to obtain a fluent and logical court opinion. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 The overall flowchart of the automatic generation method of legal opinions based on retrieval enhancement is shown.

[0061] Figure 2 A model diagram of the automatic generation method of legal opinions based on retrieval enhancement is shown. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is described in detail below with reference to examples.

[0063] like Figure 1 As shown, the present invention provides a method for automatically generating legal opinions based on retrieval enhancement. Figure 2 The model diagram of the automatic generation method of legal opinions based on retrieval enhancement is shown. The method includes the following steps.

[0064] 1. Legal Opinion Dataset Construction Steps

[0065] We crawled judgment documents from the China Judgments Online website and extracted legal opinions. We removed sensitive information (such as names, places, and court names) from the legal opinions and extracted time information and legal clause information from the legal opinions. Finally, we constructed a legal opinion dataset. We crawled legal clauses from the national laws and regulations database and constructed a legal clause knowledge base.

[0066] 2. Legal terms search steps

[0067] Step 1021: Given a legal terms knowledge base L = {L1, L2, ..., L n}, a case fact F=[f1,f2,...,f s ] and legal provisions related to the facts of the case Among them, n + The number of legal provisions that are relevant to the facts of the case. The number of legal provisions that are not relevant to the facts of the case. + . Select L +As a positive sample of the case fact F, use BM25 in LL + Search for the legal clause that is most similar to the case facts as the negative sample of the case facts F, denoted as n - The number of legal provisions that are irrelevant to the facts of the case.

[0068] Step 1022: Use the case fact encoder fact The facts of the case F=[f1,f2,...,f s ] is encoded to obtain its representation in, is an s×d dimensional matrix, where s is the length of the legal text, d is the dimension of the hidden layer state in the encoder, and f s Indicates the sth token in the legal text, Indicates the representation of the sth token in the case facts. We take The token "[CLS]" is used as a representation of the facts of the case. The specific calculation process is shown in formula (1); using the legal terms encoder Encoder law For legal terms L=[l1,l2,...,l m ] is encoded to obtain its representation in, is an m×d dimensional matrix, where m is the length of the legal clause. The "[CLS]" token is used as a legal term. The calculation process is shown in formula (2);

[0069]

[0070]

[0071] Among them, the case facts encoder Encoder fact and legal terms encoder law Both are pre-trained Lawformer models, but they do not share weights. [0,:] represents taking out the first element in a two-dimensional vector, that is, taking out the representation corresponding to the "[CLS]" token.

[0072] Step 1023: Based on the legal terms representation and case facts representation obtained in the previous step, we can obtain the training loss of the legal terms retriever And use this to optimize the case fact encoder Encoder fact and legal terms encoder lawThe calculation process is shown in formula (3):

[0073]

[0074] Among them, sim(·) is used to calculate the similarity between the case facts and the legal provisions. Here, sim(F,L)=Encoder fact (F) T Encoder law (L).

[0075] Step 1024: During the retrieval phase, we first use the legal clause encoder to encode all legal clauses in the legal clause knowledge base to obtain their representations. Then, we use the case fact encoder to encode the case facts to obtain their representations. We use MIPS to measure the similarity between the case facts and the legal clauses, and leverage the Faiss database to quickly retrieve the K legal clauses most relevant to the case facts.

[0076] 3. Steps for breaking down the facts of the case

[0077] The case facts F=[seg1,seg2,...,seg n ] is divided into n sentences, and then the n sentences are evenly divided into s fragments F = [f1,f2,...,f s ]. Each segment is of similar length and consists of multiple complete sentences.

[0078] 4. Steps for coding case facts

[0079] Step 10321: Concatenate t “[MASK]” tokens before the first segment obtained in each step 1031, and then encode it using the BERT pre-trained model. Specifically, given a segment in the case facts F Concatenate it with the "[MASK]" token to get Then input it into the pre-trained model BERT to get its representation The specific steps are shown in formulas (4) and (5):

[0080]

[0081]

[0082] BERT(·) represents the pre-trained model BERT. We use the representation corresponding to t “[MASK]” tokens as the representation of the current segment, that is, the representation of segment seg1 is

[0083] Step 10322: When encoding other segments, we will use the representation of the previous segment to initialize the embedding of t tokens. Specifically, given the i-th segment seg i Representation Combine it with the i+1th segment seg i+1 Embed Splicing to obtain The calculation process is shown in formula (6). Input into the pre-trained model BERT and take the representation of the first t tokens to get The calculation process is shown in formula (7). and Perform residual connection to obtain seg i+1 Fragment Representation This process is repeated until all the fragments are encoded, and the first t tokens of the last fragment are selected as the representation of the entire case facts H F , and its calculation steps are shown in formula (8).

[0084]

[0085]

[0086]

[0087]

[0088] Here, || represents a concatenation operation. EmbeddingLayer(·) is the embedding layer in the pre-trained BERT model, and TransformerLayers(·) is the 12-layer Transformer layer in the pre-trained BERT model. It should be noted that to accelerate model training, we use the efficient parameter fine-tuning method LoRA to train the BERT model.

[0089] 5. Steps for Automatically Generating Legal Opinions

[0090] Step 10331: Use the adapter to align the case fact representation obtained by the pre-trained model BERT with the large language model LLaMA. The specific steps are to align the case fact representation H F Projected into the word embedding space of the LLaMA model to obtain the case fact embedding The specific calculation steps are shown in formula (9).

[0091] Among them, Relu(·) is the activation function, is the weight matrix, b is the bias, and the hidden layer state dimension of the large language model LLaMA is d llm .

[0092] Step 10332: Embed the case facts The legal opinion generated by inputting it into a large language model is denoted as Training loss for the legal opinion generator The calculation process is shown in formula (10).

[0093]

[0094] Where T is the length of the actual legal opinion, Represents the legal opinion given the facts F of the case and the previous t steps To predict the probability of the token at step t.

[0095] In the specific implementation process, we implemented a retrieval-enhanced legal opinion automatic generation method based on the Pytorch deep learning framework and used the parameter-efficient fine-tuning method LoRA to train the legal opinion generation model. We tested the method proposed in this invention on the CJO dataset (criminal legal opinion dataset) and the CIVIL-DIV dataset (civil legal opinion dataset). During the test, we kept the hyperparameters consistent and used multiple automatic evaluation indicators such as ROUGE, BLEU and BERT Score to evaluate the quality of the generated legal opinions. Tables 1 and 2 show the comparative experimental results on the CJO and CIVIL-DIV datasets, respectively. Among them, R-1, R-2 and RL represent ROUGE-1, ROUGE-2 and ROUGE-L, respectively, B-1, B-2 and BN represent BLEU-1, BLEU-2 and BLEU-N, respectively, and P and R represent Precision and Recall, respectively. REG represents the method of this article.

[0096] Table 1 Comparative experimental results on the criminal law dataset CJO

[0097]

[0098] Table 2 Comparative experimental results on the civil law dataset CIVIL-DIV

[0099]

[0100]

[0101] The comparative experimental results in Tables 1 and 2 show that our method outperforms the comparative methods on both CJO and CIVIL-DIV. Furthermore, our method achieves a higher performance improvement on the CIVIL-DIV dataset than on the CJO dataset. The case facts in the CIVIL-DIV dataset are much longer than those in the CJO dataset, demonstrating that our method is well suited to handling longer case facts. As the number of parameters in Tables 1 and 2 increases, performance improves accordingly. Using a large language model can provide a certain guarantee for generating high-quality legal opinions.

Claims

1. A method for automatically generating legal opinions based on retrieval enhancement, characterized in that: The following steps are involved: Step 1: Construct a legal clause search engine, which includes a case fact encoder. fact and legal terms encoder law ; Step 2: Construct a legal clause knowledge base and a legal opinion dataset, wherein the legal opinion dataset includes time information and legal clause information in the legal opinions; Step 3: Train the case fact encoder using the legal terms knowledge base and legal opinion dataset fact and legal terms encoder law Get a trained legal terms search engine; Step 4: Training the legal opinion generation model: Input the case facts for which legal opinions need to be generated into the legal opinion generation model. The legal opinion generation model first divides the Chinese sentences into n sentences according to the punctuation marks at the end of the sentences, and then divides the n sentences into s segments on an even basis. Then, t tokens are concatenated before each segment. The segments are then encoded using the BERT pre-trained model. When processing the first segment, t "[MASK]"s are used to initialize the concatenated portion. Subsequent segments are initialized using the concatenated portion of the previous segment, i.e., the representation corresponding to the t tokens. Finally, the representation corresponding to the concatenated portion of the last concatenated segment is taken as the representation of the case facts. The obtained case facts are then projected into the word embedding space of the large language model LLaMA and the obtained case facts are input into LLaMA to generate a legal opinion; Calculate the training loss of the legal opinion generator based on the legal opinions generated by LLaMA and make After convergence, the trained legal opinion generation model is obtained; Step 5. Generate legal opinions: Input the facts of the case to be handled into the trained legal clause retriever to retrieve the legal clauses related to the facts of the case to be handled, and then input the retrieved legal clauses and the facts of the case to be handled into the trained legal opinion generation model to obtain the final legal opinion.

2. The method for automatically generating legal opinions based on retrieval enhancement according to claim 1, characterized in that: The specific steps of step 2 are as follows: In step 2, the method for constructing the legal terms knowledge base and legal opinion dataset is as follows: We crawled the judgment documents from the China Judgments Online website and extracted the legal opinions. We extracted the time information and legal clause information from the legal opinions, and finally constructed a legal opinion dataset. We crawled the legal clauses from the national laws and regulations database and constructed a legal clause knowledge base.

3. The method for automatically generating legal opinions based on retrieval enhancement according to claim 1, characterized in that: In the step 2, the legal opinion dataset includes a legal clause knowledge base L = {L1, L2, ..., L n }, case facts F = [f1, f2, ..., f s ], legal provisions related to the facts of the case Among them, n + is the number of legal provisions related to the facts of the case, select L + As a positive sample of the case facts F; the set of legal clauses that are irrelevant to the case facts is LL + ;Use BM25 in LL + Search for the legal clause that is most similar to the case facts as the negative sample of the case facts F, denoted as n - is the number of legal clauses that are irrelevant to the facts of the case; s represents the number of tokens in the facts of the case.

4. The method for automatically generating legal opinions based on retrieval enhancement according to claim 2, characterized in that: In step 3, the training loss of the legal term retriever is As shown below: Among them, sim() is used to calculate the similarity between the case facts and the legal provisions. Here, sim(F, L) = Encoder fact (F) T Encoder law (L), T represents the matrix transpose, F represents the facts of the case, Encoder fact () indicates the case facts encoder code, Encoder law () indicates the legal clause encoder code; represents the jth legal clause related to the facts of the case, represents the i-th legal clause related to the facts of the case, represents the kth legal clause that is irrelevant to the facts of the case; + represents a positive sample, and - represents a negative sample; When training a legal term retriever, Convergence to optimize the case fact encoder fact and legal terms encoder law .

5. The method for automatically generating legal opinions based on retrieval enhancement according to claim 4, characterized in that: In step 3, the case facts encoder Encoder fact and legal terms encoder law The data processing flow is as follows: Case Facts Encoder fact The facts of the case F=[f1,f2,...,f s ] is encoded to obtain the representation of the case facts F in, is an s×d dimensional matrix, where s is the length of the legal text, d is the dimension of the hidden layer state in the encoder, and f s Indicates the sth token in the legal text, Indicates the representation of the sth token in the case facts; take The token "[CLS]" is used as a representation of the facts of the case. The specific calculation process is shown in formula (1); Legal Terms Encoder law For legal terms L=[l1,l2,...,l m ] is encoded to obtain the representation of the legal clause L in, It is an m×d dimensional matrix, where m is the length of the legal clause; The "[CLS]" token is used as a legal term. The calculation process is shown in formula (2); Among them, the case facts encoder Encoder fact and legal terms encoder law Both are pre-trained Lawformer models, but they do not share weights. [0,:] represents taking out the first element in a two-dimensional vector, that is, taking out the representation corresponding to the "[CLS]" token; H F The facts of the case indicate that H L Indicates that the legal terms indicate that Table The representation of the 0th position in , Indicates taking The representation of the 0th position in , lawformer() represents the Lawformer model.

6. The method for automatically generating legal opinions based on retrieval enhancement according to claim 5, characterized in that: In step 4, the case facts are divided into n sentences, namely F = [seg1, seg2, ..., seg n ], then divide n sentences into s segments, that is, F = [f1,f2,...,f s ] is divided into multiple segments.

7. The method for automatically generating legal opinions based on retrieval enhancement according to claim 6, characterized in that: Encoder of the facts of the case fact The steps for coding the case facts are as follows: Concatenate t "[MASK]" tokens before the first segment, and then encode it using the BERT pre-trained model. Specifically, given a segment in the case facts F Concatenate the fragment seg1 with the "[MASK]" token to obtain Then input it into the pre-trained model BERT to get Representation The specific steps are shown in formulas (4) and (5): BERT() represents the pre-trained model BERT, and the representation corresponding to t "[MASK]" tokens is used as the representation of the current segment, that is, the representation of segment seg1 is mask t Indicates the tth mask, [:t,:] means taking the representation of the first t tokens, Indicates the s1th token in the 1st segment; When encoding other segments, the representation of the previous segment is used to initialize the embedding of t tokens. Specifically, given the i-th segment seg i Representation Will With the i+1th segment seg i+1 Embed Splicing to obtain The calculation process is shown in formula (6), the concatenated embedding Input into the pre-trained model BERT and take the representation of the first t tokens to get The calculation process is shown in formula (7). and Perform residual connection to get seg i+1 Fragment Representation And so on, until all the fragments are encoded, the representation of the first t tokens of the last fragment is selected as the representation of the entire case facts H F , the calculation steps are shown in formula (8): Among them, || represents the concatenation operation, EmbeddingLayer() is the embedding layer in the pre-trained BERT, and TransformerLayers() is the 12-layer Transformers in the pre-trained model BERT.

8. The method for automatically generating legal opinions based on retrieval enhancement according to claim 7, characterized in that: The parameter-efficient fine-tuning method LoRA is used to train the BERT model.

9. The method for automatically generating legal opinions based on retrieval enhancement according to claim 1, characterized in that: The steps in step 4 are as follows: Use the adapter to align the case fact representation obtained by the pre-trained model BERT with the large language model LLaMA: Align the case fact representation H F Projected into the word embedding space of the LLaMA model to obtain the case fact embedding The specific calculation steps are shown in formula (9): Among them, Relu() is the activation function, is the weight matrix, b is the bias, and the hidden layer state dimension of the large language model LLaMA is d llm ;Adatpter() is the BERT model; Embed the facts of the case Input into the large language model LLaMA to obtain the generated legal opinion, denoted as Training loss for the legal opinion generator The calculation process is shown in formula (10): Where T is the length of the actual legal opinion, Represents the legal opinion given the facts F of the case and the previous t steps To predict the probability of the token at step t.

Images