A similar case retrieval method and system based on legal elements

Abstract

The present invention relates to a similar case retrieval method and system based on legal elements, including: step 1: semantic coding; first divide into short paragraphs, and semantically code the query paragraph and the candidate paragraph; step 2: topic distribution coding; respectively obtain the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set; step 3: legal entity coding; extract legal elements from the document, and calculate the similarity between the two case paragraphs; step 4: coding aggregation and similarity calculation; code the semantic coding, topic distribution coding and legal entity coding, and calculate the similarity of the query-candidate pair, and select the first few corresponding candidate documents with the largest similarity as similar cases. The present invention uses the powerful semantic learning ability of BERT to solve the long text coding problem of legal cases, accurately mine the legal elements in legal cases, and is more suitable for legal scenarios.

Description

Technical Field

[0001] The present invention relates to a similar case retrieval method and system based on legal elements, belonging to the technical field of text similarity calculation. Background Art

[0002] With the development of smart justice in China, and against the background of the continuous and comprehensive implementation of the "mandatory search for similar and related cases" system, judicial big data has been continuously disclosed, and the application scope of "similar case search" has been continuously expanded. Therefore, the accuracy of similar case search results needs to be improved urgently.

[0003] Legal case retrieval is a special retrieval task, which refers to retrieving similar cases for a given query. Compared with traditional text retrieval, legal case retrieval has different characteristics and greater challenges, mainly reflected in the following aspects:

[0004] (I) Legal case texts are often long. Lengthy texts will cause the model to fail to take into account all useful information when establishing a vector representation of the text. At present, the most commonly used neural network models in the text field, such as the long short-term memory network, do not have strong memory capabilities and are not effective in long texts. This is also a major reason why methods and models for calculating general text similarity are not effective in the legal field.

[0005] (II) The similarity of legal cases is different from the common text similarity, and to some extent, it also exceeds the general definition of text subject relevance. It requires mining the similarity of legal elements contained in the text. Traditional text similarity calculation methods can indeed learn semantics, but because the model does not understand legal domain knowledge, it may not be able to learn the deeper legal-related logical connections beneath the surface semantics. This results in the inability to find highly similar legal cases simply using text similarity calculation methods. Therefore, it is crucial to identify the similarities of cases in terms of legal issues and legal processes.

[0006] (III) The structure of legal texts is different from that of general texts. The general text similarity model mainly considers the structural characteristics of the text, such as the syntactic structure, but it cannot represent the legal text well. Although judgment documents are unstructured texts, they have specific format requirements. If the structural characteristics of judgment documents can be combined with the calculation of traditional text similarity, better results may be achieved.

[0007] When considering whether two cases are similar, we are actually considering whether the legal elements contained in the two cases and the logic of the events that make up them are similar. These legal elements are often reflected in the defendant's behavior and are the key factors used to determine whether the defendant has violated certain laws and regulations, what crime should be sentenced, and how long the sentence should be. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention provides a similar case retrieval method based on legal elements.

[0009] The present invention also provides a similar case retrieval system based on legal elements. Summary of the invention:

[0011] The model framework of the present invention includes three modules. The first part is the legal feature encoding module, which includes a BERT-based semantic encoding module, an LDA-based topic distribution encoding module and a legal entity encoding module; the second part is the encoding aggregation module, which is responsible for encoding and aggregating the results output by the first part before entering the third part; the third part is the similarity calculation module.

[0012] In view of the shortcomings of traditional text similarity methods in legal text similarity, the present invention extracts legal element information through three encoding modules, captures the semantic context information of the case through a BERT-based semantic encoding module, clusters the subject information of the case description through a topic encoding module, and more accurately enhances the role of the legal element information through a legal element encoding module. The aggregated paragraph encoding information is input into a gated recurrent neural network GRU (Gate Recurrent Unit) to extract the paragraph-level logical relationship of the legal case description, and finally recognizes the paragraph-level key information through an attention mechanism.

[0013] Terminology explanation:

[0014] 1. BERT: An autoencoder language model that extracts the relationship features of words in a sentence and obtains semantic features based on the sentence context.

[0015] 2. [CLS] mark: placed at the beginning of the first sentence. The representation vector C obtained by BERT can be used for subsequent classification tasks.

[0016] 3. [SEP] mark: used to separate two input sentences. For example, to input sentences A and B, add the [SEP] mark after sentences A and B.

[0017] 4. LDA model: A topic probability model that can obtain the topic distribution of documents based on the inverse process of LDA-based document generation.

[0018] 5. GRU model: A variant of the long short-term memory network LSTM, which is a recursive neural network and can be used to solve long dependency problems in text encoding.

[0019] 6. Attention mechanism. The attention mechanism was first proposed by machine translation tasks. By calculating the weight distribution of each word in the encoder of the sequence generated by the decoder side of the model, a feature vector representation with weighted association information is obtained to improve the translation effect of the model. Here, it can be understood as a mechanism for allocating attention to input weights.

[0020] 7. Dirichlet distribution: Dirichlet distribution is a set of continuous multivariate probability distributions and a generalized B distribution of multivariate.

[0021] 8. Gibbs Sampling method, Gibbs sampling method, commonly used in DBM and DBN. Gibbs sampling is mainly used for inference of LDA and other model parameters.

[0022] 9. BM25 is an algorithm used to evaluate the relevance between search terms and documents. It is an algorithm based on the probabilistic retrieval model.

[0023] 10. TF-IDF (term frequency–inverse document frequency) is a commonly used weighting technique for information retrieval and text mining. It is a statistical method used to evaluate the importance of a word to a document set or one of the documents in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. If a word appears in an article with a high TF frequency and rarely appears in other articles, it is considered that this word or phrase has good category discrimination ability.

[0024] 11. LMIR is an information retrieval language model proposed in 1999. It is based on a series of data smoothing techniques, including Good-Turing estimation, curve fitting functions and model combinations. The model can be easily extended to include the probability of phrases, such as word pairs and word triplets.

[0025] 12. BERT model, Bidirectional Encoder Representations from Transformer, is a pre-trained language model, such as Figure 2As shown in the figure, it is a bidirectional encoder representation based on Transformer. As the name implies, BERT uses Transformer, and when processing a word, it can also take into account the words before and after the word to get its meaning in the context. Its purpose is to train the underlying and common partial models in the downstream tasks in advance, and then use the sample data of the downstream tasks to train their respective models, which can greatly speed up the convergence speed. For the downstream tasks of NLP, although the final goals are different, they also have common necessary steps, that is, to convert the words in the text that cannot be directly calculated into computable vectors or matrices, etc., and these digital vectors should be able to better reflect the meaning of the corresponding words in the sentence.

[0026] 13. The GRU model is a variant of the LSTM network that works very well. It has a simpler structure than the LSTM network and also works very well. Figure 3 As shown in the figure, three gate functions are introduced in LSTM: input gate, forget gate and output gate to control input value, memory value and output value. In the GRU model, there are only two gates: update gate and reset gate. GRU has fewer parameters, so the training speed is faster. GRU can also solve the long dependency problem in RNN network and reduce the risk of overfitting.

[0027] 12. BERT-PLI is a legal case retrieval model proposed in 2020. The model uses BERT to capture paragraph-level semantic relations and then infers the relevance between two cases by aggregating paragraph-level interactions.

[0028] The technical solution of the present invention is:

[0029] A similar case retrieval method based on legal elements includes the following steps:

[0030] Step 1: Semantic encoding: first divide the long text into short paragraphs according to the order of case description, and then use the BERT-based text encoding method to semantically encode the query paragraph and candidate paragraphs;

[0031] Step 2: Topic distribution encoding: This means: based on the inverse process of LDA model document generation, the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set is obtained respectively;

[0032] Step 3: Legal entity coding: extracting legal elements from the document, performing synonym expansion on the legal elements contained in each case, and finally calculating the similarity between the paragraphs of two cases;

[0033] Step 4: Coding aggregation and similarity calculation: This means: the semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and similarity calculation is performed on the query-candidate pair, and the first few corresponding candidate documents with the largest similarity are selected as similar cases.

[0034] Preferably, according to the present invention, the specific implementation process of step 1 is as follows:

[0035] Step 1.1: Formalize, for a query document q and any candidate document d k They are respectively expressed as q=(p q1 ,p q2 ,…,p qi …,p qN ), dk=(p k1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d respectively. k The total number of paragraphs; for query document q and candidate document d k For each paragraph in the query, construct a query-candidate paragraph pair (p qi ,p kj ), where 1≤i≤N and 1≤j≤M;

[0036] Step 1.2: Paragraph pair (p qi ,p kj ) is fed into the BERT model along with the retained tokens, namely [CLS] and [SEP]; the final hidden layer vector of the first token [CLS] is output as the query-candidate paragraph pair (p qi ,p kj ), as shown in formula (1):

[0037] C ij =BERT(p qi ,p kj ) (1)

[0038] In this way, the interaction matrix of all query-candidate paragraph pairs is obtained. qi ,p kj ) is represented by C ij ,C ij ∈R HB ;

[0039] Step 1.3: For each paragraph of the query document, use maxpooling to find the strongest matching paragraph of the candidate document, thus obtaining a vector sequence, denoted as p' qk =[p' qk1 ,p' qk2,...,p' qki ...,p' qkN ]; p' qki As shown in formula (2):

[0040]

[0041] Step 1.4: Use the GRU model to train p' qki Further encoding is performed to obtain the semantic encoding result h qk =[h qk1 ,h qk2 ,…,h qki …,h qkN ],h qki ∈R HR .

[0042] Preferably, according to the present invention, the specific implementation process of step 2 is as follows:

[0043] Step 2.1: The process of using the generated document to infer its hidden topic is the inverse process of LDA model document generation. Only word w in document d is observed, and topic z is hidden. Based on a large amount of known document-term information, namely word w in document d, the posterior distribution of the topic on each document is calculated, as shown in formula (3):

[0044]

[0045] in, is the topic distribution, yes The Dirichlet distribution of is the prior distribution parameter of the topic distribution, The vector of the number of topics that make up document m, is a certain topic, and the posterior distribution of the topic is solved by the Gibbs Sampling method

[0046] Step 2.2: According to formula (3), we can get paragraph P qi , P kj The posterior distribution of the topics is ZP qi =[ZP qi-1 ,ZP qi-2 ,...,ZP qi-v ] and ZP kj =[ZP kj-1 ,Z kj-2 ,...,ZP kj-v ],

[0047] The topic probability distribution interaction matrix T of the query document paragraph set and the candidate document paragraph set is calculated qik, the query document paragraph set and the candidate document paragraph set refer to the query document q=(p q1 ,p q2 ,…,p qi …,p qN ), any candidate document dk=(p k1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d, respectively. k The total number of paragraphs; as shown in formula (4):

[0048]

[0049] Among them, v is the number of topics hyperparameter, and each element is T qik express.

[0050] Preferably, the specific implementation process of step 3 is as follows:

[0051] Step 3.1: extract legal elements from the document using regular expressions, including crime name ccg, criminal behavior co, character type ct, criminal consequence cc, compensation behavior cb, and reconciliation situation r; and then perform synonym expansion on the legal elements contained in each case;

[0052] Step 3.2: Calculate the similarity between the two case paragraphs in two parts:

[0053] The similarity calculation formula of the crime or compensation behavior is shown in formula (5):

[0054]

[0055] Among them, entityName1 represents the set of crimes or compensation behaviors extracted from the paragraph;

[0056] The similarity calculation method of criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r is shown in formula (6):

[0057]

[0058] Among them, entityName2 represents the criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r extracted from the paragraphs respectively;

[0059] The similarity between paragraphs based on all legal elements sim_e(E qi ,E kj ) is calculated as shown in formula (7):

[0060]

[0061] Among them, μ1, μ2, μ3 are weight parameters, sim_e(E qi ,E kj ) refers to the similarity between paragraphs based on legal elements, E qi and E kj Represents the entity set of the query paragraph and the candidate paragraph; thus, for the query paragraph set and the candidate paragraph set, a similarity matrix LE is obtained qk =[LE q1k ,LE q2k ,…,LE qik …,LE q1k ],LE qik =sim_e(E qi ,E kj ).

[0062] Preferably, the specific implementation process of step 4 is as follows:

[0063] The semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and the similarity of the query-candidate pair is calculated. The calculation formula is shown in formula (8):

[0064] E qki =β s h qki +β T T qik +β L LE qik

[0065] =β s [GRU(h qk (i-1),p' qki )]+β T T qik +β L LE qik (8)

[0066] Among them, β s ,β T ,β L are weight parameters respectively;

[0067] For the aggregated encoding output of the query-candidate pair, an attention mechanism is added to further encode the position information. The attention weight calculation method is shown in formula (9):

[0068]

[0069] α qki 、u qk They refer to the weights and parameters in the attention mechanism respectively;

[0070] Among them, u qk The calculation method of is shown in formula (10):

[0071] u qk =W u MaxPool(E qk )+b u (10)

[0072] In formula (10), MaxPool() means selecting the largest value in each row of data, q is the query document, k is a candidate document, E qk is the aggregate code of the query document q and a candidate document k; W u ∈R HR×HR ,b u ∈R HR ;

[0073] The document-level similarity relationship is obtained by aggregation formula (11):

[0074]

[0075] Finally, all d qk After a fully connected layer and a softmax function, the similarity probability values ​​of the two legal case description documents are obtained; the document corresponding to the value with the largest similarity probability value is the required similar case.

[0076] A similar case retrieval system based on legal elements, including a semantic coding module, a subject distribution coding module, a legal entity coding module, a coding aggregation and similarity calculation module;

[0077] The semantic encoding module is configured to: first divide the long text into short paragraphs according to the case description order, and then semantically encode the query paragraph and the candidate paragraphs through a BERT-based text encoding method;

[0078] The topic distribution encoding module is configured to: obtain the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set respectively based on the inverse process of LDA model document generation;

[0079] The legal entity coding module is configured to: extract legal elements from the document, perform synonym expansion on the legal elements contained in each case, and finally calculate the similarity between two case paragraphs;

[0080] The coding aggregation and similarity calculation module is configured to: perform coding aggregation on the semantic coding, the topic distribution coding and the legal entity coding, perform similarity calculation on the query-candidate pair, and select the first several corresponding candidate documents with the largest similarity as similar cases.

[0081] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device for the method of similar case retrieval based on legal elements.

[0082] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the similar case retrieval method based on legal elements.

[0083] The beneficial effects of the present invention are:

[0084] 1. The present invention adopts a paragraph segmentation and aggregation-based architecture to divide the long legal text into short paragraphs according to the logical order of the case, and then uses a BERT-based text encoding method to semantically encode the query and candidate paragraphs. On the one hand, it can utilize BERT's powerful semantic learning ability, and on the other hand, it can solve the long text encoding problem of legal cases.

[0085] 2. The present invention mines out several legal element entities in legal cases, including charges, criminal acts, types of criminal entities, criminal consequences, compensation behaviors, and reconciliation situations, which have a decisive impact on the judgment. It also performs synonym expansion on the legal elements contained in each case, accurately mining the legal elements in legal cases.

[0086] 3. The present invention aggregates the traditional semantic coding method with the legal text topic distribution and legal case entities through linear fitting, and makes the document representation vector more suitable for legal scenarios through the deep combination of legal element entities and document topics.

[0087] 4. The present invention not only encodes the position information of legal semantics within a paragraph through convolutional neural networks and attention mechanisms, but also performs logical judgment and reinforcement on the legal elements between paragraphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic diagram of a similar case retrieval system based on legal elements of the present invention;

[0089] Figure 2 This is a schematic diagram of the network architecture of the BERT model;

[0090] Figure 3 This is a schematic diagram of the network architecture of the GRU model;

[0091] Figure 4 Schematic diagram of the pre-training process of the BERT model;

[0092] Figure 5 Schematic diagram of the LDA topic model. DETAILED DESCRIPTION

[0093] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0094] Example 1

[0095] A similar case retrieval method based on legal elements includes the following steps:

[0096] Step 1: Semantic encoding: This means firstly dividing the long text into short paragraphs according to the order of case description, and then semantically encoding the query paragraphs and candidate paragraphs through the BERT-based text encoding method. On the one hand, it can utilize the powerful semantic learning ability of BERT, and on the other hand, it can solve the long text encoding problem of legal cases.

[0097] Step 2: Topic distribution encoding: This means: based on the inverse process of LDA model document generation, the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set is obtained respectively;

[0098] Step 3: Legal entity coding; means: the present invention mainly focuses on several parts in legal cases, such as criminal behavior, character type, and compensation behavior. These three parts contain legal elements that have a decisive impact on the judgment. Among them, character type and compensation behavior are applicable to all types of legal cases, and criminal behavior varies according to the different charges. Although legal judgment documents belong to unstructured text information, the composition and writing order of judgment documents are often based on certain writing standards. Regular expressions are used to extract legal elements from the documents, and synonym expansion is performed on the legal elements contained in each case. Finally, the similarity between the two case paragraphs is calculated;

[0099] Step 4: Coding aggregation and similarity calculation: This means: the semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and similarity calculation is performed on the query-candidate pair, and the first few corresponding candidate documents with the largest similarity are selected as similar cases.

[0100] Example 2

[0101] The difference between the similar case retrieval method based on legal elements described in Example 1 is that:

[0102] The specific implementation process of step 1 is as follows:

[0103] Step 1.1: Formalize, for a query document q and any candidate document d k They are respectively expressed as q=(p q1 ,p q2 ,…,p qi …,p qN ), dk=(pk1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d respectively. k The total number of paragraphs; for query document q and candidate document d k For each paragraph in the query, construct a query-candidate paragraph pair (p qi ,p kj ), where 1≤i≤N and 1≤j≤M;

[0104] Step 1.2: Paragraph pair (p qi ,p kj ) is input into the BERT model together with the retained tokens, namely [CLS] and [SEP]; in order to capture the word order, position embeddings are added, and these embedding information is input into transformers. At each layer of transformers, a new contextualized embedding representation is generated by calculating the weighted sum with other token embeddings. The weight values ​​are calculated by a multi-head attention mechanism. Words with large attention weights are considered to be more relevant to the target word. Different attention matrices capture different types of word relationships, such as exact matches and synonym relationships. Finally, the final hidden layer vector output of the first token [CLS] is used as the query-candidate paragraph pair (p qi ,p kj ), as shown in formula (1):

[0105] C ij =BERT(p qi ,p kj ) (12)

[0106] In this way, the interaction matrix of all query-candidate paragraph pairs is obtained. qi ,p kj ) is represented by C ij ,C ij ∈R HB ;

[0107] Step 1.3: For each paragraph of the query document, use maxpooling to find the strongest matching paragraph of the candidate document, thus obtaining a vector sequence, denoted as p' qk =[p' qk1 ,p' qk2 ,...,p' qki ...,p' qkN ]; p' qki As shown in formula (2):

[0108]

[0109] Step 1.4: Model initialization Use the pre-trained BERT model to take advantage of the pre-trained language model. Use the GRU model to train p' qki Further encoding is performed to obtain the semantic encoding result h qk =[h qk1 ,h qk2 ,…,h qki …,h qkN ],h qki ∈R HR .

[0110] BERT model training is mainly divided into two stages: pre-training stage and fine-tuning stage. The pre-training stage is obtained by training on a large data set based on some pre-training tasks. The present invention uses a pre-training model based on the pre-training task Masked LM (directly using Google's open source Chinese BERT pre-training language model BERT-Base-Chinese), that is, randomly masking a part of the words in the sentence, and then using the context information to predict the masked words at the same time, so that the meaning of the word can be better understood based on the full text. Specifically:

[0111] like Figure 4 As shown in the figure, BERT only predicts the word at the [Mask] position during pre-training, so that context information can be used at the same time. However, in subsequent use, the word [Mask] will not appear in the sentence, which will affect the performance of the model. Therefore, the following strategy is adopted during training: 15% of the words in the sentence are randomly selected for masking. Among the words selected as masks, 80% are actually replaced with [Mask], 10% are not replaced, and the remaining 10% are replaced with a random word.

[0112] For example, in the sentence "my dog ​​is hairy", the word "hairy" is selected for Masking, then:

[0113] With 80% probability, the sentence "my dog ​​is hairy" is converted to the sentence "my dog ​​is [Mask]".

[0114] With a probability of 10%, keep the sentence "my dog ​​is hairy" unchanged.

[0115] With a probability of 10%, the word "hairy" is replaced by another random word, such as "apple". The sentence "my dog ​​is hairy" is converted to the sentence "my dog ​​is apple".

[0116] The fine-tuning stage is used for fine-tuning in some downstream tasks. BERT can be used for fine-tuning in downstream tasks without adjusting the structure. In the present invention, the vector output by the cls layer is used as the semantic encoding vector of the sentence pair.

[0117] The training of the GRU model is as follows:

[0118] like Figure 3 Middle t and r t They represent the update gate and reset gate respectively. The update gate is used to control the extent to which the state information of the previous moment is brought into the current state. The larger the value of the update gate, the more state information of the previous moment is brought in. The reset gate controls how much information of the previous state is written to the current candidate set. The smaller the reset gate, the less information of the previous state is written.

[0119] The forward propagation formula of the GRU network is:

[0120] r t =σ(W r ·[h t-1 , x t ])

[0121] z t =σ(W z ·[h t-1 , x t ])

[0122]

[0123]

[0124] y t =σ(W o ·h t )

[0125] Where [] indicates that two vectors are connected, and * indicates the product of matrices.

[0126] From the formula in the forward propagation process, we can see that the parameters to be learned are W r , W z , W h , W o The first three parameters are concatenated (because the previous and next vectors are also concatenated), so they need to be separated during training:

[0127] W r =W rx +W rh

[0128] W z =W zx +Wzh

[0129]

[0130] Input to the output layer:

[0131]

[0132] Output of the output layer:

[0133]

[0134] After getting the final output, we can write the loss of network transmission. The loss of a single sample at a certain moment is:

[0135]

[0136] Then the loss of a single sample at all times is:

[0137]

[0138] The backward error propagation algorithm is used to learn the network, so we first need to find the partial derivatives of the loss function with respect to each parameter (there are 7 in total):

[0139]

[0140]

[0141]

[0142]

[0143]

[0144]

[0145]

[0146] Among them, the intermediate parameters are:

[0147]

[0148]

[0149]

[0150] δ t =δ h，t ·z t ·φ′

[0151]

[0152] After calculating the partial derivatives of each parameter, the parameters can be updated and iterated until the loss converges.

[0153] The specific implementation process of step 2 is as follows:

[0154] In the LDA topic model (LDA model), Figure 5 As shown, the process of generating documents is divided into three layers: document generation->topic generation->word generation: specifically, it includes the following:

[0155] ① Select a document d according to the prior probability;

[0156] ② Based on the prior knowledge α, sample from the Dirichlet distribution (i.e., Dirichlet distribution) to generate the multinomial distribution θ of the topic of the document;

[0157] ③ Sampling from the multinomial distribution θ of the topic generates the topic z of the jth word in the document;

[0158] ④ According to the prior knowledge β, sample from the Dirichlet distribution (i.e., Dirichlet distribution) to generate the multinomial distribution φ of the words corresponding to the topic z.

[0159] ⑤ Sampling from the multinomial distribution φ of words finally generates word w.

[0160] Step 2.1: The process of using the generated document to infer its hidden topic (distribution) is the inverse process of LDA model document generation. Only word w in document d is observed, and topic z is hidden. Based on a large amount of known document term information, namely word w in document d, the posterior distribution of the topic on each document is calculated, as shown in formula (3):

[0161]

[0162] in, is the topic distribution, yes The Dirichlet distribution of is the prior distribution parameter of the topic distribution, The vector of the number of topics that make up document m, is a certain topic, and the posterior distribution of the topic is solved by the Gibbs Sampling method

[0163] Step 2.2: According to formula (3), we can get paragraph P qi , P kj The posterior distribution of the topics is ZP qi =[ZP qi-1 ,ZP qi-2 ,...,ZP qi-v] and ZP kj =[ZP kj-1 ,Z kj-2 ,...,ZP kj-v ],

[0164] The topic probability distribution interaction matrix T of the query document paragraph set and the candidate document paragraph set is calculated qik , the query document paragraph set and the candidate document paragraph set refer to the query document q=(p q1 ,p q2 ,…,p qi …,p qN ), any candidate document dk=(p k1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d, respectively. k The total number of paragraphs; as shown in formula (4):

[0165]

[0166] Among them, v is the number of topics hyperparameter, and each element is T qik express.

[0167] The specific implementation process of step 3 is as follows:

[0168] Step 3.1: Use regular expressions to extract legal elements from the document. Legal elements include crime ccg, criminal behavior co, character type ct, criminal consequence cc, compensation behavior cb, and reconciliation situation r. The present invention mainly focuses on the crime ccg, criminal behavior co, character type ct, criminal consequence cc, compensation behavior cb, and reconciliation situation r in legal cases. These parts contain legal elements that have a decisive impact on the judgment. Among them, character type and compensation behavior are applicable to all types of legal cases, and criminal behavior varies depending on the crime. Although legal judgment documents belong to unstructured text information, the composition and writing order of judgment documents are often based on certain writing specifications. Then, the legal elements contained in each case are synonymously expanded;

[0169] Step 3.2: Calculate the similarity between the two case paragraphs in two parts:

[0170] The similarity calculation formula of the crime or compensation behavior is shown in formula (5):

[0171]

[0172] Among them, entityName1 represents the set of crimes or compensation behaviors extracted from the paragraph;

[0173] The similarity calculation method of criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r is shown in formula (6):

[0174]

[0175] Among them, entityName2 represents the criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r extracted from the paragraphs respectively;

[0176] The similarity between paragraphs based on all legal elements sim_e(E qi ,E kj ) is calculated as shown in formula (7):

[0177]

[0178] Among them, μ1, μ2, μ3 are weight parameters, sim_e(E qi ,E kj ) refers to the similarity between paragraphs based on legal elements, E qi and E kj Represents the entity set of the query paragraph and the candidate paragraph; thus, for the query paragraph set and the candidate paragraph set, a similarity matrix LE is obtained qk =[LE q1k ,LE q2k ,…,LE qik …,LE q1k ],LE qik =sim_e(E qi ,E kj ).

[0179] The specific implementation process of step 4 is as follows:

[0180] The semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and the similarity of the query-candidate pair is calculated. The calculation formula is shown in formula (8):

[0181] E qki =β s h qki +β T T qik +β L LE qik

[0182] =β s [GRU(h qk (i-1),p' qki )]+β T T qik +β L LEqik (19)

[0183] Among them, β s ,β T ,β L are weight parameters respectively;

[0184] For the aggregated encoding output of the query-candidate pair, an attention mechanism is added to further encode the position information. The attention weight calculation method is shown in formula (9):

[0185]

[0186] α qki 、u qk They refer to the weights and parameters in the attention mechanism respectively;

[0187] Among them, u qk The calculation method of is shown in formula (10):

[0188] u qk =W u MaxPool(E qk )+b u (twenty one)

[0189] In formula (10), MaxPool() means selecting the largest value in each row of data, q is the query document, k is a candidate document, E qk is the aggregate code of the query document q and a candidate document k; W u ∈R HR×HR ,b u ∈R HR ;

[0190] The document-level similarity relationship is obtained by aggregation formula (11):

[0191]

[0192] Finally, all d qk After a fully connected layer and a softmax function, the similarity probability values ​​of the two legal case description documents are obtained; the document corresponding to the value with the largest similarity probability value is the required similar case.

[0193] Use softmax for multi-classification, which maps the output of multiple neurons to the interval (0,1). For example, if there are 10 candidates, then there are 10 neuron outputs. Now according to each input d qk , to predict the output probability, the resulting series of probability sequences can tell which document has the greatest similarity. For example, in one output process, the value of the output node is as follows:

[0194] [0.2,0.1,0.05,0.1,0.2,0.02,0.08,0.01,0.01,0.23]

[0195] From this, we can see that the 10th candidate document has the greatest similarity with the query document q, because 0.23 is the highest probability this time.

[0196] The present invention carried out a comparative experiment on the LeCaRD legal case retrieval dataset, and the results proved the effectiveness of the method proposed in the present invention. The results are shown in Table 1, where P@5 and P@10 are accuracy indicators, MAP is the average accuracy indicator, and NDCG@10, NDCG@20, and NDCG@30 are ranking indicators.

[0197] Table 1

[0198]

[0199] Example 3

[0200] A similar case retrieval system based on legal elements, including a semantic coding module, a subject distribution coding module, a legal entity coding module, a coding aggregation and similarity calculation module;

[0201] The semantic encoding module is configured to: first divide the long text into short paragraphs according to the case description order, and then semantically encode the query paragraph and candidate paragraphs through the BERT-based text encoding method;

[0202] The topic distribution encoding module is configured to: obtain the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set respectively based on the inverse process of LDA model document generation;

[0203] The legal entity coding module is configured to: extract legal elements from the document, perform synonym expansion on the legal elements contained in each case, and finally calculate the similarity between two case paragraphs;

[0204] The coding aggregation and similarity calculation module is configured to: encode and aggregate the three parts of semantic coding, topic distribution coding and legal entity coding, calculate the similarity of query-candidate pairs, and select the first few corresponding candidate documents with the largest similarity as similar cases.

[0205] Figure 1This is a schematic diagram of the similar case retrieval system based on legal elements of the present invention. LEGAL FEATURE ENCODING represents legal feature encoding, which includes three parts: BERT-based semantic encoding module, LDA-based topic distribution encoding module (topicalrelationship module) and legal entity encoding module (legalentityrelationship module). ENCODING AGGREGATION corresponds to the encoding aggregation module. ATTENTION is a general representative symbol of the attention mechanism.

[0206] Example 4

[0207] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device for the method of similar case retrieval based on legal elements.

[0208] Example 5

[0209] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the similar case retrieval method based on legal elements.

Claims

1. A similar case retrieval method based on legal elements, characterized in that: The steps include: Step 1: Semantic encoding: first divide the long text into short paragraphs according to the order of case description, and then use the BERT-based text encoding method to semantically encode the query paragraph and candidate paragraphs; Step 2: Topic distribution encoding: This means: based on the inverse process of LDA model document generation, the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set is obtained respectively; Step 3: Legal entity coding: extracting legal elements from the document, performing synonym expansion on the legal elements contained in each case, and finally calculating the similarity between the paragraphs of two cases; Step 4: Code aggregation and similarity calculation: This means: encode and aggregate the semantic code, topic distribution code and legal entity code, calculate the similarity of the query-candidate pair, and select the first few corresponding candidate documents with the largest similarity as similar cases; The specific implementation process of step 4 is as follows: The semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and the similarity of the query-candidate pair is calculated. The calculation formula is shown in formula (8): E qki =b s h qki +b T T qik +b L LE qik =b s [GRU(h qk (i-1),p' qki )]+b T T qik +b L LE qik (8) Among them, β s ,β T ,β L are weight parameters respectively; E qki refers to the similarity of the query-candidate pair, h qki refers to the semantic encoding result, T qik It refers to the topic probability distribution interaction matrix between the query document paragraph set and the candidate document paragraph set. qik =sim _ e(E qi ,E kj ), sim_e(E qi ,E kj ) refers to the similarity between paragraphs based on legal elements, p' qki =MaxPooling(C i1 ,C i2 ,...,C ij ...,C iM ), C ij refers to the semantic aggregation representation of the query-candidate paragraph pair, h qk (i-1) refers to the semantic encoding result; For the aggregated encoding output of the query-candidate pair, an attention mechanism is added to further encode the position information. The attention weight calculation method is shown in formula (9): α qki 、u qk They refer to the weights and parameters in the attention mechanism respectively; Among them, the calculation method of uqk is shown in formula (10): u qk =W u ·MaxPool(E qk )+b u (10) In formula (10), MaxPool() means selecting the largest value in each row of data, q is the query document, k is a candidate document, E qk is the aggregate code of the query document q and a candidate document k; W u ∈R HR×HR ,b u ∈R HR ; The document-level similarity relationship is obtained by aggregation formula (11): Finally, all dqk pass through a fully connected layer and a softmax function to obtain the similarity probability values ​​of the two legal case description documents; the document corresponding to the value with the largest similarity probability value is the required similar case.

2. A similar case retrieval method based on legal elements according to claim 1, characterized in that: The specific implementation process of step 1 is as follows: Step 1.1: Formalize, for a query document q and any candidate document d k They are respectively expressed as q=(p q1 ,p q2 ,…,p qi …,p qN ), dk=(p k1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d respectively. k The total number of paragraphs; for query document q and candidate document d k For each paragraph in the query, construct a query-candidate paragraph pair (p qi ,p kj ), where 1≤i≤N and 1≤j≤M; Step 1.2: Paragraph pair (p qi ,p kj ) is fed into the BERT model along with the retained tokens, namely [CLS] and [SEP]; the final hidden layer vector of the first token [CLS] is output as the query-candidate paragraph pair (p qi ,p kj ), as shown in formula (1): C ij =BERT(p qi ,p kj ) (1) In this way, the interaction matrix of all query-candidate paragraph pairs is obtained. qi ,p kj ) is represented by C ij ,C ij ∈R HB ; Step 1.3: For each paragraph of the query document, use maxpooling to find the strongest matching paragraph of the candidate document, thus obtaining a vector sequence, denoted as p' qk =[p' qk1 ,p' qk2 ,...,p' qki ...,p' qkN ]; p' qki As shown in formula (2): Step 1.4: Use the GRU model to train p' qki Further encoding is performed to obtain the semantic encoding result h qk =[h qk1 ,h qk2 ,…,h qki …,h qkN ],h qki ∈R HR .

3. The similar case retrieval method based on legal elements according to claim 1 is characterized in that: The specific implementation process of step 2 is as follows: Step 2.1: The process of using the generated document to infer its hidden topic is the inverse process of LDA model document generation. Only word w in document d is observed, and topic z is hidden. Based on a large amount of known document-term information, namely word w in document d, the posterior distribution of the topic on each document is calculated, as shown in formula (3): in, is the topic distribution, yes The Dirichlet distribution of is the prior distribution parameter of the topic distribution, The vector of the number of topics that make up document m, is a certain topic, and the posterior distribution of the topic is solved by the Gibbs Sampling method Step 2.2: According to formula (3), we can get paragraph P qi , P kj The posterior distribution of the topics is ZP qi =[ZP qi-1 ,ZP qi-2 ,...,ZP qi-v ] and ZP kj =[ZP kj-1 ,Z kj-2 ,...,ZP kj-v ], The topic probability distribution interaction matrix T of the query document paragraph set and the candidate document paragraph set is calculated qik , the query document paragraph set and the candidate document paragraph set refer to the query document q=(p q1 ,p q2 ,…,p qi …,p qN ), any candidate document dk=(p k1 ,p k2 ,…,p kj …,p kM ), where N and M are the query document q and the candidate document d, respectively. k The total number of paragraphs; as shown in formula (4): Among them, v is the hyperparameter of the number of topics, and each element is represented by Tqik.

4. The similar case retrieval method based on legal elements according to claim 1 is characterized in that: The specific implementation process of step 3 is as follows: Step 3.1: extract legal elements from the document using regular expressions, including crime name ccg, criminal behavior co, character type ct, criminal consequence cc, compensation behavior cb, and reconciliation situation r; and then perform synonym expansion on the legal elements contained in each case; Step 3.2: Calculate the similarity between the two case paragraphs in two parts: The similarity calculation formula of the crime or compensation behavior is shown in formula (5): Among them, entityName1 represents the set of crimes or compensation behaviors extracted from the paragraph; The similarity calculation method of criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r is shown in formula (6): Among them, entityName2 represents the criminal behavior co, character type ct, criminal consequence cc or reconciliation situation r extracted from the paragraphs respectively; Paragraph similarity based on all legal elements _ e(E qi ,E kj ) is calculated as shown in formula (7): Among them, μ1, μ2, μ3 are weight parameters, sim_e(E qi ,E kj ) refers to the similarity between paragraphs based on legal elements, E qi and E kj Represents the entity set of the query paragraph and the candidate paragraph; thus, for the query paragraph set and the candidate paragraph set, a similarity matrix LE is obtained qk =[LE q1k ,LE q2k ,…,LE qik …,LE q1k ],LE qik =sim_e(E qi ,E kj ).

5. A similar case retrieval system based on legal elements, characterized in that: It includes semantic coding module, topic distribution coding module, legal entity coding module, coding aggregation and similarity calculation module; The semantic encoding module is configured to: first divide the long text into short paragraphs according to the case description order, and then semantically encode the query paragraph and the candidate paragraphs through a BERT-based text encoding method; The topic distribution encoding module is configured to: obtain the topic probability distribution interaction matrix of the query document paragraph set and each candidate document paragraph set respectively based on the inverse process of LDA model document generation; The legal entity coding module is configured to: extract legal elements from the document, perform synonym expansion on the legal elements contained in each case, and finally calculate the similarity between two case paragraphs; The coding aggregation and similarity calculation module is configured to: perform coding aggregation on the semantic coding, the topic distribution coding and the legal entity coding, perform similarity calculation on the query-candidate pair, and select the first few corresponding candidate documents with the largest similarity as similar cases; the specific implementation process is as follows: The semantic coding, topic distribution coding and legal entity coding are coded and aggregated, and the similarity of the query-candidate pair is calculated. The calculation formula is shown in formula (8): E qki =b s h qki +b T T qik +b L LE qik =b s [GRU(h qk (i-1),p' qki )]+b T T qik +b L LE qik (8) Among them, β s ,β T ,β L are weight parameters respectively; E qki refers to the similarity of the query-candidate pair, h qki refers to the semantic encoding result, T qik It refers to the topic probability distribution interaction matrix between the query document paragraph set and the candidate document paragraph set. qik =sim_e(E qi ,E kj ), sim_e(E qi ,E kj ) refers to the similarity between paragraphs based on legal elements, p' qki =MaxPooling(C i1 ,C i2 ,...,C ij ...,C iM ), C ij refers to the semantic aggregation representation of the query-candidate paragraph pair, h qk (i-1) refers to the semantic encoding result; For the aggregated encoding output of the query-candidate pair, an attention mechanism is added to further encode the position information. The attention weight calculation method is shown in formula (9): α qki 、u qk They refer to the weights and parameters in the attention mechanism respectively; Among them, u qk The calculation method of is shown in formula (10): u qk =W u ·MaxPool(E qk )+b u (10) In formula (10), MaxPool ( ) means selecting the largest value in each row of data, q is the query document, k is a candidate document, E qk is the aggregate code of the query document q and a candidate document k; W u ∈R HR×HR ,b u ∈R HR ; The document-level similarity relationship is obtained by aggregation formula (11): Finally, all d qk After a fully connected layer and a softmax function, the similarity probability values ​​of the two legal case description documents are obtained; the document corresponding to the value with the largest similarity probability value is the required similar case.

6. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the similar case retrieval method based on legal elements as described in any one of claims 1-4.

7. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement each instruction; and the computer-readable storage medium is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the similar case retrieval method based on legal elements as described in any one of claims 1-4.

Images