A method for extracting elements from legal documents by incorporating prototype information

Abstract

The present application relates to a method for extracting elements of legal documents incorporating prototype information, the method comprising: obtaining a legal document dataset, the legal document dataset comprising legal documents and a set of element types; performing element masking processing on each sentence in the legal document based on the set of element types to obtain a set of masked sentences; inputting the set of masked sentences into an encoder to obtain an encoded representation of each sentence; calculating the prototype representation of the element type in each sentence based on the encoded representation of each sentence; collecting judicial definitions of each element type; performing comparative learning on the encoder based on the prototype representation of the element type in each sentence and the judicial definition of each element type; inputting the legal document to be extracted into the encoder after comparative learning to obtain an embedded vector representation; inputting the embedded vector representation into a decoder to obtain the result of extracting elements of the legal document. The method improves the ability to perform legal document element extraction tasks in small sample scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of legal document element extraction, and in particular to a legal document element extraction method that incorporates prototype information. Background Art

[0002] In the task of extracting elements from legal documents, the main challenges faced by existing technologies come from two aspects: the accuracy of the element extraction results and the dependence on high-quality annotated data.

[0003] First, the existing legal document element extraction technology is more likely to make mistakes in element type and boundary prediction, and it is often difficult to accurately understand and process complex and changeable judicial contexts, especially when dealing with elements with fine text description granularity and long length. The extraction effect is relatively poor. Secondly, even if the existing legal document element extraction methods can achieve high extraction accuracy, these methods usually require a large amount of annotated data for training to achieve better results. However, large-scale annotated data related to the judicial field is extremely scarce, and the amount of data that can be obtained is limited. At the same time, the cost of manual data annotation is also relatively expensive.

[0004] Therefore, how to construct an effective factor extraction scheme for legal documents in a small sample scenario and obtain high-precision factor extraction results has become a problem that needs to be solved urgently. Summary of the invention

[0005] Based on this, it is necessary to provide a method for extracting elements of legal documents that incorporate prototype information. The method includes:

[0006] S1: Obtain a legal document dataset, where the legal document dataset includes legal documents and a set of element types;

[0007] S2: performing element masking processing on each sentence in the legal document based on the element type set to obtain a masked sentence set; inputting the masked sentence set into an encoder to obtain an encoded representation of each sentence; and calculating a prototype representation of an element type in each sentence based on the encoded representation of each sentence;

[0008] S3: Collect judicial definitions of each element type;

[0009] S4: performing comparative learning on the encoder based on the prototype representation of the element type in each sentence and the judicial definition of each element type;

[0010] S5: Input the legal document to be extracted into the encoder after contrastive learning to obtain an embedded vector representation;

[0011] S6: Input the embedding vector representation into the decoder to obtain the legal document element extraction result.

[0012] Beneficial effects: This method constructs corresponding prototype representations for the element types in each sentence, and performs comparative learning on the encoder based on the constructed prototype representations of the element types and the judicial definitions of the element types. This method can learn the common feature information and semantic information of the element types, thereby improving the ability to perform legal document element extraction tasks in small sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 This is a flow chart of a method for extracting elements of legal documents incorporating prototype information according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0016] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0017] like Figure 1 As shown, this embodiment provides a method for extracting elements of legal documents incorporating prototype information, the method comprising:

[0018] S1: Obtain a legal document dataset, where the legal document dataset includes legal documents and a set of element types.

[0019] S2: Perform element masking on each sentence in the legal document based on the element type set to obtain a masked sentence set; input the masked sentence set into an encoder to obtain an encoded representation of each sentence; and calculate a prototype representation of the element type in each sentence based on the encoded representation of each sentence.

[0020] Specifically, the element shielding processing includes:

[0021] S2.1: Traverse each element type in the element type set in turn, and for the i Feature Type , the legal document i Sentences All elements that do not belong to the feature type The elements are masked with "[MASK]" to obtain the masked i Sentences ;

[0022] S2.2: Repeat S2.1 until all sentences in the legal document are processed to obtain the masked sentence set.

[0023] Furthermore, the process of obtaining the prototype representation of the element type in each sentence includes:

[0024] S2.3: Sentence-based encoding representation and element types The initial position and the end position of the elements in the legal document are used to calculate the vector representation of the elements in the sentence; the calculation formula is:

[0025] ;

[0026] ;

[0027] in, Sentences after masking Middle j A vector representation of the elements; m Indicates j The initial position of each element in the legal document; n Indicates j The final position of each element in the legal document; Indicates the legal document k The first j The encoded representation of a sentence with elements; represents the encoded representation of the sentence; Encoder(·) represents the encoder; Represents encoder parameters;

[0028] S2.4: Masked Sentences The vector representation of all elements in is averaged to obtain the first j The feature type corresponding to the feature The transition prototype is represented by; the calculation formula is:

[0029] ;

[0030] in, Indicates feature type The transition prototype representation of t Sentences after masking The feature type The number of elements;

[0031] S2.5: Repeat S2.3-S2.4 until all element types in the sentence are calculated. The transition prototype representation of

[0032] S2.6: For all element types in the sentence The transition prototype represents the average and obtains the element type The prototype representation of ; the calculation formula is:

[0033] ;

[0034] in, Indicates i Feature Type The prototype representation of Z Indicates that the mask does not belong to the feature type The number of sentences with the elements; Indicates z Mask does not belong to feature type The element type in the sentence The transition prototype representation of .

[0035] In this embodiment, it also includes a process of obtaining prototype representations of all element types;

[0036] Step 1: Based on one of the feature type sets and the feature type Different feature types Performing element masking processing on each sentence in the legal document to obtain a second masked sentence set;

[0037] Step 2: Input the second masked sentence set into the encoder to obtain a second encoded representation of each sentence;

[0038] Step 3: Based on the second encoding representation of each sentence and the element type Execute S2.3-S2.6 for the element to obtain the element type The prototype representation of

[0039] Step 4: Repeat steps 1-3 until the prototype representation of each feature type is calculated.

[0040] In order to enable the model to learn the common features of elements of each element type and improve the model's effect in the task of extracting elements from small sample legal documents; the method provided in this embodiment designs a prototype representation construction strategy to construct a prototype representation for each element type that can characterize the common feature information of elements of this type; in addition, a masking processing strategy is also designed to mask the legal documents. Each masking process obtains a text containing only a single type of element. This masking processing strategy can reduce the influence of other types of elements on the construction of the prototype representation of the element type, so that the constructed prototype representation can better characterize the common feature information of the elements of this element type.

[0041] S3: Collect judicial definitions of each element type.

[0042] In this embodiment, the collecting of judicial definitions of each element type includes: collecting the judicial definitions of each element type from legal and regulatory databases, academic papers, legal literature, judicial judgments, and government and judicial agency announcements.

[0043] S4: performing comparative learning on the encoder based on the prototype representation of the element type in each sentence and the judicial definition of each element type.

[0044] Specifically, the contrastive learning includes:

[0045] S4.1: Input the judicial definition of each element type into the encoder respectively to obtain the judicial definition vector representation of each element type; the calculation formula is:

[0046] ;

[0047] in, Indicates i Feature Type The judicial definition vector of represents ; Mean(·) represents the average operation; Encoder(·) represents the encoder; Indicates i Feature Type judicial definition of Represents encoder parameters;

[0048] S4.2: For each prototype representation H of an element type, the judicial definition vector representation of the same element type as the prototype representation H is used as the positive sample representation of the prototype representation H, otherwise it is used as the negative sample representation of the prototype representation H;

[0049] S4.3: Calculating a contrastive learning loss function based on the positive sample representation and the negative sample representation;

[0050] The calculation formula of contrastive learning loss function is:

[0051] ;

[0052] in, represents the contrastive learning loss function; M Indicates the number of feature types; e is the natural base; sim(·) represents the cosine similarity function; Indicates i Feature Type The prototype representation of Indicates i Feature Type The judicial definition vector representation of ; represents the temperature coefficient; Indicates x Feature Type Vector representation of the judicial definition of .

[0053] After constructing the corresponding prototype representation for each element type, a comparative learning task is designed on the encoder side to optimize the encoding ability of the encoder. Through the comparative learning task, the distance between the prototype representation of the same element type and the judicial definition vector representation in the vector space can be shortened, while the distance between the prototype representations of different element types and the judicial definition vector representation in the vector space can be pushed further away, thereby helping the model learn the common feature representations and semantic information of each element type, optimizing the quality of the vector representation generated by the encoder, and thus improving the model's final effect in performing small-sample legal document element extraction tasks.

[0054] S4.4: Continuously adjust the parameters of the encoder until the contrastive learning loss function converges to obtain the encoder after contrastive learning.

[0055] S5: Input the legal document to be extracted into the encoder after contrastive learning to obtain the embedded vector representation.

[0056] S6: Input the embedding vector representation into the decoder to obtain the legal document element extraction result.

[0057] In this embodiment, the decoder is an autoregressive decoder.

[0058] The decoding process of the decoder includes multiple time steps, the embedded vector representation is input into the decoder to obtain the generated character of the current time step; the calculation formula is:

[0059] ;

[0060] ;

[0061] in, Indicates cThe generated characters of time steps, Decoder(·) represents the decoder; represents the embedding vector representation; Represents the character generated in the previous time step; X Represents the legal document from which elements are to be extracted.

[0062] In this embodiment, a copy mechanism is also used to constrain the decoding process of the decoder, and the constraint process includes:

[0063] Step 1: In c The first time step is to calculate the legal document to be extracted u The attention weight of each character is calculated as follows:

[0064] ;

[0065] in, Indicates the first u The attention weight of each character; Indicates the first u A vector representation of characters; Indicates c The generated characters for time steps; v is the first parameter; is the second parameter; is the third parameter; b is the fourth parameter; T represents the transpose of the matrix;

[0066] Step 2: Perform weighted summation of the attention weights of all characters and the vector representations of their corresponding characters to obtain the context vector; the calculation formula is:

[0067] ;

[0068] in, represents the context vector; U Indicates the number of characters in the legal document to be extracted;

[0069] Step 3: Based on the context vector and the c The generated characters of time steps are calculated c The probability distribution of generated characters in time steps is calculated as follows:

[0070] ;

[0071] ;

[0072] ;

[0073] in, Indicates c The probability distribution of generated characters at time steps; A weight parameter representing the selection between selecting a word from all generated characters or copying a word from the legal document to be extracted, wherein the weight parameter is a training parameter obtained through training learning; represents the probability of a word being generated from all generated characters; Softmax(·) represents softmax Function; Linear(·) means linear function;

[0074] Step 4: Based on c The probability distribution of the generated characters in time steps is calculated to obtain the decoder at the c The loss function for time steps;

[0075] Step 5: Based on the decoder c The decoding loss function is calculated by the loss function of time steps until the decoding loss function converges to complete the decoder constraint;

[0076] The calculation formula of the decoding generation loss function is:

[0077] ;

[0078] ;

[0079] in, represents the decoding generation loss function; G Indicates the length of the extraction result of legal document elements; Indicates that the decoder is c The loss function for time steps; represents a probability distribution; Indicates the extraction result of legal document elements in c The character selected at the time step.

[0080] Further, by selecting the generated character with the maximum probability distribution as output in each time step;

[0081] The generated characters outputted at all time steps are arranged in chronological order to form the legal document element extraction result.

[0082] In this example, a joint loss function is calculated according to the contrastive learning loss function and the decoding generation loss function; the calculation formula is:

[0083] ;

[0084] in, represents the joint loss function, is the weight, represents the contrastive learning loss function; represents the decoding generation loss function;

[0085] The joint loss function is used to jointly optimize the contrastive learning loss function and the decoding generation loss function.

[0086] S7: Evaluate the extraction results of legal document elements based on the distance between the vector representation of the extraction results of legal document elements and the vector representation of the judicial definition of each element type.

[0087] Specifically, the evaluation process includes:

[0088] S7.1: Calculate the vector representation of the legal document element extraction result based on the embedding vector representation and the initial position and the end position of the legal document element extraction result in the legal document to be extracted; the calculation formula is:

[0089] ;

[0090] in, Indicates e The vector representation of the extraction results of legal document elements; q Indicates e The end position of the extraction result of the legal document elements in the legal document to be extracted; p Indicates e The initial position of the legal document element extraction result in the legal document to be extracted; Indicates the first g The first e The embedding vector of the extraction results of legal document elements;

[0091] S7.2: Calculating the Euclidean distance between the vector representation of the extraction result of the legal document elements and the vector representation of the judicial definition of each element type;

[0092] If the calculated Euclidean distance is the smallest, the result is considered credible and the corresponding legal document element extraction result is retained;

[0093] Otherwise, the result is considered unreliable and the corresponding legal document element extraction result is removed.

[0094] In order to solve the problem that it is difficult to obtain large-scale annotated data and the cost of annotating relevant data is high when performing legal document element extraction tasks; the legal document element extraction method that incorporates prototype information provided in this embodiment uses the BART pre-trained model as the basic model, and utilizes the designed prototype representation construction strategy to construct corresponding prototype representations for the element types in each sentence, and performs comparative learning on the encoder based on the constructed prototype representations of the element types and the judicial definitions of the element types, so that the model can learn the common feature information and semantic information of the element types, thereby improving the ability to perform legal document element extraction tasks in small sample scenarios.

[0095] In order to solve the problem that it is difficult to obtain a legal document element extraction model suitable for small sample scenarios and the credibility of the element extraction results is low, the legal document element extraction method that incorporates prototype information provided in this embodiment uses a decoder with a copy mechanism to directly generate the results of element extraction, and uses a joint learning mechanism to obtain a method for legal document element extraction tasks suitable for small sample scenarios. A process for evaluating and filtering the legal document element extraction results is also set up to remove element extraction results with low credibility, thereby improving the effect of element extraction on legal documents in small sample scenarios.

[0096] In order to verify the effect of the method provided in this embodiment in the task of extracting elements from small sample legal documents, a comparative experiment was conducted on the named entity recognition dataset disclosed by the China Legal Intelligence Technology Evaluation. The experimental results are shown in Tables 1 and 2.

[0097] Table 1 is a parameter comparison table when there are 5 and 10 entities in each category in the dataset;

[0098] ;

[0099] Table 2 is a parameter comparison table when there are 15 and 20 entities in each category in the dataset;

[0100] ;

[0101] The implementation principles of the other four named entity recognition methods in Table 1 and Table 2 are as follows:

[0102] The PromptNER method performs entity localization and classification based on a dual-slot prompt template. The model can process multiple prompts at the same time and extract all entities by predicting the slots in each prompt.

[0103] The W2NER method converts the named entity recognition task into a method for predicting the relationship between words in a text. It uses the BERT model as the basic encoder, combined with its designed multi-granularity two-dimensional convolutional layer and common predictor to infer the relationship between words in the text, and then decodes the predicted results to obtain the final named entity recognition results.

[0104] The PUnifiedNER method reconstructs the named entity recognition task into a sequence-to-sequence task. It introduces a prefix language modeling objective to narrow the gap between pre-training and fine-tuning, and can combine multiple datasets for joint training.

[0105] The DiffusionNER method formulates the named entity recognition task as a boundary denoising diffusion process, and then generates named entities from noisy spans. Its boundary denoising diffusion process allows for gradual refinement and dynamic sampling of entities, thus enabling the model to have efficient and flexible entity generation capabilities.

[0106] It can be seen from Tables 1 and 2 that the legal document element extraction method that incorporates prototype information provided by this embodiment is better than the other four named entity recognition methods in element extraction effect in small sample scenarios. The method provided by this embodiment improves the effect of element extraction on legal documents in small sample scenarios.

[0107] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for extracting elements of legal documents incorporating prototype information, characterized in that: include: S1: Obtain a legal document dataset, where the legal document dataset includes legal documents and a set of element types; S2: performing element masking processing on each sentence in the legal document based on the element type set to obtain a set of masked sentences; inputting the set of masked sentences into an encoder to obtain an encoded representation of each sentence; Calculating a prototype representation of the element type in each sentence based on the encoded representation of each sentence; The element shielding process includes: S2.1: Traverse each element type in the element type set in turn, and for the i Feature Type , the legal document i Sentences All elements that do not belong to the feature type The elements are masked with "[MASK]" to obtain the masked i Sentences ; S2.2: Repeat S2.1 until all sentences in the legal document are processed to obtain the masked sentence set; The process of obtaining the prototype representation of the element type in each sentence includes: S2.3: Sentence-based encoding representation and element types The initial position and the end position of the elements in the legal document are used to calculate the vector representation of the elements in the sentence; the calculation formula is: ； ； in, Sentences after masking Middle j A vector representation of the elements; m Indicates j The initial position of each element in the legal document; n Indicates j The final position of each element in the legal document; Indicates the legal document k The first j The encoded representation of a sentence with elements; represents the encoded representation of the sentence; Encoder(·) represents the encoder; Represents encoder parameters; S2.4: Masked Sentences The vector representation of all elements in is averaged to obtain the first j The feature type corresponding to the feature The transition prototype is represented by; the calculation formula is: ； in, Indicates feature type The transition prototype representation of t Sentences after masking The feature type The number of elements; S2.5: Repeat S2.3-S2.4 until all element types in the sentence are calculated. The transition prototype representation of S2.6: For all element types in the sentence The transition prototype represents the average and obtains the element type The prototype representation of ; the calculation formula is: ； in, Indicates i Feature Type The prototype representation of Z Indicates that the mask does not belong to the feature type The number of sentences with the elements; Indicates z Mask does not belong to feature type The element type in the sentence The transition prototype representation of Get the prototype representation process of all feature types: Step 1: Based on one of the feature type sets and the feature type Different feature types Performing element masking processing on each sentence in the legal document to obtain a second masked sentence set; Step 2: Input the second masked sentence set into the encoder to obtain a second encoded representation of each sentence; Step 3: Based on the second encoding representation of each sentence and the element type Execute S2.3-S2.6 for the element to obtain the element type The prototype representation of Step 4: Repeat steps 1-3 until the prototype representation of each feature type is calculated; S3: Collect judicial definitions of each element type; S4: performing comparative learning on the encoder based on the prototype representation of the element type in each sentence and the judicial definition of each element type; S5: Input the legal document to be extracted into the encoder after contrastive learning to obtain an embedded vector representation; S6: Input the embedding vector representation into the decoder to obtain the legal document element extraction result.

2. The method for extracting elements of legal documents incorporating prototype information according to claim 1 is characterized in that: In S3, the collection of judicial definitions of each element type includes: collecting the judicial definitions of each element type from legal and regulatory databases, academic papers, legal literature, judicial judgments, and government and judicial agency announcements.

3. The method for extracting elements of legal documents incorporating prototype information according to claim 1 is characterized in that: In S4, the contrastive learning includes: S4.1: Input the judicial definition of each element type into the encoder respectively to obtain the judicial definition vector representation of each element type; the calculation formula is: ； in, Indicates i Feature Type The judicial definition vector of represents ; Mean(·) represents the average operation; Encoder(·) represents the encoder; Indicates i Feature Type judicial definition of Represents encoder parameters; S4.2: For each prototype representation H of an element type, the judicial definition vector representation of the same element type as the prototype representation H is used as the positive sample representation of the prototype representation H, otherwise it is used as the negative sample representation of the prototype representation H; S4.3: Calculating a contrastive learning loss function based on the positive sample representation and the negative sample representation; S4.4: Continuously adjust the encoder parameters until the contrastive learning loss function converges to obtain the encoder after contrastive learning.

4. The method for extracting elements of legal documents incorporating prototype information according to claim 2 is characterized in that: The calculation formula of contrastive learning loss function is: ； in, represents the contrastive learning loss function; M Indicates the number of feature types; e is the natural base; sim(·) represents the cosine similarity function; Indicates i Feature Type The prototype representation of Indicates i Feature Type The judicial definition vector representation of ; represents the temperature coefficient; Indicates x Feature Type Vector representation of the judicial definition of .

5. The method for extracting elements of legal documents incorporating prototype information according to claim 1 is characterized in that: The decoding process of the decoder includes multiple time steps, the embedding vector representation is input into the decoder to obtain the generated character of the current time step; The calculation formula is: ； ； in, Indicates c The generated characters of time steps, Decoder(·) represents the decoder; represents the embedding vector representation; Represents the character generated in the previous time step; X Represents the legal document from which elements are to be extracted.

6. The method for extracting elements of legal documents incorporating prototype information according to claim 5 is characterized in that: In S6, the copy mechanism is also used to constrain the decoding process of the decoder, and the constraint process includes: Step 1: In c The first time step is to calculate the legal document to be extracted u The attention weight of each character; the calculation formula is: ； in, Indicates the first u The attention weight of each character; Indicates the first u A vector representation of characters; Indicates c The generated characters for time steps; v is the first parameter; is the second parameter; is the third parameter; b is the fourth parameter; T represents the transpose of the matrix; Step 2: Perform weighted summation of the attention weights of all characters and the vector representations of their corresponding characters to obtain the context vector; the calculation formula is: ； in, represents the context vector; U Indicates the number of characters in the legal document to be extracted; Step 3: Based on the context vector and the c The generated characters of the time step are calculated c The probability distribution of generated characters in time steps is calculated as follows: ； ； ； in, Indicates c The probability distribution of generated characters at time steps; A weight parameter representing the selection between selecting a word from all generated characters or copying a word from the legal document to be extracted, wherein the weight parameter is a training parameter obtained through training learning; represents the probability of a word being generated from all generated characters; Softmax(·) represents softmax Function; Linear(·) means linear function; Step 4: Based on c The probability distribution of the generated characters in time steps is calculated to obtain the decoder at the c The loss function for time steps; Step 5: Based on the decoder c The decoding loss function is calculated by the loss function of time steps until the decoding loss function converges to complete the decoder constraint; The calculation formula of the decoding generation loss function is: ； ； in, represents the decoding generation loss function; G Indicates the length of the extraction result of legal document elements; Indicates that the decoder is c The loss function for time steps; represents a probability distribution; Indicates the extraction result of legal document elements in c The character selected at the time step.

7. The method for extracting elements of legal documents incorporating prototype information according to claim 6 is characterized in that: In S6, the process of obtaining the legal document element extraction result includes: At each time step, the generated character with the maximum probability distribution is selected as output; The generated characters outputted at all time steps are arranged in chronological order to form the legal document element extraction result.

Images