Case factor identification method integrating label information

Abstract

The present invention relates to a case factor identification method incorporating label information, and belongs to the field of natural language processing. The present invention comprises: pre-processing a text dataset of judgment documents on private lending annotated with CAIL2019-SCM elements; using cleaned judgment documents as input; then using BERT pre-trained on civil judgment documents to encode the input representation; further using a CNN network to extract features of the fact description part; fusing the extracted representation with the attribute and label representation respectively; then identifying the labels under the attribute in turn according to the synthesized representation, and the classifier calculates the final label probability distribution. Experimental results show that the model can more effectively identify the elements on the element annotated data of the CAIL2019-SCM dataset.

Description

Technical Field

[0001] The invention relates to a case factor identification method incorporating tag information, and belongs to the field of natural language processing. Background Art

[0002] The identification of case elements in private lending aims to extract the elements that can reflect the main facts of the case according to the attribute system through the understanding of the judgment documents, and has wide application in the field of smart justice. In addition to some common basic elements such as the time, place, and institutions involved in the case, these elements also include many key elements that describe the course of the case. In the judicial field, case element identification has a wide range of applications. The identified element information can be used for similar case judgments and assist the court in the trial of similar cases. Judicial practitioners can quickly grasp the ins and outs of the case through the case elements. In actual scenarios, there are many other judicial-related businesses that use case element analysis. If the case elements can be automatically identified from the judgment documents, this will greatly improve the office efficiency of judicial practitioners.

[0003] However, the existing factor identification is mainly based on sequence labeling methods, which is not applicable to private lending cases. The present invention converts it into a multi-label text classification task, but data scarcity and label imbalance will make it difficult for the classifier to accurately and completely identify the factors. Therefore, the present invention uses different fusion strategies to fuse the label attributes and factor labels with the coding representation of the judgment documents, and finally uses the classifier to identify the factors of each type of attribute. Summary of the invention

[0004] The present invention provides a case factor identification method that incorporates label information to solve the problem that case factors of private lending cases cannot be identified through sequence labeling and that case factors are difficult to accurately identify when faced with data scarcity and label imbalance.

[0005] Invention technical solution: A case factor identification method incorporating tag information, the specific steps of the method are as follows:

[0006] Step 1: Preprocess the annotated element data of the CAIL2019-SCM dataset, extract the facts and reasons in the judgment documents through the regular expression constructed by ourselves, replace the annotated labels with duplicate names, and divide the dataset according to the proportion;

[0007] Step 2: Use the BERT model pre-trained on civil judgment documents to obtain the representation of judgment documents and element label attributes and specific labels, and use CNN to extract features of the factual description of the judgment documents to obtain the input encoding representation, including element attribute representation, element label representation, and factual description representation of the judgment documents;

[0008] Step 3: The obtained element attribute representation and element label representation are respectively integrated with the fact description representation of the judgment document, and finally the feature vectors of different attention levels of the element label under the attribute are obtained according to the element attribute;

[0009] Step 4: The feature vector is sent to the classifier corresponding to different attributes, and the classifier calculates the probability of each label under each attribute as output.

[0010] The specific steps of Step 1 are:

[0011] Step 1.1, for each judicial document, use regular expressions to extract the main part of the judicial document, specifically the facts and reasons in the judicial document;

[0012] Step 1.2. Replace the duplicate names of label information, modify some labels with consistent expressions, and add the attribute name corresponding to the label. For example, change "other" in the loan delivery form to "other loan forms" to make it more distinguishable.

[0013] Step 1.3, CAIL2019-SCM annotation feature data is divided into 8:1:1 and set as training set, validation set and test set respectively.

[0014] As a preferred solution of the present invention, an encoding module based on BERT and CNN is used to obtain element attribute representation, element label representation, and fact description representation of the judgment document. The specific steps of Step 2 are as follows:

[0015] Step 2.1. Use the "Civil BERT" pre-trained on more than 20 million civil judgments to simultaneously encode the three parts of text: fact description, element category attributes, and element label. The input text is converted into the corresponding id representation through the vocabulary, and the id representation is output after BERT encoding. The process can be abstracted as shown in formula (1-3).

[0016] H X =BERT(X) (1)

[0017] H A =BERT(A) (2)

[0018] H Y =BERT(Y) (3)

[0019] in n represents the length of the fact description text sequence, m represents the total number of category attributes, s represents the total number of labels contained in the label set, and d represents the word vector dimension;

[0020] Step 2.2: For the representation of attributes and labels, the present invention converts each attribute Label A character vector of ti ,y ti After averaging, the encoding representation of the attribute and label is obtained, ti represents the i-th character, la and ly represent the number of characters contained in a certain attribute and a certain label respectively. For detailed calculation, see formula (4-5).

[0021]

[0022]

[0023] In the above formula That is, the encoding representation of category attributes and feature labels. This is the final result after BERT encoding of the category attribute set and label set.

[0024] Step 2.3, the present invention uses a feature extraction network composed of CNN to extract features from the fact description sequence. For the output of the judgment document encoded by BERT The text representation vector is convolved using formula (6).

[0025] C i =f(W·X i:i+h-1 +b) (6)

[0026] The meaning of the above formula is that the text matrix is ​​composed of the vector representation x of n words, and a convolution kernel with a length and width h of the same dimension as the coding layer is convolved with the words in the convolution kernel window from top to bottom. W in the formula is the parameter matrix in the convolution kernel, b is a bias term, and f is a nonlinear function. In this invention, we use the ReLU function instead to obtain the feature c i Under the convolution of window size h, the feature map obtained As shown in formula (7).

[0027] C=[C1,C2,...,C n-h+1 ] (7)

[0028]

[0029] In the pooling layer, the feature map is subjected to the maximum pooling operation, as shown in formula (8). For each type of convolution with a length of h, m convolution kernels are set, and the final pooled feature map obtained by convolution with this type of convolution kernel is an m-dimensional vector. We concatenate the vectors obtained by convolution with convolution kernels of different window sizes to obtain the final feature vector. The entire feature extraction process of the fact description representation matrix can be summarized as formula (9), where the feature vector

[0030] h = CNN_Pooling (H X ) (9)

[0031] As a preferred embodiment of the present invention, the specific steps of Step 3 are as follows:

[0032] Step 3.1, feature recognition based on multi-label text classification, usually directly using text encoding to identify feature labels, it is difficult to fully identify the labels, the present invention uses the encoding representation of category attributes The row vectors of are respectively related to the feature representation vectors of the fact description Fusion, the fusion process is as follows:

[0033] (1) Weighted features: The fact description feature representation and the category attribute representation are added in align. Since the fact description text sequence itself is long and contains a large amount of information, it has the main feature contribution to the classifier, while the length of the category attribute is much smaller. Therefore, a simple fusion method of adding and averaging may not be applicable. The present invention weights the two by formula (10).

[0034] h′=αh+βh a (10)

[0035] The above formula indicates that the two representations are added proportionally with the preset weights α and β on the word vector dimension d.

[0036] (2) Feature transformation: The feature representation is sent to the feedforward neural network for further fusion. In this way, the sparse features are integrated and the representation ability is improved through network parameter learning. The process can be expressed as formula (11).

[0037]

[0038]

[0039] Step 3.2, Step 3.1W in formula (11) l is the parameter matrix, b l is the bias term, and the transformed feature representation is recorded as The matrix is ​​obtained by connecting the feature representations after fusion of all attributes and fact descriptions The representation that integrates the attribute information is weighted with the label set under the corresponding attribute based on the attention mechanism. The whole process is as follows:

[0040] (1) Extracting label sets: The label matrix after BERT encoding contains the representation of all labels. According to the length of the label set corresponding to the attribute, m subset matrices are extracted from the label matrix. For example, attribute a i , and its corresponding label set Total includes l i labels, then l i The label vectors represent the attributes a i The label matrix

[0041] (2) Calculate the attention score: The representation of the attribute information and the label set under the attribute are fused to calculate the attention score respectively. The attention score explicitly shows the degree of attention of the text to different interactive objects. Due to the large number of category attributes, in order to reduce the model complexity and calculation time, the present invention calculates the attention distribution based on the soft attention mechanism (Soft-Attention) by calculating the dot product between the representation of the fused attribute and each label vector. The calculation process is shown in formula (13-14).

[0042]

[0043]

[0044] (3) Weighted label information: Weight the label set based on the attention score obtained in formula (14) to obtain the final integrated vector, as shown in formula (15).

[0045] h oi =∑α ai Y ai (15)

[0046] It represents the output after the label set corresponding to the i-th attribute is fused with the fact description. The integrated vector integrates the information of a specific attribute and gives different attention to the labels under the attribute, which is used as the input of the classifier.

[0047] As a preferred embodiment of the present invention, the specific steps of Step 4 are as follows:

[0048] Step 4.1. The present invention predicts the labels of each category attribute separately for each integrated vector h o A classifier is constructed based on a fully connected network and softmax function. The integrated vector is mapped into a vector with the same length as the label set through the fully connected layer, and then converted into the corresponding probability representation through the softmax function. The whole process is shown in formula (16).

[0049]

[0050] in Indicates that under a certain attribute, the integrated vector h o The probability of belonging to label y, l represents the number of labels contained in the label set corresponding to the attribute, and W s is the parameter matrix of the fully connected layer for label mapping

[0051] Step 4.2: Use cross entropy to calculate the loss of the i-th attribute based on the probability distribution of the label and the true label. The calculation formula is (17).

[0052]

[0053]

[0054] Then the sample (X i ,Y i ) is obtained by summing up and averaging the losses of all attributes, as shown in formula (18), and finally each classifier is iteratively optimized by back-propagating the total sample loss.

[0055] Step 4.3: The entire model training process uses AdamW as the optimizer, with a hidden layer dimension of d=768, a maximum sequence length of 512 characters, a batch size of 6, an initial learning rate of 2e-5, and a fixed step size to reduce the learning rate.

[0056] The beneficial effects of the present invention are:

[0057] (1) The case factor identification method of the present invention incorporates label information, and uses these features to identify case factors based on a model designed based on a multi-label text classification method, thus solving the problem that it is impossible to extract case factors of private lending using sequence annotation.

[0058] (2) The case factor identification method of the present invention incorporates label information and uses BERT pre-trained on civil judgment documents to encode text. This field-pretrained BERT has better semantic sensitivity and uses CNN to extract features from text sequences. In order to alleviate the problem of sparse output features, the maximum pooling strategy is used to screen effective feature values ​​for the text feature sequence, thereby effectively extracting feature information from civil loan judgment documents.

[0059] (3) The case factor identification method that incorporates label information in the present invention integrates the attribute category to which the label belongs and the label set corresponding to the attribute into the text representation, and performs label prediction by attribute in the label prediction layer. In the case of data scarcity, the label information is used to effectively improve the recognition performance. The average F1 value on the validation set and test set of the CAIL2019-SCM element annotation dataset is 0.036 and 0.0469 higher than that of the baseline model BERT, respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the model structure of the case factor identification method incorporating label information proposed in the present invention.

[0061] Figure 2 It is the length distribution of the input data after being processed by the case factor identification method incorporating label information proposed in the present invention. DETAILED DESCRIPTION

[0062] Example 1: Figure 1 As shown, a case factor identification method incorporating tag information, the specific steps of the method are as follows:

[0063] Step 1: Preprocess the annotated element data of the CAIL2019-SCM dataset, extract the facts and reasons in the judgment documents through the regular expression constructed by ourselves, replace the annotated labels with duplicate names, and divide the dataset according to the proportion;

[0064] Step 1.1, for each judicial document, use regular expressions to extract the main part of the judicial document, specifically the facts and reasons in the judicial document;

[0065] Step 1.2. Replace the duplicate names of label information, modify some labels with consistent expressions, and add the attribute name corresponding to the label. For example, change "other" in the loan delivery form to "other loan forms" to make it more distinguishable.

[0066] Step 1.3, CAIL2019-SCM annotation feature data is divided into training set, validation set and test set according to 8:1:1. The length of the text in the data set is mostly concentrated in 550-670 characters, with an average length of 613 characters, the longest is 958 characters, and the shortest is 521 characters. The specific distribution is as follows Figure 2 shown.

[0067] Step 2: Use the BERT model pre-trained on civil judgment documents to obtain the representation of judgment documents and element label attributes and specific labels, and use CNN to extract features of the factual description of the judgment documents to obtain the input encoding representation, including element attribute representation, element label representation, and factual description representation of the judgment documents;

[0068] Step 2.1. Use the "Civil BERT" pre-trained on more than 20 million civil judgments to simultaneously encode the three parts of text: fact description, element category attributes, and element label. The input text is converted into the corresponding id representation through the vocabulary, and the id representation is output after BERT encoding. The process can be abstracted as shown in formula (1-3).

[0069] H X =BERT(X) (1)

[0070] H A =BERT(A) (2)

[0071] H Y =BERT(Y) (3)

[0072] in n represents the length of the fact description text sequence, m represents the total number of category attributes, s represents the total number of labels contained in the label set, and d represents the word vector dimension;

[0073] Step 2.2: For the representation of attributes and labels, the present invention converts each attribute Label A character vector of ti ,y ti After averaging, the encoding representation of the attribute and label is obtained, ti represents the i-th character, la and ly represent the number of characters contained in a certain attribute and a certain label respectively. For detailed calculation, see formula (4-5).

[0074]

[0075]

[0076] In the above formula That is, the encoding representation of category attributes and feature labels. This is the final result after BERT encoding of the category attribute set and label set.

[0077] Step 2.3, the present invention uses a feature extraction network composed of CNN to extract features from the fact description sequence. For the output of the judgment document encoded by BERT The text representation vector is convolved using formula (6).

[0078] C i =f(W·X i:i+h-1 +b) (6)

[0079] The meaning of the above formula is that the text matrix is ​​composed of the vector representation x of n words, and a convolution kernel with a length and width h of the same dimension as the coding layer is convolved with the words in the convolution kernel window from top to bottom. W in the formula is the parameter matrix in the convolution kernel, b is a bias term, and f is a nonlinear function. In this invention, we use the ReLU function instead to obtain the feature c i Under the convolution of window size h, the feature map obtained As shown in formula (7).

[0080] C=[C1,C2,...,C n-h+1 ] (7)

[0081]

[0082] In the pooling layer, the feature map is subjected to the maximum pooling operation, as shown in formula (8). For each type of convolution with a length of h, m convolution kernels are set, and the final pooled feature map obtained by convolution with this type of convolution kernel is an m-dimensional vector. We concatenate the vectors obtained by convolution with convolution kernels of different window sizes to obtain the final feature vector. The entire feature extraction process of the fact description representation matrix can be summarized as formula (9), where the feature vector

[0083] h = CNN_Pooling (H X ) (9)

[0084] Step 3: The obtained element attribute representation and element label representation are respectively integrated with the fact description representation of the judgment document, and finally the feature vectors of different attention levels of the element label under the attribute are obtained according to the element attribute;

[0085] Step 3.1, feature recognition based on multi-label text classification, usually directly using text encoding to identify feature labels, it is difficult to fully identify the labels, the present invention uses the encoding representation of category attributes The row vectors of are respectively related to the feature representation vectors of the fact description Fusion, the fusion process is as follows:

[0086] (1) Weighted features: The fact description feature representation and the category attribute representation are added in align. Since the fact description text sequence itself is long and contains a large amount of information, it has the main feature contribution to the classifier, while the length of the category attribute is much smaller. Therefore, a simple fusion method of adding and averaging may not be applicable. The present invention weights the two by formula (10).

[0087] h′=αh+βh a (10)

[0088] The above formula indicates that the two representations are added proportionally with the preset weights α and β on the word vector dimension d.

[0089] (2) Feature transformation: The feature representation is sent to the feedforward neural network for further fusion. In this way, the sparse features are integrated and the representation ability is improved through network parameter learning. The process can be expressed as formula (11).

[0090]

[0091]

[0092] Step 3.2, Step 3.1W in formula (11) l is the parameter matrix, b lis the bias term, and the transformed feature representation is recorded as The matrix is ​​obtained by connecting the feature representations after fusion of all attributes and fact descriptions The representation that integrates the attribute information is weighted with the label set under the corresponding attribute based on the attention mechanism. The whole process is as follows:

[0093] (1) Extracting label sets: The label matrix after BERT encoding contains the representation of all labels. According to the length of the label set corresponding to the attribute, m subset matrices are extracted from the label matrix. For example, attribute a i , and its corresponding label set Total includes l i labels, then l i The label vectors represent the attributes a i The label matrix

[0094] (2) Calculate the attention score: The representation of the attribute information and the label set under the attribute are fused to calculate the attention score respectively. The attention score explicitly shows the degree of attention of the text to different interactive objects. Due to the large number of category attributes, in order to reduce the model complexity and calculation time, the present invention calculates the attention distribution based on the soft attention mechanism (Soft-Attention) by calculating the dot product between the representation of the fused attribute and each label vector. The calculation process is shown in formula (13-14).

[0095]

[0096]

[0097] (3) Weighted label information: Weight the label set based on the attention score obtained in formula (14) to obtain the final integrated vector, as shown in formula (15).

[0098] h oi =∑α ai Y ai (15)

[0099] It represents the output after the label set corresponding to the i-th attribute is fused with the fact description. The integrated vector integrates the information of a specific attribute and gives different attention to the labels under the attribute, which is used as the input of the classifier.

[0100] Step 4: The feature vector is sent to the classifier corresponding to different attributes, and the classifier calculates the probability of each label under each attribute as output.

[0101] Step 4.1. The present invention predicts the labels of each category attribute separately for each integrated vector h oA classifier is constructed based on a fully connected network and softmax function. The integrated vector is mapped into a vector with the same length as the label set through the fully connected layer, and then converted into the corresponding probability representation through the softmax function. The whole process is shown in formula (16).

[0102]

[0103] in Indicates that under a certain attribute, the integrated vector h o The probability of belonging to label y, l represents the number of labels contained in the label set corresponding to the attribute, and W s is the parameter matrix of the fully connected layer for label mapping

[0104] Step 4.2: Use cross entropy to calculate the loss of the i-th attribute based on the probability distribution of the label and the true label. The calculation formula is (17).

[0105]

[0106]

[0107] Then the sample (X i ,Y i ) is obtained by summing up and averaging the losses of all attributes, as shown in formula (18), and finally each classifier is iteratively optimized by back-propagating the total sample loss.

[0108] Step 4.3. The entire model training process uses AdamW as the optimizer, the hidden layer dimension d=768, the maximum supported sequence length is 512 characters, the batch size is set to 6, the initial learning rate is 2e-5, and the learning rate is adjusted by a fixed step size.

[0109] In order to illustrate the effect of the present invention, the present invention is compared with 9 models on the same data set, including a neural network model based on word embedding and a method based on training language model. The experimental results are shown in Table 1:

[0110] Table 1 Comparative experimental results

[0111]

[0112]

[0113] As can be seen from Table 1, the method proposed in the present invention has achieved the best average F1 value on both the validation set and the test set on this dataset, proving the effectiveness of the method proposed in the present invention in identifying the elements of civil loan judgment documents, especially in the Macro-F1 value, which is more improved than other models, and is 0.036 and 0.0469 higher than the baseline model BERT on the validation set and test set, respectively. This shows that the method proposed in the present invention can perform better than other models in the element labels of some small samples. On the whole, except for BERT-LSTM, the method based on pre-trained language model encoding is generally better than other baseline models. This may be because civil loan judgment documents involve a large amount of amounts, dates, etc. Using CNN to capture local features can effectively learn these digital information. In the processing of judgment documents, CNN is better than LSTM. Therefore, it is reasonable for the method proposed in the present invention to use CNN as a feature extractor for the factual description of judgment documents.

[0114] In the first four word embedding-based model experiments, we pre-segmented the judicial documents. Among the four models, DPCNN performed the best, but it still had a gap compared with BERT-CNN based on character input or the method of the present invention. This also shows that BERT's powerful semantic encoding ability is enough to reduce the impact of word segmentation on the semantic information of the text context.

[0115] In order to verify the rationality of the experimental setting of using regular expressions to extract facts and reasons from judicial documents, we also retained the full text content as input and verified it on a model based on some word embeddings. The experimental results were compared with the performance of the model of the present invention on data with the header content removed. The results are shown in Table 2.

[0116] Table 2 Experimental results on the full text

[0117]

[0118] By comparing Table 1 with Table 2, it can be seen that when using the same model, whether the judgment document input retains the basic information of the persons involved in the case has little effect on the recognition result. Except for the FastText model, the improvement of other models is less than 1%. This improvement may be reflected in the recognition of the attributes of the borrower and the repayer, but the method of the present invention can even be ignored compared to this improvement. Therefore, the present invention only selects the facts and reasons as input. On the one hand, it meets the input restrictions of the pre-trained language model, and on the other hand, it uses as little information as possible in the judgment document to achieve a better factor recognition effect.

[0119] Table 3 Ablation experiment results

[0120]

[0121] In order to verify the rationality of the setting of each module in the method proposed in the present invention, Table 3 above gives the experimental results after retaining and removing different modules. It can be seen from Table 3 that the F1 values ​​of the results obtained by removing the three modules on the test set are not much different from each other, but there is a certain gap compared with the results obtained by the complete model after removing any module. The first row in the experimental results shows the recognition performance without fusing the attribute information and label information of the fact and reason encoding representation, which is 0.0198 lower than the performance of the complete model of the present invention. The second row reduces the mean F1 of the test set by 0.0206 after removing the BERT module, and the third row reduces by 0.0127 after removing the feature extraction module. From the above results, it can be seen that the rationality of each module in the method proposed in the present invention exists. Among the three modules, the module that fuses label and attribute information has the greatest improvement on the overall recognition performance. After adding the information fusion module but not using the BERT module or the feature extraction module, it still obtains results better than most baseline models. This improvement is largely due to the fact that the classifier pays different attention to the output of the integrated label information. The label information effectively guides the classifier to identify the elements under each attribute, thereby verifying the effectiveness of integrating label information proposed in the present invention.

[0122] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A case factor identification method incorporating tag information, characterized by: The specific steps of the method are as follows: Step 1: Preprocess the annotated element data of the dataset, extract the facts and reasons in the judgment documents through the constructed regular expression, replace the annotated labels with duplicate names, and divide the dataset according to the proportion; Step 2: Obtain the fact description, element attribute representation, and element label representation of the judgment document through the BERT model pre-trained on civil judgment documents; use CNN to extract features from the fact description of the judgment document; The above two steps in this step are used to obtain the input coding representation, including the element attribute representation, the element label representation, and the feature representation of the factual description of the judgment document; Step 3: The obtained element attribute representation and element label representation are respectively integrated with the fact description representation of the judgment document, and finally the feature vectors of different attention levels of the element label under the attribute are obtained according to the element attribute; Step 4: The feature vector is sent to the classifier corresponding to different attributes, and the classifier calculates the probability of each label under each attribute as output; The specific steps of Step 3 are as follows: Step 3.1: Feature recognition based on multi-label text classification usually directly uses text encoding to identify feature labels, which makes it difficult to fully identify the labels. The row vectors of are respectively related to the feature representation vectors of the fact description Fusion, the fusion process is as follows: (1) Weighted features: The fact description feature representation and the category attribute representation are added together, and the two are weighted using formula (10); h′=αh+βh a (10) The above formula indicates that the two representations are added proportionally with the preset weights α and β on the word vector dimension d; (2) Feature transformation: The feature representation is sent to the feedforward neural network for further fusion. In this way, the sparse features are integrated and the representation ability is improved through network parameter learning. The process is expressed as formula (11); Step 3.2, Step 3.1W in formula (11) l is the parameter matrix, b l is the bias term, and the transformed feature representation is recorded as The matrix is ​​obtained by connecting the feature representations after fusion of all attributes and fact descriptions The representation that integrates the attribute information is weighted with the label set under the corresponding attribute based on the attention mechanism. The whole process is as follows: (1) Extracting label sets: The label matrix after BERT encoding contains the representation of all labels. According to the length of the label set corresponding to the attribute, m subset matrices are extracted from the label matrix. i , and its corresponding label set Total includes l i labels, then l i The label vectors represent the attributes a i The label matrix (2) Calculate the attention score: The representation of the attribute information and the label set under the attribute are fused to calculate the attention score respectively. The attention score explicitly shows the degree of attention of the text to different interactive objects. The attention distribution is calculated by calculating the dot product between the representation of the fused attribute and each label vector based on the soft attention mechanism. The calculation process is shown in formulas (13)-(14); (3) Weighted label information: Weight the label set based on the attention score obtained in formula (14) to obtain the final integrated vector, as shown in formula (15); h oi =∑α ai Y ai (15) It represents the output after the label set corresponding to the i-th attribute is fused with the fact description. The integrated vector integrates the information of a specific attribute and gives different attention to the labels under the attribute, which is used as the input of the classifier.

2. The case factor identification method integrated with tag information according to claim 1, characterized in that: The specific steps of Step 1 are: Step 1.1, for each judicial document, use regular expressions to extract the main part of the judicial document, specifically the facts and reasons in the judicial document; Step 1.2, replace the duplicate tag information, modify the tags with the same expression, and add the attribute name corresponding to the tag; Step 1.3, CAIL2019-SCM annotation feature data is divided into 8:1:1 and set as training set, validation set and test set respectively.

3. The case factor identification method integrated with tag information according to claim 1, characterized in that: The encoding module based on BERT and CNN is used to obtain the element attribute representation, element label representation, and fact description representation of the judgment document. The specific steps of Step 2 are as follows: Step 2.

1. Use the "Civil BERT" pre-trained with civil judgment documents to encode the three parts of text: fact description, element category attribute, and element label. The input text is converted into the corresponding id representation through the vocabulary, and the id representation is output after being encoded by BERT. The process is abstracted as shown in formulas (1)-(3); H X =BERT(X)(1) H A =BERT(A)(2) H Y =BERT(Y)(3)where n represents the length of the fact description text sequence, m represents the total number of category attributes, s represents the total number of labels contained in the label set, and d represents the word vector dimension; Step 2.2: For the representation of attributes and labels, each attribute Label A character vector of ti ,y ti After averaging, the encoding representation of the attribute and label is obtained, where ti represents the i-th character, la and ly represent the number of characters contained in a certain attribute and a certain label respectively. The detailed calculation is shown in formulas (4)-(5); In the above formula That is, the encoding representation of category attributes and feature labels. That is the final result after BERT encoding of the category attribute set and label set; Step 2.3: Use the feature extraction network composed of CNN to extract features from the fact description sequence. For the output of the judgment document encoded by BERT The text representation vector is convolved using formula (6); C i =f(W·X i:i+h-1 +b)(6)The meaning of the above formula is that the text matrix is ​​composed of the vector representation x of n words, and a convolution kernel with a length and width of the encoding layer is convolved with the words in the convolution kernel window from top to bottom. W in the formula is the parameter matrix in the convolution kernel, b is a bias term, and f is a nonlinear function. ReLU function is used instead to obtain feature C i ; Under the convolution of window size h, the feature map obtained As shown in formula (7); C=[C1,C2,...,C n-h+1 ](7) In the pooling layer, the feature map is subjected to the maximum pooling operation, as shown in formula (8); each convolution of length h is set to m, and the final pooled feature map obtained by the convolution of the convolution kernel is an m-dimensional vector; the vectors obtained by convolution of convolution kernels of different window sizes are concatenated to obtain the final feature vector. The entire feature extraction process of the fact description representation matrix is ​​summarized as formula (9), where the feature vector h=CNN_Pooling(H X )(9)。 4. The case factor identification method integrated with tag information according to claim 1, characterized in that: The specific steps of Step 4 are as follows: Step 4.1: Predict the labels of each category attribute separately, and for each integrated vector h o The classifier is constructed based on the fully connected network and the softmax function. The integrated vector is mapped into a vector with the same length as the label set through the fully connected layer, and then converted into the corresponding probability representation through the softmax function. The whole process is shown in formula (16); in Indicates that under a certain attribute, the integrated vector h o The probability of belonging to label y, l represents the number of labels contained in the label set corresponding to the attribute, and W s It is the parameter matrix of the fully connected layer for label mapping; Step 4.2, use cross entropy to calculate the loss of the i-th attribute based on the probability distribution of the label and the true label, and the calculation formula is (17); Then the sample (X i ,Y i ) is obtained by summing and averaging the losses of all attributes, as shown in formula (18). Finally, each classifier is iteratively optimized by back-propagating the total sample loss. Step 4.

3. The entire model training process uses AdamW as the optimizer, the hidden layer dimension d=768, the maximum supported sequence length is 512 characters, the batch size is set to 6, the initial learning rate is 2e-5, and the learning rate is adjusted by a fixed step size.

Images