Criminal judicial case triple extraction model and prediction method

Abstract

The invention discloses a triad extraction model and prediction method for criminal judicial cases, belongs to the field of natural language processing, is used for law article prediction and criminal name prediction tasks in law decision prediction, and is characterized in that a data set is constructed; frequency labels are set in the data set according to the frequency of applicable law articles and / or judgment of criminal names, and data enhancement and external knowledge introduction are carried out on the data of the low-frequency labels; obtaining similar samples of the current sample; the data with the same law article and / or the judged criminal name as the current sample in the data set is a similar sample of the current sample; obtaining dissimilar samples of the current sample in the data set; the data set does not have the same law article and / or criminal name with the current sample, and the data without the low-frequency label is a dissimilar sample of the current sample; and combining the current sample, any one of the similar samples and one of the dissimilar samples into a case triple of the current sample.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and relates to a triple extraction method, a text similarity matching method, and a task prediction method and model in the field of judicial documents. Background Art

[0002] A judgment is a document written by a court based on a judgment, including civil judgments, criminal judgments, etc. There are corresponding format requirements for writing criminal judgments in my country, and the main body includes

[0003] ① Basic information of the defendant, including the defendant's name, gender, age, place of origin, address, position, whether he has been previously punished for a crime, whether he has been arrested, dates of detention, etc.;

[0004] ② The situation of the defender and the prosecutor;

[0005] ③ The facts and reasons determined in the judgment and the applicable legal basis;

[0006] ④ The judgment result, the appeal period and the court of appeal.

[0007] Public criminal verdicts are important references for judicial decisions and legal consultations, and provide examples and guidance for the trial of similar cases. If natural language technology can be used to extract case features in the verdict, similar cases can be clustered, providing more convenient similar case push services for case handlers, and enabling computers to autonomously predict the verdict of the case, thereby providing a reference for the trial of the case.

[0008] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0009] In the existing judicial case processing, many methods adopt pipeline processing. This serial processing method will lead to error accumulation. Once an error occurs in the previous step, the accuracy of the subsequent steps will be affected;

[0010] In natural language processing tasks in the judicial field, a large amount of manually annotated data is required to train the model, which is not only time-consuming and labor-intensive, but also lacks relevant standard data sets for special texts such as electronic file transcripts.

[0011] The existence of the above-mentioned problems has resulted in high labor costs in the handling and classification of judicial cases, and the results are not very ideal. Summary of the invention

[0012] In order to improve the accuracy of predicting applicable laws and charges in criminal trials according to the case, in the first aspect, according to some embodiments of the present application, a method for extracting triples of criminal justice cases includes:

[0013] Constructing a data set; in the data set, frequency labels are set according to the frequency of applicable laws and / or sentenced crimes, and data enhancement and external knowledge introduction are performed on data with low-frequency labels;

[0014] Obtain similar samples to the current sample; the data in the data set that have the same legal provisions and / or convictions as the current sample are similar samples to the current sample;

[0015] Obtain samples that are dissimilar to the current sample in the dataset; data in the dataset that do not have the same legal provisions and / or crimes as the current sample and do not have low-frequency labels are dissimilar samples to the current sample;

[0016] The current sample, any one of the similar samples, and one of the dissimilar samples are combined into the case triplet of the current sample.

[0017] According to the criminal justice case triple extraction method of some embodiments of the present application, the current sample, the sample with the highest similarity to the current sample among similar samples, and one of the dissimilar samples are combined into a case triplet of the current sample.

[0018] According to the criminal justice case triple extraction method of some embodiments of the present application, constructing a data set includes the following steps:

[0019] Obtain the case description, applicable laws, and convictions in the judgment document as a single piece of data in the data set;

[0020] Classify the data into low-frequency samples and high-frequency samples according to the frequency of applicable laws and / or convictions in a single piece of data, and set low-frequency labels and high-frequency labels accordingly;

[0021] Using a large language model to generate extended samples with different descriptions but similar semantics from the case descriptions corresponding to the low-frequency samples for data enhancement;

[0022] Using a large language model to construct external keywords for crimes of low-frequency samples, so as to introduce the external knowledge;

[0023] The data set is obtained by concatenating the expanded samples of the low-frequency samples with the external keywords.

[0024] According to the criminal justice case triple extraction method of some embodiments of the present application, the external keywords of the low-frequency sample crimes include crime elements, responsible attributes, and characteristic words.

[0025] According to the criminal justice case triple extraction method of some embodiments of the present application, obtaining similar samples of the current sample in the data set includes: extracting data with the same applicable law as the current sample from each piece of data in the data set to construct a first similar case set, wherein the first similar case set is constructed as a similar sample set; or

[0026] Wherein, obtaining similar samples of the current sample in the data set includes: extracting data with the same crime as the current sample from each piece of data in the data set to construct a second similar case set, wherein the second similar case set is constructed as a similar sample set; or

[0027] Among them, obtaining similar samples of the current sample in the data set includes: extracting data with the same applicable law as the current sample from each data in the data set to construct a first similar case set; extracting data with the same crime as the current sample from each data in the data set to construct a second similar case set; the first similar case set and the second similar case set are constructed as a similar sample set.

[0028] According to the criminal justice case triple extraction method of some embodiments of the present application, obtaining dissimilar samples of the current sample in the data set includes: extracting data with different applicable laws and different crimes from the current sample and without low-frequency labels from each data in the data set, and constructing a first dissimilar case set, wherein the first dissimilar case set is constructed as a dissimilar sample set.

[0029] According to the criminal justice case triple extraction method of some embodiments of the present application, setting frequency labels according to the frequency of applicable laws and / or sentenced crimes includes separately counting the applicable laws and / or crimes in each piece of data, calculating the frequency of occurrence of each applicable law and / or the frequency of occurrence of each crime, setting a frequency threshold, and classifying the applicable laws into high-frequency laws and low-frequency laws according to the frequency threshold, and classifying the crimes into high-frequency crimes and low-frequency crimes.

[0030] In a second aspect, a legal judgment prediction method based on case description according to some embodiments of the present application is applied to a prediction model, wherein the prediction model includes a category encoder, a category decoder, and a fully connected layer;

[0031] The decoder encodes the case description of the current sample, similar samples and dissimilar samples of the case triple;

[0032] Calculate the attention vector related to the case description, applicable laws, and charges, and impose constraints on the case triples;

[0033] The category decoder calculates the category label of the current sample, decodes the branch of the corresponding category, and obtains the decoded representation of the current sample;

[0034] The prediction result of the prediction task is calculated through the fully connected layer, and the prediction task includes a legal provision prediction task and / or a crime prediction task.

[0035] According to the legal judgment prediction method based on case description in some embodiments of the present application, the case description encoding representation v of sample f is f ; The applicable law code of sample f represents v fsim ; The applicable crime code of sample f represents v dis ;

[0036] In the step, the attention vector related to the case description, applicable laws, and crime is calculated as shown in the following formula:

[0037] r=W 3 (σ(W 1 v f +(W 2 v f +b 2 )))

[0038]

[0039] In the formula, r represents the self-attention vector of the current sample, r sim represents the attention vector of the current sample and its similar samples, r dis Represents the attention vector of the current sample and its dissimilar samples, W 1 represents the weight matrix used to obtain self-attention, W 2 Represents the weight matrix used to obtain the attention of the relationship between the current sample and similar samples, W 3 represents the weight matrix used to obtain the attention of the relationship between the current sample and similar samples, and b represents the bias vector;

[0040] In the step, relational constraints are imposed on case triples, as shown in the following formula:

[0041]

[0042] In the formula, β l represents the weight parameter related to the law, β c Represents the weight parameter related to the crime, r l Represents the self-attention vector related to the current sample and the law, Represents the attention vector related to the current sample and its similar cases and the law, represents the attention vector related to the current sample and its dissimilar cases and the legal provisions, r c Represents the self-attention vector related to the current sample and the crime, represents the attention vector related to the crime of the current sample and its similar cases, The attention vector representing the current sample and its dissimilar cases related to the crime;

[0043] In the step, the category decoder calculates the category label of the current sample as shown in the following formula:

[0044]

[0045] In the formula, Represents the weight matrix in the activation function relu, represents the weight matrix in the softmax layer, Represents the bias vector in the softmax layer;

[0046] The decoded representation of the current sample is obtained in the step, as shown in the following formula:

[0047]

[0048] represents the initial hidden state in the law prediction task, Represents the initial memory unit in the legal clause prediction task. l The final decoded representation obtained in the representation bar prediction task, c l Represents the memory unit updated by the LSTMCell layer related to the law. represents the initial hidden state in the crime prediction task, represents the initial memory unit in the crime prediction task, W c,l represents the transformation matrix in the crime prediction task, b c,l represents the bias vector in the crime prediction task, h c The final decoded representation obtained in the representation bar prediction task, c c Represents the memory unit obtained by updating the LSTMCell layer related to the crime.

[0049] In the step, the prediction results of the law prediction task and the crime prediction task are calculated through the fully connected layer

[0050]

[0051] represents the predicted value of the law, represents the crime prediction value, represents the total loss of the law prediction and crime prediction tasks, represents the parameter matrix for the law prediction task, represents the bias vector used for the law prediction task, represents the parameter matrix for the crime prediction task, represents the bias vector used for the crime prediction task, y lIndicates the true value of the law, y c Indicates the true value of the crime.

[0052] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the first and second aspects and any possible technical solution of the first aspect.

[0053] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program. When the computer program runs on an electronic device, the electronic device executes the first aspect and any possible technical solution of the first and second aspects thereof.

[0054] Beneficial effects:

[0055] The specific implementation of the model of the present invention includes modules such as data preprocessing, case triple extraction, and category decoding. In the data preprocessing stage, a legal knowledge base is first extracted with the help of a domestic large language model, and the corresponding criminal law knowledge is introduced when encoding the case description text, thereby enhancing the text features. The present invention extracts cases based on similarity, constructs case triples as the input of the model, and introduces an attention module to impose constraints on it. In the prediction task, the present invention uses a category decoder, adopts different decoding methods for cases with different frequencies of occurrence, and optimizes the decoding process.

[0056] The present invention performs two major tasks, namely, law article prediction and crime name prediction, on the CAIL2018 dataset, and conducts ablation experiments on some modules. The experimental results show that: (1) Compared with the existing judgment prediction models, the strategy of the present invention has achieved good results: based on the CAIL-small dataset, the accuracy of the model of the present invention in the law article prediction task reaches 80.06%, and the accuracy of the crime name prediction task reaches 85.46%; the performance on the CAIL-big dataset also exceeds the baseline model (2) Through ablation experiments, it can be seen that introducing external knowledge in the data preprocessing stage is effective for enhancing text features, and the strategy of extracting case triples through similarity calculation is feasible.

[0057] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1This is an example of formatted information extracted after the regularization phase of the judicial decision in the judicial case triple extraction method of this application.

[0059] Figure 2 This is a flowchart of the data processing module in the judicial case triple extraction method of this application.

[0060] Figure 3 This is an example of the text enhancement method in the data enhancement module in the judicial case triple extraction method of this application.

[0061] Figure 4 This is an example of introducing external knowledge methods in the data enhancement module of the judicial case triple extraction method of this application.

[0062] Figure 5 These are examples of similar and dissimilar cases in the judicial case triple extraction method of this application.

[0063] Figure 6 Schematic diagram of the triple selection strategy in the judicial case triple extraction method of this application.

[0064] Figure 7 This is a flowchart of the overall framework of the judicial case triple extraction method in this application. DETAILED DESCRIPTION

[0065] The following describes the embodiments of the present application in detail with reference to the accompanying drawings, examples of which are shown in the accompanying drawings. The present application provides a method and an electronic device. The method and the device are based on the same technical concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0066] Note: The topic model-based method uses word frequency information to infer that if each piece of data in a given data set contains a case description and the corresponding law, crime, and punishment, then a set of laws can be constructed = {y1, y2, ...y t}, the goal of the model is to extract the corresponding y from the legal set given a description i The same is true for the prediction tasks of charges and penalties. In real legal judgments, there are often multiple crimes, multiple legal provisions, and multiple punishments for multiple crimes, which makes the task very complicated. Therefore, this invention only studies the tasks of legal provision prediction and charge prediction, and is limited to the prediction of a single charge / single legal provision.

[0067] In the first embodiment, the judicial case triple extraction method using data enhancement technology of the present invention does not require human intervention, and guides the computer to independently construct a triple set containing similar case features and dissimilar case features, thereby simplifying the processing steps, reducing labor costs while ensuring the case clustering effect, and providing support for downstream tasks such as legal judgment prediction and similar case recommendation.

[0068] The method specifically includes the following aspects:

[0069] Preliminary processing of existing public judicial judgment datasets. Based on the fixed writing format of judgments, the three parts of the judgments, namely, case description, judgment result, and referenced legal provisions, can be extracted through regular methods. The extracted results are stored by article, and a new dataset containing only the required elements can be obtained.

[0070] Based on the processed data set, the number of crimes and applicable laws that appeared in it was counted. According to the statistical results, the classification threshold was manually set to divide the crimes that appeared into high-frequency crimes and low-frequency crimes, and similarly, the laws were classified into high-frequency laws and low-frequency laws.

[0071] According to the classification results, data enhancement is performed on low-frequency samples, specifically including two steps: text enhancement and introduction of external knowledge. The text enhancement method involved in the present invention includes sentence reorganization and text generation. In order to take into account data security and ease of operation, the present invention implements the above method on a domestic large language model. By designing corresponding prompt statements, the large model can be guided to generate extended samples with different descriptions but similar semantics based on the existing low-frequency samples as supplementary corpus for the low-frequency samples. In the step of introducing external knowledge, the present invention also builds an external knowledge base for the crime class where the low-frequency samples are located based on the powerful knowledge combing ability of the large model. The knowledge base will contain relevant information about this crime: elements of crime, responsible attributes, characteristic words, etc. In the data preprocessing stage, after screening out the low-frequency samples, the relevant information in the knowledge base will be called according to the crime category to which they belong. The acquired crime feature information will be added to the case description text in the form of phrases to participate in the subsequent process together.

[0072] After the data set is enhanced, the problem of uneven sample distribution will be improved. Then it will enter the judicial case triple extraction module. The core of judicial case triple extraction is the idea of ​​comparing cases, and the frequency information of the label also needs to be considered during extraction. The present invention first uses a threshold to divide the legal text labels into two groups: a low-frequency legal text group and a high-frequency legal text group, and divides the crime labels into two groups: a low-frequency crime group and a high-frequency crime group.

[0073] In the judicial case triple extraction module, the main basis for extracting similar cases is the similarity of the text. Specifically, for a single piece of data in the form of (case description, judgment result, reference law), natural language processing technology is first used to encode the case description part, so that the text is converted into vectorized form for storage. For any case, the process of extracting similar cases can be divided into the following steps: (1) screening out a set of candidate similar cases with the same judgment result (or reference law); (2) calculating the vector similarity between the current case and all cases in the candidate set; (3) screening out the one with the highest similarity as the constituent element of the triple.

[0074] For the current case, the present invention will extract similar case 1 from the candidate case set with the same legal provision label, and extract similar case 2 from the candidate case set with the same crime label; and then extract dissimilar cases with different legal provision and crime labels from the candidate case set (the crime label is required not to belong to the low-frequency crime group).

[0075] The ultimate goal of the judicial case triple extraction module is to construct a triple of the form (current case, similar case, dissimilar case) for each current case. In practical applications, if cases need to be clustered by legal provisions, similar cases 1 with the same legal provision label can be used to construct triples; if cases need to be clustered by crime features, similar cases 2 with the same crime label can be used to construct triples.

[0076] The extracted triplet set can be applied to many downstream tasks, such as the judgment prediction task: on the basis of the judicial case triplet set, the relational attention module is introduced to calculate the attention vector between similar cases and dissimilar cases, and then the encoded representation is decoded through the decoder to output the corresponding prediction result. The parameters for specific implementation can be obtained by training the algorithm model.

[0077] In the second embodiment, the judicial case triple extraction method includes the following steps:

[0078] Using the judicial case data processing method, the public judicial judgment data set is preliminarily processed, the key parts of the judgment are extracted through the regular expression method, and the extracted results are stored as a new data set according to conditions;

[0079] The judicial document dataset is processed using a data enhancement method, wherein the data enhancement technique includes two steps: text enhancement and introduction of external knowledge. The data enhancement step is used to enrich the features of low-frequency samples.

[0080] A triplet extraction method is used to extract a set of judicial case triples suitable for downstream training. The triplet extraction method is used for the downstream model to simultaneously learn the distribution characteristics of similar / heterogeneous samples. The triplet is composed of similar samples and heterogeneous samples.

[0081] In a preferred embodiment, the elements to be extracted by the judicial case data processing method include the case description, judgment result, and referenced legal provisions in the judgment, and the elements are extracted and matched with common keywords according to the fixed format of the judgment;

[0082] The conditional storage comprises the following steps:

[0083] Based on the processed data set, the number of crimes and applicable laws that appeared in it was counted;

[0084] According to the statistical results, we manually set the classification threshold to classify the crimes that have appeared into high-frequency crimes and low-frequency crimes. Similarly, we classify the laws into high-frequency laws and low-frequency laws.

[0085] The results are stored according to the classification.

[0086] In a preferred embodiment, the data enhancement method performs data enhancement on low-frequency samples according to the classification results, including two steps of text enhancement and introduction of external knowledge.

[0087] Implementation steps for text enhancement:

[0088] Using a large language model, based on low-frequency samples, new samples with the same label are rewritten as augmented samples.

[0089] Implementation steps for introducing external knowledge:

[0090] Use a large language model to supplement the knowledge of low-frequency tags, instruct the large model to summarize corresponding keywords from various aspects, and then add the output keywords to the case description of low-frequency samples to highlight the characteristics of the text.

[0091] In a preferred embodiment, the triple extraction method aims to obtain a set of judicial case triples in the form of (current case, similar cases, heterogeneous cases), where similar cases can be similar cases based on legal article labels or similar cases based on crime labels. The specific implementation includes two steps: judicial case classification and triple selection.

[0092] The judicial case classification module includes the following steps:

[0093] For a current case, suppose the legal article used in this case is a, the set of all legal articles is A, the crime in this case is b, and the set of all crimes is B;

[0094] When extracting similar case 1 based on the legal article label, there is a restriction: the legal article used is a, and the crime is B. Finally, the samples that meet the conditions are summarized as the similar case 1 set;

[0095] When extracting similar case 2 based on the crime label, there are restrictions: the crime is b, and the legal provision used belongs to A. Finally, the samples that meet the conditions are summarized as the set of similar cases 2;

[0096] When extracting dissimilar cases, there are restrictions: the legal provision used belongs to A but not a, the crime sentenced belongs to B but not b, and the legal provision used is not a low-frequency legal provision. Finally, the samples that meet the conditions are summarized as a set of dissimilar cases.

[0097] The triple selection module comprises the following steps:

[0098] For a current sample, after obtaining a set of similar samples, the text similarity between the current sample and each sample in the set can be calculated, and the one with the highest similarity is selected as the best similar case as a component of the judicial case triple;

[0099] In the set of dissimilar cases, a case can be randomly selected as a dissimilar case to construct a judicial case triple. It only needs to meet the conditions that the legal provisions are dissimilar, the charges are dissimilar, and the legal provisions are not low-frequency legal provisions.

[0100] In the third embodiment, since the criminal judgment is written in a certain format, it is required to contain the factual description of the case, the basis for the judgment and the result. To extract the judicial case triple set, the judgment needs to be processed first. Since the writing format of the judgment is relatively fixed, the text can be segmented by regular expressions, and finally the three major parts of the judgment are extracted: the case description, the reference law, and the judgment crime. The processed single data consists of the following: Figure 1 Example shown.

[0101] Based on the processed judgment data set, the data is further processed. The overall process of the data processing module is as follows: Figure 2 As shown. First, the number of times each legal provision and each crime appears in the data set is counted, and the threshold is manually set to divide high-frequency crimes, low-frequency crimes, high-frequency legal provisions, and low-frequency legal provisions. Samples containing low-frequency labels are selected and data enhancement is performed using a large language model.

[0102] The data enhancement process is divided into two parts. The first part is text enhancement, which is to use a large language model to rewrite new samples with the same label based on low-frequency samples as augmented samples. Figure 3 shown.

[0103] The second part of the data enhancement process is to introduce external knowledge, that is, to use a large language model to supplement the knowledge of low-frequency tags. The large model will summarize the corresponding keywords from various aspects, and then add the output keywords to the case description of the low-frequency samples to highlight the characteristics of the text. Figure 4 shown.

[0104] On this basis, data preprocessing is performed, including common data preprocessing steps: word segmentation, stop word removal, and text vectorization, so that the sample data can be converted into a vector form that can be processed by a computer.

[0105] Next, enter the judicial case triple extraction module, the steps are as follows:

[0106] (1) For a current case, suppose the legal provision used in this case is a, the set of all legal provisions is A, the crime sentenced in this case is b, and the set of all crimes is B.

[0107] When similar case 1 is extracted based on the legal article label, there is a restriction: the legal article used is a, and the crime is B. Finally, the samples that meet the conditions are summarized as the similar case 1 set.

[0108] When similar case 2 is extracted based on the crime label, there are restrictions: the crime is b and the legal provision used belongs to A. Finally, the samples that meet the conditions are summarized as a set of similar cases 2.

[0109] When extracting dissimilar cases, there are restrictions: the legal provision used belongs to A but not a, the crime sentenced belongs to B but not b, and the legal provision used is not a low-frequency legal provision. Finally, the samples that meet the conditions are summarized as a set of dissimilar cases.

[0110] Examples include Figure 5 shown.

[0111] (2) Select the best similar sample as the constituent element of the triple. Since the case description part of all samples has been encoded into vector form in the data preprocessing stage, the text similarity between samples can be calculated. The cosine vector similarity is used in the present invention. For a current sample, after obtaining a set of similar samples, the text similarity between the current sample and each sample in the set can be calculated, and the one with the highest similarity is selected as the best similar case as a component of the judicial case triple.

[0112] In the set of dissimilar cases, a case can be randomly selected as a dissimilar case to construct a judicial case triple. It only needs to meet the conditions that the legal provisions are dissimilar, the charges are dissimilar, and the legal provisions are not low-frequency legal provisions.

[0113] Select strategy such as Figure 6As shown. At the end of the present invention, a set of judicial case triplets in the form of (current case, similar cases, heterogeneous cases) will be obtained, where similar cases can be cases with similar laws or cases with similar charges, which can be selected according to downstream tasks. The triplet set contains similar features and heterogeneous features, which are input into the downstream task encoder together. The advantage of this processing method is that the model can be trained to allow similar samples to be better clustered in high-dimensional space while staying away from heterogeneous samples, thereby improving the model's learning performance on features.

[0114] The overall framework of the task prediction model is as follows: Figure 7 In this embodiment, the case triples extracted above are also used for task prediction. In this example, a legal judgment prediction model based on case triples is proposed. The legal judgment prediction task is essentially a text classification task. Figure 1 As shown in the figure, the model is required to extract text features based on the given case description text and determine the crimes and laws involved. Figure 1 As shown in the figure, when given a text describing a theft case, the model should automatically match the corresponding legal provision to Article 264 of the Criminal Law, which involves the crime of theft. For higher-performance models, even the fine and prison term of the punishment should be predicted.

[0115] If each piece of data in a given data set contains a case description and the corresponding law, crime, and penalty, then we can construct a set of laws Y = {y1, y2, ...y t}, the goal of the model is to extract the corresponding y from the legal set given a description i The same is true for the prediction of charges and penalties. In real legal judgments, there are often multiple crimes, multiple laws, and multiple punishments for multiple crimes, which makes the task very complicated.

[0116] Therefore, the present invention only studies the tasks of legal provision prediction and crime prediction, and is limited to the prediction of a single crime / single legal provision.

[0117] The model of the present invention includes a data preprocessing module for introducing a knowledge base, a case triple extraction module, and an encoder-decoder module. The overall architecture is as follows Figure 7 As shown. If the given data f=[s;y l ,y c ,y a ], where s = {s1, s2, ...s n} represents the case description text, y l Indicates the label of the law, y c Indicates the crime label, y a∈{0, 1} represents the frequency category label, indicating whether the case is a high-frequency case. In the data preprocessing stage, the low-frequency data is first enhanced using the external knowledge provided by the large model, and then converted into a vector form through mapping; it is converted into case triples through the triple module, and the encoding representation related to the case triples is constrained, and finally the model is trained through the encoder-decoder module. The final objective function to be optimized can be expressed as:

[0118] min L=L M +L R +L C +λ||θ|| (1)

[0119] Among them, L M , L R , L C They represent the prediction loss, constraint loss, and category decoder loss of multi-task learning respectively. λ and ||θ|| are the trade-off parameters and regularization terms.

[0120] (1) Coding

[0121] After constructing the case triples, the present invention will encode the fact description part of each case. The present invention selects Bi-GRU as the encoder. The sentence can be represented as s i =[w i,1 ,w i,2 ,...w i,n ,];

[0122] where w i，j Representative sentence s i The sentence-level Bi-GRU will represent each sentence as follows:

[0123]

[0124] where w i，j Represents the word embedding vector, W w is the weight matrix, b w is the bias vector, u w is a trainable context vector. Apply Bi-GRU to the sentence representation sequence, and finally obtain the encoding representation of the case description of case f Similarly, the encoding representations of the current case f and other cases in the case triple can be obtained in this way, such as the encoding representation of the current case f and and v dis . and They respectively represent the coding representation of cases with similar legal provisions to the current case and the coding representation of cases with similar crimes to the current case.

[0125] (2) Relational Attention Module

[0126] In order to extract effective relation information from case triples to improve the encoding process, the present invention introduces the attention vector between case f and its similar cases, dissimilar cases, and the attention to itself, and the formula is as follows:

[0127]

[0128] Among them, W l 1 , W l 2 , W l 3 and is the weight matrix and the bias vector. The above three formulas can be used to calculate the relational attention matrix related to the legal provisions, the r related to the crime c , The calculation method is the same. σ(·) is the sigmoid activation function. Imposing relational constraints is one of the common methods of supervised contrastive learning. Therefore, the present invention also imposes relational constraints on triples and uses this as the optimization goal:

[0129]

[0130] Among them, β l and β c is the weight parameter. Through the above calculation, the attention vector between cases can be calculated. The higher the value of a certain dimension, the more important it is in the similarity between two cases.

[0131] (3) Class Decoder Module

[0132] In order to avoid the influence between high-frequency and low-frequency cases, the present invention sets a decoder for each. So first, it is necessary to obtain the category label y of the current case a , calculated as follows:

[0133]

[0134] in, and are weight matrix and bias vector respectively. After obtaining the label of the current case, the corresponding category of branches can be selected for decoding. During decoding, the present invention adopts the decoder used in TopJudge, and the specific process is as follows:

[0135]

[0136] in, and is the initial hidden state and memory unit of the legal clause prediction task. c，l and b c，l is the transformation matrix and bias vector in the crime prediction task, h l and c l is the final decoded representation of these two tasks.

[0137] (4) Judgment prediction module

[0138] After obtaining the decoded representation of the current case, the present invention uses a fully connected layer to obtain the prediction results of the legal provision prediction task and the crime prediction task, and calculates the total loss of multi-task prediction.

[0139]

[0140] in, are the parameters of the prediction task.

[0141] Experiment and analysis

[0142] Experimental setup:

[0143] Dataset: The dataset used in this paper is the judicial decision prediction dataset CAIL-small and CAIL-big from the CAIL2018, where a single sample consists of the following content: case description, charges, legal basis, fine, and sentence. This paper mainly studies the two subtasks of law prediction and charge prediction.

[0144] Following the settings of most previous studies, the present invention first screened the data set before the experiment, deleted samples with less than 10 encodable words in the case description, and did not consider the cases under multiple laws or charged with multiple crimes. In order to verify the performance of the model of the present invention on low-frequency samples, the present invention no longer deleted any low-frequency samples, and divided the original training set CAIL-big into a training set and a validation set at a ratio of 9:1. The statistical data of the data set are shown in Table 1.

[0145] Table 1 Dataset information

[0146] data CAIL-small CAIL-big Training samples 105059 1432826 Verification Sample 14266 159372 Test samples 27953 186523 Related laws 177 181 Involved crime 191 193

[0147] Evaluation indicators: In multi-classification tasks, accuracy, macro-precision, macro-recall, and macro-F1 are usually used for measurement. The closer the value is to 1, the better the model effect. The present invention also uses the above four evaluation indicators.

[0148] Experimental environment: The hardware environment used in the experiment: GPU is NVIDIA Ge Force GTX 3090 (24GB); the operating system is Linux version 4.18.0-25-generic (Ubuntu 18.10); the software environment is CUDA11.2; the deep learning framework is TensorFlow 1.14, Keras 2.2.5, and the Python version is 3.7.

[0149] Experiment and analysis:

[0150] The present invention uses Bi-GRU as the encoder and limits the factual description of the case to within 15 sentences.

[0151] In order to verify the effectiveness of the model of the present invention, several representative models in the field of legal judgment prediction were selected for comparative experiments. The selected baseline models include:

[0152] (1) TopJudge, which transforms the legal judgment subtasks in the joint framework into a directed acyclic graph, where the subtasks share parameters;

[0153] (2) LADAN, which develops a graph neural network to learn the differences between legal articles and extract salient discriminative features from factual statements;

[0154] (3) NeurJudge, which uses the results of intermediate subtasks to separate factual statements into different cases and uses them to make predictions for other subtasks.

[0155] (4) MPBFN: This model proposes a multi-view forward and backward prediction framework to enable different subtasks to effectively share parameters.

[0156] (5) Qwen1.5-7B fine-tuning: the dataset is converted into question-answering format for large-scale model fine-tuning, and the performance is compared with this method.

[0157] All experiments are conducted on the CAIL2018 dataset, and the experimental results are shown in Tables 2 and 3.

[0158] Table 2. Experimental results of legal provisions prediction

[0159]

[0160] Table 3 Crime prediction experimental results

[0161]

[0162] From the experimental results, we can analyze:

[0163] (1) After comparative analysis of the data in the table, the TBJ model has achieved good results. Based on the CAIL-small dataset, TBJ achieved an accuracy rate of 80.06% in the legal article prediction task, and the macro precision rate reached 64.91%, which is 1.89% higher than the TopJudge model; the macro recall rate reached 67.21%, which is 0.92% higher than the NeurJudge model. In the crime prediction task, TBJ achieved an accuracy rate of 85.47%, the macro precision rate reached 72.81%, which is 1.59% higher than the Qwen model; the macro recall rate reached 75.28%, which is 6.05% higher than the Qwen model. The experimental results show that on the CAIL-small dataset, the model proposed in this invention can achieve the same or even better results as the traditional SOTA model, and is also quite competitive compared to large models in specific tasks.

[0164] (2) The performance improvement effect of this model on different tasks is also different. As can be seen from the table, compared with the four baseline models of TopJudge, LADAN, NeurJudge, and Qwen, the accuracy of the TBJ model in the legal provision prediction task is improved by about 1% to 2%, but the accuracy of the crime prediction task can be improved by 3% to 5%. The experimental results show that this model has a better improvement effect on the crime prediction task. This result is logical, because the legal provisions in the continental legal system can be regarded as the leaf nodes of the crime, and the prediction difficulty will be greater. Therefore, the prediction accuracy and improvement effect of the crime prediction task are generally higher than that of the legal provision prediction task.

[0165] (3) By comparing the F1 values, we can also find that the MPBFN model has poor prediction performance for low-frequency samples. The F1 values ​​for the law prediction and crime prediction tasks only reach 51.83% and 57.52%, respectively. LADAN and NeurJudge alleviate this problem by refining the encoding process. The method of introducing case triples proposed in this invention can further improve the prediction effect of low-frequency samples.

[0166] (4) Although the large model has outstanding language capabilities, it has not surpassed all traditional models in the experiment. In the legal clause prediction task, Qwen is very prone to confusion between similar categories, but this phenomenon is alleviated in the crime prediction task.

[0167] (5) By comparing the experimental results of each group based on the CAIL-big dataset, it can be found that the selected models have achieved a high accuracy rate: 95% in the legal article prediction task and 93% in the crime prediction task, and the accuracy difference between the groups is not obvious. However, TBJ still achieved the best results in various indicators, with significant improvements in accuracy, recall rate, and F1 value compared with other groups. By comparing the experimental results on the two datasets, it can also be seen that the increase in data volume has greatly improved the prediction performance of the model, and this is particularly obvious when predicting low-frequency samples.

[0168] Ablation experiment:

[0169] In order to verify the impact of the introduction of external knowledge base and similarity algorithm on the overall model, the present invention designed an ablation experiment. Based on the CAIL-small dataset, the performance of the following control groups was tested in the experiment:

[0170] (1) Delete only the external knowledge base module

[0171] (2) Only the similarity calculation module is deleted and the original module is replaced by a random sample.

[0172] (3) Delete the external knowledge base and similarity module at the same time and compare the performance with the complete model. The experimental results are shown in Tables 4 and 5.

[0173] Table 4. Experimental results of law prediction

[0174] Model Acc(%) MP(%) MR(%) MF1(%) w / oknowledge 78.76 64.51 67.33 65.89 w / osimilarity 79.31 64.02 66.32 65.14 w / oboth 78.01 64.22 65.98 65.09 TBJ 80.06 64.91 67.21 66.03

[0175] Table 5 Crime prediction experimental results

[0176] Model Acc(%) MP(%) MR(%) MF1(%) w / oknowledge 84.64 73.21 76.53 74.83 w / osimilarity 85.21 72.30 75.11 73.68 w / oboth 84.13 72.45 75.78 74.08 TBJ 85.47 72.81 75.28 74.02

[0177] The following conclusions can be drawn from the table:

[0178] (1) Introducing external knowledge to enhance samples is an effective strategy. In both tasks, deleting this module reduced the accuracy of model prediction: in the legal article prediction task, the accuracy dropped by 1.3%, and in the crime prediction task, the accuracy dropped by 0.8%, but the impact on the other three indicators was not obvious. In the crime prediction experiment, the control group that deleted external knowledge even achieved the highest MP, MR and MF1. The reason may be that there was a class imbalance in the training set at this time, which caused the model to tend to predict the majority class, thereby achieving high precision and recall on the majority class. The experimental results show that knowledge expansion of low-frequency samples can enable the model to better learn sample characteristics, improve the balance of sample distribution, and thus improve the model prediction accuracy, but this module has limited effect on improving other indicators.

[0179] (2) When extracting case triplets, extracting by similarity value is an effective strategy. In both tasks, deleting this module caused the model performance to decline: in the legal article prediction task, the accuracy rate dropped by 0.75%, and in the crime prediction task, the accuracy rate dropped by 0.26%, and the macro precision, recall rate, and F1 also dropped slightly. The experimental results show that by calculating similarity to extract case triplets, similar cases can be better clustered in a low-dimensional space, thereby improving model performance.

[0180] (3) When both modules are deleted at the same time, the accuracy of the model in the legal article prediction task decreases by 2.05%, and the accuracy of the crime prediction task decreases by 1.34%, which is a significant performance degradation compared to the complete model. However, it is worth noting that although adding these two modules helps improve performance, it also increases the training cost, especially the similarity calculation process will greatly increase the training time.

[0181] For the tasks of legal article prediction and crime prediction in legal judgment prediction, this paper proposes a judgment prediction model based on case triples, and uses a large model to enhance the data of samples, combining the case description of low-frequency samples with criminal law knowledge to enhance text features; it also uses a triple sampling method based on similarity to better cluster similar samples in low-dimensional space; finally, a specific encoding-decoding structure is used for model training and prediction. The results show that the method proposed in this paper is ideal, especially in the prediction of low-frequency samples.

[0182] Based on the above embodiments, the embodiments of the present application further provide a computer program, which, when executed on a computer, enables the computer to execute the methods provided in the above embodiments.

[0183] Based on the above embodiments, the embodiments of the present application further provide a computer storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0184] The storage medium may be any available medium that can be accessed by a computer. For example, but not limited to, a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0185] Based on the above embodiments, an embodiment of the present application further provides a chip, which is used to read a computer program stored in a memory to implement the method provided in the above embodiments.

[0186] Based on the above embodiments, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the method provided in the above embodiments is implemented.

[0187] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0191] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for extracting triples of criminal justice cases, characterized in that: The triple extraction method comprises: Constructing a data set; in the data set, frequency labels are set according to the frequency of applicable laws and / or sentenced crimes, and data enhancement and external knowledge introduction are performed on data with low-frequency labels; Obtain similar samples to the current sample; the data in the data set that have the same legal provisions and / or convictions as the current sample are similar samples to the current sample; Obtain samples that are dissimilar to the current sample in the dataset; data in the dataset that do not have the same legal provisions and / or crimes as the current sample and do not have low-frequency labels are dissimilar samples to the current sample; The current sample, any one of the similar samples, and one of the dissimilar samples are combined into the case triplet of the current sample.

2. The method for extracting triples of criminal justice cases according to claim 1, wherein: The current sample, the sample with the highest similarity to the current sample among similar samples, and one of the dissimilar samples are combined into the case triplet of the current sample.

3. The method for extracting triples of criminal justice cases according to claim 1, wherein: Building a dataset includes the following steps: Obtain the case description, applicable laws, and convictions in the judgment document as a single piece of data in the data set; Classify the data into low-frequency samples and high-frequency samples according to the frequency of applicable laws and / or convictions in a single piece of data, and set low-frequency labels and high-frequency labels accordingly; Using a large language model to generate extended samples with different descriptions but similar semantics from the case descriptions corresponding to the low-frequency samples for data enhancement; Using a large language model to construct external keywords for crimes of low-frequency samples, so as to introduce the external knowledge; The data set is obtained by concatenating the expanded samples of the low-frequency samples with the external keywords.

4. The method for extracting triples of criminal justice cases according to claim 3, wherein: The external keywords of low-frequency sample crimes include elements of crime, attributes of responsibility, and characteristic words.

5. The method for extracting triples of criminal justice cases according to claim 1, Its special feature is that in, Acquiring similar samples of the current sample in the data set includes: extracting data having the same applicable law as the current sample from each piece of data in the data set to construct a first similar case set, wherein the first similar case set is constructed as a similar sample set; or Wherein, obtaining similar samples of the current sample in the data set includes: extracting data with the same crime as the current sample from each piece of data in the data set to construct a second similar case set, wherein the second similar case set is constructed as a similar sample set; or Among them, obtaining similar samples of the current sample in the data set includes: extracting data with the same applicable law as the current sample from each data in the data set to construct a first similar case set; extracting data with the same crime as the current sample from each data in the data set to construct a second similar case set; the first similar case set and the second similar case set are constructed as a similar sample set.

6. The method for extracting triples of criminal justice cases according to claim 1 or 5, wherein: Obtaining dissimilar samples of the current sample in the data set includes: extracting data that has different applicable laws and different crimes from the current sample and does not have a low-frequency label from each data in the data set, and constructing a first dissimilar case set, wherein the first dissimilar case set is constructed as a dissimilar sample set.

7. The method for extracting triples of criminal justice cases according to claim 1, wherein: Setting frequency labels according to the frequency of applicable laws and / or sentenced crimes includes respectively counting the number of applicable laws and / or crimes in each piece of data, calculating the frequency of occurrence of each applicable law and / or crime, setting a frequency threshold, and classifying the applicable laws into high-frequency laws and low-frequency laws, and classifying the crimes into high-frequency crimes and low-frequency crimes according to the frequency threshold.

8. A method for predicting legal decisions based on case description, characterized in that: Applied to a prediction model, the prediction model comprising a category encoder, a category decoder, and a fully connected layer; The decoder encodes the case description of the current sample, similar samples and dissimilar samples of the case triple; Calculate the attention vector related to the case description, applicable laws, and charges, and impose constraints on the case triples; The category decoder calculates the category label of the current sample, decodes the branch of the corresponding category, and obtains the decoded representation of the current sample; The prediction result of the prediction task is calculated through the fully connected layer, and the prediction task includes a legal provision prediction task and / or a crime prediction task.

9. The method for predicting legal decisions based on case description according to claim 8, characterized in that: in, The case description encoding representation v of sample f f ; The applicable law code of sample f represents v fsim ; The applicable crime code of sample f represents v dis ; In the step, the attention vector related to the case description, applicable laws, and crime is calculated as shown in the following formula: r=W 3 (σ(W 1 v f +(W 2 v f +b 2 ))) In the formula, r represents the self-attention vector of the current sample, r sim represents the attention vector of the current sample and its similar samples, r dis Represents the attention vector of the current sample and its dissimilar samples, W 1 represents the weight matrix used to obtain self-attention, W 2 Represents the weight matrix used to obtain the attention of the relationship between the current sample and similar samples, W 3 represents the weight matrix used to obtain the attention of the relationship between the current sample and similar samples, and b represents the bias vector; In the step, relational constraints are imposed on case triples, as shown in the following formula: In the formula, β l represents the weight parameter related to the law, β c Represents the weight parameter related to the crime, r l Represents the self-attention vector related to the current sample and the law, Represents the attention vector related to the current sample and its similar cases and the law, represents the attention vector related to the current sample and its dissimilar cases and the legal provisions, r c Represents the self-attention vector related to the current sample and the crime, represents the attention vector related to the crime of the current sample and its similar cases, The attention vector representing the current sample and its dissimilar cases related to the crime; In the step, the category decoder calculates the category label of the current sample as shown in the following formula: In the formula, Represents the weight matrix in the activation function relu, represents the weight matrix in the softmax layer, Represents the bias vector in the softmax layer; The decoded representation of the current sample is obtained in the step, as shown in the following formula: represents the initial hidden state in the law prediction task, Represents the initial memory unit in the legal clause prediction task. l The final decoded representation obtained in the representation bar prediction task, c l Represents the memory unit updated by the LSTMCell layer related to the law. represents the initial hidden state in the crime prediction task, represents the initial memory unit in the crime prediction task, W c,l represents the transformation matrix in the crime prediction task, b c,l represents the bias vector in the crime prediction task, h c The final decoded representation obtained in the representation bar prediction task, c c Represents the memory unit obtained by updating the LSTMCell layer related to the crime. In the step, the prediction results of the law prediction task and the crime prediction task are calculated through the fully connected layer represents the predicted value of the law, represents the crime prediction value, represents the total loss of the law prediction and crime prediction tasks, represents the parameter matrix used for the law prediction task, represents the bias vector used for the law prediction task, represents the parameter matrix for the crime prediction task, represents the bias vector used for the crime prediction task, y l Indicates the true value of the law, y c Indicates the true value of the crime.

10. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to perform any one of the methods described in claims 1-9.

Images