Multi-role recognition method based on text mask and contrast strategy

Abstract

The present invention relates to the field of intelligent judicial assistance technology, and in particular, provides a multi-role identification method based on text masking and comparison strategies. The method includes constructing a training data set of principal and accessory pairs based on a name database; obtaining a defendant responsibility reasoning model based on the crime facts by using the masked crime fact text and guilt labels in the training data set; embedding and adding the label broadcast vector and the text embedding vector to enhance the defendant role recognition model based on the court statement; obtaining the guilt reasoning result or guilt recognition result by calling the defendant responsibility reasoning model based on the crime facts or the defendant role recognition model based on the court statement. In complex multi-role cases, this method can combine semantic information and context analysis to infer the principal and accessory responsibilities from the crime fact description, and identify the principal and accessory roles from the court statement, providing a reliable reference basis for judicial decision-making and improving judicial efficiency and judicial fairness.

Description

Technical Field

[0001] The present invention relates to the field of intelligent judicial assistance technology, and in particular to a multi-role recognition method based on text masking and comparison strategy. Background Art

[0002] With the sharp increase in the number of cases in the judicial system, the work pressure of judicial personnel has also continued to increase. Judges, prosecutors and other legal practitioners not only need to deal with the handling of massive cases, but also face increasingly complex legal issues. With the increasing maturity of technologies such as big data, artificial intelligence and machine learning, the demand for intelligent and technological tools in the judicial field has become more urgent, giving rise to a series of intelligent judicial auxiliary tools such as COMPAS. These tools analyze and process standardized judgment documents, and use complex algorithms and models to provide reliable reference for judicial decision-making, thereby improving judicial efficiency and fairness.

[0003] Existing studies have tried to use different methods to deal with the complexity of multi-role cases, but all of them have certain limitations. For example, Holger Andreas et al. used the counterfactual perturbation technique of social norms to define the causal relationship between behavior and results in the study "Counterfactuals for Causal Responsibility in Legal Contexts". However, this method relies on an idealized social order and is difficult to adapt to complex real-life scenarios. In the study "Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning", Yougang Lyu et al. used a generative model to label the individual behaviors of defendants as active behaviors or helping behaviors in the event. Although this method has improved the effect of charge and sentencing prediction in multi-defendant cases to a certain extent, the method is relatively complex in application and lacks fine-tuning for specific tasks, resulting in low model prediction accuracy and lack of professionalism. In the study "Charge Prediction For Multi-Defendant Cases with Multi-Scale Attention", Sicheng Pan et al. proposed using a local attention mechanism to focus on text descriptions related to the defendant, and supplement the overall information of the case through a global attention mechanism, thereby improving the accuracy of charge prediction in multi-defendant cases. However, their research did not delve into the level of guilt reasoning and identification in multi-defendant cases. In the study “Identifying Principals and Accessories in a Complex Case Based on the Comprehension of Fact Description”, Yakun Hu et al. combined behavioral semantic information and statistical features to model the defendant’s role. They believed that important roles should appear more frequently and more evenly in the case description. However, their method ignored the deep semantic analysis of role behavior and only relied on sentence complexity (such as sentence length, number of verbs, syntactic parsing depth, etc.) to distinguish between principal and accessory roles, making it difficult to achieve accurate identification in a complex and changing case context. In addition, the method is difficult to implement and the specific details are not disclosed. According to the relevant paper report, the highest accuracy of this method in the test was only 82.85%, which failed to achieve satisfactory results. Summary of the invention

[0004] In view of this, the present invention provides a multi-role identification method based on text masking and comparison strategy, which is used to infer the principal and accomplice responsibilities from the description of the crime facts in complex multi-role cases, and identify the principal and accomplice roles from the court statements, so as to provide a reliable reference basis for judicial decision-making and improve judicial efficiency and fairness.

[0005] In a first aspect, the present invention provides a multi-role recognition method based on text mask and comparison strategy, the method comprising:

[0006] Step 1: Based on regularization technology, extract the criminal facts, the names of all defendants in the court statement and their guilt labels from the judgment documents, and the guilt is the principal or accomplice;

[0007] Step 2: For each defendant extracted, perform a MASK mask operation on his / her name in the crime fact text or court statement text of the judgment document, and refine the masked court statement text to obtain the masked crime fact text or refined court statement text of each defendant;

[0008] Step 3: Use the name of each defendant as a keyword to build a name database [masked crime fact text, refined court statement text, guilt label]; and build a training data set of principal and accessory pairs based on the name database;

[0009] Step 4: Input the masked crime fact text and guilt label in the training data set into the classification model based on the BERT-Base-Chinese base, conduct supervised training on the classification model based on the BERT-Base-Chinese base, and obtain the defendant's responsibility reasoning model based on the crime facts after multiple rounds of optimization;

[0010] Step 5: Map the principal and accomplice prediction labels output by the defendant responsibility reasoning model based on the criminal facts to obtain the label broadcast vector; embed the refined court statement text into the classification model based on the BERT-Base-Chinese base, perform text embedding encoding and multi-head attention calculation, and obtain the text embedding vector;

[0011] Step 6: embed and add the label broadcast vector and text embedding vector, and input them into the feedforward layer of the classification model based on the bert-base-chinese base for parameter optimization. The guilt labels in the training data set are used as the target labels for training and supervised training is performed. After multiple rounds of optimization, the defendant role recognition model based on the court statement is obtained.

[0012] Step 7: Obtain guilt reasoning results or guilt identification results by calling the defendant responsibility reasoning model based on the criminal facts or the defendant role identification model based on the court statement.

[0013] Optionally, the MASK mask operation in step 2 includes:

[0014] Perform a MASK masking operation on the defendant's name in the crime fact text or the court statement text, check all names in the crime fact string or the court statement string, and replace them with '[MASK]'.

[0015] Optionally, step three includes:

[0016] According to the number of principal offenders and accomplices counted in the judgment documents, the same number of principal offenders and accomplices are extracted to obtain the corresponding name pairs of the principal and accomplices. Based on the name database, a paired training data set containing masked crime fact text, refined court statement text, and guilt labels is constructed.

[0017] Optionally, the training in step 4 or step 6 includes:

[0018] Training data preprocessing includes:

[0019] a) Comparative data generation: Screen the crime fact texts or court statement texts that contain both the principal offender and the accomplice, and extract the mask data of the principal offender and the accomplice in the crime fact texts or court statement texts in pairs to generate representative comparative training data pairs;

[0020] b) Word segmentation: The masked text is segmented based on the BertTokenizer in the Transformers library, and then the tokenizer.encode_plus() method is used to encode the text to generate the token ID and attention mask input format;

[0021] The training process includes:

[0022] h) Model selection: Fine-tune the classification model based on the bert-base-chinese base, and set the output category to 2, i.e., principal offender or accomplice;

[0023] i) Loss function: In order to eliminate the uncertainty of the system at the lowest cost, the cross-entropy loss function CrossEntropyLoss is used to optimize and reduce the relative entropy. During the training process, the classification model based on the bert-base-chinese base outputs the logits score and compares it with the true label to calculate the loss.

[0024] The loss function is defined as: ;

[0025] in Indicates that the i-th sample is in the true category y iThe predicted probability of

[0026] j) Optimizer: Use AdamW optimizer to optimize model parameters, use Scheduler to dynamically adjust the learning rate of the model, and set weight decay;

[0027] k) Multiple rounds of repeated training: The data set is divided into training set, validation set and test set by using the ten-fold cross-validation method; during the training process, different initial parameters are set to perform multiple rounds of repeated experiments; in each round of experiments, the classification model based on the bert-base-chinese base is trained for multiple epochs, and the steps of each iteration are as follows:

[0028] k1) Forward propagation: input the input data to the classification model based on the bert-base-chinese base to obtain the predicted logits score;

[0029] k2) Calculate loss: Use the loss function to calculate the loss between the output and the true label;

[0030] k3) Backward propagation: call loss.backward() to perform back propagation and calculate the gradient;

[0031] k4) Parameter update: call optimizer.step() to use the optimizer to update the model parameters.

[0032] Optionally, by calling the defendant responsibility reasoning model based on the criminal facts, the guilt reasoning results obtained include:

[0033] Based on the regularization technology, the crime fact text and the name of the defendant to be judged are extracted from the judgment document; the name of the defendant to be judged is masked in the crime fact text, all names in the crime fact string are checked, and replaced with '[MASK]' to obtain the masked crime fact text corresponding to the defendant to be judged; BERT is used to segment and encode the masked crime fact text corresponding to the defendant to be judged, and the maximum length is set to 512, so as to truncate or fill the masked crime fact text corresponding to the defendant to be judged; then, the encoded data is input into the defendant responsibility reasoning model based on the crime facts, the role reasoning of the principal or accessory is performed, and the binary logits score for judging the principal and accessory is output; the softmax formula is used to calculate the scores of the principal and accessory labels, the category with the highest score is selected, the final reasoning classification result is output, and the principal and accessory prediction results are expanded to the model embedding dimension using the broadcast mechanism. Optionally, by calling the defendant role recognition model based on the court statement, the guilt recognition results obtained include:

[0034] Based on regularization technology, the court statement text and the name of the defendant to be judged are extracted from the judgment document; the name of the defendant to be judged is masked in the crime fact text and the court statement text, and all names in the crime fact string and the court statement string are checked and replaced with '[MASK]'; the masked court statement text is divided according to the sentence level to generate a sentence list; each sentence in the list is traversed, and the sentences that do not contain [MASK] are deleted, and the retained sentences are reassembled to generate a refined court statement text; the masked facts of the same defendant to be judged are predicted based on the defendant's responsibility reasoning model based on the crime facts to obtain his guilt label; to match the model embedding dimension, the guilt label is broadcast to the corresponding dimension according to the model embedding dimension The label broadcast vector is obtained by adding the dimension size to the word size of the court statement. When the dimension size is 768, the prediction result with the label 1 is broadcast as a 1×768-dimensional all-1 vector. BERT is used to segment and encode the refined court statement text, and the maximum length is set to 512 so that the text can be truncated or padded to obtain a 1×768-dimensional embedding of the court statement, that is, a text embedding vector. Combining the criminal facts and the court statement information, the label broadcast vector is added to the text embedding vector and injected into the feedforward layer to update the parameters, identify the roles of the principal and the accomplice, and output the binary logits scores for judging the principal and the accomplice. The softmax formula is used to calculate the scores of the principal and the accomplice labels, the category with the highest score is selected, and the final recognition and classification results are output.

[0035] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the multi-role recognition method based on text mask and comparison strategy in the first aspect or any possible implementation of the first aspect.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the multi-role recognition method based on text mask and comparison strategy in the first aspect or any possible implementation of the first aspect.

[0037] In the technical solution provided by the present invention, the method includes extracting the names of all defendants in the criminal facts and court statements and their guilt labels from the judgment documents based on regularization technology, and determining whether the guilt is the principal offender or the accessory offender; for each extracted defendant, performing a MASK masking operation on his name in the criminal facts text or the court statement text of the judgment document, and refining the masked court statement text to obtain the masked criminal facts text or the refined court statement text of each defendant; using the name of each defendant as a keyword to construct a name [masked criminal facts text, refined court statement text, guilt label] database; and constructing a training data set of principal and accessory offenders in pairs based on the name database; inputting the masked criminal facts text and guilt labels in the training data set into a classification model based on the BERT-base-Chinese base, and conducting supervised training on the classification model based on the BERT-base-Chinese base, and obtaining a defendant responsibility reasoning model based on the criminal facts after multiple rounds of optimization; and the defendant responsibility reasoning model based on the criminal facts is obtained. The predicted labels of principal and accessory offenders output by the model are mapped to obtain label broadcast vectors; the refined court statement text is embedded in the classification model based on the BERT-base-Chinese base, and text embedding encoding and multi-head attention calculation are performed to obtain text embedding vectors; the label broadcast vector and the text embedding vector are embedded and added and enhanced, and input into the feedforward layer of the classification model based on the BERT-base-Chinese base for parameter optimization, and the guilt labels in the training data set are used as the target labels for training for supervised training. After multiple rounds of optimization, a defendant role recognition model based on court statements is obtained; by calling the defendant responsibility reasoning model based on criminal facts or the defendant role recognition model based on court statements, the guilt reasoning results or guilt recognition results are obtained. In complex multi-role cases, this method can combine semantic information and context analysis to infer the principal and accessory responsibilities from the description of the criminal facts, and identify the principal and accessory roles from the court statements, providing a reliable reference for judicial decision-making and improving judicial efficiency and fairness. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 An overall structural diagram of a standardized judgment document provided by an embodiment of the present invention;

[0040] Figure 2 A flowchart of a multi-role recognition method provided by an embodiment of the present invention;

[0041] Figure 3 A flowchart of another multi-role recognition method provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of obtaining guilt inference results or guilt identification results provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0047] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0048] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0049] Intelligent judicial assistance tools have strict requirements on the format and content of judicial documents. Judgment documents are not only a written expression of the judicial personnel's case trial results, but also a concentrated reflection of legal thinking, legal logic and factual basis. Standardized judicial documents usually include the following: title case number, basic information of the defendant (such as place of origin, education, address, whether there is a criminal record, etc.), description of the criminal facts ascertained by the court, evidence such as witness testimony and appraisal reports, summary of the court statement (including restatement of the facts of the crime, determination of whether the crime has been established, division of responsibilities, evaluation of the circumstances of the crime and remorse, applicable legal provisions, etc.), and the final judgment result (such as crime, type of punishment, sentence, whether probation is applicable and its duration, amount of fine, etc.). The overall structure of standardized judicial documents, such as Figure 1 shown.

[0050] In the trial process of multi-role cases, intelligent judicial assistance tools are required to have excellent role reasoning capabilities in order to correctly identify the principal offender and the accomplice. Therefore, accurately identifying the role of each defendant in the case is crucial to judicial conviction and sentencing. However, since the description of the criminal facts in multi-defendant cases usually contains complex language and implicit information, the training method based on the original text is difficult to effectively extract information on behavioral comparison and role distinction between multiple defendants. In addition, the degree of involvement of the principal offender and the accomplice in the case varies greatly, and their behavioral characteristics are difficult to describe with simple statistical characteristics. Therefore, current intelligent judicial assistance tools are rarely used in the trial process of multi-role cases.

[0051] The multi-role recognition method provided by the present invention is as follows: Figure 2 and Figure 3 As shown, the method includes:

[0052] Step 1: Based on regularization technology, extract the criminal facts, the names of all defendants in the court statement and their guilt labels from the judgment documents, and whether the guilt is the principal or the accomplice.

[0053] Step 2: For each defendant extracted, perform a MASK operation on his / her name in the crime fact text or court statement text of the judgment document, and refine the masked court statement text to obtain the masked crime fact text or refined court statement text of each defendant.

[0054] In the embodiment of the present invention, the MASK mask operation in step 2 includes:

[0055] Perform a MASK mask operation on the defendant's name in the crime fact text or court statement text, check all names in the crime fact string or court statement string, and replace them with '[MASK]', such as Figure 4 As shown, replace the text with '[MASK]'.

[0056] Step 3: Use the name of each defendant as a keyword to build a name database [masked crime fact text, refined court statement text, guilt label]; and build a training data set of principal and accessory pairs based on the name database.

[0057] In the embodiment of the present invention, step three includes:

[0058] According to the number of principal offenders and accomplices counted in the judgment documents, the same number of principal offenders and accomplices are extracted to obtain the corresponding name pairs of the principal and accomplices. Based on the name database, a paired training data set containing masked crime fact text, refined court statement text, and guilt labels is constructed.

[0059] Step 4: Input the masked crime fact text and guilt labels in the training data set into the classification model based on the BERT-Base-Chinese base, conduct supervised training on the classification model based on the BERT-Base-Chinese base, and obtain the defendant's responsibility reasoning model based on the crime facts after multiple rounds of optimization.

[0060] In the embodiment of the present invention, the classification model based on the bert-base-chinese base is a BertSequenceClassification classification model based on the bert-base-chinese base.

[0061] Step 5. Map the principal and accomplice prediction labels output by the defendant responsibility reasoning model based on the criminal facts to obtain the label broadcast vector; embed the refined court statement text into the classification model based on the BERT-Base-Chinese base, perform text embedding encoding and multi-head attention calculation, and obtain the text embedding vector.

[0062] Step 6. The label broadcast vector and the text embedding vector are embedded and added, and input into the feedforward layer of the classification model based on the BERT-Base-Chinese base for parameter optimization. The guilt labels in the training data set are used as the target labels for training for supervised training. After multiple rounds of optimization, the defendant role recognition model based on court statements is obtained.

[0063] In the embodiment of the present invention, the above training is the training of the long text data information enhancement model based on the bert-base-chinese base.

[0064] In the embodiment of the present invention, the training in step 4 or step 6 includes:

[0065] Training data preprocessing includes:

[0066] a) Comparative data generation: Screen the crime fact texts or court statement texts that contain both the principal offender and the accomplice, and extract the mask data of the principal offender and the accomplice in the crime fact texts or court statement texts in pairs to generate representative comparative training data pairs;

[0067] b) Word segmentation: The masked text is segmented based on the BertTokenizer in the Transformers library, and then the tokenizer.encode_plus() method is used to encode the text to generate the token ID and attention mask input format;

[0068] The training process includes:

[0069] h) Model selection: Fine-tune the classification model based on the bert-base-chinese base, and set the output category to 2, i.e., principal offender or accomplice;

[0070] i) Loss function: In order to eliminate the uncertainty of the system at the lowest cost, the cross-entropy loss function CrossEntropyLoss is used to optimize and reduce the relative entropy. During the training process, the classification model based on the bert-base-chinese base outputs the logits score and compares it with the true label to calculate the loss.

[0071] The loss function is defined as: ;

[0072] in Indicates that the i-th sample is in the true category y i The predicted probability of

[0073] j) Optimizer: Use AdamW optimizer to optimize model parameters, use Scheduler to dynamically adjust the learning rate of the model, and set weight decay;

[0074] k) Multiple rounds of repeated training: The data set is divided into training set, validation set and test set by using the ten-fold cross-validation method; during the training process, different initial parameters are set to perform multiple rounds of repeated experiments; in each round of experiments, the classification model based on the bert-base-chinese base is trained for multiple epochs, and the steps of each iteration are as follows:

[0075] k1) Forward propagation: input the input data to the classification model based on the bert-base-chinese base to obtain the predicted logits score;

[0076] k2) Calculate loss: Use the loss function to calculate the loss between the output and the true label;

[0077] k3) Backward propagation: call loss.backward() to perform back propagation and calculate the gradient;

[0078] k4) Parameter update: call optimizer.step() to use the optimizer to update the model parameters.

[0079] Step 7: Obtain guilt reasoning results or guilt identification results by calling the defendant responsibility reasoning model based on the criminal facts or the defendant role identification model based on the court statement.

[0080] In the embodiment of the present invention, Figure 4 As shown,

[0081] By calling the defendant's responsibility reasoning model based on the criminal facts, the guilt reasoning results obtained include:

[0082] Based on the regularization technology, the crime fact text and the name of the defendant to be judged are extracted from the judgment document, for example, the defendant to be judged is Wen; the name of the defendant to be judged is masked in the crime fact text, all the names in the crime fact string are checked and replaced with '[MASK]', and the masked crime fact text corresponding to the defendant to be judged is obtained; BERT is used to segment and encode the masked crime fact text corresponding to the defendant to be judged, and the maximum length is set to 512, so as to truncate or fill the masked crime fact text corresponding to the defendant to be judged; then, the encoded data is input into the defendant responsibility reasoning model based on the crime facts, the role reasoning of the principal or accessory is performed, and the binary logits scores for judging the principal and accessory are output; the softmax formula is used to calculate the scores of the principal and accessory labels, the category with the highest score is selected, the final reasoning classification result is output, and the principal and accessory prediction results are expanded to the model embedding dimension using the broadcast mechanism.

[0083] In the embodiment of the present invention, by calling the defendant role identification model based on the court statement, the guilt identification result is obtained, including:

[0084] Based on regularization technology, the court statement text and the name of the defendant to be judged are extracted from the judgment document; the name of the defendant to be judged is masked in the crime fact text and the court statement text, and all names in the crime fact string and the court statement string are checked and replaced with '[MASK]'; the masked court statement text is divided according to the sentence level to generate a sentence list; each sentence in the list is traversed, and the sentences that do not contain [MASK] are deleted, and the retained sentences are reassembled to generate a refined court statement text; the masked facts of the same defendant to be judged are predicted based on the defendant's responsibility reasoning model based on the crime facts to obtain his guilt label; to match the model embedding dimension, the guilt label is broadcast to the corresponding dimension according to the model embedding dimension The label broadcast vector is obtained by adding the dimension size to the word size of the court statement. When the dimension size is 768, the prediction result with the label 1 is broadcast as a 1×768-dimensional all-1 vector. BERT is used to segment and encode the refined court statement text, and the maximum length is set to 512 so that the text can be truncated or padded to obtain a 1×768-dimensional embedding of the court statement, that is, a text embedding vector. Combining the criminal facts and the court statement information, the label broadcast vector is added to the text embedding vector and injected into the feedforward layer to update the parameters, identify the roles of the principal and the accomplice, and output the binary logits scores for judging the principal and the accomplice. The softmax formula is used to calculate the scores of the principal and the accomplice labels, the category with the highest score is selected, and the final recognition and classification results are output.

[0085] In the embodiment of the present invention, the core of the method is to replace the defendant's name with [MASK] through a masking strategy, so that the technology can accurately infer the guilt of the principal and the accessory in the fact description in multi-defendant cases, and identify the role labels of the principal and the accessory in the court statement. Constructing a training data set of principal and accessory pairs enables the model to better understand the interaction between different roles, significantly improving the model's reasoning ability and accuracy in handling complex multi-defendant cases.

[0086] The present invention focuses on solving the problem of guilt reasoning and role identification of the principal offender and the accomplice, and enhances the contextual attention of the names of the defendants whose guilt is to be inferred and whose roles are to be identified in the description of the crime facts and the court statements, thereby improving the model's ability in guilt reasoning and role identification, so as to assist prosecutors and judges in making judicial decisions during the judicial trial process. This method provides effective technical support for factor extraction in intelligent judgment reasoning tasks. The present invention pre-processes the data through contrast effect enhancement technology and directional masking technology, and adopts a dual-model cascade training method to break through the model input length limit and improve the accuracy and operability of model role identification. At the same time, the present invention focuses on the generalization ability of the technology, which can be widely used in the judicial field to improve the efficiency of case handling and the accuracy of judgment.

[0087] The device architecture of the embodiment of the present invention includes:

[0088] 1. Hardware equipment. The present invention includes a processing server and a storage device. The processing server includes a high-performance CPU and an RTX 3090 GPU for training and reasoning of deep learning models. The storage device is used to store judicial document data, model parameters, and training data sets.

[0089] 2. Software environment. This paper is implemented based on Python language and mainly relies on the BERT pre-trained model provided by the transformers library. In addition, the PyTorch deep learning framework is also used for model training and reasoning.

[0090] The experimental data of the embodiment of the present invention is composed of all criminal cases disclosed on the Judgment Documents Network from 2012 to 2020. In the first-instance judgment cases of intentional injury, all cases involving multiple defendants are screened out based on the keywords "principal offender", "accomplice" and "accomplice" in the court statements. The "fact description" and "court statement" paragraphs of the case, as well as the names of the defendants, are extracted from the judgment documents through regularization technology to construct the original data set. According to the list of defendants in the judgment results, the extracted paragraph text data is anonymized, and sensitive information such as names and place names is hidden. The evaluation and correction of the data are completed by three professional labelers to ensure its accuracy and consistency. A total of 17,253 cases were finally cleaned, involving 34,828 defendants, and each case contained manually verified principal and accessory labels.

[0091] Experimental process:

[0092] 1. Data preprocessing:

[0093] 1. Generate a case dictionary:

[0094] Create a dictionary cases with the key set to case number so that the data can be organized by case number. For each case number, extract all the corresponding defendant information to ensure that all relevant data for the same case are clustered together.

[0095] 2. Training data construction:

[0096] For each case, identify all defendants and check their guilt labels as principal or accessory. Collect the IDs of principals (guilt label is 1) and accessories (guilt label is 0) separately. Construct a database according to the name [masked criminal facts text, refined court statement text, guilt label], such as {"Wen": ['Masked criminal facts text': 'The defendant [Tong] gathered the defendants [MASK], [Wang] and others and drove to the construction site of [Zheng B] at location A with steel pipes, etc., in an attempt to stop [Zheng B] from working. After being refused, the defendant [Tong], together with the defendants [MASK], [Wang] and others, beat [Zheng B], [Zheng A] and others with steel pipes, causing injuries to [Zheng B], [Zheng A], [Zheng D], [Zheng W] and others', 'Refined court statement text': 'The defendants [Tong], [MASK], and [Wang] intentionally injured others, causing minor injuries to two people. Their actions have constituted the crime of intentional injury. The defendant [Tong] played a major role in the joint crime and is the principal offender. He should be punished according to all the crimes he participated in; the defendants [MASK] and [Wang] played a minor role in the joint crime and should be given a lighter punishment. The defendants [Tong], [MASK], and [Wang] can truthfully confess their crimes after being arrested, and can be given a lighter punishment','Crime Label': 'Accomplice']}.

[0097] Compare the training data to build a technical dataset:

[0098] According to the number of principal offenders and accomplices in the same case, the same number of principal offenders and accomplices are randomly selected to form principal and accomplice criminal responsibility comparison data pairs to facilitate subsequent data processing.

[0099] Pairs Dataset:

[0100] 10% of the sample pairs are randomly selected from the principal and subordinate crime comparison data pairs generated above as the test set. Among the remaining 90% of the data pairs, the training set and the validation set are divided into a ratio of 9:1.

[0101] Random Dataset:

[0102] Keep the Pairs test set, and extract the same amount of data as the Pairs training set and validation set from all the remaining data. It is necessary to ensure that the number of principal offender data and accomplice data in the extracted data is the same, and then divide the training set and validation set in a ratio of 9:1.

[0103] Full dataset:

[0104] Keep the test set of Pairs, randomly select the principal offender data and the accomplice data from all the remaining data according to the smaller number of principal offenders and accomplices, and then divide the training set and the validation set into a ratio of 9:1.

[0105] Masking and Refinement Techniques Comparison Dataset

[0106] In order to compare the effects of masking and refining techniques, it is necessary to perform corresponding processing based on the data of the same original data set. The processing scheme is as follows:

[0107] Original data set: Based on the above-mentioned comparative training data construction technology, the case principal and accessory crime responsibility comparison data pairs are constructed, and the training set, validation set and test set are divided into training set, validation set and test set in a ratio of 8:1:1, including the defendant's name, criminal facts, court statements, and principal and accessory labels. It should be noted that the defendant's name in this data set has not been masked.

[0108] Split the dataset: Based on the crime facts and court statements in the original dataset, split them at the sentence level and only keep the sentences containing the names of the defendants of interest. Return the fact descriptions, court statements, and label information after the split.

[0109] MASK dataset (MASK): Based on the above original dataset, the criminal facts and court statements corresponding to the defendants are masked according to their names, and the masked fact descriptions, court statements, and label information are returned.

[0110] MASK Split Dataset: Based on the criminal facts and court statements of the above MASK dataset, the data is segmented at the sentence level, and only the sentences with [MASK] are retained. The masked and segmented fact descriptions, court statements, and label information are returned.

[0111] 2. Model training:

[0112] 1. Model selection and construction:

[0113] Depending on whether the text data contains the words "principal and accessory", the defendant role identification model framework based on court statements or the defendant guilt reasoning model framework based on criminal facts is selected. Then, based on the selected model framework, components such as the embedding layer, hidden layer, output layer, and corresponding activation functions are combined to build a complete model to meet specific application scenarios.

[0114] 2. Hyperparameter settings:

[0115] Define the hyperparameters used in the training process, such as repetitions, batch size, number of training rounds, etc. Use the Scheduler to reduce the learning rate of the model at fixed intervals (epochs) to adapt to different stages of training.

[0116] 3. Training process:

[0117] The training set is input into the model and the cross entropy loss is calculated by forward propagation. Subsequently, the model parameters are optimized by back propagation and the weights are updated using the AdamW optimizer. After each training cycle (epoch), the model performance is evaluated using the validation set and the performance of the model on the validation set is recorded.

[0118] 4. Model saving:

[0119] After training, save the model based on the best performance on the validation set for subsequent use or evaluation.

[0120] 3. Evaluation indicators:

[0121] Evaluation indicator calculation:

[0122] The evaluation function is used to calculate the evaluation index scores of the guilt reasoning and role recognition models on different data sets. The specific indicators include:

[0123] Accuracy (Accuracy, Acc.): The proportion of correctly classified samples to the total samples.

[0124] Precision (P.): The ratio of samples correctly predicted as positive examples to all samples predicted as positive examples.

[0125] Recall (R.): The ratio of samples correctly predicted as positive examples to all actual positive examples.

[0126] F1-score (F1): The harmonic average of precision and recall, used to comprehensively evaluate model performance.

[0127] Result output:

[0128] The evaluation results of each data set are collated and summarized to compare the impact of different data preprocessing and training methods on model performance. The accuracy, precision, recall, and F1-score of different data sets are displayed using formatted output for subsequent analysis.

[0129] The experimental results are shown in Tables 1 to 4, where Table 1 is a comparison of different data preprocessing methods for role recognition based on court statements; Table 2 is a comparison of different data preprocessing methods for guilt reasoning based on criminal facts; Table 3 is a comparison of different data training methods for role recognition based on court statements; and Table 3 is a comparison of different data training methods for guilt reasoning based on criminal facts.

[0130] Table 1

[0131] ;

[0132] Table 2

[0133] ;

[0134] Table 3

[0135] ;

[0136] Table 4

[0137] ;

[0138] From Table 1 to Table 4, we can get the experimental conclusions:

[0139] 1. When dealing with multi-defendant cases, the directional masking scheme proposed in this paper significantly improves the recognition and reasoning accuracy compared to the unprocessed raw data and the data processed by conventional sentence segmentation. This result proves that the directional masking strategy implemented for the defendant's name helps the model understand the role relationship of multiple defendants.

[0140] 2. In the task of identifying the principal and accomplice roles based on court statements, the sentence screening scheme can effectively reduce noise interference in the recognition process and more accurately locate key information, thereby significantly improving the recognition accuracy.

[0141] 3. The training method using the principal and subordinate crime comparison data significantly improved the accuracy of recognition and reasoning, and its performance was better than the training method using the complete data set. This result shows that by using paired samples for model training, the correlation between data can be more effectively captured, thereby enhancing the performance of the model in complex tasks.

[0142] The key points of the present invention are:

[0143] 1. Directional masking technology.

[0144] The technical solution of the present invention realizes masking of the defendant's name by replacing the defendant's name with the [MASK] mark in the judgment document, thereby significantly reducing the model's dependence on specific names. The present invention can enhance the model's attention to the contextual information of the defendant's name and improve the accuracy of the model. In addition, the masking solution fully considers the various structures and expressions of judgment documents when designing, ensuring its wide applicability in various application scenarios.

[0145] 2. Data construction technology for enhanced multi-role comparison effects.

[0146] The present invention extracts the mask data of different culpable roles in the same case in pairs to generate representative comparative analysis data pairs, so that the model can better understand the interaction between different roles and significantly improve the reasoning ability and accuracy of the model in handling complex multi-defendant cases. In addition, the construction process takes into account the diversity of roles and case characteristics to ensure the diversity and authenticity of samples, providing richer and more reliable data support for model training.

[0147] 3. Text refinement technology with multi-character attention enhancement.

[0148] In the role recognition task, in order to reduce the noise of multi-role recognition, the present invention analyzes the text at the sentence level attention mechanism to accurately locate the key information related to the role. This technology effectively filters out redundant information and enables the model to focus on the most influential sentences for multi-role recognition, thereby improving the recognition accuracy of the model. By strengthening the focus on important sentences, the model can have a deeper understanding of the relationship between the roles and their role in the case.

[0149] 4. Large model training technology based on mask data.

[0150] The present invention uses masked data to train and fine-tune the open source large model Bert-Base-Chinese, realizing the guilt reasoning and role identification of defendants in multi-defendant cases. The core of this technology is to combine the masking strategy with the supervised learning method to train the masked text, so that the model can effectively learn the guilt relationship and behavioral characteristics between the roles. This technology improves the model's ability to understand deep semantics and reasoning in complex situations, and also provides a reference for the processing of other types of text data. Training ideas.

[0151] 5. Dual-model cascade training technology

[0152] When training the model, supervised training is first performed based on the masked fact description text to obtain a defendant responsibility reasoning model based on criminal facts. Then, the principal and accessory prediction labels output by model 1 are broadcasted to the model embedding dimension as embedded information and injected into the training process of the defendant role recognition model based on court statements, and finally an attention-enhanced multi-defendant role recognition model is obtained. The dual-model cascade training method is adopted to effectively combine contextual information and improve the role recognition effect of multiple defendants. When calling the model, if the input text contains the words "principal" or "accessory", the defendant responsibility reasoning model based on criminal facts is called to perform principal and accessory responsibility reasoning and obtain the broadcast embedding vector. Otherwise, the defendant role recognition model based on court statements is directly called to enter the principal and accessory role reasoning task.

[0153] Compared with the prior art, the present invention has the following beneficial effects:

[0154] 1. High feasibility. The present invention preprocesses real data and performs model training based on it. The model is applied to real scenarios and can effectively deal with complex real cases with multiple defendants.

[0155] 2. Easy to call. The present invention can complete the task by simply calling the large classification language model fine-tuned based on the mask data. Its operation process is simple and easy to understand, which greatly reduces the threshold for use.

[0156] 3. Strong generalization ability. The present invention can effectively focus on the behavioral comparison and role distinction information among multiple defendants through directional masking technology, deeply deal with the guilt reasoning and role identification problems among multiple defendants, and effectively deal with complex and changeable multi-defendant cases.

[0157] 4. Focus on semantic information. The MASK of the present invention enables the model to pay more attention to the contextual information of the defendant's name, which helps the model understand the deep semantic information of the fact description text and the court statement text.

[0158] 5. Multi-information injection. The present invention found that injecting the crime fact labels derived from the guilt reasoning model into the recognition model of the court statement can significantly enhance the recognition ability of the model. By injecting multiple information of crime facts and court statements, the input length limit of the BERT model is effectively broken. When encountering a situation where the maximum input length of the model is exceeded, this strategy provides an innovative way of information injection to ensure that additional key information can also be fully utilized.

[0159] 6. High precision. The present invention is not only highly innovative, but also has an average accuracy of 91.05% and a maximum accuracy of 93.17% in the task of reasoning about the guilt of criminal facts. It also has an average accuracy of 99.25% and a maximum accuracy of 99.50% in the task of identifying the principal and the accessory roles in court statements, which significantly improves the accuracy of reasoning and identification.

[0160] 7. Fast response. The present invention takes 5.982863 seconds to process 1010 data in the test set, and an average of 0.005924 seconds per data, showing excellent response speed, and can quickly generate results to meet real-time requirements. This advantage is particularly important in the handling of multi-defendant cases, and can effectively improve judicial efficiency.

[0161] Each step of the embodiment of the present invention may be performed by an electronic device, which includes but is not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, etc.

[0162] In the technical solution provided by the present invention, the method includes extracting the names of all defendants in the criminal facts and court statements and their guilt labels from the judgment documents based on regularization technology, and determining whether the guilt is the principal offender or the accessory offender; for each extracted defendant, performing a MASK masking operation on his name in the criminal facts text or the court statement text of the judgment document, and refining the masked court statement text to obtain the masked criminal facts text or the refined court statement text of each defendant; using the name of each defendant as a keyword to construct a name [masked criminal facts text, refined court statement text, guilt label] database; and constructing a training data set of principal and accessory offenders in pairs based on the name database; inputting the masked criminal facts text and guilt labels in the training data set into a classification model based on the BERT-base-Chinese base, and conducting supervised training on the classification model based on the BERT-base-Chinese base, and obtaining a defendant responsibility reasoning model based on the criminal facts after multiple rounds of optimization; and the defendant responsibility reasoning model based on the criminal facts is obtained. The predicted labels of principal and accessory offenders output by the model are mapped to obtain label broadcast vectors; the refined court statement text is embedded in the classification model based on the BERT-base-Chinese base, and text embedding encoding and multi-head attention calculation are performed to obtain text embedding vectors; the label broadcast vector and the text embedding vector are embedded and added and enhanced, and input into the feedforward layer of the classification model based on the BERT-base-Chinese base for parameter optimization, and the guilt labels in the training data set are used as the target labels for training for supervised training. After multiple rounds of optimization, a defendant role recognition model based on court statements is obtained; by calling the defendant responsibility reasoning model based on criminal facts or the defendant role recognition model based on court statements, the guilt reasoning results or guilt recognition results are obtained. In complex multi-role cases, this method can combine semantic information and context analysis to infer the principal and accessory responsibilities from the description of the criminal facts, and identify the principal and accessory roles from the court statements, providing a reliable reference for judicial decision-making and improving judicial efficiency and fairness.

[0163] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the electronic device where the computer-readable storage medium is located is controlled to execute the above-mentioned embodiment of the multi-role recognition method based on text mask and comparison strategy.

[0164] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 5As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the multi-role recognition method based on text mask and comparison strategy in the embodiment is implemented. To avoid repetition, they are not described one by one here.

[0165] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0166] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0167] The memory 212 may be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 may also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 21. Further, the memory 212 may also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 may also be used to temporarily store data that has been output or is to be output.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-role recognition method based on text masking and contrast strategy, characterized in that: The method comprises: Step 1: Based on regularization technology, extract the criminal facts, the names of all defendants in the court statement and their guilt labels from the judgment documents, and whether the guilt is the principal or the accomplice; Step 2: For each defendant extracted, perform a MASK mask operation on his / her name in the crime fact text or court statement text of the judgment document, and refine the masked court statement text to obtain the masked crime fact text or refined court statement text of each defendant; Step 3: Use the name of each defendant as a keyword to build a name database [masked crime fact text, refined court statement text, guilt label]; and build a training data set of principal and accessory pairs based on the name database; Step 4: Input the masked crime fact text and guilt label in the training data set into the classification model based on the BERT-Base-Chinese base, conduct supervised training on the classification model based on the BERT-Base-Chinese base, and obtain the defendant's responsibility reasoning model based on the crime facts after multiple rounds of optimization; Step 5: Map the principal and accomplice prediction labels output by the defendant responsibility reasoning model based on the criminal facts to obtain the label broadcast vector; embed the refined court statement text into the classification model based on the BERT-Base-Chinese base, perform text embedding encoding and multi-head attention calculation, and obtain the text embedding vector; Step 6: embed and add the label broadcast vector and text embedding vector, and input them into the feedforward layer of the classification model based on the bert-base-chinese base for parameter optimization. The guilt labels in the training data set are used as the target labels for training and supervised training is performed. After multiple rounds of optimization, the defendant role recognition model based on the court statement is obtained. Step 7: Obtain guilt reasoning results or guilt identification results by calling the defendant responsibility reasoning model based on the criminal facts or the defendant role identification model based on the court statement.

2. The method according to claim 1, characterized in that The MASK operation in step 2 includes: Perform a MASK masking operation on the defendant's name in the crime fact text or the court statement text, check all names in the crime fact string or the court statement string, and replace them with '[MASK]'.

3. The method according to claim 1, characterized in that Step three includes: According to the number of principal offenders and accomplices counted in the judgment documents, the same number of principal offenders and accomplices are extracted to obtain the corresponding name pairs of the principal and accomplices. Based on the name database, a paired training data set containing masked crime fact text, refined court statement text, and guilt labels is constructed.

4. The method according to claim 1, characterized in that: Training in step 4 or step 6 includes: Training data preprocessing includes: a) Comparative data generation: Screen the crime fact texts or court statement texts that contain both the principal offender and the accomplice, and extract the mask data of the principal offender and the accomplice in the crime fact texts or court statement texts in pairs to generate representative comparative training data pairs; b) Word segmentation: The masked text is segmented based on the BertTokenizer word segmenter in the Transformers library, and then the tokenizer.encode_plus() method is used to encode the text to generate the token ID and attention mask input format; The training process includes: h) Model selection: Fine-tune the classification model based on the bert-base-chinese base, and set the output category to 2, i.e., principal offender or accomplice; i) Loss function: In order to eliminate the uncertainty of the system at the lowest cost, the cross-entropy loss function CrossEntropyLoss is used to optimize and reduce the relative entropy. During the training process, the classification model based on the bert-base-chinese base outputs the logits score and compares it with the true label to calculate the loss. The loss function is defined as: ; in Indicates that the i-th sample is in the true category y i The predicted probability of j) Optimizer: Use AdamW optimizer to optimize model parameters, use Scheduler to dynamically adjust the learning rate of the model, and set weight decay; k) Multiple rounds of repeated training: The data set is divided into training set, validation set and test set by using the ten-fold cross-validation method; during the training process, different initial parameters are set to perform multiple rounds of repeated experiments; in each round of experiments, the classification model based on the bert-base-chinese base is trained for multiple epochs, and the steps of each iteration are as follows: k1) Forward propagation: input the input data to the classification model based on the bert-base-chinese base to obtain the predicted logits score; k2) Calculate loss: Use the loss function to calculate the loss between the output and the true label; k3) Backward propagation: call loss.backward() to perform back propagation and calculate the gradient; k4) Parameter update: call optimizer.step() to use the optimizer to update the model parameters.

5. The method according to claim 1, characterized in that By calling the defendant's responsibility reasoning model based on the criminal facts, the guilt reasoning results obtained include: Based on the regularization technology, the crime fact text and the name of the defendant to be judged are extracted from the judgment document; the name of the defendant to be judged in the crime fact text is masked, all the names in the crime fact string are checked and replaced with '[MASK]' to obtain the masked crime fact text corresponding to the defendant to be judged; BERT is used to segment and encode the masked crime fact text corresponding to the defendant to be judged, and the maximum length is set to 512, so as to truncate or fill the masked crime fact text corresponding to the defendant to be judged; then, the encoded data is input into the defendant responsibility reasoning model based on the crime facts, the role reasoning of the principal or accessory is performed, and the binary logits scores for judging the principal and accessory are output; the softmax formula is used to calculate the scores of the principal and accessory labels, the category with the highest score is selected, the final reasoning classification result is output, and the principal and accessory prediction results are expanded to the model embedding dimension using the broadcast mechanism.

6. The method according to claim 1, characterized in that By calling the defendant role identification model based on the court statement, the guilt identification results obtained include: Based on regularization technology, the court statement text and the name of the defendant to be judged are extracted from the judgment document; the name of the defendant to be judged is masked in the crime fact text and the court statement text, and all names in the crime fact string and the court statement string are checked and replaced with '[MASK]'; the masked court statement text is divided according to the sentence level to generate a sentence list; each sentence in the list is traversed, and the sentences that do not contain [MASK] are deleted, and the retained sentences are reassembled to generate a refined court statement text; the masked facts of the same defendant to be judged are predicted based on the defendant's responsibility reasoning model based on the crime facts to obtain his guilt label; to match the model embedding dimension, the guilt label is broadcast to the corresponding dimension according to the model embedding dimension The label broadcast vector is obtained by adding the dimension size to the word size of the court statement. When the dimension size is 768, the prediction result with the label 1 is broadcast as a 1×768-dimensional all-1 vector. BERT is used to segment and encode the refined court statement text, and the maximum length is set to 512 so that the text can be truncated or padded to obtain a 1×768-dimensional embedding of the court statement, that is, a text embedding vector. Combining the criminal facts and the court statement information, the label broadcast vector is added to the text embedding vector and injected into the feedforward layer to update the parameters, identify the roles of the principal and the accomplice, and output the binary logits scores for judging the principal and the accomplice. The softmax formula is used to calculate the scores of the principal and the accomplice labels, the category with the highest score is selected, and the final recognition and classification results are output.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the multi-role recognition method based on text mask and comparison strategy according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the multi-role recognition method based on text mask and comparison strategy as described in any one of claims 1 to 6.

Images