A charge prediction multi-task learning method and system fusing behavior words
By identifying the boundaries of behavioral words in judicial documents using BERT and BiLSTM, and combining them with graph convolutional neural networks to construct correlations, this approach solves the problems of ignoring differences in case behavior and insufficient parameter sharing in existing technologies, thus achieving more efficient crime prediction and behavioral word extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU UNIV
- Filing Date
- 2023-01-04
- Publication Date
- 2026-04-24
AI Technical Summary
Existing crime prediction methods rely on text features, ignoring the differences in behavioral elements across different cases, lacking effective utilization of criminal behavior, and failing to share parameter information between different tasks, resulting in unsatisfactory prediction performance on large-scale and diverse datasets.
The BERT model is used to obtain the abstract semantic representation of text information. BiLSTM is used to extract contextual semantic dependency features, identify the boundaries of action words and generate spans. The correlation between action words is constructed through graph convolutional neural networks. A multi-task framework is used to share parameters to improve the generalization ability and recognition performance of the model.
It improves the accuracy and generalization ability of the crime prediction model, effectively utilizes key information in cases, adapts to multiple related but different tasks, and achieves end-to-end document-level behavioral word extraction and crime prediction.
Smart Images
Figure CN116050609B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information extraction, specifically relating to a multi-task learning method and system for crime prediction that integrates behavioral words. Background Technology
[0002] In recent years, with the rapid development of smart court construction and the public release of judgment data, the application of artificial intelligence technology in the judicial field has become a hot topic. People's courts at all levels are actively promoting the integration of the internet, big data, and artificial intelligence with judicial practice. Applications such as intelligent judicial Q&A, case knowledge graph construction, and relevant legal provision recommendations are gradually changing the legal services market. The combination of judicial tasks and artificial intelligence simplifies the workflow of judicial personnel, provides legal advice to judges, and offers legal consultation to those lacking legal knowledge. Courts have accumulated a large number of judgments in their long-term judicial practice, describing factors influencing case decisions, such as the circumstances of the parties, the case process, the trial procedures, and the judgment results. Therefore, judgments contain important case information and knowledge value. How to efficiently extract valuable information from these specialized textual data, and how to utilize the special information in judgments to support judicial tasks such as crime prediction, are of significant research importance in effectively improving the intelligence level of auxiliary trial work.
[0003] Early work on crime prediction tasks generally employed existing mathematical models or statistical methods. This involved comparing the case to be tested with manually constructed rule templates. Using case text information as input and then outputting the corresponding crime achieved some success. However, establishing complex rules requires a large number of legal professionals, and the construction of semantic information remains at a superficial level of text analysis. These methods are only applicable to specific cases and data, and the predictive performance of the models is not ideal, exhibiting poor generality and failing to achieve crime prediction on large-scale and diverse datasets. Furthermore, most crime prediction tasks are treated as equivalent to text classification tasks. Directly predicting the crime based on the full semantics of the case description often ignores the structural characteristics of the text content, fails to fully utilize special vocabulary information, and the accumulation of useless information in the content can interfere with the judgment.
[0004] In recent years, with the development of deep learning, the use of neural network models in the legal field has achieved good results. One approach involves introducing 10 manually labeled representative distinguishing attributes and using a neural network model combined with crime attributes for prediction. Another approach is to use a neural network model incorporating crime keyword information for crime prediction. Addressing the content differences between crime prediction and text classification, utilizing specific information from case keywords can focus on case information, express core semantics, and effectively improve the model's accuracy and generalization ability. However, using unsupervised or manually labeled methods to extract keyword information fails to establish a connection between the keyword extraction task and the crime prediction task. Furthermore, the importance and relevance of various words in a crime often vary significantly in the judgments of different crimes. Summary of the Invention
[0005] This invention provides a multi-task learning method for crime prediction that integrates behavioral words, in order to solve the problems that existing methods often rely on text features, ignore the differences in behavioral elements of different cases, lack effective utilization of criminal behavior, and cannot share parameter information between different tasks.
[0006] This invention is achieved through the following technical solution:
[0007] A multi-task learning method for crime prediction that integrates action words, specifically including the following steps:
[0008] Step 1: Use BERT pre-training to obtain an abstract semantic representation of the text information in the case description of the judgment document;
[0009] Step 2: Based on the abstract semantic representation of the text information in Step 1, use BiLSTM to obtain the contextual semantic dependency features in the text;
[0010] Step 3: Based on the contextual semantic dependency features from Step 2, locate the position of the action word in the text by identifying the boundary of the action word;
[0011] Step 4: Based on the boundary of the action word located in Step 3, the span of the action word is generated by combining different boundaries;
[0012] Step 5: Based on the span of behavioral words identified in Step 4, predict the corresponding crime by constructing correlation information between behavioral words.
[0013] A multi-task learning method for crime prediction that integrates action words, wherein step 1 specifically involves using the BERT model to obtain dynamic word vector representations that integrate contextual information for the case description in the judgment document and outputting continuous dense vectors that integrate the semantics of the entire text.
[0014] A multi-task learning method for crime prediction that integrates action words, wherein step 2 specifically involves training a continuous dense vector that integrates the semantics of the entire text bidirectionally through a BILSTM layer to further extract deep semantic features and features of long-distance dependency generated text content.
[0015] A multi-task learning method for crime prediction that integrates action words, wherein step 4 specifically involves locating the position of the action word in the text by identifying a single character as the start or end boundary of the action word, and then classifying the span generated by combining different boundary positions.
[0016] A multi-task learning method for crime prediction by fusing action words, wherein step 5 specifically involves constructing a graph structure using action words as nodes, fusing semantic information between action words through a graph convolutional neural network to establish a word structure and enhance the correlation between action words, while using a multi-objective framework to establish the correlation between different tasks; by using a multi-task model for extracting action words and predicting crimes, the performance of the two tasks can be improved by sharing parameters across tasks.
[0017] A multi-task learning method for crime prediction that integrates action words, with the case description text being... ,in Indicates the first in the text Each character;
[0018] Will The BERT model outputs a vector after passing through multiple Transformer layers. .
[0019] A multi-task learning method for crime prediction that integrates action words, Bidirectional training is performed using BILSTM layers to further extract deep semantic features and long-distance dependency feature vectors for generating text content.
[0020] The feature vector is represented as , Feature vectors of representative characters ; Indicates dimension.
[0021] A multi-task learning method for crime prediction that integrates action words, which integrates each word of the case description content. Corresponding eigenvector representation Input the boundary classifier and predict the probability of each character becoming the boundary of the action word using the Sigmoid function;
[0022] Assuming the boundary classifier predicts The character marks the beginning boundary of the action word. If the boundary index number is used, then the maximum span value is preset. ,by The index position of the character is the start position of the span, and different end boundary indices are combined. Generate span ,in Indicates the first A continuous sequence of characters ;
[0023] Similarly, if the end boundary of the action word is predicted, then different start boundaries are combined to generate the span;
[0024] The generated spans are merged into a candidate span set. ,in It refers to the number of spans;
[0025] The generated candidate span set Identification is performed using a span classifier, and then the span is... The feature vector corresponding to the index is represented by MaxPooling. ,Will The span of the starting boundary vector and the final boundary vector splicing Feature vectors representing the span;
[0026] Input the span representation into the span classifier to obtain the probability that each span is a behavior word.
[0027] A multi-task learning method for crime prediction that integrates action words is proposed. For each node in the graph, a two-layer graph convolutional neural network is used to aggregate the feature information of the node's neighbors and related nodes to obtain the feature representation vector of the node.
[0028] Assuming the boundary recognition module identifies The span of each behavioral word;
[0029] Each span Corresponding character feature vector Feature fusion within the span is performed using max pooling. Each span is represented as a feature vector of a node in the graph.
[0030] The node feature vectors of each span are concatenated to form the initial node features of the graph. ,use of Represent its graph structure, so that Representative node To the node There exists an edge such that the edges between different words are... Assign different weights;
[0031] node feature vectors and adjacency matrix Input a graph convolutional network to obtain the final representation of the nodes.
[0032] The final representation of the input node is then fed into the Softmax layer to obtain the probability of the node's crime prediction. .
[0033] A multi-task learning method for crime prediction that integrates action words, the method includes a feature extraction module, an action word span generation module, and a crime prediction module;
[0034] The feature extraction module uses BERT pre-training to obtain an abstract semantic representation of the text information for the case description in the judgment document, and then uses BiLSTM to obtain the context semantic dependency features in the text.
[0035] The behavior word span generation module locates the position of the behavior word in the text by identifying the boundary of the behavior word based on the semantic dependency features of the following text obtained by the feature extraction module; and generates the span of the behavior word by combining different boundaries.
[0036] The crime prediction module predicts the crime by integrating the correlation information between the identified behavioral words.
[0037] The beneficial effects of this invention are:
[0038] This invention incorporates behavioral word feature information to predict the corresponding crime, and constructs the correlation and content structure between behavioral words through graph convolutional neural networks.
[0039] This invention uses a multi-task framework to share parameters to improve the model's generalization ability and recognition performance. Attached Figure Description
[0040] Figure 1 This is a schematic diagram of the multi-task learning model for crime prediction based on fused action words of the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Abstract semantic representations of text information are obtained through BERT, and contextual semantic dependency features in the text are acquired using BiLSTM. Secondly, the text content is long, making it impossible to enumerate the span. Therefore, boundary recognition methods are used to locate the boundaries of action words, and the span of action words is generated by combining these boundaries. Finally, a graph is constructed using the identified action words as nodes, and a graph convolutional neural network is used to capture the correlation between different words and node information, thus transforming the crime prediction problem based on court documents into a graph node classification problem.
[0043] A multi-task learning method for crime prediction that integrates action words, specifically including the following steps:
[0044] Step 1: Use BERT pre-training to obtain an abstract semantic representation of the text information in the case description of the judgment document;
[0045] Step 2: Based on the abstract semantic representation of the text information in Step 1, use BiLSTM to obtain the contextual semantic dependency features in the text;
[0046] Step 3: Based on the contextual semantic dependency features from Step 2, locate the position of the action word in the text by identifying the boundary of the action word;
[0047] Step 4: Based on the boundary of the action word located in Step 3, the span of the action word is generated by combining different boundaries;
[0048] Step 5: Based on the span of behavioral words identified in Step 4, predict the corresponding crime by constructing the correlation information between behavioral words, and improve the generalization and learning ability of the multi-task learning model for crime prediction by using a multi-task framework to share parameters.
[0049] A multi-task learning method for crime prediction that integrates action words, wherein step 1 specifically involves using the BERT model to obtain dynamic word vector representations that integrate contextual information for the case description in the judgment document and outputting continuous dense vectors that integrate the semantics of the entire text.
[0050] The BERT model's pre-training process uses a large amount of unlabeled text to predict randomly masked text. After self-supervised training, it can learn the general linguistic features of the text.
[0051] A multi-task learning method for crime prediction that integrates action words, wherein step 2 specifically involves training a continuous dense vector that integrates the semantics of the entire text bidirectionally through a BILSTM layer to further extract deep semantic features and features of long-distance dependency generated text content.
[0052] BiLSTM overcomes the problem that traditional RNNs cannot handle long-term dependencies by introducing a gate structure. BiLSTM can further obtain bidirectional semantic dependencies and long-distance features.
[0053] A multi-task learning method for crime prediction incorporating action words is described. Step 4 specifically addresses the issue that, due to the length of the case details and the sparse nature of action words, enumeration would result in numerous low-quality spans and high computational complexity. This invention locates the position of action words in the text by identifying individual characters as the start or end boundaries. Instead of using sequence labeling to detect boundaries, this invention employs two boundary classifiers to predict start and end positions, and then classifies the spans generated by combining different boundary positions. This allows the generated spans to focus on the vicinity of the action words and reduces the computational cost of classifying the spans.
[0054] A multi-task learning method for crime prediction by fusing action words is disclosed. Step 5 specifically involves the following: Since the extracted action words are discrete in information and lack relevance and structural features, this invention constructs a graph structure using action words as nodes. Through a graph convolutional neural network, semantic information is fused between action words to establish a word structure and enhance the relevance between them. Simultaneously, a multi-objective framework is used to establish relevance between different tasks. Utilizing a multi-task model for both action word extraction and crime prediction allows for improved model performance through cross-task parameter sharing. By sharing parameters across tasks, the model can learn more effective text features and exhibit better abstraction capabilities, enabling a single model to adapt to multiple related but different tasks.
[0055] The dataset used in this invention comes from 699 criminal judgments from the China Judgments Online website. This dataset consists of the case details of the criminal judgments, and we annotated the case details with behavioral terms. The dataset covers 12 crimes, with a total of 6384 behavioral terms annotated. In terms of dataset processing, the crime prediction dataset was divided into a training set, a validation set, and a test set in a 6:2:2 ratio.
[0056] The evaluation criteria for action word recognition tasks include accuracy (P), recall (R), and F1 score, where F = 2*P*R / (P+R). For crime prediction, which is a multi-class task, the evaluation metric used is the micro-average (Micro-F).
[0057] A multi-task learning method for crime prediction that integrates action words, with the case description text being... ,in Indicates the first in the text Each character;
[0058] Will The BERT model outputs a vector after passing through multiple Transformer layers. .
[0059] A multi-task learning method for crime prediction that integrates action words, Bidirectional training is performed using BILSTM layers to further extract deep semantic features and long-distance dependency feature vectors for generating text content.
[0060] The feature vector is represented as , Feature vectors of representative characters ; Indicates dimension.
[0061] A multi-task learning method for crime prediction that integrates action words, which integrates each word of the case description content. Corresponding eigenvector representation Input the boundary classifier and predict the probability of each character becoming the boundary of the action word using the Sigmoid function;
[0062] Assuming the boundary classifier predicts The character marks the beginning boundary of the action word. If the boundary index number is used, then the maximum span value is preset. ,by The index position of the character is the start position of the span, and different end boundary indices are combined. Generate span ,in Indicates the first A continuous sequence of characters ;
[0063] Similarly, if the end boundary of the action word is predicted, then different start boundaries are combined to generate the span;
[0064] The generated spans are merged into a candidate span set. ,in It refers to the number of spans;
[0065] The generated candidate span set The purpose of using a span classifier is to categorize the span into corresponding labels. We then further classify the span... The feature vector corresponding to the index is represented by MaxPooling. Furthermore, the boundary information for each span is crucial for identifying action words. Therefore, we will... Span start boundary vector and the final boundary vector splicing Feature vectors representing the span;
[0066] Input the span representation into the span classifier to obtain the probability that each span is a behavior word.
[0067] A multi-task learning method for crime prediction that integrates action words is proposed. A graph structure is constructed using the span of action words as nodes. A graph convolutional neural network is then used to fuse semantic information between action words, establishing a word structure and enhancing the relevance between them. For each node in the graph, a two-layer graph convolutional neural network is used to aggregate feature information from nearby and related nodes to obtain the feature representation vector of that node.
[0068] Assuming the boundary recognition module identifies The span of each behavioral word;
[0069] Each span Corresponding character feature vector Feature fusion within the span is performed using max pooling. Each span is represented as a feature vector of a node in the graph.
[0070] The node feature vectors of each span are concatenated to form the initial node features of the graph. ,use of To represent its graph structure, it is usually made Representative node To the node There exists an edge, but the edges between action words may have different degrees of association. Therefore, we need to consider the edges between different words... Different weights are assigned to distinguish different dependencies. In order to maintain the relationships between data and obtain the feature information of its own nodes, the adjacency matrix is normalized and self-links are added.
[0071] node feature vectors and adjacency matrix Input a graph convolutional network to obtain the final representation of the nodes.
[0072] The final representation of the input node is then fed into the Softmax layer to obtain the probability of the node's crime prediction. The model achieves better abstraction capabilities through shared representations, and allows a single model to adapt to multiple related but different tasks. A multi-objective framework is used to achieve end-to-end document-level action word extraction and crime prediction.
[0073] A multi-task learning method for crime prediction that integrates action words, the method includes a feature extraction module, an action word span generation module, and a crime prediction module;
[0074] The feature extraction module uses BERT pre-training to obtain an abstract semantic representation of the text information for the case description in the judgment document, and then uses BiLSTM to obtain the context semantic dependency features in the text.
[0075] The behavior word span generation module locates the position of the behavior word in the text by identifying the boundary of the behavior word based on the semantic dependency features of the following text obtained by the feature extraction module; and generates the span of the behavior word by combining different boundaries.
[0076] The crime prediction module predicts the crime by fusing correlation information between the located behavioral words.
Claims
1. A multi-task learning method for crime prediction that integrates action words, characterized in that, The aforementioned multi-task learning method for crime prediction specifically includes the following steps: Step 1: Use BERT pre-training to obtain an abstract semantic representation of the text information in the case description of the judgment document; Step 2: Based on the abstract semantic representation of the text information in Step 1, use BiLSTM to obtain the contextual semantic dependency features in the text; Step 3: Based on the contextual semantic dependency features from Step 2, locate the position of the action word in the text by identifying the boundary of the action word; Step 4: Based on the boundary of the action word located in Step 3, the span of the action word is generated by combining different boundaries; Step 5: Based on the behavioral word span identified in Step 4, predict the corresponding crime by constructing correlation information between behavioral words; Step 5 specifically involves constructing a graph structure using action words as nodes, fusing semantic information between action words through a graph convolutional neural network, establishing a structure between words, enhancing the correlation between action words, and simultaneously establishing the correlation between different tasks using a multi-objective framework. The multi-task model of extracting action words and predicting crimes can improve the performance of the model by sharing parameters across tasks.
2. The multi-task learning method for crime prediction based on fused action words according to claim 1, characterized in that, Step 1 specifically involves using the BERT model to obtain dynamic word vector representations that integrate contextual information for the case description in the judgment document and outputting continuous dense vectors that integrate the semantics of the entire text.
3. The multi-task learning method for crime prediction based on fused action words according to claim 2, characterized in that, Step 2 specifically involves training the continuous dense vector that integrates the full-text semantics bidirectionally through a BILSTM layer to further extract deep semantic features and features of long-distance dependency in generating text content.
4. The multi-task learning method for crime prediction based on fused action words according to claim 1, characterized in that, Step 4 specifically involves locating the position of a behavior word in the text by identifying a single character as the start or end boundary of the behavior word, and then classifying the span generated by combining different boundary positions.
5. The multi-task learning method for crime prediction based on fused action words according to claim 2, characterized in that, The case description text is as follows ,in Indicates the first in the text Each character; Will The BERT model outputs a vector after passing through multiple Transformer layers. .
6. The multi-task learning method for crime prediction based on fused action words according to claim 5, characterized in that, Will Bidirectional training is performed using BILSTM layers to further extract deep semantic features and long-distance dependency feature vectors for generating text content. The feature vector is represented as , Feature vectors of representative characters ; Indicates dimension.
7. The multi-task learning method for crime prediction based on fused action words according to claim 6, characterized in that, Every word in the case description Corresponding eigenvector representation Input the boundary classifier and predict the probability of each character becoming the boundary of the action word using the Sigmoid function; Boundary classifier predicts The character marks the beginning boundary of the action word. If the boundary index number is used, then the maximum span value is preset. ,by The index position of the character is the start position of the span, and different end boundary indices are combined. Generate span ,in Indicates the first A continuous sequence of characters ; Similarly, by predicting the end boundary of the action word, different start boundaries can be combined to generate the span; The generated spans are merged into a candidate span set. ,in It refers to the number of spans; The generated candidate span set Identification is performed using a span classifier, and then the span is... The feature vector corresponding to the index is represented by MaxPooling. ,Will The span of the starting boundary vector and the final boundary vector splicing Feature vectors representing the span; Input the span representation into the span classifier to obtain the probability that each span is a behavior word.
8. A multi-task learning method for crime prediction based on fused action words according to claim 1 or 7, characterized in that, For each node in the graph, a two-layer graph convolutional neural network is used to aggregate the feature information of the node's neighbors and related nodes to obtain the feature representation vector of that node. The boundary recognition module identified The span of each behavioral word; Each span Corresponding character feature vector Feature fusion within the span is performed using max pooling. Each span is represented as a feature vector of a node in the graph. The node feature vectors of each span are concatenated to form the initial node features of the graph. ,use of Represent its graph structure, so that Representative node To the node There exists an edge such that the edges between different words are... Assign different weights; node feature vectors and adjacency matrix Input a graph convolutional network to obtain the final representation of the nodes. The final representation of the input node is then fed into the Softmax layer to obtain the probability of the node's crime prediction. .
9. A multi-task learning system for crime prediction that integrates action words, characterized in that, The system uses a multi-task learning method for crime prediction that integrates behavior words as described in any one of claims 1-7, and the system includes a feature extraction module, a behavior word span generation module, and a crime prediction module. The feature extraction module uses BERT pre-training to obtain an abstract semantic representation of the text information for the case description in the judgment document, and then uses BiLSTM to obtain the context semantic dependency features in the text. The behavior word span generation module locates the position of a behavior word in the text by identifying the boundaries of the behavior word based on the semantic dependency features of the following text obtained by the feature extraction module; and generates the span of the behavior word by combining different boundaries. The crime prediction module predicts the corresponding crime by fusing the correlation information between the identified behavioral words. The crime prediction module specifically constructs a graph structure using action words as nodes, integrates semantic information between action words through a graph convolutional neural network, establishes the structure between words, enhances the correlation between action words, and uses a multi-objective framework to establish the correlation between different tasks. By utilizing a multi-task model that combines the task of extracting action words and predicting crimes, the performance of the model can be improved by sharing parameters across tasks.