Criminal period prediction method based on sentencing rule knowledge graph driving

Abstract

The invention discloses a criminal period prediction method based on sentencing rule knowledge graph driving, and belongs to the technical field of criminal period prediction, and the method specifically comprises the steps: analyzing laws and regulations and guidance suggestions related to case sentencing, constructing a sentencing rule knowledge graph from the top to the bottom from a mode layer to a data layer, and summarizing and concluding a sentencing thinking chain; according to thinking chain guidance, structured data prompt engineering of the knowledge graph is used for structuring and labeling the legal documents of the criminal case, and an SSP instruction fine adjustment data set is generated; and finally, performing supervised instruction fine tuning on the large language model by adopting a low-rank adaptive LoRA parameter efficient fine tuning strategy, generating a criminal period prediction model, and constructing an interpretable criminal period prediction system driven by the sentencing rule knowledge graph. The system combines a case input by a user, completes entity recognition and retrieval through a sentencing rule knowledge graph, realizes generation retrieval enhancement of a criminal period prediction model by utilizing prompt assembly, and optimizes a criminal period prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of sentence prediction, and in particular to a sentence prediction method driven by a sentencing rule knowledge graph. Background Art

[0002] In the field of criminal justice, artificial intelligence technology is widely used in sentencing prediction to overcome human bias and improve the quality and efficiency of case handling. However, traditional intelligent sentencing methods mainly rely on classification, regression and neural network algorithms to deduce sentencing results through legal facts. The "black box" properties of these algorithms lead to a lack of interpretability in the prediction process, and their legal trust level is highly sensitive and uncertain, resulting in an inherent conflict between predictive justice and judicial justice.

[0003] In order to deal with such contradictions, the Knowledge Graph (KG) has gradually become a key tool for describing case elements and their correlations, providing basic support for the interpretability of sentencing prediction results. With the rapid development of large language models (LLM), legal big models have performed well in areas such as legal prediction, question answering, and text generation, and have therefore become the focus of intelligent sentencing research. However, most legal big models are based on general big models and are built through instruction fine-tuning and incremental training. They face problems such as opaque generation mechanisms and generated hallucinatory texts. In addition, legal big models lack an in-depth understanding of judges' adjudicative thinking, and the texts they generate are still difficult to fully present and effectively support the complete logical system of legal judgments.

[0004] To sum up, the application of knowledge graph technology alone has significant deficiencies in the logical reasoning of sentencing decisions, while relying solely on large language models is prone to model hallucinations, unable to accurately express legal language and sentencing logic, and difficult to apply in criminal justice scenarios; based on this, the present invention proposes a sentence prediction method driven by a sentencing rule knowledge graph. Summary of the invention

[0005] The purpose of this invention is to provide a sentence prediction method driven by a sentencing rule knowledge graph to solve the following technical problems: Knowledge graph technology has significant deficiencies in the logical reasoning of sentencing decisions; large language models are prone to model hallucinations, cannot accurately express legal language and sentencing logic, and are difficult to apply in criminal justice scenarios.

[0006] The purpose of the present invention can be achieved through the following technical solutions: A sentence prediction method based on sentencing rule knowledge graph drive includes the following steps: S1: Based on the criminal legal terminology set and sentencing guidelines, the basic framework of the sentencing rules knowledge graph is defined. The basic framework includes entities, attributes, relationships and knowledge tags. According to the definition of the basic framework, the annotation system of the sentencing rules knowledge graph is designed; S2: Collect the laws, regulations and guidelines on sentencing involved in the case, and perform knowledge extraction based on the BERT-BiLSTM-CRF model. The knowledge extraction includes entity recognition and relationship recognition. The entity recognition is based on the multi-label classification model of BERT-BiLSTM-CRF. The output of the multi-label classification model is used as the input of relationship recognition. The relationship recognition is based on the single-label classification model. S3: Evaluate the results of knowledge extraction based on the precision P, recall R and F1 value, align the knowledge extracted after the evaluation, and complete the construction of the sentencing rule knowledge graph; store the constructed sentencing rule knowledge graph, and generate a visualization interface of the sentencing rule knowledge graph; S4: Collect historical criminal case verdicts, use knowledge graph-based structured data prompting engineering to structure and annotate historical criminal case verdicts, generate SSP instruction fine-tuning datasets, and use the LoRA parameter efficient fine-tuning strategy based on the SSP instruction fine-tuning dataset to perform supervised instruction fine-tuning on the large language model, generate a sentence prediction model, and evaluate the generation efficiency of the sentence prediction model; S5: The sentence prediction model receives case information input by the user in real time, performs entity recognition on the case information through the sentencing rule knowledge graph, extracts key entities, retrieves legal provisions and sentencing rules related to the key entities in the knowledge graph, converts the retrieved legal provisions and sentencing rules into natural language, assembles the converted sentencing knowledge together with the case information into structured prompts to guide the large language model, and generates sentence prediction results using the large language model fine-tuned with instructions.

[0007] As a further solution of the present invention: the specific process of entity recognition is: Each sentence in the laws, regulations and guidelines for sentencing is input into the model and first processed by the BERT model to generate a word vector representation T containing contextual semantics; this representation is composed of the position vector T p , word vector T t and segment vector T s Together, T=T p +T t +T sThe BiLSTM model performs feature extraction on the word vector T generated by BERT to capture the contextual dependencies in the sequence. The output of BiLSTM is passed to the CRF model. The CRF model predicts the continuity and validity of labels by learning the adjacent relationships between labels, and outputs the corresponding BIO labels. Based on the BIO labeling results, the knowledge extraction of the sentencing rule knowledge graph is completed according to the rules.

[0008] As a further solution of the present invention: the specific process of the knowledge alignment is: The semantic similarity matching algorithm based on Siamese network is used to calculate the semantic similarity between entities and merge entities with similar semantics. The similarity calculation formula is: ; Where A and B represent the text of the input entity, respectively, and f(A) and f(B) represent the semantic vectors encoded by the shared neural network.

[0009] As a further solution of the present invention: the specific process of generating a visual interface of the sentencing rule knowledge graph is as follows: The legal entities and relationships in the sentencing rules knowledge graph are converted into nodes and edges in the Neo4j graph, and nodes and relationships are created through the Cypher query language. The built-in visualization tools of Neo4j are used to generate a visualization interface of the knowledge graph.

[0010] As a further solution of the present invention: the calculation formula for evaluating the results of knowledge extraction based on the precision P, recall R and F1 value is: ; ; ; Where i represents the type of category in the classification task; TP i is the number of samples correctly classified into category i; FP i Indicates the number of samples that are actually other classes but are classified as class i by the classifier; FN i is the number of samples that are actually in the i-th category but are incorrectly classified by the classifier. When F1 is less than the preset threshold, it means that the knowledge extraction result meets the evaluation requirements.

[0011] As a further solution of the present invention: the specific process of evaluating the generation efficiency of the sentence prediction model is as follows: The similarity between the text generated by the model and the reference text is calculated by BLEU to evaluate the accuracy of the model in text generation. The ratio of the content in the reference text covered by the text generated by the model is calculated by ROUGE-L to evaluate the recall rate of the model. The calculation formula of BLEU is: ; ; ; In the formula, CP n (C,S) represents the matching accuracy, c i To generate text, s ij For reference text, h k (c i ) and h k (s ij ) represent the number of times one or more consecutive words appear in the generated text and the reference text respectively; l c is the total length of the generated text, l s is the total length of the reference text ω n is the weight; b(C,S) is the penalty mechanism, which is used to prevent short sentences from getting high scores; The calculation formula of ROUGE-L is: ; Where LCS(output,s) represents the length of the longest common subsequence between the output and evaluation text, and length(s) represents the length of the evaluation text.

[0012] As a further solution of the present invention: collecting historical criminal case judgments, and using the knowledge graph-based structured data prompting project to structure and annotate the historical criminal case judgments. The specific process is: Initialize the structured prompt words as a copy of the prompt word template, use named entity recognition to extract key entities from historical criminal case judgments, retrieve legal provisions and sentencing guidelines related to the key entities in the sentencing rules knowledge graph, and if relevant legal provisions and sentencing guidelines are retrieved in the sentencing rules knowledge graph, map the retrieved legal provisions and sentencing guidelines to the corresponding placeholders in the prompt word template, and update the corresponding part of the structured prompt words with the retrieved knowledge; if there are unfilled template placeholders, fill them based on general legal principles or default values; output the mapped and filled structured prompt words.

[0013] Beneficial effects of the present invention: The present invention efficiently constructs a sentencing rule knowledge graph from legal texts based on the BERT-BiLSTM-CRF model. At the data level, a thinking chain is designed based on sentencing guidelines, and an SSP data set is independently constructed. At the model level, a large language model is fine-tuned based on the sentencing rule knowledge graph instructions, and then the sentencing model is generated and retrieved to enhance the accuracy and interpretability of the case sentence prediction results, ultimately obtaining an interpretable sentence prediction system driven by the sentencing rule knowledge graph. The system constructed by the present invention has significantly improved the prediction accuracy of sentence prediction tasks compared to the mainstream baseline model, and the application of methods such as thinking chain and knowledge graph retrieval enhancement has effectively improved the interpretability of sentencing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be further described below in conjunction with the accompanying drawings.

[0015] Figure 1 It is a flow chart of a sentence prediction method driven by a sentencing rule knowledge graph according to the present invention; Figure 2 It is a flow chart of constructing the knowledge graph of sentencing rules of the present invention; Figure 3 It is the knowledge graph ontology framework of sentencing rules of the present invention; Figure 4 This is the entity recognition flow chart of the BERT-BiLSTM-CRF model of the present invention; Figure 5 It is a flowchart of fine-tuning the large model based on the sentencing rule knowledge graph of the present invention; Figure 6 It is a flow chart of the sentencing knowledge graph retrieval enhanced sentence prediction model of the present invention; Figure 7 This is a comparison diagram of the loss rate change during the fine-tuning process of the large language model instruction of the present invention; Figure 8 It is a comparison chart of the fine-tuning results of different large language model instructions of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1-Figure 8 As shown, the present invention is a sentence prediction method driven by a sentencing rule knowledge graph, including the following embodiments: Embodiment 1:

[0018] Please refer to Figure 1 As shown in the figure, the overall process of the sentence prediction method driven by the sentencing rule knowledge graph is as follows: Figure 1 The method consists of two main parts: one is the construction of the sentence prediction model, and the other is user question-answering interaction.

[0019] The sentencing rules knowledge graph is the core of this method, which drives the construction of the sentence prediction model and the question-answering interaction process. In the sentence prediction model construction part, the laws, regulations and guidelines related to the sentencing of the case are analyzed, and the sentencing rules knowledge graph is constructed from top to bottom from the model layer to the data layer, and the sentencing thinking chain is summarized. According to the guidance of the thinking chain, the structured data prompt engineering of the knowledge graph is used to structure and annotate the legal documents of criminal cases, and the SSP (Sentencing Standards Prediction) instruction fine-tuning dataset is generated. Finally, the low-rank adaptation (LoRA) parameter efficient fine-tuning strategy is adopted to perform supervised instruction fine-tuning on the large language model, generate a sentence prediction model, and construct an interpretable sentence prediction system driven by the sentencing rules knowledge graph.

[0020] In the user question-and-answer interaction part, the system combines the case information input by the user, completes entity recognition and retrieval through the sentencing rule knowledge graph, and uses prompt assembly to realize the generated retrieval enhancement (Retrieval-Augmented Generation, RAG) of the sentence prediction model to optimize the sentence prediction results.

[0021] 1. The construction of the sentencing rules knowledge graph is divided into two parts: the model layer and the data layer. The process is as follows: Figure 2 shown.

[0022] At the model layer, we first define the basic framework of the knowledge graph based on the criminal legal terminology set and sentencing guidelines, including the definitions of entities, attributes, relationships, and knowledge tags, and design the graph annotation system accordingly to ensure the accuracy and logical connection of knowledge representation.

[0023] At the data layer, based on the defined model layer structure, the BERT-BiLSTM-CRF model is used to extract knowledge from the laws, regulations and guidelines related to sentencing. After aligning the extracted knowledge, the sentencing rule knowledge graph is constructed. Finally, the graph forms a sentencing rule knowledge base through storage and visualization modules, and supports the continuous updating of knowledge.

[0024] 1.1. The core of the model layer construction is to design the ontology framework of the sentencing rules knowledge graph. At present, the ontology construction in the legal field is generally divided into three categories: judicial documents, laws and regulations, and specific tasks. However, in terms of sentencing, legal norms, guidelines, etc. have described the sentencing rules in detail, but their abstractness limits practical applications. To this end, the present invention designs a knowledge graph ontology based on legal norms, especially the "Sentencing Guidelines", and adds the fourth element of "attribute" to the traditional triples to further refine the sentencing rules, thereby improving the practicality and accuracy of the knowledge graph. The knowledge graph ontology framework of sentencing rules is as follows: Figure 3 shown.

[0025] The knowledge graph ontology framework contains 15 entities, 13 groups of relationships, and 12 attributes. In addition, the present invention summarizes the chain of thinking that judges should follow in the sentencing process based on the "Guiding Opinions on Sentencing". That is, the starting point of sentencing is determined according to the crime and the number of crimes; the benchmark sentence is determined according to the number of crimes, the amount of crime, and the consequences of the crime; the sentence is determined according to various sentencing circumstances, and the consideration of fines and suspended sentences is added.

[0026] 1.2. The construction of the data layer of the sentencing rules knowledge graph includes two main steps: knowledge extraction and knowledge alignment. Knowledge extraction identifies sentencing-related entities and their relationships from legal norms, forming the core foundation of the graph data; knowledge alignment ensures the consistency of the graph data, solves the integration problem across data sources through the alignment of entities and relationships, and realizes the standardized processing of graph content.

[0027] The present invention builds a BERT-BiLSTM-CRF model to complete the knowledge extraction task. The model consists of three parts: BERT, BiLSTM and CRF. Among them, the BERT model is an encoder based on a multi-layer bidirectional Transformer, which can learn the feature representation of the domain data set through pre-training of large-scale corpus. The BiLSTM

[31] model consists of forward and backward long short-term memory networks, which is mainly used to extract the global context features of the text and can efficiently capture bidirectional semantic information. The CRF model is a probabilistic graph model that is used to capture the label dependency in the sequence labeling task, thereby improving the accuracy of label prediction.

[0028] The knowledge extraction task uses the method of joint extraction of entities and relations. First, a multi-label classification model based on BERT-BiLSTM-CRF is built for entity recognition; then the output of the model is used as the input for relation recognition to build a single-label classification model. Through this pipeline connection, information transfer between models is achieved, thereby collaboratively completing the knowledge extraction task.

[0029] Please refer to Figure 4As shown in the figure, the process of entity recognition is as follows: each sentence is input into the model and first processed by the BERT model to generate a word vector representation T containing contextual semantics; this representation is represented by the position vector T p , word vector T t and segment vector T s Together, T=T p +T t +T s The BiLSTM model performs feature extraction on the word vector T generated by BERT to capture the contextual dependencies in the sequence. The output of BiLSTM is passed to the CRF model. The CRF model predicts the continuity and validity of labels by learning the adjacent relationships between labels, and outputs the corresponding BIO labels. Based on the BIO labeling results, the knowledge extraction of the sentencing rule knowledge graph is completed according to the rules.

[0030] The goal of entity alignment is to unify entities that refer to the same object in different data sources into the same identifier. Due to differences in the expression methods in legal norms and judicial documents, the same entity may appear multiple times in the graph. For example, "theft" is expressed as "theft" or "stealing" in some documents. In order to solve the problem of inconsistent naming, the present invention adopts a semantic similarity matching algorithm based on a Siamese network to merge entities with similar semantics by calculating the semantic similarity between entities. The similarity calculation formula is: ; Where A and B represent the text of the input entity, respectively, and f(A) and f(B) represent the semantic vectors encoded by the shared neural network.

[0031] After alignment, various expressions of "theft" in different source files will be uniformly identified as the same entity to ensure the uniqueness of the entity in the graph.

[0032] Relation alignment aims to identify and unify the same or similar relationships in different data sources, ensuring the accuracy and consistency of the relationship chain in the knowledge graph. For example, different documents may use "sentence" or "rule" to express the same legal judgment. To achieve relationship alignment, the present invention adopts a rule-based automatic alignment method to identify synonymous relationships through preset semantic rules and perform standardized processing.

[0033] The present invention uses Neo4j graph database to realize efficient storage and visual display of sentencing rule knowledge graph. During the storage process, the legal entities and relationships in the sentencing rule knowledge graph are converted into nodes and edges in Neo4j, and nodes are created and relationships are constructed through Cypher query language. Subsequently, Neo4j's built-in visualization tool is used to intuitively present the knowledge graph, and users can clearly browse the legal entities and their relationships.

[0034] 2. The fusion architecture of the sentencing rule knowledge graph and the large language model provides strong technical support for the realization of explainable sentence prediction. By combining the legal rules contained in the knowledge graph with the deep semantic understanding ability of the large language model, the accuracy and explainability of sentence prediction can be significantly improved. This fusion architecture enhances the performance of the large model in two main ways: first, by optimizing the instruction fine-tuning of the large language model through the knowledge graph, and second, by enhancing the reasoning ability of the large model through knowledge graph retrieval.

[0035] 2.1. This invention uses structured data prompt engineering to deeply integrate the sentencing rule knowledge graph into the fine-tuning of the large language model, thereby improving the instruction fine-tuning effect. The specific process is as follows: Figure 5 shown.

[0036] The sentencing rules knowledge graph provides case-related laws and regulations, sentencing guidelines, and selected case information, generates instruction templates containing sentencing thinking chain step guidance, and clarifies the basic principles, methods, and sentencing standards corresponding to the circumstances. The instruction fine-tuning dataset is generated through automated annotation and manual proofreading, and the LoRA strategy is applied to fine-tune the large language model. The rich structured information provided by the knowledge graph helps the model accurately understand and apply legal rules during the fine-tuning process, thereby improving the accuracy and interpretability of the prediction results. After multiple rounds of testing and evaluation, a sentence prediction system is built based on the fine-tuned large language model.

[0037] 2.2. The second step of “graph-model fusion” is to combine the sentencing rule knowledge graph with the big model through the retrieval enhancement mechanism. The specific process is as follows: Figure 6 shown.

[0038] First, the user inputs the basic case information, and the system extracts key elements such as the crime, the amount involved, and the consequences of the crime through the entity recognition module. These entity information are then linked and retrieved in the sentencing rule knowledge graph to obtain relevant legal provisions and sentencing knowledge. Then, the retrieved sentencing knowledge is converted into natural language and integrated into the input template through the prompt assembly module. The template contains legal provisions and sentencing rules related to the case circumstances, providing structured input guidance for the large model.

[0039] In the sentence prediction model, the optimized prompt information further guides the model to make accurate sentencing predictions. By introducing the retrieval enhancement of the knowledge graph, the large model not only generates more accurate sentence prediction results, but also provides the basis for each step of the analysis process, enhancing the interpretability of the prediction results. Ultimately, the sentence prediction results generated by the system include all steps of sentencing, such as the starting point of sentencing, adjustment of the benchmark sentence, and specific circumstances of mitigation or aggravation of punishment, providing users with clear basis for judgment. Embodiment 2:

[0040] The SSP instruction fine-tuning dataset is constructed based on the criminal case judgments published on the China Judgment Documents Network from 2020 to 2023, with a total of 8,900 judgment documents. In terms of the causes of the cases, it covers 21 types of criminal cases, such as traffic accidents, dangerous driving, illegal absorption of public deposits, and fund-raising fraud. In terms of trial procedures, it covers criminal first-instance cases, criminal second-instance cases, and cases of changes in penalties and executions. The regions cover more than 20 provinces, including Jiangsu, Anhui, Hubei, Guangxi, and Sichuan, involving provincial, municipal, and county-level judicial and procuratorial organs.

[0041] In the data preprocessing stage, a structured data prompting project based on knowledge graph is adopted, following the standardized format of "instruction-input-output".

[0042] 1. Experimental environment and parameter settings: To avoid the influence of other parameter settings, this experiment was conducted under the same conditions. The operating system used in the experiment was Ubantu 18.04.6 LTS, and the hardware configuration was Intel Xeon Gold 6330 processor, 256 GB running memory, 40 TB storage space, and four NVIDIA A40 48 GB graphics cards. During the experiment, the inference method of single-machine multi-card parallel computing was used to improve the efficiency of model training.

[0043] In the construction of the knowledge graph, the number of Transformer layers of the BERT-BiLSTM-CRF model is set to 12, the maximum length is limited to 512, the hidden layer dimension is 768, the Adam optimizer is used, the learning rate is set to 0.001, the number of epochs is 32, the dropout rate is set to 0.05, and the lstm_dim parameter value is 128.

[0044] In instruction supervised fine-tuning, considering the length of the reference text in the instruction fine-tuning dataset, the maximum length of the sequence after word segmentation is limited to 1024, and the batch size is set to 4, which will be repeated 3 times per batch in a total of 100,000 steps of training. The AdamW optimizer uses a learning rate of 0.00005 and adjusts the learning rate through a cosine scheduling strategy after a warm-up period of 10,000 steps to ensure the smoothness of weight updates. In order to improve efficiency, mixed precision training (fp16) is used. The LoRA configuration includes 16 layers of low-rank updates and a learning rate multiplier of 0.1. The optimization model adapts to new tasks while maintaining the original knowledge. In order to ensure that the model fully utilizes the data and ensures the objectivity of the evaluation results, the experiment randomly divides the SSP dataset into a training set and a test set in a ratio of 9:1, and the proportion of the validation set is set to 10%.

[0045] 2. Evaluation indicators: This paper uses three indicators, namely, precision (P), recall (R), and F1 value, to evaluate the effect of the model on entity and relationship recognition and sentence prediction tasks: ; ; ; Where i represents the type of category in the classification task; TP i is the number of samples correctly classified into category i; FP i Indicates the number of samples that are actually other classes but are classified as class i by the classifier; FN i is the number of samples that are actually in the i-th category but are incorrectly classified by the classifier. When F1 is less than the preset threshold, it means that the knowledge extraction result meets the evaluation requirements.

[0046] In addition, in order to evaluate the generation efficiency of the sentence prediction model obtained after instruction supervision fine-tuning, the present invention calculates the similarity between the model-generated text and the reference text through BLEU to evaluate the accuracy of the model in text generation, and calculates the ratio of the content in the reference text covered by the model-generated text through ROUGE-L to evaluate the recall rate of the model, where the calculation formula of BLEU is: ; ; ; In the formula, CP n (C,S) represents the matching accuracy, c i To generate text, s ij For reference text, h k (c i ) and h k (s ij ) represent the number of times one or more consecutive words appear in the generated text and the reference text respectively; l c is the total length of the generated text, l s is the total length of the reference text ω n is the weight; b(C,S) is the penalty mechanism, which is used to prevent short sentences from getting high scores; The calculation formula of ROUGE-L is: ; Where LCS(output,s) represents the length of the longest common subsequence between the output and evaluation text, and length(s) represents the length of the evaluation text. Embodiment three:

[0047] 1. In order to evaluate the performance of the entity relationship extraction model, instruction fine-tuning large language model and the final interpretable sentence prediction model used in the sentencing rule knowledge graph constructed in the present invention, the present invention designed three sets of comparative experiments. First, traditional machine learning models such as HMM, LSTM-CRF, BiLSTM-CRF, BERT and BERT-BiLSTM entity relationship recognition models were selected as baseline models and compared with the BERT-BiLSTM-CRF model used in the present invention. Secondly, after analyzing the OpenLLM large model evaluation list, four mainstream large language models, GLM-4-9B-Chat, LLaMA-3-8B-Chinese-Chat, Qwen-2-7B and Baichuan2-7B-Chat, were selected. Based on the instruction fine-tuning method of optimizing the large language model based on the knowledge graph proposed in the present invention, the SSP dataset was used for experiments and evaluations. Finally, the model proposed in this paper is compared with three baseline models. The selected baseline models include the word embedding-based deep learning model MTL-Fusion, the Chinese judicial long text document pre-training model Lawformer, and the pre-training model BERT, to comprehensively evaluate the model performance differences.

[0048] The evaluation results of the BERT-BiLSTM-CRF model and the baseline model are shown in the following table.

[0049]

[0050] In the evaluation of various entity relationship extraction models, it can be found that the BERT model shows significant advantages over the traditional HMM model, mainly due to its ability to capture the characteristics of characters, words, and relationships between words and sentences, which improves the generalization ability of the model. In addition, by comparing BERT with the BERT-BiLSTM-CRF structure, it is found that the latter performs better in knowledge extraction performance. This improvement is due to the fact that the BERT-BiLSTM-CRF model effectively combines the advantages of BERT and traditional models. It not only inherits BERT's powerful ability in text feature extraction, but also extracts the correlation features of adjacent sentence labels by introducing BiLSTM and CRF layers, achieving global optimization of label sequence selection, thereby improving the accuracy of entity recognition.

[0051] 2. In order to further explore the optimal configuration parameters of the BERT-BiLSTM-CRF model, the present invention designed six groups of experiments, testing three different learning rates (0.001, 0.005, 0.01) and two training rounds (16 epochs and 32 epochs). Each group of experiments was repeated three times to eliminate accidental errors, and the average value was used as the final evaluation result to ensure the objectivity and accuracy of the results. The results are shown in the following table.

[0052]

[0053] Under the condition of learning rate of 0.001, the model performance improves significantly with the increase of training rounds. However, when the learning rate is increased to 0.005 and 0.01, although the model performance still improves slightly with the training rounds, the improvement is obviously limited. When the learning rate is 0.005, increasing the training rounds from 16 to 32 only increases the F1 score from 0.9502 to 0.9512; while when the learning rate is 0.01, the F1 score decreases slightly from 0.9495 to 0.9490. This shows that too high a learning rate may cause the model to converge prematurely, limiting the space for further optimization and making the model more likely to fall into a local optimal solution in the early stages of training. In addition, at a higher learning rate, the model is at risk of overfitting, and the increase in training rounds does not significantly improve the performance, but instead shows a slight decrease. This suggests that under high learning rate conditions, the model may have completed the learning of the training set at an early stage, and further increasing the rounds may aggravate the overfitting of the training data and affect its generalization ability on the test set.

[0054] 3. Changes in the loss rate of major language models during instruction-supervised fine-tuning training Figure 7 shown.

[0055] In general, the loss rate of each model drops rapidly at the beginning of training, indicating that the model can quickly capture effective patterns and reduce training errors. As the number of training steps increases, the loss rate decreases gradually, indicating that the model is gradually approaching the convergence state. Especially after 5000 steps, the loss curves of each model are basically stable, suggesting that the training effect is close to the optimal. Comparing the performance of each model, the initial loss rate of the LLaMA-3-8B-Chinese-Chat model is high, but the rate of decline is fast, and the final loss rate is close to that of other models. The loss curves of the GLM-4-9B-Chat and Qwen2-7B-Chat models are relatively smooth, reflecting the stability of the training process. The Baichuan2-7B-Chat model has a high loss rate in the early stage, and the loss decrease is the largest during the entire training process, indicating that it is highly sensitive to training data.

[0056] 4. The following table shows the results of instruction-supervised fine-tuning training of different large language models using the SSP dataset.

[0057]

[0058] Compared with other models, the LLama-3-8B-Chinese-Chat model has high evaluation accuracy and the lowest evaluation loss and training loss, which reflects that the model has obvious advantages in generalization ability and training effect after fine-tuning. The GLM-4-9B-Chat and Qwen-2-7B models also performed well, but the Baichuan2-7B-Chat model may be limited by the scale of model parameters and failed to fully learn the target task during the fine-tuning process, resulting in poor generalization ability, and further optimization or adjustment of the fine-tuning strategy is needed.

[0059] 5. The evaluation results of each fine-tuned model on the test set are shown in the following table.

[0060]

[0061] In general, the fine-tuning process effectively enhanced the model's adaptability to the sentencing prediction instruction task. The GLM-4-9B-Chat model performed well in four indicators. The BLEU-4 and ROUGE-1 scores after fine-tuning reached 67.63% and 79.87%, respectively, which were significantly higher than other models, showing strong language generation and text coverage capabilities. Although the LLaMA-3-8B-Chinese-Chat model did not perform as well as the GLM-4-9B-Chat in its original state, it was close to the GLM-4-9B-Chat in the ROUGE-2 and ROUGE-L indicators after fine-tuning, indicating its advantage in fine-grained text generation. Although the Qwen-2-7B model was limited by the volume of parameters and had a low original score, its various indicators were greatly improved after fine-tuning, showing good instruction following ability. In contrast, the Baichuan2-7B-Chat model had the smallest improvement, and its BLEU-4 and ROUGE series scores after fine-tuning were relatively low, indicating that its model structure had certain limitations in specific tasks. Figure 8 The comparison of fine-tuning results of each model instruction is shown. Based on the performance of the comprehensive model in the two stages of training and testing, the present invention selects the GLM-4-9B-Chat model as the basic large language model for the application method.

[0062] 6. The performance comparison of the interpretable sentence prediction model driven by the sentencing rule knowledge graph and the three benchmark models in the sentence prediction task is shown in the following table.

[0063]

[0064] The model proposed in this paper performs outstandingly in terms of accuracy, recall and F1 value in the task of sentence prediction, showing the high accuracy and stability of the model in sentence prediction. This advantage is mainly due to the introduction of the sentencing rule knowledge graph in the model construction process. By integrating legal knowledge and structured actual cases for training, the model can make full use of large-scale pre-training corpus when predicting, and at the same time combine specific knowledge in the legal field, thereby improving the pertinence and effectiveness of the prediction.

[0065] In contrast, the Lawformer model failed to outperform BERT and the model of the present invention in the comparative test because it adopted a relatively traditional architecture. Although the MTL-Fusion model is based on a multi-task learning framework and attempts to improve overall performance by optimizing the weight distribution between tasks, in the specific task of sentence prediction, the multi-task learning strategy is difficult to fully capture and respond to the complexity and specific needs of the legal field in terms of resource allocation, so its performance has certain limitations.

[0066] 7. The present invention designs a "graph-model fusion" architecture of knowledge graph and large language model, and efficiently constructs sentencing rule knowledge graph from legal text based on BERT-BiLSTM-CRF model. At the data level, it designs thinking chain based on sentencing guidelines and independently constructs SSP data set; at the model level, it fine-tunes GLM-4-9B-Chat large language model based on sentencing rule knowledge graph instructions, and then generates retrieval enhancement for sentencing model to improve the accuracy and interpretability of case sentence prediction results, and finally obtains an interpretable sentence prediction system driven by sentencing rule knowledge graph. Experimental results show that the system constructed by the present invention has significantly improved the prediction accuracy of sentence prediction tasks compared with the mainstream baseline model, and the application of thinking chain, knowledge graph retrieval enhancement and other methods effectively improves the interpretability of sentencing results.

[0067] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A sentence prediction method driven by sentencing rule knowledge graph, characterized in that: The following steps are involved: S1: Based on the criminal legal terminology set and sentencing guidelines, the basic framework of the sentencing rules knowledge graph is defined. The basic framework includes entities, attributes, relationships and knowledge tags. According to the definition of the basic framework, the annotation system of the sentencing rules knowledge graph is designed; S2: Collect the laws, regulations and guidelines on sentencing involved in the case, and perform knowledge extraction based on the BERT-BiLSTM-CRF model. The knowledge extraction includes entity recognition and relationship recognition. The entity recognition is based on the multi-label classification model of BERT-BiLSTM-CRF. The output of the multi-label classification model is used as the input of relationship recognition. The relationship recognition is based on the single-label classification model. S3: Evaluate the results of knowledge extraction based on precision P, recall R and F1 value, align the knowledge extracted after evaluation, and complete the construction of the sentencing rule knowledge graph; The constructed sentencing rule knowledge graph is stored, and a visualization interface of the sentencing rule knowledge graph is generated; S4: Collect historical criminal case verdicts, use knowledge graph-based structured data prompting engineering to structure and annotate historical criminal case verdicts, generate SSP instruction fine-tuning datasets, and use the LoRA parameter efficient fine-tuning strategy based on the SSP instruction fine-tuning dataset to perform supervised instruction fine-tuning on the large language model, generate a sentence prediction model, and evaluate the generation efficiency of the sentence prediction model; S5: The sentence prediction model receives case information input by the user in real time, performs entity recognition on the case information through the sentencing rule knowledge graph, extracts key entities, retrieves legal provisions and sentencing rules related to the key entities in the knowledge graph, converts the retrieved legal provisions and sentencing rules into natural language, assembles the converted sentencing knowledge together with the case information into structured prompts to guide the large language model, and generates sentence prediction results using the large language model fine-tuned with instructions.

2. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph driven is characterized in that: The specific process of entity recognition is as follows: Each sentence in the laws, regulations and guidelines for sentencing is input into the model and first processed by the BERT model to generate a word vector representation T containing contextual semantics; this representation is composed of the position vector T p 、Word vector T t and segment vector T s Together, T=T p +T t +T s ; The BiLSTM model extracts features from the word vector T generated by BERT to capture the contextual dependencies in the sequence; The output of BiLSTM is passed to the CRF model. The CRF model predicts the continuity and validity of labels by learning the adjacent relationship between labels, outputs the corresponding BIO labels, and completes the knowledge extraction of the sentencing rule knowledge graph according to the BIO labeling results and the rules.

3. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph drive is characterized in that: The specific process of the knowledge alignment is as follows: The semantic similarity matching algorithm based on Siamese network is used to calculate the semantic similarity between entities and merge entities with similar semantics. The similarity calculation formula is: ； Where A and B represent the text of the input entity, respectively, and f(A) and f(B) represent the semantic vectors encoded by the shared neural network.

4. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph drive is characterized in that: The specific process of generating the visual interface of the sentencing rule knowledge graph is as follows: The legal entities and relationships in the sentencing rules knowledge graph are converted into nodes and edges in the Neo4j graph, and nodes and relationships are created through the Cypher query language. The built-in visualization tools of Neo4j are used to generate a visualization interface of the knowledge graph.

5. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph drive is characterized in that: The calculation formula for evaluating the results of knowledge extraction based on precision P, recall R and F1 value is: ； ； ； In the formula, i represents the type of category in the classification task; TP i is the number of samples correctly classified into category i; FP i Indicates the number of samples that are actually other classes but are classified as class i by the classifier; FN i is the number of samples that are actually in the i-th category but are incorrectly classified by the classifier; when F1 is less than the preset threshold, it means that the knowledge extraction result meets the evaluation requirements.

6. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph drive is characterized in that: The specific process of evaluating the generation efficiency of the sentence prediction model is as follows: The similarity between the text generated by the model and the reference text is calculated by BLEU to evaluate the accuracy of the model in text generation. The ratio of the content in the reference text covered by the text generated by the model is calculated by ROUGE-L to evaluate the recall rate of the model. The calculation formula of BLEU is: ； ； ； In the formula, CP n (C,S) represents the matching accuracy, c i To generate text, s ij For reference text, h k (c i ) and h k (s ij ) represent the number of times one or more consecutive words appear in the generated text and the reference text respectively; l c is the total length of the generated text, l s is the total length of the reference text ω n is the weight; b(C,S) is the penalty mechanism, which is used to prevent short sentences from getting high scores; The calculation formula of ROUGE-L is: ； Where LCS(output,s) represents the length of the longest common subsequence between the output and evaluation text, and length(s) represents the length of the evaluation text.

7. According to claim 1, a sentence prediction method based on sentencing rule knowledge graph drive is characterized in that: The specific process of collecting historical criminal case verdicts and structuring and annotating them using the knowledge graph-based structured data prompting project is as follows: Initialize the structured prompt word as a copy of the prompt word template, use named entity recognition to extract key entities from historical criminal case judgments, retrieve legal clauses and sentencing guidelines related to the key entities in the sentencing rule knowledge graph, and if relevant legal clauses and sentencing guidelines are retrieved in the sentencing rule knowledge graph, map the retrieved legal clauses and sentencing guidelines to the corresponding placeholders in the prompt word template, and use the retrieved knowledge to update the corresponding part of the structured prompt word; If there are unfilled template placeholders, they are filled based on general legal principles or default values; Output the mapped and populated structured prompt words.

Images