Case judgment prediction method and system based on bert hidden layer information and medium

Abstract

The application provides a case judgment prediction method and system based on BERT hidden layer information and a medium, relates to the technical field of legal services, and comprises the following steps: S1, obtaining case text original data, and preprocessing the case text original data to obtain preprocessed case text; S2, performing word segmentation on the preprocessed case text to obtain preprocessed segmented case text; S3, encoding the preprocessed segmented case text according to a Chinese dictionary of BERT to obtain word encoding, and finally obtaining word vectors of the full text; S4, constructing a case prediction model and training the case prediction model to obtain a trained case prediction model; and S5, inputting the word vectors of the full text and the word vectors of the word pairs into the trained case prediction model to obtain prediction results of relevant legal articles, crimes and prison terms. The application can solve the problem of insufficient information utilization of a network model and improve the accuracy of task prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of legal services, in particular to a criminal case judgment prediction method based on BERT hidden layer information, and more particularly to a criminal case judgment prediction method and system based on BERT hidden layer information and a medium. BACKGROUND

[0002] In the prior art, the judgment and supervision of criminal cases are mostly determined by legal professionals on a case-by-case basis. This judgment form is low in efficiency and needs to provide a fair and objective, efficient, and high-accuracy judgment method.

[0003] The patent for invention with publication number CN110222866A discloses an intelligent civil case prediction system and method combining oral description and question and answer, which includes the following steps: S1. receiving oral case description input by a user; S2. determining the user's consultation intention according to the oral case description; S3. detecting whether the feature content is complete according to the consultation intention, if yes, executing step S4, otherwise prompting the user to supplement the corresponding feature content; S4. calling a prediction model to output the corresponding consultation result to the user according to the complete feature content. This prediction method can predict cases according to their features, but it involves manual participation in the entire process, cannot achieve complete fairness and objectivity, and is limited by the number of features.

[0004] The patent for invention with publication number CN110969276A discloses a judgment prediction method, a judgment prediction model obtaining method and device, obtains a case description text of a to-be-predicted case; performs word segmentation on the case description text to obtain a word sequence; obtains a matrix composed of word vectors of each word in the word sequence; inputs the matrix into a preset judgment prediction model to obtain judgment prediction information output by the preset judgment prediction model. This method can predict legal judgments, but it does not grasp the correlation between multiple sub-tasks, and the network structure used is relatively simple, and the accuracy needs to be improved.

[0005] In view of the defects in the prior art, the present application aims to provide a criminal case judgment prediction method using a deep learning network architecture, which is fair and objective and improves accuracy. SUMMARY

[0006] In view of the defects in the prior art, the present application provides a criminal case judgment prediction method, system and medium based on BERT hidden layer information.

[0007] According to the criminal case judgment prediction method, system and medium based on BERT hidden layer information provided by the present application, the scheme is as follows:

[0008] In a first aspect, a case judgment prediction method based on BERT hidden layer information is provided, and the method comprises the following steps:

[0009] In step S1, original case text data is obtained, and the original case text data is preprocessed to obtain preprocessed case text.

[0010] In step S2, the preprocessed case text is segmented to obtain preprocessed segmented case text.

[0011] In step S3, the preprocessed segmented case text is encoded according to a BERT Chinese dictionary to obtain word encoding, and finally a word vector of the whole text is obtained.

[0012] In step S4, a case prediction model is constructed and trained to obtain a trained case prediction model.

[0013] In step S5, the word vector of the whole text and the word vector of the word pair are input into the trained case prediction model to obtain a prediction result of relevant legal provisions, crime, and prison term.

[0014] Preferably, the step S1 comprises denoising the original case text data to obtain denoised case text.

[0015] Preferably, the case prediction model in step S4 comprises a deep learning network model.

[0016] The deep learning network model comprises a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit, and a relevant legal provision prediction unit.

[0017] The BERT information extraction unit arranges the word vector of the whole text obtained in step S3 into a matrix in order, then passes through a word embedding layer of deep learning, and then passes through n layers of encoding layers, the output of each encoding layer is used as the output of the next layer, and finally the hidden layer output of n layers is obtained, these outputs are in the form of vectors, and these vectors contain the main information of the whole text.

[0018] The attention mechanism information generation unit obtains the weight of the hidden layer output of n layers obtained by the BERT information extraction unit through an attention mechanism, adds the hidden layer outputs according to the weight to obtain a final information generation vector.

[0019] The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name.

[0020] The crime period prediction unit predicts the crime name by generating the information vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, converts the information vector through the linear layer into a new feature vector with the same dimension as the crime period prediction requirement, and the position of the maximum value of the feature vector is the predicted value of the crime period.

[0021] The relevant legal provision prediction unit predicts the crime name by generating the information vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, converts the information vector through the linear layer into a new feature vector with the same dimension as the relevant legal provision prediction requirement, and the position of the maximum value of the feature vector is the predicted value of the relevant legal provision.

[0022] Preferably, the attention mechanism information generation unit specifically comprises:

[0023] According to the attention mechanism, the matrix Q, K, and V relationship between the nth layer hidden layer output and other layer outputs obtained by the BERT information extraction unit is obtained, and the formula is as follows:

[0024] Q=YX Q

[0025] K=XW K

[0026] V=XW V

[0027] Wherein, X is the nth layer hidden layer output, Y is the other layer hidden layer output, W Q , W K , W V is an initialized matrix with dimension, and the hidden layer output corresponds;

[0028] Based on the Q, K, and V matrices, the weight of each hidden layer is calculated according to the attention mechanism, and the attention mechanism formula is as follows:

[0029]

[0030] Wherein, K T represents the transpose of matrix K; d is the dimension of matrix Q;

[0031] Finally, according to the weight obtained by the softmax, the hidden layer output is multiplied by the corresponding weight to obtain the final information generation vector.

[0032] Secondly, a case judgment prediction system based on BERT hidden layer information is provided, and the system comprises:

[0033] Module M1: obtaining case text original data, and preprocessing the case text original data to obtain preprocessed case text;

[0034] Module M2: the pre-processed case text is segmented to obtain a pre-processed segmented case text;

[0035] Module M3: the pre-processed segmented case text is encoded according to a Chinese dictionary of BERT to obtain word encoding, and finally a word vector of the whole text is obtained;

[0036] Module M4: a case prediction model is constructed and trained to obtain a trained case prediction model;

[0037] Module M5: the word vector of the whole text and the word pair word vector are input into the trained case prediction model to obtain a prediction result of relevant legal provisions, crimes and prison terms.

[0038] Preferably, the module M1 comprises: denoising the case text original data to obtain a denoised case text.

[0039] Preferably, the case prediction model in the module M4 comprises: a deep learning network model;

[0040] The deep learning network model comprises: a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit and a relevant legal provision prediction unit;

[0041] The BERT information extraction unit arranges the word vector of the whole text obtained in step S3 into a matrix in order, then passes through a word embedding layer of deep learning, and then passes through n layers of encoding layers, the output of each encoding layer is used as the output of the next layer, finally the hidden layer output of n layers is obtained, these outputs are in the form of vectors, and these vectors contain the main information of the whole text;

[0042] The attention mechanism information generation unit obtains the respective weights of the hidden layer outputs of n layers obtained by the BERT information extraction unit through an attention mechanism, and adds the hidden layer outputs according to the weights to obtain a final information generation vector;

[0043] The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name;

[0044] The prison term prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name;

[0045] The relevant legal provision prediction unit predicts the crime by the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, and converts the information generation vector into a new feature vector with the same dimension as the relevant legal provision prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the relevant legal provision.

[0046] Preferably, the attention mechanism information generation unit specifically comprises:

[0047] According to the attention mechanism, the matrix Q, K, V relationship of the nth layer hidden layer output and the output of other layers obtained by the BERT information extraction unit is obtained, and the formula is as follows:

[0048] Q=YX Q

[0049] K=XW K

[0050] V=XW V

[0051] Wherein, X is the output of the nth hidden layer, Y is the output of the other hidden layer, W Q , W K , W V is the initialized matrix, the dimension, the hidden layer output corresponds;

[0052] Based on the Q, K, V matrix, the weight of each hidden layer is calculated according to the attention mechanism, and the attention mechanism formula is as follows:

[0053]

[0054] Wherein, K T Indicates the transpose of matrix K; d is the dimension of matrix Q;

[0055] Finally, according to the weight obtained by softmax, the hidden layer output is multiplied by the corresponding weight and summed to obtain the final information generation vector.

[0056] In the third aspect, a computer readable storage medium storing a computer program is provided, and the computer program is executed by a processor to realize the steps in the method.

[0057] Compared with the prior art, the present application has the following beneficial effects:

[0058] 1、The present application is based on BERT hidden layer information, attention mechanism algorithm for multiple task prediction of legal text case description, which aims to analyze the case description text of the judicial field, and according to the historical court ruling, the case of the judicial field is predicted;

[0059] 2、The application makes full use of the BERT pre-training network structure with wide applicability and high accuracy in the field at present, and fully utilizes the output of each hidden layer, while obtaining the key information of the surface and deep layers of the text;

[0060] 3、The application adds the most popular attention mechanism in the field of natural language processing at present, focuses on the relationship between each layer of the network, solves the problem of insufficient information utilization of the network model, and greatly improves the accuracy of various task predictions. BRIEF DESCRIPTION OF DRAWINGS

[0061] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0062] Figure 1 Flow chart for criminal case judgment prediction;

[0063] Figure 2 Overall framework diagram of the criminal case judgment network based on BERT hidden layer information of the application;

[0064] Figure 3 BERT network structure schematic diagram. DETAILED DESCRIPTION

[0065] The application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be pointed out that, for those skilled in the art, without departing from the concept of the application, a number of changes and improvements can be made. These all belong to the protection scope of the application.

[0066] The embodiment of the application provides a case judgment prediction method based on BERT hidden layer information, researches criminal case judgment prediction technology, and aims to analyze the case description text of the judicial field, and predicts the case of the judicial field according to the historical court ruling and judgment case. Referring to Figure 1 The specific steps of the method are as follows:

[0067] Step S1: Obtain the case text original data, and pre-process the case text original data to obtain the pre-processed case text; in this step, the pre-processing is denoising processing of the case text original data, to obtain the denoised case text.

[0068] Step S2: The pre-processed case text is segmented, and the pre-processed segmented case text is obtained;

[0069] Step S3: The preprocessed case text is encoded according to the Chinese dictionary of BERT to obtain word encoding, and finally the word vector of the whole text is obtained;

[0070] Step S4: A case prediction model is constructed and trained to obtain a trained case prediction model.

[0071] The case prediction model in this step includes a deep learning network model. The deep learning network model includes a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit, and a related legal provision prediction unit.

[0072] The BERT information extraction unit arranges the word vector of the whole text obtained in step S3 into a matrix in order, then passes through a word embedding layer of deep learning, and then passes through 12 encoding layers. The output of each encoding layer is used as the output of the next layer, and finally the hidden layer output of 12 layers is obtained. These outputs are in the form of vectors, and these vectors contain the main information of the whole text.

[0073] The attention mechanism information generation unit obtains the respective weights of the 12-layer hidden layer outputs obtained by the BERT information extraction unit through the attention mechanism, and adds the hidden layer outputs according to the weights to obtain the final information generation vector. Specifically, first, the matrix Q, K, V relationship of the 12th layer hidden layer output and other layer outputs obtained by the BERT information extraction unit is calculated according to the attention mechanism, and the formula is as follows:

[0074] Q=YW Q

[0075] K=XW K

[0076] V=XW V

[0077] Wherein, X is the nth hidden layer output, Y is the other layer hidden layer output, W Q , W K , W V is an initialized matrix with a dimension of hidden layer output corresponding.

[0078] According to the Q, K, V matrix, the weight of each hidden layer can be calculated according to the attention mechanism, and the attention mechanism formula is as follows:

[0079]

[0080] Wherein, K T represents the transpose of matrix K; d is the dimension of matrix Q;

[0081] Finally, the hidden layer output is multiplied by the corresponding weight summed up according to the weight obtained by the softmax to obtain the final information generation vector.

[0082] The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the crime name prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the crime name.

[0083] The crime period prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the crime period prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the crime period.

[0084] The relevant legal provision prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the relevant legal provision prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the relevant legal provision.

[0085] Step S5: input the word vector of the full text and the word pair word vector into the trained case prediction model to obtain the prediction results of the relevant legal provisions, the crime and the crime period.

[0086] The embodiment of the application also provides a case judgment prediction system based on BERT hidden layer information, and the system is specifically as follows:

[0087] Module M1: obtain case text original data, and pre-process the case text original data to obtain pre-processed case text; in this step, the pre-processing is denoising processing on the case text original data to obtain denoised case text.

[0088] Module M2: perform word segmentation on the pre-processed case text to obtain pre-processed segmented case text.

[0089] Module M3: the pre-processed segmented case text is encoded according to the Chinese dictionary of BERT to obtain word encoding, and finally obtain the word vector of the full text.

[0090] Module M4: construct a case prediction model and train the case prediction model to obtain a trained case prediction model.

[0091] The case prediction model in this step comprises a deep learning network model. The deep learning network model comprises a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit, and a related legal provision prediction unit.

[0092] The BERT information extraction unit arranges the full-text word vectors obtained in step S3 into a matrix in order, then passes them through a word embedding layer of deep learning, and then through 12 encoding layers, the output of each encoding layer being used as the output of the next layer, and finally obtains 12 hidden layer outputs, which are in the form of vectors and contain the main information of the full text.

[0093] The attention mechanism information generation unit obtains the respective weights of the 12 hidden layer outputs obtained by the BERT information extraction unit through an attention mechanism, and adds the hidden layer outputs according to the weights to obtain a final information generation vector. Specifically, first, the matrix Q, K, and V relationship of the 12th hidden layer output and other layer outputs obtained by the BERT information extraction unit is solved according to the attention mechanism, and the formula is as follows:

[0094] Q=YX Q

[0095] K=XW K

[0096] V=XW V

[0097] wherein X is the nth hidden layer output, Y is the other layer hidden layer output, W Q , W K , and W V are initialized matrices with dimensions corresponding to the hidden layer outputs.

[0098] According to the Q, K, and V matrices, the weights of each hidden layer can be calculated according to the attention mechanism, and the formula of the attention mechanism is as follows:

[0099]

[0100] wherein K T represents the transpose of the matrix K; d is the dimension of the matrix Q;

[0101] Finally, the hidden layer outputs are multiplied by the corresponding weights obtained by the softmax and summed to obtain the final information generation vector.

[0102] The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the crime name prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the crime name.

[0103] The crime period prediction unit predicts the crime name by the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the crime period prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the crime period.

[0104] The relevant legal provision prediction unit predicts the crime name by the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model. The information generation vector is converted into a new feature vector with the same dimension as the relevant legal provision prediction requirement through the linear layer, and the position of the maximum value of the feature vector is the predicted value of the relevant legal provision.

[0105] Module M5: input the word vector of the full text and the word pair word vector into the trained case prediction model to obtain the prediction results of the relevant legal provisions, the crime, and the crime period.

[0106] Next, the application will be described in more detail.

[0107] As shown in Figure 1 , the application provides a case judgment prediction method based on BERT hidden layer information. It includes network model structure, input and output, etc.

[0108] As shown in Figure 2 , the encoded full text word vector is put into the BERT network to obtain the output of each layer, which is then weighted and added according to the attention mechanism, and finally the final prediction result is obtained through the linear layer.

[0109] The attention mechanism operation schematic diagram is as shown in Figure 3 , which aims to obtain the weights of the 12-layer hidden layer outputs obtained by the BERT information extraction unit, and add the hidden layer outputs according to the weights to obtain the final information generation vector. Specifically, first, the Q, K, V relationship between the 12th layer hidden layer output and other layer outputs obtained by the BERT information extraction unit is calculated according to the attention mechanism, and the formula is as follows:

[0110] Q=YW Q

[0111] K=XW K

[0112] V=XW V

[0113] Wherein, X is the 12th layer hidden layer output, Y is the other layer hidden layer output, W Q , W K , W Vis the initialized matrix, the dimension and the hidden layer output correspond.

[0114] According to the Q, K, V matrix, the weight of each hidden layer can be calculated according to the attention mechanism, and the attention mechanism formula is as follows:

[0115]

[0116] Wherein, d is the dimension of the matrix Q.

[0117] According to the weight obtained by the softmax, the hidden layer output is multiplied by the corresponding weight to obtain the final information generation vector. Finally, the final prediction result about the legal charge, the relevant legal provisions and the prison term is obtained through the linear layer.

[0118] The embodiment of the present application selects the currently commonly used Chinese legal text data set CAIL-2108 to generate a data set, which is used as the training data of the network proposed in the present application. The experimental results show that the accuracy of the prediction of each task reaches the SOTA level in the industry.

[0119] The test results of the embodiment show that the present application has strong commercial value on the basis of simple network design and without the need for complex manual feature selection.

[0120] The embodiment of the present application provides a case judgment prediction method and system based on BERT hidden layer information and a medium, which predicts multiple tasks of the case description of legal text based on BERT hidden layer information and attention mechanism algorithm, aims to analyze the case description text of the case in the judicial field, and predicts the case in the judicial field according to the historical court ruling and judgment case; fully use the BERT pre-training network structure with wide applicability and high accuracy in the field at present, and fully utilize the output of each hidden layer, and obtain the key information of the surface and deep layers of the text; the present application also adds the most popular attention mechanism in the field of natural language processing, focuses on the relationship between each layer of the network, solves the problem of insufficient information utilization of the network model, and greatly improves the accuracy of the prediction of each task.

[0121] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module and unit thereof in the form of pure computer readable program code, the system provided by the present application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps to achieve the same functions. Therefore, the system provided by the present application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for achieving various functions can also be considered as structures within the hardware component; the devices, modules and units for achieving various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0122] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other in any manner without conflict.

Claims

1. A case judgment prediction method based on BERT hidden layer information, characterized in that, The method comprises the following steps: Step S1: obtaining case text original data, and preprocessing the case text original data to obtain preprocessed case text; Step S2: performing word segmentation on the preprocessed case text to obtain preprocessed segmented case text; Step S3: encoding the preprocessed segmented case text according to a Chinese dictionary of BERT to obtain word encoding, and finally obtaining word vectors of the whole text; Step S4: constructing a case prediction model and training the case prediction model to obtain a trained case prediction model, wherein the case prediction model comprises a deep learning network model, and the deep learning network model comprises a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit and a related legal provision prediction unit; Step S5: inputting the word vectors of the whole text and the word vectors of the word pairs into the trained case prediction model to obtain prediction results of related legal provisions, crimes and prison terms; The BERT information extraction unit arranges the word vectors of the whole text into a matrix in a sequence, then passes through a word embedding layer of deep learning, and then passes through n encoding layers, the output of each encoding layer is used as the output of the next layer, finally the n-layer hidden layer output is obtained, these outputs are in the form of vectors, and these vectors contain the main information of the whole text; The attention mechanism information generation unit obtains the weights of the n-layer hidden layer outputs obtained by the BERT information extraction unit through an attention mechanism, and adds the hidden layer outputs according to the weights to obtain a final information generation vector; The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name; The prison term prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name; The related legal provision prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through a linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name. 2.The BERT hidden layer information-based case judgment prediction method of claim 1, wherein, The step S1 comprises denoising the case text original data to obtain denoised case text. 3.The BERT hidden layer information-based case judgment prediction method of claim 1, wherein, The attention mechanism information generation unit comprises: The relationship between the nth hidden layer output and the outputs of other layers obtained by the BERT information extraction unit is calculated according to the attention mechanism, and the formula is as follows: wherein, is the output of the n-th hidden layer, is the output of the other hidden layer, is an initialized matrix of dimension, corresponding to the hidden layer output; The weights of each hidden layer are calculated according to the attention mechanism based on the Q, K and V matrices, and the attention mechanism formula is as follows: wherein denotes the transpose of the matrix d is the dimension of the matrix Q; Finally, the hidden layer outputs are multiplied by the corresponding weights summed to yield the final information generation vector. derived weights. 4.A case judgment prediction system based on BERT hidden layer information, characterized in that, The method comprises the following steps: The module M1: obtains case text original data, and pre-processes the case text original data to obtain pre-processed case text; The module M2: carries out word segmentation on the pre-processed case text to obtain pre-processed word segmented case text; The module M3: encodes the pre-processed word segmented case text according to the Chinese dictionary of BERT to obtain word encoding, and finally obtains the word vector of the full text; The module M4: constructs a case prediction model and trains the case prediction model to obtain a trained case prediction model; The module M5: inputs the word vector of the full text and the word pair word vector into the trained case prediction model to obtain a prediction result of relevant legal provisions, crimes and prison terms; The BERT information extraction unit arranges the full text word vector obtained in step S3 into a matrix in order, then passes through a word embedding layer of deep learning, and then passes through n layers of encoding layers, the output of each encoding layer is used as the output of the next layer, and finally the hidden layer output of n layers is obtained, these outputs are in the form of vectors, and these vectors contain the main information of the full text; The case prediction model in the module M4 comprises a deep learning network model; The deep learning network model comprises a BERT information extraction unit, an attention mechanism information generation unit, a crime name prediction unit, a prison term prediction unit and a relevant legal provision prediction unit; The attention mechanism information generation unit obtains the respective weights of the hidden layer outputs of n layers obtained by the BERT information extraction unit through the attention mechanism, and adds the hidden layer outputs according to the weights to obtain a final information generation vector; The crime name prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name; The prison term prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name; The relevant legal provision prediction unit predicts the crime name of the information generation vector obtained by the attention mechanism information generation unit through the linear layer of the deep learning network model, converts the information generation vector into a new feature vector with the same dimension as the crime name prediction requirement dimension through the linear layer, and the position of the maximum value of the feature vector is the prediction value of the crime name. 5.The BERT hidden layer information based case decision prediction system of claim 4, wherein, The module M1 comprises denoising processing on the case text original data to obtain denoised case text. 6.The BERT hidden layer information based case decision prediction system of claim 4, wherein, The attention mechanism information generation unit specifically comprises: The relationship between the nth hidden layer output and the outputs of other layers obtained by the BERT information extraction unit is obtained according to the attention mechanism, and the formula is as follows: wherein, is the output of the n-th hidden layer, is the output of the other hidden layer, is an initialized matrix of dimension, corresponding to the hidden layer output; Based on the Q, K and V matrices, the weights of each hidden layer are calculated according to the attention mechanism, and the attention mechanism formula is as follows: wherein, denotes the transpose of the matrix d is the dimension of the matrix Q; Finally, the hidden layer outputs are multiplied by the corresponding weights summed to yield the final information generation vector. derived weights.

7. A computer readable storage medium storing a computer program, characterized in that, The computer program, which is executed by a processor, implements the steps of the method according to any one of claims 1 to 3.

Images