Intelligent law analysis method and system based on semantic graph model

Abstract

To provide an intelligent law analysis method and system based on a semantic graph model that efficiently provides users with services such as legal precedent feature extraction and marching.SOLUTION: An intelligent law analysis method based on a semantic graph model includes steps of: GPT language processing for adopting a deep learning model Transform network with a self-attention mechanism to convert an input legal text into a high-dimensional feature vector that can be processed by a computer; graph neural network feature optimization for optimizing the high-dimensional feature vector generated by a GPT language processing module, and realizing connection between nodes by constructing a semantic and knowledge relation so as to generate a law semantic feature, and converting text data into graph representation; and downstream task utilization for accepting the graph representation as input, and then outputting a corresponding result.SELECTED DRAWING: Figure 1

Claims

1. A bill intelligent analysis method based on a semantic graph model, comprising: It includes three steps: a GPT language processing step, a graph neural network feature optimization step, and a downstream task step; In the GPT language processing step, a deep learning model Transformer network with a self-attention mechanism is used to convert the input legal text into a high-dimensional feature vector that can be processed by a computer; In the graph neural network feature optimization step, the high-dimensional feature vector by the GPT language processing module is further optimized and improved, and the connection between the nodes is realized by building the relationship between semantics and knowledge, thereby generating bill semantic features, and converting the text data into a graphic representation; In the downstream task utilization step, the graphic representation in the graph neural network feature optimization step is received as input, and then the corresponding result is output, thereby realizing the understanding and processing of legal text and the analysis of legal precedents.

2. The method for intelligent bill analysis based on a semantic graph model according to claim 1 , wherein the downstream tasks include bill comprehension, bill summarization, bill preparation, bill compliance, bill simulation, and bill management.

3. The GPT language processing module performs supervised tasks based on the GPT-2 framework with unsupervised pre-trained models, and its core lies in language modeling, which is a form of unsupervised distribution estimation. Natural language text is sequence data composed of a set of words or phrases of variable length, and since language is naturally sequenced, the joint probability of symbols is decomposed into a product of conditional probabilities; [0010] Here, p(s) is the probability that the language model generates phrase s, and the length of phrase s after word segmentation is N. The conditional probability in the above formula is the probability of generating the first i-1 (2≦i≦N) words s in the language model. １ , …, s ｉ－１ When we give the i-th word probability estimate p(s ｉ |s １ , …, s ｉ－１ 2. The method for intelligent analysis of bills based on a semantic graph model according to claim 1, further comprising:

4. The method for intelligent analysis of bills based on semantic graph models, as described in claim 3, further comprising: for each sub-block of the standard Transformer model, placing a layer of regularization for each layer before each sub-block, adding a layer of regularization layer after the last self-attention region, expanding the size of the context to 1024 tokens, and correcting to use a batchsize of 512.

5. In the GPT language processing step, before inputting the text into the model, a quantization process is performed, and then the text is divided into words, a thesaurus is constructed, and then the token is mapped as an ID. The method for intelligent analysis of bills based on a semantic graph model as claimed in claim 4, characterized in that in the step of quantization processing, the input Chinese text is converted into Unicode, unrepresentable characters are deleted, and space characters are converted into displayable space characters, and then spaces are added before and after the Chinese characters, and words are split by spaces.

6. The Transformer uses a self-attention mechanism to process long sequences of sentences at the input, modeling dependencies and serially connecting different positions of a sequence so that each word can establish weighted dependencies on itself and other words at the reconstruction time. The weights are calculated as follows: Each word vector is divided into one query vector q ｉ and one key vector k ｉ and q ｉ And calculate the similarity or correlation of all key vectors, and normalize it with probability estimation through Softmax; [0025] Here, sim ｉｊ =q ｉ Ｔ k ｊ and a ｉｊ is the i-th query vector q ｉ and the j-th key vector k ｊ is the attention coefficient of , e is a natural number, T denotes the matrix transpose, In the third step, the Attention operation in the Transformer is performed. [0030] The query, key, and value vector sums of each word in the sentence are spliced ​​into a matrix and parallel operations are performed, where [0045] is a matrix representation of the query vector, [0050] is a matrix representation of the key vector, [006] is a matrix representation of the value vector, [0070] is the matrix representation of the output vector, and d ｋ The method for intelligent analysis of bills based on a semantic graph model according to claim 5, wherein is the number of dimensions of the query, key and value vectors.

7. The method for intelligent analysis of bills based on a semantic graph model as claimed in claim 1, characterized in that the graph neural network feature optimization includes classifying texts based on the text features extracted in the text analysis part and calculating the similarity between texts, specifically, first performing a linear transformation using a shared weight matrix on the input embedding, then using a self-attention mechanism for each adjacent node, and finally normalizing all adjacent nodes of a node using a softmax function, thereby giving a stable distribution to the output of each node.

8. A multi-head attention mechanism with multiple trainable attention vectors is used to calculate attention coefficients and node feature convolution for each vector separately, and after calculating the output structure, a nonlinear transformation of the mean function and activation function is performed; [0080] Here, W ｋ is the linear layer weight of the kth head, the superscript k denotes the index of the head in the multi-head attention mechanism, σ is a nonlinear activation function, and h ｉ is the input word vector of the i-th word, and h ｉ ’ The method for intelligent analysis of bills based on semantic graph model according to claim 7, wherein i is the output word vector of the i-th word.

9. Downstream task utilization steps include feature extraction and task adaptation; Feature extraction includes three steps: feature selection, which selects optimal features from the preprocessed data; feature extraction, which extracts the most representative features from the selected features; and feature representation, which represents the extracted features as vectors. The method for intelligent analysis of bills based on a semantic graph model as claimed in claim 1, characterized in that task adaptation includes two steps: task classification, which divides downstream tasks into different categories and processes them using different models for each category, and feature adjustment, which adjusts the extracted features according to the downstream tasks.

10. A system for intelligent analysis of bills based on a semantic graph model, which is used to implement the method according to any one of claims 1 to 9, a GPT language processing module that converts input legal text into a high-dimensional feature vector that can be processed by a computer; A graph neural network feature optimization module further optimizes and improves the high-dimensional feature vectors from the GPT language processing module to convert the text data into a graphical representation; and a downstream task utilization module for receiving as input the graphic representation in the graph neural network feature optimization step, and then outputting corresponding results, thereby realizing legal text understanding and processing, and legal precedent analysis.

Images