A case source clue classification processing method based on hierarchical graph convolutional neural network model
By adopting the hierarchical graph convolutional neural network model in case source clue classification, and using multi-threshold sparse graph and hierarchical graph convolutional network, the problems of noise learning and depth limitation are solved, and the generalization ability and classification performance of the model are improved.
Patent Information
- Application Number
- CN202111341170.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-12
- Publication Date
- 2025-02-14
- Estimated Expiration
- 2041-11-12
AI Technical Summary
The existing technology has problems of noise learning and depth limitation in the classification of case source clues, resulting in poor classification results and insufficient generalization capabilities.
Using the case source clue classification processing method based on the hierarchical graph convolution neural network model, the construction of multi-threshold sparse graph and learning of hierarchical graph convolution networks is used, and the noise in the graph data is removed and the noise in the graph data is improved. Generalization ability of the model.
Effectively reduce the impact of graph data isomorphism, remove noise, retain important information in graph structure, and improve the generalization ability and classification performance of the model.
Smart Images
Figure CN114154022B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of case source clue classification and graph learning, and in particular to a hierarchical graph convolutional network framework including a layered graph convolutional network and a sparse graph integration. Background Art
[0002] The management of case source clues is the initial link in the industrial and commercial administrative law enforcement. The case supervision and management department implements centralized management, dynamic updates, regular summary and verification of problem clues, puts forward opinions on the division of work, reports to the principal person in charge of the disciplinary inspection agency for approval, and transfers them to the undertaking department according to procedures. Strengthening the intelligent construction of case source clue management can effectively improve the law enforcement efficiency of the industrial and commercial administrative departments, avoid errors in the law enforcement process to a certain extent, and promote the intelligent and modern construction of social governance capabilities.
[0003] With the surge in the number of case source clues, the existing traditional manual case source clue allocation and processing methods have been under great pressure. Under high-intensity work pressure, burnout may occur, and even random case source allocation may occur. The accuracy of case source clue allocation cannot be guaranteed, which will cause unnecessary trouble for further case investigation and handling. Therefore, the deep learning-based method can assist in clue classification, which effectively optimizes the shortcomings of manual classification methods, reduces the pressure of manual classification, and promotes the modernization of social governance capabilities.
[0004] Case source clue classification is essentially a multi-classification problem. The expressive power of traditional classification methods, such as Naive Bayes, needs to be improved, and the classification effect is poor. Deep learning methods, especially graph learning models, have shown superior performance in classification problems. However, there are still some problems with graph learning models themselves.
[0005] In recent years, in order to analyze and classify graph data, graph embedding and graph representation methods are traditionally used to automatically learn low-dimensional feature representations of graphs. This method can also maximize the preservation of network structure. At the same time, the high-level feature representation of deep convolutional neural networks has been proven to be superior to low-level features. However, graph convolutional neural networks focus mainly on the input graph data rather than the structure of the graph. In the process of classifying graph data, there is often a lot of useless noise. Although deep GCN can be used to learn the features of the graph, the noise in the graph will also be learned into the network, and at the same time, it will face the depth limitation of GCN learning. In addition, the classification of graph data also faces the problem of limited data sets, which will limit the performance of the classifier and reduce the generalization ability of the classifier. Summary of the invention
[0006] In view of the above problems, the present invention provides a case source clue classification processing method based on a hierarchical graph convolutional neural network model. The present invention can effectively combine the nodes and structural features of the graph, learn the intrinsic connection between graphs, and effectively remove the noise in the graph data, thereby improving the generalization ability of the model.
[0007] The present invention provides a case source clue classification processing method based on a hierarchical graph convolutional neural network model, comprising:
[0008] Step 1: Preprocess the case source data set and convert the case source clue text data into graph data containing text logical relationships;
[0009] Step 2: Construction of multi-threshold sparse graphs, using different thresholds to segment the graph data converted in step 1 and construct sparse graphs of different levels;
[0010] Step 3: For the sparse graph obtained in step 2, use the hierarchical graph convolutional network model Hi-GCN to learn the topological structure of the graph and the relationship between graphs;
[0011] Step 4: Use graphs with different thresholds to build multiple Hi-GCN models, obtain the results corresponding to different thresholds, and finally form the final case source classification results through integrated classifiers.
[0012] Furthermore, the data set preprocessing described in step 1 specifically involves converting the case source clue text data into graph data. Glove is used to encode text words, convert them into nodes in the graph, and calculate the similarity of the node homes to construct the edges in the graph.
[0013] Furthermore, the multi-threshold sparse graph construction described in step 2 is specifically to set multiple thresholds in the interval of [5%, 50%] for the edges of the original graph data, and obtain sparse graphs with different sparsity levels from the original graph data.
[0014] Furthermore, the hierarchical graph convolutional network described in step three is specifically composed of two sub-networks: a graph-level sparse graph learning model f-GCN and a node-level sparse graph correlation learning model p-GCN.
[0015] Furthermore, the f-GCN described in step 3 learns the graph representation of each sparse graph through graph clustering method and graph convolution feature learning, converts the sparse graph information into a coarsened graph, and then performs graph convolution feature operation to learn a low-dimensional vector. Specifically, a learnable weight matrix is set for the input sparse graph, and the sparse graph obtained in step 2 is fused into a coarsened graph with a lower dimension by matrix multiplication, and finally the result of the coarsened graph is embedded into a low-dimensional vector through GCN.
[0016] Furthermore, the specific steps of the p-GCN are to calculate the correlation between sparse graphs based on sparse graphs of different graph data under the same threshold using the graph kernel structure matching method, and then regard the low-dimensional graph feature expression obtained by f-GCN as the point of the association graph, construct an association graph, and perform GCN feature learning based on the association graph, and put the result into a fully connected network to obtain a classification result under a threshold.
[0017] Furthermore, the integrated classifier described in step 4 specifically puts the results calculated in step 3 under each threshold into the fully connected network to calculate the prediction results under a single threshold, and finally votes on the classification results of multiple thresholds to obtain the final classification result.
[0018] Furthermore, in step 3, the graph clustering method of f-GCN is to retain points in the coarsened graph formed by weighting the importance of each point, and the edges are obtained by multiplying the importance weighted value of all edges between new points by the weight of the original edge and finally adding them together.
[0019] Furthermore, in step 3, the method of constructing a new graph by p-GCN is to convert the graph representation of f-GCN into node features, and then calculate the structural similarity between points to construct edges, thereby obtaining an association graph.
[0020] Furthermore, the entire framework uses the cross entropy function in the f-CGN, p-GCN and graph clustering process in step 3 during training.
[0021] The beneficial effects provided by the present invention are:
[0022] The present invention can effectively reduce the impact of graph data heterogeneity and remove noise while effectively retaining important information in the graph structure, thereby improving the generalization ability and classification performance of the model.
[0023] The present invention can effectively combine the nodes and structural features of the graph. At the same time, you can learn the intrinsic connection between graphs and effectively remove the noise in the graph data, thereby improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying creative labor.
[0025] Figure 1 A business logic diagram of the present invention;
[0026] Figure 2 This is a schematic diagram of data preprocessing;
[0027] Figure 3 Architecture diagram of the sparse graph hierarchical graph convolutional network framework including layered graph convolutional networks and ensemble learning.
[0028] Figure 4 This is the learning process of the HI-GCN network. DETAILED DESCRIPTION
[0029] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations are obvious to those of ordinary skill in the art, and the embodiments are selected and described to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and design various embodiments with various modifications suitable for specific purposes.
[0030] The embodiment of the present invention provides a case source clue classification processing method based on a hierarchical graph convolutional neural network model, comprising:
[0031] Step 1: Preprocess the case source data set and convert the case source clue text data into graph data containing text logical relationships;
[0032] Step 2: Construction of multi-threshold sparse graphs, using different thresholds to segment the graph data transformed in step 1 and construct sparse graphs of different levels;
[0033] Step 3: For the sparse graph obtained in step 2, use the hierarchical graph convolutional network to learn the topological structure of the graph and the relationship between graphs;
[0034] Step 4: Use graphs with different thresholds to build multiple Hi-GCN models, obtain the results corresponding to different thresholds, and finally form the final case source classification results through integrated classifiers.
[0035] like Figure 1 As shown, it is implemented by the following steps:
[0036] 1. Preprocess the input text data, convert the divided words into high-dimensional vector representations according to the glove corpus, and construct graph data (such as Figure 2 shown).
[0037] The input graph data is processed. According to the maximum value of the graph data and the threshold interval [5%, 50%], different thresholds are set with a step size of 5% to obtain sparse graphs with various sparsity levels (such as Figure 3 shown).
[0038] More specifically, for the sparse process of input graph data, the actual process is shown in formula (1):
[0039]
[0040] Q(r i ,r j ) represents a weighted edge between two nodes in the graph, is the threshold value set, where The size of is determined by the maximum value of the weighted edges in the graph. The threshold range is [5%, 50%]. Multiple thresholds can be set for the same graph to produce multiple different sparse graphs to reflect different levels of the original connectivity network topology.
[0041] 2. Hierarchical Graph Convolutional Network Hi-GCN (model such as Figure 4 As shown in Figure 2). For each threshold-derived sparse graph, the sparse network of each subject and the corresponding overall network are derived. In order to better study the multi-scale nature of graph data, all sparse graphs with the same sparsity from other graphs are also processed in the same way to construct the association matrix.
[0042] 3. Build the f-GCN subnetwork in Hi-GCN for sparse graph representation learning. The purpose of f-GCN is to hide noisy connections by grouping similar nodes into a super node, thereby highlighting the indicative edges connecting super nodes. In other words, the weights of functional connections connecting nodes across different clusters are enhanced, while nodes within the cluster and their connections are deleted. More specifically, the specific formula of this process is shown in formula (2):
[0043] A s =F T AF (2)
[0044] Among them, A is the M×M sparse graph obtained in step 1, F is the M×C learning matrix, and As is the C×C symmetric matrix.
[0045] 4. Build the p-GCN subnetwork in Hi-GCN for constructing the association graph and learning the expression of the association graph nodes. The specific steps of p-GCN are to use the low-dimensional representation vector of the sparse graph under the same threshold of different graph data obtained by f-GCN to calculate the correlation, and embed the results of p-GCN into different nodes through the graph kernel k: A new graph is constructed, and then GCN calculation is performed, and the result is placed in the fully connected network to obtain a classification result under a threshold. More specifically, the specific process of this part is shown in formulas (3)(4)(5)(6):
[0046]
[0047]
[0048]
[0049]
[0050] in, represents the local expansion of the node generated by f-CGN, represents the actual Euclidean distance between the two networks under the qth graph kernel, represents the correlation between the qth graph kernel and each sparse graph element, S I (N i , N j ) represents the correlation between two different sparse graphs under the same threshold.
[0051] More specifically, the overall loss function of network training consists of three parts: cross entropy loss and consistency regularization loss, as shown in formulas (6) and (7):
[0052] Loss = L CE (W f , W p , F)+λ1L otho (F)+λ2L bal (F)+λ3L pos (F) (8)
[0053] L otho =||F T F-diag(diag(F T F))|| (9)L bal =Var(diag(F T F)) (10)
[0054] Among them, L in formula (8) CE represents the cross entropy loss, W f and W p The weight parameter F in f-GCN and p-GCN is the same as in formula (2), L otho Used to penalize off-diagonal elements. L otho Balanced regularization for sparse graph clustering. bal It is to ensure that the elements in F are positive. λ1, λ2, λ3 are regularization parameters.
[0055] Since there are many types of violations in the existing data set, and at the same time, there are a few types of violations that contain fewer texts, less than 100, and similar types of violations are usually handled by the same functional department, so the types of violations of similar businesses are merged to classify the major categories of illegal businesses. At the same time, after observing the data, the relevant data with the violation type of "other" is generally misclassified. Considering that it is due to the drawbacks of manual classification, the classification error rate of the "other" category is relatively high, which greatly affects the training of the classifier, so the original text data of the "other" category is deleted and does not participate in the training of the classifier. The remaining data with less data and no similar business are merged into a new "other" category. After merging and sorting the business categories, there are 11 categories of existing illegal business categories, with a total of 93,627 valid data. The model is verified and analyzed on this data set. Table 1 is a comparison of the test results of the sparse graph hierarchical graph convolutional network framework provided by the present invention, including multi-layered graph convolutional networks and ensemble learning. The same parameter settings and training strategies are used on a unified data set. The comparison results are shown in Table 1:
[0056] Table 1 Comparison of the results of traditional GCN and the transfer hierarchical graph convolutional network integration framework proposed in this patent on multiple evaluation indicators
[0057]
[0058] It can be seen from the data in Table 1 that the method proposed in this invention has different degrees of improvement in comprehensive evaluation indicators such as ACC and AUC compared with traditional GCN and other models.
[0059] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention.
Claims
1. A case source clue classification processing method based on a hierarchical graph convolutional neural network model, characterized in that: include: Step 1: Preprocess the case source data set and convert the case source clue text data into graph data containing text logical relationships; Use Glove to encode text words, convert them into nodes in the graph, calculate the similarity of the node homes and construct the edges in the graph; Step 2: Construction of multi-threshold sparse graphs, using different thresholds to segment the graph data converted in step 1 and construct sparse graphs of different levels; Step 3: For the sparse graph obtained in step 2, use the hierarchical graph convolutional network model Hi-GCN to learn the topological structure of the graph and the relationship between graphs; Step 4: Use graphs with different thresholds to build multiple Hi-GCN models, obtain the results corresponding to different thresholds, and finally form the final case source classification results through the integrated classifier; The specific steps of p-GCN are to calculate the correlation between sparse graphs based on sparse graphs of different graph data under the same threshold using the graph kernel structure matching method, then regard the low-dimensional graph feature expression obtained by f-GCN as the point of the association graph, construct an association graph, and perform GCN feature learning based on the association graph, and put the result into the fully connected network to obtain a classification result under a threshold; The integrated classifier described in step 4 specifically puts the results calculated in step 3 under each threshold into the fully connected network to calculate the prediction results under a single threshold, and finally votes on the classification results of multiple thresholds to obtain the final classification result.
2. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 1 is characterized in that: The multi-threshold sparse graph construction described in step 2 is specifically to set multiple thresholds in the interval [5%, 50%] for the edges of the original graph data, and obtain sparse graphs with different sparsity levels from the original graph data.
3. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 1 is characterized in that: The hierarchical graph convolutional network described in step 3 is specifically composed of two sub-networks: a graph-level sparse graph learning model f-GCN and a node-level sparse graph correlation learning model p-GCN.
4. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 1 is characterized in that: The f-GCN described in step three learns the graph expression of each sparse graph through graph clustering method and graph convolution feature learning respectively, converts the sparse graph information into a coarsened graph, and then performs graph convolution feature operation to learn a low-dimensional vector; specifically, a learnable weight matrix is set for the input sparse graph, and the sparse graph obtained in step two is fused into a coarsened graph of lower dimension by matrix multiplication, and finally the result of the coarsened graph is embedded into a low-dimensional vector through GCN.
5. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 4 is characterized in that: In step 3, the graph clustering method of f-GCN is to retain points in the coarsened graph formed by weighting the importance of each point. The edges are obtained by multiplying the importance weighted value of all edges between new points by the weight of the original edge and finally adding them together.
6. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 1 is characterized in that: In step 3, the method of constructing a new graph by p-GCN is to convert the graph representation of f-GCN into node features, and then calculate the structural similarity between points to construct edges, and obtain the association graph.
7. The case source clue classification processing method based on the hierarchical graph convolutional neural network model according to claim 1 is characterized in that: The entire framework uses the cross entropy function in the f-CGN, p-GCN and graph clustering process in step 3 during training.