Pruning method for data graph in sub-graph matching problem based on reinforcement learning
Through the subgraph count prediction model based on reinforcement learning and the DQN reinforcement learning model, the data graphs are accurately pruned, and the problems of low subgraph matching efficiency and incomplete filtering in the existing technology are solved, and faster and more accurate subgraph matching is achieved.
Patent Information
- Application Number
- CN202510070184.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-16
- Publication Date
- 2025-04-25
AI Technical Summary
Existing subgraph matching algorithms are inefficient when processing large-scale data graphs, and existing filtering methods cannot effectively prune, resulting in the retaining of many irrelevant vertices.
Using reinforcement learning-based method, the subgraph count prediction model and the Deep Q Network (DQN) reinforcement learning model are constructed to accurately trim the data graph and delete edges that are not related to the query graph.
It effectively reduces the size of the data graph, improves the speed of sub-graph matching, and improves the accuracy of pruning.
Smart Images
Figure CN119886273A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of subgraph matching, and in particular to a method for pruning a data graph in a subgraph matching problem based on reinforcement learning. Background Art
[0002] With the rapid development of the Internet and information technology and their wide application in various fields, the amount of user data has exploded. Daily life is filled with huge amounts of related data. Graphs, nonlinear data structures that can represent the structural relationship between nodes, have been widely used due to their unique node and edge properties.
[0003] Subgraph matching, as a basic problem in graph query, has been widely used in information retrieval, computer vision, natural language processing, cheminformatics, and bioinformatics. The subgraph matching algorithm is a basic graph query operation that aims to find a subgraph in a given data graph that is isomorphic to another given query graph, and is used to discover important substructures in the graph. Since the subgraph matching problem is an NP-hard problem, when the data graph is large or the number of vertices in the query graph is large, the efficiency of the algorithm will be greatly affected. Most of the existing methods belong to the preprocessing enumeration paradigm, including DP-iso, CFL-Match, and CECI. The basic idea is to first delete the vertices in the data graph that are not related to any vertex in the matching query graph, and then use the auxiliary structure to store the pruned graph and the relevant candidates for each query vertex. Finally, the enumeration process is called to list all matches on the auxiliary structure. The size of the data graph is reduced by several orders of magnitude through pruning to improve the matching performance.
[0004] However, the filtering methods of existing algorithms still have shortcomings. For example, the filtering methods based on vertex information (LDF) and neighboring vertex information (NLF) leave many independent vertices due to the coarse filtering granularity; the filtering methods based on specific vertex structures require artificially designed vertex structures, and cannot effectively filter when the query graph is large or complex; the filtering methods based on index structures build indexes on data graphs. When the size of the data graph increases, the time and space complexity grows rapidly. Like the first two methods, this method will retain more irrelevant vertices. Therefore, how to accurately prune the data graph is an important direction for the current research on subgraph matching algorithms. Summary of the invention
[0005] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and to provide a method for pruning data graphs in subgraph matching problems based on reinforcement learning, which accurately prunes the data graph according to the query graph, thereby reducing the size of the data graph and accelerating the speed of subgraph matching.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: the present invention provides a data graph pruning method for subgraph matching problems based on reinforcement learning, comprising the following steps:
[0007] Step 1: Construct a subgraph matching dataset and obtain the query graph F in the subgraph matching dataset. F ,E F ,L F} and data graph G={V G ,E G ,L G}, construct a subgraph counting dataset for the subgraph matching dataset, and divide the subgraph counting dataset into training set, validation set and test set according to the proportion;
[0008] Furthermore, the method for constructing a subgraph counting dataset is as follows: using the subgraph matching algorithm CFL of the index enumeration framework to calculate the exact subgraph count of each query graph on the data graph in the subgraph matching dataset, and dividing the dataset into a training set, a validation set, and a test set according to the proportion;
[0009] Step 2: Build a subgraph count prediction model, extract the features of the query graph F and the data graph G respectively, fuse the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to obtain the predicted subgraph count;
[0010] Furthermore, the method of extracting features of the query graph F and the data graph G respectively is: using a graph convolutional neural network GCN to extract features of the query graph F, and using a graph attention network GAT to extract features of the data graph G;
[0011] Step 2.1, use the graph convolutional neural network GCN model to extract the node feature representation of the query graph F;
[0012] Construct an L-layer graph convolutional neural network GCN to extract the node feature representation of the query graph F. The calculation method of each layer is as follows:
[0013]
[0014] Among them, f (l) is the node feature representation of the lth layer, μ is the activation function, is the normalized adjacency matrix, is the diagonal matrix, f (l-1) is the node feature representation of the l-1th layer, w (l-1) is the weight of the l-1th layer;
[0015] In the GCN model, the feature representation of the node in the current layer depends on the neighbor nodes of the node and their feature representation in the upper layer, and the feature matrix f of the query graph F is obtained;
[0016] Step 2.2: Use the graph attention neural network GAT model to extract node feature representation of the data graph G;
[0017] Construct a K-layer graph attention neural network GAT to extract node feature representations of the data graph G. The calculation method of each layer is as follows:
[0018]
[0019] in, is the feature representation of the i-th node in the k-th layer, σ is the activation function, α is the attention coefficient, N i is the neighbor node of node i, is the feature representation of the i-th point at the k-1 layer, is the feature representation of the neighbor node j of the i-th node at the k-1 layer, W k-1 is the weight of the k-1 layer, and the feature matrix g of the data graph G is obtained;
[0020] Step 2.3, fuse the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to calculate the matching subgraph count;
[0021] Furthermore, the method for calculating the count of matched subgraphs is as follows: converting the feature matrix f of the query graph F output by the graph neural network GCN and the feature matrix g of the data graph G output by the graph attention neural network GAT into a one-dimensional vector, connecting them and inputting them into a fully connected layer to calculate the number of matched subgraphs;
[0022] The loss function of the subgraph count prediction model is shown in the following formula:
[0023]
[0024] Where m represents the number of query graphs F in a training batch, and y i represents the subgraph count predicted by the i-th query graph, represents the true subgraph count of the i-th query graph;
[0025] Step 2.4, use the training set in the subgraph counting dataset to train the subgraph counting prediction model, use the stochastic gradient descent method to update the subgraph counting prediction model parameters according to the loss function, and use the validation set in the subgraph counting dataset to verify the training effect of the subgraph counting prediction model; after the subgraph counting prediction model is trained, use the test set in the subgraph counting dataset to test the subgraph counting prediction model to determine the prediction accuracy of the subgraph counting prediction model;
[0026] Step 3: Establish a DQN reinforcement learning model, use the subgraph count prediction model as the reward function of the reinforcement learning module, find the optimal strategy for the pruning scheme, and delete the edges in the data graph that are not related to the query;
[0027] Furthermore, the DQN reinforcement learning model includes a main network for determining whether to perform a pruning operation and a target network for generating a target Q value. The subgraph count prediction model obtained in step 2 is used as the reward function of the reinforcement learning module to find the optimal strategy for the pruning scheme and delete the edges in the data graph that are not related to the query graph.
[0028] Step 3.1: Design action representation a for the DQN reinforcement learning model t , standardizes the operations performed by the reinforcement learning model; for the tth edge e in the data graph t , use a t =0 means deleting edge e in the data graph t , use a t =1 means retaining edge e in the data graph t ;
[0029] Step 3.2: Design the state representation S of the DQN reinforcement learning model t , so that the reinforcement learning model can understand the current situation, the state representation of the reinforcement learning model S t , as shown in the following formula:
[0030] S t ={g1,g2,…,g Y ,f1,f2,…,f Z}
[0031] Among them, g1,g2,…,g Y Represents the one-dimensional feature representation of the data graph, f1,f2,…,f Z A one-dimensional feature representation representing the query graph, where the features of the data graph and the query graph are calculated by the subgraph count prediction model in step 2;
[0032] Step 3.3: Use the subgraph count prediction model obtained in step 2 as the reward function of the reinforcement learning module to judge the accuracy of the selected pruning scheme. The reward function r t It is expressed as follows:
[0033]
[0034] Among them, SC(G after ) represents the number of subgraphs in the pruned data graph that match the query graph, SC(G before ) represents the number of subgraphs in the data graph that match the query graph before pruning, E G (G before) represents the number of edges in the data graph before pruning, E G (G after ) represents the number of edges in the data graph after pruning, and λ and β are hyperparameters used to control the update speed of the reinforcement learning model;
[0035] Step 3.4: Design the loss function of the DQN reinforcement learning model, train the DQN reinforcement learning model, and update the reinforcement learning model parameters through the gradient descent algorithm;
[0036] The loss function of the DQN reinforcement learning model is defined as:
[0037]
[0038] Among them, Q θ (S t ,a t ) represents the possible reward that can be obtained by selecting action at in state St, Shown in state S t Next select action a t The real rewards you can get, Indicates the next state S t+1 The maximum reward value that can be obtained under t is the current reward obtained by calculating the reward function in step 3.3, θ is the parameter of the main network, is the parameter of the target network, γ is the discount factor;
[0039] The DQN reinforcement learning model parameters are updated through the gradient descent algorithm. The calculation method is shown in the following formula:
[0040]
[0041] The trained DQN reinforcement learning model is obtained and used to delete the edges in the data graph that are not related to the query graph.
[0042] The beneficial effects of adopting the above technical solution are as follows: the present invention provides a data graph subgraph matching method based on reinforcement learning, which first constructs a subgraph counting prediction model to measure the number of subgraph matches before and after pruning and evaluate the quality of pruning effects, and uses the model as a reward function to guide the reinforcement learning module to update. Secondly, a reinforcement learning model based on the Deep Q Network algorithm is constructed to delete edges in the data graph that are not related to the query graph. The invention effectively improves the speed of subgraph matching and can be extended to related fields such as subgraph counting and subgraph matching based on machine learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1A schematic diagram of a method for pruning a data graph in a subgraph matching problem based on reinforcement learning provided by an embodiment of the present invention;
[0044] Figure 2 Schematic diagram of the subgraph matching problem, where (a) is the query graph and (b) is the data graph;
[0045] Figure 3 A structural diagram of a subgraph count prediction model provided by an embodiment of the present invention;
[0046] Figure 4 This is a structural diagram of the reinforcement learning model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0047] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0048] The data graph pruning method for the subgraph matching problem based on reinforcement learning in this embodiment is as follows: Figure 1 As shown, the following steps are included:
[0049] Step 1: Construct a subgraph matching dataset and obtain the query graph F in the subgraph matching dataset. F ,E F ,L F} and data graph G={V G ,E G ,L G}, construct a subgraph counting dataset for the subgraph matching dataset, and divide the subgraph counting dataset into training set, validation set and test set according to the proportion;
[0050] In this example, the dblp, yeast and hprd datasets are used to construct a subgraph matching dataset. The subgraph matching algorithm CFL of the index enumeration framework is used to calculate the exact subgraph count of each query graph in the dataset on the data graph, and a subgraph count dataset is constructed. The dataset is then split into a dataset, a validation set and a test set in a ratio of 7:1:2.
[0051] Subgraph matching is defined as F ,E F ,L F} and data graph G={V G ,E G ,L G}, where V represents the nodes in the graph, E represents the edges in the graph, and L represents the labels of the nodes in the graph. There exists a single mapping ξ:V F →V G , so that any two vertices u,v∈V in the query graph FF The edge (u,v)∈E F There are also edges (ξ(u),ξ(v))∈E between corresponding points in the corresponding data graph G. G , and keep the label unchanged, that is, L F (u) = L G (ξ(u)),L F (v) = L G (ξ(v)), where L F is the node label of the query graph, L G is the node label of the data graph, such as Figure 2 As shown;
[0052] Step 2: Build a subgraph count prediction model, extract the features of the query graph F and the data graph G respectively, fuse the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to obtain the predicted subgraph count;
[0053] In this embodiment, different feature extraction methods are used according to the different complexities of the query graph F and the data graph G, that is, the graph convolutional neural network GCN is used to extract the features of the query graph F, and the graph attention network GAT is used to extract the features of the data graph G. Then, the features of the query graph F and the data graph G are fused and input into the MLP (multi-layer vector perceptron) to obtain the predicted subgraph count. The subgraph count prediction model structure is as follows: Figure 3 As shown;
[0054] Step 2.1, use the graph convolutional neural network GCN model to extract the node feature representation of the query graph F;
[0055] For a query graph with fewer nodes and a simpler structure, an L-layer graph convolutional neural network GCN is constructed to extract the node feature representation of the query graph F. The calculation method of each layer is as follows:
[0056]
[0057] Among them, f (l) is the node feature representation of the lth layer, μ is the activation function, is the normalized adjacency matrix, is the diagonal matrix, f (l-1) is the node feature representation of the l-1th layer, w (l-1) is the weight of the l-1th layer.
[0058] In the GCN model, the feature representation of the node in the current layer depends on the neighbor nodes of the node and their feature representation in the upper layer, and the feature matrix f of the query graph F is obtained;
[0059] Graph Neural Network (GNN) is a deep learning model specifically designed to process graph structured data. The core idea of GNN is to learn the embedded representation of nodes in the graph through information propagation between nodes and their neighboring nodes. This representation can capture the graph structure and node features, which is convenient for downstream tasks. The advantage of GCN is that it is simple and fast. Since the query graph is small in size, the classic convolutional graph neural network (GCN) in GNN is used to extract the node feature representation of the query graph F.
[0060] Step 2.2: Use the graph attention neural network GAT model to extract node feature representation of the data graph G;
[0061] Construct a K-layer graph attention neural network GAT to extract node feature representations of the data graph G. The calculation method of each layer is as follows:
[0062]
[0063] in, is the feature representation of the i-th node in the k-th layer, σ is the activation function, α is the attention coefficient, N i is the neighbor node of node i, is the feature representation of the i-th point at the k-1 layer, is the feature representation of the neighbor node j of the i-th node at the k-1 layer, W k-1 is the weight of the k-1 layer, and the feature matrix g of the data graph G is obtained;
[0064] Since the data graph is relatively complex, in order to better extract the node representation of the data graph, the graph attention neural network GAT is used to extract the node representation of the data graph. The graph attention neural network GAT is an improved graph neural network (GNN) model. The core idea is to introduce the attention mechanism to dynamically assign different weights to the neighbors of the node, thereby enhancing the attention to important neighbor information.
[0065] Step 2.3, fuse the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to calculate the matching subgraph count;
[0066] The feature matrix f of the query graph F output by the graph neural network GCN and the feature matrix g of the data graph G output by the graph attention neural network GAT are converted into one-dimensional vectors, which are connected and input into the fully connected layer to calculate the number of matching subgraphs. The subgraph count prediction model structure is as follows: Figure 3 As shown, the loss function of the subgraph count prediction model is as follows:
[0067]
[0068] Where m represents the number of query graphs F in a training batch, and y irepresents the subgraph count predicted by the i-th query graph, represents the true subgraph count of the i-th query graph.
[0069] Step 2.4: Use the training set in the subgraph count dataset to train the subgraph count prediction model, use the stochastic gradient descent method to update the subgraph count prediction model parameters according to the loss function, and use the validation set in the subgraph count dataset to verify the training effect of the subgraph count prediction model. After the subgraph count prediction model is trained, use the test set in the subgraph count dataset to test the subgraph count prediction model to determine the accuracy of the prediction of the subgraph count prediction model;
[0070] Step 3: Establish a DQN reinforcement learning model, use the subgraph count prediction model as the reward function of the reinforcement learning module, find the optimal strategy for the pruning scheme, and delete the edges in the data graph that are not related to the query;
[0071] The DQN reinforcement learning model includes a main network for determining whether to perform pruning operations and a target network for generating target Q values. The structure of the DQN reinforcement learning model is as follows: Figure 4 As shown in the figure, the DQN reinforcement learning model is trained by randomly extracting small batches of experience samples from an experience replay buffer, using the subgraph count prediction model obtained in step 2 as the reward function of the reinforcement learning module to find the optimal strategy for the pruning scheme and delete the edges in the data graph that are not related to the query graph;
[0072] Step 3.1: Design action representation a for the DQN reinforcement learning model t , normalizes the operations performed by the reinforcement learning model, for the tth edge e in the data graph t , use a t =0 means deleting edge e in the data graph t , use a t =1 means retaining edge e in the data graph t ;
[0073] Step 3.2: Design the state representation of the DQN reinforcement learning model so that the reinforcement learning model can understand the current situation. The state representation of the reinforcement learning model is as follows:
[0074] S t ={g1,g2,…,g Y ,f1,f2,…,f Z}
[0075] Among them, g1,g2,…,g Y Represents the one-dimensional feature representation of the data graph, f1,f2,…,f Z A one-dimensional feature representation representing the query graph, where the features of the data graph and the query graph are calculated by the subgraph count prediction model in step 2;
[0076] Step 3.3: Use the subgraph count prediction model obtained in step 2 as the reward function of the reinforcement learning module to judge the accuracy of the selected pruning scheme. The reward function r t It is expressed as follows:
[0077]
[0078] Among them, SC(G after ) represents the number of subgraphs in the pruned data graph that match the query graph, SC(G before ) represents the number of subgraphs in the data graph that match the query graph before pruning, E G (G before ) represents the number of edges in the data graph before pruning, E G (G after ) represents the number of edges in the data graph after pruning, λ and β are hyperparameters used to control the update speed;
[0079] Step 3.4: Design the loss function of the DQN reinforcement learning model, train the DQN reinforcement learning model, and update the reinforcement learning model parameters through the gradient descent algorithm;
[0080] The loss function of the DQN reinforcement learning model is defined as:
[0081]
[0082] Among them, Q θ (S t ,a t ) indicates that in state S t Next select action a t Possible rewards you can get, Shown in state S t Next select action a t The real rewards you can get, Indicates the next state S t+1 The maximum reward value that can be obtained under t is the current reward obtained by calculating the reward function in step 3.3, θ is the parameter of the main network, is the parameter of the target network, and γ is the discount factor.
[0083] The DQN reinforcement learning model parameters are updated through the gradient descent algorithm. The calculation method is shown in the following formula:
[0084]
[0085] A trained DQN reinforcement learning model is obtained, which can delete edges in the data graph that are not related to the query according to the characteristics of the data graph and the query graph, thereby speeding up the query speed of subgraph matching.
[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.
Claims
1. A data graph pruning method for subgraph matching problem based on reinforcement learning, characterized by: The following steps are involved: Step 1: Construct a subgraph matching dataset and obtain the query graph F in the subgraph matching dataset. F ,E F ,L F } and data graph G={V G ,E G ,L G }, construct a subgraph counting dataset for the subgraph matching dataset, and divide the subgraph counting dataset into training set, validation set and test set according to the proportion; Step 2: Build a subgraph count prediction model, extract the features of the query graph F and the data graph G respectively, fuse the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to obtain the predicted subgraph count; Step 3: Establish a DQN reinforcement learning model, use the subgraph count prediction model as the reward function of the reinforcement learning module, find the optimal strategy for the pruning scheme, and delete the edges in the data graph that are not related to the query.
2. According to claim 1, a method for pruning a data graph in a subgraph matching problem based on reinforcement learning, characterized in that: The method for constructing the subgraph count dataset in step 1 is: using the subgraph matching algorithm CFL of the index enumeration framework to calculate the exact subgraph count of each query graph in the subgraph matching dataset on the data graph, and dividing the dataset into a training set, a validation set, and a test set according to the proportion.
3. According to claim 2, a method for pruning a data graph in a subgraph matching problem based on reinforcement learning, characterized in that: The method for extracting the features of the query graph F and the data graph G in step 2 is: using the graph convolutional neural network GCN to extract the features of the query graph F, and using the graph attention network GAT to extract the features of the data graph G.
4. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 3, characterized in that: The specific method of using the graph convolutional neural network GCN to extract the features of the query graph F is: Construct an L-layer graph convolutional neural network GCN to extract the node features of the query graph F. The calculation method of each layer is as follows: Among them, f (l) is the node feature representation of the lth layer, μ is the activation function, is the normalized adjacency matrix, is the diagonal matrix, f (l-1) is the node feature representation of the l-1th layer, w (l-1) is the weight of the l-1th layer; In the GCN model, the feature representation of the node in the current layer depends on the neighbor nodes of the node and their feature representation in the upper layer, and the feature matrix f of the query graph F is obtained.
5. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 4, characterized in that: The specific method of using the graph attention network GAT to extract the features of the data graph G is: Construct a K-layer graph attention neural network GAT to extract the features of the data graph G. The calculation method of each layer is as follows: in, is the feature representation of the i-th node in the k-th layer, σ is the activation function, α is the attention coefficient, N i is the neighbor node of node i, is the feature representation of the i-th point at the k-1 layer, is the feature representation of the neighbor node j of the i-th node at the k-1 layer, W k-1 is the weight of the k-1 layer, and the feature matrix g of the data graph G is obtained.
6. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 5, characterized in that: The specific method of fusing the features of the query graph F and the data graph G into the multi-layer vector perceptron MLP to obtain the predicted subgraph count is: The feature matrix f of the query graph F output by the graph neural network GCN and the feature matrix g of the data graph G output by the graph attention neural network GAT are converted into one-dimensional vectors, which are connected and input into the fully connected layer to calculate the number of matching subgraphs. The loss function of the subgraph count prediction model is shown in the following formula: Where m represents the number of query graphs F in a training batch, and y i represents the subgraph count predicted by the i-th query graph, represents the true subgraph count of the i-th query graph.
7. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 6, characterized in that: The step 2 also uses the training set in the subgraph count data set to train the subgraph count prediction model, uses the stochastic gradient descent method to update the subgraph count prediction model parameters according to the loss function, and uses the validation set in the subgraph count data set to verify the training effect of the subgraph count prediction model; after the subgraph count prediction model is trained, the subgraph count prediction model is tested using the test set in the subgraph count data set to determine the accuracy of the prediction of the subgraph count prediction model.
8. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 7, characterized in that: The DQN reinforcement learning model in step 3 includes a main network for determining whether to perform a pruning operation and a target network for generating a target Q value. The subgraph count prediction model obtained in step 2 is used as the reward function of the reinforcement learning module to find the optimal strategy for the pruning solution and delete the edges in the data graph that are not related to the query graph.
9. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 8, characterized in that: The specific method described in step 3 is: Step 3.1: Design action representation a for the DQN reinforcement learning model t , standardizes the operations performed by the reinforcement learning model; for the tth edge e in the data graph t , use a t =0 means deleting edge e in the data graph t , use a t =1 means retaining edge e in the data graph t ; Step 3.2: Design the state representation S of the DQN reinforcement learning model t , so that the reinforcement learning model can understand the current situation, the state representation of the reinforcement learning model S t , as shown in the following formula: S t ={g1,g2,…,g Y ,f1,f2,…,f Z } Among them, g1,g2,…,g Y Represents the one-dimensional feature representation of the data graph, f1,f2,…,f Z A one-dimensional feature representation representing the query graph, where the features of the data graph and the query graph are calculated by the subgraph count prediction model in step 2; Step 3.3: Use the subgraph count prediction model obtained in step 2 as the reward function of the reinforcement learning module to judge the accuracy of the selected pruning scheme. The reward function r t It is expressed as follows: Among them, SC(G after ) represents the number of subgraphs in the pruned data graph that match the query graph, SC(G before ) represents the number of subgraphs in the data graph that match the query graph before pruning, E G (G before ) represents the number of edges in the data graph before pruning, E G (G after ) represents the number of edges in the data graph after pruning, λ and β are hyperparameters used to control the update speed of the model; Step 3.4: Design the loss function of the DQN reinforcement learning model, train the DQN reinforcement learning model, and update the reinforcement learning model parameters through the gradient descent algorithm.
10. The method for pruning a data graph in a subgraph matching problem based on reinforcement learning according to claim 9, characterized in that: The step 3.4 comprises: The loss function of the DQN reinforcement learning model is defined as: Among them, Q θ (S t ,a t ) indicates that in state S t Next select action a t Possible rewards you can get, Shown in state S t Next select action a t The real rewards you can get, Indicates the next state S t+1 The maximum reward value that can be obtained under t is the current reward obtained by calculating the reward function in step 3.3, θ is the parameter of the main network, is the parameter of the target network, γ is the discount factor; The DQN reinforcement learning model parameters are updated through the gradient descent algorithm. The calculation method is shown in the following formula: The trained DQN reinforcement learning model is obtained and used to delete the edges in the data graph that are not related to the query graph.