A graph computing method and device based on hierarchical compression technology and related media

Abstract

The application discloses a graph computing method and device based on hierarchical compression technology and related media, and the method comprises the following steps: obtaining original graph data, compressing the original graph data by using a hierarchical compression algorithm to obtain a k-level compressed graph; generating a corresponding compression function group based on each hierarchical compressed graph; decompressing each hierarchical compressed graph by using the compression function group; and querying and responding to each hierarchical compressed graph by using the compression function group. The application realizes the effect of single-machine computing of large graph data by compressing the graph data in a hierarchical manner, thereby improving the graph processing performance.

Description

Technical Field

[0001] This invention relates to the field of graph processing technology, and in particular to a graph calculation method, apparatus and related media based on hierarchical compression technology. Background Technology

[0002] A graph is a network of vertices connected by edges. A graph network G can be represented by a pair G(V,E), where V is the set of vertices and E is the set of edges. Graph computing has a natural advantage in handling data relationships. For example, in social systems, users can be represented as different vertices, and the relationships between users can naturally be represented as edges. Therefore, user community attributes can be analyzed based on these edges, or new friends can be recommended to users. Similarly, in e-commerce, users and products can be modeled as vertices, and the relationships between users and products can be modeled as edges, allowing recommendations to be made based on these relationships.

[0003] In recent years, with the development of the Internet, graph data has exploded, posing huge challenges to traditional graph computing, such as the computation and storage of large-scale graph data. Graph data under the big data era can no longer be contained in the memory space of a single machine.

[0004] A traditional solution is to use distributed systems to distribute graph data across multiple independent computers and use communication networks to transmit information. However, the performance of many distributed systems is unsatisfactory. For example, the distributed computing system Powergraph, using 128 cores to compute PageRank on the UK-2007-05 dataset, is still slower than COST, a disk-based single-core computing system. At the algorithmic level, many graph algorithms do not have parallel scalability; that is, using more processors does not guarantee that the algorithm will become faster. In fact, some graph algorithms may even become slower when using more processors.

[0005] Another solution is to store graph data on hard disk and perform computational tasks within the memory of a single computer, treating the hard disk as an extension of memory. For example, the GraphChi system divides graph data into numerous segments and uses a parallel sliding window technique to mimic parallel systems and perform computational tasks. Similarly, the Mosaic system also divides graph data into numerous segments, logically categorizes computational tasks, and uses heterogeneous devices to compute different logical computational tasks, thus achieving acceleration.

[0006] However, current hard-disk-based graph computation systems are programmed using a vertex-centric computation model, while traditional graph computation algorithms are designed based on a graph-centric computation model. This requires users to redesign their algorithms based on the vertex-centric model before using these systems. However, this not only leads to a significant amount of redundant work in algorithm design but also results in a decrease in algorithm performance. This is because, over the past few decades, the mainstream research direction in academia and industry has been graph algorithms based on the graph-centric model, resulting in the development of numerous optimizations based on the graph-centric model, which the vertex-centric model must abandon. Summary of the Invention

[0007] This invention provides a graph computation method, apparatus, computer device, and storage medium based on hierarchical compression technology, aiming to improve graph processing performance.

[0008] In a first aspect, embodiments of the present invention provide a graph computation method based on hierarchical compression technology, comprising:

[0009] Obtain the original image data, and use a hierarchical compression algorithm to perform hierarchical compression on the original image data to generate a k-level compressed image;

[0010] Based on each level of the compression graph, generate the corresponding set of compression functions;

[0011] The compression function set is used to decompress each level of the compressed graph;

[0012] In addition, the compression function group is used to query and respond to each level of the compressed graph.

[0013] Secondly, embodiments of the present invention provide a graph computing device based on hierarchical compression technology, comprising:

[0014] A hierarchical compression unit is used to acquire the original graph data and perform hierarchical compression on the original graph data using a hierarchical compression algorithm to generate a k-level compressed graph.

[0015] The function generation unit is used to generate a corresponding set of compression functions based on each level of the compression graph;

[0016] The decompression unit is used to decompress each level of the compressed graph using the set of compression functions;

[0017] The query response unit is used to query and respond to each level of the compressed graph through the compression function group.

[0018] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the graph computation method based on hierarchical compression technology as described in the first aspect.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the graph computation method based on hierarchical compression technology as described in the first aspect.

[0020] This invention provides a graph computation method, apparatus, computer device, and storage medium based on hierarchical compression technology. The method includes: acquiring original graph data; performing hierarchical compression on the original graph data using a hierarchical compression algorithm to generate a k-level compressed graph; generating a corresponding compression function set based on each level of the compressed graph; decompressing each level of the compressed graph using the compression function set; and querying and responding to each level of the compressed graph using the compression function set. This invention achieves the effect of single-machine computation of extremely large graph data by performing hierarchical compression on graph data, thereby improving graph processing performance. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a graph computation method based on hierarchical compression technology provided in an embodiment of the present invention;

[0023] Figure 2 A schematic block diagram of a graph computing device based on hierarchical compression technology provided in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a common topology provided for embodiments of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0027] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] Please see below. Figure 1 , Figure 1 The flowchart of a graph computation method based on hierarchical compression technology provided in an embodiment of the present invention specifically includes steps S101 to S104.

[0030] S101. Obtain the original map data, and use a hierarchical compression algorithm to perform hierarchical compression on the original map data to generate a k-level compressed map;

[0031] S102. Based on each level of the compression graph, generate the corresponding compression function set;

[0032] S103. Decompress each level of the compressed graph using the compression function set;

[0033] S104, and, query and respond to each level of the compressed graph through the compression function group.

[0034] In this embodiment, the original graph data is first compressed into a k-level compressed graph using a hierarchical compression algorithm. During the compression process, a set of compression functions is generated, containing multiple different functions. This set of compression functions can then be used to decompress each level of the compressed graph and provide query responses. This embodiment achieves the effect of single-machine computation of extremely large graph data by performing hierarchical compression on the graph data, thereby improving graph processing performance.

[0035] This invention presents a single-machine graph computation method and system based on hierarchical compression technology. This system is a disk-based graph processing system that allows users to compute extremely large graph data on a single machine. Furthermore, this system uses a graph centroid model as its programming model, enabling users to utilize existing graph centroid-based algorithm logic and optimizations without needing to redesign algorithms based on vertex centroid models.

[0036] In one embodiment, S101 includes:

[0037] The first function is called to divide the original graph data into multiple subgraphs, and the original graph data is compressed according to a preset topological order. The edges of the original graph data are compressed into super edges between superpoints of the first layer compressed graph until the size of the first layer compressed graph can no longer be compressed or meets the preset compression size requirements.

[0038] The second function is called to compress the edges of the original graph data until it meets the preset compression size requirements, thereby generating the first layer of compressed graph.

[0039] The first layer compressed graph is further compressed using the first and second functions until the original graph data is compressed into a k-level compressed graph.

[0040] In this embodiment, a compressed graph Gi is generated from the (i-1)th level compressed graph Gi-1 by continuously calling the first function LCon, ultimately obtaining a k-level compressed graph. Specifically, the first function LCon divides the compressed graph Gi-1 into a large number of subgraphs g, continuously loads the subgraphs g into memory, and compresses common topological structures into superpoints according to a preset topological order T(G). Here, the subgraph g can be of any shape, as long as the size of the subgraph g is less than the memory size limit M. During the compression process, if a point v in a certain subgraph g is not compressed, then point v will be divided into the next subgraph loaded into memory until v is compressed, or it is confirmed that v cannot be compressed and is retained as an independent point in the compressed graph Gi. When the size of Gi is less than M or LCon can no longer reduce the size of the compressed graph, LCon stops. At this time, the second function TCon is called to compress the graph to meet the memory size limit M through edge compression. The second function TCon is the function used to perform edge compression. Edge compression has no requirements on the topology and can compress the graph structure without restriction. Therefore, the second function TCon can be used to compress the graph to meet the memory size limit M.

[0041] For the compressed graph G at level i-1 i-1 LCon first marks all points as "uncompressed". Then, according to the preset topology order T(G), common topologies are compressed into superpoints, and the points in the superpoints are marked as "compressed". Figure 3The diagram illustrates different topologies. For example, in the clique compression topology, LCon continuously selects uncompressed points to add to the clique, provided that the point is a neighbor of all existing points in the clique, and the number of points in the clique meets the size constraint [k1, k]. u When compressing the star topology, LCoon first selects an uncompressed point as the center point, and then selects leaf nodes from the uncompressed neighbors of the center point, provided that this point is not a neighbor of any of the selected leaf nodes. Other topologies are compressed in a similar manner. For all compressed subgraphs, all points contained in the subgraph must be uncompressed points, thus LCoon can guarantee that each point is compressed at most once, and the size of each compressed topology satisfies the size constraint [k1, k]. u ].

[0042] For a compressed sub-image segment H containing k points, the number of edges is:

[0043] (1) Cluster: |E(H)|=k(k-1) / 2;

[0044] (2) Star and Road: |E(H)|=k-1;

[0045] (3) Diamond: |E(H)|=5;

[0046] (4) Butterfly: |E(H)|=6.

[0047] By compressing the topology, significant memory savings can be achieved. For example, for a star, only the central node and leaf nodes need to be stored, instead of the entire topology. For a clique, only the nodes need to be stored, without storing any edges within the clique. Furthermore, edges connected to the topology but not within it are compressed into hyperedges.

[0048] It should also be noted that the preset topology order T(G) described in this embodiment, in conjunction with Table 1, shows the common topological structures mined from 10 types of graph data, sorted according to their frequency of occurrence in the graph. During compression, this embodiment considers that although these topological structures are common in this type of graph data, their degree of commonness varies, and we prioritize compressing the most common topological structures. Because compressing one type of topological structure may reduce the number of other topological structures present in the graph, an order is specified. For example, compression is performed according to the order in Table 1 (represented as the preset topology order T(G)), thereby better preserving the topological structure and improving the compression ratio. Note that T(G) is determined by the type of graph data, therefore it only needs to be mined once for each type of graph, rather than mining once for every piece of graph data to be compressed.

[0049] Graph data types Common topologies (sorted by frequency) Social Network Graph Group, Diamond, Star, Butterfly Communication diagram star Reference image Cluster, star, diamond, butterfly, ring Network diagram Star, Cluster, Diamond knowledge graph star Collaborative Network Group, Star, Diamond Biomedical diagram cluster Economic map star Chemical diagram Road, Group, Star Traffic network map Stars, Road

[0050] Table 1

[0051] For preset compression size requirements, such as the compressed sub-image segment H, this embodiment can specify the number of points in the sub-image within [k...]. l ,k u The specific reasons are as follows: (1) Set a lower bound k for image segment H. l Because if the lower bound is too low, the compressed graph will be over-compressed into too many superpoints, resulting in an excessively large depth of the compressed graph, which leads to excessive I / O overhead during decompression and reduces system efficiency. (2) Set an upper bound k u This limits the size of compressed segments to prevent them from becoming too large and failing to be loaded into the cache.

[0052] In one embodiment, S102 includes:

[0053] For the i-th layer compressed graph, the i-th layer compression function set is generated based on the compression relationship: in, S represents the compression function of the i-th layer compressed graph. i The summary function representing the compressed graph of the i-th layer. Let i represent the decompression function of the i-th layer compressed graph. This represents the point compression function of the i-th layer compressed graph.

[0054] In this embodiment, during the generation of the compressed graph, a corresponding set of compression functions is generated based on the compression relationships. This set of compression functions includes compression functions, summary functions, decompression functions, and point decompression functions. Specifically, the original graph data is set as G = (V, E, L), where V is the set of points in the graph, E is the set of edges, and L is the set of labels for points and edges. The corresponding compressed graph layers are G1, G2, ..., G... i G k Using compression functions This indicates that the compressed graph G of layer i-1 is... i-1 A specific sub-image segment H in the image is compressed to the i-th layer G. i Super Point V H In the middle. For each query class Q, define a summary function for the i-th level compressed graph. Furthermore, the summary function of the i-th layer is an extension of the function of the (i-1)-th layer. The function is a decompression function for the i-th layer, used to decompress the superpoint v of the i-th layer. H Decompress the image to sub-image segment H in layer i-1. The function is the point decompression function for the i-th layer, used to decompress the superpoints v of the i-th layer. H Decompress to the (i-1)th layer. The difference is, The function does not restore the topological structure H, but rather restores the points in H.

[0055] Furthermore, for each query class Q and the i-th level compressed graph Gi, the summary function... It contains information that needs to be answered for query Q, so that users can complete the query response without decompressing the compressed graph.

[0056] This section focuses first on the information needed for most queries. The summary function of the i-th level compressed graph. It is composed of the summary functions of the (i-1)th layer. That is, for the superpoint v of the compressed graph of the i-th layer. H And the segment H of the (i-1)th layer compressed graph, i.e. It is by The image is composed of points v, where each point v is a point in the image segment H of the (i-1)th layer. First, it is assumed that in the compressed image of the 0th layer (i.e., the original image data), each point has summary information. Next, we define the summary information for each type of topology.

[0057] (1) Clique: S i (υ H ).size=∑ υ∈υ(H) S i-1 `(υ).size` represents the number of points in the compressed layer 0 graph within the clique. The sizes of the following topologies are also defined in this way. S i (υ H ).type = clique.

[0058] (2) Star: S i (υ H ).c is the id of the center point of the star. S i (υ H ).type=star.

[0059] (3) Path: S i (υ H A list is an ordered arrangement of points in a topological structure. i (υ H ).type = path.

[0060] (4) Diamond: S i (v H ).s1 and Si(v H ).s2 is the ID of two shared points in the diamond. S i ( H ).type = diamond.

[0061] (5) Butterfly: S i (v H In the structure ).s, the vertex where the two triangles intersect, S i (v H ).e represents two non-intersecting edges in the structure. S i (υ H ).type = butterfly.

[0062] In one embodiment, S103 includes:

[0063] When the superpoint v of the i-th layer compressed graph H During decompression, the corresponding decompression function is queried to obtain the sub-image segment H of the (i-1)th layer compressed image;

[0064] Obtain summary information of each point in sub-image segment H of the (i-1)th layer compressed image;

[0065] Decompress the edges that are compressed into sub-image segment H in the (i-2)th layer compressed image.

[0066] In this embodiment, during query response, since the hierarchical compression algorithm simultaneously generates summary information, the user only needs to decompress the image when necessary, and most of the time only needs to query the summary information to complete the query response. Simultaneously, due to the limitation of the cache size B, we need to control the size of the data loaded into the cache during decompression.

[0067] Decompression algorithms can be divided into superpoint decompression and superedge decompression. Superedge decompression is relatively simple; it only requires decompressing the superedge (vH1, vH2).

[0068] For the superpoint vH in the i-th layer, first query To obtain the sub-image segment H of layer i-1, we then obtain summary information for each point in sub-image segment H. Finally, we decompress some of the edges in layer i-2 that were compressed into H.

[0069] Here, if the superpoint v H Since it's derived from topological compression, there's no need to decompress the topological structure; only summary information needs to be queried, and the number of edges in the subgraph can be easily calculated. If the superpoint is relative to v... HSince it's derived from edge compression, all edges need to be decompressed. Meanwhile, the size of the summary information can be obtained by querying relevant information. Furthermore, this embodiment can maintain a number N to represent the maximum number of edges compressed into a single superedge. This number N is continuously updated and maintained during the compression process. Based on N, it can be known that there are at most N*|E(H)| edges in the subgraph segment. Therefore, if the number of edges in the subgraph H is greater than N*|E(H)|, it will be loaded into the cache multiple times during decompression.

[0070] In one embodiment, S104 includes:

[0071] For each superpoint v H Set the boolean variable v H .cvg, to represent the convergent superpoint;

[0072] If based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has converged, then it skips the corresponding superpoint v. H The calculation and update are performed, and query responses are based on point decompression functions and summary functions;

[0073] If based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has not converged, then the corresponding superpoint v is interrupted and decompressed. H The included edges are traced back to the original graph data, and all nodes in the unconverged original graph data are updated. Then, query responses are performed by decompressing the hyperedges.

[0074] If based on the Boolean variable v H .cvg determines the corresponding superpoint v H If the convergence value is less than the preset convergence threshold, then the corresponding superpoint v will be... H Mark as convergent, and check all points corresponding to the superpoint v. H The system checks whether points compressed to the same superpoint have converged, marks converged superpoints, and provides query responses based on the point decompression function and the summary function.

[0075] In this embodiment, the PageRank algorithm is introduced first because it is widely used in applications such as web search and recommendation. For a graph G(V, E), the PageRank algorithm iteratively calculates the PageRank score for each point u and stores the score in a container pr. This score represents the static distribution of this point in this stochastic process, therefore ∑ u∈Vpr(u) = 1. In each iteration, (1) each point distributes its PageRank score equally among all its neighboring nodes, and (2) each point adds up all the PageRank scores obtained from its neighboring nodes to obtain its own PageRank score. When the score change of a point in two iterations is less than a given threshold ∈, the operation of this point converges and the iteration stops. When all points in graph G stop iterating, the PageRank operation stops and the calculation process ends.

[0076] For the PageRank algorithm, general summary information is sufficient to answer such queries. In fact, for each compressed graph G at layer i... i super point v in H Therefore, it can be done through S i (v H Check the existence of each edge, and then... Get the compressed to superpoint v H The points in the middle.

[0077] The following demonstrates the Pagrank algorithm based on hierarchical compressed graphs, namely PRAh, which is the method provided in this embodiment. The logic of PRAh differs from the traditional PageRank algorithm only slightly. For each superpoint v... H Define a Boolean variable vH.cvg to indicate whether the PageRank calculation at a superpoint has converged. According to the monotonicity of the PageRank algorithm, if a superpoint has converged, then this point will never become non-convergent. Therefore, in the calculation process, (1) if v H If .cvg=true, then the corresponding superpoint is determined to have converged. Therefore, all calculations and updates related to v are skipped. That is, the PRA algorithm will not distribute the PageRank value of v to its neighboring nodes, nor will it send the PageRank value of its neighboring nodes to vH. (2) If v has not converged, that is, the corresponding superpoint is determined to have not converged, then the edges contained in v will be broken and decompressed up to the 0th level compressed graph, and all nodes in the 0th level compressed graph that have not converged will be updated. (3) If it is found that the change in the PageRank value of a certain updated point is less than the given convergence threshold, then this node is marked as converged, and all points compressed to the same superpoint as this point are checked to see if they have converged. If they have all converged, then the corresponding superpoint is marked as converged.

[0078] In the processes (1) and (3) above, the user only needs to view the function. and summary information S iThis is sufficient without decompressing the graph. In process (2), the user only needs to decompress the hyperedges, not the hypervertices. Therefore, for the PageRank algorithm, the user only needs to decompress the hyperedges, not the hypervertices.

[0079] In one embodiment, S104 further includes:

[0080] To address the connectivity issues caused by tag restrictions, a strategy for generating the first summary information is set.

[0081] Get the query point u, and start expanding from the superpoint connected to the query point u as L;

[0082] When the expanded superpoints satisfy the first summary information generation strategy, the corresponding superpoints are added to a preset set.

[0083] In this embodiment, label constraint connectivity (LCC) is frequently used in path matching and program analysis. In graph G, when a path p = <υ0,υ1,...,υ... l A path is called an L-path when the label of any point in a graph G is contained in a label set L. If an L-path exists between two points u and v, then the two points are considered L-connected. The Label-Constrained Connectivity Problem (LCC) is to find whether two points are L-connected given a label set L and a pair of points (u, v) in a graph G.

[0084] Unlike the PageRank algorithm, the LCC problem is related to the point's label.

[0085] This embodiment first defines a summary information generation strategy for the LCC problem (i.e., the first summary information generation strategy). For the LCC problem, if a point u and a superpoint vH are L-connected, then u and vH are... H Every compressed point in the array is L-connected. Similarly, if two superpoints are L-connected, then every point in those two superpoints is L-connected.

[0086] To reduce the number of decompression operations, this embodiment uses the tag information of the compressed points to expand the general summary information.

[0087] At level 0,

[0088] At the i-th level,

[0089] The traditional LCC algorithm is explained below. Given a graph G and a query Q = (L, u, v), the LCC algorithm performs a breadth-first search (BFS) traversal starting from vertex u. The algorithm maintains a set S containing all vertices that are L-connected to vertex u. This set is initialized as S = {u}. At each step of the BFS traversal, LCC selects a vertex from S and adds all its neighbors whose labels are included in L to set S. When vertex u is added to set S, the LCC algorithm returns true, meaning that vertices u and v are L-connected. Alternatively, if no new vertex can be added to set S, the LCC algorithm returns false, meaning that the two vertices are not L-connected.

[0090] The LCCh algorithm based on hierarchical compression graphs (i.e., the method provided in this embodiment) and the LCC algorithm share the same logic: they expand from a superpoint vH' that is L-connected to u, and add the superpoint vH to the set S when one of the following conditions is met:

[0091] (a)v H 'It is compressed to v H and

[0092] (b) And there exists a path from v H 'to v H L-path;

[0093] (c) Decompress the superpoint v H 'and v H The superedges between them extend up to the 0th layer, thus determining whether an L-path exists.

[0094] Conditions (a) and (b) can be fulfilled using the decompression function and summary information, while condition (c) only requires decompressing the hyperedges. Therefore, the LCCh algorithm does not require decompressing the hyperedges, but only the hypervertices.

[0095] In one embodiment, S104 further includes:

[0096] To address the graph pattern matching problem, a second summary information generation strategy is set.

[0097] When a graph pattern is matched with the original graph data, superpoints are continuously decompressed into the original graph data, and the decompressed superpoints are checked according to the second summary information generation strategy to complete the graph pattern matching.

[0098] In this embodiment, the graph pattern matching (subgraph isomorphism) algorithm, also known as the subgraph isomorphism algorithm, is widely used in graph query problems and graph rule dependency problems.

[0099] First, define the graph pattern matching problem. For a graph pattern Q = (V Q E Q L Q In graph G, a matching about Q is a subgraph G′ = (V′, E′, L′) that is isomorphic to Q, meaning there exists a bijective function V. Q →V′;

[0100] (a) For any point u∈V Q L Q (u)=L(h(u));

[0101] (b) e = (u, u') is an edge in Q if and only if h(u), h(u')) is an edge in graph G and the corresponding point labels are the same. Let Q(G) denote all matchings of pattern Q in graph G.

[0102] The graph pattern matching problem is the problem of finding all matching Q(G) on a graph G, given a graph pattern Q.

[0103] This problem is a well-known NP-complete problem. This embodiment uses the classic graph pattern matching algorithm VF2 as the algorithm for the graph pattern matching problem.

[0104] First, we introduce the strategy for generating summary information for the graph pattern matching problem. For a point v in graph G and u in its matching graph pattern, we can determine that the labels of u's neighbors must be included in the labels of v's neighbors; otherwise, v and u cannot match. Therefore, by collecting information about neighboring nodes, we can further extend the summary information of the LCC problem to the summary information of the graph pattern matching problem:

[0105] At level 0,

[0106] At the i-th level,

[0107] Next, we will review the VF2 algorithm and then extend it to SubAh, a graph pattern matching algorithm based on hierarchical compression, which is the method provided in this embodiment.

[0108] Given a graph G and a graph pattern Q, the VF2 algorithm computes Q(G) using a backtracking approach. The VF2 algorithm iteratively expands partial matches s (which are initialized to an empty set). In each iteration, VF2 adds a pair of points between G and Q to the partial match, satisfying the conditions (a) the topology of the points in graph G matches the topology of the points in the graph pattern Q, and (b) the two points have the same label. When no more pairs of points can be added to the partial match, or when a matching of all points in Q has been completed, the VF2 algorithm backtracks to the previous partial match and adds a new matching pair of points.

[0109] The SubAh algorithm uses essentially the same logic as VF2, but with minor modifications. First, for point pairs (u, vH), it checks:

[0110] (a) For u,

[0111] (b) For each neighbor u' in Q,

[0112] When obtaining the matching of the graph pattern on the original graph G, the superpoint vH is continuously decompressed to the 0th level compressed graph, i.e., the original graph. During this process, each decompressed point is checked using conditions (a) and (b). At the same time, SubAh also checks whether the decompressed points meet the topological requirements by decompressing the superedges, until the 0th level compressed graph.

[0113] Throughout the process, there is no decompression of super-points, only decompression of super-edges.

[0114] In one specific embodiment, when the graph computation method based on hierarchical compression technology provided by the present invention is used to process the specified graph data, the results obtained verify the effectiveness of the graph computation method based on hierarchical compression technology (hereinafter referred to as Hier).

[0115] (1) When using 7.6% of the memory size of the graph data, the Hier method (or system) performs similarly to other fully memory-based systems.

[0116] (2) Compared with other disk-based systems GraphChi, Mosaic and COST, Hier's method (or system) is on average 705.7, 49.6 and 14.1 times faster in problems such as Pagerank, LCC, graph pattern matching, weak connectivity, maximum clique and regular path matching.

[0117] (3) Compared to the memory-based parallel computing system Powergraph, the Hier method (or system) is 74.3, 1.3, 2.4 and 404.7 times faster in LCC, Pagerank, graph pattern matching and weak connectivity problems, respectively, even though the Powergraph system uses 6 computers while the Hier method (or system) uses only 1 machine.

[0118] Figure 2 This is a schematic block diagram of a graph computing device 200 based on hierarchical compression technology provided in an embodiment of the present invention. The device 200 includes:

[0119] The hierarchical compression unit 201 is used to acquire the original graph data and perform hierarchical compression on the original graph data using a hierarchical compression algorithm to generate a k-level compressed graph.

[0120] The function generation unit 202 is used to generate a corresponding set of compression functions based on each level of the compression graph;

[0121] Decompression unit 203 is used to decompress each level of compressed graph using the compression function set;

[0122] The query response unit 204 is used to perform a query response on each level of the compressed graph through the compression function group.

[0123] In one embodiment, the hierarchical compression unit 201 includes:

[0124] The first calling unit is used to call the first function to divide the original graph data into multiple subgraphs, and to compress the original graph data according to a preset topological order, compressing the edges of the original graph data into superedges between superpoints of the first layer compressed graph, until the size of the first layer compressed graph cannot be compressed further or meets the preset compression size requirements.

[0125] The second calling unit is used to call the second function to perform edge compression on the original graph data until it meets the preset compression size requirements, thereby generating the first layer compressed graph.

[0126] A continuous compression unit is used to continue compressing the first layer compressed graph using a first function and a second function until the original graph data is compressed into a k-level compressed graph.

[0127] In one embodiment, the function generation unit 202 includes:

[0128] The function group generation unit is used to generate the compression function group of the i-th layer based on the compression relationship for the i-th layer compression graph: in, S represents the compression function of the i-th layer compressed graph. iThe summary function representing the compressed graph of the i-th layer. Let i represent the decompression function of the i-th layer compressed graph. This represents the point compression function of the i-th layer compressed graph.

[0129] In one embodiment, the decompression unit 203 includes:

[0130] The function query unit is used to query the superpoint v of the i-th layer compressed graph. H During decompression, the corresponding decompression function is queried to obtain the sub-image segment H of the (i-1)th layer compressed image;

[0131] The information acquisition unit is used to acquire summary information of each point in the sub-image segment H of the (i-1)th layer compressed image;

[0132] The edge decompression unit is used to decompress the edges that are compressed into the sub-image segment H in the (i-2)th layer compressed graph.

[0133] In one embodiment, the query response unit 204 includes:

[0134] Variable setting unit, used for each superpoint v H Set the boolean variable v H .cvg, to represent the convergent superpoint;

[0135] The first determination unit is used to determine whether, based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has converged, then it skips the corresponding superpoint v. H The calculation and update are performed, and query responses are based on point decompression functions and summary functions;

[0136] The second determination unit is used to determine whether, based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has not converged, then the corresponding superpoint v is interrupted and decompressed. H The included edges are traced back to the original graph data, and all nodes in the unconverged original graph data are updated. Then, query responses are performed by decompressing the hyperedges.

[0137] The third determination unit is used to determine whether, based on the Boolean variable v H .cvg determines the corresponding superpoint v H If the convergence value is less than the preset convergence threshold, then the corresponding superpoint v will be... H Mark as convergent, and check all points corresponding to the superpoint v. H The system checks whether points compressed to the same superpoint have converged, marks converged superpoints, and provides query responses based on the point decompression function and the summary function.

[0138] In one embodiment, the query response unit 204 further includes:

[0139] The first strategy setting unit is used to set a first summary information generation strategy for connectivity issues caused by label restrictions.

[0140] An expansion unit is used to obtain the query point u and start expansion with the superpoint connected to the query point u as L.

[0141] The set addition unit is used to add the corresponding superpoint to a preset set when the expanded superpoint satisfies the first summary information generation strategy.

[0142] In one embodiment, the query response unit 204 further includes:

[0143] The second strategy setting unit is used to set the second summary information generation strategy for the graph pattern matching problem;

[0144] The matching unit is used to continuously decompress superpoints into the original graph data when a graph pattern is obtained for matching in the original graph data, and to check the decompressed superpoints according to the second summary information generation strategy to complete the graph pattern matching.

[0145] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0146] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0147] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, power supplies, and other components.

[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0149] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A graph computation method based on hierarchical compression technology, characterized in that, include: Obtain the original image data, and use a hierarchical compression algorithm to perform hierarchical compression on the original image data to generate a k-level compressed image; Based on each level of the compression graph, generate the corresponding set of compression functions; The compression function set is used to decompress each level of the compressed graph; Furthermore, the compression function group is used to query and respond to each level of the compressed graph; The process of acquiring the original image data and then performing hierarchical compression on the original image data using a hierarchical compression algorithm to generate a k-level compressed image includes: The first function is called to divide the original graph data into multiple subgraphs, and the original graph data is compressed according to a preset topological order. The edges of the original graph data are compressed into super edges between superpoints of the first layer compressed graph until the size of the first layer compressed graph can no longer be compressed or meets the preset compression size requirements. The second function is called to compress the edges of the original graph data until it meets the preset compression size requirements, thereby generating the first layer of compressed graph. The first layer compressed graph is further compressed using the first function and the second function until the original graph data is compressed into a k-level compressed graph. The process of decompressing each level of the compressed graph using the compression function set includes: When decompressing the superpoint vH of the i-th layer compressed image, query the corresponding decompression function to obtain the sub-image segment H of the (i-1)-th layer compressed image; Obtain summary information of each point in sub-image segment H of the (i-1)th layer compressed image; Decompress the edges that are compressed into sub-image segment H in the (i-2)th layer compressed image.

2. The graph computation method based on hierarchical compression technology according to claim 1, characterized in that, The step of generating a corresponding set of compression functions based on each level of the compression graph includes: For the compressed graph at layer i, the compression function set at layer i is generated based on the compression relationship: , , , >, among which This represents the compression function of the i-th layer compressed graph. The summary function representing the compressed graph of the i-th layer. Let i represent the decompression function of the i-th layer compressed graph. This represents the point compression function of the i-th layer compressed graph.

3. The graph computation method based on hierarchical compression technology according to claim 2, characterized in that, The process of querying and responding to each level of the compressed graph using the compression function set includes: For each superpoint v H Set the boolean variable v H .cvg, to represent the convergent superpoint; If based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has converged, then it skips the corresponding superpoint v. H The calculation and update are performed, and query responses are based on point decompression functions and summary functions; If based on the Boolean variable v H If the .cvg code determines that the corresponding superpoint has not converged, then the corresponding superpoint v is interrupted and decompressed. H The included edges are traced back to the original graph data, and all nodes in the unconverged original graph data are updated. Then, query responses are performed by decompressing the hyperedges. If based on the Boolean variable v H .cvg determines the corresponding superpoint v H If the convergence value is less than the preset convergence threshold, then the corresponding superpoint v will be... H Mark as convergent, and check all points corresponding to the superpoint v. H The system checks whether points compressed to the same superpoint have converged, marks converged superpoints, and provides query responses based on the point decompression function and the summary function.

4. The graph computation method based on hierarchical compression technology according to claim 2, characterized in that, The query response for each level of the compressed graph via the compression function set also includes: To address the connectivity issues caused by tag restrictions, a strategy for generating the first summary information is set. Get the query point u, and start expanding from the superpoint connected to the query point u as L; When the expanded superpoints satisfy the first summary information generation strategy, the corresponding superpoints are added to a preset set.

5. The graph computation method based on hierarchical compression technology according to claim 4, characterized in that, The query response for each level of the compressed graph via the compression function set also includes: To address the graph pattern matching problem, a second summary information generation strategy is set. When a graph pattern is matched with the original graph data, superpoints are continuously decompressed into the original graph data, and the decompressed superpoints are checked according to the second summary information generation strategy to complete the graph pattern matching.

6. A graph computing device based on hierarchical compression technology, characterized in that, include: A hierarchical compression unit is used to acquire the original graph data and perform hierarchical compression on the original graph data using a hierarchical compression algorithm to generate a k-level compressed graph. The function generation unit is used to generate a corresponding set of compression functions based on each level of the compression graph; The decompression unit is used to decompress each level of the compressed graph using the set of compression functions; The query response unit is used to query and respond to each level of the compressed graph through the compression function group; The hierarchical compression unit includes: The first calling unit is used to call the first function to divide the original graph data into multiple subgraphs, and to compress the original graph data according to a preset topological order, compressing the edges of the original graph data into superedges between superpoints of the first layer compressed graph, until the size of the first layer compressed graph cannot be compressed further or meets the preset compression size requirements. The second calling unit is used to call the second function to perform edge compression on the original graph data until it meets the preset compression size requirements, thereby generating the first layer compressed graph. A continuous compression unit is used to continue compressing the first layer compressed graph using a first function and a second function until the original graph data is compressed into a k-level compressed graph. The decompression unit includes: The function query unit is used to query the corresponding decompression function when decompressing the superpoint vH of the i-th layer compressed image, so as to obtain the corresponding sub-image segment H of the (i-1)-th layer compressed image; The information acquisition unit is used to acquire summary information of each point in the sub-image segment H of the (i-1)th layer compressed image; The edge decompression unit is used to decompress the edges that are compressed into the sub-image segment H in the (i-2)th layer compressed graph.

7. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the graph computation method based on hierarchical compression technology as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the graph computation method based on hierarchical compression technology as described in any one of claims 1 to 5.

Images