A method and system for graph compression based on dense subgraphs
By using a graph compression method based on dense subgraphs, we obtain K-edge connected subgraphs and perform node recoding and adjacency list compression, which solves the problems of storage redundancy and low search efficiency in large-scale graph data, and realizes efficient graph data storage and query.
Patent Information
- Application Number
- CN202210926540.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-03
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2042-08-03
AI Technical Summary
Existing technologies struggle to effectively compress large-scale graph data, especially compression methods for K-edge connected subgraphs, which have not been adequately explored, resulting in high storage redundancy and low search efficiency.
A graph compression method based on dense subgraphs is adopted. By obtaining a K-edge connected subgraph, node recoding and adjacency list compression are performed to construct a node index. The characteristics of dense subgraphs are used to reduce node gaps and redundancy.
It effectively reduces the data storage space requirements of graph data, improves search efficiency, reduces the gap distance between nodes, and simplifies the storage and querying process of graph data.
Smart Images

Figure CN115439564B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer software, and relates to graph data compression, in particular to a graph compression method and system based on dense subgraphs. BACKGROUND
[0002] With the rapid development of Internet, mobile Internet and WEB3 technologies, numerous emerging products and applications are creating a large amount of data at an unprecedented speed and in a manner. As a data structure for capturing and describing relationships, a graph is generally understood as a set of relationships between nodes. Graph data is very large, and how to effectively describe, store, analyze and use the graph data has become a major opportunity and challenge faced by many industries and fields.
[0003] Numerous studies have shown that many data mining tasks (for example, detecting abnormal nodes, identifying interest groups, estimating centrality measures, etc.) can be transformed into graph mining problems using graph information, and can be solved by appropriate graph algorithms. These algorithms usually assume that graph data is stored in memory, and is stored in the form of an adjacency matrix or an adjacency list. When the size of the edges in the graph data reaches billions, hundreds of billions or even higher, such assumptions become difficult to achieve. For example, the adjacency matrix of a graph with a million nodes is about 116 GB. According to the 48th Statistical Report on the Development of China's Internet issued by China Internet Network Information Center, the number of Internet users in China has reached 1.011 billion. For the storage of relationships between Internet users, using an adjacency matrix model for storage will reach the TB scale. How to store such a large-scale graph data, domestic and foreign researchers have made a lot of research mainly from the following three aspects: (1) memory technology (graph compression technology): store the graph in memory after compression, reduce the redundancy in the graph or describe the same information in a more compact structure, and the search of the graph usually needs to be decompressed. (2) distributed technology: divide the graph, and store the graph by more devices to share the cost, but this will bring more communication overhead. (3) external storage technology: store the graph in a low-cost hard disk, and design an efficient storage scheduling algorithm to reduce the IO overhead.
[0004] Among the above three methods for solving large-scale graph storage, the present application mainly discusses the graph compression method, which can improve the performance of the other two methods and be used in device-limited scenarios.
[0005] For the compression of the storage of large-scale graph, it is important to sort the nodes so that the gap between adjacent nodes is reduced, and the connectivity between nodes does not correspond to the node number, so the graph compression is often converted into a search for frequent subsets in the graph or into a problem of finding dense substructures in the graph. The use of dense subgraphs can greatly reduce the redundant information in the graph and approach the lower limit of entropy storage. In existing methods, there is a lack of compression discussion on K-edge connected components (K-ECC). K-edge connected component refers to a subgraph G' on a graph G, which is K-edge connected, and if G' is not contained by other K-edge connected graphs, then G' is a K-edge connected component, referred to as K-ECC. SUMMARY
[0006] For the randomness problem of nodes in the graph, the present application provides a graph compression method and system based on dense subgraphs, and designs a sorting and re-encoding mode for K-ECC, which compresses the graph by reducing the gap between adjacent nodes.
[0007] The technical scheme adopted by the present application is as follows:
[0008] A graph compression method based on dense subgraphs, comprising the following steps:
[0009] Obtaining a K-edge connected component;
[0010] Re-encoding the K-edge connected component and the nodes inside the K-edge connected component so that the nodes adjacent in structure have adjacent node numbers;
[0011] Compressing the adjacency list of the nodes and constructing a node index.
[0012] Further, the re-encoding of the K-edge connected component and the nodes inside the K-edge connected component comprises: sorting the K-edge connected component, i.e. the dense set; and re-encoding the nodes inside the K-edge connected component.
[0013] Further, all nodes inside the dense set are presented to all nodes outside the dense set in the form of a super node, and the nodes inside the dense set are generated by the node sequence L in the Mas strategy.
[0014] Further, the sorting of the K-edge connected component is based on the following formula:
[0015] d V =∑deg in (V)-∑deg out (V)
[0016] Wherein, d V represents the encoded dense set, deg in(V) represents the out-degree, deg out (V) represents the out-degree, V represents the node set.
[0017] Further, the set number and the node number are distinguished by an identification bit, the identification bit is set to 8 bits, and the first bit is 0.
[0018] Further, the adjacency list of the node is compressed, comprising:
[0019] The edges are stored by using the adjacency list, the adjacency list includes the adjacency list of the node in the same set and the adjacency list of the node in the different set; all the edges in the same set are compressed by interval coding, and a 4-bit gap bit is used for recording, and a 4-bit first bit is used for recording whether there is a subsequent; for the non-same set neighbor, an identification bit is set to express that the next 8 bits are a set number;
[0020] The content stored by the adjacency list only contains the target node of the edge, and the type of the edge is stored by using another space;
[0021] BWT compression is used for character-level compression.
[0022] Further, the node index is constructed, which is a two-level index, comprising a first set index and a second node index, wherein the first set index is composed of a set array, and the second node index is composed of a node array.
[0023] A graph compression system based on dense subgraph, comprising:
[0024] A K-ECC acquisition module is configured to acquire a K-edge connected subgraph.
[0025] A re-encoding module is configured to re-encode the K-edge connected subgraph and the nodes in the K-edge connected subgraph, so that the nodes adjacent in structure have adjacent node numbers.
[0026] An adjacency list compression and index construction module is configured to compress the adjacency list of the node and construct the node index.
[0027] The key points of the present application include:
[0028] 1. The present application introduces a new dense subgraph K-ECC and a multi-layer encoding strategy for compression, controls the size of the target compression subgraph, and reduces the size of the ordered graph.
[0029] 2. The present application encodes the introduced dense subgraph and the nodes in the subgraph, so that the nodes adjacent in structure have adjacent node numbers.
[0030] 3. The present application compresses the adjacency list of the node by using gap coding and text compression, so that the storage of the adjacency list costs less.
[0031] The beneficial effects of the present application are as follows:
[0032] In the graph data, the nodes within the dense subgraph usually have more edges (because the average degree of the subgraph is greater than the global), and the nodes at both ends of the edge are repeated, which also leads to more space occupied by the nodes within the dense subgraph in storage. The present application proposes a new and relatively simple sorting method, which aims to reorder and recode the nodes using the dense subgraph, so that the sorting does not need to be performed on the large graph, thereby reducing the time of node sorting and reducing the redundancy consumption of each node in the memory of the dense subgraph. The nodes in the extracted K-edge connected subgraph are recoded, and a new storage representation of the edge is constructed, so that the space occupied by the nodes with high frequency in the graph can be reduced. For the sparse space of the graph, the compressed adjacency list is used to reduce the storage consumption and the two-level index is used to improve the search efficiency of the node neighbors. This method has three advantages. First, it effectively reduces the space required for storing the graph in the memory. Second, the method is very simple and easy to use. Finally, the method reduces the gap distance between nodes. BRIEF DESCRIPTION OF DRAWINGS
[0033] Figure 1 is a schematic diagram of a graph compression framework based on a dense subgraph. Among them, 1-4 represents a two-level node index, S1, S2 represents a one-level set index, Set1, Set2 represents a node set, node1, node2, node3 represents a node, block1, block2 represents an adjacency list compression block, N(node1) represents the adjacent nodes of node1, and "┐S1" represents a non-same set neighbor.
[0034] Figure 2 is a schematic diagram of single node storage.
[0035] Figure 3 is a schematic diagram of edge storage encoding (taking 1000020→323884332 as an example).
[0036] Figure 4 is an adjacency list storage representation graph.
[0037] Figure 5 is a node index structure diagram.
[0038] Figure 6 is a node gap comparison diagram (circles represent MLogGapA, and plus signs represent the present application scheme). DETAILED DESCRIPTION
[0039] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below through specific embodiments and drawings.
[0040] The problem of graph compression on subgraph extraction can be converted into representing the initial graph as a graph G, extracting the K-ECC subgraph, reordering the subgraph and the points in the subgraph, and re-encoding to determine the storage symbol of each node, so that each edge on the initial graph is stored in the memory without loss and the required storage space is reduced. The scheme of the present application assumes that the K-ECC subgraph has been extracted by a related algorithm.
[0041] The overall framework of the graph compression method based on dense subgraph proposed by the present scheme is shown in Figure 1 The model includes two parts: re-encoding (ReCode) and adjacency list compression and index construction (NebCompress). The ordering re-encoding encodes the nodes in the two parts, i.e., the dense set and the nodes in the dense region, so that the nodes with smaller serial numbers have more edges; the adjacency list compression and index construction reorganizes the non-dense region part to make the neighbor lookup and storage of specific nodes have good performance. If a derived set omits the element list when defined, it means that its elements are all combinations of all parent elements, and such a set is called a dense set. The dense region refers to the region with high similarity and high locality in the graph. The specific embodiments will be described in detail.
[0042] 1. Node re-encoding
[0043] In order to reduce the type of edge graph, the similar and dense characteristics of K-ECC subgraph structure can be used to set the nodes with similar node numbers to reduce the number of edge graphs, thereby reducing the coding range. The work of this step is to sort the divided subgraph and design a coding method.
[0044] The obtained dense region is sorted to make the gap between the nodes in the graph smaller and more conducive to controlling the storage cost. First, without considering the coding of the nodes in the dense set, assume that all nodes in the dense set are presented to all nodes outside the dense set in the form of a super node. Through this assumption, all node sets with high probability height in the original graph can be combined into a node, so that the locality and similarity of the local region of the original graph containing many such dense sets are reduced, and the graph becomes relatively more sparse. For such a more sparse graph than the original graph, the coding scheme of the present application is to number in turn after reordering. The comparison standard of the sorting is shown in the following formula.
[0045] d V = ∑ deg in (V) - ∑ deg out (V)
[0046] Where d V represents the encoded dense set, deg in(V) represents the in-degree, deg out (V) represents the out-degree, V represents the node set.
[0047] The dense set is understood as a node, and for a node, the in-degree of the node is important because after sorting, the storage order of all edges in memory is stored with the node as the head node, so it is difficult to find the edge search with the node as the target node, and the edge with the node as the source node can be compressed through the adjacency list to eliminate the above repeated head node. Therefore, the purpose of the rule of in-degree minus out-degree for sorting is to reduce the difficulty of modifying the node as the target node.
[0048] Secondly, only the nodes in the dense set are considered, and the coding mode of the nodes in the dense set is consistent with the coding mode of the dense set, but no sorting is done here because the nodes in the dense set obtained are generated from the node sequence L in the Mas strategy, and L has its own sorting, and the sorting rule is based on the size of the connection degree of the node v in L and the internal node joined before v in L. It should be noted that all data sets of the present scheme are based on the premise of virtual bidirectional edge. Among them, the Mas strategy uses the existing method to realize it, which divides the parent graph into two parts, one is the node sequence L, and the other is the remaining subgraph. For L, it is judged by s-t cut whether the last two nodes belong to a K-ECC, and for the remaining subgraph, it is randomly selected to re-perform the above process.
[0049] After reordering, all nodes will be converted from the original data storage of two 64-bit or 32-bit nodes to eight or four 8-bit node numbers. In order to remove the leading zeros in storage and distinguish them from subsequent nodes, an identification bit is used as an isolation here. The flag is set to 8 bits, the first bit is 0, as shown in Figure 2 This distinguishes the set number from the node number. In addition to the two ends of the edge itself, the type of the edge itself also needs to be stored. Here the present scheme takes the edge type of GitHub as the assumed edge type, and does not consider the actual character storage, but uniformly stores it with a single numerical character, assuming that it costs 4 bits, which can express 16 types of edges. The whole edge storage coding process is as shown in Figure 3
[0050] 2. Adjacency list compression and index construction
[0051] Storing each edge will consume half of the space for the head node of the edge, which can be handled by the adjacency list, but the introduction of the adjacency list will lead to the previous storage scheme of the edge is not efficient. This step will introduce the arrangement of the adjacency list and the construction of the node index. Respectively, in order to deal with the repeated head node and the compressed data search problem.
[0052] The adjacency list compression is based on the idea of gap storage. For each node out edge, the present scheme considers that all the same set of edges are compressed by interval coding. This is because interval coding can make good use of the fact that the nodes in the dense subgraph in the above work are very close in structure and close in node number. Therefore, the present scheme adopts an interval coding for these gaps and records it with a 4-bit gap. Considering that the 4-bit gap is too small, the node coding in the first part is adopted, and a flag bit is used to record whether there is a follow-up for the first 4 bits. Assuming that a dense set of points v has same set neighbor nodes 3, 4, 18 and non-same set neighbor set number 2 node number 11 and set number 3 node number 19 and 21, for example, the adjacency list representation of the same set of nodes is shown in Figure 4 The non-same set neighbor uses a new representation format, that is, setting the flag (1000) (setFlag) to express that the next 8 bits are a set number, and the rest is similar. At the same time, the content stored in the table only contains the target node of the edge, and the type of the edge is stored in another space. When the same set of edges is too many, the node number is represented by a byte number equal to the number of nodes, and when there is more than one one-way edge between two nodes, the type of the edge can be stored by using the successor bit code, further reducing the total storage of the edge.
[0053] In addition to the new representation in the adjacency list, BWT compression is also used for character-level compression. There are many ways to compress characters, and BWT compression is used because each byte contains a very short piece of information, such as a part of the set number or the node number. In this case, it is difficult to find long repeated strings using run-length encoding. Moreover, each node number in the same set appears only once, and even if there are multiple sets of node numbers, due to the scattered distribution, they rarely appear many times. Under multiple obstacles, BWT is chosen for character compression.
[0054] When the size of the edge reaches billions or even hundreds of billions, it becomes very difficult to find each node on the graph. Therefore, designing a good index representation is also very important for the compression of the graph.
[0055] When all the above steps are processed, the current data already has some characteristics, for example, each node has a set number (the nodes of the non-K-ECC subgraph are also constructed into a set to participate in sorting), and each node has an adjacency list, therefore, the application designs a two-level index, including a first set index and a second node index, the first set index is composed of a set array, and the second node index is composed of a node array. Arrays are used to facilitate direct indexing to the node when querying. The deployment structure of the index is as shown in Figure 5 .
[0056] The application designs experiments to prove the technical advancement of the scheme from three aspects. (1) Compared with the existing compression technology based on dense subgraph, how is the compression rate of the compression algorithm proposed by the application for the original graph? (2) How is the compression rate of the compression algorithm of the application on various data sets? (3) Does the scheme reduce the gap distance between the nodes of the graph after sorting?
[0057] The application is divided into two parts for comparative experiments, namely, the compression rate and compression time comparative experiment and the node gap comparative experiment after re-encoding.
[0058] The hardware configuration of the application experiment is as follows: Intel i7-11800h CPU (main frequency: 4.60GHz 8 cores 16 threads), 128GB DDR3 memory, Linux ubuntu 18.04 (64-bit) operating system. The code is realized in C++, compiled by gcc 8.2.1 and uses O3 optimization.
[0059] The application experiment uses several publicly available constructed graphs, network graphs and social networks, as shown in Table 1. The expansibility of the application is shown by using these data sets. Before running the test, all the graphs are subjected to K-ECC extraction. Because of the lack of type data sets, a type is randomly added to each edge in the experiment. The following is the data set used in the experiment:
[0060] (1) R-MAT is a constructed set, which uses a simple recursive model with only a few parameters to quickly generate realistic graphs. Using this model, weighted graphs, directed graphs and bipartite graphs can be easily generated; using R-MAT, graphs that meet the famous probability can be generated and can meet the power law distribution. We use R-MAT to generate weighted graphs with integer weights to simulate the type of edges.
[0061] (2) web-Google is a web graph, where each edge represents a hyperlink between web pages. This data was released by Google in 2002; the data can be found at https: / / snap.stanford.edu / data / web-Google.html.
[0062] (3) Twitter is a public Twitter social data set with 41 million nodes and 240 million edges.
[0063] (4) LiveJournal is a free online community with nearly 10 million members; a significant portion of these members are very active (e.g., approximately 300,000 update their content in any given 24 hours).
[0064] (5) ca-MathSciNet is a data set of co-occurrence relations of reviews, abstracts, and bibliographic references of mathematical science literature. The data set can be found at https: / / networkrepository.com / ca-MathSciNet.php.
[0065] Table 1 Main properties of the data sets used in the compression experiments
[0066]
[0067] Fig. refers to the K value of the graph with the most nodes and the processing time is less than 1000s, and the data set uses K = 8 as the split graph parameter. The edge type occupies four bits, corresponding to 16 edge types.
[0068] For different scenarios, the present application hopes to consider as many different types of general graphs as possible, and in the experiment, space and compression time are used to describe. For the compression space, the average number of bits per edge (bits per edge, hereinafter referred to as bpe) and the compression ratio are used to describe, where G c represents the total number of bytes after compression, and G represents the total number of bytes before compression. In terms of time, the total time consumption (seconds s) is used to describe. In terms of node gap, the present application subtracts the node numbers of the two ends of the edge in the graph, and for the nodes between different sets of compressed graphs processed by the present application, the following formula is used to describe.
[0069]
[0070] where Set i .n is the number of nodes in set i, node s is the node number in the head node set, and node sNode number of tail node. In order to facilitate the statistical sorting effect and solve the problem that the node after sorting is not consistent with the original node, a node is stored in the form of node+adjacency list, and the bpe is expressed as (total byte number of a node / a number of neighbors of a node).
[0071] The baseline used in the experiment is the following three algorithms:
[0072] (1) GraphZip algorithm: find the clique structure in the graph and record the first node number of the clique and the non-clique node number to compress into a supernode, so that the query graph is smaller.
[0073] (2) MLogGapA algorithm: reorder the nodes of the graph using the bipartite graph to narrow the gap between the nodes of the graph.
[0074] (3) CC algorithm: explore and compress the storage of the clique structure in the graph, and provide a new search method for the query storage of the clique.
[0075] 1. Compression rate and compression time comparison
[0076] In the experiment, the method proposed in the application is compared with the benchmark algorithms GraphZip and CC algorithm on the above several data sets to verify the effectiveness of the dense subgraph method based on node recoding in time and space. The experimental results are shown in Table 2, wherein the best experimental results in each index are marked in bold by the application.
[0077] The experimental results in Table 2 show that, compared with the existing compression scheme based on dense subgraph, the node recoding compression method proposed in the application has better compression effect and higher time efficiency for a larger graph. It can be seen from the experimental results that, compared with the GraphZip algorithm, the bpe of the compressed graph is 2-5 bits lower on the sets web-Google, Twitter and LiveJournal, and the GraphZip method cannot effectively complete the graph compression work in a short time (10000s) on a larger graph such as twitter. However, GraphZip and the present method have different efficiencies in the average degree of different scenarios. On the web-Google data set and the ca-MathSciNet data set, the present method has slightly weaker growth than GraphZip from the average degree of the graph, and the density of the web-Google data set is equivalent to that of ca-MathSciNet but the average degree is 2.3 times different. On such a denser graph, the growth rate of the present method is slightly weaker than that of GraphZip, and one of the possible reasons is that the increase in the average degree leads to an increase in the number of groups. Compared with the CC algorithm, the present method has better effect on most data sets, and the compression rate of the present method is higher than that of the CC algorithm. Since the CC algorithm allows query in compressed form, the present method actually needs to perform more operations on the graph during decompression. In addition, it can be seen from the comparison between web-Google and ca-MathSciNet that the compression of web graph is easier than that of social graph.
[0078] Table 2 results of time and space comparison
[0079]
[0080] 2. Change of node gap
[0081] In the node sorting, the present experiment will be compared with the MLogGapA algorithm. Figure 6 The figure shows the number of nodes corresponding to the gap length of the graph on the LiveJournal data set, the x-axis is the gap length, and the y-axis is the number of nodes. It can be seen that the node sorting of the present application is better than the natural sorting, and compared with MLogGapA, the present application converges more nodes in the gap length of 10-1000 and has fewer nodes in larger gaps and more nodes in low gaps. At the same time, it is found in the experiment that the graph size used by the present application in each round of sorting is the subgraph size, which also provides the possibility for parallel algorithm. It can be seen that the present application helps to reduce the node gap, which is the reason why the present method is superior to other methods in time and space compression.
[0082] Based on the same inventive concept, another embodiment of the present application provides a graph compression system based on dense subgraph, comprising:
[0083] a K-ECC obtaining module, configured to obtain a K-edge connected subgraph;
[0084] a re-encoding module, configured to re-encode the K-edge connected subgraph and the nodes inside the K-edge connected subgraph, so that the nodes that are adjacent in structure have adjacent node numbers;
[0085] a neighbor list compression and index construction module, configured to compress the neighbor list of the nodes and construct a node index.
[0086] The specific implementation process of each module can be found in the description of the method of the present application above.
[0087] Based on the same inventive concept, another embodiment of the present application provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present application.
[0088] Based on the same inventive concept, another embodiment of the present application provides a computer readable storage medium (such as ROM / RAM, magnetic disk, optical disk), which stores a computer program, and the computer program is executed by a computer to realize each step of the method of the present application.
[0089] The specific embodiments of the present application disclosed above are intended to help understand the content of the present application and to implement the same, and those skilled in the art can understand that various replacements, changes and modifications are possible without departing from the spirit and scope of the present application. The present application should not be limited to the content disclosed in the embodiments of the present application, and the protection scope of the present application is defined by the scope of the claims.
Claims
1. A graph compression method based on dense subgraphs, characterized in that, Includes the following steps: Obtain the K-edge connected subgraph; Re-encode the K-edge connected subgraph and its internal nodes so that structurally adjacent nodes have adjacent node numbers; Compress the adjacency list of the nodes and build the node index; The recoding is applied to nodes within dense sets and dense regions, so that nodes with smaller indices have more edges. If a derived set omits the element list during definition, then the elements of the set consist of all combinations of all parent elements; such a set is called a dense set. Dense regions refer to areas with high similarity and high locality in the graph. Adjacency list compression and index construction reorganize the non-dense regions to achieve good performance in finding and storing neighbors of specific nodes. The compression of the adjacency list of nodes includes: Edges are stored using an adjacency list, which includes adjacency lists for nodes within the same set and adjacency lists for nodes outside the same set. All edges within the same set are compressed using a gap encoding and recorded using 4-bit gap bits. The first bit of the 4-bit gap bits is used as an identifier to indicate whether there are any subsequent edges. For non-same set neighbors, an identifier is set to indicate that the next 8 bits are a set number. The adjacency list stores only the target node of the edge, and uses another space to store the edge type; Use BWT compression for character-level compression.
2. The method according to claim 1, characterized in that, The recoding of the K-edge connected subgraph and its internal nodes includes: sorting the K-edge connected subgraph; and recoding the internal nodes of the K-edge connected subgraph.
3. The method according to claim 2, characterized in that, All nodes within the dense set are presented to all nodes outside the dense set as a supernode. The nodes within the dense set are generated from the node sequence in the Mas policy.
4. The method according to claim 2, characterized in that, The sorting of the K-edge connected subgraph is based on the following formula: d V =Σdeg in (V)-∑deg out (V) Where, d V Deg represents the encoded dense set. in (V) represents in-degree, deg out (V) represents the out-degree, and V represents the set of nodes.
5. The method according to claim 2, characterized in that, A flag is used to distinguish the set number from the node number. The flag is set to 8 bits, with the first bit being 0.
6. The method according to claim 1, characterized in that, The construction of the node index involves creating a secondary index, which includes a primary set index and a secondary node index. The primary set index is composed of a set array, and the secondary node index is composed of a node array.
7. A graph compression system based on dense subgraphs, characterized in that, include: The K-ECC acquisition module is used to acquire K-edge connected subgraphs; The recoding module is used to recode the K-edge connected subgraph and its internal nodes so that structurally adjacent nodes have adjacent node numbers. The adjacency list compression and index building module is used to compress the adjacency list of nodes and build node indexes. The recoding is applied to nodes within dense sets and dense regions, so that nodes with smaller indices have more edges. If a derived set omits the element list during definition, then the elements of the set consist of all combinations of all parent elements; such a set is called a dense set. Dense regions refer to areas in the graph with high similarity and high locality. Adjacency list compression and index construction reorganize the non-dense regions to achieve good performance in finding and storing neighbors of specific nodes. The compression of the adjacency list of nodes includes: Edges are stored using an adjacency list, which includes adjacency lists for nodes within the same set and adjacency lists for nodes outside the same set. All edges within the same set are compressed using a gap encoding and recorded using 4-bit gap bits. The first bit of the 4-bit gap bits is used as an identifier to indicate whether there are any subsequent edges. For non-same set neighbors, an identifier is set to indicate that the next 8 bits are a set number. The adjacency list stores only the target node of the edge, and uses another space to store the edge type; Use BWT compression for character-level compression.
8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Dense graph adjacency list compression method
CN107564075A
Sparse matrix compression method and device, equipment and storage medium
CN110334067A