A method for fine-grained management of vertex updates and a multi-version graph system

Abstract

The application discloses a fine-grained vertex update management method and a multi-version graph system. The multi-version graph system performs tasks on different graph versions; the multi-version graph system comprises a graph storage module, an update storage module, an update processing module, a version switching mode selection module, a version switching module and a graph processing module; the update processing module periodically collects updates on the graph and classifies the updates, and the updates are stored in the update storage module; the version switching mode selection module adopts the fine-grained vertex update management method to select a suitable update mode for each vertex in different versions; the version switching module is used for generating a target version graph from the system; and the graph processing module performs corresponding graph calculation on the target version graph. The application utilizes the different performances of the delta mode and the copy mode in different update times, and fine-grained management of the vertex update mode, so that efficient and flexible version graph switching is realized, and the memory and switching time efficiency during version switching of the multi-version graph system is improved.

Description

Technical Field

[0001] The present invention relates to the field of dynamic graph computing, and in particular to a method for fine-grained management of vertex updates and a multi-version graph system. Background Art

[0002] Graphs are widely used in various fields to represent relationships between entities, such as in social networks, financial transactions, and viral spread. Since graphs are constantly evolving in the real world, they can generate many versions over time, with each version corresponding to a snapshot of the evolving graph at a specific point in time.

[0003] A typical multi-version graph system typically consists of two core components: version switching and graph processing. Version switching aims to access the target graph version on demand, and multi-version graph systems require access to all graph versions. Graph processing then executes graph algorithms on the target version to extract useful information. Against this backdrop, graph processing has seen significant improvements in existing high-performance graph processing systems and accelerators. However, as graph processing efficiency improves, version switching has gradually become a new bottleneck in multi-version graph systems.

[0004] Graphs are usually stored in two formats: Compressed Sparse Row (CSR format) and Vector of Vectors (VoV format).

[0005] CSR stores all edges of a graph in a neighbor array. Edges to the same vertex are stored contiguously in the neighbor array, while the offset information for each vertex is stored in the vertex array. Clearly, CSR achieves excellent storage compactness and access efficiency at the expense of scalability. However, a single change to an edge can cause modifications to the entire neighbor array and vertex array. Therefore, the CSR format is not efficient for graph updates.

[0006] In the VoV format, each vertex holds a vector to store edges, and these vectors are organized into a new vector. Using this format, graph updates are efficient. However, vectors typically record additional information (e.g., starting position, length), and maintaining a vector structure for all vertices can be expensive.

[0007] Multi-version graph systems typically use two approaches to version switching: (1) storing the complete graph version at each update (copy-based updates), and (2) storing the root version and updates between versions (delta-based updates).

[0008] Copy-based update schemes require storing all graph versions in memory. When accessing a specific graph version, the system can directly retrieve the corresponding version from memory. While copy-based update schemes significantly reduce version switching time, real-world graphs typically contain a large number of vertices and edges, and storing each complete graph version requires a significant amount of memory overhead.

[0009] Delta-based update schemes typically set a specific version as the root version of the system and save updates between adjacent versions. When accessing a specific version, the system simply starts from the root version and performs the corresponding update operations to reach the target version. Delta-based updates reduce memory usage by only saving the corresponding updates, but these update operations require additional computational time when switching versions.

[0010] In summary, existing solutions cannot provide efficient version switching. The copy method reduces the version switching time by sacrificing a large amount of memory, while the delta method reduces memory overhead but incurs additional computing time. All solutions only improve two methods to achieve better performance. In addition, the granularity of vertex updates in the version graph switching of existing systems is too coarse, that is, the system uses the same update method for the same vertex in different versions. However, since vertices are updated differently in different versions, a single update method may cause some vertices to have excessive time or memory overhead when switching versions. Therefore, fine-grained management of vertex updates and selecting the appropriate update method for each vertex in different versions are of great significance to improving the performance of multi-version graphs.

[0011] In summary, the present invention aims to propose a method and system for fine-grained management of vertex updates with high performance in terms of memory and time in multi-version graph switching.

[0012] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the inventor studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0013] In view of the deficiencies in the prior art, the present invention provides a method for fine-grained management of vertex updates and a multi-version graph system.

[0014] To achieve the above object, the present invention provides a method for fine-grained management of vertex updates, comprising the following steps:

[0015] (1) Periodically collect vertex update information, record the operation type and the updated source and destination vertices; for deletion operations, record the position of the destination vertex in the edge vector; classify and store update information by source vertex, and store operations on the same source vertex separately by addition and deletion types;

[0016] (2) Calculate and compare the update cost of each vertex in different versions through the collected update information, including memory cost and version switching time cost; select the appropriate update method for each vertex in different versions, including copy method and delta method; perform version switching when accessing a specific version and update according to the selected method.

[0017] Furthermore, the overhead of each vertex is calculated using the collected update information. The memory overhead of the copy method is the memory overhead of storing all edges of the vertex, and the time overhead is the time overhead of the system accessing the corresponding version. The memory overhead of the delta method is the memory overhead of storing the root version and version update information, and the time overhead is the time overhead of the system updating from the root version to the target version.

[0018] Furthermore, the update cost of each vertex in different versions is calculated and compared, including:

[0019] Calculate the value of each updated vertex and The value of and The size of represents the number of vertex updates, represents the vertex degree, represents the time-memory trade-off weight, Indicates the remaining memory of the system; when ,express The value of is larger, that is, the cost of the copy method is greater than that of the delta method; when ,express The value of is larger, that is, the overhead of the delta method is greater than that of the copy method.

[0020] Furthermore, we select the appropriate update method for each vertex in different versions, including:

[0021] For each version, compare the update costs of the copy method and the delta method under this version. When the copy cost is small, choose the copy method as the update method for this version; when the delta cost is small, choose the delta method as the update method for this version.

[0022] Furthermore, update storage includes copy storage and delta storage:

[0023] Copy storage: The copy method saves all edges of a vertex, and each vertex is stored in a two-level structure. The upper level is an index tree structure. Each non-leaf node in the index tree contains several sequentially arranged version numbers, which are the version numbers of the vertex using the copy method. In addition to the version number, the leaf node of the index tree also saves the index of the corresponding edge vector in memory.

[0024] Delta storage: The delta method saves vertex update information; each vertex has an add queue and a delete queue; the add queue saves the destination vertex to be added; the delete queue saves the index of the destination vertex to be deleted.

[0025] Furthermore, version switching includes version query and version update:

[0026] Version query: When a version needs to be accessed, the index tree of each vertex is queried; if the target version can be found in the index tree, the corresponding edge is obtained from the lower structure; if the target version cannot be found in the index tree, the edge of the previous version closest to the target version is obtained from the lower index;

[0027] Version update: perform update operations on all edges where the target version is not found to generate the target version edges; when updating, first execute the delete operation in the delete queue, and then execute the add operation in the add queue; when executing the delete operation, find the position of the target edge in the edge vector according to the delete index, then replace the target edge with the last edge in the edge vector, and finally delete the last edge in the edge vector; when executing the add operation, add all edges that need to be added to the end of the target edge, and finally generate the corresponding edges of the target version; the edges of all vertices are organized into VoV representation to generate the target version graph.

[0028] To achieve the above objectives, the present invention also provides a multi-version graph system for fine-grained management of vertex updates, including a graph storage module, an update storage module, an update processing module, a switching mode selection module, a version switching module, and a graph processing module;

[0029] The graph storage module is used to store the generated version graph data and the root version graph data in the delta mode, and the graph storage adopts the VoV representation mode;

[0030] The update storage module is used to store update information;

[0031] The update processing module includes an update collection submodule and an update classification submodule; the update collection submodule is used to periodically collect update information on the graph; the update classification submodule is used to receive the update information from the update collection submodule and organize and classify the update information according to vertices and update categories;

[0032] The switching mode selection module is configured to calculate and compare the overhead of each vertex in different versions of copy mode and delta mode, select the switching mode with the minimum overhead for each vertex in different versions, and store in the corresponding storage area.

[0033] The version switching module is configured to perform version switching and generate a target version graph.

[0034] The graph processing module is configured to execute a corresponding graph algorithm on the target version graph.

[0035] Further, the update storage module includes a copy storage area and a delta storage area.

[0036] The copy storage area is configured to store information of copy mode, including an index tree and edges of vertices.

[0037] The delta storage area is configured to store information of delta mode, including addition and deletion operation information.

[0038] Further, the version switching module includes a version query submodule and a version update submodule.

[0039] The version query submodule is configured to find the index tree in the copy storage area, query edges of vertices corresponding to the target version graph and edges of the nearest previous version.

[0040] The version update submodule is configured to apply updates in the delta storage area to corresponding edges to generate a target version.

[0041] To achieve the above object, the application further provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the above method for fine-grained management of vertex updates.

[0042] The application has the following advantages: based on vertex updates, the application fine-grained manages switching modes of vertices. According to different features of vertex updates, the application mixes use of copy and delta modes to maximize advantages of the two methods while avoiding their disadvantages, realizes a multi-version graph system with high memory and time performance, and improves efficiency of dynamic graph data processing. BRIEF DESCRIPTION OF DRAWINGS

[0043] To more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can be obtained from these drawings without creative labor:

[0044] Figure 1 It is a schematic diagram of a multi-version graph system for fine-grained management of vertex updates preferred by the present invention;

[0045] Figure 2 It is a schematic diagram of the process of selecting the preferred update method of the present invention;

[0046] Figure 3 This is a schematic diagram of the preferred version switching process of the present invention;

[0047] Figure 4 This is a schematic diagram of a copy in the preferred update storage of the present invention;

[0048] Figure 5 This is a schematic diagram of the preferred update storage delta and update process of the present invention;

[0049] Figure 6 This is a schematic diagram of the logical structure of the preferred update method selection of the present invention;

[0050] Figure 7 This is a schematic diagram of the logical structure of the preferred version switching of the present invention;

[0051] List of reference numerals:

[0052] 1: Graph storage module; 2: Update storage module; 3: Update processing module; 4: Version mode selection module; 5: Version switching module; 6: Graph processing module; 7: Copy storage area; 8: Delta storage area; 9: Update collection submodule; 10: Update classification submodule; 11: Version query submodule; 12: Version update submodule. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In the absence of conflict, the features in the following embodiments and implementation methods can be combined with each other.

[0054] In the current multi-version graph switching, each vertex is switched only using the delta method or the copy method. This will result in additional update calculations when the delta update method is used, resulting in a large amount of switching time overhead. When the copy method is used, copying and storing the complete graph will save a large number of redundant edges, resulting in a large amount of memory waste. The current solution cannot simultaneously meet the requirements of version switching time and system memory utilization when switching multi-version graphs. All solutions have only made good improvements to the delta method and the copy method, but have not considered the impact of vertex update changes on the delta method and the copy method overhead. Therefore, the present invention provides a method for fine-grained management of vertex updates and a multi-version graph system, which aims to select a suitable update method for vertices in each version and fully utilize the advantages of the delta method and the copy method.

[0055] The purpose of the present invention is achieved through the following technical solution: a method for fine-grained management of vertex updates. The method comprises the following steps:

[0056] (1) The system periodically processes vertex updates;

[0057] (2) Manage vertex update methods in a fine-grained manner.

[0058] Update operations mainly include adding and deleting vertices and edges;

[0059] Update methods include copy method and delta method.

[0060] First, the update of the vertex is processed, including update collection and update classification:

[0061] The process for updating a collection is as follows:

[0062] (1.1) Periodically collect vertex update information, record the update type and the updated source and destination vertices, and use [operation, source, destination] to represent it, where operation represents the update type, source represents the source vertex, and destination represents the destination vertex;

[0063] (1.2) Record the position of the destination vertex in the edge vector during the deletion operation, represented by index;

[0064] The process for updating a classification is as follows:

[0065] (1.3) Classify updates by source vertex and store updates with the same source vertex together;

[0066] (1.4) For updates to the same source vertex, classify them according to vertex operations and store add operations and delete operations separately.

[0067] Fine-grained management of vertex update mode, including:

[0068] (2.1) Calculate the update overhead of each vertex in different versions and compare;

[0069] (2.2) Select the appropriate update mode for each vertex in different versions;

[0070] (2.3) When a version needs to be accessed, version switching is performed.

[0071] Calculate the update overhead of each vertex in different versions, including:

[0072] The overhead of updating a vertex is divided into memory overhead and version switching time overhead.

[0073] The system calculates the overhead of each vertex through the collected update information. The memory overhead of the copy mode is the memory overhead of storing all edges of the vertex, and the time overhead is the time overhead of the system accessing the corresponding version. The memory overhead of the delta mode is the memory overhead of storing the root version and version update information, and the time overhead is the time overhead of the system updating from the root version to the target version.

[0074] The overhead calculation formulas of the copy mode and the delta mode are as follows:

[0075]

[0076]

[0077] wherein represents the overhead of vertex i in version j in the copy mode; represents the overhead of vertex i in version j in the delta mode; represents the switching time between two adjacent versions, represents the memory cost of storing each edge, represents the time of adding or deleting operation, represents the memory cost of storing each update, and the above parameters are system constants; is the number of versions, is the number of updates of vertex i from the (j-1)th version to the jth version; represents the degree of vertex i in version j, represents the ideal time cost, represents the ideal memory cost; parameter represents the importance of time and memory; when is 0, only the memory cost is considered; when is 1, only the time cost is considered.

[0078] Compare the update costs of vertices in different versions by using the following methods:

[0079] Since the number of versions in the system changes very little, the impact of the change in the number of versions on the overhead is ignored. By comparing the above formulas, the following approximate formula is obtained to determine the choice of delta and copy methods:

[0080]

[0081] in, Indicates: the relative cost of the delta method compared to the copy method, through To estimate, The higher the value, the greater the overhead of the delta method; Represents the time-memory trade-off weight, which is determined by the system and Decide, ; Indicates the remaining memory of the system. represents the number of vertex updates, Represents the vertex degree.

[0082] The selection threshold represents the dynamic threshold used in the system to decide between delta and copy strategies and is defined as , this threshold is used to measure the system's trade-off between memory and performance. The specific value can be set by comparing the total cost of storing / updating an edge. To balance storage and time overhead; the threshold is selected by and Jointly determine, Depend on Determine, indicating the importance of memory and time in the system. At the same time, the selection threshold will change dynamically with the change of the remaining memory in the system. The remaining memory in the system gradually decreases with the allocation of vertex switching mode. Gradually becomes higher. Greater than the selection threshold , indicating that the delta method has a high overhead, the system selects the copy method for the vertex in this version and stores all the edges of the vertex in the current version; if Less than , indicating that the copy overhead is high, the system selects the delta method for the vertex in this version, that is, storing all updates of the vertex switching to the current version;

[0083] In one embodiment, , when a new version needs to be generated, calculate the value of each updated vertex and The value of and The size of express The value of is larger, that is, the cost of the copy method is greater than the delta method, and the delta method is selected as the update method for this version; when express The value of is larger, that is, the overhead of the delta method is greater than that of the copy method, and the copy method is selected as the update method for this version.

[0084] Select the appropriate update method for each vertex in different versions, including:

[0085] For each version, compare the update costs of the copy method and the delta method under this version. When the copy cost is small, choose the copy method as the update method for this version; when the delta cost is small, choose the delta method as the update method for this version.

[0086] Update method selection Figure 2 As shown, perform the following process:

[0087] First, we process updates between versions, that is, updates from version i to version i+1. These updates are represented as [operation, source, destination]. We then perform a classification operation on the updates, assigning them to each vertex. Then, we classify the updates into either add or delete operations based on the type of update operation.

[0088] Secondly, select the appropriate update method for each vertex. By processing the update operation, calculate the updated vertex and , then and For comparison, when When , it means that the copy overhead is greater, and the delta update method is selected. When , it means that the delta overhead is greater, and the copy update method is selected.

[0089] Finally, the copy and delta storage methods. For the copy method, a two-level structure is used for storage. The upper level is the index tree. When a new copy is generated at a vertex, the version number corresponding to the copy needs to be inserted into the index tree. At this time, it is necessary to check whether the index tree meets the balance condition. The maximum number of keywords in the index tree node is , when inserting a new keyword greater than When a node is copied, it is necessary to split and rotate the node. At the same time, the copied edges are stored in order in the lower-level edge queue. For the delta method, the delete operation records the position information of the deleted vertex as an index and stores the index in the delete queue; the added vertex in the add operation is stored in the add queue.

[0090] Update storage includes copy storage and delta storage:

[0091] Copy storage. This method stores all edges of a vertex, using a two-level structure for each vertex. The upper level is an index tree structure. Each non-leaf node in the index tree contains several sequentially arranged version numbers, which correspond to the vertex's copy version number. In addition to the version number, the leaf nodes of the index tree also store the in-memory index of the corresponding edge vector.

[0092] Delta storage. The delta method stores vertex update information. Each vertex has an add queue and a delete queue. The add queue stores the destination vertex to be added, represented by destination; the delete queue stores the index of the destination vertex to be deleted, represented by index.

[0093] Figure 4 The two-level structure of the copy storage method is introduced. The figure shows the copy storage method of vertex A. The upper level is an index tree. Each node of the index tree stores the maximum Version numbers are arranged sequentially. Each node in the index tree has n indexes, where n is the number of version numbers in that node. At leaf nodes, the version number index points to the edge queue of the lower structure. The lower structure stores the copy queue for vertex A. Each element in the queue consists of two parts: the version number and the edge corresponding to the version. Elements in the queue are arranged in version number order.

[0094] Figure 5 This section describes delta storage and the delta update process. The figure shows the delta storage and update process for vertex A. During the update process for vertex A, the addition and deletion operations are stored separately. The edges of vertex A are in the order: A->B, A->C, A->D, A->E, A->F, A->G. The edges to be added are: A->H, A->I, A->J; the edges to be deleted are: A->B, A->D. In vertex A's delta storage area, the deletion queues are 1 and 3, and the addition queues are H, I, and J.

[0095] Update process: First, perform the deletion operation of vertex A. According to the deletion queue: 1, 3, we can know the position of the edge to be deleted. The first step is to delete the edge: A->B: replace the edge with index 1 with the last edge of vertex A. After that, the edges of vertex A are in the order of: A->G, A->C, A->D, A->E, A->F; the second step is to delete the edge: A->D: replace A->D with edge A->F, and then delete the last A->F. The edges of vertex A are in the order of: A->G, A->C, A->F, A->E; the third step is to add edges: A->H, A->I, A->J: the addition operation of vertex A is completed in one go, and the target edge is added to the end of the edges of vertex A in order. After that, the edges of vertex A are in the order of: A->G, A->C, A->F, A->E, A->H, A->I, A->J, and the update operation is completed.

[0096] Version switching includes version query and version update:

[0097] Version Query: When a version needs to be accessed, the system queries the index tree for each vertex. If the target version is found in the index tree, the corresponding edge is retrieved from the lower-level structure. If the target version is not found in the index tree, the edge of the previous version closest to the target version is retrieved from the lower-level index and the version update operation is performed on these edges.

[0098] Version update: Update all edges for which the target version is not found to generate edges for the target version. During the update, the delete operation in the delete queue is executed first, followed by the add operation in the add queue. First, when performing a delete operation, the position of the target edge in the edge vector is found based on the deletion index. The target edge is then replaced with the last edge in the edge vector, and finally, the last edge in the edge vector is deleted. Second, when performing an add operation, all edges to be added are appended to the end of the target edge, and finally, the corresponding edges for the target version are generated. The edges of all vertices are organized into a VoV representation to generate the target version graph.

[0099] Version switching such as Figure 3 As shown, perform the following process:

[0100] First, perform a version query. When accessing version j, find the replica version closest to version j for each vertex. Query the parent index tree of each vertex's replica storage area. If version j can be found in the index tree, then that version is the closest replica version. If version j is not found in the index tree, then version i is used as the replica version closest to version j, where i is less than j, and all versions between i and j are updated using delta. Then, based on the corresponding index information found in the index tree, find the corresponding replica edge from the lower-level edge queue. In the graph, the latest replica version of vertex 1 is j, the latest replica version of vertex 2 is i, and the latest replica version of vertex n is k.

[0101] Afterwards, perform version updates. Perform an update operation for the edge of the most recent copy of each vertex to generate an edge of version j. When the most recent copy version is j, it means that the edge of the current vertex is the target version and no further update operation is required; when the most recent copy version is i, perform update operations between version i and version j, that is, perform update operations from version i to version i+1, version i+1 to version i+2...version j-1 to version j in sequence to generate the edge corresponding to version j. Finally, combine the edges of all vertices into the target version graph. In the graph, vertex 1 does not need to continue to perform delta updates because its most recent copy version is j; vertex 2's most recent copy version is i, so a delta update from i to j is required; vertex n's most recent copy version is k, so a delta update from k to j is required.

[0102] See also Figure 1 ,The present invention also provides a multi-version graph system for fine-grained management of vertex updates, including a graph storage module 1, an update storage module 2, an update processing module 3, a switching mode selection module 4, a version switching module 5, and a graph processing module 6;

[0103] The graph storage module 1 is used to store the generated version graph data and the root version graph data in the delta mode, and the graph storage adopts the VoV representation mode;

[0104] The update storage module 2 is used to store update information;

[0105] The update storage module 2 includes a copy storage area 7 and a delta storage area 8:

[0106] The copy storage area 7 is used to store information about the copy mode, including the index tree and the edges of the vertices.

[0107] The delta storage area 8 is used to store delta mode information, including add and delete operation information.

[0108] The update processing module 3 includes an update collection submodule 9 and an update classification submodule 10:

[0109] The update collection submodule 9 is used to periodically collect update information on the graph;

[0110] The update classification submodule 10 is used to receive the update information from the update collection submodule 9 and sort and classify the update information according to vertices and update categories;

[0111] The switching mode selection module 4 is used to calculate and compare the cost of each vertex in the copy mode and delta mode of different versions, select the switching mode with the lowest cost for each vertex version, and store it in the corresponding copy storage area and delta storage area;

[0112] The version switching module 5 is used to perform version switching and generate a target version graph.

[0113] Furthermore, the version switching module 5 is divided into a version query submodule 11 and a version update submodule 12;

[0114] The version query submodule 11 is used to search the index tree in the copy storage area 7, query the edge of the vertex corresponding to the target version graph and the edge of the most recent previous version;

[0115] The version update submodule 12 is used to apply the updates in the delta storage area 8 to the corresponding edges to generate a target version.

[0116] The graph processing module 6 is used to execute corresponding graph algorithms, such as BFS / DFS, pagerank, CC, etc., on the target version graph.

[0117] In another embodiment, when accessing version 23, the index tree in the replica storage area of ​​vertex A is queried to determine that version 20 is the most recent replica of target version 23. The edge corresponding to version 20 is then found from the edges of vertex A's replica. An update operation is then performed on the edges of vertex A from version 20 to version 23. This operation is repeated for each vertex, ultimately obtaining the edges of all vertices in version 23 and assembling them into the target version graph. The corresponding graph algorithm is then executed on the target version graph.

[0118] Example 1

[0119] This embodiment introduces a fine-grained management method for vertex update switching.

[0120] S1: The collection phase. This phase aims to collect and organize update information from vertex updates. First, the system periodically retrieves updates from the graph and represents them as [operation, source, destination]. Then, an index is set for the delete operation. Finally, the updates are categorized by vertex and operation.

[0121] Specifically, for all delete operations, the corresponding vertex position is found as the index. Then, update operations are classified by vertex, and update operations for the same vertex are saved together. Finally, update operations are classified and updated and deleted operations are saved separately.

[0122] S2: Selection phase. The main goal of this phase is to select the appropriate update method for each updated vertex. First, the system obtains the updated information of each vertex through the saved update information. and value, and then calculate the vertex Compared with the selection threshold, if the vertex If the vertex is greater than the selection threshold, the copy switching mode is selected for the vertex. If it is less than the selection threshold, the vertex selects the delta switching mode.

[0123] Specifically, for the copy method, each vertex is stored in a two-level structure: an index tree at the top and a queue of edges arranged by version number at the bottom. For the delta method, each vertex uses an add queue and a delete queue to store update information. The add queue stores the sequence number of the destination vertex, and the delete queue stores the index of the destination vertex's position on the edge.

[0124] based on Figure 6 The following describes the process of fine-grained vertex switching management: The figure shows the update process of vertex A from version 0 to version 3. From version 0 to version 1, an edge A->F is added. From version 1 to version 2, three edges A->B, A->C, and A->D are deleted. From version 2 to version 3, an edge A->G is added. The selection threshold is fixed to: , when going from version 0 to version 1, vertex A: ,at this time , indicating that the cost of copy is greater than the cost of delta, and version 1 is stored in delta mode; when going from version 1 to version 2, vertex A: ,at this time , version 2 is stored in copy mode; when going from version 2 to version 3, vertex A: ,at this time , version 3 is stored in delta mode.

[0125] Example 2

[0126] This embodiment introduces version switching for fine-grained management of vertex updates.

[0127] S1: Query phase. The main purpose of this phase is to find the edge of the closest replica to the target version for each vertex. First, when version j needs to be accessed, the system searches for version j in the index tree of each vertex. If version j is found in the index tree, the edge corresponding to version j is the closest replica edge for that vertex. If version j is not found in the index tree, the edge corresponding to version i is selected as the closest replica edge, where i is less than j and i is the version closest to j. Then, based on the index of the edge of the version found in the index tree, the corresponding edge is found from the replica storage area of ​​each vertex.

[0128] S2: Update phase. The primary goal of this phase is to update the edge of the most recent copy of each vertex to generate the final target version graph. First, if the edge of the most recent copy corresponding to a vertex is version j, the vertex does not need to be updated. If the edge of the most recent copy corresponding to a vertex is version i, and i is less than j, the vertex is updated from version i to version j.

[0129] Specifically, the corresponding update information is retrieved from the vertex's delta storage, including updates from version i to i+1, i+1 to i+2, and so on. Updates between these versions are performed sequentially. First, the delete operation in the delete queue is executed, overwriting the end edge in the edge vector with the target edge based on the index. After the delete operation is completed, the edge in the add queue is added to the end of the edge vector.

[0130] based on Figure 7 The following describes the process of version switching. Figure 7 This section describes the switching process of vertex A when version 23 needs to be accessed, which mainly includes the query process and the update process.

[0131] Update process: First, the system queries the index tree of A from the copy storage of vertex A. Each node in the index tree can store at most Version numbers are arranged sequentially within nodes. By querying the index tree, we can determine that the replica of vertex A closest to version 23 is version 20. We then retrieve the edges corresponding to vertex A at version 20 from the lower structure based on the index node in the index tree. These edges are A->B, A->C, and A->D, in order. We then execute the update process from version 20 to version 23.

[0132] Update Process: The figure shows the process from version 20 to version 21. The delta storage area for vertex A from version 20 to version 21 has a delete queue of 0 and an add queue of E. First, a delete operation is performed on vertex A. Index 0 in the delete queue replaces the edge corresponding to index 0 with the last edge of vertex A. After the delete operation, the edges of vertex A are in the order A->D, A->C. Next, an add operation is performed on vertex A. The edge to be added to the add queue is A->E. After the add operation, the edges of vertex A are in the order A->D, A->C, A->E. Then, the update process from version 21 to version 22 and version 22 to version 23 is performed in sequence to obtain the edges of vertex A in version 23.

[0133] After executing the above process for all vertices, we can get the edges of each vertex in the target version, and combine them to generate the target version graph.

[0134] Corresponding to the embodiment of the above-mentioned method for fine-grained management of vertex updates, an embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for fine-grained management of vertex updates in the above-mentioned embodiment is implemented.

[0135] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0136] The above embodiments are intended only to illustrate the design concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. The scope of protection of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design concepts disclosed in the present invention are within the scope of protection of the present invention.

Claims

1. A method for fine-grained management of vertex updates, characterized in that: The following steps are involved: (1) Periodically collect vertex update information, record the operation type and the updated source and destination vertices; For the deletion operation, record the position of the destination vertex in the edge vector; The update information is stored by source vertex, and the operations on the same source vertex are stored separately by addition and deletion types; (2) Calculate and compare the update costs of the copy and delta methods for each vertex in different versions based on the collected update information, including memory overhead and version switching time overhead; select the appropriate update method for each vertex in different versions; perform version switching when accessing a specific version and update according to the selected method; The memory overhead of the copy method is the memory overhead of storing all edges in the vertex, and the time overhead is the time overhead of the system accessing the corresponding version; the memory overhead of the delta method is the memory overhead of storing the root version and version update information, and the time overhead is the time overhead of the system updating from the root version to the target version; Update storage includes copy storage and delta storage: Copy storage: The copy method saves all edges of a vertex, and each vertex is stored in a two-level structure. The upper level is an index tree structure. Each non-leaf node in the index tree contains several sequentially arranged version numbers, which are the version numbers of the vertex using the copy method. In addition to the version number, the leaf node of the index tree also saves the index of the corresponding edge vector in memory. Delta storage: The delta method saves vertex update information; each vertex has an add queue and a delete queue; the add queue saves the destination vertex to be added; The deletion queue stores the index of the destination vertex to be deleted.

2. The method for fine-grained management of vertex updates according to claim 1, characterized in that: Calculate the update cost of each vertex in different versions and compare them, including: Calculate the value of each updated vertex and The value of and The size of represents the number of vertex updates, represents the vertex degree, represents the time-memory trade-off weight, Indicates the remaining memory of the system; when ,express The value of is larger, that is, the cost of the copy method is greater than that of the delta method; when ,express The value of is larger, that is, the overhead of the delta method is greater than that of the copy method.

3. The method for fine-grained management of vertex updates according to claim 1, characterized in that: Select the appropriate update method for each vertex in different versions, including: For each version, compare the update costs of the copy method and the delta method under this version. When the copy cost is small, choose the copy method as the update method for this version; when the delta cost is small, choose the delta method as the update method for this version.

4. The method for fine-grained management of vertex updates according to claim 1, characterized in that: Version switching includes version query and version update: Version query: When a version needs to be accessed, the index tree of each vertex is queried; if the target version can be found in the index tree, the corresponding edge is obtained from the lower structure; if the target version cannot be found in the index tree, the edge of the previous version closest to the target version is obtained from the lower index; Version update: perform update operations on all edges where the target version is not found to generate the target version edges; when updating, first execute the delete operation in the delete queue, and then execute the add operation in the add queue; when executing the delete operation, find the position of the target edge in the edge vector according to the delete index, then replace the target edge with the last edge in the edge vector, and finally delete the last edge in the edge vector; when executing the add operation, add all edges that need to be added to the end of the target edge, and finally generate the corresponding edges of the target version; the edges of all vertices are organized into VoV representation to generate the target version graph.

5. A multi-version graph system for fine-grained management of vertex updates implementing the method of claim 1, characterized in that: It includes a graph storage module (1), an update storage module (2), an update processing module (3), a switching mode selection module (4), a version switching module (5), and a graph processing module (6); The graph storage module (1) is used to store the generated version graph data and the root version graph data in the delta mode, and the graph storage adopts the VoV representation mode; The update storage module (2) is used to store update information; The update processing module (3) includes an update collection submodule (9) and an update classification submodule (10); the update collection submodule (9) is used to periodically collect update information on the graph; the update classification submodule (10) is used to receive the update information from the update collection submodule (9) and organize and classify the update information according to vertices and update categories; The switching mode selection module (4) is used to calculate and compare the cost of each vertex in the copy mode and the delta mode of different versions, select the switching mode with the lowest cost for each vertex in different versions, and store it in the corresponding storage area; The version switching module (5) is used to perform version switching and generate a target version graph; The graph processing module (6) is used to execute the corresponding graph algorithm on the target version graph.

6. The multi-version graph system for fine-grained management of vertex updates according to claim 5, characterized in that: The update storage module (2) includes a copy storage area (7) and a delta storage area (8): The copy storage area (7) is used to store information about the copy mode, including the index tree and the edges of the vertices. The delta storage area (8) is used to store delta mode information, including add and delete operation information.

7. The multi-version graph system for fine-grained management of vertex updates according to claim 5, characterized in that: The version switching module (5) includes a version query submodule (11) and a version update submodule (12); The version query submodule (11) is used to search the index tree in the copy storage area (7) to query the edge of the vertex corresponding to the target version graph and the edge of the most recent previous version; The version update submodule (12) is used to apply the updates in the delta storage area (8) to the corresponding edges to generate a target version.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for fine-grained management of vertex updates according to any one of claims 1 to 4 is implemented.

Images