Graph calculation method and device

Abstract

The graph calculation method is applied to a calculation system for executing graph calculation, and comprises the following steps: respectively carrying out offline graph calculation on each graph data fragment stored in a database based on a preset graph algorithm, and storing a graph calculation result obtained by carrying out graph calculation on the graph data fragments; wherein the graph algorithm is a graph algorithm in which an output graph calculation result has monotonicity; in response to a calculation request for performing global graph calculation on a plurality of graph data fragments generated in a preset time window and stored in a database, obtaining a plurality of graph calculation results obtained by performing offline graph calculation on the plurality of graph data fragments; calculating an intermediate calculation result of global graph calculation for the plurality of graph data fragments based on the plurality of graph calculation results, and continuing to perform online graph calculation for the plurality of graph data fragments based on a preset graph algorithm on the basis of the intermediate calculation result, and obtaining a final calculation result of global graph calculation for the plurality of graph data fragments.

Description

Technical Field

[0001] The embodiments of this specification belong to the field of graph computing technology, and more particularly, to a graph computing method and device. Background Art

[0002] In some graph systems, time periods are usually used as the basis for division, and graph data generated within a period of time is saved as a subgraph to reduce system memory consumption and data redundancy. Each subgraph is usually called a graph data shard.

[0003] In practical applications, graph data composed of several graph data shards is usually very large and the data content is constantly changing; therefore, when graph analysis and calculation of graph data involves multiple graph data shards, it is usually necessary to merge the graph structures of the multiple graph data shards again, and then continue to complete subsequent graph calculations based on the merged graph data.

[0004] However, the operation of merging multiple graph data shards usually brings very large additional overhead to the system. Summary of the invention

[0005] This specification proposes a graph computing method, which is applied to a computing system that performs graph computing. The method includes:

[0006] Based on a preset graph algorithm, offline graph calculations are performed on each graph data shard stored in the database, and graph calculation results obtained by performing graph calculations on the graph data shard are saved; wherein the graph algorithm is a graph algorithm whose output graph calculation results are monotonic;

[0007] In response to a calculation request for performing global graph calculation on a plurality of graph data shards generated within a preset time window and stored in a database, obtaining a plurality of graph calculation results obtained by performing offline graph calculations on the plurality of graph data shards respectively;

[0008] Based on the multiple graph calculation results, intermediate calculation results of global graph calculation for the multiple graph data shards are calculated, and based on the intermediate calculation results, online graph calculations are continued to be performed for the multiple graph data shards based on the preset graph algorithm to obtain final calculation results of global graph calculation for the multiple graph data shards.

[0009] Optionally, the graph data shards are graph data generated within a preset unit time period.

[0010] Optionally, the graph data slices stored in the database are periodically imported into the database for storage according to the unit time period.

[0011] Optionally, offline graph computing is performed on the graph data shards stored in the database based on a preset graph algorithm, including:

[0012] In response to graph data shards being imported into the database, offline graph calculations are performed on the imported graph data shards based on a preset graph algorithm.

[0013] Optionally, the intermediate calculation result is a graph calculation result among the multiple graph calculation results that has the smallest error with a final calculation result of a global graph calculation performed on the multiple graph data shards.

[0014] Optionally, at least some of the vertices contained in each graph data slice stored in the database are the same vertices; the graph algorithm includes an iterative graph algorithm; wherein the iterative graph algorithm includes a graph algorithm that performs iterative graph calculations by traversing vertices with neighbor relationships contained in the graph data.

[0015] Optionally, the multiple graph data shards are discretely stored in the database; the computing system maintains a global graph structure corresponding to the multiple graph data shards discretely stored in the database; wherein the global graph structure is used to describe the global neighbor relationship of vertices with neighbor relationships contained in the multiple graph data shards in the multiple graph data shards; the global neighbor relationship includes the neighbor relationship between the vertex and all neighbor vertices of the vertex in each of the multiple graph data shards;

[0016] Based on the intermediate calculation results, continuing to perform online graph calculations on the multiple graph data shards based on the preset graph algorithm includes:

[0017] On the basis of the intermediate calculation results, based on the preset graph algorithm, the vertices with neighbor relationships contained in the multiple data graph slices are traversed in the global graph structure, and all the neighbor vertices of the neighbor vertices in each graph data slice are traversed to continue to perform online graph calculations on the vertices contained in the multiple graph data slices.

[0018] Optionally, the global graph structure is a global graph structure constructed based on the neighbor relationships between the vertices contained in the respective graph data shards and all neighbor vertices of the vertex in the respective graph data shards;

[0019] Before continuing to perform online graph calculation on the plurality of graph data shards based on the preset graph algorithm and on the basis of the intermediate calculation result, the method further includes:

[0020] Obtain all neighbor vertices of the vertices contained in each of the graph data slices;

[0021] The global graph structure is constructed based on the neighbor relationships between the vertices contained in the obtained graph data slices and all the neighbor vertices.

[0022] Optionally, the global graph structure adopts an adjacency list structure; wherein the adjacency list maintains vertex identifiers of vertices with neighbor relationships contained in the multiple graph data shards, and a linked list corresponding to the vertex identifiers; wherein the linked list contains pointers to all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data shards;

[0023] In the global graph structure, vertices with neighbor relationships contained in the plurality of data graph slices are traversed, and all neighbor vertices in each of the graph data slices include:

[0024] The vertex identifiers maintained in the adjacency list are traversed, and based on the pointers contained in the linked list corresponding to the vertex identifiers, all neighbor vertices of the vertex corresponding to the vertex identifier in the respective graph data fragments are further accessed.

[0025] Optionally, the linked list includes a pointer to a first neighbor vertex of the vertex corresponding to the vertex identifier in each of the graph data slices, and a pointer offset of other neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices relative to the first neighbor vertex;

[0026] Further accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices based on the pointer contained in the linked list corresponding to the vertex identifier includes:

[0027] Based on the pointer contained in the linked list corresponding to the vertex identifier, the first neighbor vertex of the vertex corresponding to the vertex identifier in each of the graph data slices is accessed, and based on the pointer offset contained in the linked list, other neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices are accessed.

[0028] Optionally, a graph calculation is performed on any target graph data shard stored in the database based on a preset graph algorithm, including:

[0029] Based on the initial set of activated vertices that need to participate in the iteration specified from the vertices contained in the target graph data slice, iteratively execute the preset calculation process until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation meets the preset algorithm convergence condition;

[0030] The preset calculation process includes:

[0031] Performing graph calculation on the activated vertices in the initial set of activated vertices based on the preset graph algorithm to obtain graph calculation results of the activated vertices in the set of activated vertices;

[0032] Traversing neighbor vertices of the activated vertices in the activated vertex set in the target graph data shard, and determining whether the traversed neighbor vertices of the activated vertex meet activation conditions corresponding to the graph algorithm based on graph calculation results of the activated vertices in the activated vertex set;

[0033] The neighboring vertices satisfying the activation condition are determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iteration is further created.

[0034] Optionally, the multiple graph calculation results include those obtained by performing offline graph calculation on vertices with neighbor relationships contained in each of the multiple graph data shards, and graph calculation results of the vertices in each of the graph data shards;

[0035] Calculating intermediate calculation results of global graph calculations for the multiple graph data shards based on the multiple graph calculation results includes:

[0036] Based on the vertices with neighbor relationships contained in each of the multiple graph data slices and the graph calculation results in each of the graph data slices, an intermediate calculation result of the global graph calculation for the vertices is calculated.

[0037] Optionally, based on the intermediate calculation results, traversing the vertices with neighbor relationships contained in the multiple data graph shards in the global graph structure based on the preset graph algorithm, all neighbor vertices of the neighbor vertices in each graph data shard, to continue to perform online graph calculations on the vertices contained in the multiple graph data shards, including:

[0038] Based on the intermediate calculation result, from vertices with neighbor relationships contained in each of the multiple graph data slices, determine an initial set of activated vertices for each of the graph data slices;

[0039] Based on the initial set of activated vertices, for each of the graph data slices, the preset calculation process is iteratively executed respectively until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation meets the preset algorithm convergence condition;

[0040] The preset calculation process includes:

[0041] Performing graph calculation on the activated vertices in the initial activated vertex set based on the preset graph algorithm to obtain graph calculation results of the activated vertices in the activated vertex set;

[0042] Traversing neighbor vertices of an activated vertex in the activated vertex set in the global graph structure, and determining whether neighbor vertices traversed to the activated vertex meet an activation condition corresponding to the graph algorithm based on graph calculation results of the activated vertices in the activated vertex set;

[0043] The neighboring vertices satisfying the activation condition are determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iteration is further created.

[0044] Optionally, based on the initial set of activated vertices, for each of the graph data slices, a preset calculation process is iteratively executed respectively until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation satisfies a preset algorithm convergence condition, including:

[0045] On the basis of the initial set of activated vertices, by traversing the neighbor vertices of the activated vertices in the set of activated vertices in each of the graph data slices, a round of the preset calculation process is performed respectively to obtain the set of activated vertices corresponding to each of the graph data slices that need to continue to participate in the next round of iteration;

[0046] The activated vertex sets corresponding to the respective graph data slices are merged to obtain a target activated vertex set corresponding to the multiple graph data slices, and based on the target activated vertex set, the preset calculation process is continued to be iteratively executed by traversing the neighbor vertices of the activated vertices in the activated vertex set in the global graph structure until it is impossible to determine the activated vertices that need to continue to participate in the next round of iteration or the iterative calculation satisfies the preset algorithm convergence condition.

[0047] Optionally, based on the vertices having neighbor relationships contained in each of the multiple graph data shards, and the graph calculation results in each of the graph data shards, calculating the intermediate calculation results of the global graph calculation for the vertices includes:

[0048] For any vertex with a neighbor relationship contained in each of the multiple graph data slices, determine, from the graph calculation results of the vertex in each of the graph data slices, a target graph calculation result having the smallest error with a final calculation result of a global graph calculation performed on the vertex;

[0049] The target graph calculation result is determined as an intermediate calculation result of a global graph calculation performed on the vertex.

[0050] Optionally, based on the intermediate calculation result, determining, from vertices having neighbor relationships contained in each of the multiple graph data slices, for each of the graph data slices, an initial set of activated vertices that need to participate in the iteration of the global graph calculation, includes:

[0051] For any graph data slice among the multiple graph data slices, determine whether a graph calculation result of a vertex with a neighbor relationship contained in the graph data slice in the graph data slice is the same as the target graph calculation result determined for the vertex;

[0052] If they are not the same, the vertex is taken as an activated vertex that needs to participate in the iteration of the global graph calculation and is added to the initial activated vertex set created for the graph data slice.

[0053] Optionally, the computing system further maintains the total number of neighbor vertices of the vertices having neighbor relationships contained in the multiple graph data shards in each of the graph data shards;

[0054] Traversing the vertex identifiers maintained in the adjacency list, and further accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices based on the pointers contained in the linked list corresponding to the vertex identifier, including:

[0055] For any vertex identifier traversed in the adjacency list, obtain the total number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data slice, and determine whether the total number obtained is greater than a threshold;

[0056] If the total number obtained is greater than the threshold, an independent thread is allocated to the vertex corresponding to the vertex identifier, and the thread is run to visit all the neighbor vertices of the vertex in the various graph data slices in sequence according to the order of the pointers contained in the linked list maintained in the adjacency list corresponding to the vertex identifier; if the total number obtained is greater than the threshold, multiple independent threads are allocated to the vertex, and the multiple independent threads are run to visit all the neighbor vertices of the vertex corresponding to the vertex identifier in the various graph data slices in parallel based on the pointers contained in the linked list corresponding to the vertex identifier.

[0057] Optionally, if the total number obtained is greater than the threshold, multiple independent threads are allocated to the vertex, including:

[0058] If the total number obtained is greater than the threshold, further calculating the difference in the number of neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices, and determining whether the difference is lower than the threshold;

[0059] If yes, a plurality of independent threads corresponding to the plurality of graph data slices are allocated to the vertex corresponding to the vertex identifier; wherein each thread in the plurality of independent threads is used to access all neighbor vertices of the vertex in the graph data slice corresponding to the thread;

[0060] If not, assign multiple independent threads with the same number of neighbor vertices that can be processed to the vertex corresponding to the vertex identifier; wherein each thread in the multiple independent threads is used to access the neighbor vertices assigned to the thread from all the neighbor vertices of the vertex in the respective graph data shards; the number of neighbor vertices assigned to each thread is the same. Optionally, before traversing the vertex identifier maintained in the adjacency list and further accessing all the neighbor vertices of the vertex corresponding to the vertex identifier in the respective graph data shards based on the pointer contained in the linked list corresponding to the vertex identifier, it also includes:

[0061] Determine a target vertex identifier from each vertex identifier maintained in the adjacency list; wherein the target vertex identifier is a vertex identifier corresponding to a target vertex whose number of neighbor vertices in each graph data slice is not greater than a threshold;

[0062] All neighbor vertices of the target vertex corresponding to the target vertex identifier in the respective graph data shards are read from the database, and the read neighbor vertices are further stored in a continuous memory space pre-allocated in the memory, and the pointers contained in the linked list maintained in the adjacency list corresponding to the target vertex identifier are further replaced with pointers corresponding to the continuous memory space, so that when traversing to the target vertex, the thread assigned to the target vertex can access all neighbor vertices of the target vertex in the respective graph data shards from the continuous memory space based on the pointers contained in the linked list.

[0063] Optionally, the computing request is a computing request triggered by a graph query operation for a plurality of graph data shards generated within a preset time window; the method further includes:

[0064] In response to a final calculation result obtained by performing global graph calculation on the multiple graph data shards, generating a query result that satisfies a query condition related to the graph query operation based on the final calculation result;

[0065] The query result is output to the initiator of the graph query operation.

[0066] Optionally, the multiple graph data shards are discretely stored in the database in CSR format.

[0067] Optionally, the vertices contained in the graph data shard represent users; and the edges connecting the vertices contained in the graph data shard represent transactions generated between users.

[0068] Optionally, the database includes a data warehouse.

[0069] This specification also proposes a graph computing device, which is applied to a computing system that performs graph computing, and the device includes:

[0070] A first computing module performs offline graph computing on each graph data shard stored in the database based on a preset graph algorithm, and saves the graph computing results obtained by performing the graph computing on the graph data shard; wherein the graph algorithm is a graph algorithm whose output graph computing results are monotonic;

[0071] An acquisition module, in response to a computation request for performing global graph computation on a plurality of graph data shards generated within a preset time window and stored in a database, acquires a plurality of graph computation results obtained by performing offline graph computations on the plurality of graph data shards respectively;

[0072] The second computing module calculates intermediate computing results of global graph computing for the multiple graph data shards based on the multiple graph computing results, and based on the intermediate computing results, continues to perform online graph computing for the multiple graph data shards based on the preset graph algorithm to obtain final computing results of global graph computing for the multiple graph data shards.

[0073] In the above embodiment, on the one hand, since the graph calculation results of the offline calculation for the multiple graph data shards can be reused during the online global graph calculation for the multiple graph data shards, it is possible to avoid repeated graph calculation for the multiple graph data shards during the online calculation stage, thereby reducing the occupation of online computing resources in the system, reducing the computing overhead, and improving the computing performance of the system for the online global graph calculation for the multiple graph data shards;

[0074] On the other hand, during the process of performing global graph calculations online for the multiple graph data shards, based on the graph calculation results of offline calculations for the multiple graph data shards, the online graph calculations for the multiple graph data shards can be continued to be completed based on the intermediate calculation results of the global graph calculations for the multiple graph data shards. When the system performs global graph calculations online for the multiple graph data shards, the system can avoid merging the graph structures of the multiple graph data shards, thereby significantly reducing the system overhead and further improving the computing performance of the global graph calculations for the multiple graph data shards online. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0076] Figure 1 is a flow chart of a graph calculation method shown in an embodiment of this specification;

[0077] Figure 2 It is a schematic diagram of a hybrid computing framework using offline computing and online computing shown in an embodiment of this specification;

[0078] Figure 3 It is a flowchart of performing online graph calculation on multiple graph data slices generated within a preset time window in one embodiment of this specification;

[0079] Figure 4 It is a schematic diagram of centrally storing neighbor nodes of a vertex in each graph data slice in a continuous memory space as shown in an embodiment of this specification;

[0080] Figure 5 is a schematic structural diagram of an electronic device shown in an embodiment of this specification;

[0081] Figure 6 It is a block diagram of a graph computing device shown in one embodiment of this specification. DETAILED DESCRIPTION

[0082] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0083] In practical applications, the size of graph data stored in the database usually continues to grow, and storing a huge graph data in the database is obviously very inefficient and inconvenient. Therefore, for a complete graph data stored in the database, it is usually divided into time periods, and the graph data generated within a period of time is saved as a subgraph. Each subgraph is usually called a graph data shard.

[0084] For example, taking the graph data stored in the data warehouse of an online payment platform as an example, the vertices contained in these graph data usually represent users, and the edges between the connecting vertices contained in the graph data usually represent payment transactions generated between users. In order to reduce the memory consumption and data redundancy of the data warehouse, the payment transactions generated between users within a certain period of time (such as one day) can usually be periodically constructed into a subgraph and imported into the data warehouse for storage. Each subgraph contains all payment transactions generated between users within a certain period of time.

[0085] In practical applications, graph computing on graph data generated within a time window is usually one of the basic applications supported by data warehouses that store graph data. Graph computing on graph data generated within a time window is not only of great significance in graph theory, but also the basis of many online graph analysis operations.

[0086] For example, when you need to query graph data generated within a time window, you usually need to first perform graph calculations on the graph data generated within the time window, and then generate query results related to this data query based on the graph calculation results.

[0087] When performing graph calculations on graph data generated within a time window, multiple graph data shards are usually involved, and the graph calculations performed on multiple graph data shards are essentially global graph calculations performed on the multiple graph data shards; therefore, it is usually necessary to merge the graph structures of the multiple graph data shards again, and then continue to complete subsequent graph calculations based on the merged graph data.

[0088] For example, still taking the scenario of performing data query on graph data generated within a time window as an example, in a graph database composed of multiple graph data shards, the data query initiated on the graph database usually needs to specify the query range by declaring a temporary query window. This time window can be used to determine which specific graph data shards will be involved in this query. Each graph data shard represents a part of the graph data in the graph database. When performing data query on graph data generated within a time window, it is necessary to merge the graph structures of each graph data shard within the scope of the time window to obtain a complete graph structure, and then perform graph calculations on the basis of the completed graph structure, and then generate query results related to this data query based on the graph calculation results.

[0089] However, in practical applications, it may be necessary to frequently perform graph calculations on graph data generated in different time windows. This frequent graph calculation requirement will inevitably generate a large number of additional operations to merge the graph structures of graph data shards in different time windows. Since the merge operation needs to process a large amount of data redundancy and structural integration, frequent merge operations will obviously bring a very large additional overhead to the system, thereby affecting the system's targeted computing performance.

[0090] Based on this, this specification proposes a hybrid computing framework that supports both offline graph computing and online graph computing.

[0091] Under this computing framework, a graph algorithm whose output graph computing results are monotonic can be used to perform offline graph computing on each graph data shard stored in the database, and save the graph computing results obtained by performing graph computing on the graph data shard.

[0092] When a global graph calculation request is required for multiple graph data slices generated within a preset time window, multiple graph calculation results obtained by performing offline graph calculations on the multiple graph data slices can be obtained, and based on the multiple graph calculation results, intermediate calculation results of global graph calculations on the multiple graph data slices can be further calculated;

[0093] Then, based on the intermediate calculation result, online graph calculation can be continued for the multiple graph data shards based on the graph algorithm to obtain the final calculation result of the global graph calculation for the multiple graph data shards.

[0094] In the above technical solution, on the one hand, since the graph calculation results of the offline calculation for the multiple graph data shards can be reused during the online global graph calculation for the multiple graph data shards, it is possible to avoid repeated graph calculation for the multiple graph data shards during the online calculation stage, thereby reducing the occupation of online computing resources in the system, reducing the computing overhead, and improving the computing performance of the system for the online global graph calculation for the multiple graph data shards;

[0095] On the other hand, during the process of performing global graph calculations online for the multiple graph data shards, based on the graph calculation results of offline calculations for the multiple graph data shards, the online graph calculations for the multiple graph data shards can be continued to be completed based on the intermediate calculation results of the global graph calculations for the multiple graph data shards. When the system performs global graph calculations online for the multiple graph data shards, the system can avoid merging the graph structures of the multiple graph data shards, thereby significantly reducing the system overhead and further improving the computing performance of the global graph calculations for the multiple graph data shards online.

[0096] See also Figure 1 , Figure 1 This is a flowchart of a graph computing method shown in this specification, wherein the solution can be applied to a computing system that performs graph computing; the method includes the following execution process:

[0097] Step 102: performing offline graph calculations on each graph data slice stored in the database based on a preset graph algorithm, and saving graph calculation results obtained by performing graph calculations on the graph data slice; wherein the graph algorithm is a graph algorithm whose output graph calculation results are monotonic;

[0098] The above-mentioned graph data shards may specifically be graph data composed of data generated within a period of time. All graph data shards stored in the database may together constitute a complete graph structure, and any graph data shard stored in the database may serve as a subgraph in the complete graph structure stored in the database.

[0099] For example, in one example, taking the above-mentioned database as a data warehouse and the above-mentioned graph data as graph data stored in the data warehouse of an online payment platform, the graph data may generally be graph data based on transaction data generated between users, and the vertices contained in the graph data generally represent users, and the edges connecting the vertices contained in the graph data generally represent payment transactions generated between users.

[0100] In order to reduce the memory consumption and data redundancy of the data warehouse, the payment transactions generated between users within a certain time period (such as one day) can usually be periodically constructed into a subgraph and imported into the data warehouse for storage. The graph data generated within a certain time period is usually called a graph data slice, and each graph data slice contains all the payment transactions generated between users within a certain time period. For example, assuming that one day is used as the unit time period, the graph data slice of the i-th day can be expressed as Gi = (V, Ei), where V represents the vertex set and Ei represents the edge set. It should be emphasized that the above-mentioned graph data can be a directed graph or an undirected graph, which is not specifically limited in this specification.

[0101] It should be noted that, in actual applications, the time periods corresponding to the various graph data slices stored in the database may be the same or different.

[0102] In one embodiment shown, in order to facilitate the unified management of graph data shards in the database, the graph data shards stored in the database can specifically be graph data constructed based on data generated within a preset unit time period. That is, the graph data shards stored in the database can all be graph data generated within a fixed time range. Among them, the specific value of the unit time period can be flexibly set based on demand and is not limited in this specification. For example, one day can be used as the unit time period, so the graph data shards can be constructed based on the data generated every day.

[0103] In practical applications, data producers can also periodically import the generated graph data shards into the database for storage according to a certain period.

[0104] For example, in one embodiment shown, the data producer may also periodically import the generated graph data shards into the database for storage according to the above-mentioned unit time period. For example, assuming that one day is used as the unit time period, in addition to constructing new graph data shards based on the data generated every day, the data producer may also periodically import the graph data shards generated every day into the database for storage according to one day as a cycle. Of course, it should be emphasized that in actual applications, the cycle used to import the generated graph data shards into the database may be different from the unit time period used to generate the graph data shards.

[0105] The above-mentioned graph algorithm may specifically be a monotonic algorithm. Monotonic algorithm is an important category of graph algorithms, and generally refers to a graph algorithm whose output graph calculation results are monotonic. Among them, the output graph calculation results are monotonic, which generally refers to the graph calculation results output by the graph algorithm for a graph data, which will undergo a single-phase change as the graph structure corresponding to the graph data changes. The so-called single-phase change generally refers to a monotonic increase or decrease in one direction as the graph structure of the graph data changes.

[0106] For example, if the graph structure corresponding to the graph data has changed (perhaps edges or nodes have been added, or edges or nodes have been reduced), and the calculation results output by the graph algorithm have monotonically increased or monotonically decreased, then the graph algorithm is a monotonic algorithm. On the contrary, if the graph structure corresponding to the graph data has changed, and the calculation results output by the graph algorithm have monotonically increased for a period of time, and then monotonically decreased after a period of time, and the calculation results output by the algorithm have not changed monotonically in one direction, then the graph algorithm cannot be called a monotonic algorithm.

[0107] In a typical monotonic algorithm, each vertex contained in the graph data can maintain a graph calculation result obtained by performing graph calculation on the vertex. During the calculation process, this calculation result usually changes monotonically in only one direction. For example, taking the shortest path algorithm in the graph algorithm as an example, in this type of algorithm, the graph calculation result for each vertex will monotonically decrease from infinity to a smaller value.

[0108] It should be noted that the specific type of the above-mentioned graph algorithm is no longer specifically limited in this specification. In practical applications, the graph algorithm may specifically include any type of monotonic algorithm.

[0109] For example, in one example, the above monotone algorithm may specifically include but is not limited to the following graph algorithms: reachability algorithm, breadth-first search algorithm, shortest path algorithm, connected component algorithm, minimum / maximum label propagation algorithm, and the like.

[0110] In practical applications, graph computing on graph data generated within a time window is usually one of the basic applications supported by databases that store graph data (such as data warehouses). Graph computing on graph data generated within a time window is not only of great significance in graph theory, but also the basis of many online graph analysis operations.

[0111] For example, when you need to query graph data generated within a time window, you usually need to first perform graph calculations on the graph data generated within the time window, and then generate query results related to this data query based on the graph calculation results.

[0112] In this specification, when it is necessary to perform global graph calculations on multiple graph data slices generated within a preset time window, in order to avoid merging the graph structures of the multiple graph data slices, a hybrid computing framework of offline computing and online computing can be used.

[0113] Based on the hybrid computing framework, before initiating global graph calculation for multiple graph data shards generated within the preset time window, the above-mentioned computing system can pre-perform offline graph calculation on each graph data shard stored in the database based on a preset graph algorithm, and save the graph calculation results for each graph data shard.

[0114] In one embodiment shown, see Figure 2 , Figure 2 A schematic diagram of a hybrid computing framework using offline computing and online computing shown in this specification, such as Figure 2As shown, assuming that the data producer can periodically import the generated graph data shards into the database for storage, then the offline calculation of the graph data shards stored in the database can be performed in advance when the graph data shards are imported into the database.

[0115] In this case, whenever a new graph data shard is imported into the database, the above-mentioned computing system can respond to the graph data shard imported into the database, perform offline graph calculation on the imported graph data shard based on a preset graph algorithm, and then save the graph calculation result of the offline graph calculation as a transition result; for example, the graph calculation result can still be saved in the database, or it can also be saved in a storage space outside the database for subsequent online computing.

[0116] Since in the database, data import and graph calculation for the graph data shards stored in the database usually do not occur at the same time, offline graph calculation is performed on the imported graph data shards in advance during the data import phase. This can not only make full use of the idle computing resources of the computing system, but also will not affect the subsequent online calculations.

[0117] It should be noted that the specific algorithm flow of offline graph computing for graph data shards stored in the database usually depends on the specific type of graph algorithm used and the programming model used when developing the graph algorithm. For example, taking graph algorithms as an example, common programming models usually include vertex-centric programming models and edge-centric programming models.

[0118] In one embodiment shown, it is assumed that the graph data slices stored in the database share vertices, that is, all vertices or at least some of the vertices contained in the graph data slices stored in the database are the same vertices. The above-mentioned graph algorithm may specifically be an iterative graph algorithm that adopts a vertex-centric programming model. The algorithm flow of the iterative graph algorithm may specifically be to traverse the vertices with neighbor relationships contained in the graph data and perform iterative graph calculations on the graph data.

[0119] In this scenario, an iterative graph algorithm based on a vertex-centric programming model is used to perform offline graph computing on any target graph data shard stored in the database. The algorithm flow can be abstracted into the following algorithm logic:

[0120] First, an initial set of activated vertices can be specified from the vertices contained in the target graph data slice; the so-called activated vertices usually refer to the vertices that need to participate in the iteration. In practical applications, the vertices contained in the graph data that may affect the final graph calculation results can be determined as activated vertices. By determining the vertices that may affect the final graph calculation results as activated vertices, the intermediate calculation results of the vertices can continue to propagate to other vertices in the algorithm, so that the final calculation results can be continuously optimized in the iteration of the algorithm.

[0121] After specifying an initial set of activated vertices from the vertices included in the target graph data slice, graph calculation can be performed on the initial set of activated vertices based on the preset graph algorithm to obtain graph calculation results of the activated vertices in the initial set of activated vertices;

[0122] Among them, for different iterative graph algorithms, the specific process of performing graph calculation on the activated vertices and the obtained graph calculation results are usually different; for example, taking the shortest path algorithm in the graph algorithm as an example, in this type of algorithm, it is usually used to calculate the distance between each activated vertex and the starting vertex, and the corresponding graph calculation result for each activated vertex is usually a distance value representing the distance between each activated vertex and the starting vertex. For another example, taking the connected component algorithm in the graph algorithm as an example, in this type of algorithm, it is usually used to calculate the connected component to which the activated vertex belongs, and the corresponding graph calculation result for each activated vertex is usually a label used to indicate the connected component to which the vertex belongs.

[0123] Furthermore, after obtaining the graph calculation results of the activated vertices in the activated vertex set, the neighbor vertices of the activated vertices in the activated vertex set can be further traversed in the target graph data shard, and based on the graph calculation results of the activated vertices in the activated vertex set, it is determined whether the neighbor vertices of the traversed activated vertices meet the activation conditions corresponding to the above-mentioned graph algorithm;

[0124] For example, in actual applications, after traversing to the neighbor vertices of the activated vertex in the activated vertex set, the update function defined by the user can be executed to update the state of the neighbor vertex based on the state of the activated vertex, and then continue to run the conditional function defined by the user to determine whether the neighbor vertex meets the activation condition executed by the user based on the latest state of the neighbor vertex. It should be noted that the activation condition is usually a condition customized by the user based on the specific needs of graph computing; for example, taking the application scenario of needing to query the graph data generated within a time window as an example, in this case, the user can define the activation condition based on the specific query requirements.

[0125] Furthermore, neighbor vertices that meet the activation condition may be determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iterations may be further created.

[0126] In the next round of iteration, the above calculation process can be repeated. Based on the determined new set of activated vertices, graph calculations can be continued for the activated vertices in the new set of activated vertices based on the above preset graph algorithm. And so on, the above calculation process is continuously iterated until it is impossible to determine the activated vertices that need to continue to participate in the next round of iteration or the calculation of the iteration meets the preset algorithm convergence conditions.

[0127] Among them, it is impossible to determine the activated vertices that need to continue to participate in the next round of iterations, which usually means that all vertices contained in the graph data that may affect the final graph calculation results have been traversed, and there are no vertices in the graph data that may affect the final graph calculation results. The above algorithm convergence conditions can be flexibly set based on specific needs; for example, in actual applications, the above algorithm convergence conditions can be specifically that the number of iterations reaches a threshold, or the graph calculation results meet the user-specified requirements.

[0128] It should be emphasized that the above-mentioned graph algorithm is taken as an example of an iterative graph algorithm using a vertex-centric programming model. In actual applications, the various graph data shards stored in the above-mentioned database may also share edges, that is, all edges or at least some of the edges contained in the various graph data shards stored in the database are the same edges. In this scenario, the above-mentioned graph algorithm may specifically be an iterative graph algorithm using an edge-centric programming model. The algorithm flow of the iterative graph algorithm using an edge-centric programming model is similar to the algorithm flow of the iterative graph algorithm using a vertex-centric programming model, and will not be described in detail in this specification.

[0129] Step 104: in response to a calculation request for performing global graph calculation on a plurality of graph data slices generated within a preset time window and stored in a database, obtaining a plurality of graph calculation results obtained by performing offline graph calculations on the plurality of graph data slices respectively;

[0130] When it is necessary to perform global graph calculation on multiple graph data shards generated within a preset time window and stored in the database, the above-mentioned computing system can respond to the computing request for performing global graph calculation on the multiple graph data shards, and obtain multiple graph calculation results calculated in the offline computing stage, which are obtained by performing offline graph calculations on the multiple graph data shards respectively.

[0131] For example, see Figure 2Assuming that the above preset time window is T2-T6, when the above computing system receives the graph data slice G within the time window t4 ~G t6 After making a global graph computing request, the graph data slice G can be obtained from the transition result saved in the offline computing phase in response to the computing request. t4 ~G t6 The transition result can be obtained by slicing the graph data G t4 ~G t6 Based on the transition results, continue to slice the graph data G t4 ~G t6 Perform online graph computation.

[0132] Step 106: Calculate intermediate calculation results of global graph calculation for the multiple graph data shards based on the multiple graph calculation results, and based on the intermediate calculation results, continue to perform online graph calculation for the multiple graph data shards based on the preset graph algorithm to obtain final calculation results of global graph calculation for the multiple graph data shards.

[0133] After obtaining multiple graph calculation results obtained by performing offline graph calculations on the above-mentioned multiple graph data shards, online graph calculations can be continued for the above-mentioned multiple graph data shards based on the multiple graph calculation results.

[0134] See also Figure 3 , Figure 3 This specification shows a flowchart of performing online graph computation on multiple graph data slices generated within a preset time window.

[0135] like Figure 3 As shown, the process of performing online graph calculation for the above-mentioned multiple graph data shards may specifically include an initialization calculation phase and an online graph calculation phase.

[0136] Among them, in the initialization stage, the intermediate calculation results of the global graph calculation for the multiple graph data slices can be calculated based on the transition results of the above multiple graph data slices, that is, the graph calculation results obtained by performing offline graph calculations on the above multiple graph data slices. In the online graph calculation stage, based on the above intermediate calculation results, the online graph calculation can be continued for the above multiple graph data slices based on the above preset graph algorithm.

[0137] In one embodiment shown, taking the vertices shared by various graph data shards stored in a database, the above-mentioned graph algorithm is an iterative graph algorithm using a vertex-centric programming model as an example. In this scenario, in the offline calculation stage, multiple graph calculation results are obtained by performing offline graph calculations on the multiple graph data shards respectively. Generally, the graph calculation results of the vertices with neighbor relationships contained in each of the multiple graph data shards are obtained by performing offline graph calculations on the vertices, which are contained in each of the multiple graph data shards, and the graph calculation results of the vertices in the above-mentioned graph data shards respectively.

[0138] In this case, when calculating the intermediate calculation results of the global graph calculation for the multiple graph data slices based on the above-mentioned multiple graph calculation results, the intermediate calculation results of the global graph calculation for the vertex can be calculated based on the graph calculation results in the above-mentioned each graph data slice respectively of the vertices with neighbor relationships contained in the multiple graph data slices.

[0139] In one embodiment shown, the intermediate calculation result may be the graph calculation result with the smallest error from the final calculation result of the global graph calculation for the multiple graph data shards among the multiple graph calculation results. It should be noted that the graph calculation result with the smallest error from the final calculation result is the graph calculation result with the closest value to the final calculation result.

[0140] In this case, for any vertex with a neighbor relationship contained in each of the multiple graph data slices, when calculating the intermediate calculation result of the global graph calculation for the vertex based on the graph calculation results of the vertex in the above-mentioned graph data slices, it is specifically possible to determine from the graph calculation results of the vertex in each of the graph data slices, the target graph calculation result with the smallest error from the final calculation result of the global graph calculation for the vertex; then, the target graph calculation result with the smallest error from the above-mentioned final calculation result can be determined as the intermediate calculation result of the global graph calculation for the vertex.

[0141] When evaluating the error between the graph calculation results of the vertex in each graph data shard and the final calculation result of the global graph calculation for the vertex, different graph algorithms usually have different error measurement methods. For example, taking the shortest path algorithm as an example, the minimum value of the shortest path of the vertex in each graph data shard can be solved as the intermediate calculation result of the global graph calculation for the vertex.

[0142] Of course, in practical applications, based on specific needs, other calculation results among the above-mentioned multiple graph calculation results, except for the graph calculation result with the smallest error with the final calculation result of the global graph calculation for the multiple graph data shards, can also be determined as the above-mentioned intermediate calculation results, which is not specifically limited in this specification.

[0143] In one embodiment shown, please continue to see Figure 3 , the above multiple graph data shards can still be stored discretely in the database.

[0144] For example, in one example, the multiple graph data fragments can be discretely stored in the database in CSR (Compressed SparseRow) format. The CSR format is widely used in efficient graph storage due to its excellent analysis performance.

[0145] In order to globally access the vertices contained in the above-mentioned multiple graph data slices discretely stored in the database, the above-mentioned computing system can maintain a global graph structure corresponding to the multiple graph data slices.

[0146] The global graph structure may be a virtual graph structure abstracted from the graph structures of the multiple graph data slices discretely stored in the database. The global graph structure may be specifically used to describe the global neighbor relationships of the vertices with neighbor relationships contained in the multiple graph data slices in the multiple graph data slices.

[0147] It should be noted that the so-called global neighbor relationship may specifically include the neighbor relationship between each vertex with neighbor relationship contained in the above-mentioned multiple graph data shards, and all neighbor vertices of these vertices in each of the multiple graph data shards. That is, the global graph structure is a global graph structure constructed based on the neighbor relationship between the vertices contained in each graph data shard and all neighbor vertices of the vertex in each graph data shard.

[0148] In practical applications, the above-mentioned computing system can obtain all neighbor vertices of the vertices contained in each of the multiple graph data slices in each graph data slice before continuing to perform online graph calculations on the multiple graph data slices based on the intermediate calculation results, and then construct the global graph structure based on the neighbor relationships between the vertices contained in each graph data slice and these neighbor vertices.

[0149] In this case, in the online graph calculation stage, when the above-mentioned computing system continues to perform online graph calculations on the multiple graph data slices based on the above-mentioned preset graph algorithm on the basis of the above-mentioned intermediate calculation results, it can specifically traverse the vertices with neighbor relationships contained in the multiple data graph slices in the global graph structure, and all the neighbor vertices of the neighbor vertices in each graph data slice, so as to continue to perform online graph calculations on the vertices contained in the multiple graph data slices.

[0150] In one embodiment shown, please continue to see Figure 3 Assuming that the graph data slices stored in the database share vertices, the above graph algorithm is an iterative graph algorithm that adopts a vertex-centric programming model. In the initialization stage, in addition to calculating the intermediate calculation results of the global graph calculation for the vertex based on the vertices with neighbor relationships contained in each of the above multiple graph data slices, it is usually also necessary to determine an initial set of activated vertices for each graph data slice from the vertices with neighbor relationships contained in each of the multiple graph data slices based on the intermediate calculation results.

[0151] For example, in one example, as mentioned above, the intermediate calculation result may be the graph calculation result with the smallest error from the final calculation result of the global graph calculation for the multiple graph data shards among the multiple graph calculation results. In other words, the graph calculation result with the smallest error from the final calculation result is the graph calculation result with the closest value to the final calculation result.

[0152] In this case, the computing system, based on the intermediate computing results, determines the initial set of activated vertices that need to participate in the iteration of the global graph computing for each graph data slice from the vertices with neighbor relationships contained in each of the multiple graph data slices. For any of the multiple graph data slices, it can be determined whether the graph computing result (that is, the above transition result) of the vertices with neighbor relationships contained in the graph data slice is the same as the target graph computing result with the minimum error from the above final computing result determined for the vertex. If they are not the same, it means that the vertex may affect the final graph computing result. At this time, the vertex can be added to the initial set of activated vertices created for the graph data slice as an activated vertex that needs to participate in the iteration of the global graph computing. Of course, if they are the same, it means that the vertex has completed the traversal and the vertex no longer needs to participate in the next round of iteration as an activated vertex.

[0153] Among them, it should be noted that, since any graph data slice among the multiple graph data slices contains vertices with neighbor relationships, the graph calculation result in the graph data slice is different from the target graph calculation result determined for the vertex with the minimum error from the above-mentioned final calculation result, which usually means that the graph calculation results of the vertex in each graph data slice are not exactly the same, and there will be certain differences; therefore, taking these vertices as activated vertices is equivalent to determining the vertices with differences in the graph calculation results in each graph data slice as activated vertices that need to participate in the global graph calculation iteration.

[0154] In this way, in the context of shared vertices among various graph data shards, vertices with different transition results in various graph data shards can be used as activated vertices, and continuously participate in the iteration of the graph algorithm, thereby continuously optimizing the final calculation results.

[0155] Please continue to see Figure 3 After an initial set of activated vertices is determined for each graph data shard from the vertices with neighbor relationships contained in each of the above-mentioned multiple graph data shards, in the online graph calculation stage, the above-mentioned computing system can continue to use the iterative graph algorithm of the vertex-centric programming model to iteratively perform online graph calculations for each graph data shard.

[0156] In this scenario, an iterative graph algorithm based on a vertex-centric programming model is used to perform offline graph computing on any target graph data shard stored in the database. The algorithm flow can be abstracted into the following algorithm logic:

[0157] First, based on the initial activated vertex set of each graph data shard, graph calculation can be performed on the activated vertices in the initial activated vertex set based on the above-mentioned preset graph algorithm to obtain the graph calculation results of the activated vertices in the activated vertex set;

[0158] Furthermore, after obtaining the graph calculation results of the activated vertices in the activated vertex set, the neighbor vertices of the activated vertices in the activated vertex set can be further traversed in the global graph structure, and based on the graph calculation results of the activated vertices in the activated vertex set, it is determined whether the neighbor vertices of the traversed activated vertices meet the activation conditions corresponding to the graph algorithm.

[0159] Furthermore, neighbor vertices that meet the activation condition may be determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iterations may be further created.

[0160] In the next round of iteration, the above calculation process can be repeated. Based on the determined new set of activated vertices, graph calculations can be continued for the activated vertices in the new set of activated vertices based on the above preset graph algorithm. And so on, the above calculation process is continuously iterated until it is impossible to determine the activated vertices that need to continue to participate in the next round of iteration or the calculation of the iteration meets the preset algorithm convergence conditions.

[0161] In one embodiment shown, the computing node may, based on the initial activated vertex set of each of the multiple graph data slices, traverse the neighbor vertices of the activated vertices in the activated vertex set in the graph structure of each graph data slice, and perform one or more rounds of iterative calculations of the computing process for each graph data slice to obtain a new activated vertex set corresponding to each graph data slice that needs to continue to participate in the next round of iteration.

[0162] Furthermore, new activated vertex sets corresponding to each graph data slice obtained after a round of iteration for each graph data slice can be merged to obtain a target activated vertex set corresponding to the above-mentioned multiple graph data slices, and then, based on the merged target activated vertex set, the above calculation process can be iteratively executed by traversing the neighbor vertices of the activated vertices in the activated vertex set in the above-mentioned global graph structure until it is impossible to determine the activated vertices that need to continue to participate in the next round of iteration or the iterative calculation meets the preset algorithm convergence condition.

[0163] In this way, when the computing system performs global graph calculations on the above-mentioned multiple graph data shards, it no longer needs to traverse neighbor vertices in each graph data shard. It only needs to summarize the activated vertices obtained in one round of iterations in each graph data shard, and then continue to traverse neighbor vertices on the global graph to complete the iterative calculations, thereby avoiding repeated iterative calculations on each graph data shard and reducing the amount of calculations when performing global graph calculations on the above-mentioned multiple graph data shards.

[0164] It should be noted that the structural type adopted by the above-mentioned global graph structure is not particularly limited in this specification.

[0165] In one embodiment shown, the global graph structure may specifically adopt an adjacency list structure.

[0166] Specifically, the adjacency list may maintain vertex identifiers of vertices with neighbor relationships in the above-mentioned multiple graph data slices, and a linked list corresponding to the vertex identifier. The linked list may specifically include pointers to all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data slice.

[0167] In this case, when the above-mentioned computing system traverses all neighbor vertices of the vertices with neighbor relationships contained in the multiple data graph shards in the global graph structure, it can specifically traverse the vertex identifiers maintained in the adjacency list, and based on the pointer contained in the linked list maintained in the adjacency list corresponding to the traversed vertex identifier, further access all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard that are discretely stored in the database.

[0168] In one embodiment shown, in order to improve the storage space utilization of the linked list maintained in the adjacency list, the pointers included in the linked list may specifically adopt a double-layer pointer design.

[0169] Specifically, the first-level pointer in the linked list corresponding to each vertex identifier maintained in the adjacency list may specifically include a pointer to the first neighbor vertex of the vertex corresponding to the vertex identifier in each graph data shard; the second-level pointer in the linked list may specifically include the pointer offsets of other neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard relative to the first neighbor vertex.

[0170] In this case, in the process of further accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer contained in the linked list corresponding to the traversed vertex identifier, the computing system can access the first neighbor vertex of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer of the first layer contained in the linked list corresponding to the vertex identifier, and then further access other neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer offset of the second layer contained in the linked list.

[0171] In this way, the linked list corresponding to each vertex identifier maintained in the adjacency list can only maintain the complete pointer of the first neighbor vertex corresponding to the vertex identifier in each graph data shard, while other neighbor vertices corresponding to the vertex identifier in each graph data shard do not need to maintain complete pointers, but only need to maintain a pointer offset relative to the first neighbor vertex, thereby significantly reducing the size of the storage space occupied by the linked list corresponding to each vertex identifier maintained in the adjacency list.

[0172] In actual applications, when the computing system further accesses all neighbor vertices of a vertex in each graph data shard based on the pointers contained in the above-mentioned linked list, all neighbor vertices of a vertex in each graph data shard are discretely stored in the database, and the number of neighbor vertices of different vertices in each graph data shard may also vary to a certain extent; therefore, if the computing system only accesses the neighbor vertices discretely stored in the database according to a single access mode, it may affect the computing system's access performance to the neighbor vertices.

[0173] Based on this, in this specification, the computing system can access all neighbor vertices of the vertices discretely stored in the database in each graph data shard according to two access modes: sequential access and parallel access.

[0174] In one embodiment shown, a computing system may maintain the total number of neighbor vertices in each graph data shard of the vertices having neighbor relationships contained in the above-mentioned multiple graph data shards; in the process of traversing the vertex identifiers maintained in the adjacency list and further accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer contained in the linked list corresponding to the vertex identifier, the computing system may obtain the total number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard for any vertex identifier traversed in the adjacency list, and determine whether the total number obtained is greater than a threshold value;

[0175] On the one hand, if the total number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard is not greater than the threshold, then only one independent thread can be allocated to the vertex corresponding to the vertex identifier, and then the thread can be run to access all neighbor vertices of the vertex in each graph data shard in sequence according to the order of the pointers contained in the linked list maintained in the adjacency list corresponding to the vertex identifier. This access mode can be called a sequential access mode.

[0176] On the other hand, if the total number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard is greater than a threshold, multiple independent threads can be allocated to the vertex, and then the multiple independent threads can be run to access all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard in parallel based on the pointers contained in the linked list corresponding to the vertex identifier. This access mode can be called a parallel access mode.

[0177] By providing the computing system with two access modes, sequential access and parallel access, the computing system can flexibly determine the access mode based on the total number of neighbor vertices of the vertex in each graph data shard during the process of traversing the vertex, thereby making more reasonable use of the existing access resources of the computing system.

[0178] In one embodiment shown, in order to further optimize the access performance to all neighbor vertices of a vertex in each graph data shard, the computing system can traverse the vertex identifiers maintained in the adjacency list, and based on the pointer contained in the linked list corresponding to the vertex identifier, further access all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard, and can determine in advance from each vertex identifier maintained in the adjacency list the vertex identifier of the target vertex whose total number of neighbor vertices in each graph data shard is not greater than a threshold.

[0179] Then, all neighbor vertices of the target vertex corresponding to the target vertex identifier in each graph data shard can be read from the database, and all the read neighbor vertices can be further stored in a continuous memory space pre-allocated in the memory for storage, and then the pointer contained in the linked list maintained in the adjacency list corresponding to the target vertex identifier can be further replaced with the pointer corresponding to the continuous memory space.

[0180] In this way, when the computing system traverses the vertex identifiers maintained in the adjacency list and further accesses all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer contained in the linked list corresponding to the vertex identifier, if a target vertex is traversed and the total number of neighbor vertices in each graph data shard is not greater than a threshold, the neighbor vertices of the target vertex can be directly loaded from the continuous storage space in the memory, thereby further improving the access performance to the neighbor vertices.

[0181] For example, see Figure 4 Assume that a vertex V has three neighbor vertices in three different graph data shards, namely vertices 1, 2, and 3. Since these three neighbor vertices are discretely stored in different locations in the database, the computing system usually needs to access all the neighbor nodes of the vertex V three times to load all the neighbor nodes of the vertex V into the memory. Moreover, in practical applications, among the vertices contained in the graph data, the vertices with a small total number of neighbor vertices in each graph data shard usually account for the majority, which will significantly affect the computing system's access performance to the neighbor nodes of the vertex in each graph data shard.

[0182] Please continue to see Figure 4If the three neighbor nodes of the vertex V are stored in advance in a continuous memory space pre-allocated in the memory, when the computing system runs an independent thread allocated for the vertex V to access all the neighbor nodes of the vertex V, the thread only needs to access once based on the replaced pointer contained in the linked list corresponding to the vertex V, and it can access all the neighbor nodes of the vertex V from the above continuous memory space. In this way, it is obvious that the access performance of all neighbor vertices of a vertex in each graph data shard can be further optimized.

[0183] In practical applications, when the computing system accesses vertices whose total number of neighbor nodes in each graph data shard is greater than a threshold based on the above-mentioned parallel access mode, in the process of accessing all neighbor vertices in each graph data shard, since the number of neighbor vertices of the vertex in each graph data shard may vary greatly, this may cause the load of multiple threads assigned to the vertex to be unbalanced; therefore, if the computing system does not consider the fact that the number of neighbor vertices of the vertex in each graph data shard may vary greatly when accessing all neighbor vertices of the vertex in each graph data shard based on the above-mentioned parallel access mode, it may affect the computing system's access performance to the neighbor vertices.

[0184] Based on this, in this specification, the computing system, based on the above-mentioned parallel access mode, accesses vertices whose total number of neighbor nodes in each graph data shard is greater than a threshold. In the process of accessing all neighbor vertices in each graph data shard, the specific parallel access strategy can be dynamically determined based on whether the distribution of neighbor vertices of the vertex in each graph data shard is balanced.

[0185] In one embodiment shown, the computing system traverses the vertex identifiers maintained in the adjacency list, and further accesses all neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard based on the pointer contained in the linked list corresponding to the vertex identifier. For any vertex identifier traversed in the adjacency list, if the total number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard is greater than a threshold, the difference in the number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard can be further calculated, and it can be determined whether the difference is lower than the threshold;

[0186] On the one hand, if the difference in the number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data shard is lower than the threshold, it indicates that the distribution of the neighbor vertices of the vertex in each graph data shard is relatively balanced. At this time, multiple independent threads corresponding to the multiple graph data shards can be allocated to the vertex corresponding to the vertex identifier. That is, each thread in the multiple independent threads allocated to the vertex corresponds to one of the multiple graph data shards, and each thread is only used to access all the neighbor vertices of the vertex in the graph data shard corresponding to the thread. Since the distribution of the neighbor vertices of the vertex in each graph data shard is relatively balanced at this time, the load of the multiple independent threads finally allocated to the vertex usually does not have an imbalance problem. This parallel access strategy can be called a subgraph-level parallel access strategy.

[0187] On the other hand, if the difference in the number of neighbor vertices of the vertex corresponding to the vertex identifier in each graph data slice is greater than the threshold, it indicates that the distribution of the neighbor vertices of the vertex in each graph data slice is unbalanced. At this time, multiple independent threads with the same number of neighbor vertices that can be processed can be assigned to the vertex corresponding to the vertex identifier, that is, each thread in the multiple independent threads assigned to the vertex can process exactly the same number of neighbor vertices. In practical applications, all neighbor vertices of the vertex in each graph data slice can be evenly assigned to each thread in the multiple independent threads according to the total number, and each thread is only used to access the neighbor vertices assigned to the thread from all the neighbor vertices of the vertex in each graph data slice.

[0188] At this time, although the distribution of neighbor vertices of the vertex in each graph data shard is not balanced, since each of the multiple independent threads assigned to the vertex can process exactly the same number of neighbor vertices, the load of the multiple independent threads assigned to the vertex will not be unbalanced. This parallel access strategy can be called an edge-level parallel access strategy.

[0189] In this specification, when the computing system completes the online graph calculation for the multiple graph data slices generated within the above-mentioned preset time window and obtains the final calculation results of the global graph calculation for the multiple graph data slices, the final calculation results can be used to perform graph data analysis on the multiple graph data slices using the monotonicity of the final calculation results.

[0190] The specific application scenario of performing graph data analysis on the multiple graph data slices is not particularly limited in this specification.

[0191] For example, in one embodiment shown, an application scenario of performing graph data analysis on the multiple graph data shards may specifically be an application scenario of performing graph queries on the multiple graph data shards. In this application scenario, a computational request for performing global graph computation on the multiple graph data shards is typically a computational request triggered by a graph query operation on the multiple graph data shards.

[0192] In this case, after the computing system completes the online graph calculation for the multiple graph data shards generated within the above-mentioned preset time window and obtains the final calculation results of the global graph calculation for the multiple graph data shards, the computing system can respond to the final calculation results obtained by performing global graph calculation on the multiple graph data shards, and generate a query result that satisfies the query conditions related to the graph query operation based on the final calculation results; then, the query result is output to the initiator of the graph query operation.

[0193] It should be noted that, in actual applications, the query scenario of performing graph queries on the multiple graph data shards may cover any form of daily graph queries on the various graph data shards stored in the database, and is not specifically limited in this specification.

[0194] For example, in one example, the vertices contained in the graph data slices represent users; the edges between the connected vertices contained in the graph data slices represent transactions between users. In this case, the graph query for the multiple graph data slices can specifically query the connected components contained in the multiple graph data slices generated within the preset time window. The connected component usually refers to the set of vertices connected by edges contained in the graph data. Any two vertices in a connected component will be connected by an edge. A connected component can usually be used to represent a user group with a transaction relationship.

[0195] In the above technical solution, on the one hand, since the graph calculation results of the offline calculation for the multiple graph data shards can be reused during the online global graph calculation for the multiple graph data shards, it is possible to avoid repeated graph calculation for the multiple graph data shards during the online calculation stage, thereby reducing the occupation of online computing resources in the system, reducing the computing overhead, and improving the computing performance of the system for the online global graph calculation for the multiple graph data shards;

[0196] On the other hand, during the process of performing global graph calculations online for the multiple graph data shards, based on the graph calculation results of offline calculations for the multiple graph data shards, the online graph calculations for the multiple graph data shards can be continued to be completed based on the intermediate calculation results of the global graph calculations for the multiple graph data shards. When the system performs global graph calculations online for the multiple graph data shards, the system can avoid merging the graph structures of the multiple graph data shards, thereby significantly reducing the system overhead and further improving the computing performance of the global graph calculations for the multiple graph data shards online.

[0197] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of an apparatus, an electronic device, and a storage medium.

[0198] Figure 5 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and may also include other required hardware. One or more embodiments of this specification may be implemented based on software, such as the processor 502 reading the corresponding computer program from the non-volatile memory 510 into the memory 508 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or logic devices.

[0199] like Figure 6 As shown, Figure 6 is a block diagram of a graph computing device according to an exemplary embodiment of the present specification. The device can be applied to Figure 5 In the electronic device shown, the technical solution of this specification is implemented. The device can be applied to any consensus node participating in the consensus in the blockchain system, wherein the process address space of the consensus process running on the consensus node includes a reserved mapping area; the virtual memory address in the mapping area establishes a one-to-one mapping relationship with the physical storage address in the non-volatile physical storage space carried by the consensus node for persistent storage of consensus data; the device 600 includes:

[0200] The first calculation module 601 performs offline graph calculations on each graph data shard stored in the database based on a preset graph algorithm, and saves the graph calculation results obtained by performing the graph calculations on the graph data shards; wherein the graph algorithm is a graph algorithm whose output graph calculation results are monotonic;

[0201] An acquisition module 602, in response to a computation request for performing global graph computation on a plurality of graph data shards generated within a preset time window and stored in a database, acquires a plurality of graph computation results obtained by performing offline graph computations on the plurality of graph data shards respectively;

[0202] The second computing module 603 calculates the intermediate computing results of the global graph computing for the multiple graph data shards based on the multiple graph computing results, and based on the intermediate computing results, continues to perform online graph computing for the multiple graph data shards based on the preset graph algorithm to obtain the final computing results of the global graph computing for the multiple graph data shards.

[0203] Accordingly, the present specification also provides an electronic device, which includes a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the steps in all the method flows described above.

[0204] Accordingly, the present specification also provides a computer-readable storage medium on which executable computer program instructions are stored; wherein, when the instructions are executed by a processor, the steps in all the method flows described above are implemented.

[0205] Accordingly, the present specification also provides a computer program product having executable computer program instructions stored thereon; wherein, when the computer program instructions are executed by a processor, the steps in all the method flows described above are implemented.

[0206] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0207] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0208] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0209] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0210] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0211] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0215] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0216] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0217] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0218] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0219] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0220] The above description is only an example of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims.

Claims

1. A graph computing method, applied to a computing system for performing graph computing, the method include: Based on a preset graph algorithm, offline graph calculations are performed on each graph data shard stored in the database, and graph calculation results obtained by performing graph calculations on the graph data shard are saved; wherein the graph algorithm is a graph algorithm whose output graph calculation results are monotonic; In response to a calculation request for performing global graph calculation on a plurality of graph data shards generated within a preset time window and stored in a database, obtaining a plurality of graph calculation results obtained by performing offline graph calculations on the plurality of graph data shards respectively; Based on the multiple graph calculation results, intermediate calculation results of global graph calculation for the multiple graph data shards are calculated, and based on the intermediate calculation results, online graph calculations are continued to be performed for the multiple graph data shards based on the preset graph algorithm to obtain final calculation results of global graph calculation for the multiple graph data shards.

2. According to the method as claimed in claim 1, the graph data shards are graph data generated within a preset unit time period.

3. According to the method described in claim 2, the graph data slices stored in the database are periodically imported into the database for storage according to the unit time period.

4. The method according to claim 1, performing offline graph calculation on the graph data shards stored in the database based on a preset graph algorithm, include: In response to graph data shards being imported into the database, offline graph calculations are performed on the imported graph data shards based on a preset graph algorithm.

5. According to the method as claimed in claim 1, the intermediate calculation result is the graph calculation result among the multiple graph calculation results, which has the smallest error with the final calculation result of the global graph calculation performed on the multiple graph data shards.

6. The method according to claim 1, wherein at least some of the vertices contained in each graph data fragment stored in the database are the same vertices; and the graph algorithm comprises an iterative graph algorithm; in, The iterative graph algorithm includes a graph algorithm that performs iterative graph calculations by traversing vertices with neighbor relationships contained in the graph data.

7. The method of claim 6, wherein the plurality of graph data shards are discretely stored in the database; the computing system maintains a global graph structure corresponding to the plurality of graph data shards discretely stored in the database; in, The global graph structure is used to describe the global neighbor relationship of the vertices with neighbor relationship contained in the multiple graph data slices in the multiple graph data slices; The global neighbor relationship includes a neighbor relationship between the vertex and all neighbor vertices of the vertex in each of the multiple graph data shards; Based on the intermediate calculation results, continuing to perform online graph calculations on the multiple graph data shards based on the preset graph algorithm includes: On the basis of the intermediate calculation results, based on the preset graph algorithm, the vertices with neighbor relationships contained in the multiple data graph slices are traversed in the global graph structure, and all the neighbor vertices of the neighbor vertices in each graph data slice are traversed to continue to perform online graph calculations on the vertices contained in the multiple graph data slices.

8. The method according to claim 7, wherein the global graph structure adopts an adjacency list structure; in, The adjacency list maintains vertex identifiers of vertices with neighbor relationships contained in the multiple graph data fragments, and a linked list corresponding to the vertex identifiers; wherein the linked list contains pointers to all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data fragments; In the global graph structure, vertices with neighbor relationships contained in the plurality of data graph slices are traversed, and all neighbor vertices in each of the graph data slices include: The vertex identifiers maintained in the adjacency list are traversed, and based on the pointers contained in the linked list corresponding to the vertex identifiers, all neighbor vertices of the vertex corresponding to the vertex identifier in the respective graph data fragments are further accessed.

9. The method as claimed in claim 8, wherein the linked list includes a pointer to a first neighbor vertex of the vertex corresponding to the vertex identifier in each of the graph data slices, and a pointer offset of other neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices relative to the first neighbor vertex; Based on the pointer contained in the linked list corresponding to the vertex identifier, further access all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices, include: Based on the pointer contained in the linked list corresponding to the vertex identifier, the first neighbor vertex of the vertex corresponding to the vertex identifier in each of the graph data slices is accessed, and based on the pointer offset contained in the linked list, other neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices are accessed.

10. The method according to claim 7, performing graph calculation on any target graph data slice stored in the database based on a preset graph algorithm, include: Based on the initial set of activated vertices that need to participate in the iteration specified from the vertices contained in the target graph data slice, iteratively execute the preset calculation process until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation meets the preset algorithm convergence condition; The preset calculation process includes: Performing graph calculation on the activated vertices in the initial set of activated vertices based on the preset graph algorithm to obtain graph calculation results of the activated vertices in the set of activated vertices; Traversing neighbor vertices of the activated vertices in the activated vertex set in the target graph data shard, and determining whether the traversed neighbor vertices of the activated vertex meet activation conditions corresponding to the graph algorithm based on graph calculation results of the activated vertices in the activated vertex set; The neighboring vertices satisfying the activation condition are determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iteration is further created.

11. The method according to claim 10, wherein the plurality of graph calculation results include graph calculation results of the vertices with neighbor relationships contained in each of the plurality of graph data shards obtained by performing offline graph calculation on the vertices in the graph data shards, respectively; Calculate intermediate calculation results of global graph calculations for the multiple graph data shards based on the multiple graph calculation results, include: Based on the vertices with neighbor relationships contained in each of the multiple graph data slices and the graph calculation results in each of the graph data slices, an intermediate calculation result of the global graph calculation for the vertices is calculated.

12. The method as claimed in claim 11, based on the intermediate calculation results, traverses the vertices with neighbor relationships contained in the multiple data graph slices in the global graph structure based on the preset graph algorithm, and all neighbor vertices of the neighbor vertices in each graph data slice, so as to continue to perform online graph calculations on the vertices contained in the multiple graph data slices, include: Based on the intermediate calculation result, from vertices with neighbor relationships contained in each of the multiple graph data slices, determine an initial set of activated vertices for each of the graph data slices; Based on the initial set of activated vertices, for each of the graph data slices, the preset calculation process is iteratively executed respectively until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation meets the preset algorithm convergence condition; The preset calculation process includes: Performing graph calculation on the activated vertices in the initial set of activated vertices based on the preset graph algorithm to obtain graph calculation results of the activated vertices in the set of activated vertices; Traversing neighbor vertices of the activated vertices in the activated vertex set in the global graph structure, and determining whether the traversed neighbor vertices of the activated vertex meet an activation condition corresponding to the graph algorithm based on the graph calculation results of the activated vertices in the activated vertex set; The neighboring vertices satisfying the activation condition are determined as new activated nodes, and based on the determined new activated vertices, an activated vertex set that needs to continue to participate in the next round of iteration is further created.

13. The method according to claim 12, based on the initial set of activated vertices, iteratively executes the preset calculation process for each of the graph data slices, until no activated vertices that need to continue to participate in the next round of iteration can be determined or the iterative calculation meets the preset algorithm convergence condition, include: On the basis of the initial set of activated vertices, by traversing the neighbor vertices of the activated vertices in the set of activated vertices in each of the graph data slices, a round of the preset calculation process is performed respectively to obtain the set of activated vertices corresponding to each of the graph data slices that need to continue to participate in the next round of iteration; The activated vertex sets corresponding to the respective graph data slices are merged to obtain a target activated vertex set corresponding to the multiple graph data slices, and based on the target activated vertex set, the preset calculation process is continued to be iteratively executed by traversing the neighbor vertices of the activated vertices in the activated vertex set in the global graph structure until it is impossible to determine the activated vertices that need to continue to participate in the next round of iteration or the iterative calculation satisfies the preset algorithm convergence condition.

14. The method of claim 11, based on the vertices with neighbor relationships contained in each of the plurality of graph data shards, and the graph calculation results in each of the graph data shards, calculates the intermediate calculation results of the global graph calculation for the vertices, include: For any vertex with a neighbor relationship contained in each of the multiple graph data slices, determine, from the graph calculation results of the vertex in each of the graph data slices, a target graph calculation result having the smallest error with a final calculation result of a global graph calculation performed on the vertex; The target graph calculation result is determined as an intermediate calculation result of a global graph calculation performed on the vertex.

15. The method of claim 12, further comprising: determining, based on the intermediate calculation results, from vertices with neighbor relationships contained in each of the plurality of graph data shards, for each of the graph data shards, an initial set of activated vertices that need to participate in the iteration of the global graph calculation, include: For any graph data slice among the multiple graph data slices, determine whether a graph calculation result of a vertex with a neighbor relationship contained in the graph data slice in the graph data slice is the same as the target graph calculation result determined for the vertex; If they are not the same, the vertex is taken as an activated vertex that needs to participate in the iteration of the global graph calculation and is added to the initial activated vertex set created for the graph data slice.

16. The method according to claim 8, wherein the computing system further maintains the total number of neighbor vertices of the vertices with neighbor relationships contained in the multiple graph data shards in each graph data shard; Traversing the vertex identifiers maintained in the adjacency list, and further accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices based on the pointers contained in the linked list corresponding to the vertex identifier, including: For any vertex identifier traversed in the adjacency list, obtain the total number of neighbor vertices of the vertex corresponding to the vertex identifier maintained in each of the graph data slices, and determine whether the total number obtained is greater than a threshold; If the total number obtained is not greater than the threshold, an independent thread is allocated to the vertex corresponding to the vertex identifier, and the thread is run to visit all the neighbor vertices of the vertex in the various graph data slices in sequence according to the order of the pointers contained in the linked list maintained in the adjacency list corresponding to the vertex identifier; if the total number obtained is greater than the threshold, multiple independent threads are allocated to the vertex, and the multiple independent threads are run to visit all the neighbor vertices of the vertex corresponding to the vertex identifier in the various graph data slices in parallel based on the pointers contained in the linked list corresponding to the vertex identifier.

17. The method of claim 16, wherein if the total number obtained is greater than the threshold, multiple independent threads are allocated to the vertex, include: If the total number obtained is greater than the threshold, further calculating the difference in the number of neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices, and determining whether the difference is lower than the threshold; If yes, a plurality of independent threads corresponding to the plurality of graph data slices are allocated to the vertex corresponding to the vertex identifier; wherein each thread in the plurality of independent threads is used to access all neighbor vertices of the vertex in the graph data slice corresponding to the thread; If not, multiple independent threads with the same number of neighbor vertices that can be processed are assigned to the vertex corresponding to the vertex identifier; wherein each thread in the multiple independent threads is used to access the neighbor vertices assigned to the thread from all the neighbor vertices of the vertex in each graph data slice; the number of neighbor vertices assigned to each thread is the same.

18. The method of claim 16, further comprising traversing the vertex identifiers maintained in the adjacency list and, based on the pointers contained in the linked list corresponding to the vertex identifiers, accessing all neighbor vertices of the vertex corresponding to the vertex identifier in each of the graph data slices. include: Determine a target vertex identifier from each vertex identifier maintained in the adjacency list; wherein the target vertex identifier is a vertex identifier corresponding to a target vertex whose total number of neighbor vertices in each graph data fragment is not greater than a threshold; All neighbor vertices of the target vertex corresponding to the target vertex identifier in each graph data shard are read from the database, and the read neighbor vertices are further stored in a continuous memory space pre-allocated in the memory, and the pointer contained in the linked list maintained in the adjacency list corresponding to the target vertex identifier is further replaced by a pointer corresponding to the continuous memory space.

19. The method of claim 6, wherein the computing request is a computing request triggered by a graph query operation for a plurality of graph data shards generated within a preset time window; The method also include: In response to a final calculation result obtained by performing global graph calculation on the multiple graph data shards, generating a query result that satisfies a query condition related to the graph query operation based on the final calculation result; The query result is output to the initiator of the graph query operation.

20. The method of claim 7, wherein the plurality of graph data shards are discretely stored in the database in a CSR format.

21. In the method as claimed in claim 1, the vertices contained in the graph data shards represent users; and the edges between the connecting vertices contained in the graph data shards represent transactions generated between users.

22. The method of claim 1, wherein the database comprises a data warehouse.

23. A graph computing device, applied to a computing system for performing graph computing, the device include: A first computing module performs offline graph computing on each graph data shard stored in the database based on a preset graph algorithm, and saves the graph computing results obtained by performing the graph computing on the graph data shard; wherein the graph algorithm is a graph algorithm whose output graph computing results are monotonic; An acquisition module, in response to a computation request for performing global graph computation on a plurality of graph data shards generated within a preset time window and stored in a database, acquires a plurality of graph computation results obtained by performing offline graph computations on the plurality of graph data shards respectively; The second computing module calculates intermediate computing results of global graph computing for the multiple graph data shards based on the multiple graph computing results, and based on the intermediate computing results, continues to perform online graph computing for the multiple graph data shards based on the preset graph algorithm to obtain final computing results of global graph computing for the multiple graph data shards.

24. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, in, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 22 are implemented.

25. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 22.

26. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 22 are implemented.

Images