Flow graph processing method and processing system based on sub-graph hybrid calculation

Abstract

The invention belongs to the technical field related to information analysis, and discloses a flow graph processing method and processing system based on sub-graph hybrid computation.The processing method comprises the steps that an original flow graph in a graph database is divided into a plurality of sub-graphs with the vertex total weight distributed in a balanced mode; executing a graph updating operation on the sub-graphs; matching a corresponding graph calculation mode for each updated sub-graph, matching the corresponding updated sub-graph with a full-amount calculation mode when the proportion of the vertexes influenced by the updating operation in the updated sub-graphs exceeds a preset proportion, and otherwise, matching with an incremental calculation mode; extracting a dependency relationship between the sub-graphs; performing graph calculation on each updated sub-graph in parallel by adopting the matched graph calculation mode to respectively obtain an intermediate result of each sub-graph; and transmitting and updating the intermediate result of each sub-graph based on the dependency relationship, and outputting a global processing result of the flow graph. According to the processing mode, the locality characteristic of the graph can be fully utilized, the calculation overhead is reduced, and the accuracy of a calculation result is ensured.

Description

Technical Field

[0001] This invention belongs to the field of information analysis technology, and more specifically, relates to a flow graph processing method and system based on subgraph hybrid computation. Background Technology

[0002] Information graphs can be used to represent entities and the relationships between them. They have vertices and edges connecting the vertices. Vertices represent entities involved in the analysis, and edges between two vertices represent the relationship path between the two entities. Examples include common social network graphs and traffic network graphs. Traditional information processing methods are usually based on static graphs, that is, the structure of the graph remains unchanged during the calculation process, which is suitable for one-time analysis or offline batch processing tasks.

[0003] However, in real-world applications, the relationships within a graph are often dynamic. For example, users in social networks continuously add friends or follow accounts, and road conditions in traffic networks change in real time. Graphs that change over time are called streaming graphs. In streaming graph scenarios, the graph structure changes frequently, including the insertion and deletion of edges. To address this dynamic nature, streaming graph computation typically employs either incremental computation or full computation. Full computation involves recalculating all vertices and edges of the entire graph after each update. Full computation is generally simple to implement and yields accurate results, but it incurs high computational overhead when the graph is large or updated frequently. Incremental computation, on the other hand, performs local computation only on the vertices or edges affected by the update and reuses historical computation results, thereby reducing redundant computational overhead. Incremental computation is more efficient in scenarios with small update scales or strong locality, but it is complex to implement and requires accurate identification of the affected area; otherwise, it may lead to inaccurate results. In other words, full computation is simple to implement but incurs high computational overhead due to a large amount of redundant computation; incremental computation can effectively reduce redundant computation but is complex to implement and may introduce computational bias.

[0004] Therefore, there is an urgent need to propose a flow graph processing scheme that can balance computational overhead and computational accuracy. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a flow graph processing method and system based on subgraph hybrid computation, which aims to balance the computational overhead and computational accuracy of flow graph processing.

[0006] To achieve the above objectives, the following technical solution is proposed.

[0007] According to a first aspect of the present invention, a flow graph processing method based on subgraph hybrid computation is provided, comprising: In response to the graph processing request, the original flow graph in the graph database is divided into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent the relationship or traffic path between the two entities. The closer the relationship or the longer the traffic path, the greater the weight of the edge. Perform graph update operations on the subgraph, including deleting and adding edges; For each updated subgraph, a corresponding graph computation method is matched. The graph computation method includes full computation method and incremental computation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full computation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental computation method. After the graph is updated, the dependencies between subgraphs are extracted. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph. The matching graph computation method is used to perform graph computations in parallel on each updated subgraph, and intermediate results are obtained for each subgraph. Based on the dependencies between subgraphs, the intermediate results of each subgraph are passed and updated. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate results of the current subgraph are updated. The passing and updating of the intermediate results of the subgraphs are executed sequentially, and the global processing result of the flow graph is output.

[0008] According to a second aspect of the present invention, a flow graph processing system based on subgraph hybrid computation is provided, comprising: The graph partitioning module is used to respond to graph processing requests and divide the original flow graph in the graph database into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent the relationship or traffic path between the two entities. The closer the relationship or the longer the traffic path, the greater the weight of the edge. The graph update module is used to perform graph update operations on subgraphs, including deleting and adding edges. The calculation method selection module is used to match the corresponding graph calculation method for each updated subgraph. The graph calculation methods include full calculation method and incremental calculation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full calculation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental calculation method. The dependency construction module is used to extract the dependencies between subgraphs after the graph is updated. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph. The graph computation module is used to perform graph computations on each updated subgraph in parallel using the matched graph computation method, and obtain the intermediate results of each subgraph. The inter-graph scheduling module is used to pass and update intermediate results of each subgraph based on the dependencies between subgraphs. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate result of the current subgraph is updated. The passing and updating of intermediate results of subgraphs are executed sequentially, and the global processing result of the flow graph is output.

[0009] In summary, compared with the prior art, the technical solutions conceived in this invention have the following main advantages: This invention first divides the original flow graph into several subgraphs of similar size based on vertex weights. Then, each subgraph performs an update operation based on its update information. For each subgraph, an appropriate calculation method is selected based on the proportion of affected vertices. When the affected range exceeds a set threshold, it indicates that many vertices in the subgraph need to be recalculated, thus a full calculation method is used. When the affected range is below the threshold, it indicates that fewer vertices are affected, so an incremental calculation method is used. This method enables efficient incremental calculation on subgraphs with small local updates, while full calculation is used on subgraphs with large update ranges or significant structural changes, achieving efficient hybrid computation of the flow graph. Furthermore, by partitioning the subgraph, intermediate results are calculated in parallel using an appropriate graph computation method, and then the intermediate results are passed and updated based on the dependencies between subgraphs. Compared to traditional single incremental or full calculation methods, the hybrid subgraph computation and parallel processing method can fully utilize the locality of graphs, improve computational parallelism and resource utilization efficiency, while ensuring the accuracy and consistency of the calculation results. Attached Figure Description

[0010] Figure 1 This is a flowchart of the steps of a flow graph processing method based on subgraph hybrid computation in one embodiment of the present invention; Figure 2This is a simplified operation diagram of a flow graph processing method based on subgraph hybrid computation in one embodiment of the present invention; Figure 3 This is a schematic diagram of determining the affected vertices of the updated subgraph in one embodiment of the present invention; Figure 4 This is a schematic diagram of constructing a directed dependency graph in one embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the execution of graph calculations in different subgraphs according to an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the transfer and updating of intermediate results of each subgraph in one embodiment of the present invention; Figure 7 This is an architecture diagram of a flow graph processing system based on subgraph hybrid computation according to an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0012] In a first aspect, the present invention claims a flow graph processing method based on subgraph hybrid computation.

[0013] like Figure 1 The diagram shown is a flowchart of the steps of a flow graph processing method based on subgraph hybrid computation according to an embodiment of the present invention, as follows: Figure 2 The diagram shown is a simplified operation diagram of a flow graph processing method based on subgraph hybrid computation according to an embodiment of the present invention. The sequence of steps in the diagram is only one feasible approach and is not limited thereto. The following is combined with... Figure 1 This method will be described in detail.

[0014] S1. In response to the graph processing request, the original flow graph in the graph database is divided into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent the relationship or traffic path between the two entities. The closer the relationship or the longer the traffic path, the greater the weight of the edge.

[0015] Specifically, the original flow graph is divided into multiple subgraphs based on the structural features of the graph, including the degree of vertices, the weight of edges, and the connection relationships between vertices.

[0016] In one embodiment, the process of partitioning a subgraph may include several sub-steps.

[0017] S11. First, divide the original flow graph into multiple subgraphs.

[0018] The main goal of this step is to divide the original large graph into multiple subgraphs of more manageable size, thereby reducing the overall cost of subsequent graph computations. For example, community detection algorithms (such as the Louvain algorithm) can be used to initially divide the original flow graph into multiple subgraphs, resulting in several subgraphs with relatively compact structures.

[0019] Since the size of the subgraphs after the initial partitioning is often uneven, some large subgraphs may become performance bottlenecks in subsequent calculations. Therefore, it is necessary to use subsequent sub-steps to fine-tune the subgraphs to achieve approximate size of each subgraph and avoid large-scale subgraphs.

[0020] S12. Fine-tune the subgraph partitioning according to the weight balancing mechanism; the weight balancing mechanism includes: calculating the weight of each vertex in its subgraph, identifying high-weight subgraphs and low-weight subgraphs according to the total weight of the vertices in the subgraph, partitioning some vertices in the high-weight subgraphs to other subgraphs with connectivity, and merging the low-weight subgraphs.

[0021] The purpose of this step is to further adjust the subgraph based on the weights of the vertices within it, splitting vertices with excessively high weights into new subgraphs. For subgraphs that are too small, a merging operation is performed based on their connection structure with neighboring subgraphs, thereby obtaining a more balanced subgraph partitioning result with a more reasonable overall structure.

[0022] Optionally, the weight of vertex v in its subgraph is calculated. The calculation formula is: ; In the formula, This represents the edge connecting vertex u and vertex v. Representing an edge The weights are such that vertices u and v belong to the same subgraph.

[0023] Optionally, the high-weight subgraph and low-weight subgraph can be identified by setting a maximum value and a minimum value. When the total weight of the vertices of a subgraph is greater than the set maximum value, the subgraph is considered to be a high-weight subgraph. When the total weight of the vertices of a subgraph is less than the set minimum value, the subgraph is considered to be a low-weight subgraph.

[0024] Optionally, partitioning some vertices in the high-weight subgraph into other subgraphs with connectivity relationships includes: calculating the degree ratio of each vertex in the high-weight subgraph. The vertex v with the smallest degree ratio is assigned to the other subgraph with the smallest total weight of vertices connected to it, where the degree ratio of any vertex v is... The calculation formula is: ; In the formula, Let v be the degree by which vertex v is connected to all other vertices within its subgraph. Let v be the degree of any vertex v connected to any other vertex outside its subgraph. For a directed graph, the degree of any edge is the number of its outgoing edges.

[0025] In this embodiment, the vertices with the smallest degree ratio are moved to other subgraphs. The smallest degree ratio indicates that the vertex has a lower degree of association with its subgraph. Therefore, choosing to move it out of the subgraph can not only balance the weights, but also make the vertices within each subgraph more interconnected, which facilitates subsequent update operations.

[0026] Optionally, low-weight subgraphs may be merged, including prioritizing the merging of subgraphs with the lowest total vertex weights after the merge. For example, subgraphs with lower weights may be merged. Figure 1 For a low-weight subgraph, its relationship with sub-graphs is... Figure 2 Kazuko Figure 3 All have a connection relationship, if the child Figure 1 +sub Figure 2 The total weight of the vertices is less than that of the sub-vertices. Figure 1 +sub Figure 3 Given the total weight of the vertices, the child vertices are given priority. Figure 1 Merge to sub Figure 2 .

[0027] S2. Perform graph update operations on the subgraph. Graph update operations include deleting and adding edges.

[0028] The main objective of this phase is to collect, organize, and perform graph structure updates related to vertices.

[0029] Specifically, graph update operations mainly include two categories: adding edges and deleting edges. Based on the range of vertices involved, these operations are divided into corresponding subgraphs. Then, the corresponding update operation is executed within each subgraph, generating the updated subgraph.

[0030] In practice, each edge to be updated can be recorded as a triple [operator, source, destination] for unified management and processing. The source represents the source vertex and the destination represents the destination vertex. The update operation is divided into different subgraphs according to the subgraph to which the source belongs. Then, the update operation is applied to the subgraph to generate a new subgraph.

[0031] S3. Match the corresponding graph calculation method to each updated subgraph. The graph calculation method includes full calculation method and incremental calculation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full calculation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental calculation method.

[0032] The main objective of this step is to select the most suitable computation method for each updated subgraph based on the update information. If a subgraph has a large number of affected vertices, incremental computation will lose its advantage, and full computation will be more efficient. In this case, the system will choose to perform full computation on the subgraph to maintain algorithm efficiency and overall consistency. Conversely, when the update's impact is small, incremental computation can effectively reduce redundant operations in full computation, thereby improving computational efficiency. The system prioritizes incremental computation, calculating only the affected local regions, thus significantly reducing redundant computation overhead. Through this selection mechanism, the system can automatically adapt the optimal computation strategy in different update scenarios, improving overall processing efficiency.

[0033] Vertex proportion affected by update operation The calculation formula is: ; In the formula, denoted as , where is the number of affected vertices in the subgraph, and n is the total number of vertices in the subgraph.

[0034] when When the weight is greater than the preset weight, it indicates that more vertices in the subgraph are affected; when When the weight is less than or equal to the preset weight, it means that not many vertices in the subgraph are affected. The preset weight can be set to 50%.

[0035] like Figure 3 The diagram shown is a schematic representation of determining the affected vertices in an updated subgraph according to an embodiment of the present invention. Figure 1 Add operation ,son Figure 2 Add operation , in the child Figure 1 In the middle, the vertices directly affected are ,vertex The affected vertices after taking one step forward are After taking another step forward, the affected vertex is Therefore, Figure 1 The number of affected vertices is 4; similarly, the number of sub-vertices is 4. Figure 2 The affected vertices are The number of affected vertices is 1. Assuming a preset weight of 50%, the number of sub-vertexes can be calculated. Figure 1The affected vertices account for 80%, therefore the full calculation method is chosen; sub Figure 2 The affected vertex ratio is 20%, so the incremental calculation method is chosen.

[0036] Alternatively, a formula can be used to quickly estimate the number of affected vertices in the subgraph; If the flow graph algorithm used is a computationally structure-related algorithm, such as breadth-first search, shortest path search, or connected component calculation, the number of affected vertices in the subgraph can be predicted using the following formula: ; In the formula, Let d be the set of vertices reached after d-1 steps of propagation from the self-updated edges. Belongs to set The vertex in This represents the degree measure of the outgoing edge direction from vertex u. The propagation weights are used to estimate the proportion of vertices affected in each round of propagation. This indicates the set propagation depth, typically 2 to 3. Figure 3 sub Figure 1 For example, suppose At this time, the set include The degree measures of its outgoing directions are 3 and 1 respectively. include The degree measures of their outgoing directions are 2 and 1 respectively, which can be calculated using the above formula. .

[0037] If the flow graph algorithm used is one with numerical propagation characteristics, such as the Pagrerank algorithm or the Louvain algorithm, the number of affected vertices in the subgraph can be predicted using the following formula: ; In the formula, U is the set of vertices directly affected by the update operation. Let U be the number of vertices in set U. To propagate weight, To update the average degree of vertices in a subgraph, we need to update the total degree of all vertices in the subgraph divided by the number of vertices.

[0038] S4. After the graph is updated, extract the dependencies between subgraphs. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph.

[0039] The main objective of this stage is to construct the dependency relationships between subgraphs based on the dependency characteristics of specific graph algorithms. These dependency relationships will serve as the basis for subsequent subgraph scheduling.

[0040] Specifically, the dependencies between vertices can be determined based on the dependencies between them. When a vertex v in subgraph SG1 depends on a vertex u in subgraph SG2 during computation, SG1 is considered to depend on SG2, denoted as SG1→SG2. For example, when the result of vertex v is determined by vertex u, it means that vertex v depends on u, denoted as... Determine the dependencies between subgraphs. If vertex v in subgraph SG1 depends on vertex u in subgraph SG2, then subgraph SG1 depends on subgraph SG2, denoted as: when... Furthermore, to eliminate dependencies between subgraphs, vertices causing dependencies can be grouped into the same subgraph. For example, if V2 affects V3, V3 affects V4, and V4 affects V2, then V2, V3, and V4 are considered mutually dependent vertices. By traversing all subgraphs and their internal vertices' dependency information, a directed dependency graph composed of subgraphs can be obtained. In subsequent operations, the system performs a topological sort on this dependency graph. Subgraphs without prior dependencies can immediately perform graph computation, while subgraphs with prior dependencies must wait for their preceding subgraphs to complete their computation before performing their own.

[0041] like Figure 4 The diagram shown illustrates the construction of a directed dependency graph in one embodiment of the present invention. Taking a traffic network graph as an example, the graph computation algorithm used is the single-source shortest path algorithm. Vertices represent stations, and edges between vertices represent traffic routes between stations. The weight of an edge is the length of the traffic path between two points. The value of each vertex is the shortest path length from the source vertex of the original flow graph to the corresponding vertex before the graph update. Figure 4 There are two add operations, for Before the update, source to The distance is 4. Through subgraph partitioning, the original flow graph is divided into three subgraphs: sub... Figure 1 ,son Figure 2 ,son Figure 3 Among them, sub Figure 2 Mid-vertex The result is from the sub Figure 1 Mid-vertex Decision, son Figure 3 Mid-vertex Yuko Figure 2 Mid-vertex Decision, therefore Figure 2 Dependency Figure 1 ,son Figure 3 Dependency Figure 2 , that is, son Figure 1 For son Figure 2 The preceding subgraph, sub Figure 3 For son Figure 2 The preceding subgraph can be used to construct a subgraph dependency graph.

[0042] Furthermore, flow graphs can also be social network flow graphs. In a social network flow graph, vertices represent users, edges between vertices represent social relationships between users, and edge weights characterize the strength of these relationships; the stronger the relationship, the greater the edge weight. Taking content platforms like TikTok as an example, the following relationships between users can be abstracted into a directed social network graph, where users represent vertices and following relationships represent edges. The edge weights can be determined by a combination of factors, such as follower level, interaction frequency, likes, and comments, all of which reflect the strength of the relationship between users. Building on this, graph algorithms (such as PageRank and its variants) characterize the impact of certain users' behaviors or preferences on their followers and potential follower groups through weighted propagation on the graph structure. For example, when a celebrity shows a clear preference for a particular content or product, this preference may propagate along social relationships in a decaying manner, thus influencing the interests of their followers and even a wider range of users.

[0043] S5. Using the matched graph computation method, perform graph computations in parallel on each updated subgraph to obtain intermediate results for each subgraph.

[0044] The goal of this stage is to perform parallel computation of intermediate results within each executable subgraph. The computation within a subgraph can be regarded as a local task independent of other subgraphs. Therefore, each subgraph can perform its own graph computation in parallel to obtain the intermediate results of each subgraph. That is, the intermediate results of a subgraph are local results that only consider the subgraph itself and do not take into account the influence of other subgraphs.

[0045] For example, the intermediate results of each subgraph reflect the best path for other vertices within the subgraph relative to the reference vertex within the subgraph, which is the vertex that influences other vertices within the subgraph but is not influenced by them.

[0046] like Figure 5 The diagram shown illustrates graph computation performed by different subgraphs in one embodiment of the present invention, wherein the subgraphs... Figure 1 China has added The edge, child Figure 3 Added The edge, child Figure 1 Kazuko Figure 3 To update the subgraph, Figure 2 Not updated, therefore, only the sub-items need to be updated. Figure 1 , 3 Parallel graph computation is performed to obtain updated intermediate results, sub- Figure 2 Keeping the data from the original flow graph as an intermediate result, the sub-flow graph can be calculated based on the update. Figure 1 Affected vertices are ,son Figure 3Affected vertices are It should be noted that, although point to ,but Compared to the baseline vertex The shortest path has not changed, therefore Unaffected by update operations, therefore, child Figure 1 Kazuko Figure 3 Incremental calculation is used for all calculations. First, incremental calculation is performed. The shortest distance is 1, recorded as (Before the update) ), The shortest distance is 2, recorded as 2 (Before the update) 3) Based on the results before the update, we can then know that... , , The distances are 1, 2, 1, 1, 2 respectively; since these vertices are not affected by the update in the subgraph, the relative distances between the vertices do not change, and these results remain as intermediate results.

[0047] S6. Based on the dependencies between subgraphs, the intermediate results of each subgraph are passed and updated. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate results of the current subgraph are updated. The passing and updating of the intermediate results of the subgraphs are executed sequentially, and the global processing result of the flow graph is output.

[0048] The main goal of this stage is to merge the final results based on the relationships between the subgraphs after completing the local computations of all subgraphs.

[0049] Specifically, when the computation of subgraph SG1 depends on intermediate results from SG2, the system will pass the messages, states, or aggregate values ​​generated by SG2 to SG1 after SG2 completes its computation. As scheduling progresses and dependencies are satisfied, the system will trigger the passing and updating of subsequent subgraphs in topological order until the dependency chains of all subgraphs converge completely. Finally, when all subgraphs in the dependency graph have completed their computation, the final global result can be obtained.

[0050] like Figure 6 The diagram shown illustrates the transfer and updating of intermediate results for each subgraph in one embodiment of the present invention. As can be seen from the dependency graph, initially only the subgraph... Figure 1 It can be calculated, where the vertices The result is 1. When the child Figure 1 After the update is complete, the child Figure 2 It can perform calculations, because of the sub Figure 2 The calculation results of the interior vertices depend on ,Will The value is passed to the child Figure 2 ,son Figure 2 The reference vertex in The value is updated to 2, child Figure 2 The relative distances of other vertices to the base vertex (i.e., intermediate results) are known; by superimposing these distances, the other vertices can be obtained. The results were 2, 3, 4, 3 respectively; when the sub- Figure 2 After the calculation is completed, the sub Figure 3 It can perform calculations, because of the sub Figure 3 The calculation results of the interior vertices depend on ,Will The value is passed to the child Figure 3 , The result is updated to 4, 5, 6, 6, which is the final updated result. The value of each vertex in the final result reflects the shortest path length of each vertex from the source vertex in the original flow graph after the update.

[0051] Overall, the flow graph processing method proposed in this invention first divides the flow graph into multiple subgraphs of approximately the same size. Then, each subgraph performs an update operation based on its update information and counts the proportion of affected vertices within the subgraph. When the affected range exceeds a set threshold, it indicates that many vertices in the subgraph need to be recalculated, so a full calculation method is adopted. When the affected range is below the threshold, it indicates that fewer vertices are affected, so an incremental calculation method is adopted. Afterward, a dependency graph is constructed based on the dependencies between subgraphs. When the calculation result of a vertex in a certain subgraph depends on the calculation result of other subgraphs, it indicates that the subgraph depends on the corresponding other subgraphs. Since the subgraphs are independent of each other, each subgraph can first complete its internal calculation independently and generate the corresponding intermediate results. Finally, scheduling is performed according to the dependency graph: all dependent subgraphs are executed first, and subsequent dependent subgraphs are triggered sequentially according to the topological sort. When all subgraphs in the dependency graph have been updated, the final global result can be obtained. Compared to traditional single incremental or full computation methods, subgraph hybrid computation and parallel processing can make full use of the locality of graphs, improve computational parallelism and resource utilization efficiency, while ensuring the accuracy and consistency of computation results.

[0052] Secondly, the present invention also relates to a flow graph processing system based on subgraph hybrid computation.

[0053] like Figure 7 The diagram shown is an architecture diagram of a flow graph processing system based on subgraph hybrid computation according to an embodiment of the present invention, which includes a graph partitioning module, a graph update module, a computation method selection module, a dependency relationship construction module, an intra-graph computation module, and an inter-graph scheduling module.

[0054] The graph partitioning module, in response to graph processing requests, divides the original flow graph in the graph database into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent a relationship path between the two entities. The shorter the relationship path, the smaller the weight of the edge.

[0055] The graph update module is used to perform graph update operations on subgraphs, including deleting and adding edges. The calculation method selection module is used to match the corresponding graph calculation method for each updated subgraph. The graph calculation methods include full calculation method and incremental calculation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full calculation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental calculation method.

[0056] The dependency building module is used to extract the dependencies between subgraphs after a graph update. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph.

[0057] The graph computation module is used to perform graph computations on each updated subgraph in parallel using the matched graph computation method, and obtain the intermediate results of each subgraph.

[0058] The inter-graph scheduling module is used to pass and update intermediate results of each subgraph based on the dependencies between subgraphs. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate result of the current subgraph is updated. The passing and updating of intermediate results of subgraphs are executed sequentially, and the global processing result of the flow graph is output.

[0059] Specifically, the above processing system can implement the flow graph processing method described above. For details, please refer to the previous description, which will not be repeated here.

[0060] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. It should be noted that the terms "in one embodiment," "for example," and "again" are intended to illustrate the present invention and are not intended to limit the present invention.

[0061] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A flow graph processing method based on subgraph hybrid computation, characterized in that, include: In response to the graph processing request, the original flow graph in the graph database is divided into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent the relationship or traffic path between the two entities. The closer the relationship or the longer the traffic path, the greater the weight of the edge. Perform graph update operations on the subgraph, including deleting and adding edges; For each updated subgraph, a corresponding graph computation method is matched. The graph computation method includes full computation method and incremental computation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full computation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental computation method. After the graph is updated, the dependencies between subgraphs are extracted. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph. The matching graph computation method is used to perform graph computations in parallel on each updated subgraph, and intermediate results are obtained for each subgraph. Based on the dependencies between subgraphs, the intermediate results of each subgraph are passed and updated. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate results of the current subgraph are updated. The passing and updating of the intermediate results of the subgraphs are executed sequentially, and the global processing result of the flow graph is output.

2. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, The original flow graph in the graph database is divided into multiple subgraphs with a balanced distribution of vertex total weights, including: First, divide the original flow graph into multiple subgraphs; Then, the subgraph partitioning is fine-tuned according to the weight balancing mechanism; the weight balancing mechanism includes: calculating the weight of each vertex in its subgraph, identifying high-weight subgraphs and low-weight subgraphs according to the total weight of the vertices in the subgraph, assigning some vertices in the high-weight subgraphs to other subgraphs with connectivity, and merging the low-weight subgraphs.

3. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, Weight of any vertex v in its subgraph The calculation formula is: ； In the formula, This represents the edge connecting vertex u and vertex v. Representing an edge The weights are such that vertices u and v belong to the same subgraph.

4. The flow graph processing method based on subgraph hybrid computation as described in claim 2 or 3, characterized in that, Dividing some vertices in a high-weight subgraph into other subgraphs with connectivity includes: calculating the degree ratio of each vertex in the high-weight subgraph. The vertices v with the smallest degree ratio are assigned to other subgraphs with the smallest total weight of vertices connected to them. Wherein, the degree ratio of any vertex v The calculation formula is: ； In the formula, Let v be the degree by which vertex v is connected to all other vertices within its subgraph. Let v be the degree by which vertex v is connected to other vertices outside its subgraph.

5. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, When partitioning a subgraph, vertices with interdependent relationships are partitioned into the same subgraph.

6. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, The default weight is 50%. When the weight of vertices affected by the update operation in the updated subgraph exceeds 50%, the corresponding updated subgraph matches the full calculation method; otherwise, the corresponding updated subgraph matches the incremental calculation method.

7. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, When performing graph update operations on a subgraph, each edge to be updated is recorded as a triple [operator, source, destination]. The operator represents the type of update operation, which is divided into two types: addition and deletion. The source represents the source vertex of the updated edge, and the destination represents the target vertex of the updated edge. The source vertex points to the target vertex. The update operation corresponding to the source is assigned to update the subgraph to which the source belongs.

8. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, The intermediate results within each subgraph are the optimal paths of other vertices within the subgraph relative to the reference vertex within the subgraph. The reference vertex within the subgraph is the vertex that influences other vertices in the subgraph but is not influenced by other vertices in the subgraph.

9. The flow graph processing method based on subgraph hybrid computation as described in claim 1, characterized in that, The original flow graph is a social network flow graph or a traffic network flow graph; In a traffic network flow graph, vertices represent stations, edges between vertices represent traffic paths between two stations, and the weight of an edge represents the length of the traffic path; the shorter the traffic path, the smaller the weight of the edge. In a social network flow graph, vertices represent users, edges between vertices represent social relationships between users, and edge weights are used to characterize the tightness of social relationships between users; the tighter the social relationship, the greater the weight of the corresponding edge.

10. A flow graph processing system based on subgraph hybrid computation, characterized in that, include: The graph partitioning module is used to respond to graph processing requests and divide the original flow graph in the graph database into multiple subgraphs with a balanced distribution of vertex weights. The weight of any vertex in its subgraph is proportional to the sum of the weights of the edges connecting that vertex to other vertices in the same subgraph. The original flow graph is an information relationship graph, which has vertices and edges connecting the vertices. Vertices represent entities participating in the analysis, and the edges between two vertices represent the relationship or traffic path between the two entities. The closer the relationship or the longer the traffic path, the greater the weight of the edge. The graph update module is used to perform graph update operations on subgraphs, including deleting and adding edges. The calculation method selection module is used to match the corresponding graph calculation method for each updated subgraph. The graph calculation methods include full calculation method and incremental calculation method. When the proportion of vertices affected by the update operation in the updated subgraph exceeds the preset proportion, the corresponding updated subgraph is matched with the full calculation method. When the proportion of vertices affected by the update operation in the updated subgraph does not exceed the preset proportion, the corresponding updated subgraph is matched with the incremental calculation method. The dependency construction module is used to extract the dependencies between subgraphs after the graph is updated. If the current subgraph has vertices that are affected by another subgraph, it means that the current subgraph depends on the other subgraph, and the other subgraph is the predecessor subgraph of the current subgraph. The graph computation module is used to perform graph computations on each updated subgraph in parallel using the matched graph computation method, and obtain the intermediate results of each subgraph. The inter-graph scheduling module is used to pass and update intermediate results of each subgraph based on the dependencies between subgraphs. After the predecessor subgraph of the current subgraph has completed the passing and updating, the updated result of the predecessor subgraph is passed to the current subgraph and the intermediate result of the current subgraph is updated. The passing and updating of intermediate results of subgraphs are executed sequentially, and the global processing result of the flow graph is output.

Images