Processing method and apparatus for a graph
By determining the threshold for the number of received increments and the traversal order when the graph changes, invalid calculations are reduced, redundancy problems in the graph update process are solved, and update efficiency is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2022-06-15
- Publication Date
- 2026-04-17
AI Technical Summary
When the graph changes, the update process of vertex states in related technologies involves invalid and redundant computational overhead, which affects update efficiency.
By determining a threshold for the number of incremental visits for each affected vertex, the traversal process is executed until the number of visits reaches the threshold. The edges are arranged in the traversal order for state propagation, the root vertex is selected, and the traversal stops when the condition is met, thus reducing invalid computation.
This reduces invalid and redundant calculations during vertex state update, thus improving update efficiency.
Smart Images

Figure CN117271834B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for processing graphs. Background Technology
[0002] A graph is a non-linear data structure that can simply and intuitively represent the relationships between any two vertices. Transportation networks, social networks, and the state execution of computers can all be represented using graphs. A dynamic graph refers to a graph whose structure changes over time. During dynamic graph analysis, when the graph changes, the states of the affected vertices need to be updated.
[0003] In related technologies, after a graph changes, the initial vertices directly affected by the change are first identified, and the state of each initial vertex is updated. Then, each initial vertex generates an increment and propagates it along the graph's topology. During this propagation, each vertex that receives an increment updates its own state based on the increment and generates a new increment to continue propagating along the graph's topology.
[0004] In the process of realizing this invention, the inventors discovered that the related technology has at least the following problems: during the propagation of the related technology, the propagation of the increment generated by each vertex is irregular, which leads to invalid and redundant computational overhead in the process of updating vertex state, thereby affecting the update efficiency. Summary of the Invention
[0005] This application provides a graph processing method and apparatus that can reduce invalid and redundant calculations during the vertex state update process and improve update efficiency.
[0006] In a first aspect, this application provides a graph processing method, the method comprising: when a graph changes, determining a threshold for the number of received increments for each of a plurality of affected vertices during state propagation, wherein the affected vertices are those whose vertex states need to be updated when the graph changes; performing at least one traversal process based on the threshold for the number of received increments for each affected vertex until the number of visits to each affected vertex reaches its respective threshold for the number of received increments, thereby obtaining a plurality of edges arranged in traversal order; and performing state propagation based on the plurality of edges arranged in traversal order; wherein the traversal process comprises: determining a root vertex from at least one active vertex among the plurality of affected vertices; and traversing a state propagation path starting from the root vertex. The traversal of the current state propagation path is stopped when an affected vertex satisfying the first stopping condition is reached, until all state propagation paths starting from the root vertex have been traversed; wherein, the at least one active vertex includes: an initial vertex that has never been a root vertex and an affected vertex that has never been a root vertex and satisfies the first stopping condition, wherein the difference between the received increment threshold and the number of visits of the root vertex is the smallest among the at least one active vertex, and the initial vertex is the vertex directly affected when the graph changes; the first stopping condition includes: when the number of visits of the affected vertex is less than the received increment threshold when the affected vertex is reached, and / or when the affected vertex is reached, the affected vertex has no accessible neighbor affected vertices.
[0007] A change in a graph refers to a change in its structure. Changes in graph structure include at least one of the following: adding edges, deleting edges, updating edges, adding vertices, deleting vertices, and updating vertices.
[0008] Vertex state has different meanings in different graph computation algorithms. For example, in the PageRank algorithm, the vertex state of any vertex is the current PageRank value of the vertex; in the Single Source Shortest Path (SSSP) algorithm, the vertex state of any vertex is the length of the shortest path between the source vertex and any vertex.
[0009] When a graph changes, the initial vertex is determined first. Then, the state propagation path starting from the initial vertex is traversed to determine all affected vertices and the threshold for the number of increments received by each affected vertex. The final list of affected vertices includes the initial vertex and all vertices reachable from it. The initial vertex is the first affected vertex whose state needs to be updated and whose increments generated when the graph changes.
[0010] During the traversal, reaching any affected vertex from other affected vertices along the state propagation path is considered as visiting that affected vertex once. The same affected vertex may be visited multiple times, and the same edge may also be visited multiple times.
[0011] During the first traversal, the root vertex is determined from the initial vertices. In subsequent traversals, the root vertex is determined from the initial vertices that have never been used as the root vertex and from the vertices that have never been used as the root vertex and satisfy the second stopping condition.
[0012] The root vertex determined by the graph processing device is the active vertex with the smallest difference between the number of received increments threshold and the number of times it has been visited among at least one active vertex, and there exists a state propagation path starting from the root vertex that has not been traversed. During the traversal, after visiting an affected vertex that satisfies the first stopping condition, that affected vertex is set as an active vertex. After each root vertex is determined, it is set as an inactive vertex.
[0013] The state propagation path starting from the root vertex is all the paths that the initial vertex takes during state propagation, that is, the paths between all endpoints that can be reached from the root vertex.
[0014] During the traversal, each time an affected vertex is visited, it is equivalent to the affected vertex receiving an increment during state propagation. The vertex with the smallest difference between the threshold for the number of received increments and the number of times it has been visited among at least one active vertex is designated as the root vertex. This is equivalent to designating the vertex with the fewest unreceived increments among at least one active vertex during state propagation as the root vertex.
[0015] The increment represents the change in vertex state. The meaning of the increment differs across algorithms. For example, in the PageRank algorithm, the increment represents the change in vertex state, while in the SSSP algorithm, the increment represents the sum of the updated vertex state and the weights of the edges that sent from the vertex.
[0016] The affected vertex receives the increment through the receiving edge. In this embodiment, the graph may include directed graphs and undirected graphs. When the graph is directed, the receiving edge of the affected vertex is the incoming edge, and the sending edge is the outgoing edge. When the graph is undirected, an edge connecting the affected vertex can be regarded as a sending edge and a receiving edge.
[0017] In the first stopping condition, the affected vertex has no accessible neighbors. The affected vertex can include two cases: one is that all the edges of the affected vertex to other neighboring affected vertices have been visited (i.e., all the sending edges of the affected vertex connected to neighboring affected vertices have been visited), and the other is that the affected vertex is not connected to any other neighboring affected vertices (i.e., the affected vertex does not have any sending edges connected to neighboring affected vertices).
[0018] During the traversal, the order in which each edge is visited can be recorded. For example, a queue can be created, and the edge identifier of each edge visited can be stored in the queue.
[0019] In the embodiments of this application, the graph can be a complete graph or a partial structure of an initial graph. For example, a graph processing device (e.g., graph processing system software running on a core) can configure a complete graph on a core for processing, or it can divide the initial graph into blocks, configuring each block on a core for processing.
[0020] In this method, after determining the threshold for the number of received increments, when the difference between the threshold for the number of received increments and the number of times a certain active vertex has been visited is the smallest among at least one active vertex, it is equivalent to the active vertex not receiving the fewest increments among at least one active vertex during subsequent state propagation. This active vertex is then designated as the root vertex for traversal. When an affected vertex is visited, if the number of times the affected vertex has been visited is less than its threshold for the number of received increments, it is equivalent to the vertex not receiving enough increments during subsequent state propagation, and therefore cannot continue to be visited. This ensures that an active vertex can only be designated as the root vertex when the difference between the threshold for the number of received increments and the number of times it has been visited is minimal, and it can only continue to visit its neighboring affected vertices when the difference between the threshold for the number of received increments and the number of times it has been visited is zero during traversal. When the state propagation process follows this traversal order, it starts with the active vertex that has received the fewest increments. The affected vertices that have received increments will only continue the state propagation after all increments have been accumulated. This normalizes the state propagation process, effectively reduces the number of updates for subsequently visited affected vertices, thereby reducing invalid and redundant calculations in the process of updating vertex states and improving update efficiency.
[0021] In one possible implementation, the state propagation based on the multiple edges arranged in the traversal order includes: selecting the edge that needs to be propagated through state propagation according to the arrangement order of the multiple edges; obtaining the vertex states of the affected vertices at both ends of the selected edge; and performing state propagation based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
[0022] Graph processing devices can determine the increment based on the weight of the selected edge and / or the vertex states of the affected vertices at both ends of the selected edge, and then propagate the increment along the selected edge.
[0023] In one possible implementation, the method further includes: determining a plurality of hot vertices from the graph, wherein the threshold for the number of received increments of the hot vertices is greater than or equal to a quantity threshold; storing the vertex states of the plurality of hot vertices in a contiguous storage area; and obtaining the vertex states of the affected vertices at both ends of a selected edge, including: when the affected vertex at either end of the selected edge is a hot vertex, obtaining the vertex state of the affected vertex at that end from the contiguous storage area; and when the affected vertex at either end is a non-hot vertex, obtaining the vertex state of the affected vertex at that end from a memory area.
[0024] Hot vertices are those that are frequently visited during state propagation, i.e., vertices with a high access frequency. The number threshold can be obtained by the access frequency of each vertex and a custom ratio α, where 0 ≤ α ≤ 1. In this embodiment, the threshold for the number of received increments of a vertex is approximated as the access frequency of the vertex, and the vertices whose access frequency ranks in the top α (including α) among all vertices are determined as hot vertices. That is, the minimum access frequency among the vertices whose access frequency ranks in the top α among all vertices is determined as the number threshold.
[0025] In graph computation, most state propagation processes only involve a small subset of vertices that are affected and need to update their states. That is, the percentage of vertices that need to update their states is small among all vertices, so the states of most vertices are not needed in the graph computation process.
[0026] In related technologies, the vertex states of each vertex are sparsely stored in main memory. During graph computation, a single cache line stores a small number of vertex states required for the computation, sometimes even only one. This leads to numerous random data accesses when retrieving vertex states, resulting in high data access overhead. In this embodiment, the vertex states of hot vertices are merged and stored in contiguous storage areas. This allows a single cache line to store a larger number of vertex states required for graph computation, meaning one cache line can handle most of the vertex state accesses during state propagation. Vertex state retrieval becomes continuous and regular, thereby reducing data access overhead.
[0027] In one possible implementation, determining the threshold for the number of received increments for each of the multiple affected vertices during state propagation includes: traversing the state propagation path starting from each initial vertex, stopping the traversal of the current state propagation path when an affected vertex satisfying a second stopping condition is reached, until all state propagation paths starting from each initial vertex have been traversed; determining the number of times the affected vertex is visited as the threshold for the number of received increments for the affected vertex; the second stopping condition includes: the affected vertex is another initial vertex, and / or the affected vertex has no accessible neighboring affected vertices when it is visited. In this case, edges on the state propagation path are visited only once, and common edges of different state propagation paths are also visited only once.
[0028] After updating the vertex state, the initial vertex needs to generate an increment for state propagation. The state propagation path starting from the initial vertex is all the paths that the initial vertex takes during state propagation, that is, the paths between all endpoints that can be reached from the initial vertex.
[0029] In one possible implementation, traversing the state propagation path starting from each initial vertex includes: traversing the edges on the state propagation path starting from the initial vertex, wherein the number of times the affected vertex is visited is the number of edges from the affected vertex's neighboring affected vertices to the affected vertex; traversing the state propagation path starting from the root vertex includes: traversing the edges on the state propagation path starting from the root vertex.
[0030] In this implementation, the number of times an affected vertex is visited is the number of edges from its neighboring affected vertices to that affected vertex, i.e., the number of receiving edges connecting the affected vertex to its neighboring affected vertices. Visiting an affected vertex once means that one of its receiving edges connecting it to its neighboring affected vertices is visited.
[0031] When traversing the state propagation path starting from each initial vertex and the edges on the state propagation path starting from the root vertex, it is necessary to ensure that the edges between any two affected vertices in the graph are visited after all traversals are completed. Therefore, a vertex may be visited multiple times during the traversal process.
[0032] In one possible implementation, traversing the edges of the state propagation path starting from the initial vertex includes: traversing the edges of the state propagation path starting from the initial vertex based on a traversal algorithm; traversing the edges of the state propagation path starting from the root vertex includes: traversing the edges of the state propagation path starting from the root vertex based on the traversal algorithm, wherein the traversal algorithm includes: depth-first traversal algorithm and / or breadth-first traversal algorithm.
[0033] Secondly, this application provides a graph processing apparatus, the apparatus comprising: a first determining module, configured to determine, when the graph changes, a threshold for the number of received increments for each of a plurality of affected vertices during state propagation, wherein the affected vertices are vertices whose states need to be updated when the graph changes; a traversal module, configured to perform at least one traversal process based on the threshold for the number of received increments for each affected vertex, until the number of visits to each affected vertex reaches its respective threshold for the number of received increments, thereby obtaining a plurality of edges arranged in traversal order; and a state propagation module, configured to perform state propagation based on the plurality of edges arranged in traversal order; the traversal process comprising: determining a root vertex from at least one active vertex among the plurality of affected vertices; traversing the... The state propagation path starting from the root vertex stops traversing the current state propagation path when it reaches an affected vertex that satisfies the first stopping condition, until all state propagation paths starting from the root vertex have been traversed; wherein, the at least one active vertex includes: an initial vertex that has never been a root vertex and an affected vertex that has never been a root vertex and satisfies the first stopping condition, wherein the difference between the received increment threshold and the number of visits of the root vertex is the smallest among the at least one active vertex, and the initial vertex is the vertex directly affected when the graph changes; the first stopping condition includes: when the number of visits of the affected vertex is less than the received increment threshold when it is reached, and / or when the affected vertex is reached, the affected vertex has no accessible neighbor affected vertices.
[0034] In one possible implementation, the state propagation module is specifically used to: select the edge that needs to be propagated through state according to the arrangement order of the multiple edges; obtain the vertex states of the affected vertices at both ends of the selected edge; and perform state propagation based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
[0035] In one possible implementation, the device further includes: a second determining module, configured to determine a plurality of hot vertices from the graph, wherein the threshold for the number of received increments of the hot vertices is greater than or equal to a quantity threshold; a storage module, configured to store the vertex states of the plurality of hot vertices in a contiguous storage area; and the state propagation module, specifically configured to: obtain the vertex state of the affected vertex at either end of a selected edge from the contiguous storage area when the affected vertex at either end is a hot vertex; and obtain the vertex state of the affected vertex at either end from a memory area when the affected vertex at either end is a non-hot vertex.
[0036] In one possible implementation, the first determining module is specifically configured to: traverse the state propagation path starting with each initial vertex, stop traversing the current state propagation path when an affected vertex satisfying a second stopping condition is reached, until all state propagation paths starting with each initial vertex have been traversed; determine the number of times the affected vertex has been visited as a threshold for the number of received increments of the affected vertex; the second stopping condition includes: the affected vertex is another initial vertex, and / or the affected vertex does not have any accessible neighboring affected vertices when it is visited.
[0037] In one possible implementation, the first determining module is specifically used to: traverse the edges on the state propagation path starting from the initial vertex, wherein the number of times the affected vertex is visited is the number of edges from the affected vertex's neighboring affected vertices to the affected vertex; the traversal module is specifically used to: traverse the edges on the state propagation path starting from the root vertex.
[0038] In one possible implementation, the first determining module is specifically used to: traverse the edges on the state propagation path starting from the initial vertex based on a traversal algorithm; the traversal module is specifically used to: traverse the edges on the state propagation path starting from the root vertex based on the traversal algorithm, wherein the traversal algorithm includes: depth-first traversal algorithm and / or breadth-first traversal algorithm.
[0039] Thirdly, this application provides a graph processing apparatus, the apparatus comprising: one or more processors; a memory for storing one or more computer programs or instructions; wherein when the one or more computer programs or instructions are executed by the one or more processors, the one or more processors perform the method as described in any one of the first aspects.
[0040] Fourthly, this application provides a graph processing apparatus, including a processor for performing the method as described in any one of the first aspects.
[0041] Fifthly, this application provides a computer-readable storage medium including a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of the first aspects. Attached Figure Description
[0042] Figure 1 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;
[0043] Figure 2 This is a partial structural schematic diagram of an image processing device provided in an embodiment of this application;
[0044] Figure 3 A schematic flowchart illustrating a method for processing a graph provided in an embodiment of this application;
[0045] Figure 4 A schematic flowchart illustrating another method for processing diagrams provided in an embodiment of this application;
[0046] Figure 5 This is a schematic diagram of a directed graph provided in an embodiment of this application;
[0047] Figure 6 This is a schematic diagram of an undirected graph provided in an embodiment of this application;
[0048] Figure 7 This is a schematic diagram of the structure of a TDTU provided in an embodiment of this application;
[0049] Figure 8 This is a schematic diagram of configuration information provided in an embodiment of this application;
[0050] Figure 9 This is a schematic diagram of a process for determining the threshold number of received increments for each affected vertex by traversing, as provided in an embodiment of this application.
[0051] Figure 10 A flowchart illustrating a process of performing at least one traversal based on a threshold for the number of received increments for each affected vertex, provided in an embodiment of this application;
[0052] Figure 11 This application provides a schematic diagram of vertex state storage for a hot vertex.
[0053] Figure 12 A block diagram of a diagram processing apparatus provided in an embodiment of this application;
[0054] Figure 13 A block diagram of another diagram processing apparatus provided in an embodiment of this application;
[0055] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0056] Figure 15 This is a schematic diagram of a diagram processing device provided in an embodiment of this application. Detailed Implementation
[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0058] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0059] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, "first range" and "second range," etc., are used to distinguish different ranges, rather than to describe a specific order of ranges.
[0060] In the embodiments of this application, the words "in one example," "examplely," or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "in one example," "examplely," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "in one example," "examplely," or "for example" is intended to present the relevant concepts in a specific manner.
[0061] In the description of the embodiments in this application, unless otherwise stated, "at least one" means one or more, and "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.
[0062] A graph is a non-linear data structure consisting of edges connecting vertices, with each edge connecting two vertices. Graphs can be classified into directed graphs and undirected graphs based on whether the edges have a direction. In directed graphs, data needs to be transmitted between vertices according to the direction of the edges, while in undirected graphs, data can be transmitted bidirectionally between vertices.
[0063] A dynamic graph is a graph whose structure changes over time. A change in the graph means a change in its structure, which includes at least one of the following: adding edges, deleting edges, updating edges, adding vertices, deleting vertices, and updating vertices. In dynamic graph analysis, when the graph changes, graph computation is needed to update the vertex state of each affected vertex. First, an initial vertex is determined, and its state is updated first. Then, the initial vertex needs to propagate its state to update the states of other affected vertices. The vertex state has different meanings in different graph computation algorithms. For example, in the PageRank algorithm, the vertex state of any vertex is its current PageRank value; in the SSSP algorithm, the vertex state of any vertex is the length of the shortest path between the source vertex and that vertex. The initial vertex is the directly affected vertex when the graph changes; that is, it is the first vertex whose state needs to be updated and whose incremental affected vertex is generated when the graph changes.
[0064] In the PageRank algorithm, if an edge needs to be added, the source vertex of the new edge first sends its previous convergent inverse value to its connected neighboring vertices through the changed sending edge, while simultaneously deleting all sending edges from the source vertex. Then, all actual sending edges (including the new edge) are added back, an increment is generated, and the increment is sent to the neighboring vertices connected to the source vertex's sending edge. The vertex receiving the increment is designated as the initial vertex, updates its vertex state, and begins the state propagation process. If an edge needs to be deleted, the source vertex of the deleted edge generates a negative increment and sends it to its connected neighboring vertices through the changed sending edge, while simultaneously deleting all sending edges from the source vertex. Then, all actual sending edges are added back, an increment is generated, and the increment is sent to the neighboring vertices connected to the source vertex's sending edge. The vertex receiving the increment is designated as the initial vertex, updates its vertex state, and begins the state propagation process.
[0065] In the SSSP algorithm, if an edge needs to be added, the destination vertex of the added edge is determined as the initial vertex. The initial vertex updates its vertex state and begins the state propagation process. If an edge is deleted, the vertex states of all affected vertices are first reset to their initial values, and each affected vertex receives the states of its neighbors through receiving edges. Then, the destination vertex of the deleted edge is determined as the initial vertex, the initial vertex updates its vertex state, and begins the state propagation process.
[0066] Adding a vertex can be considered as adding an edge connecting that vertex, and deleting a vertex can be considered as deleting an edge connecting that vertex. Therefore, the processes of adding and deleting vertices can be referred to the graph computation processes for adding and deleting edges, respectively. When updating an edge, the destination vertex of the updated edge is determined as the initial vertex, the initial vertex updates its state, and the state propagation process begins. When updating a vertex, the vertex to be updated is directly determined as the initial vertex, the initial vertex updates its state, and the state propagation process begins.
[0067] In this graph computation algorithm, a vertex's sending edge represents the edge used to send data, and a vertex's receiving edge represents the edge used to receive data. One edge of a vertex is used to send and / or receive data. For a directed graph, the incoming edges connecting vertices are the receiving edges, the outgoing edges are the sending edges, and the vertex pointed to by the arrow on the edge is the destination vertex. For an undirected graph, the edges connecting vertices are considered as two edges: a sending edge and a receiving edge. The vertex at either end of the edge can be considered the source vertex and the destination vertex.
[0068] Currently, state propagation is typically implemented using increments, where the data propagated during state propagation is an increment representing the change in vertex state. The meaning of the increment varies across different algorithms. For example, in the PageRank algorithm, the increment represents the change in vertex state value, while in the SSSP algorithm, the increment represents the sum of the updated vertex state value and the weights of the edges sent by the vertex. During state propagation, for any affected vertex, updating its state includes: obtaining the vertex state; updating the affected vertex state based on the current vertex state and the increment received via the receiving edge; generating a new increment after the update; obtaining graph structure data; determining neighboring vertices based on the graph structure data; and sending the new increment to neighboring vertices via the sending edge. The graph structure data includes the set of edges of the current vertex in the graph.
[0069] In related technologies, each initial vertex, after updating its vertex state, generates an increment and propagates it along the graph's topology. During this propagation process, each vertex that receives the increment updates its own vertex state based on the increment and generates a new increment to continue propagating along the graph's topology.
[0070] However, the propagation of increments generated by each vertex during this process is irregular. The increments generated by different vertices will propagate along different state propagation paths. Therefore, the state propagation of different vertices may reach the same vertex at different times. This means that the same vertex needs to update its state multiple times, and the graph structure data associated with the same vertex also needs to be acquired multiple times. This results in invalid and redundant computational overhead during the vertex state update process, thus affecting the update efficiency.
[0071] This application provides a graph processing method that reduces invalid and redundant computational overhead during vertex state update, thereby improving vertex state update efficiency. This method can be applied to graph processing devices requiring high-performance graph processing to achieve efficient data analysis, such as anomaly detection and social network analysis. The graph processing device may include, but is not limited to, computer devices (e.g., servers, personal computers, and handheld devices such as mobile phones and tablets).
[0072] Please refer to Figure 1 , Figure 1This is a schematic diagram of a graph processing device provided in an embodiment of this application. The graph processing device includes multiple cores, a Last Level Cache (LLC), main memory, L1 caches corresponding to each core, L2 caches corresponding to each core, and Topology Driven Graph (TDGraph) modules corresponding to each core. Each core is coupled to its corresponding TDGraph module, and the TDGraph module corresponding to any core is located between that core and its corresponding L2 cache. The LLC, main memory, and L2 cache can all be used to store graph data.
[0073] It should be noted that, Figure 1 This explanation uses the example of a TDGraph module being a hardware module coupled to the core. Optionally, the TDGraph module can also be a software module running on the core, or it can consist of a software module running on the core and a hardware module coupled to the core. The hardware module can include, but is not limited to, Field Programmable Gate Array (FPGA) circuits and / or Application Specific Integrated Circuit (ASIC) circuits.
[0074] Please refer to Figure 2 , Figure 2 This is a partial structural schematic diagram of an image processing device provided in an embodiment of this application. Figure 2 The structure of the TDGraph module corresponding to a single core will be explained using this example. Figure 2 As shown, the TDGraph module includes a lightweight hardware unit: the Topology-Driven Traversing Unit (TDTU). Optionally, the TDGraph module may also include a lightweight hardware unit connected to the TDTU: the VertexStates Coalescing Unit (VSCU).
[0075] It should be noted that, Figure 1 and Figure 2 The structure shown is merely illustrative, and the embodiments of this application do not limit the number of modules in the graph processing device or the number of each module.
[0076] Please refer to Figure 3 , Figure 3This is a flowchart illustrating a graph processing method provided in an embodiment of this application. The method describes the graph processing procedure when the graph changes during dynamic graph analysis. This method can be applied to graph processing devices, such as… Figure 1 and Figure 2 The image processing device shown. Specifically, this method can be applied to... Figure 1 and Figure 2 The TDGraph module shown, when graph processing is required, calls the corresponding TDGraph module of the core to execute the following method procedure. This embodiment uses the TDGraph module corresponding to one core as an example to illustrate this method; the execution process of the TDGraph modules corresponding to other cores can refer to the following procedure, such as... Figure 3 As shown, the method may include the following procedures:
[0077] 101. When the graph changes, determine the threshold for the number of received increments for each of the multiple affected vertices during state propagation. The affected vertices are those whose vertex states need to be updated when the graph changes.
[0078] 102. Based on the threshold number of received increments for each affected vertex, perform at least one traversal process until the number of visits to each affected vertex reaches its respective threshold number of received increments, resulting in multiple edges arranged in traversal order. The traversal process includes: determining the root vertex from at least one active vertex among the multiple affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex that satisfies the first stopping condition is visited, until all state propagation paths starting from the root vertex have been traversed.
[0079] During this traversal, reaching any affected vertex from other affected vertices along the state propagation path is considered as visiting that affected vertex once. The same affected vertex may be visited multiple times, and the same edge may also be visited multiple times.
[0080] The first stopping conditions include: when the number of times the affected vertex has been visited is less than the threshold of the number of received increments, and / or when the affected vertex has no accessible neighboring affected vertices.
[0081] The state propagation process for 103 follows the traversal order of edges in the previous traversal. Therefore, the traversal process for 102 is equivalent to a simulated state propagation process. The root vertex is equivalent to the vertex that propagates the increment to subsequent affected vertices during state propagation. Therefore, active vertices should be those that can propagate the increment to subsequent affected vertices during state propagation. At the beginning of the state propagation process, the increment propagates from the initial vertex to subsequent affected vertices. Other affected vertices have not yet generated increments and cannot propagate state. Therefore, during the first traversal, active vertices only include the initial vertex. The traversal of the current state propagation path stops after visiting a vertex that meets the first stopping condition. Vertices that meet the first stopping condition are equivalent to vertices that have received some increments during state propagation. They may also propagate increments to subsequent affected vertices during state propagation. Therefore, vertices that meet the first stopping condition can also be set as active vertices.
[0082] The root vertex determined by different traversal processes is different. Therefore, based on the above description, at least one active vertex includes: the initial vertex that has never been a root vertex and the affected vertex that has never been a root vertex and satisfies the first stopping condition.
[0083] The root vertex determined by the graph processing device is the active vertex with the smallest difference between the number of received increments threshold and the number of times it has been visited among at least one active vertex, and there exists a state propagation path starting from the root vertex that has not been traversed. During the traversal, after visiting an affected vertex that satisfies the first stopping condition, that affected vertex is set as an active vertex. After each root vertex is determined, it is set as an inactive vertex.
[0084] During the traversal, each time an affected vertex is visited, it is equivalent to the affected vertex receiving an increment during state propagation. The vertex with the smallest difference between the threshold for the number of received increments and the number of times it has been visited among at least one active vertex is designated as the root vertex. This is equivalent to designating the vertex with the fewest unreceived increments among at least one active vertex during state propagation as the root vertex.
[0085] 103. Propagate state based on multiple edges arranged in traversal order.
[0086] The graph processing device can select the edge that needs to be propagated through state according to the arrangement order of multiple edges, obtain the vertex states of the affected vertices at both ends of the selected edge, and then perform state propagation based on the selected edge and the vertex states of the affected vertices at both ends of the edge.
[0087] In summary, the graph processing method provided in this application, when the graph changes, determines a threshold for the number of received increments for each of the multiple affected vertices during state propagation. Then, based on the threshold for the number of received increments for each affected vertex, at least one traversal process is performed until the number of visits to each affected vertex reaches its respective threshold for the number of received increments, resulting in multiple edges arranged in traversal order. State propagation is then performed based on these edges arranged in traversal order. The traversal process includes: determining a root vertex from at least one active vertex among the multiple affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex satisfying a first stopping condition is visited, until all state propagation paths starting from the root vertex have been traversed. After determining the threshold for the number of received increments, when the difference between the threshold for the number of received increments and the number of visits to an active vertex is the smallest among at least one active vertex, it is equivalent to the active vertex not receiving the fewest increments among at least one active vertex in subsequent state propagation processes, and this active vertex is then used as the root vertex to begin traversal. When a visited affected vertex is reached, if the number of times that affected vertex has been visited is less than its received increment threshold, it means that the vertex has not received enough increments in the subsequent state propagation process, and therefore cannot continue to be visited. This ensures that an active vertex can only become the root vertex when the difference between the received increment threshold and the number of times it has been visited is minimized, and it can only continue to visit its neighboring affected vertices when the difference between the received increment threshold and the number of times it has been visited is zero. When the state propagation process follows this traversal order, it starts with the active vertex that has received the fewest increments, and affected vertices that have received increments only continue state propagation after accumulating all increments. This normalizes state propagation, effectively reduces the number of updates for subsequently visited affected vertices, thereby reducing invalid and redundant calculations in the vertex state update process and improving update efficiency.
[0088] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating another graph processing method provided in an embodiment of this application. The method describes the graph processing procedure when the graph changes during dynamic graph analysis. This method can be applied to graph processing devices, such as… Figure 1 and Figure 2 The image processing device shown. Specifically, this method can be applied to... Figure 1 and Figure 2 The TDGraph module shown, when graph processing is required, calls the corresponding TDGraph module of the core to execute the following method procedure. This embodiment uses the TDGraph module corresponding to one core as an example to illustrate this method; the execution process of the TDGraph modules corresponding to other cores can refer to the following procedure, such as... Figure 4 As shown, the method may include the following procedures:
[0089] 201. When the graph changes, determine the threshold for the number of received increments for each of the multiple affected vertices during state propagation. The affected vertices are those whose vertex states need to be updated when the graph changes.
[0090] When a graph changes, the initial vertex can be determined first. Then, the state propagation path starting from the initial vertex can be traversed to determine all affected vertices and the threshold number of received increments for each affected vertex. The finally determined multiple affected vertices include the initial vertex and the vertices reachable from the initial vertex.
[0091] The determination of initial vertices is related to the graph computation algorithm used when the graph changes. This application uses the PageRank algorithm and the SSSP algorithm as examples to illustrate the process of determining initial vertices. In the PageRank algorithm, for a newly added edge, the initial vertex is the vertex connected to the sending edge of the source vertex of the newly added edge; for a deleted edge, the initial vertex is the vertex connected to the sending edge of the source vertex of the deleted edge before deletion. In the SSSP algorithm, for a newly added edge, the initial vertex is the destination vertex of the newly added edge; for a deleted edge, the initial vertex is the destination vertex of the deleted edge before deletion.
[0092] For example, please refer to Figure 5 and Figure 6 , Figure 5 This is a schematic diagram of a directed graph provided in an embodiment of this application. Figure 6 This is a schematic diagram of an undirected graph provided in an embodiment of this application. Figure 5 and Figure 6 Each graph shows 11 vertices V0 to V10. Assume the graph was changed by adding edges (V0, V4) and (V0, V5) and deleting edge (V1, V2). For the PageRank algorithm, Figure 5 The initial vertices determined in the process are V1, V2, V4, and V5. Figure 6 The initial vertices determined are V0, V1, V2, V3, V4, V5, and V6. For the SSSP algorithm, Figure 5 The initial vertices determined in the process are V2, V4, and V5. Figure 6 The initial vertices determined in the process are V0, V1, V2, V4, and V5.
[0093] The graph processing device then traverses the state propagation path starting from each initial vertex. When it reaches an affected vertex that satisfies the first stopping condition, it stops traversing the current state propagation path, until all state propagation paths starting from each initial vertex have been traversed. The vertices visited during the traversal are identified as affected vertices, and the number of times each affected vertex is visited is determined as the threshold for the number of received increments for affected vertices.
[0094] As described above, the initial vertex first updates its vertex state and then needs to generate an increment for state propagation. The state propagation path starting from the initial vertex is all the paths traversed by the initial vertex during state propagation, that is, the paths between all endpoints reachable from the initial vertex. For example, such as... Figure 5 As shown, the state propagation paths starting from the initial vertex V2 include: V2→V4→V5→V6→V7→V8→V9 and V2→V3→V4→V5→V6→V7→V8→V9.
[0095] The first stopping condition includes: Condition 1. The affected vertex is another initial vertex, and / or Condition 2. When visiting the affected vertex, the affected vertex has no reachable neighboring affected vertices. The absence of reachable neighboring affected vertices can include two cases: one is that all edges connecting the affected vertex to other neighboring affected vertices have been visited (i.e., all sending edges connecting the affected vertex to neighboring affected vertices have been visited); the other is that the affected vertex has no connecting edges to other neighboring affected vertices (i.e., the affected vertex has no sending edges connecting it to neighboring affected vertices). When the first stopping condition includes multiple conditions, satisfying at least one condition is considered to satisfy the first stopping condition; not satisfying all conditions is considered not satisfying the first stopping condition.
[0096] The affected vertex receives the increment through the receiving edge. In this embodiment, the graph may include directed graphs and undirected graphs. When the graph is directed, the receiving edge of the affected vertex is the incoming edge, and the sending edge is the outgoing edge. When the graph is undirected, an edge connecting the affected vertex can be regarded as a sending edge and a receiving edge.
[0097] like Figure 5 and Figure 6 As shown, for the PageRank algorithm, Figure 5 The affected vertices, determined through traversal, include V1 to V9. Figure 6 The affected vertices determined through traversal include V0 to V10; for the SSSP algorithm, Figure 5 The affected vertices, determined through traversal, include V2 to V9. Figure 6 The affected vertices, determined by traversal, include V0 to V10.
[0098] This traversal process can be implemented by traversing all edges on the state propagation path. Optionally, the graph processing device can traverse the edges on the state propagation path starting from the initial vertex until all edges on the state propagation path starting from the initial vertex have been traversed. At this point, each edge on the state propagation path is visited only once, and the common edges of different state propagation paths are also visited only once. In this implementation, for condition 2 in the aforementioned first stopping condition, if the affected vertex still has unvisited sending edges connecting it to its neighboring affected vertices, then it is determined that the affected vertex still has accessible neighboring affected vertices; if all sending edges connecting the affected vertex to its neighboring affected vertices have been visited, or if the affected vertex does not have any sending edges connecting it to its neighboring affected vertices, then it is determined that the affected vertex does not have any accessible neighboring affected vertices. Furthermore, in this implementation, the number of times an affected vertex is visited is the number of edges from its neighboring affected vertices to the affected vertex, that is, the number of receiving edges connecting the affected vertex to its neighboring affected vertices.
[0099] Optionally, the edges along the state propagation path starting from the initial vertex can be traversed using a traversal algorithm. Traversal algorithms include depth-first search and / or breadth-first search. It is necessary to ensure that all edges between any two affected vertices in the graph have been visited after all traversals are completed; therefore, a vertex may be visited multiple times during the traversal process.
[0100] It should be noted that during the traversal, an edge on the state propagation path is visited only once, and a common edge between different state propagation paths is also visited only once. For an undirected graph, when any affected vertex is visited, the edge connecting that affected vertex to its neighboring affected vertices can be considered as a sending edge and a receiving edge. Therefore, an edge on the state propagation path in an undirected graph can be visited at most twice, and the affected vertex and its neighboring affected vertices can send increments to each other.
[0101] The following are respectively based on Figure 5 and Figure 6 Taking an example, we will explain the process of traversing all edges on the state propagation path starting from the initial vertex. We assume that the subsequent graph computation algorithm is the SSSP algorithm, and the first stopping condition includes conditions 1 and 2. Figure 5 As shown, traverse the edges on the state propagation path starting with V2.
[0102] like Figure 5As shown, taking the depth-first traversal algorithm as an example, first traverse edge (V2, V3), incrementing the visit count of V3 by 1. V3 does not satisfy the first stopping condition, so traverse edge (V3, V4), incrementing the visit count of V4 by 1. V4 satisfies condition 1, so return to V3. V3 then satisfies condition 2, so return to V2. Next, traverse (V2, V4), incrementing the visit count of V4 by 1. V4 satisfies condition 1, so return to V2. At this point, both edges on the two state propagation paths starting from V2 have been traversed, and the traversal process starting from V2 can be terminated. The traversal process starting from other initial vertices can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0103] like Figure 5 As shown, taking the breadth-first traversal algorithm as an example, edges (V2, V3) and (V2, V4) are traversed first, and the visit count of both V3 and V4 is incremented by 1. V3 does not meet the first stopping condition, so edge (V3, V4) is traversed, and the visit count of V4 is incremented by 1. V4 meets condition 1, so the algorithm returns to V2. At this point, the edges on both state propagation paths starting from V2 have been traversed, and the traversal process starting from V2 can be terminated. The traversal process starting from other initial vertices can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0104] After the traversal is completed, the number of times each affected vertex is visited is as follows: V2(0), V3(1), V4(2), V5(1), V6(1), V7(1), V8(1), V9(1).
[0105] like Figure 6 As shown, during the first traversal, since the edges connecting affected vertices to their neighboring affected vertices are undirected, an edge connecting an affected vertex to its neighboring affected vertices can be considered as both a sending edge and a receiving edge. For example... Figure 6 As shown, the edges on the state propagation path starting from the initial vertex V4 are traversed.
[0106] like Figure 6As shown, taking the depth-first traversal algorithm as an example, first traverse edge (V4, V0), incrementing the visit count of V0 by 1. V0 satisfies condition 1 of the first stopping condition, so return to V4. Traverse edge (V4, V2), incrementing the visit count of V2 by 1. V2 satisfies condition 1 of the first stopping condition, so return to V4. Traverse edge (V4, V5), incrementing the visit count of V5 by 1. V5 satisfies condition 1 of the first stopping condition, so return to V4. Traverse edge (V4, V3), incrementing the visit count of V3 by 1. V3 does not satisfy the first stopping condition. Traverse edge (V3, V4), incrementing the visit count of V4 by 1. V4 satisfies condition 1 of the first stopping condition, so return to V3. Traverse edge (V3, V2), incrementing the visit count of V2 by 1. V2 satisfies condition 1 of the first stopping condition, so return to V3. Traverse edge (V3, V10), incrementing the visit count of V10 by 1. V10 does not satisfy the first stopping condition, so the edge (V10, V3) is traversed, and the visit count of V3 is incremented by 1. V3 satisfies condition 2 of the first stopping condition, so the process returns to V4. All state propagation paths starting from V4 have been traversed, and the traversal process starting from V4 can be terminated. The process of traversing the edges on all state propagation paths starting from other initial vertices can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0107] like Figure 6 As shown, taking the breadth-first search algorithm as an example, we traverse edges (V4, V0), (V4, V2), (V4, V3), and (V4, V5), incrementing the visit count of each of V0, V2, V3, and V5 by 1. V0 satisfies condition 1 of the first stopping condition, so we return to V2. V2 satisfies condition 1 of the first stopping condition, so we return to V3. V3 does not satisfy the first stopping condition, so we traverse edges (V3, V4), (V3, V2), and (V3, V10), incrementing the visit count of each of V4, V2, and V10 by 1, and return to V5. V5 satisfies condition 1 of the first stopping condition, so we return to V4. V4 satisfies condition 1 of the first stopping condition, so we return to V2. V2 satisfies condition 1 of the first stopping condition, so we return to V10. V10 does not satisfy the first stopping condition, so we traverse edge (V10, V3), incrementing the visit count of each of V3 by 1. V3 satisfies condition 2 of the first stopping condition. All state propagation paths starting with V4 have been traversed, and the traversal process starting with V4 can be terminated. The process of traversing the edges on all state propagation paths starting with other initial vertices can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0108] After the traversal is completed, the number of times each affected vertex is visited is as follows: V0(3), V1(1), V2(2), V3(3), V4(4), V5(3), V6(2), V7(2), V8(2), V9(1), V10(1).
[0109] The depth-first traversal algorithm requires a stack during traversal, while the breadth-first traversal algorithm requires a queue. The following explanation, from the perspective of a stack, further clarifies the process of using the depth-first traversal algorithm to determine the threshold number of received increments for each affected vertex.
[0110] First, the execution entity for determining the threshold number of received increments for each affected vertex is defined. This entity can be a TDGraph module, such as the TDTU within the TDGraph module. For an example, please refer to [link to example]. Figure 7 , Figure 7 This is a schematic diagram of the structure of a TDTU provided in an embodiment of this application. The TDTU includes a Fetch Root module, a Fetch Offsets module, a Fetch Neighbors module, a SynchronizePropagation module, and a Fetch states module.
[0111] Before beginning the traversal process, the graph processing device (e.g., graph processing system software running on the core) needs to initialize the configuration information required for determining the threshold number of received increments for each affected vertex. This configuration information can be located at... Figure 1 The memory region shown, for example, is located in Figure 1 In the LLC, main memory, or L2 cache.
[0112] Alternatively, please refer to Figure 8 , Figure 8 This is a schematic diagram of configuration information provided in an embodiment of this application. Figure 8 This shows the name of the storage location where each configuration information is stored, the size of the configuration information, the base address of the storage location, and the configuration information itself. For example... Figure 8 As shown, the storage locations include: an offset array, a neighbor array, and a topology list.
[0113] The Neighbor Array stores the neighboring vertices of each vertex in the graph, with an address base of Addr1. The Offset Array stores the start and / or end positions of the neighboring vertices of each vertex in the Neighbor Array, with an address base of Addr0. The i-th cell in the Offset Array stores the start position of the neighboring vertices of the vertex with identity document (ID) i in the Neighbor Array. For example, if the neighboring vertices of the vertex with ID i are stored in cells a to b-1 in the Neighbor Array, then the i-th cell in the Offset Array stores the value 'a', and the (i+1)-th cell stores the value 'b'. The Topology List includes the cells corresponding to each vertex. In this process 201, the value stored in the cell corresponding to any vertex represents the number of times that vertex has been visited. The value stored in the i-th cell represents the number of times vertex Vi has been visited, with an address base of Addr9.
[0114] As mentioned above Figure 5 The graph shown and the subsequent graph computation algorithm used are based on the SSSP algorithm. Figure 8 In this process, the Neighbor Array consists of 12 units, each with a size of 12, storing the following values in sequence: V1, V4, V5, V3, V4, V4, V5, V6, V7, V8, V9, and V3. The Offset Array also consists of 12 units, each with a size of 12, storing the following values in sequence: 0, 3, 3, 5, 6, 7, 8, 9, 10, 11, 11, and 12. The Topology List consists of 11 units, each with a size of 11, corresponding to each of the 11 vertices. In this process 201, the values stored in these 11 units represent the number of times each of the 11 vertices, from V0 to V10, has been visited. Before traversal begins, the number of visits for each vertex is 0.
[0115] After initializing the configuration information, TDTU uses the configuration information (e.g.) Figure 8 (The configuration information shown) performs a traversal process to determine the threshold for the number of received increments for each affected vertex. Please refer to [the relevant documentation / reference]. Figure 9 , Figure 9 A flowchart illustrating a process for determining the threshold number of received increments for each affected vertex through traversal, as provided in this embodiment of the application, is shown below:
[0116] 2011. Obtain the initial vertex from the Active Vertices Array and push the initial vertex onto the stack as the root vertex.
[0117] This process can be performed by Figure 7The Fetch Root module, as shown, is executed. The process by which the Fetch Root module determines and obtains the initial vertices can be referred to the foregoing description; it will not be repeated here in the embodiments of this application. Figure 5 and Figure 8 For example, the Fetch Root module obtains the initial vertex V2, which can then be pushed onto the stack.
[0118] 2012. Read the starting and / or ending positions of the neighboring vertices of the top-level vertex in the stack from the Offset Array.
[0119] This process can be performed by Figure 7 The Fetch Offsets module shown is executed. Figure 5 and Figure 8 For example, at this time, the topmost vertex in the stack is V2. The Fetch Offsets module reads the storage start position of V2's neighboring vertices from the Offset Array, which is 3, and the storage end position is 4.
[0120] 2013. Based on the storage start and / or storage end positions of the neighboring vertices of the top-level vertex in the stack, retrieve the cache line containing the ID of the unvisited neighbor vertex of the top-level vertex from the Neighbor Array.
[0121] This process can be performed by Figure 7 The Fetch Neighbors module shown is executed. Figure 5 and Figure 8 For example, the FetchNeighbors module retrieves the cache line containing the unvisited neighbor vertex IDs from the Neighbor Array based on the start and end positions of the storage, thus obtaining the unvisited neighbor vertex IDs as V3 and V4, respectively.
[0122] 2014. Push any unvisited neighbor of the top-level vertex in the stack onto the stack, set the edge between the previous top-level vertex and the current top-level vertex in the stack to be visited, and simultaneously increment the value of the current top-level vertex in the Topology List by 1.
[0123] This process can be performed by Figure 7 The Synchronize Propagation module shown is executed. Figure 5 and Figure 8 For example, the Synchronize Propagation module pushes V3 onto the stack, sets the edge (V2, V3) as visited, and simultaneously increments the value of V3 in the Topology List by 1 until it becomes 1, at which point V3 is visited.
[0124] In the aforementioned process, when the current top-level vertex in the stack meets the first stopping condition, it is popped from the stack. When the current top-level vertex in the stack does not meet the first stopping condition, the aforementioned process 2012 to 2014 is executed again, pushing the unvisited neighbor vertices of the current top-level vertex onto the stack, setting the edge between the previous top-level vertex and the current top-level vertex in the stack as visited, and simultaneously incrementing the value of the current top-level vertex in the Topology List by 1. This continues until the top-level vertex in the stack is the initial vertex that was first pushed onto the stack, and the initial vertex meets the first stopping condition. Then, the next initial vertex is pushed onto the stack and the aforementioned process is re-executed until all state propagation paths starting from each initial vertex have been traversed, thereby obtaining the threshold for the number of received increments for each affected vertex.
[0125] For example, with Figure 5 and Figure 8 For example, if the current top-level vertex is V3, and V3 does not meet the first stopping condition, then the aforementioned process 2012 to 2014 is re-executed for V3. V4 is pushed onto the stack, the edge (V3, V4) is marked as visited, and the value of V4 in the Topology List is incremented by 1, becoming 1. At this point, V4 is visited. The current top-level vertex is V4, and V4 meets condition 1 of the first stopping condition, so V4 is popped from the stack. If the current top-level vertex is V3, and V3 meets the first stopping condition, V3 is popped from the stack. If the current top-level vertex is V2, and V2 does not meet the first stopping condition, then the aforementioned process 2012 to 2014 is re-executed for V2. V4 is pushed onto the stack, the edge (V2, V4) is marked as visited, and the value of V4 in the Topology List is incremented by 1, becoming 2. At this point, V4 is visited. The current top-level vertex is V4, and V4 meets condition 1 of the first stopping condition, so V4 is popped from the stack. The current top-level vertex is V2. V2 satisfies the first stopping condition. At this point, all state propagation paths starting from V2 have been traversed. V2 is popped from the stack, and the traversal process starting from the initial vertex V2 ends. The traversal then begins from the next initial vertex, and this process can be referenced from the traversal process starting from V2. This embodiment of the application will not elaborate further here.
[0126] After all the state propagation paths starting from each initial vertex have been traversed, the Topology List stores the following values in sequence: 0, 0, 0, 1, 2, 1, 1, 1, 1, 1, and 0. At this point, the values corresponding to the vertices stored in the Topology List also represent the threshold for the number of incremental values received by each vertex.
[0127] In this embodiment, the graph can be a complete graph or a partial structure of a complete initial graph. For example, a graph processing device (e.g., graph processing system software running on a core) can configure a complete graph on a core for processing, or it can divide the complete initial graph into blocks, configuring each block on a core for processing by that core and the corresponding TDGraph module.
[0128] 202. Based on the threshold number of received increments for each affected vertex, perform at least one traversal process until the number of visits to each affected vertex reaches its respective threshold number of received increments, resulting in multiple edges arranged in traversal order. The traversal process includes: determining the root vertex from at least one active vertex among the multiple affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex that satisfies the first stopping condition is visited, until all state propagation paths starting from the root vertex have been traversed.
[0129] The state propagation path starting from the root vertex can be referred to the aforementioned description of the state propagation path starting from the initial vertex, and will not be repeated here in the embodiments of this application. As can be seen from the aforementioned process 102, during the first traversal, the root vertex is determined from the initial vertices. During subsequent traversals, the root vertex is determined from the initial vertices that have never been the root vertex and the vertices that have never been the root vertex and satisfy the second stopping condition.
[0130] It should be noted that, generally, the vertex whose difference between the number of received increments threshold and the number of visits is 0 should be selected from at least one active vertex as the root vertex. This is equivalent to determining the vertex that receives all increments as the root vertex during the state propagation process, so that the root vertex only needs to be updated once and the state propagation only needs to be performed once.
[0131] When the initial vertex included in an active vertex does not form a cycle with other affected vertices, the difference between the received increment threshold and the number of visits for that initial vertex is 0. However, if the initial vertex included in an active vertex forms a cycle with other affected vertices, the difference between the received increment threshold and the number of visits for that initial vertex may not be 0. In this case, to minimize the number of times the root vertex is updated and the number of state propagations during state propagation, the affected vertex with the smallest difference between the received increment threshold and the number of visits can be selected as the root vertex from at least one active vertex. For example, such as... Figure 5 As shown, if edge (V4, V2) still exists, for the SSSP algorithm, before the first traversal begins, at least one active vertex includes all initial vertices V2, V4, and V5. V2 and V4 form a cycle, and there are no vertices in V2, V4, and V5 whose received increment threshold and visited count threshold are both 0.
[0132] The second stopping condition includes: Condition 1. When visiting an affected vertex, the number of times the affected vertex has been visited is less than the threshold for the number of received increments, and / or Condition 2. When visiting an affected vertex, the affected vertex has no accessible neighboring affected vertices. For an explanation of Condition 2, please refer to the relevant description of Condition 2 in the aforementioned first stopping condition; this embodiment will not repeat it here. Condition 1 is equivalent to the following: during state propagation, when visiting a vertex, the number of increments received by the vertex has not yet reached the threshold for the number of received increments, meaning there are still unreceived increments. Therefore, visiting further vertices is not allowed to avoid repeated updates of the vertex state of the same vertex, thereby reducing invalid and redundant calculations during vertex state updates and improving update efficiency. When the second stopping condition includes multiple conditions, satisfying at least one condition is considered as satisfying the second stopping condition; not satisfying all conditions is considered as not satisfying the second stopping condition.
[0133] During the traversal, the order in which each edge is visited can be recorded. For example, a queue can be created, and the edge identifier of each edge visited can be stored in the queue.
[0134] When the aforementioned process 201 is implemented by traversing all edges on the state propagation path, correspondingly, process 202 can also be implemented by traversing all edges on the state propagation path. In this case, each edge on the state propagation path is visited only once, and the common edge of different state propagation paths is also visited only once. As described in process 201, the threshold for the number of received increments of the affected vertex in this implementation is the number of received edges connecting the affected vertex to its neighboring affected vertices. Visiting an affected vertex once means that one received edge connecting the affected vertex to its neighboring affected vertices is visited.
[0135] Optionally, the edges along the state propagation path starting from the root vertex can be traversed using a traversal algorithm. When a vertex satisfying the second stopping condition is reached, the traversal of the current state propagation path ends, and the traversal of other state propagation paths starting from the root vertex continues until all state propagation paths starting from the root vertex have been traversed, at which point the traversal starting from that root vertex ends. Traversal algorithms include depth-first traversal and / or breadth-first traversal. It is necessary to ensure that the edges between any two affected vertices in the graph are visited after all traversals are completed; therefore, a vertex may be visited multiple times during the traversal process.
[0136] It should be noted that during the traversal, an edge on the state propagation path is visited only once, and a common edge between different state propagation paths is also visited only once. For an undirected graph, when any affected vertex is visited, the edge connecting that affected vertex to its neighboring affected vertices can be considered as a sending edge and a receiving edge. Therefore, an edge on the state propagation path in an undirected graph can be visited at most twice, and the affected vertex and its neighboring affected vertices can send increments to each other.
[0137] The following are respectively based on Figure 5 and Figure 6 Taking the root vertex as an example, we will explain the process of traversing all edges on the state propagation path. We assume that the subsequent graph computation algorithm is the SSSP algorithm, and the first stopping condition includes conditions 1 and 2. Figure 5 As shown, before the first traversal begins, at least one active vertex includes V2, V4 and V5. Based on the aforementioned process 201, among V2, V4 and V5, only V2 has the smallest difference between the threshold of the number of received increments and the number of visited received edges connected to the affected neighbor vertices. This difference is 0. Therefore, V2 can be determined as the root vertex and the edges on the state propagation path starting with V2 can be traversed.
[0138] like Figure 5 As shown, taking the depth-first traversal algorithm as an example, first traverse edge (V2, V3), incrementing the visit count of V3 by 1. V3 does not meet the second stopping condition, traverse edge (V3, V4), incrementing the visit count of V4 by 1. V4 meets condition 1, so return to V3. V3 now meets condition 2, so return to V2. V2 does not meet the second stopping condition, traverse edge (V2, V4), incrementing the visit count of V4 by 1. V4 now does not meet the second stopping condition, traverse edge (V4, V5), incrementing the visit count of V5 by 1. Referring to the aforementioned traversal process, subsequently traverse (V5, V6), (V6, V7), (V7, V8), and (V8, V9), incrementing the visit count of V6, V7, V8, and V9 by 1 respectively. At this point, both edges on the two state propagation paths starting from V2 have been traversed, and the visit count of each affected vertex has reached its respective threshold for the number of received increments, so the traversal process can end.
[0139] like Figure 5As shown, taking the breadth-first search algorithm as an example, first traverse edge (V2, V3), then traverse edge (V2, V4), incrementing the visit count of both V3 and V4 by 1. V3 no longer meets the second stopping condition, so traverse edge (V3, V4), incrementing the visit count of V4 by 1. V4 no longer meets the second stopping condition, so traverse edge (V4, V5), incrementing the visit count of V5 by 1. Referring to the aforementioned traversal process, (V5, V6), (V6, V7), (V7, V8), and (V8, V9) are traversed sequentially, incrementing the visit count of V6, V7, V8, and V9 by 1. At this point, all edges on the two state propagation paths starting from V2 have been traversed, and the visit count of each affected vertex has reached its respective threshold for the number of received increments, allowing the traversal process to end.
[0140] like Figure 6 As shown, during the first traversal, since the edges connecting affected vertices to their neighboring affected vertices are undirected, these edges can be considered as a sending edge and a receiving edge. Before the first traversal begins, at least one active vertex includes V0, V1, V2, V4, and V5. Based on the aforementioned process 201, among V0, V1, V2, V4, and V5, V1 has the smallest difference between the threshold for the number of received increments and the number of times it has been visited, which is 1. Therefore, V1 can be determined as the root vertex and the edge on the state propagation path starting with V1.
[0141] like Figure 6 As shown, taking the depth-first traversal algorithm as an example, the edge (V1, V0) is traversed first, and the number of times V0 has been visited is incremented by 1. V0 satisfies condition 1 of the second stopping condition, and V1 is returned. At this point, all state propagation paths starting from V1 have been traversed, and the root vertex needs to be selected again.
[0142] Currently, at least one active vertex includes V0, V2, V4, and V5. Among V0, V2, V4, and V5, V0 and V2 have the smallest difference between the received increment threshold and the number of times they have been visited, which is 2. Therefore, either V0 or V2 can be determined as the root vertex. Assuming V0 is chosen as the root vertex, traversing the edge (V0, V1), the number of times V1 has been visited is incremented by 1. V1 satisfies condition 2 of the second stopping condition, so return to V0. Traversing the edge (V0, V4), the number of times V4 has been visited is incremented by 1. V4 satisfies condition 1 of the second stopping condition, so return to V0. Traversing the edge (V0, V5), the number of times V5 has been visited is incremented by 1. V5 satisfies condition 1 of the second stopping condition, so return to V0. At this point, all state propagation paths starting from V0 have been traversed, and a new root vertex needs to be selected.
[0143] Currently, at least one active vertex includes V2, V4, and V5. Among V2, V4, and V5, V2 and V5 have the smallest difference between the received increment threshold and the number of times they have been visited, which is 2. Therefore, either V2 or V5 can be determined as the root vertex. Assuming V2 is selected as the root vertex, the edge (V2, V3) is traversed, and the number of times V3 has been visited is incremented by 1. V3 satisfies condition 1 of the second stopping condition, and returns to V2. The edge (V2, V4) is traversed, and the number of times V4 has been visited is incremented by 1. V4 satisfies condition 1 of the second stopping condition, and returns to V2. At this point, all state propagation paths starting from V2 have been traversed, and a new root vertex needs to be selected. The subsequent process of re-determining the root vertex and traversing the edges on all state propagation paths starting from the new root vertex can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0144] like Figure 6 As shown, taking the breadth-first search algorithm as an example, when traversing the edge (V1, V0), the number of times V0 has been visited is incremented by 1. V0 satisfies condition 1 of the second stopping condition, so V1 is returned. At this point, all state propagation paths starting from V1 have been traversed, and the root vertex needs to be selected again.
[0145] Currently, at least one active vertex includes V0, V2, V4, and V5. Among V0, V2, V4, and V5, V0 and V2 have the smallest difference between the received increment threshold and the number of times they have been visited, which is 2. Therefore, either V0 or V2 can be determined as the root vertex. Assuming V0 is chosen as the root vertex, we traverse edges (V0, V1), (V0, V4), and (V0, V5), incrementing the number of visits for V1, V4, and V5 by 1. V1 satisfies condition 2 of the second stopping condition, so we return to V4. V4 satisfies condition 1 of the second stopping condition, so we return to V5. V5 satisfies condition 1 of the second stopping condition, so we return to V0. At this point, all state propagation paths starting from V0 have been traversed, and we need to reselect the root vertex.
[0146] Currently, at least one active vertex includes V2, V4, and V5. Among V2, V4, and V5, V2 and V5 have the smallest difference between the received increment threshold and the number of times they have been visited, which is 2. Therefore, either V2 or V5 can be determined as the root vertex. Assuming V2 is selected as the root vertex, the edges (V2, V3) and (V2, V4) are traversed, and the number of times V3 and V4 have been visited is incremented by 1. V3 satisfies condition 1 of the second stopping condition, so V4 is returned. V4 satisfies condition 1 of the second stopping condition, so V2 is returned. At this point, all state propagation paths starting from V2 have been traversed, and a new root vertex needs to be selected. The subsequent process of re-determining the root vertex and traversing the edges on all state propagation paths starting from the new root vertex can be referred to the foregoing description, and will not be repeated here in the embodiments of this application.
[0147] In the foregoing Figure 9Building upon this, the following section further explains, from a stack perspective, the at least one traversal process performed using a depth-first traversal algorithm based on a threshold for the number of received increments for each affected vertex. The execution entity for this process can be a TDGraph module, such as TDTU within the TDGraph module (e.g., Figure 7 (TDTU shown). Based on the foregoing Figure 9 The illustrated embodiment obtains a threshold for the number of received increments for each affected vertex (i.e., the number of received edges connected to neighboring affected vertices), and then the TDTU bases its decisions on configuration information (e.g., ...). Figure 8 The configuration information shown employs a depth-first traversal algorithm to perform at least one traversal process to obtain multiple edges arranged in traversal order.
[0148] like Figure 8 As shown, the storage location may also include an Active Vertices Array, which is used to record in real time whether each vertex in the graph is an active vertex, with an address base of Addr6. The Active Vertices Array includes the bits corresponding to each vertex in the graph, and the i-th bit is used to record whether Vi is an active vertex.
[0149] For example, the Active Vertices Array can record whether each vertex is active or inactive using a bit vector. For instance, "1" indicates an active vertex, and "0" indicates an inactive vertex. As mentioned earlier, before traversal begins, the initial value of each vertex in the Active Vertices Array is "1". When an active vertex is determined to be the root vertex, it is set to inactive; for example, its corresponding value in the Active Vertices Array can be set to 0. During traversal, each time an affected vertex that satisfies the second stopping condition is visited, it is set to active; for example, its corresponding value in the Active Vertices Array can be set to 1.
[0150] like Figure 5 and Figure 8 As shown, the Active Vertices Array comprises 11 bits corresponding to each of the 11 vertices in the graph, with a size of 11. Before traversal begins, the 2nd, 4th, and 5th bits in the Active Vertices Array are 1, and the other bits are 0. Please refer to [reference needed]. Figure 10 , Figure 10 This application provides a flowchart illustrating a process of performing at least one traversal based on a threshold for the number of received increments for each affected vertex. Figure 10 During the traversal process shown, each time an affected vertex is visited, it is connected to a neighboring affected vertex by a receiving edge. This means the value of that affected vertex in the Topology List is decremented by 1. Therefore, during this traversal, the values stored in the Topology List represent the difference between the threshold number of received increments for the 11 vertices from V0 to V10 and the number of times they were visited. Before the traversal begins, the values stored in the Topology List represent the threshold number of received increments for the 11 vertices from V0 to V10. The traversal process is shown below:
[0151] 2021. Determine the root vertex from at least one active vertex, push the root vertex onto the stack, and set the root vertex as an inactive vertex in the ActiveVertices Array.
[0152] This process can be performed by Figure 7 The Fetch Root module is executed as shown above. Figure 9 As described in the corresponding embodiment, the Active Vertices Array records whether each vertex is active, and the Topology List records the difference between the received increment threshold and the number of times each vertex has been visited. A smaller difference between the received increment threshold and the number of times visited indicates fewer unvisited receiving edges connecting the vertex to its neighboring affected vertices. A difference of 0 indicates that all unvisited edges connecting the vertex to its neighboring affected vertices are sending edges. Therefore, active vertices can be determined using the Active Vertices Array, and the active vertex with the smallest value in the Topology List is selected as the root vertex. Figure 5 and Figure 8 For example, the Fetch Root module determines that among the active vertices V2, V4, and V5 in the Active Vertices Array, V2 has the smallest value in the Topology List, which is 0. V2 is pushed onto the stack, and the state of V2 in the Active Vertices Array is set to inactive, for example, by setting the second bit in the Active Vertices Array from 1 to 0.
[0153] 2022. Read the starting and / or ending positions of the neighboring vertices of the top-level vertex in the stack from the Offset Array.
[0154] This process can be performed by Figure 7 The Fetch Offsets module shown is executed. Figure 5 and Figure 8For example, at this time, the topmost vertex in the stack is V2. The Fetch Offsets module reads the storage start position of V2's neighboring vertices from the Offset Array, which is 3, and the storage end position is 4.
[0155] 2023. Based on the storage start and / or storage end positions of the neighboring vertices of the top-level vertex in the stack, retrieve the cache line containing the ID of the unvisited neighbor vertex of the top-level vertex from the Neighbor Array.
[0156] This process can be performed by Figure 7 The Fetch Neighbors module shown is executed. Figure 5 and Figure 8 For example, the FetchNeighbors module retrieves the cache line containing the unvisited neighbor vertex IDs from the Neighbor Array based on the start and end positions of the storage, thus obtaining the unvisited neighbor vertex IDs as V3 and V4, respectively.
[0157] 2024. Push any unvisited neighbor of the top-level vertex in the stack onto the stack, mark the edge between the previous top-level vertex and the current top-level vertex in the stack as visited and add it to the queue, and decrement the value of the current top-level vertex in the Topology List by 1.
[0158] This process can be performed by Figure 7 The Synchronize Propagation module shown is executed. Figure 5 and Figure 8 For example, the Synchronize Propagation module pushes V3 onto the stack, marks edge (V2, V3) as visited, adds edge (V2, V3) to the queue, and decrements the value of V3 in the Topology List by 1, making it 0. At this point, V3 is accessed. The queue can include a First-In-First-Out (FIFO) queue, and its name can be the FetchedBuffer.
[0159] In this embodiment, since the aforementioned process decrements the value of a vertex in the Topology List by 1 after visiting a vertex connected to a neighboring affected vertex via a receiving edge, it is possible to determine whether a vertex satisfies condition 1 of the second stopping condition by the current value of that vertex in the Topology List. If the value of a vertex in the current Topology List is 0 when a vertex is visited, then the vertex does not satisfy condition 1 of the second stopping condition. If the value of a vertex in the current Topology List is not 0 when a vertex is visited, then the vertex satisfies condition 1 of the second stopping condition.
[0160] In the aforementioned process, when the current top-level vertex in the stack satisfies the second stopping condition, it is popped from the stack. When the current top-level vertex in the stack does not satisfy the second stopping condition, the aforementioned process 2022 to 2024 is executed again. Unvisited neighbor vertices of the current top-level vertex are pushed onto the stack, the edge between the previous top-level vertex and the current top-level vertex in the stack is marked as visited and added to the queue, and the value of the current top-level vertex in the Topology List is decremented by 1. This continues until the top-level vertex in the stack is the root vertex that was initially pushed onto the stack, and the root vertex satisfies the second stopping condition. Then, the next root vertex is pushed onto the stack and the aforementioned process is re-executed until the number of visits to each affected vertex reaches its respective threshold for the number of received increments, i.e., the value of each affected vertex in the Topology List is 0, thus obtaining a queue storing multiple edges in traversal order.
[0161] For example, with Figure 5 and Figure 8For example, if the current top-level vertex is V3, and V3 does not meet the second stopping condition, then the aforementioned process 2022 to 2024 is re-executed for V3. V4 is pushed onto the stack, the edge (V3, V4) is marked as visited and added to the queue, and the value of V4 in the Topology List is decremented by 1, becoming 1. At this point, V4 is visited. If the current top-level vertex is V4, and V4 meets condition 1 of the second stopping condition, V4 is popped from the stack. If the current top-level vertex is V3, and V3 meets condition 2 of the second stopping condition, V3 is popped from the stack. If the current top-level vertex is V2, and V2 does not meet the second stopping condition, then the aforementioned process 2022 to 2024 is re-executed for V2. V4 is pushed onto the stack, the edge (V2, V4) is marked as visited and added to the queue, and the value of V4 in the Topology List is decremented by 1, becoming 0. At this point, V4 is visited. The current top-level vertex is V4. Since V4 does not meet the second stopping condition, the aforementioned process from 2022 to 2024 is re-executed for V4. V5 is pushed onto the stack, the edge (V4, V5) is marked as visited and added to the queue, and the value of V5 in the Topology List is decremented by 1 to become 0. At this point, V5 is visited. Subsequently, edges (V5, V6), (V6, V7), (V7, V8), and (V8, V9) are marked as visited and added to the queue in sequence.
[0162] The aforementioned traversal process yields a queue (e.g., a Fetched Buffer), which stores multiple edges arranged in traversal order.
[0163] The subsequent state propagation process can be carried out by Figure 1 The kernel shown (e.g., graph processing system software running on the kernel) executes the process. Referring to process 201 above, multiple edge identifiers can be stored in a queue in traversal order. The TDTU sends this queue to the corresponding kernel, which performs the state propagation process in graph computation based on the queue.
[0164] The state propagation process also requires the vertex states of the affected vertices at both ends of each edge in the queue. In one implementation, the graph processing device can obtain the vertex states of the affected vertices at both ends of each edge in the queue after the aforementioned traversal process is completed; the vertex states of the vertices at both ends of each edge can also be stored in the queue. Alternatively, the vertex states of the affected vertices at both ends of each edge can be obtained as soon as it is visited.
[0165] In another implementation, during subsequent state propagation, each time an edge that needs to be propagated through the state is selected, the vertex states of the affected vertices at both ends of the selected edge are obtained. See procedures 203 and 204 below, which will be used as examples to illustrate this implementation.
[0166] 203. Select the edge that needs to be propagated through the state according to the order of the multiple edges.
[0167] 204. Get the vertex states of the affected vertices at both ends of the selected edge.
[0168] Image processing equipment can... Figure 1 The vertex states of the affected vertices at both ends of the selected edge are obtained from the LLC, main memory, or L2 cache shown.
[0169] Optionally, the graph processing device can identify multiple hot vertices from the graph and then store the vertex states of these hot vertices in a contiguous storage area. Hot vertices are those frequently accessed during state propagation, i.e., vertices with high access frequency, and whose received increment threshold is greater than or equal to a certain threshold. If the traversal process is performed by iteration, the number of received edges connecting hot vertices to their neighboring affected vertices is greater than or equal to a certain threshold. When obtaining the vertex states of affected vertices at both ends of a selected edge, if the affected vertex at either end of the selected edge is a hot vertex, the vertex state of the affected vertex at that end is obtained from the contiguous storage area. If the affected vertex at either end is a non-hot vertex, the vertex state of the affected vertex at that end is obtained from the memory area.
[0170] The number threshold can be obtained by the access frequency of each vertex and a custom ratio α, where 0 ≤ α ≤ 1. In this embodiment, the threshold for the number of received increments of a vertex is approximated as the access frequency of the vertex, and the vertices whose access frequency ranks in the top α (including α) among all vertices are determined as hot vertices. That is, the minimum access frequency among the vertices whose access frequency ranks in the top α among all vertices is determined as the number threshold.
[0171] Optionally, it can be based on configuration information (e.g. Figure 8 (The configuration information shown) retrieves the vertex states of the affected vertices at both ends of the selected edge. For example... Figure 8 As shown, the storage locations may also include: Vertex StatesArray, Coalesced StatesArray, Hash Tables (H Tables), and Hot VerticesArray. Figure 8 The diagram shows two memory regions, Vertex States Array1 and Vertex States Array2, which are used to store the vertex state of each vertex, with address bases Addr2 and Addr3 respectively. Figure 8The diagram shows two contiguous storage areas, Coalesced States Array1 and Coalesced States Array2. Coalesced States Array1 and Coalesced States Array2 are used to store the vertex states of hot vertices, with address bases of Addr4 and Addr5, respectively. H Tables are used to store the index information of hot vertices, with an address base of Addr7.
[0172] The Hot Vertices Array is used to record whether each vertex is a hot vertex, with an address base of Addr8. The Hot Vertices Array includes bits corresponding to each vertex in the graph, with the i-th bit used to record whether Vi is a hot vertex. For example, the Hot Vertices Array can record whether each vertex is a hot vertex in bit vector form. For example, "1" can indicate that the vertex is a hot vertex, and "0" can indicate that the vertex is not a hot vertex.
[0173] Optionally, the configuration information also includes the first vertex ID and the last vertex ID of the graph structure. Here, the Vertex States Array corresponds to a memory region, and the Coalesced States Array corresponds to a contiguous storage region.
[0174] The process of obtaining the vertex states of the affected vertices at both ends of the selected edge can be performed by the TDTU's Fetchstates module and VSCU. For an affected vertex at either end of the selected edge, VSCU first determines whether the affected vertex at that end is a hot vertex. If the access frequency of the affected vertex ranks in the top α among all vertices, the affected vertex is determined to be a hot vertex, and the bit position corresponding to the affected vertex in the Hot Vertices Array is set to 1. If the access frequency of the affected vertex does not rank in the top α among all vertices, the affected vertex is determined to be a non-hot vertex, and the bit position corresponding to the affected vertex in the Hot Vertices Array is set to 0.
[0175] If the affected vertex is determined to be a non-hot vertex, the VSCU obtains and provides the Fetch states module with the address of the affected vertex's vertex state in the Vertex States Array. The Fetch states module then retrieves the affected vertex's vertex state from the Vertex States Array based on the address provided by the VSCU.
[0176] If the affected vertex is determined to be a hot vertex, VSCU first checks whether this is the first time the affected vertex has been visited. If it is not the first time, VSCU retrieves the address of the affected vertex's state in the Coalesced States Array from the H Tables and provides this address to the Fetch states module. The Fetch states module then retrieves the affected vertex's state from the Coalesced States Array based on the address provided by VSCU.
[0177] When the affected vertex is visited for the first time, its vertex state is stored in the VertexStates Array. VSCU retrieves the affected vertex state from the Vertex States Array, stores it in the Coalesced States Array, and adds the index information of the affected vertex to the H Tables to represent its address in the Coalesced States Array. For example, it adds the index information of the affected vertex V.<v,vertex offset> The vertex offset represents the offset address of the affected vertex V in the Coalesced StatesArray.
[0178] Please refer to Figure 11 , Figure 11 This is a schematic diagram of vertex state storage for a hot vertex provided in an embodiment of this application. Figure 11 This shows a cache line 1 located in LLC. Cache line 1 is a contiguous storage area that stores the vertex states of multiple hot vertices. Figure 11 (4 examples are shown). In graph computation, most state propagation processes involve only a small subset of vertices that need to update their states. This means that the percentage of vertices requiring state updates is small, and therefore, the states of most vertices are not needed during graph computation. In related technologies, vertex states are sparsely stored in main memory. During graph computation, a single cache line stores a small number of vertex states required for the computation, sometimes even only one. This leads to numerous random data accesses when retrieving vertex states, resulting in high data access overhead. In this embodiment, the vertex states of hot vertices are merged and stored in contiguous storage areas. This allows a single cache line to store a larger number of vertex states required for graph computation, meaning one cache line can handle most vertex state accesses during state propagation. Vertex state retrieval is continuous and regularized, thus reducing data access overhead.
[0179] 205. Perform state propagation based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
[0180] Graph processing devices can determine the increment based on the weight of the selected edge and / or the vertex states of the affected vertices at both ends of the selected edge, and then propagate the increment along the selected edge.
[0181] As described above, the traversal order of edges during state propagation is determined by the storage order of edges in the queue. This storage order is determined by two factors: the root vertex in process 202 and whether a vertex satisfies the second stopping condition when visited. After determining the threshold for the number of received increments, when the difference between the threshold for the number of received increments of at least one active vertex and the number of visited received edges connecting to affected vertices with neighbors is minimized, it means that the active vertex has received the fewest increments during state propagation, and this active vertex is used as the root vertex for traversal. When an affected vertex is visited, if the number of times the affected vertex has been visited is less than its threshold for the number of received increments, satisfying condition 1 of the second stopping condition, it means that the affected vertex has not received all increments during state propagation, and therefore cannot continue visiting.
[0182] This ensures that an active vertex can only become the root vertex when the difference between the received increment threshold and the number of visits is minimized. During traversal, it can only continue visiting its neighboring affected vertices when the difference between the received increment threshold and the number of visits is zero. When state propagation follows this visiting order, an affected vertex will only proceed with state propagation after accumulating most or even all increments. This normalizes state propagation, effectively reducing the number of updates for subsequently visited affected vertices, thereby reducing invalid and redundant computations during vertex state updates and improving update efficiency.
[0183] When both of the aforementioned traversal processes are implemented by traversing the edges on the state propagation path, as described above... Figure 9 and Figure 10 As described in the embodiment, the root vertex and whether condition 1 in the second stopping condition is satisfied can be determined by the values of each affected vertex in the Topology List. The values of each affected vertex in the Topology List are... Figure 9 The process 2014 and Figure 10 The process will change in 2024. Figure 9 In the traversal shown, when an affected vertex is visited along an edge connecting it to a neighboring affected vertex, the value of that affected vertex in the Topology List is incremented by 1. Figure 9The traversal yielded the threshold for the number of received increments for each affected vertex (i.e., the number of received edges that connect the affected vertex to its neighboring affected vertex).
[0184] exist Figure 10 In the traversal shown, when an affected vertex is visited and connected to a neighboring affected vertex via a receiving edge, it's equivalent to a state propagation process having passed through that affected vertex (i.e., the affected vertex has received an increment), and the value of that affected vertex in the Topology List is decremented by 1. When the value of an active vertex in the Topology List is at its minimum, for example, 0, it's equivalent to the active vertex not having received the fewest increments among at least one active vertex in subsequent state propagation processes (e.g., all increments passing through that active vertex have been received), and traversal begins with that active vertex as the root vertex. When an affected vertex is visited and its value in the Topology List is not 0, condition 1 of the first stopping condition is satisfied, meaning that the affected vertex still has unreceived increments in the state propagation process, and therefore traversal cannot continue.
[0185] In this embodiment of the application, by means of Figure 9 and Figure 10 In both traversals, the values in the Topology List are modified so that a vertex can only continue traversing after most or all of its receiving edges connected to affected neighbor vertices have been visited. When the state propagation process follows this traversal order, a vertex will only continue propagating its state after receiving most or all increments. This normalizes state propagation, reducing the number of updates and increment generation for each affected vertex. For example, if a vertex continues traversing only after all its receiving edges connected to affected neighbor vertices have been visited, each affected vertex only needs to update its state once and generate an increment once. This reduces invalid and redundant computations during vertex state updates, improving update efficiency.
[0186] In summary, the graph processing method provided in this application, when the graph changes, determines the threshold for the number of received increments for each of the multiple affected vertices during state propagation. Based on this threshold, at least one traversal process is performed until the number of visits to each affected vertex reaches its respective threshold, resulting in multiple edges arranged in traversal order. Then, the edge currently requiring state propagation is selected according to the order of the multiple edges, the vertex states of the affected vertices at both ends of the selected edge are obtained, and state propagation is performed based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge. The traversal process includes: determining the root vertex from at least one active vertex among the multiple affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex satisfying a first stopping condition is reached, until all state propagation paths starting from the root vertex have been traversed. After determining the threshold for the number of received increments, when the difference between the threshold for the number of received increments and the number of times a given active vertex has been visited is the smallest among at least one active vertex, it means that the number of increments that active vertex has not received during subsequent state propagation is the smallest among at least one active vertex, and this active vertex is designated as the root vertex for traversal. When an affected vertex is visited, if the number of times that affected vertex has been visited is less than its threshold for the number of received increments, it means that the number of increments that vertex has received during subsequent state propagation is insufficient, and therefore, traversal cannot continue. This ensures that an active vertex can only become the root vertex when the difference between the threshold for the number of received increments and the number of times it has been visited is the smallest, and that it can only continue to visit its neighboring affected vertices when the difference between the threshold for the number of received increments and the number of times it has been visited is 0 during traversal. When the state propagation process follows this traversal order, it starts with the active vertex that has received the fewest increments. The affected vertices that have received increments will only continue the state propagation after all increments have been accumulated. This normalizes the state propagation process, effectively reduces the number of updates for subsequently visited affected vertices, thereby reducing invalid and redundant calculations in the process of updating vertex states and improving update efficiency.
[0187] When both traversal processes are implemented by traversing the edges along the state propagation path, the values in the Topology List are modified during both traversal processes so that a vertex can only continue traversing after most or all of its receiving edges connected to affected neighbor vertices have been visited. When the state propagation process follows this traversal order, a vertex will only continue propagating its state after receiving most or all of its increments. This normalizes state propagation, reducing the number of updates and increment generation for each affected vertex. For example, if a vertex continues traversing only after all its receiving edges connected to affected neighbor vertices have been visited, each affected vertex only needs to update its state once and generate an increment once. This reduces invalid and redundant computations during vertex state updates, improving update efficiency.
[0188] Furthermore, the graph processing device can identify multiple hot vertices from the graph and then store the vertex states of these hot vertices in a contiguous storage area. The threshold for the number of received increments for hot vertices is greater than or equal to a quantity threshold. When obtaining the vertex state of an affected vertex at either end of a selected edge, if the affected vertex is a hot vertex, its vertex state is retrieved from the contiguous storage area. If the affected vertex is not a hot vertex, its vertex state is retrieved from the memory area. During graph computation, most state propagation processes involve only a small subset of vertices that are affected and require state updates. This means that the percentage of vertices requiring state updates is small, and therefore, the vertex states of most vertices are not needed during graph computation. Merging the vertex states of hot vertices and storing them in a contiguous storage area allows a single cache line to store a larger number of vertex states required for graph computation. This means a single cache line can be used for most vertex state access processes during state propagation, making vertex state retrieval continuous and regular, thereby reducing data access overhead.
[0189] The order of the methods provided in the embodiments of this application can be adjusted appropriately, and the process can also be added or removed as appropriate. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and the embodiments of this application do not limit this.
[0190] The foregoing mainly describes the diagram processing method provided in the embodiments of this application from the perspective of the device. It is understood that, in order to achieve the above functions, the device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0191] This application embodiment can divide the device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.
[0192] Figure 12 This is a block diagram of a graph processing apparatus provided in an embodiment of this application. Exemplarily, the graph processing apparatus can be a graph processing device, or a chip or other combined device or component having the functions of the graph processing apparatus described above. When each functional module is divided according to its corresponding function, the graph processing apparatus 300 includes:
[0193] The first determining module 301 is used to determine the threshold number of received increments for each of the multiple affected vertices during state propagation when the graph changes, wherein the affected vertex is the vertex whose vertex state needs to be updated when the graph changes;
[0194] Traversal module 302 is used to perform at least one traversal process based on the threshold number of received increments for each affected vertex until the number of visits to each affected vertex reaches its respective threshold number of received increments, thereby obtaining multiple edges arranged in traversal order;
[0195] The state propagation module 303 is used to propagate the state based on the multiple edges arranged in the traversal order;
[0196] The traversal process includes: determining a root vertex from at least one active vertex among the plurality of affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex that satisfies a first stopping condition is reached, until all state propagation paths starting from the root vertex have been traversed.
[0197] The at least one active vertex includes: an initial vertex that has never been a root vertex and an affected vertex that has never been a root vertex and satisfies the first stopping condition. Among the at least one active vertex, the difference between the received increment threshold and the number of visits of the root vertex is the smallest. The initial vertex is the vertex directly affected when the graph changes. The first stopping condition includes: when the affected vertex is visited, the number of visits of the affected vertex is less than the received increment threshold, and / or when the affected vertex is visited, the affected vertex does not have any accessible neighbor affected vertices.
[0198] In conjunction with the above scheme, the state propagation module 303 is specifically used for: selecting the edge that needs to be propagated through state according to the arrangement order of the multiple edges; obtaining the vertex states of the affected vertices at both ends of the selected edge; and performing state propagation based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
[0199] Combining the above solutions, Figure 13 A block diagram of another diagram processing apparatus provided in an embodiment of this application, in Figure 12 Based on this, the device further includes:
[0200] The second determining module 304 is used to determine multiple hot vertices from the graph, wherein the threshold for the number of received increments of the hot vertices is greater than or equal to a quantity threshold.
[0201] Storage module 305 is used to store the vertex states of the plurality of hot vertices in a contiguous storage area;
[0202] The state propagation module 303 is specifically used for: when the affected vertex at either end of the selected edge is a hot vertex, obtaining the vertex state of the affected vertex at either end from the contiguous storage area; when the affected vertex at either end is a non-hot vertex, obtaining the vertex state of the affected vertex at either end from the memory area.
[0203] In conjunction with the above scheme, the first determining module 301 is specifically used to: traverse the state propagation path starting from each initial vertex, and stop traversing the current state propagation path when an affected vertex that meets the second stopping condition is reached, until all state propagation paths starting from each initial vertex have been traversed; determine the number of times the affected vertex is visited as the threshold for the number of received increments of the affected vertex; the second stopping condition includes: the affected vertex is another initial vertex, and / or the affected vertex does not have any accessible neighbor affected vertices when it is visited.
[0204] In conjunction with the above scheme, the first determining module 301 is specifically used to: traverse the edges on the state propagation path starting from the initial vertex, wherein the number of times the affected vertex is visited is the number of edges from the affected vertex's neighboring affected vertices to the affected vertex; the traversal module 302 is specifically used to: traverse the edges on the state propagation path starting from the root vertex.
[0205] In conjunction with the above scheme, the first determining module 301 is specifically used to: traverse the edges on the state propagation path starting from the initial vertex based on the traversal algorithm; the traversal module 302 is specifically used to: traverse the edges on the state propagation path starting from the root vertex based on the traversal algorithm, wherein the traversal algorithm includes: depth-first traversal algorithm and / or breadth-first traversal algorithm.
[0206] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 400 can be an image processing device, or a chip or functional module within an image processing device. Figure 14 As shown, the electronic device 400 includes a processor 401, a transceiver 402, and a communication line 403.
[0207] Among them, processor 401 is used to perform such as Figure 3 , Figure 4 , Figure 9 and Figure 10 In any step of the method embodiment shown, when performing processes such as sending and receiving data, the transceiver 402 and communication line 403 may be invoked to complete the corresponding operation.
[0208] Furthermore, the electronic device 400 may also include a memory 404. The processor 401, memory 404, and transceiver 402 can be connected via a communication line 403.
[0209] The processor 401 can be a processor, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 401 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.
[0210] Transceiver 402 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 402 can be a module, circuit, transceiver, or any device capable of enabling communication.
[0211] The transceiver 402 is mainly used for data transmission and reception, and may include a transmitter and a receiver to send and receive signals respectively; operations other than signal transmission and reception are implemented by the processor, such as information processing and calculation.
[0212] Communication line 403 is used to transmit information between the various components included in electronic device 400.
[0213] In one design, the processor can be viewed as a logic circuit, and the transceiver as an interface circuit.
[0214] Memory 404 is used to store instructions. These instructions can be computer programs.
[0215] The memory 404 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM). Memory 404 can also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices. It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0216] It should be noted that the memory 404 can exist independently of the processor 401, or it can be integrated with the processor 401. The memory 404 can be used to store instructions, program code, or some data, etc. The memory 404 can be located inside or outside the electronic device 400, without limitation. The processor 401 is used to execute the instructions stored in the memory 404 to implement the method provided in the above embodiments of this application.
[0217] In one example, processor 401 may include one or more processors, for example Figure 14 Processor 0 and processor 1 in the system.
[0218] As an optional implementation, the electronic device 400 includes multiple processors, for example, besides Figure 14 In addition to processor 401, it may also include processor 407.
[0219] As an optional implementation, the electronic device 400 also includes an output device 405 and an input device 406. For example, the input device 406 is a device such as a keyboard, mouse, microphone, or joystick, and the output device 405 is a device such as a display screen or speaker.
[0220] It should be noted that the electronic device 400 can be a chip system or... Figure 14 Devices with similar structures. The chip system can be composed of chips or include chips and other discrete components. Actions, terminology, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages used for interaction between devices in the embodiments of this application are merely examples; other names can be used in specific implementations without limitation. Furthermore, Figure 14 The structural composition shown does not constitute a limitation on the electronic device 400, except... Figure 14 In addition to the components shown, the electronic device 400 may include more than Figure 14 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.
[0221] The processor and transceiver described in this application can be implemented on integrated circuits (ICs), analog ICs, radio frequency integrated circuits, mixed-signal ICs, application-specific integrated circuits (ASICs), printed circuit boards (PCBs), electronic devices, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal-oxide semiconductors (CMOS), n-metal-oxide-semiconductor (NMOS), positive-channel metal-oxide semiconductors (PMOS), bipolar junction transistors (BJTs), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.
[0222] Figure 15This is a schematic diagram of a diagram processing apparatus provided in an embodiment of this application. This diagram processing apparatus is applicable to the scenarios shown in the above-described method embodiments. For ease of explanation, Figure 15 Only the main components of the processing device shown in the figure are illustrated, including a processor, memory, control circuitry, and input / output devices. The processor is primarily used to process communication protocols and data, execute software programs, and process the data within those programs. The memory is mainly used to store software programs and data. The control circuitry is primarily used for power supply and the transmission of various electrical signals. The input / output devices are primarily used to receive user input data and output data to the user.
[0223] When the image processing device is an image processing equipment, the control circuit can be a motherboard, the memory includes storage media such as hard disks, RAM, and ROM, and the processor can include a baseband processor and a central processing unit. The baseband processor is mainly used to process communication protocols and communication data, while the central processing unit is mainly used to control the entire image processing device, execute software programs, and process the data of the software programs. Input / output devices include displays, keyboards, and mice. The control circuit can further include or be connected to transceiver circuits or transceivers, such as network cable interfaces, for sending or receiving data or signals, such as data transmission and communication with other devices. Furthermore, it can also include an antenna for transmitting and receiving wireless signals for data / signal transmission with other devices.
[0224] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the methods described in the embodiments of this application.
[0225] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by a computer or an information processing device, using computer programs or instructions to control related hardware. The computer program or set of instructions can be stored in the computer-readable storage medium. When executed, the computer program or set of instructions can include the processes described in the above method embodiments. The computer-readable storage medium can be an internal storage unit of the server in any of the foregoing embodiments, such as the server's hard drive or memory. The computer-readable storage medium can also be an external storage device of the server, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the server. Further, the computer-readable storage medium can include both internal storage units and external storage devices of the server. The computer-readable storage medium is used to store the computer program or instructions and other programs and data required by the server. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0226] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0227] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0228] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0229] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0230] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0231] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.
[0232] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method of processing a graph, characterized by, The method includes: When the graph changes, a threshold for the number of received increments for each of the multiple affected vertices is determined during state propagation, wherein the affected vertices are those whose vertex states need to be updated when the graph changes. Based on the threshold number of received increments for each affected vertex, at least one traversal process is performed until the number of visits to each affected vertex reaches its respective threshold number of received increments, resulting in multiple edges arranged in traversal order. State propagation is performed based on the multiple edges arranged in the traversal order; The traversal process includes: determining a root vertex from at least one active vertex among the plurality of affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex that satisfies a first stopping condition is reached, until all state propagation paths starting from the root vertex have been traversed. The at least one active vertex includes: an initial vertex that has never been a root vertex and an affected vertex that has never been a root vertex and satisfies the first stopping condition. Among the at least one active vertex, the difference between the received increment threshold and the number of visits of the root vertex is the smallest. The initial vertex is the vertex directly affected when the graph changes. The first stopping condition includes: when the affected vertex is visited, the number of visits of the affected vertex is less than the received increment threshold, and / or when the affected vertex is visited, the affected vertex does not have any accessible neighbor affected vertices. The method further includes: determining multiple hot vertices from the graph, wherein the threshold for the number of received increments of the hot vertices is greater than or equal to a quantity threshold; The vertex states of the multiple hot vertices are stored in a contiguous storage area.
2. The method according to claim 1, characterized in that, The state propagation based on the multiple edges arranged in the traversal order includes: Select the edge that needs to be propagated through the state according to the order of the multiple edges; Get the vertex states of the affected vertices at both ends of the selected edge; State propagation is performed based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
3. The method according to claim 2, characterized in that, The process of obtaining the vertex states of the affected vertices at both ends of the selected edge includes: When the affected vertex at either end of the selected edge is a hot vertex, the vertex state of the affected vertex at either end is obtained from the contiguous storage area; When the affected vertex at either end is a non-hot vertex, the vertex state of the affected vertex at either end is obtained from the memory region.
4. The method according to any one of claims 1 to 3, characterized in that, Determining the threshold for the number of received increments for each of the multiple affected vertices during state propagation includes: The state propagation path starting from each initial vertex is traversed separately. When an affected vertex that satisfies the second stopping condition is visited, the traversal of the current state propagation path is stopped until all state propagation paths starting from each initial vertex have been traversed. The number of times the affected vertex is accessed is determined as the threshold for the number of received increments of the affected vertex; The second stopping condition includes: the affected vertex is another initial vertex, and / or the affected vertex does not have any accessible neighboring affected vertices when accessing the affected vertex.
5. The method according to claim 4, characterized in that, The step of traversing the state propagation path starting from each initial vertex includes: Traverse the edges on the state propagation path starting from the initial vertex, and the number of times the affected vertex is visited is the number of edges from the affected vertex's neighboring affected vertices to the affected vertex; The traversal of the state propagation path starting from the root vertex includes: Traverse the edges along the state propagation path starting from the root vertex.
6. The method according to claim 5, characterized in that, The traversal of the edges along the state propagation path starting from the initial vertex includes: The edges on the state propagation path starting from the initial vertex are traversed based on the traversal algorithm. The traversal of the edges along the state propagation path starting from the root vertex includes: The traversal algorithm is used to traverse the edges on the state propagation path starting from the root vertex. The traversal algorithm includes: depth-first traversal algorithm and / or breadth-first traversal algorithm.
7. A diagram processing apparatus, characterized in that, The device includes: The first determining module is used to determine the threshold number of received increments for each of the multiple affected vertices during state propagation when the graph changes, wherein the affected vertex is the vertex whose vertex state needs to be updated when the graph changes; The traversal module is used to perform at least one traversal process based on the threshold number of received increments for each affected vertex, until the number of visits to each affected vertex reaches its respective threshold number of received increments, thereby obtaining multiple edges arranged in traversal order; A state propagation module is used to propagate the state based on the multiple edges arranged in the traversal order; The traversal process includes: determining a root vertex from at least one active vertex among the plurality of affected vertices; traversing the state propagation path starting from the root vertex, stopping the traversal of the current state propagation path when an affected vertex that satisfies a first stopping condition is reached, until all state propagation paths starting from the root vertex have been traversed. The at least one active vertex includes: an initial vertex that has never been a root vertex and an affected vertex that has never been a root vertex and satisfies the first stopping condition. Among the at least one active vertex, the difference between the received increment threshold and the number of visits of the root vertex is the smallest. The initial vertex is the vertex directly affected when the graph changes. The first stopping condition includes: when the affected vertex is visited, the number of visits of the affected vertex is less than the received increment threshold, and / or when the affected vertex is visited, the affected vertex does not have any accessible neighbor affected vertices. The device further includes: The second determining module is used to determine multiple hot vertices from the graph, wherein the threshold for the number of received increments of the hot vertices is greater than or equal to a quantity threshold. A storage module is used to store the vertex states of the multiple hot vertices in a contiguous storage area.
8. The apparatus according to claim 7, characterized in that, The state propagation module is specifically used for: Select the edge that needs to be propagated through the state according to the order of the multiple edges; Get the vertex states of the affected vertices at both ends of the selected edge; State propagation is performed based on the selected edge and the vertex states of the affected vertices at both ends of the selected edge.
9. The apparatus according to claim 8, characterized in that, The state propagation module is specifically used for: When the affected vertex at either end of the selected edge is a hot vertex, the vertex state of the affected vertex at either end is obtained from the contiguous storage area; When the affected vertex at either end is a non-hot vertex, the vertex state of the affected vertex at either end is obtained from the memory region.
10. The apparatus according to any one of claims 7 to 9, characterized in that, The first determining module is specifically used for: The state propagation path starting from each initial vertex is traversed separately. When an affected vertex that satisfies the second stopping condition is visited, the traversal of the current state propagation path is stopped until all state propagation paths starting from each initial vertex have been traversed. The number of times the affected vertex is accessed is determined as the threshold for the number of received increments of the affected vertex; The second stopping condition includes: the affected vertex is another initial vertex, and / or the affected vertex does not have any accessible neighboring affected vertices when accessing the affected vertex.
11. The apparatus according to claim 10, characterized in that, The first determining module is specifically used for: Traverse the edges on the state propagation path starting from the initial vertex, and the number of times the affected vertex is visited is the number of edges from the affected vertex's neighboring affected vertices to the affected vertex; The traversal module is specifically used for: Traverse the edges along the state propagation path starting from the root vertex.
12. A diagram processing apparatus, characterized in that, The device includes: One or more processors; Memory, used to store one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors perform the method as described in any one of claims 1 to 6.
13. A computer-readable storage medium, characterized in that, It includes a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Graph processing memory access optimization method and system based on activeness layout
CN110737804A
Dynamic graph processing method based on FPGA
CN111104224A