Graph data partitioning method and apparatus, graph data processing method and apparatus, and device and medium
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-12
- Publication Date
- 2026-08-13
Smart Images
Figure CN2025082097_13082026_PF_FP_ABST
Abstract
Description
Graph data partitioning methods, graph data processing methods, devices, equipment and media
[0001] This application is based on and claims priority to Chinese Invention Application No. 202510137663.6, filed on February 7, 2025, entitled "Graph Data Partitioning Method, Graph Data Processing Method, Apparatus, Device and Medium". Technical Field
[0002] This application applies to the field of data processing, and particularly relates to a graph data partitioning method, graph data processing method, apparatus, equipment and medium. Background Technology
[0003] In the era of big data, graph data, as the carrier of complex information networks, is growing in scale, leading to a surge in demand for graph data processing and analysis. Traditional large-scale graph computing systems use parallel methods of data partitioning, integrating the resources of multiple computers to complete graph computing tasks. Considering the high maintenance and construction costs, a series of single-machine-based large-scale graph processing systems have emerged. These systems utilize disks as an extension of memory to process large graphs, employing either a vertex-centric computing model (which confines information transfer during computation to nodes) or a subgraph-centric computing model (which transfers information between subgraphs). However, despite the progress made by single-machine-based large-scale graph processing systems in processing large graph data, unreasonable subgraph partitioning still exists when using a subgraph-centric computing model and utilizing disks as external storage for graph data to achieve large-scale graph analysis tasks on a single machine. This unreasonable partitioning affects memory utilization efficiency, increases disk I / O overhead, and reduces the parallelism of computing tasks, thus impacting the overall performance of graph processing. Therefore, optimizing graph data partitioning strategies to improve graph processing performance has become an urgent problem to be solved. Summary of the Invention
[0004] In view of this, embodiments of this application provide a graph data partitioning method, graph data processing method, apparatus, device, and medium to address the problem of how to optimize graph data partitioning strategies to improve graph processing performance.
[0005] In a first aspect, embodiments of this application provide a graph data partitioning method, the graph data partitioning method comprising:
[0006] Obtain target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data;
[0007] Based on the target node, the target graph data is divided to obtain at least two initial subgraphs, and edge nodes in each initial subgraph are determined. Each initial subgraph includes the target node, and the edge node is a node that is included by at least two initial subgraphs.
[0008] Obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal that the number of nodes of the edge nodes is within a preset value range, to obtain an updated subgraph;
[0009] For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to be executed until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
[0010] Secondly, embodiments of this application provide a graph data processing method. After obtaining all updated subgraphs using the graph data partitioning method of the first aspect described above, the graph data processing method includes:
[0011] In one round of iterative processing, all updated subgraphs in the partition of the disk are read and transferred to the memory buffer.
[0012] The memory capacity occupied by the update subgraph in the memory buffer is monitored, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area.
[0013] According to the preset graph algorithm, the update subgraphs in the memory execution area are processed in parallel, and the state of each update subgraph is updated according to the processing procedure.
[0014] For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then the processing of the updated subgraph is stopped, the processed updated subgraph is obtained, and the processed updated subgraph is written to the disk.
[0015] Thirdly, embodiments of this application provide a graph data partitioning apparatus, the graph data partitioning apparatus comprising:
[0016] The acquisition module is used to acquire target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data;
[0017] The partitioning module is used to partition the target graph data according to the target node to obtain at least two initial subgraphs and determine the edge nodes in each initial subgraph, wherein each initial subgraph includes the target node and the edge node refers to a node that is included by at least two initial subgraphs;
[0018] An adjustment module is used to obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal of the number of nodes of the edge nodes being within a preset value range, so as to obtain an updated subgraph.
[0019] The first loop module is used to, for any updated subgraph, if the storage capacity required to store the updated subgraph is not within a preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to execute until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
[0020] Fourthly, embodiments of this application provide a graph data processing apparatus. After obtaining all updated subgraphs using the graph data partitioning method of the first aspect described above, the graph data processing apparatus includes:
[0021] The read module is used to read all the updated subgraphs in the partition of the disk in one round of iteration processing, and transfer all the updated subgraphs in the partition to the memory buffer.
[0022] The scheduling module is used to monitor the memory capacity occupied by the update subgraph in the memory buffer, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area.
[0023] The running module is used to perform parallel processing on the update subgraphs in the memory execution area according to a preset graph algorithm, and to update the state of each update subgraph according to the processing procedure.
[0024] The write module is used to stop processing the update subgraph if the state of the update subgraph reaches a preset termination state for any update subgraph, obtain the processed update subgraph, and write the processed update subgraph to the disk.
[0025] Fifthly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the graph data partitioning method as described in the first aspect, or the graph data processing method as described in the second aspect.
[0026] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the graph data partitioning method as described in the first aspect, or the graph data processing method as described in the second aspect.
[0027] The beneficial effects of this application embodiment compared with the prior art are as follows: The graph data partitioning method of this application determines the node with the largest out-degree from the target graph data as the target node, partitions the target graph data according to the target node to obtain at least two initial subgraphs, determines the edge nodes in each initial subgraph, and each initial subgraph includes the target node. An edge node is a node included by at least two initial subgraphs. The number of edge nodes in all initial subgraphs is obtained. With the number of edge nodes being within a preset value range as the target, the node paths corresponding to the edge nodes in at least one initial subgraph are adjusted to obtain an updated subgraph. For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, the updated subgraph is designated as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned until all updated subgraphs are obtained.
[0028] In one iteration of the graph data processing method of this application, the updated subgraph read from the disk partition is transferred to the memory buffer. When the content capacity occupied by the updated subgraph in the memory buffer reaches a preset threshold, the updated subgraph in the memory buffer is transferred to the memory execution area. According to the preset graph algorithm, the updated subgraph in the memory execution area is processed in parallel, and the state of each updated subgraph is updated according to the processing process. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, the processing of the updated subgraph is stopped, the processed updated subgraph is obtained, and the processed updated subgraph is written to the disk.
[0029] Specifically, by selecting the node with the highest degree as the target node and using it as a benchmark to partition the target graph data, an initial subgraph is obtained. This ensures the connectivity between the initial subgraphs and avoids separating highly connected nodes, thereby reducing communication overhead in subsequent graph processing. Based on this, the initial subgraphs are adjusted according to the number of edge nodes in all initial subgraphs to obtain updated subgraphs. This achieves load balancing of nodes or edges among the updated subgraphs while maintaining connectivity. Furthermore, the updated subgraphs are adjusted based on their required storage capacity, optimizing storage and processing performance, thus optimizing the overall graph data partitioning strategy. Therefore, when parallel graph processing is performed on the updated subgraphs obtained from the above partitioning based on the preset graph algorithm, the good connectivity between the updated subgraphs greatly promotes the information synchronization speed between them. During processing, information can be transmitted more quickly between the updated subgraphs, improving the convergence efficiency of graph processing and thus enhancing graph processing performance. Attached Figure Description
[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 is a schematic diagram of an application environment for a graph data partitioning method and a graph data processing method provided in Embodiment 1 of this application;
[0032] Figure 2 is a flowchart illustrating a graph data partitioning method provided in Embodiment 2 of this application;
[0033] Figure 3 is a schematic diagram of target image data provided in Embodiment 2 of this application;
[0034] Figure 4 is a schematic diagram of an updated sub-graph provided in Embodiment 2 of this application;
[0035] Figure 5 is a flowchart illustrating a graph data partitioning method provided in Embodiment 3 of this application;
[0036] Figure 6 is a schematic flowchart of a graph data processing method provided in Embodiment 4 of this application;
[0037] Figure 7 is a schematic diagram of a system architecture for graph data processing in a single-machine multi-core environment provided in Embodiment 4 of this application;
[0038] Figure 8 is a flowchart illustrating a graph data processing method provided in Embodiment 5 of this application;
[0039] Figure 9 is a schematic diagram of a graph data partitioning device provided in Embodiment 6 of this application;
[0040] Figure 10 is a schematic diagram of a graph data processing device provided in Embodiment 7 of this application;
[0041] Figure 11 is a schematic diagram of the structure of a computer device provided in Embodiment 8 of this application. Detailed Implementation
[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0043] To illustrate the technical solution of this application, specific embodiments are described below.
[0044] This application provides a graph data partitioning method and a graph data processing method, applicable to the application environment shown in Figure 1. The server and client communicate, with the server providing graph data partitioning and processing services, and the client triggering graph data partitioning and processing tasks to the server. The client includes, but is not limited to, devices such as handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server-side computer device can be implemented using a dedicated server or a server cluster consisting of multiple servers.
[0045] Referring to Figure 2, which is a flowchart illustrating a graph data partitioning method according to Embodiment 2 of this application, this graph data partitioning method is applied to the server in Figure 1. The server connects to the client to obtain the target graph data sent by the client. As shown in Figure 2, the graph data partitioning method may include the following steps:
[0046] Step S201: Obtain the target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data.
[0047] Step S202: Based on the target node, divide the target graph data to obtain at least two initial subgraphs, and determine the edge nodes in each initial subgraph.
[0048] In this embodiment, the target graph data can refer to the graph structure data to be partitioned, the target node can refer to the node with the largest out-degree in the target graph data, the initial subgraph can refer to the graph data obtained by partitioning the target graph data according to the target node, each initial subgraph includes the target node, and the edge node refers to the node that is included by at least two initial subgraphs.
[0049] Specifically, for any node in the target graph data, determine the number of edges emanating from that node. Select the node with the most emanating edges from the target graph data as the target node. Divide the target graph data starting from the target node to obtain at least two initial subgraphs. Each initial subgraph includes the target node. For any initial subgraph, determine the edge nodes in the initial subgraph that are included by at least two initial subgraphs. Since each initial subgraph includes the target node, the target node is an edge node in each initial subgraph.
[0050] Step S203: Obtain the number of edge nodes in all initial subgraphs. With the goal of the number of edge nodes being within a preset range, adjust the node paths corresponding to the edge nodes in at least one initial subgraph to obtain an updated subgraph.
[0051] In this embodiment, a node path can refer to a path that includes edge nodes, and updating a subgraph can refer to graph data in which the number of edge nodes is within a preset range after adjusting the node paths corresponding to edge nodes in at least one initial subgraph.
[0052] Specifically, obtain the number of edge nodes in all initial subgraphs. If the number of edge nodes in all initial subgraphs is within a preset range, then each initial subgraph can be determined as an updated subgraph.
[0053] If the number of edge nodes in all initial subgraphs is not within a preset range, then for any initial subgraph, the edge nodes in that initial subgraph are determined. For any edge node in that initial subgraph other than the target node, a node path containing that edge node is determined in that initial subgraph. For any node path, that node path is randomly added to any initial subgraph other than the initial subgraph that includes that edge node. Edge nodes are merged to reduce the number of all edge nodes. When the number of all edge nodes is within a preset range, an updated subgraph is obtained.
[0054] Step S204: For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, then the updated subgraph is converted to target graph data, and the process returns to the step of determining the out-degree of each node in the target graph data until all updated subgraphs are obtained.
[0055] In this embodiment, storage capacity can refer to the storage space required for updating the subgraph in the storage device, and can be measured using basic units of data storage capacity such as kilobytes (KB) and megabytes (MB).
[0056] For example, if the preset capacity range is KB-MB, for any updated subgraph, if the storage capacity required to store the updated subgraph is in the KB-MB range, then no further partitioning of the updated subgraph is required. If the storage capacity required to store the updated subgraph is not in the KB-MB range, then further partitioning of the updated subgraph is required, and the updated subgraph becomes the new target graph data. Then, the steps S201 to S203 above are returned to be executed until the storage capacity required to store all updated subgraphs is in the KB-MB range, all updated subgraphs are obtained, and the obtained updated subgraphs are partitioned and stored on the disk.
[0057] For example, referring to Figure 3, which is a schematic diagram of target graph data provided in Embodiment 2 of this application. As shown in Figure 3, the target graph data has a total of 10 nodes, namely node 0, node 1, node 2, node 3, node 4, node 5, node 6, node 7, node 8 and node 9. The out-degree of node 0 is 5, the out-degree of node 1 is 2, the out-degree of node 2 is 2, the out-degree of node 3 is 3, the out-degree of node 4 is 3, the out-degree of node 5 is 3, the out-degree of node 6 is 3, the out-degree of node 7 is 1, the out-degree of node 8 is 1, and the out-degree of node 9 is 3. Therefore, the node with the largest out-degree in the target graph data can be determined as node 0, that is, node 0 is determined as the target node.
[0058] Referring to Figure 4, which is a schematic diagram of an updated subgraph provided in Embodiment 2 of this application, the updated subgraph is obtained by dividing the target graph data shown in Figure 3. As shown in Figure 4, the target graph data shown in Figure 3 is divided into 5 updated subgraphs, namely updated subgraph B1, updated subgraph B2, updated subgraph B3, updated subgraph B4 and updated subgraph B5. All updated subgraphs are interconnected. The number of edge nodes in all updated subgraphs is within a preset value range. The difference in the total number of corresponding nodes between all updated subgraphs is less than a preset value. Furthermore, the storage capacity required to store each updated subgraph is also within a preset capacity range.
[0059] In this embodiment, the node with the highest out-degree is selected as the target node, and the target graph data is partitioned based on this node to obtain an initial subgraph. This ensures the connectivity between the initial subgraphs and avoids separating highly connected nodes, thereby reducing communication overhead in subsequent graph processing. Based on this, the initial subgraphs are adjusted according to the number of edge nodes in all initial subgraphs to obtain an updated subgraph. This achieves load balancing of nodes or edges between the updated subgraphs while maintaining connectivity. Furthermore, the updated subgraphs are adjusted based on the required storage capacity, optimizing storage and processing performance, thus optimizing the overall graph data partitioning strategy. Therefore, when the updated subgraphs obtained from the above partitioning are processed in parallel using a preset graph algorithm, the good connectivity between the updated subgraphs greatly promotes the information synchronization speed between them. During processing, information can be transmitted more quickly between the updated subgraphs, improving the convergence efficiency of graph processing and thus enhancing graph processing performance.
[0060] Referring to Figure 5, which is a flowchart illustrating a graph data partitioning method provided in Embodiment 3 of this application, as shown in Figure 5, step S203 above, with the goal of ensuring that the number of edge nodes is within a preset numerical range, adjusts the node paths corresponding to the edge nodes in at least one initial subgraph to obtain an updated subgraph, and may include the following steps:
[0061] Step S501: For any initial subgraph, determine the edge nodes in the initial subgraph; for any edge node in the initial subgraph other than the target node, determine the node path containing the edge node in the initial subgraph.
[0062] Step S502: For any node path, add the node path to any initial subgraph that includes edge nodes, except for the initial subgraph, to obtain the subgraph to be evaluated.
[0063] Step S503: Determine the total number of nodes in each subgraph to be evaluated and the number of edge nodes in all subgraphs to be evaluated. For any subgraph to be evaluated, calculate the difference between the total number of nodes in the subgraph to be evaluated and the total number of nodes in each subgraph to be evaluated except the subgraph to be evaluated.
[0064] Step S504: If the difference between all the subgraphs to be evaluated is less than the preset value, and the number of edge nodes in all the subgraphs to be evaluated is within the preset value range, then each subgraph to be evaluated is determined to be an updated subgraph.
[0065] In this embodiment, the subgraph to be evaluated can refer to the graph data after updating the initial subgraph based on the node paths of the edge nodes.
[0066] Optionally, after calculating the difference between the total number of nodes in any subgraph to be evaluated and the total number of nodes in every other subgraph to be evaluated in step S503 above, the method further includes:
[0067] If there is a subgraph to be evaluated whose difference is not less than a preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated will be set as the initial subgraph, and the process of determining the edge nodes in the initial subgraph will be repeated until each subgraph to be evaluated is determined to be an updated subgraph.
[0068] That is, if there is a difference in the subgraph to be evaluated that is not less than a preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated is set as a new initial subgraph, and the above steps S501 to S503 are returned to be executed until each subgraph to be evaluated is determined to be an updated subgraph.
[0069] For example, in the process of determining the updated subgraph, if the target graph data is divided to obtain two initial subgraphs, namely initial subgraph A1 and initial subgraph A2, the number of edge nodes in the two initial subgraphs is 4. Initial subgraph A1 has two edge nodes, namely node 0 and node 2, and initial subgraph A2 has two edge nodes, namely node 0 and node 2. Among them, node 0 is the target node.
[0070] In the initial subgraph A1, node 2 corresponds to a node path: node 2 → node 4. In the initial subgraph A2, node 2 corresponds to a node path: node 2 → node 5. For the edge node 2 in the initial subgraph A2, if the node path corresponding to node 2: node 2 → node 5 is added to another initial subgraph A1 that includes node 2, after the addition, we get subgraph A1 and subgraph A2 to be evaluated. Subgraph A1 to be evaluated has two paths starting from node 2, namely node 2 → node 4 and node 2 → node 5. Subgraph A2 to be evaluated does not have node 2 and the path corresponding to node 2. At this time, the number of edge nodes in the two subgraphs to be evaluated is 2.
[0071] If the preset value range is [1,3], then the number of edge nodes in the two subgraphs to be evaluated is within the preset value range [1,3]. If the difference in the total number of corresponding nodes in the two subgraphs to be evaluated is also less than the preset value, then both subgraphs to be evaluated are updated subgraphs.
[0072] In this embodiment, by adjusting the node paths corresponding to edge nodes in at least one initial subgraph with the goal of ensuring that the number of edge nodes is within a preset range and that the differences between all subgraphs to be evaluated are less than a preset value, an updated subgraph is obtained. This maintains the load balance between updated subgraphs while ensuring connectivity between them, optimizes the storage and processing performance of the subgraphs, and improves the convergence efficiency of graph processing when performing graph processing based on the updated subgraphs, thereby improving the performance of graph processing.
[0073] Referring to Figure 6, which is a flowchart illustrating a graph data processing method according to Embodiment 4 of this application, as shown in Figure 6, after obtaining all updated subgraphs using a graph data partitioning method, the graph data processing method may include the following steps:
[0074] In step S601, during one round of iterative processing, all updated subgraphs in the partitions of the disk are read and transferred to the memory buffer.
[0075] Step S602: Monitor the memory capacity occupied by the updated subgraph in the memory buffer. When the memory capacity reaches a preset threshold, transfer the updated subgraph in the memory buffer to the memory execution area.
[0076] Step S603: According to the preset graph algorithm, the update subgraphs in the memory execution area are processed in parallel, and the state of each update subgraph is updated according to the processing.
[0077] Step S604: For any update subgraph, if the state of the update subgraph reaches a preset termination state, stop processing the update subgraph, obtain the processed update subgraph, and write the processed update subgraph to the disk.
[0078] In this embodiment, the memory buffer is used to temporarily store data read from external devices or data sources, the memory execution area is used to store the program instructions being executed and related data, and the preset graph algorithm can refer to a pre-set algorithm for processing the updated subgraph. For example, the preset graph algorithm can be an algorithm for traversing, updating and querying the updated subgraph.
[0079] The state of an updated subgraph can refer to its processing status. For example, the state of an updated subgraph can include an active state, a waiting-to-be-computed state, a computing state, and a converged state. The first two states indicate that the updated subgraph is on disk, while the remaining states indicate that the updated subgraph is in memory. Each updated subgraph is initially in the active state, meaning it is waiting to be read into memory. When the updated subgraph is transferred to a memory buffer or memory execution area, its state is in the waiting-to-be-computed state. When the updated subgraph is processed according to a preset graph algorithm, its state is in the computing state. When the updated subgraph converges during processing, its state is in the converged state.
[0080] The preset termination state can refer to the state that the updated subgraph should reach when processing the updated subgraph is stopped. For example, if it is necessary to detect whether the updated subgraph has converged by the state of the updated subgraph, the preset termination state can be set to the state corresponding to the convergence of the updated subgraph. For example, when the state of the updated subgraph is the converged state, the preset termination state is reached.
[0081] This application addresses the processing of all updated subgraphs corresponding to the target graph data. The overall approach follows a synchronous model. In one round of iterative processing, all updated subgraphs in the partitions read from the disk are calculated and converged. After synchronization, the next round of iterative processing begins. That is, the calculations between updated subgraphs are performed asynchronously, and the synchronization is controlled by the partition where the updated subgraph is located.
[0082] Specifically, a round of iterative processing can be divided into three periods: the read period, the run period, and the write period. These three periods operate in a pipelined manner. During the read period, since the storage capacity of the updated subgraphs obtained by partitioning the target graph data using the aforementioned graph data partitioning method is within a preset capacity range, for example, controlled at the KB-MB level, an asynchronous I / O library can be used to randomly read all the updated subgraphs from the disk partition. The read updated subgraphs are then transferred to the memory buffer, or directly transferred to the memory execution area for processing, thus achieving parallel processing of graph processing and I / O operations. If the read updated subgraphs are transferred to the memory buffer, further processing of the memory buffer is also required. The memory usage of the updated subgraph is monitored. When the memory usage reaches a preset threshold, the updated subgraph in the memory buffer is transferred to the memory execution area for processing. During runtime, the updated subgraphs in the memory execution area are processed in parallel according to the preset graph algorithm. For any updated subgraph, the state of each node in the updated subgraph is updated according to the processing procedure. Based on the state of all nodes in the updated subgraph, the state of the updated subgraph is updated. If the state of any updated subgraph reaches a preset termination state or the state of the updated subgraph cannot be changed further, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained. During the write phase, the processed updated subgraph is written to the disk.
[0083] For example, referring to Figure 7, which is a schematic diagram of a system architecture for graph data processing in a single-machine multi-core environment provided in Embodiment 4 of the application. As shown in Figure 7, the system includes a read module, an executor, a write module, a disk, and a control module. The control module includes a scheduler, a state manager, and a configuration manager. The system memory is divided into two parts: a buffer and a runtime, namely, a memory buffer and a memory execution area. All updated subgraph partitions obtained by partitioning the target graph data based on the above graph data partitioning method are stored on the disk.
[0084] The system also provides application programming interfaces (APIs). Before processing graph data, users can set preset graph algorithms for processing updated subgraphs based on the APIs, and set algorithm parameters and parallel parameters based on the configuration manager.
[0085] The overall process for graph data processing based on this system architecture can be summarized as follows:
[0086] In one round of iterative processing, the read module reads all the updated subgraphs from the disk partitions and transfers them to the memory buffer. The scheduler monitors the memory capacity occupied by the updated subgraphs in the memory buffer and, when a preset threshold is reached, transfers the updated subgraphs in the memory buffer to the memory execution area. The executor processes the updated subgraphs in the memory execution area in parallel according to a preset graph algorithm. For any updated subgraph, the state manager updates the state of each node in the updated subgraph according to the processing procedure and updates the state of the updated subgraph based on the state of all nodes in the updated subgraph. If the state of any updated subgraph reaches a preset termination state, processing of the updated subgraph stops, and the processed updated subgraph is obtained. The write module then writes the processed updated subgraph to the disk.
[0087] In this embodiment of the application, when the updated subgraphs obtained by the above division are processed in parallel based on the preset graph algorithm, the good connectivity between the updated subgraphs greatly promotes the information synchronization speed between the updated subgraphs. During the processing, information can be transmitted between the updated subgraphs more quickly, which improves the convergence efficiency of graph processing and thus improves the performance of graph processing.
[0088] Referring to Figure 8, which is a flowchart illustrating a graph data processing method provided in Embodiment 5 of this application, as shown in Figure 8, the state of each node includes an active state and an inactive state. The active state indicates that the node needs to be scheduled and processed in the current round, and the inactive state indicates that the node does not need to be scheduled and processed in the current round. In step S604 above, for any updated subgraph, if the state of the updated subgraph reaches a preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained. This may include the following steps:
[0089] Step S801: For any updated subgraph, determine the state of each node in the updated subgraph;
[0090] Step S802: If the state of each node in the updated subgraph is inactive, then the state of the updated subgraph has reached the preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained.
[0091] In this embodiment, the state of each node includes an active state and an inactive state. The active state indicates that the node needs to be scheduled and processed in the current round, and the inactive state indicates that the node does not need to be scheduled and processed in the current round.
[0092] For any updated subgraph, if the state of each node in the updated subgraph is inactive, that is, if no further processing is needed for any node in the updated subgraph in the current round, then the state of the updated subgraph is determined to have reached the preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained.
[0093] In this embodiment, when the state of each node in the updated subgraph is inactive, the state of the updated subgraph is determined to have reached a preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained. This avoids unnecessary calculations and iterations and improves the efficiency of graph processing.
[0094] Corresponding to the graph data partitioning method in the above embodiments, Figure 9 shows a structural block diagram of the graph data partitioning apparatus provided in Embodiment Six of this application. The graph data partitioning apparatus is applied to the server in Figure 1. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0095] Referring to Figure 9, the data partitioning device includes:
[0096] The acquisition module 91 is used to acquire target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data;
[0097] The partitioning module 92 is used to partition the target graph data according to the target node to obtain at least two initial subgraphs and determine the edge nodes in each initial subgraph, wherein each initial subgraph includes the target node and the edge node refers to a node that is included by at least two initial subgraphs;
[0098] The adjustment module 93 is used to obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node path corresponding to the edge node in at least one initial subgraph with the goal that the number of nodes of the edge node is within a preset value range, so as to obtain an updated subgraph.
[0099] The first loop module 94 is used to, for any updated subgraph, if the storage capacity required to store the updated subgraph is not within a preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to be executed until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
[0100] Optionally, the adjustment module 93 includes:
[0101] The first determining unit is configured to, for any initial subgraph, determine the edge nodes in the initial subgraph, and for any edge node in the initial subgraph other than the target node, determine the node path in the initial subgraph that includes the edge node;
[0102] The first update unit is used to add the node path to any initial subgraph that includes the edge node, except for the initial subgraph, for any given node path, to obtain the subgraph to be evaluated.
[0103] The calculation unit is used to determine the total number of nodes in each subgraph to be evaluated and the number of edge nodes in all subgraphs to be evaluated. For any subgraph to be evaluated, the difference between the total number of nodes in the subgraph to be evaluated and the total number of nodes in each subgraph to be evaluated other than the subgraph to be evaluated is calculated.
[0104] The second determining unit is used to determine each subgraph to be evaluated as an updated subgraph if the difference between all the calculated subgraphs to be evaluated is less than a preset value, and the number of edge nodes in all subgraphs to be evaluated is within the range of the preset value.
[0105] Optionally, the adjustment module 93 also includes:
[0106] The second loop unit is used to, if there is a subgraph to be evaluated whose difference is not less than the preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated is the initial subgraph, and the step of determining the edge nodes in the initial subgraph is returned to be executed until each subgraph to be evaluated is determined to be an updated subgraph.
[0107] Corresponding to the graph data processing method in the above embodiments, FIG10 shows a structural block diagram of the graph data processing apparatus provided in Embodiment 7 of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0108] Referring to Figure 10, after obtaining all updated subgraphs using the graph data partitioning method, the graph data partitioning device includes:
[0109] The read module 1001 is used to read all the updated subgraphs in the partition of the disk in one round of iterative processing, and transfer all the read updated subgraphs in the partition to the memory buffer;
[0110] The scheduling module 1002 is used to monitor the memory capacity occupied by the update subgraph in the memory buffer, and when the memory capacity reaches a preset threshold, transfer the update subgraph in the memory buffer to the memory execution area.
[0111] The running module 1003 is used to perform parallel processing on the update subgraphs in the memory execution area according to a preset graph algorithm, and to update the state of each update subgraph according to the processing procedure.
[0112] The write module 1004 is used to, for any update subgraph, if the state of the update subgraph reaches a preset termination state, stop processing the update subgraph, obtain the processed update subgraph, and write the processed update subgraph to the disk.
[0113] Optionally, module 1003 includes:
[0114] The node state management unit is used to update the state of each node in any updated subgraph according to the processing procedure.
[0115] The graph state management unit is used to update the state of the updated subgraph based on the state of all nodes in the updated subgraph.
[0116] Optionally, write module 1004, which includes:
[0117] The third determining unit is used to determine the state of each node in any updated subgraph;
[0118] The fourth determining unit is configured to determine that the state of the updated subgraph has reached the preset termination state if the state of each node in the updated subgraph is the inactive state, stop processing the updated subgraph, and obtain the processed updated subgraph.
[0119] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram is shown in Figure 11. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a readable storage medium, and a database. The internal memory provides an environment for the operation of the operating system and the readable storage medium. The database of the computer device is used to store target graph data and update subgraphs. The network interface of the computer device is used to communicate with external terminals via a network connection. When executed by the processor, the readable storage medium implements a graph data partitioning method or a graph data processing method.
[0120] In one embodiment, a computer device is provided, including a first memory, a first processor, and a readable storage medium stored in the first memory and operable on the first processor. When the first processor executes the readable storage medium, it implements the steps of the graph data partitioning method in the above embodiments, such as steps S201-S204 shown in FIG2, or the steps shown in FIG3 to FIG5. To avoid repetition, these steps will not be described again here. Alternatively, when the first processor executes the readable storage medium, it implements the functions of each module / unit in the above embodiments of the graph data partitioning device, such as the functions of the acquisition module 91, partitioning module 92, adjustment module 93, and first loop module 94 shown in FIG9. To avoid repetition, these functions will not be described again here. In one embodiment, a computer device is provided, including a second memory, a second processor, and a readable storage medium stored on the second memory and operable on the second processor. When the second processor executes the readable storage medium, it implements the steps of the graph data processing method in the above embodiments, such as steps S601-S604 shown in FIG. 6, or the steps shown in FIG. 7 to FIG. 8. To avoid repetition, these steps will not be described again here. Alternatively, when the second processor executes the readable storage medium, it implements the functions of each module / unit in the above embodiments of the graph data processing device, such as the functions of the read module 1001, scheduling module 1002, running module 1003, and write module 1004 shown in FIG. 10. To avoid repetition, these functions will not be described again here.
[0121] In one embodiment, one or more readable storage media storing computer-readable instructions are provided. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps of the graph data partitioning method in the above embodiments, such as steps S201-S204 shown in FIG2, or the steps shown in FIG3 to FIG5. To avoid repetition, these steps will not be described again here. Alternatively, when the first processor executes the readable storage medium, it performs the functions of each module / unit in the above-described graph data partitioning device embodiments, such as the functions of the acquisition module 91, partitioning module 92, adjustment module 93, and first loop module 94 shown in FIG9. To avoid repetition, these steps will not be described again here. In one embodiment, one or more readable storage media storing computer-readable instructions are provided. When executed by one or more processors, these computer-readable instructions cause the processors to perform the steps of the graph data processing method described in the above embodiments, such as steps S601-S604 shown in FIG. 6, or the steps shown in FIG. 7 and 8. To avoid repetition, these steps will not be described again here. Alternatively, when a second processor executes the readable storage medium, it performs the functions of each module / unit in the graph data processing device embodiments described above, such as the functions of the read module 1001, scheduling module 1002, running module 1003, and write module 1004 shown in FIG. 10. To avoid repetition, these functions will not be described again here. The readable storage medium in this embodiment includes non-volatile readable storage media and volatile readable storage media.
[0122] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by instructing related hardware through a readable storage medium. The readable storage medium can be stored in a non-volatile computer-readable storage medium, which, when executed, can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0123] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0124] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for partitioning graph data, wherein, The graph data partitioning method includes: Obtain target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data; Based on the target node, the target graph data is divided to obtain at least two initial subgraphs, and edge nodes in each initial subgraph are determined. Each initial subgraph includes the target node, and the edge node is a node that is included by at least two initial subgraphs. Obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal that the number of nodes of the edge nodes is within a preset value range, to obtain an updated subgraph; For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to be executed until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
2. The graph data partitioning method according to claim 1, wherein, The step of adjusting the node paths corresponding to the edge nodes in at least one initial subgraph to obtain an updated subgraph, with the goal of ensuring that the number of nodes at the edge nodes is within a preset numerical range, includes: For any initial subgraph, determine the edge nodes in the initial subgraph; for any edge node in the initial subgraph other than the target node, determine the node path in the initial subgraph that includes the edge node. For any node path, add the node path to any initial subgraph that includes the edge node, except for the initial subgraph, to obtain the subgraph to be evaluated; Determine the total number of nodes in each subgraph to be evaluated and the number of edge nodes in all subgraphs to be evaluated. For any subgraph to be evaluated, calculate the difference between the total number of nodes in the subgraph to be evaluated and the total number of nodes in each subgraph to be evaluated except the subgraph to be evaluated. If the calculated difference values for all subgraphs to be evaluated are less than a preset value, and the number of edge nodes in all subgraphs to be evaluated is within the preset value range, then each subgraph to be evaluated is determined to be an updated subgraph.
3. The graph data partitioning method according to claim 2, wherein, After calculating the difference between the total number of nodes in any subgraph to be evaluated and the total number of nodes in every other subgraph to be evaluated, the method further includes: If there is a difference in the subgraph to be evaluated that is not less than the preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated is designated as the initial subgraph, and the step of determining the edge nodes in the initial subgraph is returned to be executed until each subgraph to be evaluated is determined to be an updated subgraph.
4. A graph data processing method, wherein, After obtaining all updated subgraphs using the graph data partitioning method as described in any one of claims 1 to 3, the graph data processing method includes: In one round of iterative processing, all updated subgraphs in the partition of the disk are read and transferred to the memory buffer. The memory capacity occupied by the update subgraph in the memory buffer is monitored, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area. According to the preset graph algorithm, the update subgraphs in the memory execution area are processed in parallel, and the state of each update subgraph is updated according to the processing procedure. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then the processing of the updated subgraph is stopped, the processed updated subgraph is obtained, and the processed updated subgraph is written to the disk.
5. The graph data processing method according to claim 4, wherein, The step of updating the state of each updated subgraph according to the processing procedure includes: For any updated subgraph, the state of each node in the updated subgraph is updated according to the processing procedure. The state of the updated subgraph is updated based on the state of all nodes in the updated subgraph.
6. The graph data processing method according to claim 5, wherein, Each node's state includes an active state and an inactive state. The active state indicates that the node needs to be scheduled for processing in the current round, and the inactive state indicates that the node does not need to be scheduled for processing in the current round. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then processing of the updated subgraph is stopped, and a processed updated subgraph is obtained, including: For any updated subgraph, determine the state of each node in the updated subgraph; If the state of each node in the updated subgraph is the inactive state, then the state of the updated subgraph is determined to have reached the preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained.
7. A graph data partitioning device, wherein, The graph data partitioning device includes: The acquisition module is used to acquire target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data; The partitioning module is used to partition the target graph data according to the target node to obtain at least two initial subgraphs and determine the edge nodes in each initial subgraph, wherein each initial subgraph includes the target node and the edge node refers to a node that is included by at least two initial subgraphs; An adjustment module is used to obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal of the number of nodes of the edge nodes being within a preset value range, so as to obtain an updated subgraph. The first loop module is used to, for any updated subgraph, if the storage capacity required to store the updated subgraph is not within a preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to execute until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
8. A graph data processing apparatus, wherein, After obtaining all updated subgraphs using the graph data partitioning method as described in any one of claims 1 to 3, the graph data processing apparatus includes: The read module is used to read all the updated subgraphs in the partition of the disk in one round of iteration processing, and transfer all the updated subgraphs in the partition to the memory buffer. The scheduling module is used to monitor the memory capacity occupied by the update subgraph in the memory buffer, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area. The running module is used to perform parallel processing on the update subgraphs in the memory execution area according to a preset graph algorithm, and to update the state of each update subgraph according to the processing procedure. The write module is used to stop processing the update subgraph if the state of the update subgraph reaches a preset termination state for any update subgraph, obtain the processed update subgraph, and write the processed update subgraph to the disk.
9. A computer device comprising a first memory, a first processor, and a readable storage medium stored in the first memory and operable on the first processor, wherein, When the first processor executes the readable storage medium, it performs the following steps: Obtain target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data; Based on the target node, the target graph data is divided to obtain at least two initial subgraphs, and edge nodes in each initial subgraph are determined. Each initial subgraph includes the target node, and the edge node is a node that is included by at least two initial subgraphs. Obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal that the number of nodes of the edge nodes is within a preset value range, to obtain an updated subgraph; For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to be executed until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
10. The computer device according to claim 9, wherein, The step of adjusting the node paths corresponding to the edge nodes in at least one initial subgraph to obtain an updated subgraph, with the goal of ensuring that the number of nodes at the edge nodes is within a preset numerical range, includes: For any initial subgraph, determine the edge nodes in the initial subgraph; for any edge node in the initial subgraph other than the target node, determine the node path in the initial subgraph that includes the edge node. For any node path, add the node path to any initial subgraph that includes the edge node, except for the initial subgraph, to obtain the subgraph to be evaluated; Determine the total number of nodes in each subgraph to be evaluated and the number of edge nodes in all subgraphs to be evaluated. For any subgraph to be evaluated, calculate the difference between the total number of nodes in the subgraph to be evaluated and the total number of nodes in each subgraph to be evaluated except the subgraph to be evaluated. If the calculated difference values for all subgraphs to be evaluated are less than a preset value, and the number of edge nodes in all subgraphs to be evaluated is within the preset value range, then each subgraph to be evaluated is determined to be an updated subgraph.
11. The computer device according to claim 10, wherein, After calculating the difference between the total number of nodes in any subgraph to be evaluated and the total number of nodes in every other subgraph to be evaluated, the method further includes: If there is a difference in the subgraph to be evaluated that is not less than the preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated is designated as the initial subgraph, and the step of determining the edge nodes in the initial subgraph is returned to be executed until each subgraph to be evaluated is determined to be an updated subgraph.
12. A computer device comprising a second memory, a second processor, and a readable storage medium stored in the second memory and operable on the second processor, wherein, After obtaining all updated subgraphs using the graph data partitioning method as described in any one of claims 1 to 3, the second processor executes the following steps when running the readable storage medium: In one round of iterative processing, all updated subgraphs in the partition of the disk are read and transferred to the memory buffer. The memory capacity occupied by the update subgraph in the memory buffer is monitored, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area. According to the preset graph algorithm, the update subgraphs in the memory execution area are processed in parallel, and the state of each update subgraph is updated according to the processing procedure. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then the processing of the updated subgraph is stopped, the processed updated subgraph is obtained, and the processed updated subgraph is written to the disk.
13. The computer device according to claim 12, wherein, The step of updating the state of each updated subgraph according to the processing procedure includes: For any updated subgraph, the state of each node in the updated subgraph is updated according to the processing procedure. The state of the updated subgraph is updated based on the state of all nodes in the updated subgraph.
14. The computer device according to claim 13, wherein, Each node's state includes an active state and an inactive state. The active state indicates that the node needs to be scheduled for processing in the current round, and the inactive state indicates that the node does not need to be scheduled for processing in the current round. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then processing of the updated subgraph is stopped, and a processed updated subgraph is obtained, including: For any updated subgraph, determine the state of each node in the updated subgraph; If the state of each node in the updated subgraph is the inactive state, then the state of the updated subgraph is determined to have reached the preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained.
15. One or more readable storage media storing computer-readable instructions, wherein, When the computer-readable instructions are executed by one or more processors, the one or more processors cause the processors to perform the following steps: Obtain target graph data, determine the out-degree of each node in the target graph data, and determine the node with the largest out-degree as the target node from the target graph data; Based on the target node, the target graph data is divided to obtain at least two initial subgraphs, and edge nodes in each initial subgraph are determined. Each initial subgraph includes the target node, and the edge node is a node that is included by at least two initial subgraphs. Obtain the number of nodes of edge nodes in all initial subgraphs, and adjust the node paths corresponding to edge nodes in at least one initial subgraph with the goal that the number of nodes of the edge nodes is within a preset value range, to obtain an updated subgraph; For any updated subgraph, if the storage capacity required to store the updated subgraph is not within the preset capacity range, then the updated subgraph is set as the target graph data, and the step of determining the out-degree of each node in the target graph data is returned to be executed until all updated subgraphs are obtained, wherein the updated subgraphs are used for partitioned storage on the disk.
16. The readable storage medium according to claim 15, wherein, The step of adjusting the node paths corresponding to the edge nodes in at least one initial subgraph to obtain an updated subgraph, with the goal of ensuring that the number of nodes at the edge nodes is within a preset numerical range, includes: For any initial subgraph, determine the edge nodes in the initial subgraph; for any edge node in the initial subgraph other than the target node, determine the node path in the initial subgraph that includes the edge node. For any node path, add the node path to any initial subgraph that includes the edge node, except for the initial subgraph, to obtain the subgraph to be evaluated; Determine the total number of nodes in each subgraph to be evaluated and the number of edge nodes in all subgraphs to be evaluated. For any subgraph to be evaluated, calculate the difference between the total number of nodes in the subgraph to be evaluated and the total number of nodes in each subgraph to be evaluated except the subgraph to be evaluated. If the calculated difference values for all subgraphs to be evaluated are less than a preset value, and the number of edge nodes in all subgraphs to be evaluated is within the preset value range, then each subgraph to be evaluated is determined to be an updated subgraph.
17. The readable storage medium according to claim 16, wherein, After calculating the difference between the total number of nodes in any subgraph to be evaluated and the total number of nodes in every other subgraph to be evaluated, the method further includes: If there is a difference in the subgraph to be evaluated that is not less than the preset value, or if the number of edge nodes in all subgraphs to be evaluated is not within the preset value range, then for any subgraph to be evaluated, the subgraph to be evaluated is designated as the initial subgraph, and the step of determining the edge nodes in the initial subgraph is returned to be executed until each subgraph to be evaluated is determined to be an updated subgraph.
18. One or more readable storage media storing computer-readable instructions, wherein, After obtaining all updated subgraphs using the graph data partitioning method as described in any one of claims 1 to 3, when the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps: In one round of iterative processing, all updated subgraphs in the partition of the disk are read and transferred to the memory buffer. The memory capacity occupied by the update subgraph in the memory buffer is monitored, and when the memory capacity reaches a preset threshold, the update subgraph in the memory buffer is transferred to the memory execution area. According to the preset graph algorithm, the update subgraphs in the memory execution area are processed in parallel, and the state of each update subgraph is updated according to the processing procedure. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then the processing of the updated subgraph is stopped, the processed updated subgraph is obtained, and the processed updated subgraph is written to the disk.
19. The readable storage medium according to claim 18, wherein, The step of updating the state of each updated subgraph according to the processing procedure includes: For any updated subgraph, the state of each node in the updated subgraph is updated according to the processing procedure. The state of the updated subgraph is updated based on the state of all nodes in the updated subgraph.
20. The readable storage medium according to claim 19, wherein, Each node's state includes an active state and an inactive state. The active state indicates that the node needs to be scheduled for processing in the current round, and the inactive state indicates that the node does not need to be scheduled for processing in the current round. For any updated subgraph, if the state of the updated subgraph reaches a preset termination state, then processing of the updated subgraph is stopped, and a processed updated subgraph is obtained, including: For any updated subgraph, determine the state of each node in the updated subgraph; If the state of each node in the updated subgraph is the inactive state, then the state of the updated subgraph is determined to have reached the preset termination state, the processing of the updated subgraph is stopped, and the processed updated subgraph is obtained.