Graph partitioning method and system

By determining and numbering the community structure during graph partitioning, the performance issues caused by cross-process communication in existing technologies are resolved, resulting in more efficient distributed system performance.

CN121350310APending Publication Date: 2026-01-16ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511398304.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-09-26
Publication Date
2026-01-16

AI Technical Summary

Technical Problem

Existing technologies result in a large amount of inter-process communication during graph partitioning, which reduces system performance.

Method used

By determining the community structure of the original graph and numbering the community identifiers, nodes are evenly distributed among processes in the distributed system to reduce cross-process communication.

Benefits of technology

By effectively utilizing the structural information between nodes, the overhead of cross-process data exchange and communication can be reduced, thereby improving the performance of distributed systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121350310A_ABST
    Figure CN121350310A_ABST
Patent Text Reader

Abstract

The invention provides a graph partitioning method and system, and the method comprises the steps: obtaining an original graph to be partitioned, enabling the original graph to comprise nodes and edges, enabling the nodes to represent an entity, enabling the edge between any two nodes to represent the correlation between any two nodes, determining a plurality of communities corresponding to the original graph, enabling one community to correspond to one community identifier, and enabling the community identifier to correspond to one community identifier; one community comprises a plurality of nodes, each community identifier is numbered, each node module is mapped to a corresponding worker according to the numbered identifier, the numbering processing is used for enabling the N workers to be balanced in the number of allocated communities, and the nodes in one community are allocated to the same worker. According to the method and the device, the structural information among the nodes in the original graph is effectively utilized, and a large amount of cross-worker communication is avoided, so that the cross-worker data exchange and communication overhead can be reduced, and the performance of a distributed system can be remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a graph partitioning method and system. Background Technology

[0002] Graph data structures, also known simply as graph data or graphs, are data structures composed of nodes (vertex) and edges (edges).

[0003] Graph partitioning refers to the process of dividing a graph into nodes or edges to accommodate the needs of distributed computing. Its core objective is to divide the graph into multiple subgraphs, balancing storage load and reducing communication costs across subgraphs.

[0004] In related technologies, graph partitioning is typically performed using node partitioning or edge partitioning. However, this often leads to extensive inter-process (worker) communication, significantly reducing system performance.

[0005] It should be noted that the above-mentioned related technologies are only information known to the inventor personally, and do not mean that the above information had entered the public domain before the application date of this specification, nor do they mean that it can be considered prior art in this specification. Summary of the Invention

[0006] This specification provides a graph partitioning method and system to avoid at least one of the above-mentioned technical problems.

[0007] Firstly, this specification provides a graph partitioning method applied to a distributed system comprising N processes, where N is an integer greater than 1. The method includes:

[0008] Obtain the original graph to be partitioned, wherein the original graph includes nodes and edges, nodes represent entities, and edges between any two nodes represent the association between the two nodes;

[0009] Multiple communities corresponding to the original graph are identified, wherein each community corresponds to a community identifier, and each community includes multiple nodes; and

[0010] Each community identifier is assigned a number, and each node is mapped to the corresponding process based on the numbered identifier. The numbering process is used to ensure that the N processes are evenly distributed across the number of communities, and that nodes in a community are assigned to the same process.

[0011] Secondly, this specification provides a graph partitioning system, including:

[0012] At least one storage medium stores at least one instruction set for partitioning the original graph to be partitioned;

[0013] At least one processor is communicatively connected to the at least one storage medium, wherein when the at least one processor is running, it reads the at least one instruction set and executes the method as described in the first aspect according to the instructions of the at least one instruction set.

[0014] Thirdly, this specification provides a computer-readable non-transitory storage medium, wherein the computer-readable non-transitory storage medium stores at least one instruction set, which is executed by at least one processor to implement the method as described in the first aspect.

[0015] As can be seen from the above technical solutions, the graph partitioning method and system provided in this specification, by numbering each community, ensures that the number of communities assigned to each worker is balanced, and that nodes in a community are assigned to the same worker. Therefore, by mapping the nodes in the original graph to the corresponding workers through the numbered identifiers, it is possible to ensure that the number of communities assigned to each worker is balanced. That is, nodes in the same community are assigned to the same worker, and different workers are partitioned into approximately the same number of communities. This effectively utilizes the structural information between nodes in the original graph, avoids a large amount of cross-worker communication, thereby reducing cross-worker data exchange and communication overhead, and thus significantly improving the performance of the distributed system.

[0016] Other functions of the map partitioning methods and systems provided in this specification will be partially listed in the following description. The inventive aspects of the map partitioning methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of the graph partitioning method provided in the embodiments of this specification;

[0019] Figure 2 This is a schematic diagram of the structure of the graph partitioning system provided in the embodiments of this specification;

[0020] Figure 3A schematic flowchart illustrating a graph partitioning method provided in one embodiment of this specification;

[0021] Figure 4 A schematic flowchart illustrating another embodiment of the graph partitioning method provided in this specification;

[0022] Figure 5 This is a schematic flowchart of a graph partitioning method provided in yet another embodiment of this specification. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0024] It should be understood that the terms “comprising” and “having”, and any variations thereof, in the embodiments of this specification are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0025] The term "and / or" in the embodiments of this specification describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0026] In the embodiments of this specification, the term "multiple" refers to two or more, and other quantifiers are similar.

[0027] The terms “first,” “second,” “third,” etc., used in this specification are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, in situations where implementation can proceed in an order other than those given in the embodiments illustrated or described in this specification.

[0028] As used in this specification, the term "unit / module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0029] To avoid at least one of the technical problems mentioned in the background section above, this specification proposes a technical concept developed through inventive effort: determining the community structure corresponding to the original graph to be partitioned, partitioning the original graph from the dimension of the community structure, and assigning the nodes in the original graph to the corresponding workers.

[0030] For example, firstly, for the original graph to be partitioned, the graph partitioning system can determine the community structure corresponding to the original graph. This community structure includes multiple communities. A community includes multiple nodes; a community can be understood as a set of nodes. Each community corresponds to a community identifier, allowing different communities to be distinguished through different identifiers.

[0031] Then, the graph partitioning system can number each community identifier so that each node in the original graph can be mapped to the corresponding worker using the numbered identifier.

[0032] It is worth noting that the numbering process is used to ensure that each worker is assigned a balanced number of communities, and that nodes in a community are assigned to the same worker.

[0033] The technical solution provided in this specification is based on the above-described technical concept. As can be seen from the description of the above technical concept, in the technical solution provided in this specification, each community is numbered to ensure a balanced distribution of the number of communities assigned to each worker, and nodes within a community are assigned to the same worker. Therefore, by mapping the nodes in the original graph to the corresponding workers through the numbered identifiers, a balanced distribution of the number of communities assigned to each worker can be achieved. That is, nodes within the same community are assigned to the same worker, and different workers are partitioned into approximately the same number of communities. This effectively utilizes the structural information between nodes in the original graph, avoids extensive cross-worker communication, thereby reducing cross-worker data exchange and communication overhead, and significantly improving the performance of the distributed system.

[0034] To facilitate readers' understanding of this manual, the application scenarios of this manual are introduced below.

[0035] The technical solutions provided in this specification are applicable to scenarios requiring graph partitioning. For example, the technical solutions provided in this specification can be applied to scenarios in the fields of social network analysis, recommendation systems, and knowledge graphs.

[0036] Taking the above social network analysis scenario as an example:

[0037] The nodes in the original graph can represent users, and the edges can represent interactive relationships such as following, forwarding, commenting, and private messaging.

[0038] The original graph may be very large, such as having 100 million users and 1 billion edges. Therefore, a single-machine system may not be able to store and compute such a large original graph. This could result in high computational complexity for global algorithms, or the need for low-latency responses in real-time recommendations and risk detection.

[0039] Therefore, graph partitioning and a distributed system can be used to implement this. For example, a graph partitioning system uses the graph allocation method provided in this specification to distribute the nodes in the original graph across multiple workers in the distributed system for parallel processing. One worker might process a subgraph of the original graph that includes some of the nodes.

[0040] It should be noted that the above examples are only used to illustrate the application scenarios to which the technical solutions in this specification can be applied, and should not be construed as limiting the application scenarios.

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of the graph partitioning method according to embodiments of this specification. The graph partitioning method of this specification can be applied to, for example... Figure 1 Scene 100 is shown.

[0042] like Figure 1 As shown, scenario 100 may include the original graph 101 to be partitioned. The original graph 101 includes multiple nodes and multiple edges. Based on the above analysis, it can be seen that the content represented by nodes and edges may differ for different application scenarios.

[0043] in addition, Figure 1 This is merely a illustrative display of some nodes and edges in the original Figure 101. It should not be interpreted as a limitation on the size of the original Figure 101.

[0044] Continue reading Figure 1 Similar to the example above, scenario 100 can include distributed system 102. Distributed system 102 includes N workers, where N is an integer greater than 1. For example, N can be 4, or a larger or smaller integer.

[0045] Using the graph partitioning method provided in this manual, each node in the original graph 101 can be assigned to the corresponding worker.

[0046] Figure 2 A hardware structure diagram of a graph partitioning system 200 according to an embodiment of this specification is shown. The graph partitioning system 200 can perform the graph partitioning method described in this specification. The graph partitioning method is described in other parts of this specification.

[0047] like Figure 2As shown, the graph partitioning system 200 may include at least one storage medium 203 and at least one processor 202. In some embodiments, the graph partitioning system 200 may also include a communication port 204 and an internal communication bus 201. The graph partitioning system 200 may also include I / O components 205.

[0048] The internal communication bus 201 can connect to different system components. For example, the internal communication bus 201 can connect to storage medium 203, processor 202, communication port 204, and I / O component 205.

[0049] I / O component 205 supports input / output between graph partitioning system 200 and other components.

[0050] Communication port 204 is used for data communication between the graph partitioning system 200 and the outside world. For example, communication port 204 can be used for data communication between the graph partitioning system 200 and a network. Communication port 204 can be a wired communication port or a wireless communication port.

[0051] Storage medium 203 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 2031, a read-only storage medium (ROM) 2032, or a random access storage medium (RAM) 2033. Storage medium 203 also includes at least one instruction set stored in the data storage device. The instruction set includes computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the graph partitioning method provided in this specification.

[0052] At least one processor 202 may be communicatively connected to at least one storage medium 203. The at least one processor 202 is used to execute the at least one instruction set described above. When the graph partitioning system 200 is running, the at least one processor 202 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the graph partitioning method provided in this specification. The processor 202 may execute all the steps included in the graph partitioning method. The processor 202 may be in the form of one or more processors. In some embodiments, the processor 202 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0053] For illustrative purposes only, only one processor 202 is shown in the accompanying drawings of the graph partitioning system 200. However, it should be noted that the graph partitioning system 200 may also include multiple processors. Therefore, the operation and / or method steps disclosed herein may be executed by one processor or by multiple processors in combination. For example, if the processor 202 of the graph partitioning system 200 is described in this specification as performing steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 202 (e.g., the first processor performs step A, the second processor performs step B, or the first and second processors jointly perform steps A and B).

[0054] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating a graph partitioning method provided in one embodiment of this specification. Wherein, Figure 3 The graph partitioning method shown can be applied to distributed systems with multiple workers. Furthermore, the execution entity of the graph partitioning method can be the graph partitioning system itself. For a description of graph partitioning systems, please refer to the example above; it will not be repeated here.

[0055] like Figure 3 As shown, the method includes the following steps S301 to S303:

[0056] S301: Obtain the original graph to be partitioned, wherein the original graph includes nodes and edges, nodes represent entities, and edges between any two nodes represent the association between any two nodes.

[0057] The original graph can be understood as a complete graph data structure that has not been processed and needs to be partitioned. The original graph includes nodes and edges. Nodes are the basic units in the original graph, representing entities such as users and products. Edges are line segments connecting two nodes, representing the associations or interactions between them.

[0058] For example, as can be seen from the above examples of social network analysis, nodes represent social users, and edges represent the interactive relationships between social users, such as following, forwarding, commenting, and private messaging.

[0059] For example, the nodes in the original graph represent social users, such as User1, User2, etc. The edges in the original graph represent the follow relationships between users, such as User1 following User2, etc.

[0060] The original graph may contain millions or even hundreds of millions of nodes and edges to form a large social network. Accordingly, the original graph can be called a social network graph.

[0061] Based on the above analysis, it can be seen that the graph partitioning method provided in this specification can be applied to different domains and scenarios. Therefore, when the graph partitioning method provided in this specification is applied to a social network analysis scenario, the original graph can be a graph data structure of a social network (such as a social network graph). When the graph partitioning method provided in this specification is applied to a recommendation scenario, the original graph can be a graph data structure of a recommendation system. When the graph partitioning method provided in this specification is applied to a knowledge graph scenario, the original graph can be a graph data structure of a knowledge graph.

[0062] In other words, the original graph can be one of the following: a graph data structure for a social network, a graph data structure for a recommendation system, or a graph data structure for a knowledge graph. The specific structure can be determined based on the domain, scenario, etc.

[0063] Furthermore, this embodiment does not limit the method of obtaining the original graph. Taking the social network analysis scenario as an example, the graph partitioning system can obtain the original graph from social network platforms, etc.

[0064] S302: Identify multiple communities corresponding to the original graph, where each community corresponds to a community identifier and each community includes multiple nodes.

[0065] A community can be understood as a set of nodes in a graph that are densely connected internally and sparsely connected externally, representing a set of nodes (including multiple nodes) with similar behaviors or attributes. The structure of each community is called the community structure. Taking social network analysis as an example, a community could be a circle of friends, interest groups, etc.

[0066] A community identifier can be understood as a unique identifier assigned to each community to distinguish them (such as commid=1, commid=2, etc.).

[0067] In other words, in this step, the graph partitioning system can partition the nodes in the original graph to obtain multiple communities. Each community corresponds to a set of nodes, which includes multiple nodes. Relatively speaking, nodes within a community are densely connected, while nodes between communities are sparsely connected.

[0068] It is worth noting that this embodiment does not limit the method by which the graph partitioning system determines multiple communities. For example, the graph partitioning system can use community detection algorithms (such as the Louvain algorithm, the Leiden algorithm, the Girvan-Newman algorithm, etc.) to determine multiple communities.

[0069] S303: Number each community identifier and map each node to the corresponding process according to the numbered identifier. The numbering process is used to ensure that the N processes are evenly distributed in terms of the number of communities, and that nodes in a community are assigned to the same process.

[0070] Numbering can be understood as reordering or mapping the identifiers of each community so that the reordered or mapped identifiers are more suitable for subsequent load balancing.

[0071] Modular mapping can be understood as a simple hash-based allocation strategy. It assigns nodes to N workers based on their assigned community identifiers. Nodes within the same community are assigned to the same worker, while nodes from different communities are assigned to different workers. The number of communities assigned to each of the N workers is approximately the same.

[0072] A worker can be understood as a unit of work (such as a server or compute node) in parallel computing, responsible for processing the subgraph containing multiple nodes assigned to it. In the example above, a community's set of nodes might be assigned to a worker. Correspondingly, that worker is responsible for processing the set of nodes assigned to its community.

[0073] Based on the analysis of S301 to S303 above, it can be seen that in this embodiment, by numbering each community, the numbering process ensures that each worker is assigned a balanced number of communities, and nodes in a community are assigned to the same worker. Therefore, by mapping the nodes in the original graph to the corresponding workers through the numbered identifiers, the number of communities assigned to each worker can be balanced. That is, nodes in the same community are assigned to the same worker, and different workers are partitioned into approximately the same number of communities. This effectively utilizes the structural information between nodes in the original graph, avoids a large amount of cross-worker communication, thereby reducing cross-worker data exchange and communication overhead, and thus significantly improving the performance of the distributed system.

[0074] To help readers gain a deeper understanding of the implementation principles of the graph partitioning method provided in this manual, the following is a combination of... Figure 4 The diagram partitioning method provided in this manual will be described in more detail.

[0075] in, Figure 4 This is a schematic flowchart illustrating another embodiment of the graph partitioning method provided in this specification. Figure 4 As shown, the method includes:

[0076] S401: Obtain the original graph to be partitioned, wherein the original graph includes nodes and edges, nodes represent entities, and edges between any two nodes represent the association between any two nodes.

[0077] It is understood that, in order to avoid tedious descriptions, this embodiment will not repeat the same or similar content as the examples above.

[0078] For example, regarding the understanding of S401, please refer to the description of S301 in the example above, which will not be repeated here.

[0079] S402: The original graph is processed using a community detection algorithm to obtain multiple communities. Each community corresponds to a community identifier and contains multiple nodes.

[0080] Similarly, for an understanding of S402, please refer to the description of S302 in the example above, which will not be repeated here.

[0081] S403: Determine the original identifier list corresponding to each node in the original graph based on each community identifier. In the original list, each node has its own corresponding node identifier, and the node identifier of any node is the community identifier of the community to which the node belongs.

[0082] The original identifier list can be understood as an array or mapping structure that records the "temporary identifier" of each node in the graph. Here, the identifier is equal to the community identifier of its community before it was numbered.

[0083] Correspondingly, a node identifier can be understood as the identifier corresponding to a node. In this step, the node identifier of a node is the community identifier of the community to which the node belongs before it is numbered.

[0084] For example, a community comprises multiple nodes. A community has a corresponding community identifier. For each node, the graph partitioning system can assign an identifier to that node to obtain its node identifier. Specifically, the graph partitioning system uses the community identifier of the community to which the node belongs as the node identifier for that node.

[0085] For example, if the community identifier of a node is 3, then the node identifier of the node can also be 3.

[0086] S404: Number the node identifiers in the original identifier list with consecutive integers to obtain a new identifier list, and map each node to the corresponding worker according to the node identifiers in the new identifier list.

[0087] The numbering of consecutive integers can be understood as remapping potentially sparse and discontinuous identifiers into consecutive integers starting from 0 (such as 0, 1, 2, 3...) to facilitate subsequent even distribution.

[0088] In this embodiment, by using consecutive integer numbering, the number of communities allocated to N processes can be balanced, and nodes within a community are assigned to the same process. That is, by using consecutive numbering and modulo mapping, the number of communities allocated to each worker can be balanced, avoiding overload of some workers. Furthermore, all nodes in the same community share the same identifier to be assigned to the same worker, avoiding cross-worker partitioning. This achieves higher load balancing and higher computational efficiency.

[0089] The new identifier list can be understood as a list of node identifiers generated after numbering, where each node's new identifier is a consecutive number within its community.

[0090] For example, to facilitate distinction, the identifiers in the original identifier list can be called the original identifiers, and the identifiers in the new identifier list can be called the new identifiers. Assume the original identifier list is [3,7,3,2,7,2]. As shown in Table 1, the original identifier of node A is 3, the original identifier of node B is 7, the original identifier of node C is 3, the original identifier of node D is 2, the original identifier of node E is 7, and the original identifier of node F is 2.

[0091] Table 1

[0092] node Original Identifier New logo A 3 1 B 7 2 C 3 1 D 2 0 E 7 2 F 2 0

[0093] The graph partitioning system can process the original identifier list [3,7,3,2,7,2] with consecutive integers to renumber them as consecutive integers starting from 0, resulting in the identifier list [1,2,1,0,2,0].

[0094] Referring to Table 1, the new identifier for node A is 1, the new identifier for node B is 2, the new identifier for node C is 1, the new identifier for node D is 0, the new identifier for node E is 2, and the new identifier for node F is 0.

[0095] Correspondingly, the graph partitioning system can map each node module to the corresponding worker based on the node identifiers in the new identifier list. For example, the graph partitioning system can map the node A module to the corresponding worker based on the new identifier 1 of node A.

[0096] In some embodiments, prior to S404, the graph partitioning system may perform deduplication on the node identifiers in the original identifier list to obtain a deduplicated identifier list.

[0097] Correspondingly, in S404, "the identifiers in the original identifier list are numbered consecutively by integers to obtain a new identifier list" can include: numbering the identifier list after deduplication to obtain a new identifier list.

[0098] Continuing with the example above, the original identifier list is [3,7,3,2,7,2]. The graph partitioning system can remove duplicates from the original identifier list [3,7,3,2,7,2], resulting in a unique identifier list with the value [3,7,2].

[0099] Accordingly, the graph partitioning system can renumber the deduplicated identifier list [3,7,2], such as by renumbering consecutive integers. For example, the original identifier 2 can be renumbered as 0; the original identifier 3 as 1; and the original identifier 7 as 2. Thus, the new identifier list can be obtained as [1,2,1,0,2,0].

[0100] In this embodiment, the graph segmentation system avoids duplicate numbering by first deduplicating and then numbering, thereby improving the efficiency of obtaining a new list of identifiers.

[0101] In some embodiments, the numbering process includes renumbering the identifiers in the original identifier list based on one of the frequency, size, number, or order of occurrence of the node identifiers in the original identifier list.

[0102] Continuing with the example above, the original identifier list is [3,7,3,2,7,2]. The graph partitioning system can remove duplicate identifiers from the original identifier list [3,7,3,2,7,2], resulting in a unique identifier list with the value [3,7,2].

[0103] Then, the graph partitioning system can sort the deduplicated identifier list [3,7,2] from smallest to largest to obtain the sorted identifier list [2,3,7].

[0104] Finally, the graph partitioning system can renumber the sorted identifier list [2,3,7] with consecutive integers starting from 0, resulting in a new identifier list [1,2,1,0,2,0].

[0105] The frequency of a node identifier in the original identifier list can be understood as the ratio of the number of times a certain node identifier appears in the original identifier list to the total number of nodes.

[0106] For example, a graph partitioning system can renumber the node identifiers in the original identifier list as consecutive integers starting from 0, based on descending (or ascending) frequency.

[0107] For implementation details regarding quantity, order of appearance, etc., please refer to the examples above; they will not be listed here again.

[0108] In this embodiment, the graph partitioning system renumbers the identifiers in the original identifier list based on factors such as frequency, size, quantity, and order of appearance. This allows for flexibility and diversity in renumbering, and also improves the efficiency of renumbering.

[0109] In some embodiments, "mapping each node module to the corresponding worker according to the node identifier in the new identifier list" in S404 may include the following steps 11 and 12:

[0110] Step 11: Based on the node identifier corresponding to any node in the new identifier list, perform a modulo operation on N to obtain the target worker identifier corresponding to any node.

[0111] Step 12: Assign any node to the worker corresponding to the target worker identifier.

[0112] For example, one worker corresponds to one worker identifier to distinguish different workers.

[0113] Correspondingly, the graph partitioning system can determine the worker identifier corresponding to the node based on the formula “worker_id(v)=new_groupid[v]%N”.

[0114] Where new_groupid[v] is the new identifier[v], worker_id(v) is the worker number assigned to the node with the new identifier v, and % is the modulo operation.

[0115] For example, suppose N = 3, and the 3 workers are worker0, worker1, and worker2, and the worker identifiers of the 3 workers are 0, 1, and 2 respectively.

[0116] Continuing with the example above, as shown in Table 2, nodes D and F are assigned to worker0, nodes A and C are assigned to worker1, and nodes B and E are assigned to worker2.

[0117] Table 2

[0118] worker node worker 0 D,F worker1 A,C worker2 B,E

[0119] Correspondingly, any worker can execute a distributed task based on the module mapping to the node of that worker.

[0120] Based on the above analysis, in the example above, the graph partitioning system mainly reorders and numbers the corresponding identifiers of the nodes (such as a new identifier list including the node identifiers corresponding to each node) as consecutive integers, and then assigns each node to the corresponding worker through a modulo algorithm.

[0121] In other embodiments, the graph partitioning system may also reorder and number the corresponding identifiers as consecutive integers from the community dimension, and then assign each node to the corresponding worker using a modulo algorithm.

[0122] For example, such as Figure 5 As shown, the graph partitioning method may include the following steps S501 to S504:

[0123] S501: Obtain the original graph to be partitioned, wherein the original graph includes nodes and edges, nodes represent entities, and edges between any two nodes represent the association between any two nodes.

[0124] S502: Identify multiple communities corresponding to the original graph, where each community corresponds to a community identifier and each community includes multiple nodes.

[0125] Similarly, this embodiment will not repeat the same or similar content as the examples above.

[0126] For example, regarding the understanding of S501 and S502, please refer to the relevant descriptions in the above examples (such as S301 and S302), which will not be repeated here.

[0127] S503: Perform consecutive integer numbering on each community identifier to obtain a new community identifier corresponding to each community.

[0128] Similarly, regarding the numbering of consecutive integers, although the example above deals with node identifiers, such as the original identifier list including the original identifiers corresponding to each node, while this step deals with community identifiers, the principle is similar, so please refer to the description in the example above.

[0129] For example, in the above example, there are 3 communities. Let's assume that the 3 communities are Community A, Community B, and Community C, and that the community identifiers of the 3 communities are 3, 7, and 2, respectively.

[0130] Similarly, the graph partitioning system can renumber and sort the community identifiers 3, 7, and 2 to obtain new community identifiers for each of the three communities. For example, the new community identifier for community A is 1, the new community identifier for community B is 2, and the new community identifier for community C is 0.

[0131] S504: Assign node identifiers to each node, where the node identifier of a node is the new community identifier of its community. For the first node in the original graph, map the first node to the corresponding process according to the new community identifier of the first node; or, for each node in any community, map each node in any community to the corresponding process according to the new community identifier of any community.

[0132] In other words, S504 can include two implementation methods:

[0133] Implementation Method 1: Assign node identifiers to each node. The node identifier of a node is the new community identifier of its community. For the first node in the original graph, the first node is mapped to the corresponding process according to the new community identifier of the first node.

[0134] Implementation Method 2: For each node in any community, map each node in any community to the corresponding process based on the new community identifier of any community.

[0135] The description of implementation method 1 is as follows:

[0136] This implementation is closer to the one in the example above. In this implementation, the graph partitioning system first reorders and numbers the corresponding identifiers as consecutive integers from the community dimension, and then performs modulo operation from the node dimension to assign each node to the corresponding worker.

[0137] Continuing with the example above, a community may include multiple nodes. For instance, community A may contain one thousand nodes, and these one thousand nodes have corresponding node identifiers, such as the new community identifier. For example, the node identifier for all one thousand nodes in community A is the new community identifier 1.

[0138] The first node can be any node in the original graph. For example, the first node can be any node among the one thousand nodes in community A. Similarly, the node identifier of the first node is also the new community identifier 1 of community A.

[0139] Therefore, the graph partitioning system can assign the first node to the corresponding worker by performing a modulo operation on 1 versus N. See the description in the example above for details, which will not be repeated here.

[0140] The description of implementation method 2 is as follows:

[0141] This implementation differs significantly from the one described in the previous example. In this implementation, the graph partitioning system first reorders and numbers the corresponding identifiers as consecutive integers from the community dimension, and then performs modulo operations from the community dimension to assign each node to the corresponding worker on a community-by-community basis.

[0142] Continuing with the example above, for one thousand nodes in community A, the graph partitioning system can use the new community identifier 1 of community A to perform a modulo operation on N to determine the worker corresponding to community A, and then assign the one thousand nodes in community A to that worker.

[0143] Based on the analysis in S501 to S504 above, it can be seen that in this embodiment, numbering can be performed from the community dimension to assign each node in the original graph to the corresponding worker, thereby achieving diversity and flexibility in graph partitioning. Furthermore, numbering from the community dimension improves the efficiency of numbering, which in turn improves the efficiency of graph partitioning.

[0144] It is worth noting that the above examples are merely illustrative of possible implementations of the graph partitioning method in this specification, and should not be construed as limiting the implementation of the graph partitioning method in this specification. For example, based on the above technical concept, some of the technical features described above can be combined to obtain new embodiments; new technical features can be added to the above examples to obtain new embodiments; some technical features can be removed from the above examples to obtain new embodiments; some technical features in the above examples can be replaced with other technical features; some technical features and their order in the above examples can be adjusted to obtain new embodiments, and so on, which will not be listed here.

[0145] Based on the above technical concept, this specification also provides a computer-readable non-transitory storage medium storing at least one instruction set, which, when executed by a processor, implements the steps of the graph partitioning method described in this specification.

[0146] In some possible implementations, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on the graph partitioning system 200, the program code causes the graph partitioning system 200 to perform the steps of the graph partitioning method described in this specification. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the graph partitioning system 200. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on graph partitioning system 200, partially on graph partitioning system 200, as a standalone software package, partially on graph partitioning system 200 and partially on a remote graph partitioning system, or entirely on remote graph partitioning system 200.

[0147] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user-related information involved in the technical solutions of this specification all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0148] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0149] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0150] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0151] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

[0152] Every patent, patent application, publication of a patent application, and other material cited herein, such as articles, books, specifications, publications, documents, and literature (excluding any related historical examination documents), is cited for all purposes relevant to this document, including, for example, in the specification and claims herein. However, in the event of any inconsistency or conflict between the descriptions, definitions, and / or terms used in the foregoing and those used herein, the descriptions, definitions, and / or terms used herein shall prevail.

[0153] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A graph partitioning method, applied to a distributed system comprising N processes, N being an integer greater than 1, the method comprising: obtaining an original graph to be partitioned, wherein the original graph comprises nodes and edges, a node representing an entity and an edge between any two nodes representing an association between the two nodes; determining a plurality of communities corresponding to the original graph, wherein a community corresponds to a community identifier and a community comprises a plurality of nodes; and numbering the community identifiers and mapping the nodes to corresponding processes according to the numbered identifiers, wherein the numbering is used to balance the number of communities allocated to the N processes and the nodes in a community are allocated to the same process. The numbering the community identifiers and mapping the nodes to corresponding processes according to the numbered identifiers comprises: determining, according to the community identifiers, a list of original identifiers corresponding to the nodes in the original graph, wherein in the original list, each node has a node identifier corresponding to the community identifier of the community to which the node belongs; numbering the node identifiers in the original list of identifiers to obtain a new list of identifiers, and mapping the nodes to corresponding processes according to the node identifiers in the new list of identifiers. The mapping the nodes to corresponding processes according to the new list of identifiers comprises: performing a modulo operation on N according to the node identifier corresponding to the node in the new list of identifiers to obtain a target process identifier corresponding to the node; and allocating the node to the process corresponding to the target process identifier. The numbering the node identifiers in the original list of identifiers comprises: numbering the node identifiers in the original list of identifiers with consecutive integers.

2. The method of claim 1, wherein, The method further comprises: performing a deduplication process on the node identifiers in the original list of identifiers to obtain a deduplicated list of identifiers; and numbering the identifiers in the original list of identifiers to obtain a new list of identifiers, comprising: numbering the deduplicated list of identifiers to obtain the new list of identifiers. The numbering comprises renumbering the identifiers in the original list of identifiers based on one of frequency, size, number, and order of appearance of the node identifiers in the original list of identifiers. The original graph is one of a graph data structure of a social network, a graph data structure of a recommendation system, and a graph data structure of a knowledge graph.

3. The method of claim 2, wherein, Any process performs a corresponding distributed task based on the nodes mapped to the process by modulo. The numbering the community identifiers and mapping the nodes to corresponding processes according to the numbered identifiers comprises: numbering the community identifiers with consecutive integers to obtain new community identifiers corresponding to the community identifiers, respectively; and ​ 4. The method of claim 2, wherein, ​ ​ 5. The method of claim 2, wherein, ​ ​ ​ 6. The method of claim 2, wherein, ​ 7. The method of claim 1, wherein, ​ 8. The method of claim 1, wherein, ​ 9. The method of claim 1, wherein, ​ ​ allocating a node identifier to each node, wherein the node identifier of a node is a new community identifier of a community to which the node belongs, and mapping each node in the original graph to a corresponding process according to the new community identifier of the node; or mapping each node in an arbitrary community to a corresponding process according to the new community identifier of the community.

10. A graph partitioning system, comprising: at least one storage medium storing at least one instruction set for partitioning an original graph to be partitioned; at least one processor communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set when executed and performs the method according to any one of claims 1 to 9 as instructed by the at least one instruction set.