Method and apparatus for processing graph data

By employing parallel processing and merging techniques within a device cluster, the communication bottlenecks and high complexity issues encountered during community subgraph generation were resolved, achieving efficient community subgraph generation.

CN119719423BActive Publication Date: 2026-01-06ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411777623.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-04
Publication Date
2026-01-06
Estimated Expiration
2044-12-04

AI Technical Summary

Technical Problem

When processing massive graph data, existing technologies struggle to efficiently generate community subgraphs, especially in hot communities where the large number of edge information leads to communication congestion and high processing time complexity.

Method used

The second set of devices in the device cluster collects and generates partial subgraphs in parallel, and the target device merges these partial subgraphs to generate a community subgraph. The computing devices in the device cluster are used for distributed processing, and the stored point and edge information is distributed to reduce communication bottlenecks and processing time.

Benefits of technology

It improves the efficiency of generating community subgraphs, reduces the risk of communication congestion and processing time complexity, and enables efficient analysis of large-scale graph data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119719423B_ABST
    Figure CN119719423B_ABST
Patent Text Reader

Abstract

Embodiments of the present specification provide a kind of graph data processing method and equipment cluster.The method is used to generate the community subgraph corresponding to target community from original graph, original graph includes multiple point edge information, and the multiple point edge information is stored in the first equipment set of equipment cluster scatteredly.In the method, equipment cluster determines multiple target point edge information belonging to target community in original graph, and determines M storage equipment storing the multiple target point edge information in the first equipment set;The multiple target point edge information is collected from the M storage equipment by second equipment set to generate N partial subgraph;The N partial subgraph is collected from second equipment set by target equipment, and it is merged to generate the community subgraph corresponding to target community;Wherein, the multiple target point edge information is collected in parallel by at least part of equipment in second equipment set, and N partial subgraph is generated in parallel by at least part of equipment in second equipment set.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method and device cluster for processing graph data. Background Technology

[0002] In today's data-driven era, graph data has become a core research subject in numerous fields, including but not limited to social network analysis, bioinformatics, recommender systems, information retrieval and text analysis, cybersecurity, and graph neural network learning. However, understanding and analyzing the inherent structure of massive graph data presents a significant challenge. Against this backdrop, community clustering (community detection) techniques have emerged, offering an effective solution for graph data analysis.

[0003] Building upon community clustering, community subgraphs for specific communities can be generated from the original graph, simplifying the data structure and allowing the analysis to focus on specific communities, thus improving the efficiency of data analysis. This is a crucial step in data analysis. Therefore, researching efficient methods for generating community subgraphs is of great significance for advancing the development of graph data analysis.

[0004] The information in the background section is merely information known only to the inventor and does not imply that such information had entered the public domain before the date of this application, nor does it imply that it can be considered prior art in this disclosure. Summary of the Invention

[0005] This specification provides a method for processing graph data and a device cluster that can improve the efficiency of generating community subgraphs.

[0006] Firstly, this specification provides a graph data processing method applied to a device cluster, used to generate a community subgraph corresponding to a target community from an original graph. The original graph includes multiple vertex-edge information, which is distributed and stored in a first device set within the device cluster. The method includes: determining multiple target vertex-edge information belonging to the target community in the original graph, and identifying M storage devices in the first device set to store the multiple target vertex-edge information, where M is an integer greater than 1; collecting the multiple target vertex-edge information from the M storage devices through a second device set in the device cluster, and generating N partial subgraphs based on the multiple target vertex-edge information, wherein the multiple target vertex-edge information is collected in parallel by at least some devices in the second device set, and the N partial subgraphs are generated in parallel by at least some devices in the second device set, where N is an integer less than M; and collecting the N partial subgraphs from the second device set through a target device in the device cluster, and merging the N partial subgraphs to generate the community subgraph corresponding to the target community.

[0007] In some embodiments, the number of computing devices in the second device set is greater than N, and the second device set is divided into K subsets, where K is an integer greater than 1, and the number of devices in the i-th subset is P. i P i >P i+1 P k =N;

[0008] The step of collecting the multiple target point edge information from the M storage devices through the second device set in the device cluster, and generating N partial subgraphs based on the multiple target point edge information, includes:

[0009] The first operation is performed in parallel by the computing devices in the first subset to obtain P1 intermediate subgraphs. The first operation includes: collecting partial point-edge information from the multiple target point-edge information, and generating intermediate subgraphs based on the collected partial point-edge information.

[0010] Following the order of i from 2 to K, the second operation is performed in parallel on each computing device in the i-th subset to obtain P. i A subgraph, the second operation includes: collecting P... i-1 The collected intermediate subgraphs are selected from a set of intermediate subgraphs, and then merged to generate new intermediate subgraphs.

[0011] The P generated by each computing device in the Kth subset k There are 1 intermediate subgraph, which serve as the N partial subgraphs.

[0012] In some embodiments, the step of performing the first operation in parallel through the computing devices in the first subset to obtain P1 intermediate subgraphs includes:

[0013] The M storage devices are divided into P1 device groups;

[0014] Determine the first correspondence between P1 computing devices in the first subset and the P1 device groups; and

[0015] Based on the first correspondence, the first operation is performed in parallel by P1 computing devices in the first subset to obtain the P1 intermediate subgraphs, wherein the P1 computing devices collect the target point edge information from the corresponding device group when performing the first operation.

[0016] In some embodiments, dividing the M storage devices into P1 device groups includes:

[0017] Based on the distribution of the multiple target point edge information in the M storage devices and P1, the M storage devices are divided into P1 device groups, wherein the number of target point edge information stored in each of the P1 device groups satisfies a preset quantity balance condition.

[0018] In some embodiments, the second operation is performed in parallel by the computing devices in the i-th subset to obtain P. i The intermediate subgraphs include:

[0019] Divide the computing devices in the (i-1)th subset into P i One equipment group;

[0020] Determine P in the i-th subset i A computing device and the P i The second correspondence between each equipment group; and

[0021] Based on the second correspondence, through P in the i-th subset i Each computing device executes the second operation in parallel to obtain P. i An intermediate subgraph, wherein the P i Each computing device collects the partial intermediate subgraph from its corresponding device group when performing the second operation.

[0022] In some embodiments, the division of computing devices in the (i-1)th subset into P i A group of devices, including:

[0023] Based on the number P of computing devices in the (i-1)th subset i-1 and the P i The computing devices in the (i-1)th subset are divided into the P... i A group of devices, wherein the P i The number of computing devices in each device group meets the preset quantity balance condition.

[0024] In some embodiments, the number of computing devices in the second device set is equal to N, and the step of collecting the plurality of target point edge information from the M storage devices through the second device set in the device cluster, and generating N partial subgraphs based on the plurality of target point edge information, includes:

[0025] The following operations are performed in parallel by N computing devices in the second device set to obtain the N partial subgraphs: collecting partial point edge information from the multiple target point edge information, and generating partial subgraphs based on the collected partial point edge information.

[0026] In some embodiments, each point edge information includes: information about an edge in the original graph and information about the two nodes connected by that edge.

[0027] In some embodiments, merging the N partial subgraphs to generate a community subgraph corresponding to the target community includes:

[0028] One of the N partial subgraphs is taken as the first partial subgraph, and the remaining partial subgraphs are added to the subgraph set. The community subgraph is obtained by performing N-1 updates on the first partial subgraph, wherein in each update:

[0029] Determine a second subgraph from the set of subgraphs that has at least one node in common with the first subgraph; and

[0030] Based on the common nodes between the first subgraph and the second subgraph, the vertex and edge information in the second subgraph is updated in the first subgraph, and the second subgraph is deleted from the subgraph set.

[0031] In some embodiments, the step of collecting the multiple target point edge information from the M storage devices through a second set of devices in the device cluster, and generating N partial subgraphs based on the multiple target point edge information, includes:

[0032] If the number of edge information of the multiple target points is greater than a preset number, the multiple edge information of the multiple target points is collected from the M storage devices through the second set of devices in the device cluster, and N partial subgraphs are generated based on the multiple edge information of the multiple target points.

[0033] In some embodiments, the method further includes: when the number of edge information of the plurality of target points is less than or equal to the preset number, collecting the edge information of the plurality of target points from the M storage devices through the target device, and generating the community subgraph based on the edge information of the plurality of target points.

[0034] Secondly, this specification also provides a device cluster including multiple computing devices. Each computing device includes: at least one storage medium storing at least one instruction set for processing graph data; and at least one processor communicatively connected to the at least one storage medium, wherein, when the device cluster is running, the processors in the multiple computing devices read the corresponding instruction set and implement the method as described in the first aspect according to the instructions of the instruction set.

[0035] The methods for processing graphical data and other functions of the device clusters provided in this specification are partially listed in the following description. The inventive aspects of the methods for processing graphical data and the device clusters provided in this specification can be fully explained through practice or by using the graphical data processing methods, device clusters, and combinations described in the detailed examples below. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A schematic diagram of an original drawing provided according to an embodiment of this specification is shown;

[0038] Figure 2 A schematic diagram of a method for processing graph data according to an embodiment of this specification is shown;

[0039] Figure 3 A schematic diagram of a community subgraph provided according to an embodiment of this specification is shown;

[0040] Figure 4 A schematic diagram illustrating an application scenario of graph data processing according to an embodiment of this specification is shown.

[0041] Figure 5 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0042] Figure 6 A flowchart illustrating another method for processing graph data according to an embodiment of this specification is shown;

[0043] Figure 7 A detailed schematic diagram of another method for processing graph data according to an embodiment of this specification is shown;

[0044] Figure 8 Another specific schematic diagram is shown of another method for processing graph data provided according to embodiments of this specification; and

[0045] Figure 9 A schematic diagram of N partial sub-diagrams provided according to embodiments of this specification is shown. Detailed Implementation

[0046] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0047] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0048] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0049] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0050] For ease of understanding, the terms that will appear in this manual are explained as follows.

[0051] Community clustering (community discovery) is a technique that divides nodes into different communities based on the connections between nodes in a complex network structure of an original graph. In the process of dividing nodes into different communities, nodes within the same community are relatively tightly connected, forming a locally dense network structure; while the connections between nodes in different communities are relatively sparse, exhibiting obvious separation characteristics.

[0052] Community subgraph generation techniques: These techniques extract community subgraphs corresponding to specific communities from the original graph, ensuring that the subgraph retains the key characteristics of the original graph to identify and understand important structures. By generating community subgraphs, research can focus on local structures, i.e., community subgraphs, reducing the complexity of data analysis.

[0053] Original graph: The graph data used before community clustering, which may include vertex and edge information involving multiple communities, for example... Figure 1 As shown, each area enclosed by a dashed box can be considered a community.

[0054] In some embodiments, such as Figure 1 As shown, the original graph includes vertex and edge information involving multiple communities, including the target community. This vertex and edge information is stored across multiple storage devices. The target device is used to generate a community subgraph corresponding to the target community from the original graph.

[0055] For example, the target device determines multiple target point edge information belonging to the target community in the original graph, and determines M storage devices for storing the multiple target point edge information, where M is an integer greater than 1; then it collects the multiple target point edge information from the M storage devices, and generates a community subgraph corresponding to the target community based on the multiple target point edge information.

[0056] For example, Figure 2 As shown, the M storage devices identified by the target device include storage devices 1 to 7. Each storage device stores at least one target point edge information, and the number of target point edge information stored in each storage device is different. The target device needs to collect all target point edge information from storage devices 1 to 7, and then generate a function based on the collected target point edge information. Figure 3 The community subgraph shown.

[0057] In the above embodiments, the steps of collecting target point edge information and generating community subgraphs are all completed by a single target device. For ordinary communities (communities involving a small number of point edge information), the implementation process is not a major problem. However, for hot communities (communities involving more than a preset number of point edge information), the following problems exist:

[0058] 1) Hotspot communities involve a large amount of point-edge information. The communication rate of the target device is limited. Collecting a large amount of point-edge information sequentially through a single target device may cause communication bottlenecks such as communication congestion or information loss.

[0059] 2) A target device generates a community subgraph based on all the point and edge information. The time complexity of the target device is relatively high, that is, the process of generating the community subgraph is relatively time-consuming.

[0060] To overcome the above problems, embodiments of this specification provide a method for processing graph data and a device cluster.

[0061] The following is combined Figure 4 This manual introduces the application scenarios of the graph data processing provided.

[0062] Figure 4 A schematic diagram illustrating an application scenario of graph data processing provided according to embodiments of this specification is shown. For example... Figure 4 As shown, the application scenario includes a device cluster 100, which includes a first device set 10, a second device set 20, and a target device 30.

[0063] The first device set 10 includes multiple computing devices, each used to store partial target point edge information in the original graph. To distinguish it from the second device set 20 mentioned later, this specification refers to the computing devices in the first device set 10 as storage devices.

[0064] The second device set 20 includes multiple computing devices, at least some of which are used to collect target point edge information belonging to the target community from the first device set in parallel, and at least some of which are used to generate partial subgraphs in parallel based on the collected target point edge information. The partial subgraph is a partial graph structure within the community subgraph corresponding to the target community. It is understood that the second device set 20 ultimately generates multiple partial subgraphs.

[0065] The target device 30 is used to collect multiple partial subgraphs from the second device set 20, thereby generating a community subgraph corresponding to the target community based on the collected multiple partial subgraphs.

[0066] In some embodiments, the device cluster 100 may further include a control device ( Figure 4 (Not shown), the control device can be used to distribute the point and edge information of multiple points in the original graph to various storage devices in the first device set 10. The control device can also control the communication between various storage devices in the first device set 10, various computing devices in the second device set 20, and the target device 30, as well as the operations performed based on the communication content. The specific operations are described in detail in the method P100 below.

[0067] It should be noted that the second device set 20 can be devices from the first device set 10; or, the second device set 20 can be devices outside the first device set 10; or, some devices in the second device set 20 are devices from the first device set 10, and some devices are devices outside the first device set 10. The target device 30 can be a device from the second device set 20, or it can be a device outside the second device set 20. The control device can be any device from the first device set 10, the second device set 20, or the target device 30, or it can be a device outside of the first device set 10, the second device set 20, or the target device 30. This specification does not impose any limitations on this.

[0068] In some embodiments, each device in the device cluster 100 may store data and instructions for implementing graph data processing methods, and may execute or be used to execute the data and instructions. In some embodiments, each device in the device cluster 100 may include hardware devices with data information processing capabilities and the necessary programs required to drive the hardware devices to work.

[0069] For example, the point edge information included in the original graph is distributed and stored in the first set of devices in the device cluster 100. In the process of generating the community subgraph corresponding to the target community from the original graph, the device cluster 100 performs the following steps:

[0070] Determine the edge information of multiple target points belonging to the target community in the original graph, and determine M storage devices in the first device set to store the edge information of the multiple target points, where M is an integer greater than 1;

[0071] The second device set collects the edge information of the plurality of target points from M storage devices, and generates N partial subgraphs based on the edge information of the plurality of target points, wherein the edge information of the plurality of target points is collected in parallel by at least some devices in the second device set, and the N partial subgraphs are generated in parallel by at least some devices in the second device set, wherein N is an integer less than M; and

[0072] The target device collects the N partial subgraphs from the second set of devices and merges the N partial subgraphs to generate a community subgraph corresponding to the target community.

[0073] Specifically, the specific operations performed by each device in the device cluster 100 will be explained in detail in method P100 later, and will not be repeated here.

[0074] Figure 5 A hardware structure diagram of an electronic device 200 provided according to an embodiment of this specification is shown.

[0075] Electronic device 200 can be any device in device cluster 100, such as a storage device in first device set 10, a computing device in second device set 20, target device 30, or control device.

[0076] like Figure 5 As shown, the electronic device 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the electronic device 200 may also include a communication port 250 and an internal communication bus 210. Furthermore, the electronic device 200 may also include I / O components 260.

[0077] The internal communication bus 210 can connect to different system components. For example, the internal communication bus 210 can connect to storage medium 230, processor 220, communication port 250, and I / O component 260.

[0078] I / O component 260 supports input / output between electronic device 200 and other components.

[0079] Communication port 250 is used for data communication between electronic device 200 and the outside world. For example, communication port 250 can be used for data communication between electronic device 200 and a network. Communication port 250 can be a wired communication port or a wireless communication port.

[0080] In some embodiments, the network can be any type of wired or wireless network, or a combination thereof. For example, the network may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network™, a ZigBee™ short-range wireless network, a near field communication (NFC) network, or a similar network.

[0081] In some embodiments, the network may include one or more network access points. For example, the network may include wired or wireless network access points, such as base stations or internet switching points. Through these access points, one or more components of various devices corresponding to electronic device 200 can connect to the network to exchange data or information.

[0082] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 236. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the graph data processing methods provided in this specification.

[0083] Processor 220 can be communicatively connected to storage medium 230. Processor 220 is used to execute at least one of the above-described instruction sets. When electronic device 200 is running, processor 220 reads the at least one instruction set and executes the graph data processing method provided in this specification according to the instructions of the at least one instruction set.

[0084] Processor 220 may be in the form of one or more processors. In some embodiments, processor 220 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), central processing units (CPUs), graphics processing units (GPUs), physical processing units (PPUs), microcontroller units, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), advanced RISC machines (ARMs), programmable logic devices (PLDs), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0085] For the purpose of illustrating the point only, in the appendix Figure 5 Only one processor 220 is shown in the electronic device 200. However, it should be noted that the electronic device 200 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 220 of the electronic device 200 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 220 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0086] Figure 6A flowchart of a graph data processing method P100 according to an embodiment of this specification is shown. As previously described, each device in the device cluster 100 can cooperate to execute the graph data processing method P100 to generate a community subgraph corresponding to a target community from the original graph. Specifically, the processor in each device in the device cluster 100 can read the instruction set stored in its local storage medium, and then cooperate to execute the graph data processing method P100 according to the specifications of the instruction set.

[0087] The original graph includes multiple point-edge information, which is distributed and stored in the first device set of the device cluster 100.

[0088] For example, the original image is as follows Figure 1 As shown, this includes vertex-edge information corresponding to multiple communities, where multiple communities include the target community. The vertex-edge information related to the target community is called target vertex-edge information. The original graph structure can contain multiple nodes and multiple edges. For example, a node can represent an entity, and an edge represents a relationship between two connected entities. In this specification, each vertex-edge information represents information about one or more nodes in the original graph, as well as information about one or more edges.

[0089] In some embodiments, each point edge information may include information about an edge in the original graph and information about the two nodes connected by that edge.

[0090] For example, the specific format of each point edge information can be found in [reference needed]. Figure 2 , Figure 7 and Figure 8 For example, the edge information ①-⑥ represents the information of node ①, the information of node ⑥, and the information of the edge between node ① and node ⑥. Other edge information can be understood similarly. Node information can represent entities, such as people, groups, organizations, objects, locations, etc. Edge information can represent the relationships between the nodes connected by the edge, such as the type of relationship and the degree of closeness of the relationship. Edges can be directed or undirected; this specification does not impose any restrictions on this.

[0091] The original graph data units are vertex and edge information, which can intuitively show the structure of the graph data. This representation makes the relationships between entities in complex networks clearer and helps in understanding and analyzing graph data.

[0092] like Figure 6 As shown, the processing of graph data P100 includes the following steps.

[0093] S110: Determine the edge information of multiple target points belonging to the target community in the original graph, and determine M storage devices in the first device set that store the edge information of the multiple target points, where M is an integer greater than 1.

[0094] The vertex and edge information of the original graph is scattered across a first set of devices. Some storage devices in the first set store the target vertex and edge information belonging to the target community, while others store the vertex and edge information belonging to other communities. Therefore, device cluster 100 needs to determine M storage devices from the first set to store the target vertex and edge information.

[0095] For example, after the control device in device cluster 100 identifies the target community and other communities from the original graph through community clustering, it can distinguish the point-edge information corresponding to different communities using community identifiers. During the process of distributing multiple point-edge information from the original graph to the first device set, the control device can inform the storage devices in the first device set of the community identifiers corresponding to the allocated point-edge information. Thus, the control device can determine from the first device set M storage devices that store multiple target point-edge information corresponding to the community identifiers of the target community. The M storage devices are, for example... Figure 7 and Figure 8 Storage devices 1 to 7 are shown.

[0096] S120: Collect the multiple target point edge information from M storage devices through the second device set, and generate N partial subgraphs based on the multiple target point edge information, wherein the multiple target point edge information is collected in parallel by at least some devices in the second device set, and the N partial subgraphs are generated in parallel by at least some devices in the second device set, and N is an integer less than M.

[0097] It can be understood that the process of device cluster 100 generating a community subgraph corresponding to the target community based on the edge information of the multiple target points can be divided into two stages. The first stage is the process of generating the N partial subgraphs based on the edge information of the multiple target points. The second stage is the process of generating the community subgraph based on the N partial subgraphs.

[0098] The first stage can be completed by the various computing devices in the second set of devices working together, while the second stage can be completed by the target device. The second set of devices can implement the computing task of the first stage in several ways, and two examples are given below. Method 1: The first stage is not further divided; Method 2: The first stage is further divided into multiple sub-stages, and each sub-stage is implemented in parallel by computing devices in a subset of the second set of devices.

[0099] Method 1

[0100] In some embodiments, the number of computing devices in the second device set is equal to N. The device cluster 100 performs the following operations in parallel using the N computing devices in the second device set to obtain the N partial subgraphs: collecting partial point edge information from the plurality of target point edge information, and generating partial subgraphs based on the collected partial point edge information.

[0101] In this embodiment, the first stage is not further subdivided, and the operation performed in parallel by each computing device in the second device set is relatively simple, which can reduce the internal processing complexity of the second device set in completing the first stage task. At the same time, the N partial subgraphs can be generated in parallel by multiple computing devices, thereby improving the efficiency of generating the N partial subgraphs and thus improving the efficiency of generating community subgraphs.

[0102] In some embodiments, the device cluster 100 divides the M storage devices into N device groups, determines the correspondence between the N computing devices in the second device set and the N device groups, and then each computing device can collect target point edge information from the corresponding device group based on the correspondence, and generate a partial subgraph based on the collected target point edge information. In this process, the device cluster 100 assigns tasks to the N computing devices to collect target point edge information from the storage devices corresponding to different device groups, avoiding the omission of some target point edge information during collection or avoiding the collection of duplicate target point edge information by the same computing device, thus avoiding wasting computing resources and time.

[0103] For example, the device cluster 100 divides the M storage devices into N device groups based on the distribution of the multiple target point edge information across the M storage devices and N. The number of target point edge information stored in each of the N device groups satisfies a preset quantity balance condition #1, thereby ensuring that the time taken by each computing device to collect target point edge information and generate a partial subgraph is basically the same. This allows the operation of each computing device to be synchronized as much as possible, reducing the waiting time when the target device collects a partial subgraph later.

[0104] For example, the control device in device cluster 100 generates an identifier that uniquely identifies each target point edge information, generates device indexes for N computing devices, and determines a hash mapping function. Based on the identifier corresponding to each target point edge information and the hash mapping function, the target point edge information corresponding to each computing device is determined. The hash mapping function is, for example, D. id =edge id %M, where D id The edge represents the device index of the computing device. id An identifier representing edge information of the target point.

[0105] Assuming N=3, such as Figure 7As shown, the second set of devices includes computing device 1, computing device 2, and computing device 3. The control device determines the target edge information corresponding to computing device 1 as ①-⑥, ①-⑦, ②-⑥, ②-③, ⑤-④, ⑥-⑦ using a hash mapping function; the target edge information corresponding to computing device 2 as ③-②, ③-⑥, ③-④, ④-③, ④-⑥, ④-⑤; and the target edge information corresponding to computing device 3 as ⑥-①, ⑥-②, ⑥-③, ⑥-④, ⑦-①, ⑦-⑥. Therefore, the control device can determine the correspondence between computing device 1 and device group #1 (storage device 1, storage device 2, and storage device 5), computing device 2 and device group #2 (storage device 3 and storage device 4), and computing device 3 and device group #3 (storage device 6 and storage device 7).

[0106] Further, computing device 1 collects target point edge information (①-⑥, ①-⑦, ②-⑥, ②-③, ⑤-④, ⑥-⑦) from device group #1, and generates partial subgraph #1 based on the collected target point edge information. Computing device 2 collects target point edge information (③-②, ③-⑥, ③-④, ④-③, ④-⑥, ④-⑤) from device group #2, and generates partial subgraph #2 based on the collected target point edge information. Computing device 3 collects target point edge information (⑥-①, ⑥-②, ⑥-③, ⑥-④, ⑦-①, ⑦-⑥) from device group #3, and generates partial subgraph #3 based on the collected target point edge information. Partial subgraph #1, partial subgraph #2, and partial subgraph #3 are the N partial subgraphs.

[0107] It should be noted that the preset quantity balance condition for the number of target point edge information stored in each of the N device groups can be understood as follows: the number of target point edge information stored in each device group should be as equal as possible. For example, quantity balance condition #1 means that the difference between the number of target point edge information stored in different device groups is within a first preset range; or the ratio between the number of target point edge information stored in different device groups is within a second preset range. Specifically, the N device groups include device group #1, device group #2, and device group #3. The number of target point edge information stored by all storage devices in device group #1 is quantity #1, the number of target point edge information stored by all storage devices in device group #2 is quantity #2, and the number of target point edge information stored by all storage devices in device group #3 is quantity #3. The difference between any two of quantities #1, #2, and #3 is within the first preset range (e.g., a range centered on 0); or the ratio between any two of quantities #1, #2, and #3 is within the second preset range (e.g., a range centered on 1). In other words, quantities #1, #2, and #3 should be as close as possible to each other.

[0108] Method 2

[0109] In some embodiments, the number of computing devices in the second device set is greater than N, and the second device set is divided into K subsets, where K is an integer greater than 1, and the number of devices in the i-th subset is P. i P i >P i+1 P K =N. Device cluster 100 performs the first operation in parallel through the computing devices in the first subset to obtain P1 intermediate subgraphs, and performs the second operation in parallel through the computing devices in the i-th subset according to the order of i values ​​from 2 to K to obtain P. i An intermediate subgraph, which combines the P generated by each computing device in the Kth subset. K Each intermediate subgraph is used as one of the N partial subgraphs.

[0110] The first operation includes: collecting partial point-edge information from the multiple target point-edge information, and generating an intermediate subgraph based on the collected partial point-edge information.

[0111] The second step includes: collecting P i-1 The collected intermediate subgraphs are used to generate new intermediate subgraphs by merging some of the intermediate subgraphs.

[0112] In this embodiment, the device cluster 100 further subdivides the first stage into multiple sub-stages, making full use of the computing resources of each computing device in the second device set. This significantly shortens the time for the second device set to complete the first stage task, improving efficiency. The advantages of this multi-sub-stage parallel computing are even more pronounced when the amount of target point edge information is enormous. This multi-sub-stage approach allows for adjustment of the number of computing devices corresponding to each sub-stage according to task requirements. Therefore, when the amount of target point edge information increases, leading to increased computing demands, additional computing devices can be added to each sub-stage to better adapt to constantly changing task requirements.

[0113] In some embodiments, the device cluster 100 divides the M storage devices into P1 device groups, determines a first correspondence between the P1 computing devices in the first subset and the P1 device groups, and then, based on the first correspondence, performs a first operation in parallel through the P1 computing devices in the first subset to obtain the P1 intermediate subgraphs. During the execution of the first operation, each of the P1 computing devices collects the target point edge information from its corresponding device group. In this process, the device cluster 100 assigns tasks to the P1 computing devices in the first subset, collecting partial target point edge information from storage devices corresponding to different device groups. This avoids the omission of some target point edge information during collection or prevents the same computing device in the first subset from collecting duplicate target point edge information, thus avoiding wasted computing resources and time.

[0114] For example, the device cluster 100 divides the M storage devices into P1 device groups based on the distribution of the multiple target edge information across the M storage devices and P1. The number of target edge information stored in each of the P1 device groups satisfies a preset quantity balancing condition #1. This ensures that each computing device in the first subset takes approximately the same amount of time to collect target edge information and generate an intermediate subgraph, allowing for more synchronized operations and reducing waiting time when the first subset subsequently collects the intermediate subgraph.

[0115] Specifically, the method by which device cluster 100 divides the M storage devices into P1 device groups can be referred to the method of dividing the M storage devices into N device groups described above, and will not be repeated here.

[0116] For example, Figure 8 As shown, the second device set includes 6 computing devices, designated as computing devices 1 to 6. Assume K = 2, meaning device cluster 100 divides the devices in the second device set into two subsets. The first subset includes computing devices P1 = 4, specifically computing devices 1 to 4. The second subset includes computing devices P2 = N = 2, specifically computing devices 5 and 6.

[0117] The first correspondence determined by the control device includes: the correspondence between computing device 1 and the first storage device group (including storage device 1 and storage device 2), the correspondence between computing device 2 and the second storage device group (including storage device 3 and storage device 5), the correspondence between computing device 3 and the third storage device group (including storage device 4 and storage device 7), and the correspondence between computing device 4 and the fourth storage device group (including storage device 6).

[0118] The first operation performed by computing device 1 is as follows: computing device 1 collects target point edge information (①-⑥, ①-⑦, ②-⑥, ②-③) from storage device 1 and storage device 2, and generates intermediate subgraph #1 based on the collected target point edge information. Similarly, computing device 2 generates intermediate subgraph #2, computing device 3 generates intermediate subgraph #3, and computing device 4 generates intermediate subgraph #4.

[0119] For example, device cluster 100 divides each computing device in the (i-1)th subset into v device groups; determines P in the i-th subset i A computing device and the P i The second correspondence relationship of each device group; and based on the second correspondence relationship, through P in the i-th subset. i Each computing device executes the second operation in parallel to obtain P.i An intermediate subgraph, wherein the P i When performing the second operation, each computing device collects the intermediate subgraph from its corresponding device group. During this process, the device cluster 100 divides the computing devices in the i-th subset into groups to collect the corresponding intermediate subgraphs from different device groups after dividing the (i-1)-th subset. This avoids the omission of some intermediate subgraphs during the collection process or avoids the same computing device in the i-th subset collecting duplicate intermediate subgraphs, thus saving computing resources and time.

[0120] In some embodiments, the device cluster 100 computes the number P of devices in the (i-1)th subset. i-1 and the P i Divide the computing devices in the (i-1)th subset into P i A group of devices, wherein the P i The number of computing devices in each device group satisfies the preset quantity balance condition #2. This ensures that the time spent by each computing device in the i-th subset on collecting intermediate subgraphs and generating new intermediate subgraphs is basically the same, so that the operations of each computing device in the i-th subset can be synchronized as much as possible, and the waiting time can be reduced when the (i+1)-th subset collects new intermediate subgraphs.

[0121] Specifically, device cluster 100 can also divide the computing devices in the (i-1)th subset into P groups based on a hash mapping function. i The equipment group is detailed here, but the specifics will not be repeated.

[0122] The P i The requirement that the number of computing devices in a device group meets the preset quantity balance condition #2 can be understood as follows: the number of computing devices in each device group should be as equal as possible. For example, quantity balance condition #2 means that the difference between the number of computing devices in different device groups falls within a third preset range; or the ratio between the number of computing devices in different device groups falls within a fourth preset range. The third preset range may be the same as or different from the first preset range. The fourth preset range may be the same as or different from the second preset range, and this specification does not impose any restrictions on this.

[0123] For example, Figure 8 As shown, the second correspondence determined by the control device includes: the correspondence between computing device 5 and the first computing device group (including computing device 1 and computing device 2), and the correspondence between computing device 6 and the second computing device group (including computing device 3 and computing device 4).

[0124] The second operation performed by computing device 5 is as follows: computing device 5 collects intermediate subgraph #1 from computing device 1, collects intermediate subgraph #2 from computing device 2, and generates intermediate subgraph #1-2 based on intermediate subgraph #1 and intermediate subgraph #2.

[0125] The second operation performed by computing device 6 is: computing device 6 collects intermediate subgraph #3 from computing device 3, collects intermediate subgraph #4 from computing device 4, and generates intermediate subgraph #3-4 based on intermediate subgraph #3 and intermediate subgraph #4.

[0126] Since the second subset is the last subset after the second set of devices is divided, intermediate subgraphs #1-2 and #3-4 are considered as the N partial subgraphs.

[0127] It should be noted that when K is an integer of 3 or higher, the operations performed by each subset in the second device set can be deduced from the description above, and will not be repeated here.

[0128] S130: Collect the N partial subgraphs from the second set of devices through the target device, and merge the N partial subgraphs to generate a community subgraph corresponding to the target community.

[0129] In some embodiments, the device cluster 100 takes one of the N partial subgraphs as the first partial subgraph and adds the remaining partial subgraphs to the subgraph set. It then performs N-1 updates on the first partial subgraph to obtain the community subgraph, wherein in each update:

[0130] A second subgraph with at least one identical node to the first subgraph is determined from the set of subgraphs; then, based on the identical nodes between the first and second subgraphs, the vertex and edge information in the second subgraph is updated in the first subgraph, and the second subgraph is deleted from the set of subgraphs.

[0131] For example, such as Figure 9 As shown, the N partial subgraphs include partial subgraph #1, partial subgraph #2, and partial subgraph #3. Device cluster 100 selects partial subgraph #1 as the first partial subgraph and adds partial subgraphs #2 and #3 to the subgraph set. In the first update, partial subgraph #2 is selected from the subgraph set as the second partial subgraph. Based on the common nodes between partial subgraphs #2 and #1, the target point edge information in partial subgraph #2 is updated to the first partial subgraph, thus obtaining the first partial subgraph. Figure 2 0, and remove subgraph #2 from the subgraph set. In the second update, select subgraph #3 from the subgraph set as the second subgraph, based on the relationship between subgraph #3 and the first subgraph. Figure 2The same nodes between .0 will update the target point edge information in part of subgraph #3 to the first part of the subgraph. Figure 2 In .0, the first part is obtained. Figure 3 0, and remove subgraph #3 from the subgraph set. At this point, there are no other subgraphs in the subgraph set, so the first subgraph... Figure 3 .0 as Figure 3 The community subgraph shown.

[0132] In some embodiments, when the number of edge information of the plurality of target points is greater than a preset number, the device cluster 100 executes S120 and S130. That is, when the target community is a hot community, the device cluster 100 uses S120 and S130 to generate the community subgraph, which can significantly improve the efficiency of generating the community subgraph.

[0133] In some embodiments, if the number of edge information of the plurality of target points is less than or equal to a preset number, the device cluster 100 does not execute S120 and S130, but executes the third operation.

[0134] The third operation involves the target device collecting the target point edge information from the M storage devices and generating the community subgraph based on this information. In other words, for non-hotspot communities, the target device can complete the collection of target point edge information and the generation of the community subgraph. The interaction between devices within the device cluster of 100 is relatively simple, and the amount of point edge information involved in non-hotspot communities is small, thus having little impact on computational efficiency.

[0135] The solutions provided in this manual allow for flexible execution of the device cluster 100 in different ways for both non-hotspot areas and hotspot communities.

[0136] Specifically, assuming the number of target edge points in the target community is m, the number of computing devices in the second device set is n, and the time complexity of device cluster 100 executing S110, S120, and S130 is O(m). The time complexity of device cluster 100 executing S110 and the third operation is O(m). It is evident that when m is much larger than n, Approximately In other words, when the amount of target point edge information involved in the target community is large enough, the execution of S110, S120 and S130 by device cluster 100 can be nearly n times faster than the execution of S110 and the third operation.

[0137] In method P100, when each of the M storage devices sends the target point edge information to the corresponding computing device in the second device set, it can be achieved through MapReduce technology.

[0138] It should be noted that in this specification, "parallelism" can be understood as mutual non-interference, for example... Figure 8 As shown, computing devices 1, 2, 3, and 4 collect target point edge information and generate intermediate subgraphs independently. The collection and generation operations of computing devices 1, 2, 3, and 4 can be performed synchronously, and the timing of collection can be dynamically scheduled by the control device; this specification does not impose any restrictions on this.

[0139] Similarly, the collection of intermediate subgraphs and the generation of partial subgraphs by computing devices 5 and 6 are also independent of each other. The collection and generation operations of computing devices 5 and 6 can be performed synchronously, and the timing of collection can be dynamically scheduled by the control device; this specification does not impose any restrictions on this.

[0140] In summary, the graph data processing method and device cluster provided in this specification divide the task of generating a target community based on the edge information of multiple target points involved in the target community into at least two stages. The computational tasks of each stage can be processed in parallel by different computing devices, making reasonable use of computing resources, thereby improving the overall processing efficiency and reducing the communication overhead of each computing device.

[0141] This specification, in another aspect, provides a computer-readable non-transitory storage medium storing at least one set of executable instructions for performing a method of processing graph data. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of method P100 described in this specification. In some possible embodiments, various aspects of this specification may also be implemented as a program product comprising program code. When the program product is run on device cluster 100, the program code is used to cause device cluster 100 to perform the steps of method P100 described in this specification. The program product for implementing the above method may employ a portable compact disk read-only memory (CD-ROM) containing program code and may run on device cluster 100. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on device cluster 100, partially on device cluster 100, as a standalone software package, partially on device cluster 100 and partially on a remote computing device, or entirely on a remote computing device.

[0142] The term "and / or" in the embodiments of this specification describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0143] The terms “first”, “second”, etc., used in this specification are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence.

[0144] Unless otherwise stated, the term "multiple" in this specification shall be understood as two or more.

[0145] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0146] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0147] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0148] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0149] Every patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein, except for those inconsistent with or conflicting with this document, or those having a restrictive effect on the widest scope of the claims, may be incorporated herein by reference for all purposes now or hereafter associated with this document. Furthermore, in the event of any inconsistency or conflict between the description, definition, and / or use of relevant terms in any material and the description, definition, and / or use of relevant terms in this document, the terms in this document shall prevail.

[0150] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for processing graph data, applied to a device cluster, for generating a community subgraph corresponding to a target community from an original graph, the original graph comprising a plurality of point-edge information, the plurality of point-edge information being stored in a first set of devices in the device cluster, the method comprising: determining a plurality of target point-edge information belonging to the target community in the original graph, and determining M storage devices in the first set of devices storing the plurality of target point-edge information, M being an integer greater than 1; collecting, by a second set of devices in the device cluster, the plurality of target point-edge information from the M storage devices, and generating N partial subgraphs based on the plurality of target point-edge information, wherein the plurality of target point-edge information is collected in parallel by at least part of the second set of devices, and the N partial subgraphs are generated in parallel by at least part of the second set of devices, N being an integer less than M; and collecting, by a target device in the device cluster, the N partial subgraphs from the second set of devices, and merging the N partial subgraphs to generate the community subgraph corresponding to the target community. The collecting, by the second set of devices in the device cluster, the plurality of target point-edge information from the M storage devices, and generating N partial subgraphs based on the plurality of target point-edge information, comprises: performing, by each computing device in a first subset in parallel, a first operation to obtain P1 intermediate subgraphs, the first operation comprising: collecting part of the plurality of target point-edge information, and generating an intermediate subgraph based on the collected part of the plurality of target point-edge information. The performing, by each computing device in the first subset in parallel, the first operation to obtain P1 intermediate subgraphs, comprises: dividing the M storage devices into P1 device groups; determining a first correspondence between P1 computing devices in the first subset and the P1 device groups; and based on the first correspondence, performing, by the P1 computing devices in the first subset in parallel, the first operation to obtain the P1 intermediate subgraphs, wherein the P1 computing devices respectively collect the target point-edge information from the corresponding device groups when performing the first operation. The dividing the M storage devices into P1 device groups, comprises: dividing the M storage devices into the P1 device groups based on a distribution of the plurality of target point-edge information in the M storage devices and the P1, wherein a quantity of target point-edge information stored in each of the P1 device groups satisfies a preset quantity balance condition. The number of computing devices in the second set of devices is equal to N, and the collecting, by the second set of devices in the device cluster, the plurality of target point-edge information from the M storage devices, and generating N partial subgraphs based on the plurality of target point-edge information, comprises: performing, by N computing devices in the second set of devices in parallel, an operation to obtain the N partial subgraphs, the operation comprising: collecting part of the plurality of target point-edge information, and generating a partial subgraph based on the collected part of the plurality of target point-edge information.

2. The method of claim 1, wherein, the number of computing devices in the second set of devices is greater than N, the second set of devices is divided into K subsets, the K being an integer greater than 1, wherein the number of devices in the i-th subset is P i , P i > N i+1 , P K = N; ​ ​ performing, in parallel by the computing devices in the i-th subset, a second operation to obtain P i intermediate subgraphs in order, for i ranging from 2 to K i-1 intermediate subgraphs, and merging the collected partial intermediate subgraphs to generate a new intermediate subgraph, and P generated by each computing device in the Kth subset k as the N partial subgraphs.

3. The method of claim 2, wherein, ​ ​ ​ ​ 4. The method of claim 3, wherein, ​ ​ 5. The method of claim 2, wherein, performing the second operation in parallel by the computing devices in the ith subset to obtain P i intermediate subgraphs, comprising: dividing each computing device in the i-1th subset into P i device groups; determining P i computing devices and a second correspondence between the P i devices and the P computing devices; and based on the second correspondence, performing the second operation by P i computing devices in the ith subset in parallel to obtain P i intermediate sub-graphs, wherein the P i computing devices collect the partial intermediate sub-graphs from the corresponding device groups respectively when performing the second operation.

6. The method of claim 5, wherein, The dividing each computing device in the i-1th subset into P i device groups comprises: a number P of computing devices in the i-1th subset i-1 and the P i dividing each computing device in the i-1th subset into the P i device groups, wherein the P i device groups contain a number of computing devices satisfying a preset number balance condition.

7. The method of claim 1, wherein, ​ ​ 8. The method of claim 1, wherein, Each point edge information comprises information of an edge in the original graph and information of two nodes connected by the edge.

9. The method of claim 8, wherein, The merging of the N partial subgraphs to generate the community subgraph corresponding to the target community comprises: Taking one of the N partial subgraphs as a first partial subgraph and adding the remaining partial subgraphs to a subgraph set, performing N-1 times of updating on the first partial subgraph to obtain the community subgraph, wherein in each time of updating: Determining a second partial subgraph having at least one same node as the first partial subgraph from the subgraph set; and Updating point edge information in the second partial subgraph to the first partial subgraph based on the same node between the first partial subgraph and the second partial subgraph, and deleting the second partial subgraph from the subgraph set.

10. The method of claim 1, wherein, The collecting of the plurality of target point edge information from the M storage devices by the second device set in the device cluster and the generating of N partial subgraphs based on the plurality of target point edge information comprises: In a case where the number of the plurality of target point edge information is greater than a preset number, collecting the plurality of target point edge information from the M storage devices by the second device set in the device cluster and generating N partial subgraphs based on the plurality of target point edge information.

11. The method of claim 10, wherein, The method further comprises: In a case where the number of the plurality of target point edge information is less than or equal to the preset number, collecting the plurality of target point edge information from the M storage devices by the target device and generating the community subgraph based on the plurality of target point edge information.

12. A device cluster comprising a plurality of computing devices, each computing device comprising: at least one storage medium storing an instruction set for processing of graph data; and at least one processor communicatively connected to the at least one storage medium, wherein when the device cluster is running, the processors in the plurality of computing devices read the corresponding instruction set and implement the method according to any one of claims 1-11 as indicated by the instruction set. ​

Citation Information

Patent Citations

  • Community division method and device, computing equipment and storage medium

    CN109408722A

  • GPU cluster deep learning task parallelization method, device and electronic equipment

    CN110399222A