Method, device, apparatus and program product for graph segmentation

Abstract

The embodiment of the invention relates to a graph segmentation method, equipment, a device and a program product. According to an exemplary method, a limited projective plane is determined from a number of partitions of a graph. The limited projective plane includes a plurality of points and a plurality of lines, each point corresponding to one of the partitions, and each line including a subset of the plurality of points. A first line and a second line of the plurality of lines are determined from vertexes of edges of the graph. A partition of the edge is determined from the first line and the second line. The graph is segmented by assigning each edge to a corresponding partition. The embodiment of the invention provides a deterministic and balanced method for graph segmentation, which can ensure a bounded replication factor of a segmented graph so as to accelerate the performance of distributed graph processing.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of computer technology, and more particularly to a method, apparatus, device and computer program product for graph segmentation. Background Technology

[0002] In many application systems, such as social networks, information retrieval, video conferencing, product recommendation systems, knowledge management systems, and other systems, large-scale graph data (e.g., vertices representing documents, people, products, or other items linked by edges) are generated.

[0003] Existing methods for querying graph data and performing computations using it to control applications and other systems are typically time-consuming and do not scale well to network-scale applications involving large amounts of data. Therefore, there is a need to efficiently partition large-scale graph data into smaller graphs for processing on computing devices.

[0004] The embodiments described below are not limited to implementations that address any or all the drawbacks of known methods for processing graph data. Summary of the Invention

[0005] Overall, the embodiments of the present invention provide a graph segmentation scheme.

[0006] Firstly, a method is provided. This method includes: determining multiple points and multiple lines in a finite projective plane based on the number of partitions in the graph, where each point corresponds to one partition and each line comprises a subset of multiple points; determining a first line and a second line among the multiple lines based on the vertices of each edge in the graph; identifying a partition for each edge based on the first and second lines; and partitioning the graph by assigning each edge to its corresponding partition. In this way, a bounded replication factor can be used to partition the graph, improving the performance of distributed graph processing.

[0007] Secondly, an electronic device is provided. The electronic device includes a processor and a memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, cause the device to perform actions. These actions include: determining a plurality of points and a plurality of lines in a finite projective plane based on the number of partitions of a graph, each point corresponding to one partition and each line comprising a subset of points; determining a first line and a second line among the plurality of lines based on the vertices of each edge among the plurality of edges of the graph; identifying a partition for each edge based on the first line and the second line; and partitioning the graph by assigning each edge to a corresponding partition.

[0008] Thirdly, an apparatus is provided. The apparatus includes: components for determining a plurality of points and a plurality of lines in a finite projective plane based on the number of partitions of a graph, each point corresponding to one of the partitions, and each line comprising a subset of points; components for determining a first line and a second line among the plurality of lines based on the vertices of each edge of the graph; components for identifying a partition for each edge based on the first line and the second line; and components for segmenting the graph by assigning each edge to a corresponding partition.

[0009] Fourthly, a computer program product tangibly stored in a computer-readable medium and including machine-executable instructions is provided, which, when executed, cause a machine to perform the method according to the first aspect of the invention.

[0010] Fifthly, a computer-readable medium including machine-executable instructions is provided, which, when executed, cause a machine to perform the method according to the first aspect of the invention.

[0011] It should be understood that the summary section is not intended to identify key or essential features of the embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] Some embodiments will now be described with reference to the accompanying drawings, in which:

[0013] Figure 1 A schematic diagram of an exemplary environment in which various embodiments of the present invention may be implemented is shown;

[0014] Figure 2 A flowchart illustrating an exemplary method for segmenting a graph according to some embodiments of the present invention is shown;

[0015] Figure 3 A schematic diagram illustrating an exemplary workflow based on a finite projective plane segmentation diagram according to some embodiments of the present invention is shown;

[0016] Figure 4 Exemplary diagrams for segmentation according to some embodiments of the present invention are shown;

[0017] Figure 5 A schematic block diagram of a device that can be used to implement embodiments of the present invention is shown.

[0018] In all the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0019] The principles of the invention are described below with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement the invention, and do not impose any limitation on the scope of the invention. The disclosure described herein can be implemented in various ways other than those described below.

[0020] In the following detailed description and claims, unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as those known to a person skilled in the art.

[0021] In this invention, references to "one embodiment," "some embodiments," "embodiment," etc., indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment necessarily includes such specific features, structures, or characteristics. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with some embodiments, it is understood by those skilled in the art that such feature, structure, or characteristic will be implemented in conjunction with other embodiments, whether explicitly described or not.

[0022] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed items.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments. Unless the context clearly indicates otherwise, the singular forms “a” and “described” as used herein are also intended to include the plural forms. It should also be understood that the terms “comprising,” “having,” and / or “including,” when used herein, indicate the presence of the described features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0024] To better understand the embodiments of the present invention, definitions of relevant terms mentioned in the present invention are provided below.

[0025] Graph: A set of pairs (V, E), where V is the set of vertices and E is the set of vertex pairs (v, u) (i.e., edges); v is called the source and u is called the destination.

[0026] Subgraph: Subgraph of graph G=(V,E): Graph =( ), making , .

[0027] Graph partitioning: distributing graph elements into smaller sets. There are two types of graph partitioning: vertex partitioning and edge partitioning; this invention considers the latter. For example, given an input value n (i.e., the number of partitions), each edge receives an integer number from 0 to n–1.

[0028] Replication factor: The average number of duplicate vertices, equal to the number of vertices divided by partitions. (in, All subgraphs formed by ) The sum of the number of vertices in the array divided by the number of vertices |V|:

[0029]

[0030] It shows the number of repeating vertices.

[0031] Balance: The maximum number of edges in a partition divided by the average number of edges in the partition; a good balance should be close to 1.0.

[0032]

[0033] Projective plane: A set of lines, a set of points, and the relationship between points and lines is called a correlation, which has the following properties:

[0034] — Given any two distinct points, there is exactly one line connecting these two points;

[0035] — Given any two distinct lines, there is exactly one point that is connected to both lines;

[0036] There are 4 points such that no line is associated with more than 2 of them.

[0037] Finite field Fq: A finite set of q elements with two associative and commutative operations (Fq). and ), satisfying the following rules:

[0038] 1. There exists and ( ), so that for each ,

[0039] 2. For each ,

[0040] 3. For each , , making

[0041] 4. For each , making , Make

[0042] Finite projective plane: The projective plane on.

[0043] Graph partitioning is a computational task involving distributing graph elements into a small set of disjoint parts. Graph partitioning can be performed through vertex partitioning and edge partitioning. In this invention, the latter case is considered. The input to the graph partitioning system is a graph represented as a set of edges and a number of partitions n. The output of the system is the partitioned graph, where each edge corresponds to a number from 0 to n–1.

[0044] Graph partitioning is a crucial step in distributed computing. This task is essential for analyzing large graphs due to their long runtime. In practice, graphs are distributed across several machines in some way to optimize storage, but can also significantly speed up computation. The way a graph is partitioned has a significant impact on load balancing and the amount of interaction between machines, which is important for runtime. However, in most cases, achieving an ideal partition of the graph across machines without any data duplication is impossible.

[0045] After edge splitting, some vertices are "cut," appearing in different partitions. Interactions occur between different machines as these vertices are affected in the algorithm. To make the algorithm faster, reducing the number of duplicate vertices is meaningful; the average number of duplicate vertices is called the replication factor.

[0046] Numerous partitioning methods exist across various literature and frameworks, applied to graph algorithms to accelerate runtime. Research focuses on graph partitioning quality metrics such as balance and replication factor. Methods and algorithms typically aim to optimize these metrics. However, these improvements are highly dependent on the graph structure and the target application requiring acceleration. There is no universal approach applicable to all situations. Furthermore, partitioning methods achieving better quality often employ more complex computations, thus incurring significant time costs. Long partitioning times are unavoidable in tasks where partitioning is performed before each application runs. This is why complex graph partitioning methods are unsuitable for scenarios where partitioning is an integral part of the overall computational pipeline.

[0047] Some methods guarantee that the partition has a bounded replication factor. For example, the replication factor of the EdgePartition2D method, which is generally considered the baseline, is less than [a certain value]. ,in, This indicates the number of partitions. Another method, called circular partitioning, may have the maximum replication factor. It is smaller than the approximate replication factor of EdgePartition2D. However, these methods cannot obtain the theoretical boundary (for the complete graph, it is...). ).

[0048] Existing sharding methods that achieve better sharding quality typically employ more complex computations, thus incurring significant time costs. Long sharding times are unavoidable in tasks where sharding is performed before each application runs. This is why complex graph sharding algorithms are unsuitable for scenarios where sharding is part of the overall computational pipeline. Furthermore, the replication factor has not reached its possible lower bound.

[0049] Therefore, a method for partitioning a graph based on a finite projective plane is provided. In this method, a computing device creates a finite projective plane based on the number of partitions, n. The finite projective plane consists of n points, each corresponding to a partition of the graph, and n lines, each composed of a subset of the points in the finite projective plane. For each edge of the graph to be partitioned, the computing device maps the two vertices of the edge to two lines and determines the points in the finite projective plane based on these two lines. Then, the computing device assigns the edge to the corresponding partition. By assigning all edges of the graph to their corresponding partitions, the device generates the partitioned graph. In this way, the time for partitioning the graph can be reduced, and an improved replication factor boundary is achieved.

[0050] The principles and embodiments of the present invention will now be described in detail with reference to the accompanying drawings. First, refer to... Figure 1 The diagram illustrates an exemplary environment 100 in which various embodiments of the present invention can be implemented.

[0051] Exemplary environment 100 includes application system 110 and distributed computer system 120. Distributed computer system 120 includes multiple computing devices 101 and 102, which may be located in the same or different geographical locations. In distributed computer system 120, computing device 101 is coupled to application system 110 and used to perform graph partitioning methods.

[0052] Computing device 101 takes a graph 106 containing multiple edges as input, each edge represented by a pair of vertices. Graph 106 can be a directed graph, where some edges have directions. Graph 106 can be an undirected graph, where no edge has a direction. In both cases, each edge can be represented by a source (src) vertex and a destination (dst) vertex. The vertices of graph 106 have identifiers. Vertex identifiers can include one or more of numbers, strings, characters, etc. Computing device 101 can receive or access graph 106 as a data stream from application system 110 to use the graph for control or modification. In some examples, the graph can be stored (on computing device 101 or elsewhere). Computing device 101 can segment graph 106 into subgraphs 108 and distribute subgraphs 108 to computing devices 102 associated with graph analysis applications.

[0053] Application system 110 may include multiple applications 112 that will use graph 106 to control or change it. Application system 110 may be any computer-implemented system that observes and / or records events, which can be recorded as one or more connections between items represented as graph vertices. Depending on the specific application, each graph vertex may represent at least one item, such as a person, product, document, email account, video conferencing account, or other item. Each graph edge represents a relationship between the items represented by the vertices it connects to.

[0054] For example, one application in 112 could be a social networking system that stores user accounts and allows connections to be established between user accounts and / or communication to be sent between them. In this case, Figure 106 could be a social graph, where operations are user interactions defined by social engagements represented by graph edges. Application 112 could be an information retrieval system for obtaining addresses of documents or other items, as well as information about links or other connections between documents. Documents can be represented by graph vertices, and links between documents can be represented by edges. Application 112 could be a video conferencing system that uses graph vertices to represent video conferencing accounts associated with people. Communication events between users in a video conferencing system can be represented by graph edges. Application 112 could be a recommendation system that uses user and product features to recommend products to users. For example, graph vertices could represent users and / or products, while edges could represent events such as whether a user has purchased a given product. The system could be a knowledge management system. For example, the graph could be a knowledge graph, where vertices represent people, places, and items, and edges represent relationships such as "likes," "dislikes," "life," or "works."

[0055] Once Figure 106 is available, it can be used to control application system 110. For example, queries on Figure 106 can produce results that can be used to improve information retrieval results, improve recommendations, recommend new friends in social networks, and for other purposes. However, in many cases, the scale of Figure 106 is so large that it is impractical to perform calculations on the graph data on a real-world timescale.

[0056] As described above, Figure 106 can be divided into subgraphs 108, such that the size of subgraphs 108 is smaller than the size of the entire Figure 106. Each subgraph can be stored in, or accessed by, one or more computing devices 102 in a data center, or distributed across various locations and communicating with each other. Each computing device 102 can perform computations and aggregate the results to accommodate large-scale graph data. For example, queries on graph data may be large, and each query may be executed on many clusters. The computation results 109 can be sent to application system 110 and used to control the system.

[0057] In some embodiments, computing devices 101 and 102 can be implemented as various user terminals or service terminals with computing capabilities. Service terminals can be servers, large-scale computing devices, etc., provided by various service providers. For example, user terminals can be mobile terminals, fixed terminals, or any type of portable terminal, including mobile phones, sites, units, devices, multimedia computers, multimedia tablets, internet nodes, communicators, desktop computers, laptop computers, netbook computers, tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any other combination thereof, including accessories and peripherals of these devices or any other combination thereof. It is also understood that computing devices 101 and 102 can support any type of user-specific interface (e.g., "wearable" circuitry, etc.).

[0058] Alternatively or additionally, the functionality of the computing devices 101 and 102 described herein may be performed at least in part by one or more hardware logic components of an electronic device. For example, exemplary types of hardware logic components that may be used, without limitation, include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), and graphics processing units (GPUs).

[0059] It should be understood that the architecture and functionality described in environment 100 are for illustrative purposes only and do not imply any limitations. Other devices, systems, or components not shown in environment 100 may also be present. Furthermore, embodiments of the present invention can be applied to other environments with different structures and / or functions.

[0060] refer to Figure 2 A flowchart of an exemplary method 200 for segmenting a graph according to some embodiments of the present invention is shown. For example, the exemplary method 200 may be provided by... Figure 1 The computing device 101 shown performs this operation. It should be understood that method 200 may also include additional actions not shown, and the scope of the invention is not limited thereto. The following is in conjunction with... Figure 1 The exemplary environment 100 describes method 200 in detail.

[0061] In box 210, computing device 101 determines multiple points and multiple lines in a finite projective plane based on the number of partitions in the graph. In the finite projective plane, each point may correspond to one partition, and each line may include a subset of multiple points.

[0062] In some embodiments, the number of points and the number of lines are equal to the number of partitions. Let n denote the number of partitions. A finite projective plane may include n points, which, as a whole, are... It represents n lines, each line consisting of S. i This means that i = 0 ... n.

[0063] In some embodiments, the number of partitions n can be the value of an irreducible polynomial that is a prime power. For example, ,in, Let N be a prime number and k be a positive integer. In some embodiments, the computing device 101 may receive an input number N indicating the number of partitions required by the user. If N is not the value of an irreducible polynomial that is a power of a prime, the computing device can determine the appropriate number of partitions n based on the number of inputs N. For example, n could be... The maximum value of the form, but less than or equal to the number of inputs N, in which case the number of partitions is... and any The results are the same.

[0064] To create a finite projective plane, computing device 101 uses irreducible polynomials, such as We obtain a prime number p and an integer k. For p and k, computing device 101 can find points in a finite projective plane. In some embodiments, each point can be represented as an array of 3 elements having a finite field F, which has a prime number. The order of the point. In some embodiments, each element of the point may be represented by a number from 0 to q–1.

[0065] In some embodiments, computing device 101 can find a subset of lines or points such that for any and , Graph partitioning might look like this. Consider the mapping function of lines from vertices to the projective plane. For the edge Consider two subsets: and A subset has a non-empty intersection, and a partition can be selected for the edge from this intersection. In some embodiments, the index of the subset or line can be a function of the identity (ID) of the two vertices of the edge. For example, functions It can be the remainder when the vertex ID is divided by the total number of partitions: .

[0066] In some embodiments, each line can be defined by a point such that all points in the line have a scalar product equal to zero with that point. For example, each line can be defined by a point By definition, it is all points. , making It is a scalar product.

[0067] In some embodiments, the segmentation uses the projective plane as the point set. Points can correspond to partition indexes, subsets These are lines on the plane. As mentioned above, the number of subsets or lines may equal the actual number of partitions. ,in, It is a prime number. Because of the subset Since it is a line on the projective plane, the intersection of two (different) subsets contains exactly one point. Each vertex corresponds to a subset (mapped). Furthermore, every subset or line of the projective plane is a power of a prime number plus 1 (i.e., It consists of 4 points, therefore no vertex can be copied more than 10 times. Next (otherwise in the corresponding Among all intersections with other subsets, there are those greater than (This is incorrect.) It should be noted that... This ensures that the replication factor is no greater than [a certain value]. .

[0068] In some embodiments, a finite projective plane comprising points and lines can be created solely based on the number of partitions. The computing device 101 can create the finite projective plane offline without actual graph input. This further reduces partitioning time.

[0069] In box 220, computing device 101 determines a first line and a second line among a plurality of lines based on the vertices of each edge in the graph. For a given edge... The computing device 101 can use functions The vertices of the edges are mapped onto lines in the projective plane, as described above. Therefore, the computing device obtains the first line and the second line.

[0070] In box 230, computing device 101 identifies partitions for each edge based on the first line and the second line. Computing device 101 can determine points in a finite projective plane based on the first line and the second line, and determine the partitions corresponding to those points for each edge.

[0071] As described above, the intersection of two distinct lines in a finite projective plane contains exactly one point. In some embodiments, if the first line and the second line are different lines, the computing device 101 can determine the point to be segmented as the intersection of the first line and the second line. If the first line and the second line are the same line, i.e., the two vertices of an edge map to the same line or the same subset, then the intersection of these two lines contains multiple points of that line. In this case, the computing device 101 can determine the point to be segmented as a point of that line based on the mapping from the line to points within that line: , making In some embodiments, mapping This can be achieved using Kuhn's algorithm. This eliminates random selection, retains the advantages of all methods, and improves the uniformity of the results obtained by the current method. In this way, computing device 101 determines the intersection points of multiple lines for different lines and, based on the line-to-point mapping... The points corresponding to these lines are determined. In some embodiments, the computing device 101 may store the intersection points and points associated with the multiple lines, for example, in matrix form called an intersection matrix.

[0072] In block 240, computing device 101 partitions the graph by assigning each edge to a corresponding partition. Computing device 101 can associate partition identification information with edges to include edges in the corresponding partitions. By processing each edge in the graph, the graph is partitioned into a set of subgraphs. In some embodiments, computing device 101 can distribute the partitioned graph (i.e., subgraphs) to multiple computing devices 102. Computing devices 102 use the subgraphs to perform graph algorithms. It should be noted that computing device 101 can perform the actions in blocks 220, 230, and 240 sequentially or in parallel to improve the performance of graph partitioning.

[0073] The above reference Figures 1 to 2 A scheme for partitioning a graph according to an embodiment of the present invention is described. This scheme uses lines on a finite projective plane as a subset of points corresponding to partitions. Compared to conventional methods, it reduces partitioning processing time and ensures a high replication factor. The boundary is defined by n, where n is the number of partitions, thus reducing the graph processing time. In some embodiments, it also eliminates random selection, retaining the advantages of all methods and improving the balance of results.

[0074] Figure 3 A schematic diagram illustrating an exemplary workflow based on a finite projective plane segmentation map according to some embodiments of the present invention is shown. The subsequent work can be... Figure 1 Implemented on computing device 101. Figure 3 In a graph, a graph is considered a set of vertices and edges, where an edge is an ordered pair of vertices. A partitioned graph is a set of edges, where each edge has an index indicating the partition it belongs to.

[0075] The given graph 302 is represented as a set of edges. Another input parameter is the number of input segments (N) 305, which is likely desired by the user. In box 304, after the given graph is read into memory, the following operations are performed.

[0076] In box 306, find the actual number of partitions n and the corresponding prime powers based on the input quantity N. In some embodiments, it can first find the prime power q based on the input parameter N. The prime power can be a prime number. and natural numbers maximum value ,in, The actual number of partitions can then be set to... Other irreducible polynomials also apply.

[0077] In box 308, find A finite projective plane of points and above line The input for box 308 can include the actual number of partitions. and prime powers The output can include a representation of size [size missing]. The points are represented as Array[Array[Int]] and the lines are represented as Array[Array[Int]] of that size. To find all the points, each point is a line of size 3. An array of order 1 finite field The element of this field Each element (in, ) represents an array of size k. (It can be considered a polynomial) The following describes some examples.

[0078] Example 1

[0079] q=2, p=2, k=1.

[0080] The elements are {0, 1}.

[0081] Suppose we find a point (7 = 4 + 2 + 1).

[0082] These are domains A one-dimensional line in the three-dimensional space V.

[0083] 1 point (0, 0, 1)

[0084] Two points with coordinates (0, 1, a): {(0, 1, 0), (0, 1, 1)}

[0085] Four points with coordinates in the form of (1, a, b): {(1, 0, 0), (1, 1, 0), (1, 0, 1), (1, 1, 1)}

[0086] Example 2

[0087] q=4, p=2, k=2

[0088] Irreducible polynomials , For its in The roots in the multiplicative group generate the multiplicative group. The element, which is .

[0089] Suppose we find points (there are 21 = 16 + 4 + 1 points).

[0090] These are domains A one-dimensional line in the three-dimensional space V.

[0091] One point ((0,0), (0,0), (1,0)).

[0092] Four points with coordinates ((0,0), (1,0), a): {((0,0), (1,0), (0,0)), ((0,0), (1,0), (1,0)), ((0,0), (1,0), (0,1)), ((0,0), (1,0), (0,1)), ((0,0), (1,0), (1,1))}.

[0093] 16 points with coordinates ((1,0), a, b): {((1,0), (0,0), (0,0)), ((1,0), (0,0), (1,0)) ... ((0,0), (1,1), (1,1))}

[0094] Then, find all the projective lines. Each line can be represented as a finite field. A 2D subspace of a 3D vector space on the vector plane. Each line can be defined by a point u, which is all points v such that... , it is The scalar product over the given surface. An example is described below.

[0095] Example 3

[0096] Consider the points in Example 1.

[0097] The line is formed by the homogeneous equation (with) Such an equation) is defined by 3 = It consists of several points.

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] in, , , Elements representing points, The line represents the line, where i = 0……6.

[0106] Find all the intersections in box 310. The input to box 310 may include a subset (represented as a set of size ). The array[Array[Int]] is an array of projective lines; the lines are... (An array of points). The output of box 310 may include the intersection (represented as an array of size ). An Array[Array[Int]] can be considered as an intersection matrix. The intersection of elements (i) and (j) contains and The intersection of; with the first The elements corresponding to the subsets are retained in the intersection matrix at the th position. (Among the diagonal elements). Execute the following statement:

[0107] if ,but ;if ,but .

[0108] when At that time, the latter situation occurs.

[0109] To find all intersections, the size is... The intersection of objects, Array[Array[Int]], can be viewed as an intersection matrix. The intersection of elements (i) and (j) can contain and The intersection of the elements, element (i) is not yet defined. Since there is a single element, an Int value is sufficient, and the main diagonal contains only zeros.

[0110] ( When constructing a mapping Use this to fill the main diagonal of the intersection matrix. Construct this mapping. , making It can be proven that such a mapping always exists. An exemplary mapping is shown below.

[0111] Example 4

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120] The intersection matrix is ​​shown below:

[0121]

[0122] As can be seen, each vertex here cannot have more than q+1=3 partitions (that is, the number of rows or columns with equal values). This is calculated for the entire graph. A matrix can be stored for use by graphs with the corresponding number of partitions. Dialog box elements correspond to vertices mapped to the same line, while other elements correspond to vertices mapped to different lines.

[0123] Continue to refer to Figure 3 The following is the calculation for each edge. A given edge 312 is represented as a source vertex and a destination vertex pair. In box 314, find the line connecting the two vertices of the edge. The input to box 314 can include the vertices of the edge. The ID, and the output can include the line index. For example, mapping Defined as the remainder when vertex Id is divided by n: , .

[0124] In 316, the partition is determined based on this line. In this case, the function Choose was applied. from Choose from Obtain the position from the intersection matrix The elements in.

[0125] In box 318, the splitting result of given edge 312 is determined. In box 320, after processing all edges, the split graph is output.

[0126] Figure 5 An exemplary diagram for segmentation according to some embodiments of the present invention is shown. Figure 5 An example vertex ID is shown. It will be understood that there could be other IDs as well.

[0127] consider Figure 5 The diagram in the middle. Let... For the edge The partitioning is as follows, according to some embodiments of the present invention.

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136] The provided method can accelerate some graph applications. A pipeline refers to a process where the graph is sliced ​​before each application runs. This method uses a relatively fast slicing process, making it beneficial in this scenario. While the method is not limited to this scenario, it can also be applied to situations where a slicing is performed once and the application runs multiple times.

[0137] In some exemplary embodiments, the apparatus capable of performing method 200 (e.g., computing device 101) may include components for performing the various steps of method 200. These components may be implemented in any suitable form. For example, these components may be implemented in a circuit or software module.

[0138] In some exemplary embodiments, the apparatus includes: means for determining a plurality of points and a plurality of lines in a finite projective plane based on the number of partitions of a graph, each point corresponding to one of the partitions and each line comprising a subset of the plurality of points; means for determining a first line and a second line among the plurality of lines based on the vertices of each of the plurality of edges of the graph; means for identifying a partition for each edge based on the first line and the second line; and means for segmenting the graph by assigning each edge to a corresponding partition.

[0139] In some exemplary embodiments, the components for identifying partitions for each edge may include: components for determining points in a finite projective plane based on a first line and a second line; and components for determining partitions corresponding to points for each edge.

[0140] In some exemplary embodiments, the means for determining a point in a finite projective plane based on a first line and a second line may include: means for determining a point as the intersection of the first line and the second line in response to determining that the first line and the second line are different lines.

[0141] In some exemplary embodiments, the means for determining a point in a finite projective plane based on a first line and a second line may include: means for determining a point as a point in a line based on a line-to-point mapping in response to determining that the first line and the second line are the same line.

[0142] In some exemplary embodiments, the apparatus may further include: components for determining the intersection points of the multiple lines; components for determining points corresponding to the multiple lines respectively according to a mapping; and components for storing the determined intersection points and points associated with the multiple lines. In some example embodiments, the mapping may be formed using the Kuhn algorithm.

[0143] In some exemplary embodiments, the number of partitions can be the value of an irreducible polynomial that is a prime power.

[0144] In some exemplary embodiments, the apparatus may further include: a component for obtaining the input number of partitions; and a component for determining the number of partitions to be less than or equal to the maximum value of the input number of partitions. In some exemplary embodiments, the number of points and the number of lines may be equal to the number of partitions.

[0145] In some exemplary embodiments, each line may include a prime power plus one point. In some exemplary embodiments, each point may be represented as an array of three elements having a finite field of the order of a prime power. In some exemplary embodiments, each of the multiple lines may be defined by a point such that the scalar product of all points in the line with that point is equal to zero.

[0146] In some exemplary embodiments, the component for determining a first line and a second line among a plurality of lines may include a component of a function for determining the indices of the first line and the second line as the identities (IDs) of two vertices of the edges, respectively.

[0147] In some exemplary embodiments, the function can be defined by the remainder of the vertex ID divided by the number of partitions.

[0148] In some exemplary embodiments, the apparatus may include components for distributing the segmented graph to multiple computing devices for graph processing.

[0149] Figure 5 A schematic block diagram of a device 500 that can be used to implement embodiments of the present invention is shown. Device 500 can be the device or apparatus described in the embodiments of the present invention, such as computing device 110. Figure 5As shown, device 500 includes a processor 501, which can perform various appropriate actions and processes according to computer program instructions to execute the method of the present invention (e.g., method 200). For example, the computer program instructions can be stored in read-only memory (ROM) 502 or loaded from storage unit 508 into random access memory (RAM) 503. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504. Furthermore, although not shown in FIG. 9, device 500 may also include a coprocessor.

[0150] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various displays, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless communication transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks (such as the Internet) and / or various telecommunications networks.

[0151] The various methods or processes described above can be executed by processor 501. For example, in some embodiments, the method may be embodied as a computer software program tangibly included in a machine-readable medium, such as in storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed onto device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by processor 501, one or more steps or actions of the methods or processes described herein may be performed.

[0152] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for performing various aspects of the present invention are loaded.

[0153] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. For example, computer-readable storage media can be, but is not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (not an exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punched cards or raised structures in recesses storing instructions, and any suitable combination of the foregoing. The computer-readable storage media used herein should not be construed as transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through optical fibers), or electrical signals transmitted through wires.

[0154] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded to external computers or external storage devices via networks such as the Internet, local area networks, wide area networks, and / or wireless networks. Networks may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives and forwards the computer-readable program instructions from the network for storage on a computer-readable storage medium within each computing / processing device.

[0155] The computer program instructions used to perform the operations of this invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​and traditional procedural programming languages). The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer as a standalone software package, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or to an external computer (e.g., connected to the Internet via an Internet service provider). In some embodiments, electronic circuits such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) are customized using state information from the computer-readable program instructions. The electronic circuits can execute the computer-readable program instructions to implement various aspects of this invention.

[0156] These computer-readable program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, or another programmable data processing apparatus to create a machine such that, when executed by the processing unit of the computer or the other programmable data processing apparatus, components are created to implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Therefore, a computer-readable medium storing instructions includes an article of writing comprising instructions that implement aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0157] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other equipment, such that a series of operational steps can be performed on the computer, other programmable data processing apparatus or other equipment to produce a computer-implemented process, such that the instructions that execute on the computer, other programmable data processing apparatus or other equipment can perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a portion of a module, program segment, or instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may appear in a different order than those marked in the drawings. For example, two consecutive blocks may actually execute substantially simultaneously, and sometimes they may execute in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of special hardware and computer instructions.

[0159] Various embodiments of the invention have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the illustrated embodiments. The terminology used herein is chosen to best explain the principles and practical application of the various embodiments or improvements to the technology on the market, or to enable others skilled in the art to understand the various embodiments disclosed herein.

Claims

1. A method, characterized in that, include: Based on the number of partitions in the graph, multiple points and multiple lines in a finite projective plane are determined, where each point corresponds to one of the partitions, and each line includes a subset of the multiple points; The first and second lines among the multiple lines are determined based on the vertices of each edge in the graph. Each edge is partitioned based on the first line and the second line; The graph is partitioned by assigning each edge to a corresponding partition.

2. The method according to claim 1, characterized in that, Identifying partitions for each edge includes: Points in the finite projective plane are determined based on the first line and the second line; Determine the partition corresponding to the point for the edge.

3. The method according to claim 2, characterized in that, Determining points in the finite projective plane based on the first line and the second line includes: In response to determining that the first line and the second line are different lines, the point is determined as the intersection of the first line and the second line.

4. The method according to claim 3, characterized in that, Determining points in the finite projective plane based on the first line and the second line further includes: In response to determining that the first line and the second line are the same line, the point is determined as a point in the line according to the line-to-point mapping.

5. The method according to claim 4, characterized in that, Also includes: Determine the intersection points of the multiple lines; Based on the mapping, determine the points corresponding to the multiple lines respectively; Store the determined intersections and points associated with the multiple lines.

6. The method according to claim 4, characterized in that, The mapping is formed by the Kuhn algorithm.

7. The method according to any one of claims 1 to 5, characterized in that, The number of partitions is the value of an irreducible polynomial raised to the power of a prime number.

8. The method according to claim 7, characterized in that, Also includes: Get the number of partitions input; The number of partitions is determined to be the maximum number that is less than or equal to the input number of partitions.

9. The method according to any one of claims 1 to 8, characterized in that, The number of points and the number of lines are equal to the number of partitions.

10. The method according to any one of claims 7 to 9, characterized in that, Each line includes the prime power plus 1 point.

11. The method according to any one of claims 7 to 10, characterized in that, Each point is represented as an array of three elements having a finite field, the finite field having the order of the prime power.

12. The method according to claim 11, characterized in that, Each of the plurality of lines is defined by one of the points such that all points in the line have a scalar product equal to zero with the point.

13. The method according to any one of claims 1 to 12, characterized in that, Determining the first and second lines among the plurality of lines includes: The indices of the first line and the second line are determined as functions of the identity (ID) of the two vertices of the edge.

14. The method according to claim 13, characterized in that, The function is defined by the remainder when the vertex ID is divided by the number of partitions.

15. The method according to any one of claims 1 to 14, characterized in that, Also includes: The segmented graph is distributed to multiple computing devices for graph processing.

16. An electronic device, characterized in that, include: processor; A memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, cause the device to perform an action including the following steps: Based on the number of partitions in the graph, multiple points and multiple lines in a finite projective plane are determined, where each point corresponds to one of the partitions, and each line includes a subset of the multiple points; The first and second lines among the multiple lines are determined based on the vertices of each edge in the graph. Each edge is partitioned based on the first line and the second line; The graph is partitioned by assigning each edge to a corresponding partition.

17. An apparatus, characterized in that, include: A component for determining multiple points and multiple lines in a finite projective plane based on the number of partitions in a graph, wherein each point corresponds to one of the partitions and each line comprises a subset of the multiple points; Components for determining the first and second lines among the multiple lines based on the vertices of each edge among the multiple edges of the graph; A component for identifying partitions for each edge based on the first line and the second line; A component used to segment the graph by assigning each edge to a corresponding partition.

18. A computer program product tangibly stored in a computer-readable medium and comprising machine-executable instructions, characterized in that, When the machine-executable instructions are executed, the machine performs the method according to any one of claims 1 to 15.

Images