Distributed dynamic graph neural network training-oriented graph data partitioning method

Abstract

The invention provides a graph data partitioning method for distributed dynamic graph neural network training, and relates to the technical field of distributed dynamic graph neural network training and graph partitioning. The method specifically comprises the following steps: acquiring a plurality of continuous original snapshots and constructing an original snapshot set; reducing the original snapshot set by adopting a second-order construction frequency reduction strategy, and calculating the total use frequency of each original snapshot in the original snapshot set; according to the method, a part of original snapshots are reserved from an original snapshot set by adopting a frequency-based adaptive snapshot reservation strategy, a weighted combined graph containing all edges in the reserved original snapshots is constructed, and then the weighted combined graph is subjected to graph division by adopting an existing static graph division algorithm, so that a division result of the original snapshot set is obtained. According to the method, high-quality large-scale discrete time dynamic graph data division operation is effectively and efficiently supported, so that the cross-node communication overhead is reduced, and the training efficiency of a large-scale distributed dynamic graph neural network system is further improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of distributed dynamic graph neural network training and graph partitioning, and particularly relates to a graph data partitioning method for distributed dynamic graph neural network training. Background Art

[0002] With the rapid development of the Internet and mobile communication, discrete time dynamic graph (DTDG) data is generated in large quantities and collected in the form of regular snapshots, such as the daily activities of social media, the vehicle flow per minute of traffic sensors, and the hourly data of weather stations. The edge-life technology enhances temporal smoothness and alleviates the instability of dynamic changes by extending the life cycle of edges, and has become an important means to optimize discrete time dynamic graph modeling. Currently, the dynamic graph neural network (DGNN) is the state-of-the-art technology for processing discrete time dynamic graphs. It combines the graph neural network (GNN) to capture graph structure changes and the recursive neural network (RNN) to model temporal dynamics, achieving efficient analysis of discrete time dynamic graph data. The sliding window-based DGNN training is a common training mode. By using N s (N s (N < T) snapshots to replace all T snapshots for training, the computational complexity is reduced and the sample diversity is enhanced, thereby improving the training efficiency and the generalization ability of the model. However, even with the sliding window, the single machine processing ability is still difficult to meet the training requirements of large-scale discrete time dynamic graphs. Therefore, distributed dynamic graph neural network training has become a hot topic in current research, and graph partitioning, as the primary task of large-scale distributed dynamic graph neural network training, significantly affects the training efficiency of the system.

[0003] Graph partitioning technology splits large-scale graph data into multiple subgraphs and distributes them to different computing nodes or machines for parallel processing to optimize computing efficiency and reduce inter-node communication overhead. The quality of partitioning directly affects the performance of distributed dynamic graph neural network training. A good partitioning strategy can improve training efficiency, reduce memory usage, and accelerate the overall computing process. Existing static graph partitioning methods, including Hash, Metis, Scotch, SBV, and TopoX, perform well in static graph partitioning, but have poor adaptability to discrete-time dynamic graphs. This is because reducing communication within a single snapshot alone cannot effectively reduce the communication overhead between snapshots of the entire discrete-time dynamic graph. Existing discrete-time dynamic graph partitioning strategies such as DGC and DynaHB are mainly aimed at DGNN training of full snapshots and cannot meet the needs of sliding window training. However, splicing discrete-time dynamic graphs into a single graph will bring significant overhead. Therefore, there is an urgent need for a lightweight and effective graph partitioning strategy to meet the requirements of sliding window DGNN training. Summary of the invention

[0004] In view of the shortcomings of the above-mentioned prior art, the present invention considers factors such as the edge-life conversion technology of the edge and the sliding window, uses the original snapshot data before the smaller-scale edge-life conversion technology, and analyzes the communication frequency differences between different snapshots. A graph data partitioning method for distributed dynamic graph neural network training is proposed, aiming to effectively and efficiently support high-quality large-scale discrete-time dynamic graph data partitioning operations, thereby reducing cross-node communication overhead and further improving the training efficiency of large-scale distributed dynamic graph neural network systems.

[0005] The present invention proposes a graph data partitioning method for distributed dynamic graph neural network training, the method comprising:

[0006] Get several consecutive original snapshots and construct an original snapshot set R;

[0007] The original snapshot set R is restored using the second-order construction count restoration strategy, and the total number of times each original snapshot in the original snapshot set R is calculated;

[0008] According to the total number of times each original snapshot in the original snapshot set R is used, a frequency-based adaptive snapshot retention strategy is used to retain some original snapshots from the original snapshot set R, and a weighted merged graph containing all edges in the retained original snapshots is constructed. Then, the weighted merged graph is partitioned using an existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R.

[0009] The method for restoring the original snapshot set R by using the second-order construction frequency restoration strategy is as follows: the original snapshot set R is converted into T snapshots G by using the edge-life conversion technology, a sliding window is set and a first-order frequency restoration array is constructed by calculating the number of times each snapshot is used in the sliding window, and then the first-order frequency restoration array is used to calculate the number of times each original snapshot in any snapshot is used and a second-order frequency restoration matrix is ​​constructed, and the total number of times each original snapshot in the original snapshot set R is obtained by summing the columns of the second-order frequency restoration matrix;

[0010] The process of restoring the original snapshot set R using the second-order construction number restoration strategy includes the following steps:

[0011] Step I-1: Set the length N of the sliding window S s and the time span z of the edge-life conversion technique, the number of original snapshots in the original snapshot set R is recorded as N r , and uses edge-life conversion technology to convert N r The original snapshots are converted into T snapshots G;

[0012] Step I-2: For any snapshot G t , calculate the snapshot G in the sliding window S t Number of times UG is used t , and build a first-order frequency reduction array based on the number of times all snapshots are used; where t represents the index of the snapshot, and the value of t is any integer in the interval [0, T-1];

[0013] Step I-3: Restore the array based on the first-order order. For any snapshot G t , using the edge-life conversion technique to calculate the snapshot G t The number of times each original snapshot is used in the snapshots, and a second-order number of times reduction matrix is ​​constructed according to the number of times each original snapshot is used in all snapshots; the rows of the second-order number of times reduction matrix are used to represent the number of times each original snapshot is used in a specific snapshot; the columns of the second-order number of times reduction matrix are used to represent the number of times an original snapshot is used in each snapshot;

[0014] Step I-4: Sum each column of the second-order frequency reduction matrix to obtain the total number of times each original snapshot is used AR r ;

[0015] In step I-1, the edge-life conversion technique is used to convert N rThe method of converting original snapshots into T snapshots G is as follows: based on the set time span z of the edge-life conversion technology, for any original snapshot, the edge-life conversion technology forms a snapshot G by retaining the edge of the original snapshot to the subsequent z original snapshots; each snapshot G is composed of multiple original snapshots, and the sliding window S contains multiple snapshots G;

[0016] The snapshot G in the sliding window S in step I-2 t Number of times UG is used t It is expressed as:

[0017] UG t =min(t+1, Tt, N s )

[0018] Where min(·) represents the minimum value operation;

[0019] The first-order reduction array is expressed as:

[0020] {UG0, UG1, ..., UG T-1}

[0021] UG0 is the number of times the first snapshot G0 is used; UG1 is the number of times the second snapshot G1 is used; T-1 is the Tth snapshot G T-1 The number of times it is used;

[0022] The method for constructing the second-order reduction matrix in step I-3 is: for snapshot G t Any original snapshot R r , calculate the snapshot G t Original snapshot R r Number of times UR is used (t，r) ; where r is the index of the original snapshot, and the value of r is in the interval [0, N r -1];

[0023]

[0024] When the original snapshot R r Located in Snapshot G t When the time span of the edge-life conversion technique is within the time span, the snapshot G in the sliding window S t Number of times UG is used t As a snapshot G t Original snapshot R r Number of times UR is used (t，r) ; When the original snapshot R r Not in snapshot G t The edge-life conversion technology is within the time span of the snapshot Gt Original snapshot R r Number of times UR is used (t，r) is 0;

[0025] According to the original snapshot R in the snapshot Gt r Number of times UR is used (t，r) , construct the second-order power reduction matrix, expressed as:

[0026]

[0027] Among them UR (0,0) Indicates the number of times the first original snapshot in the first snapshot G0 is used; Indicates the Nth snapshot in the first snapshot G0 r The number of times the original snapshot is used; UR (T-1，0) is the Tth snapshot G T-1 The number of times the first original snapshot is used; is the Tth snapshot G T-1 N r The number of times the original snapshot was used;

[0028] The specific content of the frequency-based adaptive snapshot retention strategy is: calculating the threshold δ by counting the highest number of times the original snapshots in the original snapshot set R are used, and the threshold δ is used to divide the original snapshots to be retained from the original snapshot set R, and all edges in the retained original snapshots are merged to construct a weighted merged graph containing all edges in the retained original snapshots, and then the weighted merged graph is partitioned using the existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R;

[0029] The process of the frequency-based adaptive snapshot retention strategy includes the following steps:

[0030] Step II-1: Based on the total usage count of each original snapshot in the original snapshot set R, count the highest usage count of the original snapshot in the original snapshot set R and record it as r max , and count the number of times used as r max The number of original snapshots According to and r max Calculate the threshold δ;

[0031] The threshold δ is expressed as:

[0032]

[0033] Where δ represents the threshold used to filter the original snapshots; Indicates rounding up operation;

[0034] Step II-2: Calculate the original snapshot retention interval [R begin , R end ) starts at begin and ends at end; where R begin Represents the original snapshot at the starting position begin; R end Represents the original snapshot at the end position end;

[0035] Step II-3: Retain intervals according to the original snapshot begin , R end ) Divide the original snapshots that need to be retained from the original snapshot set R, and construct a weighted merged graph G containing all the edges in the retained original snapshots by merging the edges in all the divided original snapshots ret ;

[0036] Step II-4: Use the existing static graph partitioning algorithm to weighted merge graph G ret Perform graph partitioning to obtain the partitioning result of the original snapshot;

[0037] The starting position begin and the ending position end in step II-2 are expressed as:

[0038]

[0039] end=N r -begin

[0040] AR i Represents the original snapshot R i The total number of times it is used; i is the index of the original snapshot; AR i ≥δ represents the filter condition of retaining the original snapshot, that is, filtering out the original snapshots whose usage times are greater than or equal to the threshold δ; {i|·} represents the set of indexes of all original snapshots that meet the filter condition; Represents the operation of finding the minimum index that satisfies the filtering condition;

[0041] Step II-3 constructs a weighted merge graph G containing all the edges in the original snapshot that are preserved. ret The method is: merge all the edges in the original snapshot to construct the edge set of the original snapshot. For any edge e in the edge set, count the edge e in the original snapshot retention interval [R begin , R end ) in all original snapshots and use it as the weight of edge e. The edge set E of the weighted merged graph is constructed using edge e and edge e’s weight count(e). ret , expressed as:

[0042]

[0043] Where E i represents the set of edges in the i-th original snapshot; Represents the union operation of the edges in all original snapshots ranging from index begin to end-1;

[0044] Construct the vertex set V of the weighted merged graph based on the vertices of all original snapshots in the original snapshot set R, and use the edge set E of the weighted merged graph ret and the vertex set V of the weighted merged graph, construct a weighted merged graph G containing all the edges in the original snapshot that are retained ret , expressed as:

[0045] G ret =(V,E ret ).

[0046] The beneficial effects of adopting the above technical solution are:

[0047] The graph data partitioning method for distributed dynamic graph neural network training proposed by the method of the present invention is the first graph partitioning method for discrete-time dynamic graphs, providing important technical support for future distributed training of discrete-time dynamic graphs. Considering the characteristics of edge-life conversion technology in discrete-time dynamic graphs and the ability to support DGNN training based on sliding windows, the method of the present invention uses the original snapshot data before the edge-life conversion technology with a smaller scale by considering factors such as edge life cycle conversion technology and sliding windows, and designs an optimized graph partitioning method by analyzing the differences in communication frequencies between different original snapshots, thereby reducing the total communication volume between sliding windows and reducing the cost of data partitioning.

[0048] The frequency-based adaptive snapshot retention strategy designed by the method of the present invention focuses on weighted edge merging of frequently and stably used original snapshots, achieving high-quality graph partitioning for discrete-time dynamic graphs, significantly reducing the communication overhead caused by graph partitioning, and maximizing the computational efficiency of partition preprocessing.

[0049] The method of the present invention effectively optimizes the communication and computing bottlenecks in the distributed dynamic graph neural network training system, significantly improves the training efficiency of the dynamic graph neural network, especially in high-frequency communication and large-scale distributed dynamic graph neural network training scenarios, and significantly enhances the processing capability and scalability of the distributed dynamic graph neural network training system.

[0050] The method of the present invention effectively improves the training speed of the dynamic graph neural network model, thereby promoting research innovation in the field of dynamic graph neural network algorithms and the rapid implementation of practical projects. For example: in the field of transportation, it can be applied to real-time traffic flow prediction and optimization of intelligent transportation systems; in the financial field, it can assist in risk assessment and behavior prediction of dynamic trading networks; in social networks, it can support user behavior modeling and accurate recommendations; in addition, the method of the present invention can also be applied to dynamic load prediction and scheduling optimization of power systems, as well as dynamic network analysis in bioinformatics. The above applications demonstrate the broad value and potential of the method of the present invention in multiple industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a graph data partitioning method for distributed dynamic graph neural network training in this embodiment;

[0052] Figure 2 This is an example diagram of the second-order construction number reduction strategy in this implementation method;

[0053] Figure 3 This is an example diagram of the frequency-based adaptive snapshot retention policy in this implementation. DETAILED DESCRIPTION

[0054] In order to facilitate the understanding of the present application, the specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thoroughly understood.

[0055] The graph data partitioning method for distributed dynamic graph neural network training in this embodiment is as follows: Figure 1 As shown, the method includes:

[0056] Get several consecutive raw snapshots and construct a raw snapshot set R.

[0057] In this embodiment, real data sets are obtained from open source websites, such as Slashdot, Stackexch, Ratings, and Flickr, which are also commonly used data sets in current distributed dynamic graph neural network systems, and several continuous original snapshot graphs are obtained from the above real data sets as the original discrete-time dynamic graph.

[0058] The original snapshot set R is restored using the second-order build count restoration strategy, and the total usage count of each original snapshot in the original snapshot set R is calculated.

[0059] The method for restoring the original snapshot set R using the second-order construction frequency restoration strategy is as follows: the original snapshot set R is converted into T snapshots G using the edge-life conversion technology, a sliding window is set, and a first-order frequency restoration array is constructed by calculating the number of times each snapshot is used in the sliding window, and then the first-order frequency restoration array is used to calculate the number of times each original snapshot in any snapshot is used and a second-order frequency restoration matrix is ​​constructed, and the total number of times each original snapshot in the original snapshot set R is obtained by summing the columns of the second-order frequency restoration matrix.

[0060] In this embodiment, if Figure 2 As shown in the figure, the second-order construction frequency reduction strategy comprehensively considers the impact of edge-life conversion technology and sliding window mode on communication during the training process of dynamic graph neural network, combines the data characteristics and training mode of discrete-time dynamic graph, and calculates the number of times the original snapshot is used in the training process of distributed dynamic graph neural network (Dynamic Graph Neural Network, DGNN) based on sliding window by constructing the first-order frequency reduction array and the second-order frequency reduction matrix. The original snapshot refers to the state of graph data at a specific time point. In DGNN training, snapshots processed by edge-life conversion technology are usually used as graph data representing a specific time point for training. Under the sliding window mechanism, these snapshots are used to construct serialized discrete-time dynamic graph data so that the model can effectively capture the dynamic evolution of graph data in the time dimension. Therefore, before the start of DGNN training, it is necessary to determine the number of times each original snapshot appears in these sequences in order to construct the corresponding training samples to ensure the consistency and predictability of data during training and optimize the efficiency of the training process.

[0061] The process of restoring the original snapshot set R using the second-order construction number restoration strategy includes the following steps:

[0062] Step I-1: Set the length N of the sliding window S s and the time span z of the edge-life conversion technique, the number of original snapshots in the original snapshot set R is recorded as N r , and uses edge-life conversion technology to convert N r The original snapshots are converted into T snapshots G;

[0063] The edge-life conversion technology is used to convert N rThe method for converting original snapshots into T snapshots G is as follows: based on the set time span z of the edge-life conversion technology, for any original snapshot, the edge-life conversion technology retains the edges of the original snapshot to the subsequent z original snapshots to form a snapshot G; each snapshot G is composed of multiple original snapshots, and the sliding window S contains multiple snapshots G.

[0064] In this embodiment, if Figure 2 As shown, input the original snapshot set {R0, ..., R7}, record the hyperparameters such as the number of original snapshots N r To 8, set the length of the sliding window N s The time span z of the edge-life conversion technology is set to 2, and the time span z of the edge-life conversion technology is set to 3; the edge-life conversion technology is used to generate T snapshots G. The discrete time dynamic graph, that is, the continuous original snapshots, are naturally formed into a new discrete time dynamic graph, that is, continuous snapshots after edge-life conversion, and the continuous snapshots are used for DGNN training, where T can be obtained from the data set used for DGNN training, or by the expression T = N r -z+1 is calculated, so the total number T of snapshots G obtained by applying the edge-life conversion technology in this implementation manner is 6.

[0065] Step I-2: For any snapshot G t , calculate the snapshot G in the sliding window S t Number of times UG is used t , and construct a first-order frequency reduction array based on the number of times all snapshots are used; where t represents the index of the snapshot, and the value of t is any integer in the interval [0, T-1].

[0066] The snapshot G within the sliding window S t Number of times UG is used t It is expressed as:

[0067] UG t =min(t+1, Tt, N s ) (1)

[0068] Where min(·) represents the minimum value operation.

[0069] The first-order reduction array is expressed as:

[0070] {UG0, UG1, ..., UG T-1} (2)

[0071] UG0 is the number of times the first snapshot G0 is used; UG1 is the number of times the second snapshot G1 is used; T-1 is the Tth snapshot G T-1The number of times it is used.

[0072] In this embodiment, if Figure 2 As shown, for T snapshots G0, G1, ..., G T-1 , calculate each snapshot G in the sliding window S separately t The number of times the sliding window S0 contains G0 and G1, and S1 contains G1 and G2, and so on. Under the constraints of the sliding window length and snapshot length, the formula (1) is used to calculate the number of times each snapshot G in the sliding window S is used. t Number of times UG is used t Taking G0 as an example, UG0=min(0+1, 6-0, 2)=1, thereby calculating the number of times all snapshots are used in the sliding window and constructing a first-order number reduction array.

[0073] Step I-3: Restore the array based on the first-order order. For any snapshot G t , using the edge-life conversion technique to calculate the snapshot G t The number of times each original snapshot is used in the snapshots, and a second-order number of times restoration matrix is ​​constructed according to the number of times each original snapshot is used in all snapshots; the rows of the second-order number of times restoration matrix are used to represent the number of times each original snapshot is used in a specific snapshot; the columns of the second-order number of times restoration matrix are used to represent the number of times an original snapshot is used in each snapshot.

[0074] The method for constructing the second-order reduction matrix is ​​as follows: for the snapshot G t Any original snapshot R r , calculate the snapshot G t Original snapshot R r Number of times UR is used (t，r) ; where r is the index of the original snapshot, and the value of r is in the interval [0, N r -1];

[0075]

[0076] When the original snapshot R r Located in Snapshot G t When the time span of the edge-life conversion technique is within the time span, the snapshot G in the sliding window S t Number of times UG is used t As a snapshot G t Original snapshot R r Number of times UR is used (t，r) ; When the original snapshot R r Not in snapshot G t The edge-life conversion technique is used to convert the time span of snapshot Gt to the original snapshot R. rUsage times UR (t，r) is 0.

[0077] According to the usage times UR r of the original snapshot R within the snapshot Gt (t，r) , construct a second-order degree reduction matrix, expressed as:

[0078]

[0079] where UR (0，0) represents the usage times of the first original snapshot in the first snapshot G0; represents the usage times of the Nth r original snapshot in the first snapshot G0; UR (T-1，0) is the usage times of the first original snapshot in the Tth snapshot G T-1 ; is the usage times of the Nth T-1 original snapshot in the Tth snapshot G r .

[0080] In this embodiment, as shown in Figure 2 , for the N r original snapshots , use the time span z of the edge-life conversion technology to calculate the usage times of each original snapshot R t in the snapshot G r respectively, and construct a second-order degree reduction matrix. For the original snapshots, G0 uses R0, R1 and R2, while G1 uses R1, R2 and R3. Combining the usage times UG t of each snapshot G t in the sliding window S, calculate the usage times of each original snapshot R t in the snapshot G r according to formula (3). When the original snapshot R r is within the time span z of the edge-life conversion technology of the snapshot G t , that is, when t ≤ r < t + z, the usage times UR r of the original snapshot R t in G (t，r) is equal to the usage times UG t of G t . If it is not within the effective range, that is, r < t or t + z ≤ r, UR (t，r) is 0. Taking R0 and R1 as examples, the usage times UR (0，0) of R0 under G0 = UG0 = 1, and the usage times UR (0，1) of R1 = UG0 = 1. The usage times UR (1，1)=UG1=2, thereby calculating the number of times all original snapshots are used in the snapshots, and constructing a second-order frequency reduction matrix.

[0081] Step I-4: Sum each column of the second-order frequency reduction matrix to obtain the total number of times each original snapshot is used AR r .

[0082] The total number of times the original snapshot is used AR r for:

[0083]

[0084] in Represents the original snapshot R in the second-order restoration matrix r The usage count in all snapshots is accumulated and summed.

[0085] In this implementation, the usage count of each original snapshot is the sum of its usage counts in all snapshots. t Each original snapshot R r Number of times UR is used (t，r) , calculate the original snapshot R according to formula (5) r Taking R1 as an example, the number of times R1 is used under G0 is UR (0，1) =1, the number of times R1 is used under G1 is UR (1，1) =2, add them together to get the total number of times R1 is used AR r =UR (0，1) +UR (1，1) =3, and the usage count of all original snapshots is calculated.

[0086] According to the total number of times each original snapshot in the original snapshot set R is used, a frequency-based adaptive snapshot retention strategy is adopted to retain some original snapshots from the original snapshot set R, and a weighted merged graph containing all edges in the retained original snapshots is constructed. The weighted merged graph is then partitioned using the existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R.

[0087] The specific content of the frequency-based adaptive snapshot retention strategy is: the threshold δ is calculated by counting the highest number of times the original snapshots in the original snapshot set R are used, and the threshold δ is used to divide the original snapshots that need to be retained from the original snapshot set R, and all edges in the retained original snapshots are merged to construct a weighted merged graph containing all edges in the retained original snapshots, and then the weighted merged graph is partitioned using the existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R.

[0088] In this embodiment, if Figure 3As shown in the figure, based on the symmetrical distribution and variation characteristics of snapshot usage frequency, the retention threshold is first calculated to filter high-frequency snapshots, and the threshold δ is designed according to the data characteristics of snapshot usage frequency. Then, the original snapshot retention interval is calculated and the original snapshots to be retained are filtered out from the original snapshot set. The edges are merged based on the screening results to complete the construction of the weighted merged graph. Finally, the weighted merged graph is input into a static graph partitioning algorithm such as Metis to complete the graph partitioning and obtain the original snapshot, that is, the graph partitioning result of the discrete-time dynamic graph. The frequency-based adaptive snapshot retention strategy significantly reduces the partitioning preprocessing time on the basis of fully considering the edge-life conversion technology and sliding window factors. This technology uses smaller original snapshots for snapshot screening and weighted merged graph construction, combined with the characteristics of discrete-time dynamic graph edges and training patterns, to achieve the partitioning of high-quality discrete-time dynamic graphs, significantly reducing communication overhead and improving partitioning efficiency.

[0089] The process of constructing a weighted merged graph containing edges in all original snapshots using a frequency-based adaptive snapshot retention strategy comprises the following steps:

[0090] Step II-1: Based on the total usage count of each original snapshot in the original snapshot set R, count the highest usage count of the original snapshot in the original snapshot set R and record it as r max , and count the number of times used as r max The number of original snapshots According to and r max Calculate the threshold δ.

[0091] The threshold δ is expressed as:

[0092]

[0093] Where δ represents the threshold used to filter the original snapshots; Indicates a round-up operation.

[0094] In this embodiment, if Figure 3 As shown, based on the total number of times the original snapshot is used AR r It can be seen that in the training of DGNN, there are significant differences in the usage frequency of the original snapshots. Some original snapshots appear repeatedly in multiple sliding windows, while some are hardly used. In formula (6), Represents the ratio of snapshots with the maximum number of uses among all original snapshots. Then, multiply this ratio by the maximum number of uses r max , and an expected value based on actual usage is obtained. Finally, by rounding up, δ is ensured to be an integer, forming a threshold that can actually filter and retain the original snapshots with high usage frequency and stability. rThe usage frequency of is centrally symmetrically distributed and decreases from the middle to both sides. Therefore, the design can adaptively adjust the formula of the threshold δ according to the characteristics of different data sets. Since the threshold δ is calculated based on the usage frequency of the snapshots in the actual training process, the threshold δ has dynamic adaptability.

[0095] Step II-2: Calculate the original snapshot retention interval [R begin , R end ) starts at begin and ends at end; where R begin Represents the original snapshot at the starting position begin; R end Represents the original snapshot at end position end.

[0096] The starting position begin and the ending position end are expressed as:

[0097]

[0098] end=N r -begin (8)

[0099] AR i Represents the original snapshot R i The total number of times it is used; i is the index of the original snapshot; AR i ≥δ represents the filter condition of retaining the original snapshot, that is, filtering out the original snapshots whose usage times are greater than or equal to the threshold δ; {i|·} represents the set of indexes of all original snapshots that meet the filter condition; Represents the operation of finding the minimum index that satisfies the filter condition.

[0100] In this embodiment, it is necessary to retain a portion of snapshots from the original snapshot set R for subsequent edge merging operations. The threshold δ represents the standard for high-frequency use of snapshots. Only snapshots whose usage times reach or exceed the threshold δ will be retained for subsequent training. Since the usage frequency of the original snapshot R presents a centrally symmetrical distribution and decreases from the center to both sides, the first snapshot whose usage times reach or exceed the threshold δ can be determined as the starting position of the snapshot interval, and the end position can be calculated based on its symmetry by subtracting the starting position from the total number of snapshots. This embodiment uses formula (7) and formula (8) to calculate the starting position of the original snapshot end=N r -begin = 8 - 1 = 7. Therefore, the original snapshot retained is [R1, R7) = {R1, R2, ..., R6}.

[0101] Step II-3: Retain intervals according to the original snapshotbegin , R end ) Divide the original snapshots that need to be retained from the original snapshot set R, and construct a weighted merged graph G containing all the edges in the retained original snapshots by merging the edges in all the divided original snapshots ret .

[0102] The construction contains a weighted merged graph G containing all edges in the original snapshot that are preserved ret The method is: merge all the edges in the original snapshot to construct the edge set of the original snapshot. For any edge e in the edge set, count the edge e in the original snapshot retention interval [R begin , R end ) in all original snapshots and use it as the weight of edge e. The edge set E of the weighted merged graph is constructed using edge e and edge e’s weight count(e). ret , expressed as:

[0103]

[0104] Where E i represents the set of edges in the i-th original snapshot; Represents the union operation of the edges in all original snapshots from index begin to end-1.

[0105] Construct the vertex set V of the weighted merged graph based on the vertices of all original snapshots in the original snapshot set R, and use the edge set E of the weighted merged graph ret and the vertex set V of the weighted merged graph, construct a weighted merged graph G containing all the edges in the original snapshot that are retained ret , expressed as:

[0106] G ret =(V, E ret ) (10)

[0107] Step II-4: Use the existing static graph partitioning algorithm to weighted merge graph G ret Perform graph partitioning and obtain the partitioning result of the original snapshot.

[0108] Calculate the weighted merge graph G ret Afterwards, the existing static graph partitioning algorithm is used to partition the graph G ret Perform graph partitioning to generate the partitioning result of the original snapshot, that is, the final graph partitioning result PR for the discrete-time dynamic graph. There are many existing static graph partitioning algorithms, such as Hash, Metis, etc. This implementation method uses Metis to partition the weighted merged graph G. ret Metis is an efficient graph partitioning algorithm, whose goal is to merge the weighted graph G retDivide it into several subgraphs to minimize the number of split edges while maintaining the load balance of the subgraphs. Metis adopts a multi-level partitioning strategy (MultilevelPartitioning), which first aggregates the weighted merge graph G by aggregating vertices and edge pairs. ret The method is used to coarsen the graph to generate a smaller approximate graph; then perform the initial partitioning on the approximate graph; finally, restore the original graph by layer-by-layer refinement while optimizing the partitioning quality. This method is widely used in graph partitioning tasks due to its high efficiency and reliability.

[0109] This implementation method uses two strategies, namely a second-order construction frequency reduction strategy and a frequency-based adaptive snapshot retention strategy, to construct a weighted merged graph using smaller-scale original snapshot data, and uses a static graph partitioning algorithm, such as Metis, to generate partitioning results, thereby achieving an efficient and high-quality graph data partitioning for discrete-time dynamic graphs.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A graph data partitioning method for distributed dynamic graph neural network training, characterized in that: The method comprises the following steps: Get several consecutive original snapshots and construct an original snapshot set R; The original snapshot set R is restored using the second-order construction count restoration strategy, and the total number of times each original snapshot in the original snapshot set R is calculated; According to the total number of times each original snapshot in the original snapshot set R is used, a frequency-based adaptive snapshot retention strategy is adopted to retain some original snapshots from the original snapshot set R, and a weighted merged graph containing all edges in the retained original snapshots is constructed. The weighted merged graph is then partitioned using the existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R.

2. The graph data partitioning method for distributed dynamic graph neural network training according to claim 1 is characterized in that: The method for restoring the original snapshot set R using the second-order construction frequency restoration strategy is as follows: the original snapshot set R is converted into T snapshots G using the edge-life conversion technology, a sliding window is set, and a first-order frequency restoration array is constructed by calculating the number of times each snapshot is used in the sliding window, and then the first-order frequency restoration array is used to calculate the number of times each original snapshot in any snapshot is used and a second-order frequency restoration matrix is ​​constructed, and the total number of times each original snapshot in the original snapshot set R is obtained by summing the columns of the second-order frequency restoration matrix.

3. The graph data partitioning method for distributed dynamic graph neural network training according to claim 2 is characterized in that: The process of restoring the original snapshot set R using the second-order construction number restoration strategy includes the following steps: Step I-1: Set the length N of the sliding window S s and the time span z of the edge-life conversion technique, the number of original snapshots in the original snapshot set R is recorded as N r , and uses edge-life conversion technology to convert N r The original snapshots are converted into T snapshots G; Step I-2: For any snapshot G t , calculate the snapshot G in the sliding window S t Number of times UG is used t , and build a first-order frequency reduction array based on the number of times all snapshots are used; where t represents the index of the snapshot, and the value of t is any integer in the interval [0, T-1]; Step I-3: Restore the array based on the first-order order. For any snapshot G t , using the edge-life conversion technique to calculate the snapshot G t The number of times each original snapshot is used in the snapshots, and a second-order number of times reduction matrix is ​​constructed according to the number of times each original snapshot is used in all snapshots; the rows of the second-order number of times reduction matrix are used to represent the number of times each original snapshot is used in a specific snapshot; the columns of the second-order number of times reduction matrix are used to represent the number of times an original snapshot is used in each snapshot; Step I-4: Sum each column of the second-order frequency reduction matrix to obtain the total number of times each original snapshot is used AR r .

4. The graph data partitioning method for distributed dynamic graph neural network training according to claim 3 is characterized in that: In step I-1, the edge-life conversion technique is used to convert N r The method for converting original snapshots into T snapshots G is as follows: based on the set time span z of the edge-life conversion technology, for any original snapshot, the edge-life conversion technology retains the edges of the original snapshot to the subsequent z original snapshots to form a snapshot G; each snapshot G is composed of multiple original snapshots, and the sliding window S contains multiple snapshots G.

5. The graph data partitioning method for distributed dynamic graph neural network training according to claim 4 is characterized in that: The snapshot G in the sliding window S in step I-2 t Number of times UG is used t It is expressed as: and t =min(t+1,Tt,N s ) Where min(·) represents the minimum value operation; The first-order reduction array is expressed as: {UG0,UG1,…,UG T-1 } UG0 is the number of times the first snapshot G0 is used; UG1 is the number of times the second snapshot G1 is used; T-1 is the Tth snapshot G T-1 The number of times it is used.

6. The graph data partitioning method for distributed dynamic graph neural network training according to claim 5 is characterized in that: The method for constructing the second-order reduction matrix in step I-3 is: for snapshot G t Any original snapshot R r , calculate the snapshot G t Original snapshot R r Number of times UR is used (r,r) ; where r is the index of the original snapshot, and the value of r is in the interval [0, N r -1]; When the original snapshot R r Located in Snapshot G t When the time span of the edge-life conversion technique is within the time span, the snapshot G in the sliding window S t Number of times UG is used t As a snapshot G t Original snapshot R r Number of times UR is used (t,r) ; When the original snapshot R r Not in snapshot G t The edge-life conversion technology is within the time span of the snapshot G t Original snapshot R r Number of times UR is used (t,r) is 0; According to snapshot G t Original snapshot R r Number of times UR is used (t,r) , construct the second-order power reduction matrix, expressed as: Among them UR (0,0) Indicates the number of times the first original snapshot in the first snapshot G0 is used; Indicates the Nth snapshot in the first snapshot G0 r The number of times the original snapshot is used; UR (T-1,0) is the tth snapshot G T-1 The number of times the first original snapshot is used; is the Tth snapshot G T-1 N r The number of times the original snapshot was used.

7. The graph data partitioning method for distributed dynamic graph neural network training according to claim 6 is characterized in that: The specific content of the frequency-based adaptive snapshot retention strategy is: the threshold δ is calculated by counting the highest number of times the original snapshots in the original snapshot set R are used, and the threshold δ is used to divide the original snapshots that need to be retained from the original snapshot set R, and all edges in the retained original snapshots are merged to construct a weighted merged graph containing all edges in the retained original snapshots, and then the weighted merged graph is partitioned using the existing static graph partitioning algorithm to obtain the partitioning result of the original snapshot set R.

8. The graph data partitioning method for distributed dynamic graph neural network training according to claim 7 is characterized in that: The process of the frequency-based adaptive snapshot retention strategy includes the following steps: Step II-1: Based on the total number of times each original snapshot is used in the original snapshot set R, count the highest number of times the original snapshot is used in the original snapshot set R and record it as r max , and count the number of times used as r max The number of original snapshots According to and r max Calculate the threshold δ; The threshold δ is expressed as: Where δ represents the threshold used to filter the original snapshots; Indicates rounding up operation; Step II-2: Calculate the original snapshot retention interval [R begin ,R end ) starts at begin and ends at end; where R begin Represents the original snapshot at the starting position begin; R end Represents the original snapshot at the end position end; Step II-3: Retain interval [R begin ,R end ) Divide the original snapshots that need to be retained from the original snapshot set R, and construct a weighted merged graph G containing all the edges in the retained original snapshots by merging the edges in all the divided original snapshots ret ; Step II-4: Use the existing static graph partitioning algorithm to weighted merge graph G ret Perform graph partitioning and obtain the partitioning result of the original snapshot.

9. The graph data partitioning method for distributed dynamic graph neural network training according to claim 8 is characterized in that: The starting position begin and the ending position end in step II-2 are expressed as: end=N r -begin AR i Represents the original snapshot R i The total number of times it is used; i is the index of the original snapshot; AR i ≥δ represents the filter condition of retaining the original snapshot, that is, filtering out the original snapshots whose usage times are greater than or equal to the threshold δ; {i|·} represents the set of indexes of all original snapshots that meet the filter condition; Represents the operation of finding the minimum index that satisfies the filter condition.

10. The graph data partitioning method for distributed dynamic graph neural network training according to claim 9 is characterized in that: Step II-3 constructs a weighted merged graph G containing all the edges in the original snapshot that are retained ret The method is: merge all the edges in the original snapshot to construct the edge set of the original snapshot. For any edge e in the edge set, count the edge e in the original snapshot retention interval [R begin ,R end ) in all original snapshots and use it as the weight of edge e. The edge set E of the weighted merged graph is constructed using edge e and edge e’s weight count(e). ret , expressed as: Where E i represents the set of edges in the i-th original snapshot; Represents the union operation of the edges in all original snapshots ranging from index begin to end-1; Construct the vertex set V of the weighted merged graph based on the vertices of all original snapshots in the original snapshot set R, and use the edge set E of the weighted merged graph ret and the vertex set V of the weighted merged graph, construct a weighted merged graph G containing all the edges in the original snapshot that are retained ret , expressed as: G ret =(V,E ret )。

Images