A large-scale directed network topology fast generation method and system

Abstract

The present application relates to a kind of large-scale directed network topology fast generation method and system, belong to computer network field.Its generation method includes: obtaining user input parameter, calculating reciprocity, in-degree and out-degree sequence, creating empty topology and adding node, calculating expected reciprocity edge and directed edge number, calculating reciprocity edge and directed edge self-loop probability, calculating reciprocity edge sampling probability while applying excess degree edge complement strategy to obtain reciprocity edge sequence, calculating directed edge sampling probability while applying excess degree edge complement strategy to obtain directed edge sequence, add edge sequence to topology.The system framework includes: user input module, node degree sampling module, reciprocity edge sampling module, directed edge sampling module, network topology generation module.The present application is based on the dynamic construction idea, uses the current degree value of node in edge sampling probability instead of initial degree value, and reallocates the excess degree of each node, solves the problems of existing generation method, such as complex calculation, low network topology generation quality and slow generation time.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer networks, in particular to a large-scale directed network topology fast generation method and system. BACKGROUND

[0002] Network topology, as a graphical representation of how components (such as computers, routers, switches, etc.) in a network system are connected and communicate with each other, plays a crucial role in network design, management, optimization, and troubleshooting. However, the process of crawling network topology is usually very time-consuming, especially when they consist of millions of nodes and edges. In addition, due to the sensitivity of user data and data protection regulations, information in network topology needs to be handled carefully.

[0003] In order to generate a graph with similar characteristics to real-world network topology, an edge connection algorithm and an edge rewiring algorithm with a time complexity of O(n 3 ) are used to generate directed network topology (with reciprocal edges), which only relies on static node degree values during edge sampling. The construction method is complex and has high time complexity, making it difficult to meet the fast response requirements of large-scale directed network topology generation.

[0004] Therefore, the present application proposes a large-scale directed network topology fast generation method and system under the perspective of dynamic construction. SUMMARY

[0005] The present application proposes a large-scale directed network topology fast generation method and system to address the problems of static nature, complex construction method, and high time complexity in existing directed network topology construction. The dynamic construction method simplifies the construction method, improves the generation quality, and speeds up the generation time. The technical solution adopted by the present application is as follows:

[0006] A large-scale directed network topology fast generation method, the method comprising the following steps:

[0007] (1) obtaining user input parameters;

[0008] (2) calculating the linear correlation coefficient between the reciprocal degree, in-degree, and out-degree according to the parameters input by the user in step (1);

[0009] (3) sampling N times from the multivariate normal distribution according to the parameters input by the user in step (1) and the linear correlation coefficient calculated in step (2), and then applying the cumulative distribution function and the inverse cumulative distribution function to obtain the reciprocal degree sequence, the in-degree sequence, and the out-degree sequence;

[0010] (4) creating an empty topology and adding the number of nodes;

[0011] (5) calculating the expected number of reciprocal edges and the expected number of directed edges according to the sequence of reciprocity, the sequence of in-degree and the sequence of out-degree calculated in step (3);

[0012] (6) calculating the probability of reciprocal edge self-loop and the probability of directed edge self-loop according to the sequence of reciprocity, the sequence of in-degree and the sequence of out-degree calculated in step (3) and the expected number of reciprocal edges and the expected number of directed edges calculated in step (5);

[0013] (7) calculating the probability of reciprocal edge sampling according to the sequence of reciprocity calculated in step (3), the expected number of reciprocal edges calculated in step (5) and the probability of reciprocal edge self-loop calculated in step (6);

[0014] (8) double-loop traversing the node pairs, for each node i and node j (i

[0015] (9) calculating the probability of directed edge sampling according to the sequence of in-degree and the sequence of out-degree calculated in step (3), the expected number of directed edges calculated in step (5) and the probability of directed edge self-loop calculated in step (6);

[0016] (10) double-loop traversing the node pairs, for each node i and node j (i

[0017] (11) adding the sequence of reciprocal edge connection obtained in step (8) and the sequence of directed edge connection obtained in step (10) to the network topology created in step (4).

[0018] The user input parameters obtained in step (1) include: the total number of nodes, the rank correlation coefficient between reciprocity and in-degree, the rank correlation coefficient between reciprocity and out-degree, the rank correlation coefficient between in-degree and out-degree, the degree of freedom and position parameter of the reciprocity distribution, the degree of freedom and position parameter of the in-degree distribution, and the degree of freedom and position parameter of the out-degree distribution.

[0019] The linear correlation coefficient between reciprocity, in-degree and out-degree calculated in step (2) is: multiplying the rank correlation coefficient between reciprocity, in-degree and out-degree by π / 6 to take the sine value and then multiplying by 2.

[0020] The sampling N times from the multivariate normal distribution in step (3) is: generating N samples according to the probability density function of the multivariate normal distribution, where each sample is a 3-dimensional vector, and transposing the sample to obtain the sampled multivariate data.

[0021] The application of the cumulative distribution function and the inverse cumulative distribution function to obtain the reciprocity degree sequence, the in-degree sequence and the out-degree sequence in the above step (3) is: applying the cumulative distribution function (CDF) to normalize the sampled multivariate data into uniformly distributed data, and then applying the inverse cumulative distribution function (PPF) to convert the uniformly distributed data into power-law distribution data and take the integer to obtain the reciprocity degree sequence, the in-degree sequence and the out-degree sequence.

[0022] The calculation of the expected reciprocity edge number in the above step (5) is: the cumulative sum of the reciprocity degree sequence.

[0023] The calculation of the expected directed edge number in the above step (5) is: the cumulative sum of the in-degree sequence and the out-degree sequence divided by 2.

[0024] The calculation of the reciprocity edge self-loop probability in the above step (6) is: the square of the reciprocity degree value of each node i divided by the cumulative sum of the expected reciprocity edge number, and then divided by 1 / 2 of the product of the number of nodes with the reciprocity degree greater than 0.

[0025] The calculation of the directed edge self-loop probability in the above step (6) is: the product of the in-degree value and the out-degree value of each node i divided by the cumulative sum of the expected directed edge number, and then divided by 1 / 2 of the product of the number of nodes with the in-degree and the out-degree greater than 0.

[0026] The calculation of the reciprocity edge sampling probability in the above step (7) is: according to the product of the current reciprocity degree values of node i and node j divided by the difference between the expected reciprocity edge number and the length of the reciprocity edge edge sequence, and then adding the reciprocity edge self-loop probability. After each edge is established, the reciprocity degree values of node i and node j are both reduced by 1, and the length of the reciprocity edge edge sequence is increased by 2.

[0027] The degree complement edge strategy in the above step (8) is: after the end of one round of sampling of the i-th node, if the current degree value of node i is not 0, find other nodes with non-zero degree values adjacent to node i, and connect node i to the found nodes until the current degree value of node i is 0.

[0028] The calculation of the directed edge sampling probability in the above step (9) is: according to the product of the current in-degree value of node i and the current out-degree value of node j divided by the difference between the expected directed edge number and the length of the directed edge edge sequence, and then adding the directed edge self-loop probability. If a directed edge from node j to node i is established, the in-degree of node i is reduced by 1, the out-degree of node j is reduced by 1, and the length of the directed edge edge sequence is increased by 1.

[0029] The application also relates to a large-scale directed network topology fast generation system, which comprises the following modules:

[0030] (1) User input module: the starting end of the network topology generation system, used for obtaining user input;

[0031] (2) Node degree sampling module: used for obtaining reciprocity degree sequence, in-degree sequence and out-degree sequence, first sampling N times from multivariate normal distribution according to the input of the user, then applying cumulative distribution function and inverse cumulative distribution function to convert the multivariate data into power-law distribution data and rounding;

[0032] (3) Reciprocal edge sampling module: used for obtaining reciprocal edge edge sequence, first calculating the expected number of reciprocal edges, then calculating the reciprocal edge loop probability, secondly calculating the reciprocal edge sampling probability, and using the residual degree edge supplement strategy to nest and use for reciprocal edge sampling;

[0033] (4) Directed edge sampling module: used for obtaining directed edge edge sequence, first calculating the expected number of directed edges, then calculating the directed edge loop probability, secondly calculating the directed edge sampling probability, and using the residual degree edge supplement strategy to nest and use for directed edge sampling;

[0034] (5) Network topology generation module: the terminal of the network topology generation system, used for generating network topology, first adding the number of nodes, secondly adding the reciprocal edge edge sequence and the directed edge edge sequence.

[0035] Advantages

[0036] According to the technical scheme provided by the application, the following technical advantages are disclosed:

[0037] (1) High network topology generation quality, the edge sampling probability of the application no longer depends on the initial fixed degree value but the current degree value of the node pair, in addition, the residual degree of each node is redistributed, avoiding overfitting and underfitting of the node degree, so that the network topology generation quality of the application reaches a high level.

[0038] (2) Fast network topology generation time, the time complexity of the application is O(n 2 ), by performing residual degree edge supplement in the edge sampling process, the network topology generation time is greatly reduced.

[0039] (3) Low complexity, the large-scale directed network topology fast generation method and system proposed by the application have low complexity and are easy to understand. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a large-scale directed network topology fast generation method flowchart;

[0041] Figure 2 is a reciprocal edge sampling example diagram;

[0042] Figure 3 is a directed edge sampling example diagram;

[0043] Figure 4 is a large-scale directed network topology fast generation system architecture diagram. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0045] Embodiment one:

[0046] This embodiment is used to provide a large-scale directed network topology fast generation method, as shown in Figure 2 The reciprocity edge sampling in the method includes the following steps:

[0047] (1) According to the user input parameters, the number of nodes N = 4, the reciprocity degree sequence and the expected number of reciprocity edges is 8;

[0048] (2) The reciprocity edge sampling probability pr r of node a and node b is: the product of the current reciprocity degree values of node a and node b divided by the expected number of reciprocity edges plus the reciprocity edge self-loop probability, since pr r <1, node a and node b do not establish an edge, and the remaining edge number is 8;

[0049] (3) The reciprocity edge sampling probability pr r of node a and node c is: the product of the current reciprocity degree values of node a and node c divided by the remaining edge number plus the reciprocity edge self-loop probability, since pr r >1, node a and node c establish an edge, and the reciprocity degree values of node a and node c are both reduced by 1, and the remaining edge number is 6;

[0050] (4) The reciprocity edge sampling probability pr r of node a and node d is: the product of the current reciprocity degree values of node a and node d divided by the remaining edge number plus the reciprocity edge self-loop probability, since pr r >1, node a and node d establish an edge, and the reciprocity degree values of node a and node d are both reduced by 1, and the remaining edge number is 4;

[0051] (5) After one round of sampling of node a, the remaining degree is 1, the current reciprocity degree value of adjacent node b is greater than 0, node a and node b establish an edge, and the reciprocity degree values of node a and node b are both reduced by 1, and the remaining edge number is 2;

[0052] (6) Since the current reciprocity degree value of node b is 0, the sampling of node b is skipped;

[0053] (7) The reciprocity edge sampling probability pr r of node c and node d is: the product of the current reciprocity degree values of node c and node d divided by the remaining edge number plus the reciprocity edge self-loop probability, since pr rIf the number of edges is less than 1, no edge is established between node c and node d, and the number of remaining edges is 2.

[0054] (8) After one round of sampling, node c has a remainder of 1. The current reciprocity value of the adjacent node d is greater than 0. Node c and node d establish an edge. The reciprocity values ​​of node c and node d are both reduced by 1, and the number of remaining edges is 0.

[0055] like Figure 3 As shown, the directed edge sampling method includes the following steps:

[0056] (1) According to the user input parameters, the number of nodes N = 4 and the in-degree sequence is... out-degree sequence The expected number of directed edges is 3;

[0057] (2) Calculate the sampling probability pr of the directed edge from node a to b. d The product of node a's current out-degree and node b's current in-degree, divided by the expected number of directed edges, plus the probability of self-loops in directed edges, is given by pr. d If the value is less than 1, no edge is established between nodes a and b, and the number of remaining edges is 3.

[0058] (3) Calculate the sampling probability pr of the directed edge from node a to c. d The product of the current out-degree of node a and the current in-degree of node c, divided by the expected number of directed edges, plus the probability of self-loops in directed edges, is given by pr. d >1. Establish an edge from node a to c. Decrease the out-degree of node a by 1 and the in-degree of node c by 1. The number of remaining edges is 2.

[0059] (4) Since the current in-degree of node d is 0, skip the calculation of the sampling probability of the directed edge from node a to d;

[0060] (5) After one round of sampling, the out-degree remainder of node a is 1, the current in-degree value of the adjacent node b is greater than 0, a connection is established between node a and b, the out-degree value of node a is reduced by 1, the in-degree value of node b is reduced by 1, and the number of remaining edges is 1.

[0061] (6) Since the current out-degree of node b is 0, skip the calculation of the sampling probability of the directed edges of nodes b→a, b→c, and b→d;

[0062] (7) Since the current out-degree of node c is 0, skip the calculation of the sampling probability of the directed edges of nodes b→a, b→c, and b→d;

[0063] (8) Since the current in-degree of nodes a and b is 0, skip the calculation of the sampling probability of the directed edge from node d to a and d to b;

[0064] (9) Calculate the sampling probability pr of the directed edge from node d to c. dis: the product of the current out-degree value of node d and the current in-degree value of node c divided by the expected number of directed edges plus the probability of a directed edge loop, since pr d = 1, node d→c establishes an edge, the out-degree value of node d is reduced by 1, the in-degree value of node c is reduced by 1, and the remaining number of edges is 0.

[0065] It is worth noting that, in reciprocal edge sampling, each node only calculates the edge probability with the nodes after its own serial number; while in directed edge sampling, each node calculates the edge probability with all the nodes except itself. When there are both reciprocal edges and directed edges in the network topology, the reciprocal edges are calculated first, then the directed edges, and finally the network topology is generated.

[0066] Embodiment Two:

[0067] This embodiment is used to provide a large-scale directed network topology fast generation system, and a system architecture diagram thereof is shown in Figure 4 The system comprises the following modules:

[0068] (1) User input module: the starting end of the network topology generation system, used to obtain user input, including: total number of nodes, rank correlation coefficient between reciprocal degree and in-degree, rank correlation coefficient between reciprocal degree and out-degree, rank correlation coefficient between in-degree and out-degree, degree of freedom and location parameter of reciprocal degree distribution, degree of freedom and location parameter of in-degree distribution, degree of freedom and location parameter of out-degree distribution;

[0069] (2) Node degree sampling module: used to obtain reciprocal degree sequence, in-degree sequence and out-degree sequence, first, the rank correlation coefficient between reciprocal degree, in-degree and out-degree is multiplied by π / 6 to take the sine value and then multiplied by 2 to obtain the linear correlation coefficient between reciprocal degree, in-degree and out-degree; then, N samples are generated in the covariance matrix of two-by-two combination with mean value of 0, diagonal elements of 1 and other elements of linear correlation coefficient according to the probability density function of multivariate normal distribution, wherein each sample is a 3-dimensional vector, and the sample is transposed to obtain 3 multivariate data; then, the cumulative distribution function is applied to standardize the 3 multivariate data into uniform distribution data; then, the inverse cumulative distribution function is applied to convert the uniform distribution data into power-law distribution data and take the integer to obtain the reciprocal degree sequence, in-degree sequence and out-degree sequence;

[0070] (3) Reciprocal edge sampling module: used to obtain reciprocal edge edge sequence, first, the reciprocal degree sequence is summed to obtain the expected number of reciprocal edges; then, the reciprocal edge loop probability is calculated, which is the square of the reciprocal degree value of each node i divided by the cumulative sum of the expected number of reciprocal edges, then divided by the product of 1 / 2 of the number of nodes with reciprocal degree greater than 0; secondly, the reciprocal edge sampling probability is calculated, for the specific steps, refer to Figure 3 The reciprocal edge is sampled by using the reciprocal edge sampling probability and the residual edge complement strategy nesting strategy;

[0071] (4)Directed edge sampling module: used to obtain directed edge sequence, first, the sum of the in-degree sequence and the out-degree sequence is accumulated and then divided by 2 to obtain the expected number of directed edges, then the product of the in-degree value and the out-degree value of each node i is calculated, divided by the accumulated sum of the expected number of directed edges, and then divided by the product of the number of nodes with in-degree and out-degree greater than 0, and the result is divided by 1 / 2 to obtain the directed edge self-loop probability, second, the directed edge sampling probability is calculated, and the specific steps are referred to Figure 4 and the directed edge sampling is nested with the degree complement edge strategy;

[0072] (5)Network topology generation module: the terminal of the network topology generation system, used to generate network topology, first, the number of nodes is added, second, the reciprocal edge sequence and the directed edge sequence are added, and finally the generated network topology is displayed on the system interface.

[0073] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the claims of the present application.

Claims

1. A method for rapidly generating large-scale directed network topologies, characterized in that, It includes the following steps: (1) Obtain user input parameters including: the total number of nodes, the rank correlation coefficient between reciprocity and in-degree, the rank correlation coefficient between reciprocity and out-degree, the rank correlation coefficient between in-degree and out-degree, the degrees of freedom and location parameters of the reciprocity distribution, the degrees of freedom and location parameters of the in-degree distribution, and the degrees of freedom and location parameters of the out-degree distribution; (2) Calculate the linear correlation coefficients between reciprocity, in-degree, and out-degree pairwise according to the parameters input by the user in step (1): Multiply the rank correlation coefficients between reciprocity, in-degree, and out-degree pairwise by π / 6, take the sine value, and then multiply by 2; (3) Generate N samples according to the parameters input by the user in step (1) and the linear correlation coefficients calculated in step (2) according to the probability density function of the multivariate normal distribution. Each sample is a 3D vector. Transpose the samples to obtain the sampled multivariate data. Apply the cumulative distribution function (CDF) to standardize the sampled multivariate data into uniformly distributed data, and then apply the inverse cumulative distribution function (PPF) to convert the uniformly distributed data into power-law distributed data and round to obtain the reciprocity sequence, in-degree sequence, and out-degree sequence; (4) Create an empty topology and add the number of nodes; (5) According to the reciprocity sequence, in-degree sequence, and out-degree sequence calculated in step (3), calculate the expected number of reciprocal edges by accumulating and summing the reciprocity sequence, and calculate the expected number of directed edges by accumulating and summing the in-degree sequence and out-degree sequence and then dividing by 2; (6) Calculate the self-loop probability of reciprocal edges and the self-loop probability of directed edges according to the reciprocity sequence, in-degree sequence, and out-degree sequence calculated in step (3) and the expected number of reciprocal edges and expected number of directed edges calculated in step (5). The self-loop probability of reciprocal edges is the square of the reciprocity value of each node i divided by the accumulated sum of the expected number of reciprocal edges and then divided by 1 / 2 of the product of the number of nodes with reciprocity greater than 0. The self-loop probability of directed edges is the product of the in-degree value and out-degree value of each node i divided by the accumulated sum of the expected number of directed edges and then divided by 1 / 2 of the product of the number of nodes with in-degree and out-degree greater than 0; (7) Calculate the sampling probability of reciprocal edges according to the reciprocity sequence calculated in step (3), the expected number of reciprocal edges calculated in step (5), and the self-loop probability of reciprocal edges calculated in step (6) as the product of the current reciprocity values of nodes i and j divided by the difference between the expected number of reciprocal edges and the length of the reciprocal edge connection sequence plus the self-loop probability of reciprocal edges; After each connection is established, the reciprocity values of nodes i and j are both reduced by 1, and the length of the reciprocal edge connection sequence is increased by 2; (8) Traverse the node pairs in a double-layer loop. For each node i and node j (i < j), obtain the reciprocal edge connection sequence according to the sampling probability of reciprocal edges calculated in step (7) and the redundant edge complement strategy. The redundant edge complement strategy is that after a round of sampling of the i-th node, if the current degree value of node i is not 0, then find other nodes with non-zero degree values of adjacent ordinals to connect with node i until the current degree value of node i is 0; (9) Calculate the directed edge sampling probability based on the in-degree sequence and out-degree sequence calculated in step (3), the expected number of directed edges calculated in step (5), and the directed edge self-loop probability calculated in step (6). The directed edge sampling probability is calculated by dividing the product of the current in-degree value of node i and the current out-degree value of node j by the difference between the expected number of directed edges and the length of the directed edge connection sequence, plus the directed edge self-loop probability. If an edge is established from node j to node i, the in-degree of node i is reduced by 1, the out-degree of node j is reduced by 1, and the length of the directed edge connection sequence is increased by 1. (10) Double loop traverse the node pairs. For each node i and node j (i≠j), obtain the directed edge connection sequence according to the directed edge sampling probability and redundancy supplementation strategy calculated in step (9). (11) Add the reciprocal edge connection sequence obtained in step (8) and the directed edge connection sequence obtained in step (10) to the network topology created in step (4).

2. A system for rapidly generating large-scale directed network topologies to implement the method of claim 1, characterized in that, Includes the following modules: (1) User input module: The starting point of this network topology generation system, used to obtain user input; (2) Node degree sampling module: used to obtain the reciprocity degree sequence, in-degree sequence and out-degree sequence. First, it samples N times from the multivariate normal distribution according to the user's input. Then, it applies the cumulative distribution function and the inverse cumulative distribution function to convert the multivariate data into power law distribution data and rounds it. (3) Reciprocal edge sampling module: used to obtain the sequence of reciprocal edges. First, the expected number of reciprocal edges is calculated, then the probability of reciprocal edge self-loop is calculated, then the reciprocal edge sampling probability is calculated, and it is used in nested with the redundancy edge supplementation strategy to perform reciprocal edge sampling. (4) Directed edge sampling module: used to obtain the sequence of directed edges. First, the expected number of directed edges is calculated, then the probability of self-loop of directed edges is calculated, then the sampling probability of directed edges is calculated, and it is used in nested with the redundancy supplementation strategy to perform directed edge sampling. (5) Network topology generation module: The terminal of this network topology generation system is used to generate network topology. First, add the number of nodes, and then add the mutual edge connection sequence and the directed edge connection sequence.

Images