Flow monitoring and analysis methods, devices and media
By acquiring normal network topology and test data, and using a denoising diffusion probability model to generate an estimated network topology, the problem of insufficient link traffic information analysis is solved, enabling network fault location and traffic prediction, and improving the effectiveness of network management.
Patent Information
- Application Number
- CN202311703992.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-12
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-12-12
AI Technical Summary
Existing technologies lack effective means to conduct in-depth analysis of link traffic information, resulting in difficulties in locating network faults and untimely prediction of traffic congestion, making it difficult to achieve effective traffic supervision.
By acquiring the network topology map and test data at normal time points, an estimated network topology map is generated using a denoising diffusion probability model, which shows the node performance of each traffic flow at the estimated time point, thus enabling in-depth analysis and prediction of link traffic information.
Effective analysis of link traffic information, location of network fault nodes, and prediction of network congestion provide a more timely and effective means of traffic monitoring, thereby improving network quality.
Smart Images

Figure CN117560274B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates at least to the field of communication technology, and in particular to a traffic regulatory analysis method, a traffic regulatory analysis device, and a computer-readable storage medium. Background Technology
[0002] Some traffic policing methods primarily involve monitoring traffic on individual links and performing statistical analysis of link traffic over time. With the development of network technology, the demand for traffic policing and analysis is increasing. In-depth analysis of link traffic information is needed, and traffic should be managed in a tiered manner based on the analysis results to improve network quality. However, currently, effective means of analyzing link traffic information are lacking. Summary of the Invention
[0003] The technical problem to be solved by this disclosure is to address the above-mentioned shortcomings by providing a traffic regulatory analysis method, a traffic regulatory analysis device, and a computer-readable storage medium, so as to solve the problem of how to effectively analyze link traffic information and effectively realize traffic regulation.
[0004] Firstly, this disclosure provides a traffic monitoring and analysis method, the method comprising:
[0005] Obtain and estimate the time point x t The normal time point x0 of the relevant network;
[0006] Obtain the normal network topology at normal time point x0. The normal network topology includes the nodes that the traffic from user terminals accessing the application server passes through.
[0007] Obtain the first network test data of each traffic passing through each node in the normal network topology at normal time point x0. The first network test data includes node performance data collected from each node.
[0008] The estimated time point x is generated based on the normal network topology diagram and the first network test data. t The estimated network topology includes the traffic flow at the time point x to be estimated. t
[0009] The estimated node performance is represented by each node.
[0010] Secondly, this disclosure provides a traffic monitoring and analysis device, the device comprising:
[0011] The time acquisition module is used to obtain the time point x to be estimated. t The normal time point x0 of the relevant network;
[0012] The normal topology module, connected to the time acquisition module, is used to obtain the normal network topology map at normal time point x0. The normal network topology map includes the nodes that the traffic from the user terminal to the application server passes through.
[0013] The data acquisition module, connected to the normal topology module, is used to acquire the first network test data of each traffic passing through each node in the normal network topology at normal time point x0. The first network test data includes node performance data collected from each node.
[0014] The topology estimation module, connected to the data acquisition module, is used to generate the estimated time point x based on the normal network topology diagram and the first network test data. t The estimated network topology includes the traffic flow at the time point x to be estimated. t The estimated node performance is represented by each node.
[0015] Thirdly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the traffic monitoring and analysis method described above.
[0016] This disclosure provides a traffic policing analysis method, a traffic policing analysis device, and a computer-readable storage medium. It acquires a normal network topology map and first network test data based on normal network time points, generates an estimated network topology map for the time point to be estimated based on the normal network topology map and the first network test data, and displays the estimated node performance representation of each traffic flow passing through each node at the time point to be estimated in the estimated network topology map. This effectively analyzes link traffic information, and further, it obtains the estimated node performance results based on the estimated network topology map. These estimation results can at least be used to locate network faulty nodes and predict network congestion, thus providing a more effective means of implementing traffic policing. Attached Figure Description
[0017] Figure 1 This is a flowchart of a traffic monitoring and analysis method according to an embodiment of this disclosure;
[0018] Figure 2 This is a schematic diagram of the structure of a traffic monitoring and analysis system according to an embodiment of this disclosure;
[0019] Figure 3 This is a schematic diagram of a timeline according to an embodiment of the present disclosure;
[0020] Figure 4 This is a schematic diagram of a normal network topology according to an embodiment of this disclosure;
[0021] Figure 5 This is a schematic diagram of an estimated network topology according to an embodiment of this disclosure;
[0022] Figure 6 This is a schematic diagram of the structure of another traffic monitoring and analysis system according to an embodiment of this disclosure;
[0023] Figure 7 This is a schematic diagram of the structure of a traffic monitoring and analysis device according to an embodiment of this disclosure. Detailed Implementation
[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings.
[0025] It is understood that the specific embodiments and accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure.
[0026] It is understood that, without conflict, the various embodiments and features in the embodiments of this disclosure can be combined with each other.
[0027] It is understood that, for ease of description, only the parts relevant to this disclosure are shown in the accompanying drawings, while parts unrelated to this disclosure are not shown in the drawings.
[0028] It is understood that each unit or module involved in the embodiments of this disclosure may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.
[0029] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this disclosure may occur in a different order than that marked in the accompanying drawings.
[0030] It is understood that the flowcharts and block diagrams of this disclosure illustrate the architecture, functions, and operations of possible implementations of systems, apparatuses, devices, and methods according to various embodiments of this disclosure. Each block in a flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagrams and flowcharts may be implemented using a hardware-based system to implement the specified function, or using a combination of hardware and computer instructions.
[0031] It is understood that the units and modules involved in the embodiments of this disclosure can be implemented by software or by hardware, for example, the units and modules can be located in a processor.
[0032] To facilitate understanding of this disclosure, the inventive concept of this disclosure will first be explained in conjunction with two possible application approaches.
[0033] One way operators monitor traffic is by statistically analyzing the traffic of a single link on a single device over time; another way is by statistically analyzing the total traffic for a region, such as the total inbound and outbound traffic in Foshan City or Guangzhou City over a certain period of time.
[0034] One potential use of traffic monitoring is to allow network administrators to log into the relevant device's network management system to view and analyze historical traffic on a specific link and investigate the cause when a user reports a fault, such as packet loss or slow internet access at a certain time in the past. The current problem is that because user internet access often crosses multiple ISPs from the user's client to the server, historical faults cannot be analyzed using ping (Packet InternetGroper) or trace (a method for tracing information transmission paths), leaving no relevant evidence of the fault. Network administrators cannot possibly view the traffic at every node of the network link, nor can they check the traffic of multiple ISPs, making it impossible to pinpoint the cause of the fault, which often recurs after a period of time.
[0035] Another possible application is that operators, to avoid traffic congestion, periodically analyze link traffic. When traffic reaches a specified threshold (e.g., peak traffic reaches 80%), they will expand the link capacity. Link expansion typically involves first planning spare ports and fiber optic cables on the equipment, having the construction team install the fiber optic cables on-site, and finally configuring data on the equipment to allow traffic. The ultimate goal is to ensure that peak traffic never exceeds the limit, thus preventing network congestion. However, this method of avoiding congestion through link expansion is too slow; sudden peak traffic cannot be avoided. When interconnection between multiple operators fails, operators need to negotiate link expansion, which takes too long. Therefore, it is difficult to resolve network congestion problems in a timely manner.
[0036] In view of this, there is an urgent need to propose a method for in-depth analysis of traffic, and to classify and manage traffic based on the analysis results to improve network quality.
[0037] This disclosure proposes a new technology system for network traffic collection and analysis, including a traffic topology analysis platform based on the principle of a noise-reducing diffusion probability model. Utilizing the characteristics of the diffusion model, it traces back historical topology traffic maps based on historical traffic data to analyze the causes of historical failures; based on current traffic data, it predicts future topology traffic maps and, combining the principles of DNS (Domain Name System) and CDN (Content Delivery Network), performs traffic diversion in advance to avoid traffic congestion.
[0038] Example 1:
[0039] like Figure 1 As shown, this disclosure provides a traffic monitoring and analysis method, the method comprising:
[0040] S11. Obtaining and estimating the time point x t The normal time point x0 of the relevant network;
[0041] S12. Obtain the normal network topology at normal time point x0. The normal network topology includes the nodes that the traffic from the user terminal to the application server passes through.
[0042] S13. Obtain the first network test data of each traffic passing through each node in the normal network topology at normal time point x0. The first network test data includes node performance data collected from each node.
[0043] S14. Generate the estimated time point x based on the normal network topology diagram and the first network test data. t The estimated network topology includes the traffic flow at the time point x to be estimated. t The estimated node performance is represented by each node.
[0044] In this embodiment, it can be combined with Figure 2-6 To understand the proposed traffic regulation analysis method, for example, for Figure 2 As shown, the traffic of a computer user accessing the application server will pass through nodes 1, 2, and 3 sequentially. If traffic monitoring and analysis tasks obtain data such as... Figure 3 The estimated time point x is shown. t And the normal time point x0, according to Figure 2 The network topology diagram shown below illustrates the traffic from PC users accessing the application server. Figure 4 As shown, after obtaining the first network test data of all traffic passing through nodes 1, 2, and 3 at the normal time point x0, the estimated time point x is generated based on the normal network topology and the first network test data. t The estimated network topology is as follows: Figure 5 As shown, the estimated network topology diagram displays the traffic flow at the time point x to be estimated. t The estimated node performance representations based on nodes 1, 2, and 3 are as follows: Figure 5 The black dots represent nodes whose performance is estimated to be poor when traffic passes through them. The above process effectively analyzes link traffic information, allowing for the estimation of node performance based on the estimated network topology. This estimation can at least be used to locate faulty network nodes; or, if for example… Figure 6 The system shown can perform similar traffic policing analysis to predict, for example... Figure 6Will the network shown at the estimated time point x t Congestion occurs, and traffic is diverted according to the time of congestion, thus providing a more effective means of traffic regulation.
[0045] In one implementation, the time point x to be estimated is obtained. t Before the normal time point x0 of the relevant network, the method further includes:
[0046] The system receives network test data sent by the application server. The network test data is obtained by the user terminal sending network test packets to the application server according to the set scheduling time. The network test packets collect node performance data at each node they pass through in sequence. The network test data includes test time, network connectivity status, IP (Internet Protocol) address of each node, and latency and packet loss rate of that node.
[0047] In this embodiment, the method is applied to a traffic monitoring platform, such as... Figure 2 As shown in Figure 6, sufficient network test data should be obtained before conducting traffic policing analysis. The first network test data is obtained from the collected network test data.
[0048] The traffic data collection principle of the traffic monitoring platform is as follows: When a user accesses an application server via the Internet, the data is collected through devices such as users, enterprises, and operators. The traffic monitoring platform uses the user's client as the data collection point. When the client accesses the application server, a network test is performed in the client's background at a certain time. The client sends the collected data to the application server (i.e., sending network test packets from the client to the application server, and collecting performance data from each node). The application server then sends the data to the traffic monitoring platform. The client can be a computer, mobile phone, tablet, or other device (i.e., a user terminal). When the client accesses the application server, the client's background secretly uses network tools to test the network parameters from the access point to the application server, including test time, connectivity, average latency, packet loss rate, and access nodes. The generated network test result format is (time = t, [connected | not connected], node = node IP address [latency, packet loss rate], node IP address [latency, packet loss rate]... node n's IP address [latency, packet loss rate]). The relevant parameters are explained in Table 1 below.
[0049] Table 1 Network Test Package Parameters
[0050]
[0051]
[0052] The following example illustrates network test data, such as... Figure 2 As shown, various user terminals such as computers, mobile phones, and tablets access the application server from different locations. Taking a scheduling time of 10 minutes as an example, every 10 minutes, the terminal user will perform a network test in the background and send the corresponding test results to the application server. The application server will then forward the test results to the traffic monitoring platform.
[0053] For example: Node 3's IP address is 11.11.11.11; Node 2's IP address is 22.22.22.22; Node 1's IP address is 33.33.33.33;
[0054] The network test data obtained by PC users is as follows:
[0055] (Time = 2023-10-25 18:10, Connected, Nodes = 11.11.11.11[39,0], 22.22.22.22[80,0], 33.33.33.33[21,0]);
[0056] (Time = 2023-10-25 18:20, Connected, Nodes = 11.11.11.11[37,1%], 22.22.22.22[67,0], 33.33.33.33[20,0]);
[0057] The network test data obtained by mobile users is as follows:
[0058] (Time = 2023-10-25 18:10, Connected, Node = 11.11.11.11[20,0], 22.22.22.22[60,8%]);
[0059] (Time = 2023-10-25 18:20, Connected, Node = 11.11.11.11[19,0, 22.22.22.22[67,0]);
[0060] The network test data obtained by tablet users is as follows:
[0061] (Time = 2023-10-25 18:10, not working, node =);
[0062] (Time = 2023-10-25 18:20, Connected, Node = 11.11.11.11[13,0]).
[0063] In one implementation, the time point x to be estimated is obtained. t The relevant network normal time point x0 specifically includes:
[0064] In response to obtaining the network failure time point based on user complaints, let the network failure time point be x.t Based on user complaints, x is filtered out t The point in time when the user previously had normal network access is x0; or,
[0065] In response to the need to predict whether network congestion will occur at a certain future point in time, let the certain future point in time be x. t The current time point when the network is not congested is obtained as x0.
[0066] In this embodiment, steps S11, S12, and S13 differ slightly for the two different applications described above. When used for network fault tracing, the principle is that when a user complains about a network fault occurring during a historical period, such as... Figure 2 For user complaints on the computer terminal, fault point analysis can be performed using historical network test data combined with the forward noise addition principle of the DDPM (Denoising Diffusion Probabilistic Models) model. The time point is determined by selecting multiple different time points as x, based on the fault time period reported by the user. t Filter out a normal time point x0 related to this period, at which time point x0 the computer user can access the application server normally.
[0067] In one embodiment, a normal network topology map is obtained at a normal time point x0. The normal network topology map includes the nodes traversed by traffic from user terminals accessing the application server, specifically including:
[0068] Obtain the first network test data of the complaining user at time x0. Based on the first network test data of the complaining user, obtain the nodes and node order through which the complaining user's traffic passes from the user terminal to the application server. Draw a normal network topology diagram based on the nodes and node order. Or,
[0069] Obtain the nodes and node order that a user accesses on a certain link of an application server at time x0, and draw a normal network topology based on the nodes and node order.
[0070] In this embodiment, a normal network topology map is generated based on the network test data at the normal network time point x0, and then... Figure 2 Taking a PC user accessing the application server as an example, at a normal time point x0, the PC user performs a network test in the background, generating network test data as follows (time = x0, connectivity, nodes = node1[41,0], node2[31,0], node3[21,0]). This network test data is transmitted to the application server, which forwards it to the traffic monitoring platform. Based on this test data, the traffic monitoring platform generates a normal network topology diagram as follows. Figure 4 As shown.
[0071] In one embodiment, acquiring first network test data for each traffic flow passing through each node in the normal network topology at a normal time point x0 specifically includes:
[0072] Obtain the first network test data for all user traffic at x0 that passes through the same nodes and in the same node order as the complaining user; or...
[0073] Obtain the first network test data of all user access traffic passing through a certain link at x0.
[0074] In this embodiment, the example of a user complaint is approximately during the time period x0 to x... t The network has been abnormal recently, x t Because data loss occurred due to network anomalies at a specific time point, the cause of the failure cannot be directly analyzed. Therefore, the traffic monitoring platform will filter out network test data (the first network test data) at time point x0 for failure analysis. For example, the network test data for three users at time point x0 is filtered as follows:
[0075] User 1: (Time = x0, Connectivity, Node = Node 1 [39,0], Node 2 [80,0], Node 3 [21,0]);
[0076] User 2: (Time = x0, Connectivity, Node = Node 1 [41,0], Node 2 [31,0], Node 3 [21,8%]);
[0077] User 3: (Time = x0, not connected, Node = Node 0 [0,0], Node 0 [0,0], Node 3 [0,0]).
[0078] In one implementation, the estimated time point x is generated based on the normal network topology diagram and the first network test data. t The estimated network topology includes the traffic flow at the time point x to be estimated. t The estimated node performance is represented by the following:
[0079] Transform a normal network topology graph into a normal network topology graph array y0;
[0080] Pixel values are generated based on the node performance data of each node in the test data of each first network. Each pixel value represents the node performance of a certain traffic passing through a certain node. A Gaussian noise array ∈ representing the node performance of each traffic passing through each node is obtained.
[0081] Based on the normal network topology graph array y0 and the Gaussian noise array ∈, the denoising diffusion probability model DDPM is used to obtain the time point to be estimated x. t The estimated network topology graph array y t ;
[0082] The estimated network topology graph array y t Transform into an estimated network topology graph, where each pixel in the estimated network topology graph represents the flow at the time point x to be estimated. t The estimated node performance is represented by each node.
[0083] In this embodiment, based on the aforementioned example, the normal network topology needs to be converted into an array for computation. The network test data is converted into a Gaussian noise array ∈ for computation. Both the normal network topology array y0 and the Gaussian noise array ∈ are image-related data, therefore pixel representation is used. The processed normal network topology array y0 and Gaussian noise array ∈ are substituted into the forward noise addition formula of the DDPM diffusion model. The network fault topology is derived using the normal network topology. The network fault topology is as follows: Figure 5 As shown, the corresponding pixels (black dots in the image) will be displayed in the corresponding node, and these pixels represent the estimated node performance.
[0084] In one embodiment, converting a normal network topology graph into a normal network topology graph array y0 specifically includes:
[0085] Obtain the first RGB value representation of the normal network topology graph, and standardize and normalize each first RGB value to obtain the normal network topology graph array y0.
[0086] In this embodiment, the normal network topology graph is saved in RGB24 (a color standard) format. Each pixel in the topology graph is represented by three channels: R (red), G (green), and B (blue). The value of each channel is between [0, 255]. The value of each channel is normalized and mapped to the range of [-1.0, +1.0] using the formula: (pixel value / 255)*2-1. The topology graph can be processed using tools such as OpenCV (Open Source Computer Vision Library, an open-source cross-platform computer vision and machine learning software library). Finally, a normalized normal network topology graph array y0 is generated.
[0087] In one embodiment, pixel values are generated based on the node performance data of each node in each first network test data. Each pixel value represents the node performance of a certain traffic passing through a certain node. A Gaussian noise array ∈ representing the node performance of each traffic passing through each node is obtained, specifically including:
[0088] Based on the node performance data of each node in the test data of each first network, pixel values representing white to black are generated. White pixel values indicate that the node performance of a certain traffic is optimal when passing through a certain node, and black pixel values indicate that the node performance of a certain traffic is worst when passing through a certain node.
[0089] After obtaining the pixel values representing the node performance of each traffic flow, the pixel values are arranged according to the topological position of the corresponding node in the normal network topology graph to obtain the Gaussian noise array ∈.
[0090] In this embodiment, the network test data at time point x0 is normalized and standardized to generate a Gaussian noise array ∈. Since the RGB values of white are all 255, the normalized array is [1,1,1], and the RGB values of black are all 0, the normalized array is [-1,-1,-1]. One of the purposes of this disclosure is to generate black dots at the locations of nodes with network problems on the estimated topology map. Therefore, the Gaussian noise array ∈ at noise points with network problems should tend to [-1,-1,-1]. After obtaining the pixel values, the approximate position of the pixel of each node in the topology map is calculated, and the noise array is placed in the corresponding position to generate the final Gaussian noise array ∈.
[0091] In one embodiment, pixel values representing white to black are generated based on the node performance data of each node in each first network test data, specifically including:
[0092] In response to a first network test result indicating network connectivity issues, a standardized and normalized black pixel value is generated for each node; otherwise...
[0093] In response to a certain network test data reflecting that the latency and packet loss rate of the corresponding traffic passing through a certain node are within their respective ranges, the standardized and normalized color pixel values corresponding to the range are obtained, and the smaller of the two obtained color pixel values is taken as a pixel value of a certain node.
[0094] In this embodiment, pixels can be represented by color or entirely by black. The size of the black dots can be used to distinguish numerical values, or a threshold can be set, with values greater than the threshold taking a value of 1 and values less than the threshold taking a value of -1. The method for assigning pixel values to a given data point includes:
[0095] If the network test fails, you need to [do something - the name is missing in the original text]. t At this point in time, black dots are generated for all nodes;
[0096] If the network latency is between 31ms and 50ms, users will have a good experience accessing the network; if the network latency is between 51ms and 100ms, users will experience pauses; if the network latency is greater than 100ms, users will feel that they cannot access the network. Therefore, the processing results for obtaining normalized pixel values based on latency are shown in Table 2 below:
[0097] Table 2 shows the pixel values for time delay.
[0098] Network latency pixel value Less than 50ms [1,1,1] 51ms to 100ms -(((delay-50) / 50)*2-1) Greater than 100ms [-1,-1,-1]
[0099] If the normal packet loss rate is between 0.1% and 2%, the user's network access is good; if the packet loss rate is between 2% and 10%, the user's network access is slow; if the packet loss rate is greater than 10%, the user feels that the network is inaccessible. Therefore, the processing results of obtaining normalized pixel values for packet loss rate are shown in Table 3 below:
[0100] Table 3 shows the pixel values for packet loss rate.
[0101] Packet loss rate pixel value Less than 2% [1,1,1] 2.1% to 10% -(((packet loss rate-2) / 8)*2-1) Greater than 10% [-1,-1,-1]
[0102] When both latency and packet loss rate show abnormal data at the same node, the minimum value between the two arrays of latency and packet loss rate is taken as the pixel value.
[0103] Based on the previous example, User 1 corresponds to the values in Table 2 above based on latency, with pixel values for node 1 [1,1,1], node 2 [-0.2,-0.2,-0.2], and node 3 [1,1,1]. User 2 corresponds to the values in Table 3 above based on packet loss rate, with pixel values for node 1 [1,1,1], node 2 [-0.5,-0.5,-0.5], and node 3 [1,1,1]. User 3's network test failed, so it needs to be checked in x. t At this point in time, black dots are generated on nodes 1, 2, and 3. Therefore, the pixel values at this time are: [-1, -1, -1] for node 1, [-1, -1, -1] for node 2, and [-1, -1, -1] for node 3. Through corresponding topological pixel calculations, it is determined that node 1 is located approximately in the lower 1 / 3 of the topological graph, node 2 is in the middle of the topological graph, and node 3 is located approximately in the upper 1 / 3 of the topological graph. Placing the corresponding pixel values into the corresponding node positions, the final Gaussian noise array ∈ is: [.........]
[0105] [1,1,1]
[0106] [1,1,1]
[0107] [-1,-1,-1] ...........
[0109] [-0.2,-0.2,-0.2]
[0110] [-0.5,-0.5,-0.5]
[0111] [-1,-1,-1] .............
[0113] [1,1,1]
[0114] [1,1,1]
[0115] [-1,-1,-1] ................]
[0117] In one implementation, the time point to be estimated, x, is obtained using the Denoising Diffusion Probability Model (DDPM) based on the normal network topology graph array y0 and the Gaussian noise array ∈. t The estimated network topology graph array y t Specifically, it includes:
[0118] The time point to be estimated, x, is obtained from the following formula. t The estimated network topology graph array y t :
[0119]
[0120] in, It is in x t Previously (including x) t The cumulative product of all linear combination parameters in the network topology graph.
[0121] In this embodiment, the normalized data is substituted into the forward noise-adding formula of the DDPM diffusion model. The topology of the network fault is derived using the normal network topology, thereby locating the faulty nodes and promoting network remediation at the fault points. The forward formula of the DDPM diffusion model is shown below:
[0122]
[0123] Among them, y t Let y0 be the final network fault topology array derived from a certain fault time point, y0 be the normal network topology array at a past time point, and ∈ be the Gaussian noise array standardized from the network test data at time point x0. It is the cumulative product of all linearly combined parameters (source image weights) up to and including the time step. Multiple different time points x can be taken. t Generate different x t The topological graphs are analyzed and compared.
[0124] In one implementation, the estimated time point x is generated based on the normal network topology diagram and the first network test data. t After estimating the network topology, the method further includes:
[0125] In response to x t To pinpoint the network failure time points obtained from user complaints, the node with the most black pixels in the estimated network topology map is identified as the network failure point.
[0126] In response to x t To predict whether network congestion will occur at a future point in time, we determine whether the number of black pixels in the estimated network topology map exceeds a set threshold. If so, we determine that the link corresponding to the estimated network topology map will be congested at point x. t Network congestion may occur.
[0127] In this embodiment, for example, if the fault analysis yields such as Figure 5 The estimated topology diagram shown indicates that node 2 is most likely to experience a network failure leading to user complaints. A thorough analysis of node 2 is needed to identify the cause of the network failure, generate a failure report, and prevent future network failures. Multiple topology diagrams can be generated at different time points for comparison and multi-faceted analysis to improve accuracy.
[0128] In one implementation, it is determined that the link corresponding to the estimated network topology will be in x t After network congestion occurs, the method further includes:
[0129] Obtain the first application server corresponding to the estimated network topology map, obtain the second application server providing the same application service as the first application server, obtain a subset of users corresponding to the estimated network topology map, and allocate the traffic of the subset of users to x. t The time-sharing traffic is routed to the link connecting to the second application server.
[0130] In this embodiment, for example, regarding Figure 6 The system shown performs traffic prediction, generates a future topology map in advance, and adjusts traffic planning accordingly. This avoids network congestion caused by sudden traffic spikes, preventing user complaints. Since internet companies now deploy multiple servers in clusters for the same application to ensure uninterrupted 24 / 7 website access, and deploy dedicated lines from different ISPs to improve user experience, allowing users to access the application through different pathways within the same ISP, traffic prediction results can also be combined with DNS resolution and CDN servers for proactive traffic distribution to avoid congestion.
[0131] The steps for pre-emptive traffic splitting are as follows:
[0132] 1. Following the aforementioned method for obtaining the estimated network topology, using the current time x0 as the origin and combining it with the Gaussian noise array ∈ of the current time x0, the forward noise addition formula of the DDPM diffusion model is used to generate the future time point x. t Topological graph at time step, analyzing x t Network congestion points at any given time;
[0133] 2. Application server 1 and application server 2 are application servers of the same enterprise, and the same application is deployed. Application server 1 and application server 2 use dedicated lines from different operators. The traffic monitoring platform will send the analyzed future network congestion points to the DNS server and CDN server. When the DNS server and CDN server learn that the traffic accesses application server 1 through the path (node 1'-node 2'-node 3'), traffic congestion will occur.
[0134] 3. When a computer user requests the DNS resolution of application server 1 from the DNS server and CDN server, the DNS server and CDN server will interpret it as the IP address of application server 2 for the user. The traffic will then access application server 2 through the path (node 4-node 5-node 6), thereby avoiding traffic congestion.
[0135] 4. At fixed intervals (such as every 2 hours, 4 hours, etc.), the traffic monitoring platform continuously communicates with the DNS server and CDN server based on the analysis results of future traffic topology, changes the DNS resolution results, and controls the direction of user traffic.
[0136] Example 2:
[0137] like Figure 7 As shown, this disclosure provides a traffic monitoring and analysis device, the device comprising:
[0138] Time acquisition module 11 is used to acquire the time point x to be estimated. t The normal time point x0 of the relevant network;
[0139] Normal topology module 12, connected to time acquisition module 11, is used to acquire the normal network topology map at normal time point x0. The normal network topology map includes the nodes that the traffic of the user terminal accessing the application server passes through.
[0140] The data acquisition module 13 is connected to the normal topology module 12 and is used to acquire the first network test data of each traffic passing through each node in the normal network topology at the normal time point x0. The first network test data includes node performance data collected from each node.
[0141] Topology estimation module 14, connected to data acquisition module 13, is used to generate the estimated time point x based on the normal network topology diagram and the first network test data. t The estimated network topology includes the traffic flow at the time point x to be estimated. t The estimated node performance is represented by each node.
[0142] In one embodiment, the apparatus further includes:
[0143] The data receiving module, connected to the data acquisition module, is used to receive network test data sent by the application server. The network test data is obtained by the user terminal sending network test packets to the application server according to the set scheduling time. The network test packets collect node performance data at each node they pass through in sequence. The network test data includes test time, network connectivity status, IP address of each node, and latency and packet loss rate of that node.
[0144] In one embodiment, the time acquisition module 11 specifically includes:
[0145] The first-time acquisition unit is used to respond to obtaining the network failure time point based on user complaints, with the network failure time point as x. t Based on user complaints, x is filtered out t The point in time when the user's network access was previously normal is x0;
[0146] The second time acquisition unit is used to respond to the need to predict whether network congestion will occur at a certain future time point, with said future time point as x. t The current time point when the network is not congested is obtained as x0.
[0147] In one embodiment, the normal topology module 12 specifically includes:
[0148] The first normal topology unit is used to obtain the first network test data of the complaining user at x0, obtain the nodes and node order through which the complaining user's traffic accesses the application server from the user terminal based on the first network test data, and draw the normal network topology diagram based on the nodes and node order through which the traffic passes.
[0149] The second normal topology unit is used to obtain the nodes and node order that a certain traffic accessed by a user on a certain link of a certain application server at time x0 passes through, and to draw a normal network topology diagram based on the nodes and node order.
[0150] In one embodiment, the data acquisition module 13 specifically includes:
[0151] The first data acquisition unit is used to acquire the first network test data of all user traffic that passes through the same nodes and node order as the traffic of the complaining user at x0.
[0152] The second data acquisition unit is used to acquire the first network test data of all user access traffic passing through the certain link at x0.
[0153] In one embodiment, the topology estimation module 14 specifically includes:
[0154] The image-to-array unit is used to convert a normal network topology graph into a normal network topology graph array y0;
[0155] A Gaussian noise array unit is used to generate pixel values based on the node performance data of each node in the test data of each first network. Each pixel value represents the node performance of a certain traffic passing through a certain node, and a Gaussian noise array ∈ representing the node performance of each traffic passing through each node is obtained.
[0156] The array operation unit, connected to the image-to-array unit and the Gaussian noise array unit, is used to obtain the estimated time point x using the Denoising Diffusion Probability Model (DDPM) based on the normal network topology graph array y0 and the Gaussian noise array ∈. t The estimated network topology graph array y t ;
[0157] The array-to-image unit, connected to the array operation unit, is used to convert the estimated network topology graph array y into an image. t Transform into an estimated network topology graph, where each pixel in the estimated network topology graph represents the flow at the time point x to be estimated. t The estimated node performance is represented by each node.
[0158] In one embodiment, the image-to-array unit specifically includes:
[0159] The RGB conversion unit is used to obtain the first RGB numerical representation of a normal network topology graph.
[0160] The standardization unit, connected to the RGB conversion unit, is used to standardize and normalize each first RGB value to obtain the normal network topology array y0.
[0161] In one embodiment, the Gaussian noise array unit specifically includes:
[0162] The pixel value generation unit is used to generate pixel values representing white to black based on the node performance data of each node in the test data of each first network. The white pixel value indicates that the node performance of a certain traffic is optimal when passing through a certain node, and the black pixel value indicates that the node performance of a certain traffic is worst when passing through a certain node.
[0163] The topology location acquisition unit is used to obtain the pixel values representing the node performance of each traffic flow through each node, and then arrange the pixel values according to the topology location of the node corresponding to each pixel value in the normal network topology diagram to obtain the Gaussian noise array ∈.
[0164] In one embodiment, the pixel value generation unit specifically includes:
[0165] The first pixel value generation unit is used to generate a standardized and normalized black pixel value for each node in response to a certain first network test data reflecting network disconnection.
[0166] The second pixel value generation unit, when a certain first network test data reflects network connectivity, is used to respond to the fact that the latency and packet loss rate of the corresponding traffic passing through a certain node are within their respective corresponding ranges, to obtain the standardized and normalized color pixel values corresponding to the certain range, and to take the smaller of the two obtained color pixel values as a pixel value of a certain node.
[0167] In one embodiment, the array operation unit is specifically used for:
[0168] The time point to be estimated, x, is obtained from the following formula. t The estimated network topology graph array y t :
[0169]
[0170] in, It is in x t Previously (including x) t The cumulative product of all linear combination parameters in the network topology graph.
[0171] In one embodiment, the apparatus further includes:
[0172] The fault location unit, connected to the topology estimation module, is used in response to x t To pinpoint the network failure time points obtained from user complaints, the node with the most black pixels in the estimated network topology map is identified as the network failure point.
[0173] The prediction unit, connected to the topology estimation module, is used in response to x t To predict whether network congestion will occur at a future point in time, we determine whether the number of black pixels in the estimated network topology map exceeds a set threshold. If so, we determine that the link corresponding to the estimated network topology map will be congested at point x. t Network congestion may occur.
[0174] In one embodiment, the prediction unit further includes:
[0175] The splitting unit is used to determine whether the link corresponding to the estimated network topology will be in x. t After network congestion occurs, the system retrieves the link connections corresponding to the estimated network topology map, retrieves the second application server providing the same application service as the first application server, retrieves a subset of users corresponding to the estimated network topology map, and distributes the traffic of these users in x... t The time-sharing traffic is routed to the link connecting to the second application server.
[0176] Example 3:
[0177] Embodiment 3 of this disclosure provides a computer-readable storage medium storing a computer program. When the computer program is run by a processor, it implements the traffic monitoring and analysis method as described in Embodiment 1, or the traffic monitoring and analysis device as described in Embodiment 2.
[0178] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other memory technologies, CD-ROM (Compact Disc Read-Only Memory), DVD or other optical disc storage, cartridges, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.
[0179] In addition, this disclosure may also provide a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the traffic monitoring and analysis method as described in Embodiment 1, and the computer device may be the traffic monitoring and analysis device as described in Embodiment 2.
[0180] The memory is connected to the processor. The memory can be flash memory, read-only memory or other types of memory. The processor can be a central processing unit or a microcontroller.
[0181] Embodiments 1-3 of this disclosure provide a traffic policing analysis method, a traffic policing analysis device, and a computer-readable storage medium. The method acquires a normal network topology map and first network test data based on normal network time points. It then generates an estimated network topology map for the time point to be estimated based on the normal network topology map and the first network test data. The estimated network topology map displays the estimated node performance representation of each traffic flow passing through each node at the time point to be estimated. This effectively analyzes link traffic information, and allows for the acquisition of node performance estimation results based on the estimated network topology map. These estimation results can at least be used to locate network faulty nodes and predict network congestion, thus providing a more effective means of implementing traffic policing.
[0182] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.
Claims
1. A traffic monitoring and analysis method, characterized in that, The method includes: Acquisition and estimated time points Normal time points of the relevant network ; Obtaining normal time points A normal network topology diagram, which includes the nodes that traffic from user terminals accessing application servers passes through. Get at normal time point The first network test data is obtained by analyzing the traffic of each node in the normal network topology diagram. The first network test data includes node performance data collected from each node. The estimated time points are generated based on the normal network topology diagram and the first network test data. The estimated network topology includes the traffic flow at the time points to be estimated. The estimated node performance is represented by the values of each node. Acquisition and estimated time points Normal time points of the relevant network Specifically, it includes: In response to obtaining the network failure time point based on user complaints, the network failure time point is used as... Filtered based on user complaints At a certain point in time when the user's network access was normal, ;or, In response to the need to predict whether network congestion will occur at a certain future point in time, the certain future point in time is used as... To obtain the current time point when the network is not congested. ; The estimated time points are generated based on the normal network topology diagram and the first network test data. The estimated network topology includes: Convert a normal network topology diagram into a normal network topology diagram array. ; Pixel values are generated based on the node performance data of each node in the first network test data. Each pixel value represents the node performance of a certain traffic passing through a certain node, and a Gaussian noise array representing the node performance of each traffic passing through each node is obtained. ; Based on the normal network topology diagram array and Gaussian noise array The time points to be estimated are obtained using the Denoising Diffusion Probability Model (DDPM). Estimated network topology graph array ; Estimated network topology graph array Transform into an estimated network topology graph, where each pixel represents the traffic flow at the time point to be estimated. The estimated node performance is represented by each node.
2. The method according to claim 1, characterized in that, Acquisition and estimated time points Normal time points of the relevant network Previously, the method also included: The system receives network test data sent by the application server. The network test data is obtained by the user terminal sending network test packets to the application server according to the set scheduling time. The network test packets collect node performance data at each node they pass through in sequence. The network test data includes test time, network connectivity status, IP address of each node, and latency and packet loss rate of that node.
3. The method according to claim 1, characterized in that, Obtaining normal time points A normal network topology diagram includes the nodes that traffic from user terminals accessing application servers passes through, specifically including: Get The first network test data from the complaining user is used to obtain the nodes and node order through which the complaining user's traffic passes from the user terminal to the application server; a normal network topology diagram is then drawn based on the nodes and node order. Alternatively, Get This involves mapping the nodes and their order of access for a specific user traffic on a specific link of an application server, and then drawing a normal network topology based on these nodes and their order.
4. The method according to claim 3, characterized in that, Get at normal time point The first network test data, which includes the traffic flow at each node in a normal network topology diagram, specifically includes: Get in The first network test data of all user traffic that passed through the same nodes and node order as the complaining user; or... Get in The first network test data of all user access traffic passing through a certain link.
5. The method according to claim 1, characterized in that, Convert a normal network topology diagram into a normal network topology diagram array. Specifically, it includes: Obtain the first RGB value representation of the normal network topology graph, and then standardize and normalize each first RGB value to obtain an array of normal network topology graphs. .
6. The method according to claim 1, characterized in that, Pixel values are generated based on the node performance data of each node in the first network test data. Each pixel value represents the node performance of a certain traffic passing through a certain node, and a Gaussian noise array representing the node performance of each traffic passing through each node is obtained. Specifically, it includes: Based on the node performance data of each node in the test data of each first network, pixel values representing white to black are generated. White pixel values indicate that the node performance of a certain traffic is optimal when passing through a certain node, and black pixel values indicate that the node performance of a certain traffic is worst when passing through a certain node. After obtaining the pixel values representing the node performance of each traffic flow, the pixel values are arranged according to the topological position of the corresponding node in the normal network topology graph to obtain a Gaussian noise array. .
7. The method according to claim 6, characterized in that, Pixel values representing white to black are generated based on the node performance data of each node in the first network test data, specifically including: In response to a first network test result indicating network connectivity issues, a standardized and normalized black pixel value is generated for each node; otherwise... In response to a certain network test data reflecting that the latency and packet loss rate of the corresponding traffic passing through a certain node are within their respective ranges, the standardized and normalized color pixel values corresponding to the range are obtained, and the smaller of the two obtained color pixel values is taken as a pixel value of a certain node.
8. The method according to claim 1, characterized in that, Based on the normal network topology diagram array and Gaussian noise array The time points to be estimated are obtained using the Denoising Diffusion Probability Model (DDPM). Estimated network topology graph array Specifically, it includes: The time point to be estimated is obtained using the following formula. Estimated network topology graph array : = + in, Is Previously (including) The cumulative product of all linear combination parameters in the network topology graph.
9. The method according to claim 6 or 7, characterized in that, The estimated time points are generated based on the normal network topology diagram and the first network test data. After estimating the network topology, the method further includes: In response to To pinpoint the network failure time points obtained from user complaints, the node with the most black pixels in the estimated network topology map is identified as the network failure point. In response to To predict whether network congestion will occur at a future point in time, we determine whether the number of black pixels in the estimated network topology map exceeds a set threshold. If so, we determine that the link corresponding to the estimated network topology map will be congested. Network congestion may occur.
10. The method according to claim 9, characterized in that, Determine if the link corresponding to the estimated network topology will be in After network congestion occurs, the method further includes: Obtain the first application server corresponding to the estimated network topology map, obtain the second application server providing the same application service as the first application server, obtain a subset of users corresponding to the estimated network topology map, and allocate the traffic of the subset of users in... The time-sharing traffic is routed to the link connecting to the second application server.
11. A flow monitoring and analysis device, characterized in that, The device includes: The time acquisition module is used to acquire the time point to be estimated. Normal time points of the relevant network ; The normal topology module connects to the time acquisition module to obtain normal time points. A normal network topology diagram, which includes the nodes that traffic from user terminals accessing application servers passes through. The data acquisition module, connected to the normal topology module, is used to acquire data at normal time points. The first network test data is obtained by analyzing the traffic of each node in the normal network topology diagram. The first network test data includes node performance data collected from each node. The topology estimation module, connected to the data acquisition module, is used to generate estimated time points based on the normal network topology diagram and the first network test data. The estimated network topology includes the traffic flow at the time points to be estimated. The estimated node performance is represented by the values of each node. The time acquisition module is specifically used for: responding to obtaining the network failure time point based on user complaints, and using the network failure time point as... Filtered based on user complaints At a certain point in time when the user's network access was normal, Alternatively, in response to the need to predict whether network congestion will occur at a certain future point in time, the certain future point in time is used as... To obtain the current time point when the network is not congested. ; The topology estimation module is specifically used to: convert a normal network topology graph into a normal network topology graph array. ; Pixel values are generated based on the node performance data of each node in the first network test data. Each pixel value represents the node performance of a certain traffic passing through a certain node, and a Gaussian noise array representing the node performance of each traffic passing through each node is obtained. ; Based on the normal network topology diagram array and Gaussian noise array The time points to be estimated are obtained using the Denoising Diffusion Probability Model (DDPM). Estimated network topology graph array ; Estimated network topology graph array Transform into an estimated network topology graph, where each pixel represents the traffic flow at the time point to be estimated. The estimated node performance is represented by each node.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the traffic monitoring and analysis method as described in any one of claims 1-10.
Citation Information
Patent Citations
Network flow prediction method and device, computer equipment and readable storage medium
CN110995520A
Traffic exception processing method and device, electronic equipment and storage medium
CN116723115A
