A network fault detection and localization method based on partial traffic monitoring

Abstract

The application discloses a network fault detection and positioning method based on partial flow monitoring, which comprises the following steps: firstly, a monitoring switch is used to modify the packet header of data packets flowing through the monitoring switch, and ID information of the monitoring switch is written into the code as path information; the monitoring switch is configured with a flow meter to count data packet information of a large flow to be monitored; and a diagnosis server reasons the packet loss rate of different links based on network topology and packet loss information according to the collected data packet information, and locates a fault link. The method solves the problem of locating network faults based on the flow monitoring scheme, and realizes lightweight packet loss detection and positioning.

Description

Technical Field

[0001] This invention relates to the field of network fault measurement technology, and in particular to a method for network fault detection and location based on partial traffic monitoring. Background Technology

[0002] Packet loss is a very common problem in network communication. With the continuous expansion of network scale and the sustained growth of data traffic, the impact of packet loss on network performance and user experience is becoming increasingly apparent. Packet loss can lead to communication delays, compromised data integrity, and even complete network service unavailability. Component failure is one of the main causes of packet loss, specifically software errors and hardware failures on switches. These failures cause random packet loss, usually without any warning. Furthermore, various failures can occur on the link itself, such as connector contamination, fiber optic cable damage, transmitter or transceiver failure, etc., all of which can lead to packet corruption. In data centers, this unnoticed packet loss and corruption is very common and difficult to detect in a timely manner because upstream switches cannot detect packet loss, and downstream switches cannot recover any useful information from lost or corrupted packets. This poses a serious threat to network reliability and stability.

[0003] To detect and locate network faults, a common method is active probing. In this method, the system selects multiple terminal hosts to send and receive probe packets. These packets propagate along a pre-defined path, collecting probe results along the way, and ultimately inferring the fault location based on the results. The advantage of active probing is that it can proactively discover potential problems in the network and quickly locate faults. However, sending additional probe packets increases network traffic, impacting network performance and increasing network load. To reduce traffic overhead, some methods send probe packets at a lower rate; for example, in NetBouncer, each terminal host sends only 100 packets per 5 minutes. Low-rate probing reduces the impact of extra traffic but makes it difficult to detect low-probability packet loss events, reducing detection accuracy. Furthermore, the multi-path routing methods widely used in data centers introduce uncertainty into the probe packet path, posing a significant challenge to fault location.

[0004] Another common approach is passive probing. These systems collect information about incoming data packets from switches at different locations and detect faults by comparing this information. The advantage of passive probing is that it doesn't require sending additional probe packets, thus not increasing network traffic. For example, Netflow, a widely used information gathering technique in routers, reduces processing overhead by sampling data packets. However, sampling increases the difficulty of packet loss detection, especially for silent packet loss caused by faulty components, which neither reports to network administrators nor reveals problems in the sampling results. Therefore, to detect packet loss, researchers must monitor all network traffic, which undoubtedly places high demands on memory. To reduce memory overhead, some studies (such as LossRadar, FlowRadar, and ChameleMon) have used the sketch probabilistic data structure to detect packet loss. However, the drawback of these methods is that unless deployed at every hop, they can only detect packet loss but cannot accurately pinpoint the specific location of the fault. Summary of the Invention

[0005] The purpose of this invention is to provide a network fault detection and location method based on partial traffic monitoring. This method solves the problem of locating network faults on rooftops based on traffic monitoring schemes and achieves lightweight packet loss detection and location.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A method for network fault detection and location based on partial traffic monitoring, the method comprising:

[0008] Step 1: Modify the header of the data packets flowing through the monitoring switch using the monitoring switch, and write the ID information of the monitoring switch as path information into the encoding.

[0009] Step 2: The monitoring switch uses the configured flow meter to collect data packet information of the large flow to be monitored;

[0010] Step 3: Based on the data packet information collected in Step 2, the diagnostic server infers the packet loss rate of different links based on network topology and packet loss information, and locates the faulty link.

[0011] An electronic device includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method.

[0012] A computer storage medium storing a plurality of instructions adapted for loading and executing the method by a processor.

[0013] As can be seen from the technical solution provided by the present invention, the above method solves the problem of locating network faults on rooftops based on traffic monitoring schemes, realizes lightweight packet loss detection and location, and can adaptively select more large network flows on the path segment for measurement, simplifying the work of network administrators. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the network fault detection and location method based on partial traffic monitoring provided in an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram illustrating a topology and the distribution of monitoring switches in an example of the present invention.

[0017] Figure 3 This is a schematic diagram illustrating the process of using HashPipe to select large streams as described in this invention.

[0018] Figure 4 This is a flowchart illustrating a packet loss detection process as described in the example of this invention.

[0019] Figure 5 This is a schematic diagram illustrating the process of locating faulty links based on the success rate of path segments, as described in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0021] like Figure 1 The diagram shows a flowchart of a network fault detection and location method based on partial traffic monitoring provided by an embodiment of the present invention. The method includes:

[0022] Step 1: Modify the header of the data packets flowing through the monitoring switch using the monitoring switch, and write the ID information of the monitoring switch as path information into the encoding.

[0023] In this step, at the control layer, a unique ID is assigned to each monitoring switch using the MAC address in the monitoring switch interface to identify the monitoring switch; the path information consists of the monitoring switch ID string, which is stored in the header of each data packet through a Bloom filter, and its encoding and decoding are both performed by the Bloom filter.

[0024] When a data packet passes through the monitoring switch, the monitoring switch records the relevant data packet information and stores the monitoring switch's ID information as path information in the Bloom filter within the data packet header, denoted as path BF. During subsequent decoding, the diagnostic server tests the corresponding Bloom filter to see if the monitoring switch's ID appears. Specifically:

[0025] Each monitoring switch has a static list of hash functions. When a data packet arrives, it processes its own ID through the hash function to obtain the mapping position, finds the corresponding bit in the Bloom filter in the data packet, and sets all bits to 1 to indicate that the mapping is complete.

[0026] In this context, for a Bloom filter of length b, U represents the ID field of the monitoring switch, and m hash functions are represented as: U→{1,···,b}, i=1,···,m; i is the hash function index, representing the i-th hash function;

[0027] The monitoring switch s uses a list function to hash the ID of the monitoring switch to obtain h_i(s.ID), i = 1, ..., m, and sets the specified bit pkt.pathBF[hi(s.ID)] of the Bloom filter in the data packet to 1.

[0028] In practical implementation, to simplify information storage, path information needs to be segmented at appropriate locations. This application achieves this through monitoring switches. Flow meters are configured on some switches to collect relevant information about the data packets flowing through them; these switches are called monitoring switches. Considering all the monitoring switches in a given network path, the switch closest to a given switch is called its upstream switch, and this switch is called the downstream switch of that upstream switch. A pair of upstream and downstream switches covers a path segment, represented by a combination of their IDs. Each path consists of multiple path segments. Each switch encodes its own ID into the data packets flowing through it, but the monitoring switches clear the Bloom filter in the data packets before encoding, ensuring that the Bloom filter always contains only information about one path segment.

[0029] Step 2: The monitoring switch uses the configured flow meter to collect data packet information of the large flow to be monitored;

[0030] In this step, the flow meters configured in the monitoring switch are divided into downstream flow meters and upstream flow meters. The downstream flow meter is represented by DSM and is used to count the data packets of a group of flows received by the monitoring switch; the upstream flow meter is represented by USM and is used to count the data packets of a group of data packets sent by the monitoring switch.

[0031] Both DSM and USM consist of d rows, each row corresponds to an independent hash function, and each row has w buckets;

[0032] For DSM, each bucket contains the fields dirtyFlag, flowID, pathBF, and count; for USM, each bucket contains the fields dirtyFlag, flowID, and count. Here, dirtyFlag is a 1-bit flag indicating the availability of data in the bucket; flowID represents the ID of the recorded flow in the bucket (e.g., a 5-tuple); pathBF represents a b-bit Bloom filter, i.e., the path information carried by the data packet; and count represents the number of data packets belonging to the corresponding flow received by the flow meter.

[0033] The flow meter designs a rule table based on the selected flow and updates the fields in the bucket using data packets that match the rules; the DSM rule table matches based on the flowID and pathBF fields, while the USM rule table matches based on the flowID field. The specific field update operation is as follows:

[0034] 1) For each row, use the corresponding function to map the flowID carried by the data packet to a certain bucket;

[0035] 2) If the dirtyFlag field is equal to 1 within a bucket, then the bucket contains more than one stream, the data is unavailable, and the current row does not need to be updated;

[0036] 3) If the dirtyFlag field in the bucket is not equal to 1 and all other fields are empty, it means that the current data packet corresponds to a new flow that needs to be stored. Then, the fields carried by the data packet are transferred to the flow meter, and the count field in the bucket is initialized to 1.

[0037] 4) If the dirtyFlag field in the bucket is not equal to 1, other fields are not empty, and the information carried by the data packet matches the rule table in the flow meter, it means that the current data packet corresponds to a flow that has already been stored. Then, increment the count field in the bucket by 1 to indicate that a new data packet has been received.

[0038] In practice, for large flows to be monitored, the time is divided into several periods. After each period, the diagnostic server collects data from the monitoring switch to detect packet loss. If the diagnostic server finds that not enough information has been collected on a certain path segment after receiving the data, the network controller will send an instruction to the corresponding upstream and downstream switches to monitor more network flows in the next period.

[0039] To save costs, this embodiment uses HashPipe to detect large flows. HashPipe is a data plane algorithm for detecting large flows, containing a hash table with multiple stages. As data packets pass through, they sequentially pass through each stage. Through data exchange and comparison, it is ensured that the flow stored in each stage is larger than the flow carried by the data packet. When the diagnostic server finds that the monitored data packets or traffic on a path segment is less than the threshold, it tells the controller that more large flows (Heavy hitters, denoted as hh) need to be selected. The controller then encodes the required path segment as pathBF and inserts it into the HH table on the switch. In the next cycle, the switch will detect large flows through the multi-level hash table of HashPipe, and then match the detected flows with the HH table. Successfully matched flows will be monitored, reported to the controller, and then distributed to other switches by the controller.

[0040] Step 3: Based on the data packet information collected in Step 2, the diagnostic server infers the packet loss rate of different links based on network topology and packet loss information, and locates the faulty link.

[0041] In this step, the diagnostic server divides the diagnostic problems of all links in the network into several non-overlapping sub-problems. Each sub-problem covers a part of the network. For example, in a fat-tree network with k=4, each pod and the links between the pod's upstream switch and all top-level switches constitute a sub-problem, and the entire network consists of 4 sub-problems. In each sub-problem, the diagnostic server compares each downstream flow meter (DSM) with all its upstream flow meters (USMs) and stores the comparison results in a subtractive hash table, denoted as MM.

[0042] Each entry in MM has the following fields: 1) flowID: Represents the ID of the monitored flow; 2) pathBF: Represents a Bloom filter that stores data flow path information; 3) sent: Represents the number of corresponding flow data packets sent along the path segment in the current period; 4) lost: Represents the number of corresponding flow data packets lost on the path segment in the current period.

[0043] The specific process of MM updating is as follows:

[0044] 1) For DSM, find all available buckets of flow based on the dirtyFlag field, use a hash function to map flowID to a bucket in MM, and transfer all fields in DSM bucket except dirtyFlag to the corresponding fields in MM bucket;

[0045] 2) For each bucket in MM, check all USMs. If there is a clean bucket with the same flowID, subtract the count field in MM from the count field in USM bucket to get the lost field, record the number of lost packets, and transfer the count field in USM bucket to MM bucket to record the actual number of data packets in the flow.

[0046] 3) Finally, iterate through each bucket in MM and delete the entries whose lost field is empty.

[0047] In practice, the diagnostic server calculates the success rate of path segments using MM (Master-Side Analyzer) and infers the success rate of each link through the following optimization problem:

[0048]

[0049] Where x i y is the success rate of link i; λ is a constant parameter with a default value of 1; p Represents the path segment stored in MM p The success rate is defined as follows:

[0050]

[0051] Where F p It is the collection of all path segments stored in MM; f.sent and f.lost represent the sent and lost fields in the bucket of the stream f, respectively;

[0052] The first term of formula (1) is the difference of squares, which incorporates the success rate of the path segment into the inference; the second term is the regularization term, which incorporates x... i Push towards both ends to reduce false positive errors caused by measurement;

[0053] The above optimization problem (1) is described as follows: By adjusting the success rate of each link, under the constraint that the success rate of each link is non-negative and does not exceed 1, the combination of the most likely link success rates is inferred by minimizing the sum of the squared differences between the success rates of all path segments in MM and the corresponding link success rates, as well as the regularization term, to avoid false positive errors caused by measurement.

[0054] The above optimization problem (1) is solved using a multi-stage coordinate descent algorithm. Each stage focuses only on path segments with a success rate within a specific range. The specific process is as follows:

[0055] 1) Divide the path segments with non-zero packet loss rate into 5 groups, namely [0.2,1], [0.1,0.2], [0.05,0.1], [0.01,0.05], and [0.001,0.01];

[0056] 2) Sort the groups in descending order of packet loss rate. For each group, first map the packet loss rate of the path segments in the group to [0, 0.8]. For groups with a packet loss rate greater than its own, map all the packet loss rates of the path segments in the group to 1, and vice versa.

[0057] 3) Apply the coordinate descent algorithm to the packet loss rate of the mapped path segment to obtain the link success rate. This process is as follows:

[0058] First, initialize the success rate x of each link based on the following formula. i :

[0059] x i =avg({y j |link i ∈path j})

[0060] That is, the success rate x of each link i It equals the average success rate of all path segments that contain it;

[0061] Then the iteration begins, with X representing the success rate of all current links, Y representing the success rate of all current path segments, and f(X,Y) representing the result of formula (1) for X and Y. The success rate of each link is determined by the following formula:

[0062]

[0063] That is, x i As variables, the success rates of other links in X are fixed as constants. The success rate of each link in X is found by minimizing f(X,Y). i The value;

[0064] The above process is iterated k times. If f(X,Y) < ε in the two iterations before and after, where ε is the path segment error threshold, the iteration is stopped early. After the iteration stops, the link success rate inferred in the current stage is obtained. Links with a success rate less than 1 are judged as faulty links, and the success rate of these links will not be inferred in the next stage.

[0065] Additionally, when the monitoring switch reports to the diagnostic server, the diagnostic server decodes the path of the corresponding flow based on the received pathBF. The diagnostic server can obtain network topology and configuration information through the control layer, i.e., understand the ID of the monitoring switch and the path segment composition between adjacent monitoring switches. Since the pathBF contains only one path segment, the diagnostic server first tests whether the ID of the adjacent monitoring switch appears in the Bloom filter, finding all possible path segments. If the path segment between these two switches is unique, then decoding can be successful directly; otherwise, it continues to test whether the ID of the non-monitoring switch appears, thus decoding the path. Although the Bloom filter has false positive errors, which may cause incorrect path segments to be misjudged, according to the characteristics of the CLOS structure, incorrect path segments may only overlap with correct path segments at the first and last switches, proving that the false judgment probability P is negligible. fail as follows:

[0066]

[0067] Where m represents the number of hash functions; b represents the number of bits in the Bloom filter; and h is the number of hops in the end-to-end path segment.

[0068] There is a very small probability of decoding failure in the CLOS structure, but since the switch ID is static, path segments can be enumerated offline and the potential for decoding failure can be detected. If a potential decoding failure is detected, the ID of the switch that could cause decoding failure can be reassigned to ensure that all path segments can be uniquely decoded.

[0069] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to specific examples, such as... Figure 2 The diagram shown illustrates a topology and monitoring switch distribution as exemplified by this invention. Figure 2 This describes a Fat-Tree topology that can be used with this algorithm. Fat-Tree is a commonly used network topology in data centers, consisting of three layers: core, aggregation, and access. In each layer, the forwarding port is determined based on the source and destination IP addresses of the received IP packets, effectively distributing packets evenly across different paths according to IP addresses. By using a multi-rooted tree structure, the number of switches in each layer is increased, widening the network topology and ensuring that each switch has k ports. There are k pods, each consisting of k switches. There are k paths between any two pods, and packet paths are evenly distributed according to IP addresses.

[0071] The current topology distributes the monitoring switches as follows: all core layer and access layer switches are configured as monitoring switches, meaning the top and bottom layer switches function as monitoring switches. The processing pipeline for the monitoring switches is as follows:

[0072] Step 1: The DSM matches the flow ID and pathBF according to the rule table. If a match is successful, the DSM table is updated with packet information.

[0073] Step 2: The switch calls the HashPipe algorithm to update the flow stored in the multi-level hash table, and then matches the pathBF and the number of packets according to the rule table. The flow that exceeds the threshold is selected as the monitoring flow and reported to the switch.

[0074] Step 3: Clear the Bloom filter in the data packet, and then the USM matches the flow ID according to the rule table. If a match is successful, the USM table is updated with the data packet information. After the update is completed, the switch encodes its own ID into the pathBF on the data packet.

[0075] like Figure 3 The diagram illustrates the process of selecting large streams using HashPipe as described in this invention, including the following steps:

[0076] Step 1: If the diagnostic tool determines that a certain path segment needs to have a monitoring stream added, it sends a message to the controller indicating that a monitoring stream needs to be added to the corresponding path segment.

[0077] Step 2: The controller inserts the new rule into the HH table, requiring the target flow to carry the same pathBF as the corresponding path segment, belonging to a large flow and not yet being monitored;

[0078] Step 3: After each data packet arrives, determine whether it is a large flow through a multi-level hash table, and search for a matching rule in the HH table. If a flow is successfully matched, it is treated as a new monitoring object and reported to the controller.

[0079] Step 4: After receiving the message, the controller inserts the corresponding flow rules into the USM table and DSM table, informing the switch that monitoring is required;

[0080] Step 5: The controller deletes the matching rule from the HH table, terminating the selection.

[0081] like Figure 4 The diagram shows a packet loss detection workflow of an example of the present invention, where each DSM or USM bucket is represented by a triple (flowID, dirtyFlag, count), and each MM bucket is represented by a quadruple (flowID, pathBF, sent, lost).

[0082] To calculate the MM table on the monitoring switch s6, first locate its corresponding DSM table and all USM tables. Observing the DSM, we can find the clean buckets corresponding to f1, f2, f3, and f4. First, search for the clean bucket for f1 in all USM tables. We find that a target bucket exists on the monitoring switch s17. Therefore, the path for f1 is s17→s7→s6, and we can calculate sent = 20 and lost = 1, indicating a packet loss event has occurred. Repeat the above process for f2, f3, and f4 to obtain the MM table shown in the figure. The final detection result is that packet loss occurred for f1 and f4.

[0083] like Figure 5 The diagram illustrates the process of locating faulty links based on path segment success rate, as described in this invention. Figure 5 (a) shows the sub-problems selected in the embodiment and the actual values, where the packet loss rate for s78→s40 is 0.3 and the packet loss rate for s40→s8 is 0.002; Figure 5 (b) Lists 7 path segments where packet loss was detected, all starting with s78 or ending with s8.

[0084] The above path segments are divided into different groups. The four path segments from s78 to s40 have a packet loss rate around 0.3, so they are assigned to group [0.2, 1]. The other path segments have a packet loss rate close to 0.002, so they are assigned to group [0.0001, 0.01]. Based on this division, the first iteration selects group [0.2, 1], maps the success rate of all path segments in this group to 1, and applies the coordinate descent algorithm. Figure 5 (c) shows the state after the first iteration, successfully identifying s78→s40 as the faulty link, explaining the cause of packet loss on the relevant path segment, and removing it from the subproblem. Then, the second iteration is performed, selecting the group [0.0001, 0.01]. Since the success rate of the path segments in this group is too high, a logarithmic-based function is used to map it to the range [0, 0.8], and then the coordinate descent algorithm is applied. Figure 5 (d) shows the state after the second iteration, successfully identifying s40→s8 as the faulty link and explaining the reason for packet loss on the remaining path segment.

[0085] In summary, the method described in the embodiments of the present invention has the following advantages:

[0086] 1. Effective location: Path information is stored in the packet header to achieve fault location, which solves the problem of locating network faults on rooftops based on traffic monitoring solutions;

[0087] 2. Lightweight: This method is based on passive traffic monitoring and only needs to monitor a portion of the traffic, rather than the entire network traffic, which reduces the network burden and saves measurement costs.

[0088] 3. Adaptive: This method ensures that enough data packets are calculated on each path segment, and when there are insufficient data packets to monitor on a path segment due to traffic dynamics, it can adaptively select more large network flows on that path segment for measurement, simplifying the work of network administrators.

[0089] 4. Accuracy: This method directly monitors data packets in the network, which can collect more comprehensive information compared to active probing. It also improves the inference algorithm and can detect faulty links that were ignored in previous algorithms.

[0090] Furthermore, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0091] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A network fault detection and location method based on partial traffic monitoring, characterized in that, The method includes: Step 1: Modify the header of the data packets flowing through the monitoring switch using the monitoring switch, and write the ID information of the monitoring switch as path information into the encoding. Step 2: The monitoring switch uses the configured flow meter to collect data packet information of the large flow to be monitored; Step 3: Based on the data packet information collected in Step 2, the diagnostic server infers the packet loss rate of different links based on network topology and packet loss information, and locates the faulty link.

2. The network fault detection and location method based on partial traffic monitoring according to claim 1, characterized in that, In step 1, at the control layer, a unique ID is assigned to each monitoring switch using the MAC address in the monitoring switch interface to identify the monitoring switch; the path information consists of the monitoring switch ID string, which is stored in the header of each data packet through a Bloom filter, and its encoding and decoding are both performed by the Bloom filter. When a data packet passes through the monitoring switch, the monitoring switch records the relevant data packet information and stores the monitoring switch's ID information as path information in the Bloom filter within the data packet header, denoted as path BF. During subsequent decoding, the diagnostic server tests the corresponding Bloom filter to see if the monitoring switch's ID appears. Specifically: Each monitoring switch has a static list of hash functions. When a data packet arrives, it processes its own ID through the hash function to obtain the mapping position, finds the corresponding bit in the Bloom filter in the data packet, and sets all bits to 1 to indicate that the mapping is complete. In this context, for a Bloom filter of length b, U represents the ID field of the monitoring switch, and m hash functions are represented as: U→{1,···,b}, i=1,···,m; i is the hash function index, representing the i-th hash function; The monitoring switch s uses a list function to hash the ID of the monitoring switch to obtain h_i(s.ID), i = 1, ..., m, and sets the specified bit pkt.path BF[hi(s.ID)] of the Bloom filter in the data packet to 1.

3. The network fault detection and location method based on partial traffic monitoring according to claim 1, characterized in that, In step 2, the flow meters configured in the monitoring switch are divided into downstream flow meters and upstream flow meters. The downstream flow meter is represented by DSM and is used to count the data packets of a group of flows received by the monitoring switch; the upstream flow meter is represented by USM and is used to count the data packets of a group of data packets sent by the monitoring switch. Both DSM and USM consist of d rows, each row corresponds to an independent hash function, and each row has w buckets; For DSM, each bucket contains the fields dirtyFlag, flowID, pathBF, and count; for USM, each bucket contains the fields dirtyFlag, flowID, and count. Here, dirtyFlag is a 1-bit flag indicating the availability of data in the bucket; flowID represents the ID of the stream recorded in the bucket; pathBF represents a b-bit Bloom filter, i.e., the path information carried by the data packet; and count represents the number of data packets belonging to the corresponding stream received by the flow meter. The flow meter designs a rule table based on the selected flow and updates the fields in the bucket using data packets that match the rules; the DSM rule table matches based on the flowID and pathBF fields, while the USM rule table matches based on the flowID field. The specific field update operation is as follows: 1) For each row, use the corresponding function to map the flowID carried by the data packet to a certain bucket; 2) If the dirtyFlag field is equal to 1 within a bucket, then the bucket contains more than one stream, the data is unavailable, and the current row does not need to be updated; 3) If the dirtyFlag field in the bucket is not equal to 1 and all other fields are empty, it means that the current data packet corresponds to a new flow that needs to be stored. Then, the fields carried by the data packet are transferred to the flow meter, and the count field in the bucket is initialized to 1. 4) If the dirtyFlag field in the bucket is not equal to 1, other fields are not empty, and the information carried by the data packet matches the rule table in the flow meter, it means that the current data packet corresponds to a flow that has already been stored. Then, increment the count field in the bucket by 1 to indicate that a new data packet has been received.

4. The network fault detection and location method based on partial traffic monitoring according to claim 1, characterized in that, In step 2, for the large flow to be monitored, the time is divided into several periods. After each period, the diagnostic server collects data from the monitoring switch to detect packet loss. If the diagnostic server finds that not enough information has been collected on a certain path segment after receiving the data, the network controller will issue an instruction to the corresponding upstream and downstream switches that more network flows need to be monitored in the next period.

5. The network fault detection and location method based on partial traffic monitoring according to claim 1, characterized in that, In step 3, the diagnostic server divides the diagnostic problems of all links in the network into several non-overlapping sub-problems. Each sub-problem covers a part of the network. In each sub-problem, the diagnostic server compares each downstream flow meter (DSM) with all its upstream flow meters (USM) and stores the comparison results in a subtractive hash table, denoted as MM. Each entry in MM has the following fields: 1) flowID: Represents the ID of the monitored flow; 2) pathBF: Represents a Bloom filter that stores data flow path information; 3) sent: Represents the number of corresponding flow data packets sent along the path segment in the current period; 4) lost: Represents the number of corresponding flow data packets lost on the path segment in the current period. The specific process of MM updating is as follows: 1) For DSM, find all available buckets of flow based on the dirtyFlag field, use a hash function to map flowID to a bucket in MM, and transfer all fields in DSM bucket except dirtyFlag to the corresponding fields in MM bucket; 2) For each bucket in MM, check all USMs. If there is a clean bucket with the same flowID, subtract the count field in MM from the count field in USM bucket to get the lost field, record the number of lost packets, and transfer the count field in USM bucket to MM bucket to record the actual number of data packets in the flow. 3) Finally, iterate through each bucket in MM and delete the entries whose lost field is empty.

6. The network fault detection and location method based on partial traffic monitoring according to claim 5, characterized in that, In step 3, the diagnostic server calculates the success rate of path segments using MM and infers the success rate of each link through the following optimization problem: minimize∑ p (y p -Π i∈p x i ) 2 +λ∑ i x i (1-x i ) s. t. 0≤x i ≤1 (1) Where x i It is the success rate of link i; λ is a constant parameter with a default value of 1; y p Represents the path segment stored in MM p The success rate is defined as follows: Where F p It is the collection of all path segments stored in MM; f.sent and f.lost represent the sent and lost fields in the bucket of the stream f, respectively; The first term of formula (1) is the difference of squares, which incorporates the success rate of the path segment into the inference; the second term is the regularization term, which incorporates x... i Push towards both ends to reduce false positive errors caused by measurement; The above optimization problem (1) is described as follows: By adjusting the success rate of each link, under the constraint that the success rate of each link is non-negative and does not exceed 1, the combination of the most likely link success rates is inferred by minimizing the sum of the squared differences between the success rates of all path segments in MM and the corresponding link success rates, as well as the regularization term, to avoid false positive errors caused by measurement. A multi-stage coordinate descent algorithm is used to solve the above optimization problem (1). Each stage only focuses on path segments with a success rate within a specific range. The specific process is as follows: 1) Divide the path segments with non-zero packet loss rate into 5 groups, namely [0.2,1], [0.1,0.2], [0.05,0.1], [0.01,0.05], and [0.001,0.01]; 2) Sort the groups in descending order of packet loss rate. For each group, first map the packet loss rate of the path segments in the group to [0, 0.8]. For groups with a packet loss rate greater than its own, map all the packet loss rates of the path segments in the group to 1, and vice versa. 3) Apply the coordinate descent algorithm to the packet loss rate of the mapped path segment to obtain the link success rate.

7. The network fault detection and location method based on partial traffic monitoring according to claim 6, characterized in that, The coordinate descent algorithm is applied to the packet loss rate of the mapped path segment to obtain the link success rate. The specific process is as follows: First, initialize the success rate x of each link based on the following formula. i : x i =avg({y j |link i ∈path j }) That is, the success rate x of each link i It equals the average success rate of all path segments that contain it; Then the iteration begins, with X representing the success rate of all current links, Y representing the success rate of all current path segments, and f(X,Y) representing the result of formula (1) for X and Y. The success rate of each link is determined by the following formula: That is, x i As variables, the success rates of other links in X are fixed as constants. The success rate of each link in X is found by minimizing f(X,Y). i The value; The above process is iterated k times. If the calculated f(X,Y) in the previous two iterations is less than ε, where ε is the path segment error threshold, then the iteration is stopped early. After the iteration stops, the link success rate inferred in the current stage is obtained. Links with a success rate of less than 1 are judged as faulty links, and the success rate of these links will not be inferred in the next stage.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method of any one of claims 1 to 7.

Images