Flow sampling method, system, device, computer device and readable storage medium

Abstract

This application relates to a traffic sampling method, system, apparatus, computer equipment, and readable storage medium, belonging to the field of data communication technology. The method includes: acquiring initial traffic from a service center channel; the initial traffic includes traffic from multiple services; performing flow classification, tagging, and filtering processing on the initial traffic sequentially to obtain an aggregate index and target traffic for each service; the aggregate index represents the correspondence between the target traffic and each service, with each aggregate index pointing to multiple members carrying the target traffic; mapping based on the aggregate index of each service to obtain a mirror member index for each service; the mirror member index corresponds one-to-one with the aggregate index, with each mirror member index pointing to multiple mirror members used for sampling the target traffic; sampling the target traffic based on each mirror member index and preset sampling configuration rules to obtain the sampled traffic, and sending it to a traffic analysis server. This method can improve efficiency.

Description

Technical Field

[0001] This application relates to the field of data communication technology, and in particular to a traffic sampling method, system, apparatus, computer equipment, and readable storage medium. Background Technology

[0002] Currently, with the rapid development of cloud services, the complexity and scale of data center networks are also growing exponentially. This poses a great challenge to the analysis of traffic data and the location of hidden faults. In order to obtain effective network information, traffic sampling methods are used to collect and analyze the traffic of data center networks.

[0003] In related technologies, optical traffic sampling is employed. On the main network link, an optical splitter is installed to "copy" the traffic to an analysis server. A custom algorithm analyzes the traffic to obtain information helpful for operations and optimization. However, the optical splitter completely "copys" all the traffic to the analysis server, requiring a significant amount of time to obtain useful information. Therefore, these traffic collection methods suffer from low efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, system, device, computer equipment, and readable storage medium that can improve the flow sampling method, system, apparatus, and readable storage medium to address the above-mentioned technical problems.

[0005] Firstly, this application provides a traffic sampling method, including:

[0006] Obtain the initial traffic of the business center channel; the initial traffic includes traffic from various services;

[0007] The initial traffic is sequentially processed by flow classification, tagging, and filtering to obtain an aggregate index and target traffic for each of the services. The aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic.

[0008] The aggregate index of each service is mapped to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic;

[0009] The target traffic is sampled based on each of the mirror member indices and the preset sampling configuration rules to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server.

[0010] In one embodiment, the step of sequentially performing flow classification, tagging, and filtering on the initial traffic to obtain an aggregated index and target traffic for each service includes:

[0011] The initial traffic is classified according to a preset matching rule to obtain the initial traffic for each service.

[0012] The initial traffic for each of the aforementioned services is tagged to obtain the tagged traffic and aggregated index for each of the aforementioned services;

[0013] The marked traffic is filtered to obtain the target traffic for each of the services.

[0014] In one embodiment, the filtering process of the tagged traffic to obtain the target traffic for each of the services includes:

[0015] Based on the dual LAN tag and packet truncation tag of the tagged traffic, the tagged traffic is filtered to obtain the target traffic for each of the services.

[0016] In one embodiment, the step of performing flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic for each of the services includes:

[0017] Obtain the pre-configured message configuration for each of the aforementioned services;

[0018] The initial traffic is parsed to obtain the parsed traffic;

[0019] According to the preset matching rules and the preset message configuration for each service, the parsed traffic is subjected to flow classification processing to obtain the initial traffic for each service.

[0020] In one embodiment, sampling the target traffic based on each of the mirror member indices and preset sampling configuration rules to obtain the sampled traffic includes:

[0021] Obtain the total number of target traffic segments for all the aforementioned services.

[0022] The target traffic is sampled based on the number of mirror members pointed to by the mirror member index, the total number of target traffic segments, and the preset sampling configuration rules to obtain the sampled traffic.

[0023] In one embodiment, the step of sampling the target traffic based on the number of mirror members pointed to by the mirror member index, the total number of target traffic components, and the preset sampling configuration rules to obtain the sampled traffic includes:

[0024] The number of cyclic sampling times N is determined based on the total number of target traffic components and the number of mirror members;

[0025] According to the preset sampling configuration rules, each of the mirror members is called, and each of the target traffic is sampled sequentially N-1 times, and the sampling order of the mirror members in the Nth sampling is in reverse order; wherein, the first of the mirror members is sampled by default for the first time for the first time for the first portion of the target traffic and the second portion of the target traffic.

[0026] Secondly, this application also provides a traffic sampling system, including: a splitter, a programmable switch, and an analysis server; wherein,

[0027] The splitter is located in the service center channel and connected to the programmable switch. It is used to copy the initial traffic of the service center channel and send the initial traffic to the programmable switch.

[0028] The programmable switch is connected to the analysis server and is used to execute the steps of the method described in any of the above embodiments;

[0029] The analysis server is used to receive the sampled traffic sent by the programmable switch and to analyze and process the sampled traffic.

[0030] Thirdly, this application also provides a flow sampling device, comprising:

[0031] The traffic acquisition module is used to acquire the initial traffic of the business center channel; the initial traffic includes traffic from various services.

[0032] The traffic processing module is used to perform flow classification, tagging, and filtering on the initial traffic in sequence to obtain an aggregate index and target traffic for each of the services; the aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic;

[0033] The mirror mapping module is used to map according to the aggregate index of each of the services to obtain the mirror member index of each of the services; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic;

[0034] The traffic sampling module is used to sample the target traffic based on each of the mirror member indices and preset sampling configuration rules, obtain the sampled traffic, and send the sampled traffic to the traffic analysis server.

[0035] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0036] Obtain the initial traffic of the business center channel; the initial traffic includes traffic from various services;

[0037] The initial traffic is sequentially processed by flow classification, tagging, and filtering to obtain an aggregate index and target traffic for each of the services. The aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic.

[0038] The aggregate index of each service is mapped to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic;

[0039] The target traffic is sampled based on each of the mirror member indices and the preset sampling configuration rules to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server.

[0040] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0041] Obtain the initial traffic of the business center channel; the initial traffic includes traffic from various services;

[0042] The initial traffic is sequentially processed by flow classification, tagging, and filtering to obtain an aggregate index and target traffic for each of the services. The aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic.

[0043] The aggregate index of each service is mapped to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic;

[0044] The target traffic is sampled based on each of the mirror member indices and the preset sampling configuration rules to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server.

[0045] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0046] Obtain the initial traffic of the business center channel; the initial traffic includes traffic from various services;

[0047] The initial traffic is sequentially processed by flow classification, tagging, and filtering to obtain an aggregate index and target traffic for each of the services. The aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic.

[0048] The aggregate index of each service is mapped to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic;

[0049] The target traffic is sampled based on each of the mirror member indices and the preset sampling configuration rules to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server.

[0050] The aforementioned traffic sampling method, system, device, computer equipment, and readable storage medium acquire initial traffic from the business center channel. This initial traffic includes traffic from various services, serving as the basis for subsequent traffic sampling. The initial traffic undergoes sequential flow classification, tagging, and filtering to obtain aggregated indexes and target traffic for each service. The aggregated indexes represent the correspondence between target traffic and each service, with each index pointing to multiple members of the target traffic load. The flow classification, tagging, and filtering of the initial traffic enables fine-grained flow classification of traffic from different services, facilitating subsequent server offloading analysis, extraction of target traffic, and reducing the burden on the system. The sampling traffic load is increased, which also reduces the processing burden on the subsequent analysis server. Furthermore, the aggregated index of each service is mapped to obtain the mirror member index of each service. The mirror member index corresponds one-to-one with the aggregated index, and each mirror member index points to multiple mirror members used for sampling target traffic. The target traffic is sampled based on each mirror member index and the preset sampling configuration rules to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server. The mirror group does not need to sample all traffic of the service center channel, but only needs to sample the target traffic of each service after a series of processing, thereby improving the traffic sampling efficiency. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a diagram illustrating the application environment of the flow sampling method in one embodiment;

[0053] Figure 2 This is a flowchart illustrating a traffic sampling method in one embodiment;

[0054] Figure 3 This is a flowchart illustrating the target traffic determination step in one embodiment;

[0055] Figure 4 This is a schematic diagram of the flow sampling system in one embodiment;

[0056] Figure 5 This is a flowchart illustrating a flexible sampling method based on a programmable switching chip in one embodiment;

[0057] Figure 6 This is a structural block diagram of a flow sampling device in one embodiment;

[0058] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] The flow sampling method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the application scenario includes: a service center channel, a splitter 102, a programmable switch 104, and an analysis server 106. The splitter 102 is located in the service center channel and is connected to the programmable switch 104. The programmable switch 104 is communicatively connected to the analysis server. The programmable switch can be a programmable chip, and the analysis server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The programmable switch 104 obtains the initial traffic of the service center channel through the splitter 102. The initial traffic includes traffic from various services, and performs flow classification, tagging, and filtering processing on the initial traffic to obtain an aggregate index and target traffic for each service. The aggregate index represents the correspondence between the target traffic and each service, and each aggregate index points to multiple members that carry the target traffic. Further, the programmable switch 104 maps according to the aggregate index of each service to obtain a mirror member index for each service. The mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic. Then, the programmable switch 104 samples the target traffic based on each mirror member index and the preset sampling configuration rules, obtains the sampled traffic, and sends the sampled traffic to the traffic analysis server 106.

[0061] In one exemplary embodiment, such as Figure 2 As shown, a flow sampling method is provided, which is applied to... Figure 1 Taking the programmable switch 104 as an example, the explanation includes the following steps S202 to S208. Wherein:

[0062] Step S202: Obtain the initial traffic of the business center channel.

[0063] The initial traffic includes traffic from various services.

[0064] The business center channel can be the data transmission channel for the enterprise system to process various business operations, and all business-related traffic will be transmitted through this channel.

[0065] Optionally, the programmable switch acquires the initial traffic transmitted in the service center channel through an optical splitter installed in the service center channel. The optical splitter is a passive optical device that can redistribute the power intensity of the optical signal according to the required ratio. After receiving the initial traffic, the programmable switch processes it through a primary forwarding process, a Layer 2 forwarding process, and a Layer 3 forwarding process before it enters the target traffic extraction process.

[0066] Step S204: Perform flow classification, tagging, and filtering on the initial traffic in sequence to obtain the aggregated index and target traffic for each service.

[0067] Among them, the aggregated index represents the correspondence between the target traffic and each service. Each aggregated index points to multiple members of the target traffic. The aggregated index can be an identifier.

[0068] Among them, flow classification processing can refer to classifying the initial flow; tagging processing can refer to labeling the initial flow with tags; and filtering processing can refer to removing target information that does not need to be sampled from the initial flow.

[0069] The target traffic can be the traffic to be analyzed, which is determined based on the actual traffic analysis requirements.

[0070] Optionally, the programmable switch performs flow classification, tagging, and filtering on the initial traffic to obtain the aggregation index and target traffic for each service. This series of actions can be performed in the ACL (Access Control List) of the programmable switch. The ACL is used to define which traffic can be mirrored or monitored. Through the ACL, it is possible to precisely control which traffic is copied to the mirror port. Specifically, the initial traffic is flow classified to obtain the initial traffic corresponding to each service. The initial traffic corresponding to each service is tagged with the corresponding label to obtain the initial traffic after each service is tagged. The initial traffic after each service is filtered to obtain the target traffic for each service. And based on the label corresponding to the target traffic and the members that load the target traffic, the aggregation index corresponding to each service is determined.

[0071] Step S206: Map the aggregated index of each service to obtain the mirror member index of each service.

[0072] Among them, the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling target traffic.

[0073] Optionally, the programmable switch maps the aggregate index and the members included in the aggregate index to the mirror port according to each service, simulating the process of building the aggregate index, and obtains the mirror member index of each service. At this time, the mirror member index only contains the mirror member index identifier and the associated mirror member, and the traffic load of the mirror member is empty.

[0074] Step S206: Sample the target traffic based on each mirror member index and the preset sampling configuration rules to obtain the sampled traffic, and send the sampled traffic to the traffic analysis server.

[0075] The preset sampling configuration rules can be the number of samples collected by the mirror members for the target traffic, the collection order, etc.

[0076] Optionally, the programmable switch instructs each mirror member to sample the target traffic based on the multiple mirror members included in each mirror member index and the preset sampling configuration rules, until the target traffic of all services has been sampled by the mirror members to obtain the sampled traffic. The programmable switch then performs packet encapsulation and editing on the sampled traffic, and then sends the encapsulated sampled traffic to the traffic analysis server to analyze the sampled traffic and realize the monitoring of traffic in the service center channel.

[0077] In the aforementioned traffic sampling method, initial traffic from the business center channel is acquired. This initial traffic includes traffic from various services and serves as the basis for subsequent traffic sampling. The initial traffic undergoes sequential flow classification, tagging, and filtering to obtain aggregated indexes and target traffic for each service. The aggregated indexes represent the correspondence between target traffic and each service, with each index pointing to multiple members of the target traffic load. Performing flow classification, tagging, and filtering on the initial traffic enables fine-grained flow classification of traffic from different services, facilitating subsequent server offloading and extraction of target traffic. This reduces the traffic load on the sampled traffic and improves efficiency. While increasing the sampling load, it also reduces the processing burden on the subsequent analysis server. Furthermore, it maps the aggregated index of each service to obtain the mirror member index of each service. The mirror member index corresponds one-to-one with the aggregated index, and each mirror member index points to multiple mirror members used for sampling target traffic. Based on each mirror member index and the preset sampling configuration rules, the target traffic is sampled to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server. The mirror group does not need to sample all the traffic of the service center channel, but only needs to sample the target traffic of each service after a series of processing, thereby improving the traffic sampling efficiency.

[0078] In one exemplary embodiment, such as Figure 3 As shown, step S204 sequentially performs flow classification, tagging, and filtering on the initial traffic to obtain the aggregated index and target traffic for each service, including steps 302 to 306. Wherein:

[0079] Step 302: Perform flow classification processing on the initial traffic according to the preset matching rules to obtain the initial traffic for each service.

[0080] The preset matching rules can be the classification criteria for each business. For example, the traffic characteristics corresponding to each business can be matched with the traffic characteristics of the initial traffic. The initial traffic that can be successfully matched with the traffic characteristics corresponding to any business belongs to the initial traffic of that business.

[0081] Optionally, the programmable switch performs flow classification processing on the initial traffic according to preset matching rules, dividing the initial traffic into service categories that meet the preset matching rules, and obtaining the initial traffic for each service.

[0082] Step 304: Tag the initial traffic for each service to obtain the tagged traffic and aggregated index for each service.

[0083] Optionally, the programmable switch can tag the initial traffic for each service. Based on traffic analysis requirements, it can determine the part of the initial traffic packets that needs to be extracted, and add preset tags at the start and end points of that part. In addition, based on the load member identifier of the initial traffic corresponding to each service, it can construct an aggregate index for each service, which points to multiple members of the initial traffic that loads the service.

[0084] Step 306: Filter the marked traffic to obtain the target traffic for each service.

[0085] Optionally, the programmable switch determines the start and end points of the target traffic to be extracted based on the tags in the tagged traffic, filters the tagged traffic based on the start and end points, extracts the target traffic, deletes the remaining interfering traffic, and obtains the target traffic for each service.

[0086] In this embodiment, through preset matching rules, the programmable switch can accurately identify and classify traffic of different service types, thereby improving the utilization efficiency of network resources and ensuring that different service traffic is properly processed. By tagging the initial traffic, traffic of each service can be quickly identified and processed in subsequent traffic management, avoiding forwarding all traffic from the service center to the mirror port for sampling and improving the efficiency of traffic sampling.

[0087] In an exemplary embodiment, step S306 filters the tagged traffic to obtain the target traffic for each service, including:

[0088] Based on the dual LAN tagging and packet truncation tagging of the tagged traffic, the tagged traffic is filtered to obtain the target traffic for each service.

[0089] The dual LAN tag can be an inner LAN tag (inner_vlan_tag) and an outer LAN tag (outer_vlan_tag). The VLAN tag is a field added to the Ethernet frame, mainly used to identify which VLAN the frame belongs to. In this embodiment, adding dual LAN tags to the initial traffic packet can identify the type of service to which the initial traffic belongs.

[0090] The message truncation marker can be the cutoff point of the target traffic to be extracted from the initial traffic. The position of the message truncation marker is set according to the actual traffic collection requirements.

[0091] Optionally, the programmable switch determines the start and end points of the target traffic in the initial traffic based on the dual LAN tag and packet truncation tag of the tagged traffic, and filters the tagged traffic to obtain the target traffic for each service.

[0092] This embodiment further illustrates the specific implementation of the tags set for initial traffic tagging and the filtering to obtain target traffic. Based on dual LAN tags and packet truncation tags, the target traffic can be extracted more accurately, further improving traffic sampling efficiency.

[0093] In an exemplary embodiment, step S302 performs flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic for each service, including:

[0094] The initial traffic is parsed to obtain the parsed traffic; according to the preset matching rules and the preset message configuration for each service, the parsed traffic is classified to obtain the initial traffic for each service.

[0095] The message configuration can include the message IP (Internet Protocol) address, protocol number, port number, etc., of the traffic messages corresponding to each service.

[0096] Optionally, the programmable switch obtains the preset message configuration corresponding to each service, obtains the initial traffic of the service center channel through the optical splitter, performs message parsing on the initial traffic to obtain the parsed traffic. The parsed traffic includes the message configuration of the initial traffic. According to the preset matching rules and the preset message configuration for each service, the parsed traffic is processed by flow classification. The parsed traffic that can match the message configuration of any service is taken as the initial traffic of that service type, thereby obtaining the initial traffic of each service.

[0097] In this embodiment, by utilizing the parsed traffic and preset packet configuration, different types of service traffic can be clearly identified and classified, ensuring that each type of service traffic is properly processed and monitored. Multiple services in the network can be independently supported through effective traffic classification and processing. This not only prevents mutual interference between different services, but also improves security through VLAN isolation.

[0098] In an exemplary embodiment, step S208 samples the target traffic based on each mirror member index and preset sampling configuration rules to obtain the sampled traffic, including:

[0099] Obtain the total number of target traffic segments for all services; sample the target traffic based on the number of mirror members pointed to by the mirror member index, the total number of target traffic segments, and the preset sampling configuration rules to obtain the sampled traffic.

[0100] The sampled traffic is mirrored traffic, and the mirror group members are set in the mirror group.

[0101] Optionally, the programmable switch obtains the total number of target traffic segments for all services, and determines the number of target traffic segments to be collected and the collection order for each mirror member based on the number of mirror members pointed to by the mirror member index, the total number of target traffic segments, and the preset sampling configuration rules. The programmable switch then performs load balancing on the traffic collected by each mirror member to obtain the traffic sampled by each mirror member.

[0102] In this embodiment, the target traffic is sampled by the number of mirror members, the total number of target traffic segments, and preset sampling configuration rules to perform load balancing on the traffic sampled by each mirror member, reduce the risk of load imbalance, and ensure that the mirror members can work normally.

[0103] In an exemplary embodiment, the steps of the above embodiments to sample the target traffic based on the number of mirror members pointed to by the mirror member index, the total number of target traffic components, and preset sampling configuration rules to obtain the sampled traffic include:

[0104] Based on the total number of target traffic segments and the number of mirror members, determine the number of cyclic sampling times N; according to the preset sampling configuration rules, call each mirror member and sequentially perform N-1 cyclic samplings on each target traffic segment, and the sampling order of the mirror members in the Nth sampling is in reverse order.

[0105] The first mirror member samples the first and second target traffic samples by default.

[0106] The number of cyclic sampling N can be the number of iterations in which all mirror members perform one target traffic sampling in turn. For example, each mirror member performs one target traffic sampling in turn, which is one cycle. However, usually, the first mirror group member will sample the first and second traffic in the first cycle sampling, while the remaining mirror group members will only sample one traffic each time. Therefore, there will be an imbalance in traffic load among the members.

[0107] Optionally, the ratio between the total number of target traffic segments and the number of mirror members is used to determine the number of cyclic sampling iterations N. If the division between the total number of target traffic segments and the number of mirror members is not even and leaves a remainder, then 1 is added to the quotient, and this remainder is used as the number of cyclic sampling iterations. According to the preset sampling rules, each mirror member is called sequentially to perform N-1 iterations on each traffic segment. During the Nth sampling iteration, the sampling order of the mirror members is reversed; that is, during the Nth sampling iteration, the last mirror member is called first for sampling, followed by the second-to-last mirror member, until all target traffic segments have been sampled. Table 1 provides an example of cyclic sampling using mirror members.

[0108] Table 1

[0109]

[0110] Where LAG MEMBERS is the number of mirror members, INDEX is the index of each target traffic, and the table shows which mirror member each target traffic is assigned to. For example, when the number of mirror members is 2, there are 16 target traffic. The 0th and 1st traffic are assigned to the 1st member, the 2nd traffic is assigned to the 2nd member, the 3rd traffic is assigned to the 1st member, the 4th traffic is assigned to the 2nd member, and so on, until the last cycle, when the allocation order is reversed and the 15th traffic is assigned to the 2nd member.

[0111] In one exemplary embodiment, such as Figure 4 As shown, a flow sampling system is provided, including:

[0112] The system includes optical splitters, programmable switches, and analysis servers, as well as BRs (Bandwidth Routers), which communicate with each other via wavelength division multiplexing (WDM).

[0113] The splitter is located in the service center channel and connected to the programmable switch. It is used to copy the initial traffic of the service center channel and send the initial traffic to the programmable switch. The programmable switch is connected to the analysis server and is used to execute the steps of the traffic sampling method described in any of the above embodiments. The analysis server is used to receive the sampled traffic sent by the programmable switch and analyze and process the sampled traffic.

[0114] Optionally, the programmable switch uses the TD4 (Trident4) programmable chip. Based on traditional switching chips, the TD4 chip integrates: fixed-function components, where the fixed functional flow in the pipeline is uneditable, similar to the fixed functional components of traditional chips, divided into inlet and outlet; flexible switching logic, which makes decisions based on flow switching and is the main manifestation of programmable logic, also divided into inlet and outlet, with chip functions implemented in both inlet and outlet directions; and a bus structure, capable of defining metadata transmitted across stages and pipelines, divided into three types of data transmission: inlet, memory management unit, and outlet. Packet processing in the TD4 chip utilizes a flexible bus architecture. The flexible bus carries packet data and metadata derived in the packet processing pipeline, and context is passed between pipelines through different bus structures via the memory management unit. Furthermore, NPL (Network Programming Language) is a high-level network programmable language unique to the TD4 chip, transmitting data via the bus and performing programmable logic decisions within the flexible switching logic components.

[0115] In one exemplary embodiment, such as Figure 5 As shown, a flexible sampling method based on a programmable switching chip is provided, including:

[0116] Step 1: In the programmable switch, add the required ACL actions, including: VLAN tagging (dual LAN tagging), load balancing for mirror members of the mirror group, sampling, packet truncation, etc. These fields are placed in the data bus and assigned to the corresponding information in ing_data_bus (data bus) and ing_cmd_bus (control bus) for subsequent data packet transmission.

[0117] For example, flow classification is performed based on Port (port number), Protocol, and TCP flag (Transmission Control Protocol Flag) to achieve traffic classification for cloud services, and a dual-layer VLAN tagging is added to distinguish different cloud services. Sampling supports 252 (Samplerentry) sampling rate configuration entries, with a sampling rate range of (2^28):1 to 1:1. Truncation supports packet truncation with configurable length, minimum 64 bytes. Load balancing is implemented using LAG load balancing, simulated by chip mirror groups.

[0118] First, the packet enters the INGRESS (inbound) phase: After passing through the PARSING (pre-parsing), L2 (Layer 2 forwarding), and L3 (Layer 3 forwarding) logic, it directly enters the INGRESS_FP_PROCESS IFP (ACL processing) unit. The extracted valid information from the packet is matched against the ENTRY (ACL entry) content stored in the SLICE-GREOUP (ACL storage unit), including the packet's IP address, protocol number, and port number. If a match is found, an action is performed. The ACL action adds ifp_sampling_lag_id (aggregate index), outer_vlan_tag (outer VLAN), inner_vlan_lag (inner VLAN), and pkt_truncate_ctrl (packet truncation mark) action information. Then, the IFP FSL editable module's ACL programmable logic, by judging the aggregate index carried by the bus and the bus's packet truncation mark, transmits the bus's mirrored member index bus data for subsequent use by the MIRROR component logic.

[0119] Step 2: The data is then edited and processed in subsequent MIRROR and L2 (Layer 2 forwarding) processes.

[0120] For example, in the last editable module of the IPOSTFSL (inbound direction) editable module, the tpid (Tag Protocol Identifier) ​​is used for bus data editing and transmission. The mirror sampling module simulates the load output using the aggregated index MIRROR_SESSION (mirror group) and configures sampling using MIRROR_SFLOW.SAMPLE_ING_FLEX_RATE (sampling).

[0121] Step 3: Simulate LAG load using MIRROR_SEESION, and use algorithms to achieve load balancing as much as possible.

[0122] For example, in this component, MIRROR_SEESION supports 15 (1-15) indices, and the packet generates 16 (0-15) random numbers. Random numbers 0 and 1 are assigned to index 1, random number 2 to index 2, and so on. When a packet passes through, the random number generated is 0 or 1, which will cause index 1 to account for twice the proportion of other mapped indices. The mapped index 1 occupies 2 parts of traffic, which leads to poor traffic uniformity. This paper proposes a new algorithm that uses the total number of random index values ​​% aggregation member number = loop count. In the last loop, the index values ​​are reversed to fill the MIRROR_SEESION group in turn, so as to achieve load balancing of MIRROR_SEESION simulated aggregation as much as possible. For example, two LAGMEMBERs (aggregate members) are filled into 15 MIRROR_SEESION index groups. The output ratio of the two MEMBERs is 7 (index1, 3, 5, 7, 9, 11, 13): 8 (2, 4, 6, 8, 10, 12, 14, 15). Index1 occupies two portions of traffic, and the others each occupy one portion, thus achieving load balancing. Then, the packet encapsulation and editing are performed in the EGRESS (outbound) stage.

[0123] In addition, the WRAP__FSL2_FUNCTIONS_PKT_TRUNCATE (packet truncation module) transmits the outer_vlan_tag (outer LAN tag), and the EDIT_L2_FORWARDING_HEADERS (Layer 2 module) edits the packet header redo. Through this implementation, massive amounts of data from different types of cloud services can be optically sampled and distributed to different servers, and VLAN information can be used for differentiation and labeling, facilitating backend server processing. This method can achieve all the above functions with a single click, greatly simplifying the deployment process, extracting effective packet payloads, and reducing costs.

[0124] In this embodiment, different types of service packets are tagged (VLAN_Tag) by matching packet IP or service type, enabling fine-grained flow classification and facilitating subsequent server offloading. Packet truncation capabilities are used to extract valid packet header fields, reducing server analysis and processing overhead. A new load balancing algorithm mechanism is proposed, using MIRROR groups to simulate LAG (Link Aggregation Group) load aggregation, ensuring full load balancing of traffic. An effective sampling rate is configured to reduce effective traffic throughput and analyze effective information. The above methods leverage the programmable technology of the Broadcom TD4 chip, allowing direct modification of the corresponding logic in the chip pipeline via NPL language editing to guide matching and forwarding. This implementation enables flexible sampling to distinguish different service types while achieving full load balancing forwarding.

[0125] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0126] Based on the same inventive concept, this application also provides a flow sampling device for implementing the flow sampling method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more flow sampling device embodiments provided below can be found in the limitations of the flow sampling method described above, and will not be repeated here.

[0127] In one exemplary embodiment, such as Figure 6 As shown, a traffic sampling device 600 is provided, including: a traffic acquisition module 602, a traffic processing module 604, a mirroring module 606, and a traffic sampling module 608, wherein:

[0128] The traffic acquisition module 602 is used to acquire the initial traffic of the business center channel; the initial traffic includes traffic from various services.

[0129] The traffic processing module 604 is used to perform flow classification, tagging and filtering on the initial traffic in sequence to obtain the aggregate index and target traffic for each service. The aggregate index represents the correspondence between the target traffic and each service, and each aggregate index points to multiple members of the target traffic.

[0130] The mirror mapping module 606 is used to map according to the aggregate index of each service to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling target traffic.

[0131] The traffic sampling module 608 is used to sample the target traffic based on each mirror member index and preset sampling configuration rules, obtain the sampled traffic, and send the sampled traffic to the traffic analysis server.

[0132] Furthermore, in one embodiment, the traffic processing module 604 is also used to perform flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic of each service; to perform tagging processing on the initial traffic of each service to obtain the tagged traffic and aggregate index of each service; and to perform filtering processing on the tagged traffic to obtain the target traffic of each service.

[0133] Furthermore, in one embodiment, the traffic processing module 604 is also configured to filter the marked traffic based on the dual LAN tag and packet truncation tag of the marked traffic to obtain the target traffic for each service.

[0134] Furthermore, in one embodiment, the traffic processing module 604 is also used to obtain the pre-set message configuration corresponding to each service; perform message parsing processing on the initial traffic to obtain the parsed traffic; and perform flow classification processing on the parsed traffic according to the preset matching rules and the preset message configuration for each service to obtain the initial traffic for each service.

[0135] Furthermore, in one embodiment, the traffic sampling module 608 is also used to obtain the total number of target traffic segments for all services; and to sample the target traffic according to the number of mirror members pointed to by the mirror member index, the total number of target traffic segments, and the preset sampling configuration rules to obtain the sampled traffic.

[0136] Furthermore, in one embodiment, the traffic sampling module 608 is also used to determine the number of cyclic sampling times N based on the total number of target traffic components and the number of mirror members; according to the preset sampling configuration rules, each mirror member is called to perform sequential cyclic sampling N-1 times on each target traffic component, and the sampling order of the mirror members in the Nth sampling is in reverse order; wherein, the first mirror member's first sampling defaults to sampling the first target traffic component and the second target traffic component.

[0137] Each module in the aforementioned flow sampling device 600 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0138] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as initial traffic, target traffic, and aggregated indexes. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a traffic sampling method.

[0139] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A flow rate sampling method, characterized in that, The method includes: Obtain the initial traffic of the business center channel; the initial traffic includes traffic from various services; The initial traffic is sequentially processed by flow classification, tagging, and filtering to obtain an aggregate index and target traffic for each of the services. The aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic. The aggregate index of each service is mapped to obtain the mirror member index of each service; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic; Obtain the total number of target traffic segments for all the services; determine the number of cyclic sampling times N based on the total number of target traffic segments and the number of mirror members; according to the preset sampling configuration rules, call each mirror member and sequentially perform N-1 cyclic samplings on each target traffic segment, with the sampling order of the mirror member in the Nth sampling being in reverse order, and send the sampled traffic to the traffic analysis server; wherein, the first mirror member's first sampling defaults to sampling the first and second target traffic segments.

2. The method according to claim 1, characterized in that, The process of sequentially performing flow classification, tagging, and filtering on the initial traffic to obtain the aggregated index and target traffic for each service includes: The initial traffic is classified according to a preset matching rule to obtain the initial traffic for each service. The initial traffic for each of the aforementioned services is tagged to obtain the tagged traffic and aggregated index for each of the aforementioned services; The marked traffic is filtered to obtain the target traffic for each of the services.

3. The method according to claim 2, characterized in that, The step of filtering the marked traffic to obtain the target traffic for each service includes: Based on the dual LAN tag and packet truncation tag of the tagged traffic, the tagged traffic is filtered to obtain the target traffic for each of the services.

4. The method according to claim 2, characterized in that, The step of performing flow classification processing on the initial traffic according to preset matching rules to obtain the initial traffic for each of the services includes: The initial traffic is parsed to obtain the parsed traffic; According to the preset matching rules and the preset message configuration for each service, the parsed traffic is subjected to flow classification processing to obtain the initial traffic for each service.

5. A flow sampling system, characterized in that, The system includes: a beam splitter, a programmable switch, and an analysis server; wherein... The splitter is located in the service center channel and connected to the programmable switch. It is used to copy the initial traffic of the service center channel and send the initial traffic to the programmable switch. The programmable switch is connected to the analysis server and is used to execute the steps of the method according to any one of claims 1 to 4; The analysis server is used to receive the sampled traffic sent by the programmable switch and to analyze and process the sampled traffic.

6. A flow sampling device, characterized in that, The device includes: The traffic acquisition module is used to acquire the initial traffic of the business center channel; the initial traffic includes traffic from various services. The traffic processing module is used to perform flow classification, tagging, and filtering on the initial traffic in sequence to obtain an aggregate index and target traffic for each of the services; the aggregate index represents the correspondence between the target traffic and each of the services, and each aggregate index points to multiple members that carry the target traffic; The mirror mapping module is used to map according to the aggregate index of each of the services to obtain the mirror member index of each of the services; the mirror member index corresponds one-to-one with the aggregate index, and each mirror member index points to multiple mirror members used for sampling the target traffic; The traffic sampling module is used to obtain the total number of target traffic segments for all the services; determine the number of cyclic sampling times N based on the total number of target traffic segments and the number of mirror members; according to the preset sampling configuration rules, call each mirror member to perform sequential cyclic sampling N-1 times on each target traffic segment, and the sampling order of the mirror member in the Nth sampling is in reverse order to obtain the sampled traffic, and send the sampled traffic to the traffic analysis server; wherein, the first mirror member's first sampling defaults to sampling the first and second target traffic segments.

7. The apparatus according to claim 6, characterized in that, The traffic processing module is further configured to perform flow classification processing on the initial traffic according to preset matching rules to obtain the initial traffic for each service; to perform tagging processing on the initial traffic for each service to obtain the tagged traffic and aggregate index for each service; and to perform filtering processing on the tagged traffic to obtain the target traffic for each service.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

Images