Traffic sampling method, system and device, computer equipment and readable storage medium

Abstract

The invention relates to a traffic sampling method, system and device, computer equipment and a readable storage medium, and relates to the technical field of data communication. The method comprises the following steps: acquiring initial flow of a service center channel; the initial traffic comprises traffic of multiple services; performing flow classification processing, marking processing and filtering processing on the initial flow in sequence to obtain an aggregation index and a target flow of each service; the aggregation index represents a corresponding relationship between the target traffic and each service, and each aggregation index points to a plurality of members loading the target traffic; mapping according to the aggregation index of each service to obtain a mirror image member index of each service; the mirror image member indexes are in one-to-one correspondence with the aggregation indexes, and each mirror image member index points to a plurality of mirror image members used for sampling the target traffic; and sampling the target traffic based on each mirror image member index and a preset sampling configuration rule to obtain sampled traffic, and sending the sampled traffic to a traffic analysis server. By adopting the method, the efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of data communication technology, and in particular to a flow sampling method, system, device, computer equipment and readable storage medium. Background Art

[0002] At present, with the rapid development of cloud services, the complexity and scale of data center networks are also growing exponentially, which brings great challenges to the analysis of traffic data and the location of hidden faults. In order to obtain effective network information, traffic sampling methods are used to collect and analyze the traffic of data center networks.

[0003] In the related art, the traffic is sampled by using the traffic splitting sampling method. On the main network link, a splitter is installed to "copy" the traffic to the analysis server, and the traffic is analyzed by a custom algorithm to obtain information that is helpful for operation and optimization. However, the splitter will completely "copy" all the traffic to the analysis server, and it takes a long time to obtain effective information. Therefore, the traffic collection method of the related art has the problem of low efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a method, system, device, computer equipment and readable storage medium that can improve flow sampling in response to the above technical problems.

[0005] In a first aspect, the present application provides a flow sampling method, comprising:

[0006] Obtaining the initial flow of the service center channel; the initial flow includes the flow of multiple services;

[0007] The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents a corresponding relationship between the target traffic and each of the services, and each aggregation index points to multiple members that carry the target traffic;

[0008] Mapping is performed according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic;

[0009] The target traffic is sampled based on each mirror member index and a preset sampling configuration rule to obtain sampled traffic, and the sampled traffic is sent to a traffic analysis server.

[0010] In one embodiment, the initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregate index and target traffic of each service, including:

[0011] Performing flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic of each service;

[0012] Marking the initial traffic of each of the services to obtain the marked traffic and aggregation index of each of the services;

[0013] The marked traffic is filtered to obtain the target traffic of each service.

[0014] In one embodiment, filtering the marked traffic to obtain target traffic for each service includes:

[0015] According to the double LAN mark and the message truncation mark of the marked traffic, the marked traffic is filtered to obtain the target traffic of each service.

[0016] In one of the embodiments, the performing flow classification processing on the initial traffic according to the preset matching rule to obtain the initial traffic of each service includes:

[0017] Obtaining the pre-set message configuration corresponding to each of the services;

[0018] Performing message parsing on the initial traffic to obtain parsed traffic;

[0019] According to the preset matching rule and the preset message configuration for each of the services, the parsed traffic is subjected to flow classification processing to obtain the initial traffic of each of the services.

[0020] In one of the embodiments, sampling the target traffic based on each of the mirror member indexes and a preset sampling configuration rule to obtain the sampled traffic includes:

[0021] Obtaining the total number of target traffic shares of the target traffic of all the services;

[0022] The target traffic is sampled according to the number of the mirror members pointed to by the mirror member index, the total number of target traffic, and the preset sampling configuration rule to obtain the sampled traffic.

[0023] In one of the embodiments, sampling the target traffic according to the number of the mirror members pointed to by the mirror member index, the total number of the target traffic, and the preset sampling configuration rule to obtain the sampled traffic includes:

[0024] Determine the number of cyclic sampling times N according to the total number of target traffic and the number of mirror members;

[0025] According to the preset sampling configuration rule, each mirror member is called to perform N-1 sequential cyclic sampling on each portion of the target traffic in turn, and the sampling order of the mirror member for the Nth sampling is in reverse order; wherein, the first mirror member samples the first portion of the target traffic and the second portion of the target traffic by default for the first sampling.

[0026] In a second aspect, the present application also provides a flow sampling system, including: an optical splitter, a programmable switch and an analysis server; wherein,

[0027] The optical splitter is arranged in the service center channel and connected to the programmable switch, and is used to copy the initial traffic of the service center channel and send the initial traffic to the programmable switch;

[0028] The programmable switch is connected to the analysis server and is used to perform the steps of any one of the methods in the above embodiments;

[0029] The analysis server is used to receive the sampled traffic sent by the programmable switch and analyze and process the sampled traffic.

[0030] In a third aspect, the present application also provides a flow sampling device, comprising:

[0031] A traffic acquisition module, used to acquire the initial traffic of the service center channel; the initial traffic includes traffic of multiple services;

[0032] A traffic processing module, used to perform flow classification processing, labeling processing and filtering processing on the initial traffic in sequence, to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents the corresponding relationship between the target traffic and each of the services, and each of the aggregation indexes points to multiple members that carry the target traffic;

[0033] A mirror mapping module, used for mapping according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic;

[0034] The traffic sampling module is used to sample the target traffic based on each mirror member index and a preset sampling configuration rule to obtain the sampled traffic, and send the sampled traffic to a traffic analysis server.

[0035] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Obtaining the initial flow of the service center channel; the initial flow includes the flow of multiple services;

[0037] The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents a corresponding relationship between the target traffic and each of the services, and each aggregation index points to multiple members that carry the target traffic;

[0038] Mapping is performed according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic;

[0039] The target traffic is sampled based on each mirror member index and a preset sampling configuration rule to obtain sampled traffic, and the sampled traffic is sent to a traffic analysis server.

[0040] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0041] Obtaining the initial flow of the service center channel; the initial flow includes the flow of multiple services;

[0042] The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents a corresponding relationship between the target traffic and each of the services, and each aggregation index points to multiple members that carry the target traffic;

[0043] Mapping is performed according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic;

[0044] The target traffic is sampled based on each mirror member index and a preset sampling configuration rule to obtain sampled traffic, and the sampled traffic is sent to a traffic analysis server.

[0045] In a sixth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0046] Obtaining the initial flow of the service center channel; the initial flow includes the flow of multiple services;

[0047] The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents a corresponding relationship between the target traffic and each of the services, and each aggregation index points to multiple members that carry the target traffic;

[0048] Mapping is performed according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic;

[0049] The target traffic is sampled based on each mirror member index and a preset sampling configuration rule to obtain sampled traffic, and the sampled traffic is sent to a traffic analysis server.

[0050] The above-mentioned traffic sampling method, system, device, computer equipment and readable storage medium obtain the initial traffic of the service center channel, and the initial traffic includes the traffic of multiple services as the basis for subsequent traffic sampling; the initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain the aggregation index and target traffic of each service; wherein the aggregation index represents the corresponding relationship between the target traffic and each service, and each aggregation index points to members of multiple load target traffic; the initial traffic is subjected to flow classification processing, labeling processing and filtering processing, which can realize fine-grained flow classification of traffic of different services, facilitate subsequent analysis of server unloading processing, extract the target traffic, and reduce The sampled traffic load increases the sampling load while also reducing the processing burden of the subsequent analysis server; further, mapping is performed according to the aggregation index of each business to obtain the mirror member index of each business; the mirror member index corresponds to the aggregation index one by one, and each mirror member index points to multiple mirror members for sampling target traffic; based on each mirror member index and the preset sampling configuration rules, the target traffic is sampled to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server. The mirror group does not need to sample all the traffic of the business center channel, but only needs to sample the target traffic of each business obtained after a series of processing, thereby improving the sampling efficiency of the traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0052] FIG1 is a diagram showing an application environment of a flow sampling method according to an embodiment;

[0053] FIG2 is a schematic flow chart of a flow sampling method in one embodiment;

[0054] FIG3 is a schematic flow chart of a target flow determination step in one embodiment;

[0055] FIG4 is a schematic diagram of the structure of a flow sampling system in one embodiment;

[0056] FIG5 is a schematic diagram of a flow chart of a flexible sampling method based on a programmable switching chip in one embodiment;

[0057] FIG6 is a structural block diagram of a flow sampling device in one embodiment;

[0058] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The traffic sampling method provided in the embodiment of the present application can be applied to the application environment shown in FIG1 , and the application scenario includes: a service center channel, a splitter 102, a programmable switch 104, and an analysis server 106. The splitter 102 is arranged in the service center channel, the splitter 102 is connected to the programmable switch 104, and the programmable switch 104 is connected to the analysis server in communication. The programmable switch can be a programmable chip, and the analysis server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The programmable switch 104 obtains the initial traffic of the service center channel through the splitter 102, and the initial traffic includes the traffic of multiple services, and performs flow classification processing, labeling processing, and filtering processing on the initial traffic in turn to obtain the aggregation index and target traffic of each service, wherein the aggregation index represents the corresponding relationship between the target traffic and each service, and each aggregation index points to a member of multiple load target traffic. Further, the programmable switch 104 performs mapping according to the aggregation index of each service to obtain the mirror member index of each service, wherein the mirror member index corresponds to the aggregation index one by one, and each mirror member index points to multiple mirror members for sampling the target traffic. Then the programmable switch 104 samples the target traffic based on each mirror member index and the preset sampling configuration rule to obtain the sampled traffic, and sends the sampled traffic to the traffic analysis server 106.

[0061] In an exemplary embodiment, as shown in FIG2 , a traffic sampling method is provided, which is described by taking the method applied to the programmable switch 104 in FIG1 as an example, and includes the following steps S202 to S208 . Among them:

[0062] Step S202, obtaining the initial flow of the service center channel.

[0063] The initial traffic includes traffic of various services.

[0064] Among them, the business center channel can be a data transmission channel for the enterprise system to process various businesses, and the traffic related to each business will be transmitted through this channel.

[0065] Optionally, the programmable switch obtains the initial traffic transmitted in the service center channel through an optical splitter set in the service center channel. The optical splitter is a passive optical device that can redistribute the power intensity of the optical signal according to the required proportion. After the programmable switch receives the initial traffic, it goes through the original forwarding process, the layer 2 forwarding process, and the layer 3 forwarding process, and then enters the target traffic extraction process.

[0066] Step S204, the initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and target traffic of each service.

[0067] The aggregation index represents the corresponding relationship between the target flow and each service, each aggregation index points to members of multiple load target flows, and the aggregation index can be an identifier.

[0068] Among them, flow classification processing can refer to classification processing of initial traffic; labeling processing can refer to the operation of marking initial traffic with labels; filtering processing can refer to filtering out target information that does not need to be sampled in the initial traffic.

[0069] The target flow may be the flow to be analyzed determined according to actual flow analysis requirements.

[0070] Optionally, the programmable switch performs flow classification, labeling and filtering on the initial traffic in sequence to obtain an aggregation index and target traffic for each service. This series of actions can be performed in the ACL (Access Control List) actions of the programmable switch. The ACL is used to define which traffic can be mirrored or monitored. The ACL can accurately control which traffic is copied and sent to the mirror port. Specifically, the initial traffic is subjected to flow classification to obtain the initial traffic corresponding to each service, the initial traffic corresponding to each service is labeled with a corresponding label to obtain the marked initial traffic for each service, the marked initial traffic for each service is filtered to obtain the target traffic for each service, and the aggregation index corresponding to each service is determined based on the label corresponding to the target traffic and the member that loads the target traffic.

[0071] Step S206, mapping is performed according to the aggregation index of each service to obtain the mirror member index of each service.

[0072] The mirror member index corresponds to the aggregation index one by one, and each mirror member index points to multiple mirror members for sampling target traffic.

[0073] Optionally, the programmable switch maps the mirror port according to the aggregation index of each service and the members included in the aggregation index, simulates the process of building the aggregation index, and obtains the mirror member index of each service. At this time, the mirror member index only contains the mirror member index identifier and the associated mirror member, and the traffic load of the mirror member is empty at this time.

[0074] Step S206: sampling the target traffic based on each mirror member index and the preset sampling configuration rule to obtain the sampled traffic, and sending the sampled traffic to the traffic analysis server.

[0075] The preset sampling configuration rule may be the pre-configured collection quantity and collection sequence of the mirror member for the target traffic.

[0076] Optionally, the programmable switch instructs each mirror member to sample the target traffic separately based on the multiple mirror members included in each mirror member index and the preset sampling configuration rules until the target traffic of all services is sampled by the mirror members to obtain the sampled traffic. The programmable switch performs message encapsulation and editing on the sampled traffic, and then sends the encapsulated sampled traffic to the traffic analysis server to analyze the sampled traffic and monitor the traffic of the service center channel.

[0077] In the above traffic sampling method, the initial traffic of the business center channel is obtained, and the initial traffic includes the traffic of multiple businesses, which serves as the basis for subsequent traffic sampling; the initial traffic is successively subjected to flow classification processing, labeling processing and filtering processing to obtain the aggregation index and target traffic of each business; wherein the aggregation index represents the corresponding relationship between the target traffic and each business, and each aggregation index points to members of multiple load target traffic; the initial traffic is subjected to flow classification processing, labeling processing and filtering processing, so as to realize fine-grained flow classification of traffic of different businesses, facilitate subsequent analysis of server unloading processing, extract the target traffic, reduce the sampled traffic load, and improve While increasing the sampling load, it also reduces the processing burden of the subsequent analysis server; further, mapping is performed according to the aggregation index of each business to obtain the mirror member index of each business; the mirror member index corresponds to the aggregation index one by one, and each mirror member index points to multiple mirror members for sampling target traffic; based on each mirror member index and the preset sampling configuration rules, the target traffic is sampled to obtain the sampled traffic, and the sampled traffic is sent to the traffic analysis server. The mirror group does not need to sample all the traffic of the business center channel, but only needs to sample the target traffic of each business obtained after a series of processing, thereby improving the sampling efficiency of the traffic.

[0078] In an exemplary embodiment, as shown in FIG3 , step S204 performs flow classification processing, labeling processing and filtering processing on the initial traffic in order to obtain an aggregate index and target traffic of each service, including steps 302 to 306. Among them:

[0079] Step 302: Perform flow classification processing on the initial flow according to the preset matching rule to obtain the initial flow of each service.

[0080] Among them, the preset matching rules can be the classification criteria for each business, such as matching the traffic characteristics corresponding to each business with the traffic characteristics of the initial traffic. The initial traffic that can successfully match the traffic characteristics corresponding to any business belongs to the initial traffic of the business.

[0081] Optionally, the programmable switch performs flow classification processing on the initial traffic according to a preset matching rule, divides the initial traffic into service categories that meet the preset matching rule, and obtains the initial traffic of each service.

[0082] Step 304: Mark the initial traffic of each service to obtain the marked traffic and aggregation index of each service.

[0083] Optionally, the programmable switch labels the initial traffic of each service and can determine the part of the initial traffic message that needs to be extracted based on traffic analysis requirements, and add preset marks to the starting point and end point of the part. In addition, based on the load member identifier of the initial traffic corresponding to each service, an aggregation index for each service is constructed, and the aggregation index points to multiple members that load the initial traffic of the service.

[0084] Step 306: filter the marked traffic to obtain the target traffic of each service.

[0085] Optionally, the programmable switch determines the starting point and end point of the target traffic to be extracted based on the mark in the marked traffic, filters the marked traffic according to the starting point and end point, extracts the target traffic, deletes the remaining interfering traffic, and obtains the target traffic for each service.

[0086] In this embodiment, through the preset matching rules, the programmable switch can accurately identify and classify traffic of different business types, improve the utilization efficiency of network resources, and ensure that different business traffic is properly processed. By marking the initial traffic, the traffic of each business can be quickly identified and processed in the subsequent traffic management, avoiding forwarding all traffic of the business center to the mirror port for sampling, and improving the efficiency of traffic sampling.

[0087] In an exemplary embodiment, step S306 filters the marked traffic to obtain target traffic for each service, including:

[0088] According to the double LAN mark and the message truncation mark of the marked traffic, the marked traffic is filtered to obtain the target traffic of each service.

[0089] Among them, the double LAN tag can be an inner LAN tag (inner_vlan_tag) and an outer LAN tag (outer_vlan_tag). The VLAN Tag is a field added to the Ethernet frame, which is mainly used to identify which VLAN the frame belongs to. In this embodiment, adding a double LAN tag to the message of the initial traffic can identify the type of service to which the initial traffic belongs.

[0090] The message truncation mark may be the cutoff point of the target flow that needs to be extracted from the initial flow, and the position of the message truncation mark is set according to the actual flow collection requirements.

[0091] Optionally, the programmable switch determines the starting point and the ending point of the target traffic in the initial traffic according to the dual LAN tag and the message truncation tag of the marked traffic, respectively, and filters the marked traffic to obtain the target traffic for each service.

[0092] In this embodiment, the labels set for marking the initial traffic and the specific implementation method of filtering to obtain the target traffic are further explained. According to the dual LAN marking and the message truncation marking, the target traffic can be extracted more accurately, further improving the sampling efficiency of the traffic.

[0093] In an exemplary embodiment, step S302 performs flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic of each service, including:

[0094] The initial traffic is parsed to obtain the parsed traffic; according to the preset matching rules and the preset message configuration for each service, the parsed traffic is classified to obtain the initial traffic of each service.

[0095] The message configuration may include a message IP (Internet Protocol) address, a protocol number, a port number, etc. of a traffic message corresponding to each service.

[0096] Optionally, the programmable switch obtains a preset message configuration corresponding to each pre-set service, obtains the initial traffic of the service center channel through a splitter, and then performs message parsing on the initial traffic to obtain the parsed traffic, wherein the parsed traffic includes the message configuration of the initial traffic, and performs flow classification processing on the parsed traffic according to the preset matching rules and the message configuration preset for each service, and uses the parsed traffic that can match the message configuration of any service as the initial traffic of the service type, thereby obtaining the initial traffic of each service.

[0097] In this embodiment, by utilizing the parsed traffic and preset message configuration, different types of business traffic can be clearly identified and classified to ensure that each business traffic is properly processed and monitored. Multiple businesses in the network can be independently supported through effective classification and processing of traffic, which not only prevents mutual interference between different businesses, but also improves security through VLAN isolation.

[0098] In an exemplary embodiment, step S208 samples the target traffic based on each mirror member index and a preset sampling configuration rule to obtain the sampled traffic, including:

[0099] The total number of target traffic of all services is obtained; the target traffic is sampled according to the number of mirror members pointed to by the mirror member index, the total number of target traffic and the preset sampling configuration rule to obtain the sampled traffic.

[0100] The sampled traffic is mirrored traffic, and the mirror group members are set in the mirror group.

[0101] Optionally, the programmable switch obtains the total number of target traffic flows of all services, determines the number of target traffic flows and the collection order that each mirror member needs to collect according to the number of mirror members pointed to by the mirror member index, the total number of target traffic flows and preset sampling configuration rules, load balances the traffic collected by each mirror member, and obtains the traffic after sampling by each mirror member.

[0102] In this embodiment, the target traffic is sampled according to the number of mirror members, the total number of target traffic, and the preset sampling configuration rules, so as to load balance the traffic sampled by each mirror member, reduce the risk of load imbalance, and ensure that the mirror members can work normally.

[0103] In an exemplary embodiment, the steps of the above embodiment sample the target traffic according to the number of mirror members pointed to by the mirror member index, the total number of target traffic and a preset sampling configuration rule to obtain the sampled traffic, including:

[0104] According to the total number of target traffic and the number of mirror members, determine the number of cyclic sampling times N; according to the preset sampling configuration rule, call each mirror member to perform N-1 sequential cyclic sampling on each target traffic in turn, and the sampling order of the mirror member for the Nth sampling is in reverse order.

[0105] The first mirror member samples the first target traffic and the second target traffic by default for the first sampling.

[0106] The number of cyclic sampling times N may be the number of iterations in which all mirror members perform target traffic sampling in turn. For example, each mirror member performs a target traffic sampling in turn as one cycle. However, usually, the first mirror group member samples the first traffic and the second traffic in the first cyclic sampling, while the remaining mirror group members only sample one traffic each time. Therefore, there may be a traffic load imbalance among the members.

[0107] Optionally, according to the ratio between the total number of target traffic and the number of mirror members, the ratio is determined as the number of cyclic sampling times N. If the total number of target traffic and the number of mirror members cannot be divided evenly and there is a remainder, the quotient is added with 1 as the number of cyclic sampling times. According to the preset sampling rule, each mirror member is called to perform N-1 sequential cycles on each traffic in turn, and at the Nth sampling, the sampling order of the mirror members is in reverse order, that is, at the Nth sampling, the last mirror member is called first for sampling, and then the second to last mirror member is called for sampling, until all target traffic samples are sampled. As shown in Table 1, an example of cyclic sampling of mirror members is provided.

[0108] Table 1

[0109]

[0110] Among them, LAG MEMBERS is the number of mirror members, INDEX is the index of each target flow, and the table shows the mirror member to which each target flow is assigned for sampling. For example, when the number of mirror members is 2, there are 16 target flows, and the 0th and 1st are assigned to the 1st member, the 2nd is assigned to the 2nd member, the 3rd is assigned to the 1st member, and the 4th is assigned to the 2nd member. This cycle repeats until the last cycle, in which the allocation order is reversed and the 15th is assigned to the 2nd member.

[0111] In an exemplary embodiment, as shown in FIG4 , a flow sampling system is provided, comprising:

[0112] The optical splitter, programmable switch and analysis server also include: BR (Bandwidth Router), and BR routers communicate with each other through wavelength division.

[0113] Among them, the splitter is arranged in the business center channel and connected to the programmable switch, which is used to copy the initial traffic of the business center channel and send the initial traffic to the programmable switch; the programmable switch is connected to the analysis server, which is used to execute the steps of the traffic sampling method described in any one of the above embodiments; the analysis server is used to receive the sampled traffic sent by the programmable switch and analyze and process the sampled traffic.

[0114] Optionally, the programmable switch uses the TD4 (Trident4) programmable chip. Based on the traditional switching chip, the TD4 chip integrates: fixed function components. The fixed function process in the pipeline is not editable, similar to the solidified function components of traditional chips, divided into inlet and outlet; flexible switch logic, which makes decisions based on flow switches, is the main embodiment of programmable logic, and is also divided into inlet and outlet. The chip function can be realized in both inlet and outlet directions; bus structure, which can define metadata transmitted across stages and across pipelines, and is divided into three types of data transmission: inlet, memory management unit, and outlet. Packet processing in the TD4 chip relies on a flexible bus architecture. The flexible bus carries the packet data and metadata derived in the packet processing pipeline, and the context is passed between pipelines through different bus structures through the memory management unit. In addition, the NPL (Network Programming Language) language is a high-level network programming language unique to the TD4 chip. Data reports are transmitted through the bus, and programmable logic decisions are made in the flexible switch logic component.

[0115] In an exemplary embodiment, as shown in FIG5 , a flexible sampling method based on a programmable switching chip is provided, including:

[0116] Step 1: In the programmable switch, add the required ACL actions, including: VLAN tagging (double LAN tagging), load sharing, sampling, message truncation and other fields of the mirror members of the mirror group, and put them into the data bus, assigning the corresponding information to ing_data_bus (data bus) and ing_cmd_bus (control bus) for subsequent data message transmission.

[0117] For example, traffic classification is performed based on Port (port number), Protocol (protocol), TCPflag (Transmission Control Protocol Flag, Transmission Control Protocol Flag), etc. to achieve traffic classification of cloud services, and double-layer VLAN tags are added to distinguish different cloud services. Sampling, supports 252 (Samplerentry) sampling rate configuration table items, and the sampling rate range is (2^28): 1~1:1. Truncation, supports message truncation, and the length is configurable, with a minimum of 64 bytes. Load balancing, LAG load balancing, and chip MIRROR (mirror) group simulation implementation.

[0118] First, enter the INGRESS (inbound direction) stage: the message goes through the PARSING (pre-message parsing), L2 (Layer 2 forwarding), and L3 (Layer 3 forwarding) forwarding logic, and directly enters the INGRESS_FP_PROCESS IFP (ACL processing) processing unit. The effective information extracted from the message is matched with the ENTRY (ACL entry) content stored in the SLICE-GREOUP (ACL storage unit), including the message IP address, protocol number, port number, etc., and the action processing is performed if it is hit. Add ifp_sampling_lag_id (aggregation index), outer_vlan_tag (outer VLAN), inner_vlan_lag (inner VLAN), and pkt_truncate_ctrl (message truncation mark) action information in (ACL action). Then enter the IFP FSL editable module ACL programmable logic to judge the aggregation index carried by the bus and the message truncation mark of the bus, and transmit the bus mirror member index bus data, which will be used by the MIRROR component logic later.

[0119] Step 2: Edit and process the data in subsequent MIRROR and L2 (Layer 2 forwarding).

[0120] For example, the last editable module in the IPOSTFSL (incoming direction) edits and transmits the bus data with tpid (Tag Protocol Identifier). The mirror sampling module simulates the load output of the aggregate index MIRROR_SESSION (mirror group) and configures the sampling with MIRROR_SFLOW.SAMPLE_ING_FLEX_RATE (sampling).

[0121] Step 3: Use MIRROR_SEESION to simulate LAG load and use the algorithm to achieve load balancing as much as possible.

[0122] For example, MIRROR_SEESION in this component supports 15 (1-15) indexes, and the message generates 16 random numbers (0-15). The random numbers 0 and 1 correspond to index 1, and the random number 2 corresponds to index 2, and the latter correspond one to one. When the message passes, the random numbers 0 and 1 generated will cause index 1 to occupy twice the proportion of other mapped indexes. The mapped index 1 occupies 2 shares of the traffic, which leads to poor traffic uniformity. This article proposes a new algorithm, which uses the total number of random index values ​​% number of aggregation members = number of cycles. The index values ​​of the last round of cycles are in reverse order to fill the MIRROR_SEESION group in turn, so as to achieve load balancing of MIRROR_SEESION simulated aggregation as much as possible. For example, 2 LAGMEMBERs (aggregation members) are filled in 15 MIRROR_SEESION index groups, and the output ratio of the 2 MEMBERs is 7 (index1, 3, 5, 7, 9, 11, 13): 8 (2, 4, 6, 8, 10, 12, 14, 15). Index1 accounts for 2 portions of traffic, and the others each account for 1 portion of traffic, so that load balancing can be achieved. Enter the EGRESS (outbound) stage and perform message encapsulation editing.

[0123] In addition, WRAP__FSL2_FUNCTIONS_PKT_TRUNCATE (message truncation module) transmits outer_vlan_tag (outer LAN tag), EDIT_L2_FORWARDING_HEADERS (layer 2 module), and redo edits the message header. Through the above implementation, the massive data of different types of cloud services can be split and sampled to different servers, and distinguished and marked with the help of VLAN information, which is convenient for the back-end server to process. This method can realize all the above functions in one click, greatly simplify the deployment process, extract the effective message load, and reduce costs.

[0124] In this embodiment, by matching the message IP or service type, different types of service messages are tagged (VLAN_Tag), and fine-grained flow classification is performed to facilitate subsequent server unloading processing; through the message truncation capability, the valid message header field is extracted to reduce the server analysis and processing overhead; the MIRROR group is used to simulate the LAG (link aggregation group) aggregation load, and a new load algorithm mechanism is proposed to ensure full load balancing of traffic; the effective sampling rate is configured to reduce the effective traffic throughput and analyze the effective information. The above method uses the Broadcom TD4 chip programmable technology to directly modify the corresponding logic in the chip pipeline through NPL language editing to guide matching and forwarding. Through the above implementation, different service types can be flexibly sampled and distinguished according to different service types, and full load balancing forwarding can be achieved.

[0125] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0126] Based on the same inventive concept, the embodiment of the present application also provides a flow sampling device for implementing the flow sampling method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more flow sampling device embodiments provided below can refer to the limitations on the flow sampling method above, and will not be repeated here.

[0127] In an exemplary embodiment, as shown in FIG6 , a flow sampling device 600 is provided, comprising: a flow acquisition module 602, a flow processing module 604, a mirror mapping module 606 and a flow sampling module 608, wherein:

[0128] The traffic acquisition module 602 is used to acquire the initial traffic of the service center channel; the initial traffic includes the traffic of multiple services.

[0129] The traffic processing module 604 is used to perform flow classification processing, labeling processing and filtering processing on the initial traffic in sequence to obtain the aggregation index and target traffic of each business; the aggregation index represents the correspondence between the target traffic and each business, and each aggregation index points to members of multiple load target traffic.

[0130] The mirror mapping module 606 is used to map according to the aggregation index of each service to obtain the mirror member index of each service; the mirror member index corresponds to the aggregation index one by one, and each mirror member index points to multiple mirror members for sampling target traffic.

[0131] The traffic sampling module 608 is used to sample the target traffic based on each mirror member index and a preset sampling configuration rule, obtain the sampled traffic, and send the sampled traffic to the traffic analysis server.

[0132] Furthermore, in one embodiment, the traffic processing module 604 is also used to perform flow classification processing on the initial traffic according to preset matching rules to obtain the initial traffic of each service; perform labeling processing on the initial traffic of each service to obtain the labeled traffic and aggregation index of each service; perform filtering processing on the labeled traffic to obtain the target traffic of each service.

[0133] Furthermore, in one embodiment, the traffic processing module 604 is further configured to filter the marked traffic according to the dual LAN tag and the message truncation tag of the marked traffic to obtain the target traffic of each service.

[0134] Furthermore, in one embodiment, the traffic processing module 604 is also used to obtain the pre-set message configuration corresponding to each service; perform message parsing on the initial traffic to obtain the parsed traffic; perform flow classification on the parsed traffic according to the preset matching rules and the preset message configuration for each service to obtain the initial traffic of each service.

[0135] Furthermore, in one embodiment, the traffic sampling module 608 is also used to obtain the total number of target traffic for all services; sample the target traffic according to the number of mirror members pointed to by the mirror member index, the total number of target traffic and the preset sampling configuration rules to obtain the sampled traffic.

[0136] Furthermore, in one embodiment, the traffic sampling module 608 is also used to determine the number of cyclic sampling times N according to the total number of target traffic and the number of mirror members; according to the preset sampling configuration rule, each mirror member is called to perform N-1 sequential cyclic sampling on each target traffic in turn, and the sampling order of the mirror member for the Nth sampling is in reverse order; wherein, the first mirror member samples the first target traffic and the second target traffic by default for the first sampling.

[0137] Each module in the above-mentioned flow sampling device 600 can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0138] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as initial traffic, target traffic and aggregate indexes. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a traffic sampling method is implemented.

[0139] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0143] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A flow sampling method, characterized in that: The method comprises: Obtaining the initial flow of the service center channel; the initial flow includes the flow of multiple services; The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents a corresponding relationship between the target traffic and each of the services, and each aggregation index points to multiple members that carry the target traffic; Mapping is performed according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic; The target traffic is sampled based on each mirror member index and a preset sampling configuration rule to obtain sampled traffic, and the sampled traffic is sent to a traffic analysis server.

2. The method according to claim 1, characterized in that The initial traffic is subjected to flow classification processing, labeling processing and filtering processing in sequence to obtain an aggregation index and target traffic of each service, including: Performing flow classification processing on the initial traffic according to a preset matching rule to obtain the initial traffic of each service; Marking the initial traffic of each of the services to obtain the marked traffic and aggregation index of each of the services; The marked traffic is filtered to obtain the target traffic of each service.

3. The method according to claim 2, characterized in that The filtering process of the marked traffic to obtain the target traffic of each service includes: According to the double LAN mark and the message truncation mark of the marked traffic, the marked traffic is filtered to obtain the target traffic of each service.

4. The method according to claim 2, characterized in that: The performing flow classification processing on the initial traffic according to the preset matching rule to obtain the initial traffic of each service includes: Performing message parsing on the initial traffic to obtain parsed traffic; According to the preset matching rule and the preset message configuration for each of the services, the parsed traffic is subjected to flow classification processing to obtain the initial traffic of each of the services.

5. The method according to claim 1, characterized in that The sampling of the target traffic based on each of the mirror member indexes and the preset sampling configuration rule to obtain the sampled traffic includes: Obtaining the total number of target traffic shares of the target traffic of all the services; The target traffic is sampled according to the number of the mirror members pointed to by the mirror member index, the total number of the target traffic, and the preset sampling configuration rule to obtain the sampled traffic.

6. The method according to claim 5, characterized in that The sampling of the target traffic according to the number of the mirror members pointed to by the mirror member index, the total number of the target traffic and the preset sampling configuration rule to obtain the sampled traffic includes: Determine the number of cyclic sampling times N according to the total number of target traffic and the number of mirror members; According to the preset sampling configuration rule, each mirror member is called to perform N-1 sequential cyclic sampling on each portion of the target traffic in turn, and the sampling order of the mirror member for the Nth sampling is in reverse order; wherein, the first mirror member samples the first portion of the target traffic and the second portion of the target traffic by default for the first sampling.

7. A flow sampling system, characterized in that: The system comprises: an optical splitter, a programmable switch and an analysis server; wherein, The optical splitter is arranged in the service center channel and connected to the programmable switch, and is used to copy the initial traffic of the service center channel and send the initial traffic to the programmable switch; The programmable switch is connected to the analysis server and is used to perform the steps of the method according to any one of claims 1 to 6; The analysis server is used to receive the sampled traffic sent by the programmable switch and analyze and process the sampled traffic.

8. A flow sampling device, characterized in that: The device comprises: A traffic acquisition module, used to acquire the initial traffic of the service center channel; the initial traffic includes traffic of multiple services; A traffic processing module, used to perform flow classification processing, labeling processing and filtering processing on the initial traffic in sequence, to obtain an aggregation index and a target traffic of each of the services; the aggregation index represents the corresponding relationship between the target traffic and each of the services, and each of the aggregation indexes points to multiple members that carry the target traffic; A mirror mapping module, used for mapping according to the aggregation index of each of the services to obtain a mirror member index of each of the services; the mirror member index corresponds to the aggregation index one by one, and each of the mirror member indexes points to multiple mirror members for sampling the target traffic; The traffic sampling module is used to sample the target traffic based on each mirror member index and a preset sampling configuration rule to obtain the sampled traffic, and send the sampled traffic to a traffic analysis server.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images