Large flow detection method and system in high-speed network scene

Abstract

The embodiment of the invention provides a large flow detection method and system in a high-speed network scene, and belongs to the technical field of electric digital signal processing. The method comprises the steps that a first bucket array and a second bucket array which are used for recording data flow information are constructed, each of the first bucket array and the second bucket array comprises a plurality of buckets and an independent hash function, and each bucket comprises a plurality of units; acquiring the data stream information; calculating an index of a bucket corresponding to a flow label according to the flow label of the data flow information by adopting hash functions of the first bucket array and the second bucket array; determining a to-be-searched unit set of the flow label according to the index; matching and searching in the to-be-searched unit set according to the stream label, and judging whether a target unit can be matched and searched in the to-be-searched unit set or not; and under the condition that the target unit can be searched in the search unit set in a matched manner, updating the main counter and the auxiliary counter of the target unit.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical digital signal processing, and in particular to a method and system for detecting a large flow in a high-speed network scenario. Background Art

[0002] With the rapid development of Internet technology and the continuous growth of the number of mobile devices, the scale of data flows in high-speed networks has increased exponentially. In high-speed network scenarios, the distribution of data flows is extremely skewed, that is, the vast majority of data flows are small flows of small scale, while a small number of large flows of large scale occupy the vast majority of traffic in high-speed networks, and they are more likely to have a significant impact on network performance. Therefore, in high-speed networks, detecting large flows with online detection scale (i.e., the number of packets contained in the flow) exceeding a certain threshold is crucial for many network applications. These applications include congestion control, traffic engineering, and anomaly detection, and large flow detection can provide key basic data support for them. In practical applications, flows are uniquely identified by flow labels, and a flow consists of all packets carrying the flow label. Flow labels can be flexibly defined according to specific application requirements, such as source IP address, destination IP address, or a combination of any header fields. In the process of network traffic measurement, we need to process each arriving data packet in real time and analyze them quickly to extract useful information. However, the current packet forwarding speed in high-speed networks has reached hundreds of Gbps or even Tbps. In order to keep up with the increasing line speed, we need to use high-speed on-chip storage to complete large flow detection tasks, while also minimizing the online processing of data packets. However, the capacity of high-speed on-chip storage is extremely limited (for example, the SRAM of a central switch is usually less than 10MB), and most of the storage needs to be allocated to core network functions such as packet forwarding. Therefore, how to achieve flexible, fast, and accurate large flow detection with the lowest possible computing and storage overhead has become a hot topic in recent years.

[0003] To address this problem, there have been a large number of studies on online large flow detection based on compact data structures, and many efficient detection methods have been achieved. These technologies usually try to separate large flows from small flows based on the size of the flow. When more flows cannot be stored, they hope to save large flows and expel small flows. However, using only the size of the flow cannot fully explore the difference between large flows and small flows. Although the size of each small flow is small, the number of small flows in high-speed network scenarios is much higher than the number of large flows, and the total traffic size they occupy is relatively large. Moreover, at the beginning of the arrival of the large flow, it has not yet grown to a large enough scale. If there is no space to store the newly arrived flow at this time, the large flow at the beginning of the arrival is easily replaced by a large number of small flows by mistake. At this time, the information of the large flow will be discarded, resulting in a decrease in the accuracy of large flow detection, and the erroneously saved small flows waste storage resources, making it difficult for existing technologies to efficiently store large flows. In addition, frequent flow replacement will result in a large number of memory read and write operations, thereby reducing the processing speed of data packets. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for detecting a large flow in a high-speed network scenario, wherein the method and system can achieve accurate and efficient large flow detection.

[0005] In order to achieve the above object, an embodiment of the present invention provides a method for detecting a large flow in a high-speed network scenario, including:

[0006] Acquiring the data flow information;

[0007] Calculate the index of the bucket corresponding to the flow label according to the flow label of the data flow information by using the hash function of the first bucket array and the second bucket array;

[0008] Determine a set of units to be searched for the flow label according to the index;

[0009] According to the flow label, a match search is performed in the set of units to be searched, and it is determined whether a target unit can be matched and searched in the set of units to be searched;

[0010] In the case that the target unit can be matched and searched in the search unit set, the main counter and the sub-counter of the target unit are updated.

[0011] Optionally, the method further comprises:

[0012] In the case where it is determined that the target unit cannot be matched and searched in the search unit set, determining whether there is an empty unit in the search unit set;

[0013] When it is determined that there are vacant units in the search unit set, one of the vacant units is used as a target unit, and a main counter and a sub-counter of the target unit are updated.

[0014] Optionally, the method further comprises:

[0015] In the case where it is determined that there is no vacant unit in the search unit set, selecting a unit with a minimum value of a main counter in the search unit set as a target unit;

[0016] Determine whether the secondary counter of the target unit is 0;

[0017] In the case where it is determined that the secondary counter of the target unit is not 0, the secondary counter of the selected target unit is decayed by using a preset first probability value;

[0018] Determining whether the secondary counter of the target unit is decayed successfully;

[0019] If it is determined that the secondary counter of the target unit is successfully decayed, the value of the secondary counter of the target unit is reduced by 1, and the flow information corresponding to the current flow label is discarded;

[0020] When it is determined that the secondary counter of the target unit has not been successfully decayed, the flow information corresponding to the current flow label is discarded.

[0021] Optionally, when it is determined that the secondary counter of the target unit is not 0, attenuating the secondary counter of the selected target unit by using a preset first probability value includes:

[0022] The first probability value is calculated according to formula (1):

[0023]

[0024] Among them, p1 is the first probability value, A i [h i (f)][l].fc is the value of the sub-counter.

[0025] Optionally, the method further comprises:

[0026] When it is determined that the secondary counter of the target unit is 0, replacing the current flow label with the selected target unit according to a preset second probability value;

[0027] Determine whether the current flow label is replaced successfully;

[0028] If it is determined that the current flow label is not replaced successfully, discard the flow information corresponding to the flow label;

[0029] If it is determined that the current flow label is replaced successfully, updating the secondary counter of the target unit;

[0030] Determine whether the value of the main counter of the target unit before replacement is greater than a preset main counter threshold;

[0031] When it is determined that the value of the main counter of the target unit before replacement is less than or equal to the main counter threshold, the replaced flow label is discarded.

[0032] Optionally, when it is determined that the secondary counter of the target unit is 0, replacing the current flow label with the selected target unit according to a preset second probability value includes:

[0033] The second probability value is calculated according to formula (2):

[0034]

[0035] Among them, p2 is the second probability value, A i [h i (f)][l].lc is the value of the main counter.

[0036] Optionally, the method further comprises:

[0037] When it is determined that the value of the main counter of the target unit before replacement is greater than the main counter threshold, another hash function corresponding to the target unit is selected, and the replaced flow label is input into the hash function to obtain an index of a bucket corresponding to the replaced flow label;

[0038] Determine a new search match set according to the index of the bucket corresponding to the replaced flow label;

[0039] Determine whether there is an empty cell in the search match set;

[0040] When it is determined that there are vacant units in the search match set, the replaced flow label is stored in one of the vacant units, and the flow label, the primary counter and the secondary counter of the vacant unit are updated.

[0041] Optionally, the method further comprises:

[0042] In the case where it is determined that there is no vacant unit in the search matching set, taking the unit with the smallest value of the main counter in the search matching set as the target unit;

[0043] Determine whether the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label;

[0044] If it is determined that the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label, the replaced flow label is discarded;

[0045] When it is determined that the value of the main counter of the target unit is less than the value of the main counter corresponding to the replaced flow label, the replaced flow label is replaced into the target unit, and the flow label in the target unit before the replacement is used as the new replaced flow label;

[0046] Update replacement times;

[0047] Determining whether the number of replacements is greater than or equal to a preset number threshold;

[0048] If it is determined that the number of replacements is greater than or equal to the number threshold, discarding the replaced flow label;

[0049] When it is determined that the number of replacements is less than the number threshold, the process returns to the step of determining whether the value of the main counter of the target unit before replacement is greater than a preset main counter threshold.

[0050] On the other hand, the present invention further provides a large flow detection system in a high-speed network scenario, the system comprising a processor, and the processor is configured to execute any of the above methods.

[0051] Through the above technical solution, the embodiment of the present invention provides a method and system for detecting a large flow in a high-speed network scenario. The method and system detect and filter each flow label in the data flow information by setting a first bucket array and a second bucket array in the server. When facing large flow information, it is dynamically updated in real time based on the recorded information of the first bucket array and the second bucket array. Compared with the detection method of the prior art, the method and system provided by the present invention have higher efficiency and accuracy due to the use of dynamic real-time update.

[0052] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0054] Figure 1 is a partial flow chart of a large flow detection method in a high-speed network scenario according to an embodiment of the present invention;

[0055] Figure 2 is a partial flow chart of a large flow detection method in a high-speed network scenario according to an embodiment of the present invention;

[0056] Figure 3is a partial flow chart of a large flow detection method in a high-speed network scenario according to an embodiment of the present invention;

[0057] Figure 4 is a partial flow chart of a large flow detection method in a high-speed network scenario according to an embodiment of the present invention;

[0058] Figure 5 It is a partial flow chart of a large flow detection method in a high-speed network scenario according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0060] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In the embodiments of this application, some existing solutions in the industry such as certain software, components, and models may be mentioned, which should be considered as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0061] like Figure 1 FIG. 1 is a flow chart of a method for detecting a large flow in a high-speed network scenario according to an embodiment of the present invention. Figure 1 In the method, the method may include the following steps:

[0062] In step S10, a first bucket array and a second bucket array for recording data flow information are constructed, wherein the first bucket array and the second bucket array each include a plurality of buckets and an independent hash function, and each bucket includes a plurality of units;

[0063] In step S11, data flow information is obtained;

[0064] In step S12, a hash function of the first bucket array and the second bucket array is used to calculate the index of the bucket corresponding to the flow label according to the flow label of the data flow information;

[0065] In step S13, a set of units to be searched of the flow label is determined according to the index;

[0066] In step S14, a match search is performed in the to-be-searched unit set according to the flow label to determine whether the target unit can be matched and searched in the to-be-searched unit set;

[0067] In step S15, when the target unit can be matched and searched in the search unit set, the main counter and the sub-counter of the target unit are updated.

[0068] In this Figure 1 In the method shown, step S10 is used to construct a first bucket array and a second bucket array. Wherein, the first bucket array and the second bucket array may include multiple buckets and an independent hash function, and each bucket includes multiple units. The hash functions of the first bucket array and the second bucket array are independent of each other. Each unit is correspondingly provided with a main counter and a secondary counter.

[0069] Step S11 may be used to obtain data flow information. The data flow information may include a plurality of continuous data packets, each of which corresponds to a flow label.

[0070] Steps S12 to S14 are used to search for the target unit. The set of units to be searched determined in step S13 includes two buckets. The two buckets are from the first bucket array and the second bucket array, respectively, and their corresponding indexes are from the results of two hash function calculations. In addition, the specific method for the matching search in step S14 can be in various forms known to those skilled in the art. In one example of the present invention, the matching search method can be to directly compare one by one according to the flow labels and search for completely identical flow labels.

[0071] In the case where the corresponding target unit can be matched and searched in the search unit set, the main counter and the sub-counter of the target unit can be directly updated at this time, thereby completing the recording of a single data packet (flow information). The method for updating the main counter and the sub-counter can be various forms known to those skilled in the art. In one example of the present invention, the method for updating the main counter and the sub-counter can be to add 1 to the value of the main counter and the sub-counter.

[0072] On the contrary, if the corresponding target unit cannot be found in the search unit set, the following needs to be executed: Figure 2 Specifically, in the Figure 2 In the method, the method may further include the following steps:

[0073] In step S16, when it is determined that the target unit cannot be matched in the search unit set, it is determined whether there is an empty unit in the search unit set;

[0074] In step S17, when it is determined that there are vacant cells in the search cell set, one of the vacant cells is used as the target cell, and the main counter and the sub-counter of the target cell are updated. Considering that the initial values ​​of the main counter and the sub-counter of the vacant cell are generally 0, the method of updating the main counter and the sub-counter of the target cell in step S17 may be to update the values ​​of the main counter and the sub-counter to 1.

[0075] When it is determined that there are empty cells in the search unit set, it means that the bucket array has recorded flow information that has not been recorded. At the same time, the unit itself can also complete the recording of new flow information. Therefore, the flow information can be directly recorded in the empty cells, thereby completing the recording of the flow information. On the contrary, if it is determined that there are no empty cells in the search unit set. This means that the records of the two buckets in the search unit set are full, and further methods are needed to record. Specifically, the method may also include the following: Figure 3 In the steps shown in Figure 3 In the method, the method may include the following steps:

[0076] In step S18, when it is determined that there is no vacant unit in the search unit set, the unit with the smallest value of the main counter in the search unit set is selected as the target unit;

[0077] In step S19, it is determined whether the secondary counter of the target unit is 0;

[0078] In step S20, when it is determined that the secondary counter of the target unit is not 0, the secondary counter of the selected target unit is decayed by using a preset first probability value;

[0079] In step S21, it is determined whether the secondary counter of the target unit is successfully decayed;

[0080] In step S22, when it is determined that the secondary counter of the target unit is successfully decayed, the value of the secondary counter of the target unit is reduced by 1, and the flow information corresponding to the current flow label is discarded;

[0081] In step S23, when it is determined that the secondary counter of the target unit has not been successfully decayed, the flow information corresponding to the current flow label is discarded.

[0082] In this Figure 3 In the method shown, step S18 is used to select the unit with the smallest value of the main counter in the search unit set as the target unit, and determine whether to put the current flow information into the target unit through steps S19 to S23. Specifically, step S19 can be used to determine whether the sub-counter of the target unit is 0. If it is determined that the sub-counter of the target unit is not 0, it means that the flow information currently recorded by the target unit still has large flow characteristics. At this time, it means that the current flow information cannot replace the flow information in the target unit, so the current flow information can be directly discarded. However, considering the impact of the new flow information, for the target unit, it can be attenuated through step S20, that is: the sub-counter of the selected target unit is attenuated using a preset first probability value. The first probability value can be obtained by a method known to those skilled in the art. In one example of the present invention, the first probability value can be calculated using the following formula (1):

[0083]

[0084] Among them, p1 is the first probability value, A i [h i (f)][l].fc is the value of the sub-counter.

[0085] When it is determined that the secondary counter of the target unit is 0, it means that the current stream information can be replaced into the target unit. Specifically, the method may include: Figure 4 In the steps shown in Figure 4 In the method, the method may further include the following steps:

[0086] In step S24, when it is determined that the secondary counter of the target unit is 0, the current flow label is replaced into the selected target unit according to the preset second probability value;

[0087] In step S25, it is determined whether the current flow label is replaced successfully;

[0088] In step S26, if it is determined that the current flow label is not replaced successfully, the flow information corresponding to the flow label is discarded;

[0089] In step S27, when it is determined that the current flow label is replaced successfully, the secondary counter of the target unit is updated;

[0090] In step S28, it is determined whether the value of the main counter of the target unit before replacement is greater than a preset main counter threshold;

[0091] In step S29, when it is determined that the value of the main counter of the target unit before replacement is less than or equal to the main counter threshold, the replaced flow label is discarded.

[0092] In this Figure 4 In the method shown, when it is determined that the secondary counter of the target unit is 0, the current flow information can be considered to be replaced into the target unit. However, when determining whether to replace, it is also necessary to combine whether the current flow information has a greater probability of being recorded as a large flow feature, so it is necessary to determine whether to replace through a second probability value. The second probability value can be determined by a method known to those skilled in the art. In one example of the present invention, the second probability value can be calculated using the following formula (2):

[0093]

[0094] Among them, p2 is the second probability value, A i [h i (f)][l].lc is the value of the main counter.

[0095] If the replacement is not successful, the current flow label can be directly discarded, that is, step S26. On the contrary, if the replacement is successful, on the one hand, the replaced flow information needs to be processed, and on the other hand, the sub-counter of the target unit needs to be processed. Specifically, if the replacement is successful, the sub-counter of the target unit can be updated to 1. As for the replaced flow information, it is necessary to determine whether the flow information is still necessary to be processed. Therefore, step S28 can be used to determine whether the main counter is greater than the preset main counter threshold. If it is less than or equal to the main counter threshold, it means that the flow information is no longer necessary to be processed, and the replaced flow information (flow label) can be directly discarded, that is, step S29.

[0096] When it is determined that the value of the main counter is greater than the main counter threshold, it means that the flow information can be further processed. Therefore, the method can also include the following steps: Figure 5 In the steps shown in Figure 5 In the method, the method may include the following steps:

[0097] In step S30, when it is determined that the value of the main counter of the target unit before replacement is greater than the main counter threshold, another hash function corresponding to the target unit is selected, and the replaced flow label is input into the hash function to obtain the index of the bucket corresponding to the replaced flow label;

[0098] In step S31, a new search match set is determined according to the index of the bucket corresponding to the replaced flow label;

[0099] In step S32, it is determined whether there is an empty cell in the search matching set;

[0100] In step S33, when it is determined that there is an empty unit in the search match set, the replaced flow label is stored in one of the empty units, and the flow label, the primary counter and the secondary counter of the empty unit are updated;

[0101] In step S34, when it is determined that there is no vacant cell in the search matching set, the cell with the smallest value of the main counter in the search matching set is used as the target cell;

[0102] In step S35, it is determined whether the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label;

[0103] In step S36, if it is determined that the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label, the replaced flow label is discarded;

[0104] In step S37, when it is determined that the value of the main counter of the target unit is less than the value of the main counter corresponding to the replaced flow label, the replaced flow label is replaced into the target unit, and the flow label in the target unit before the replacement is used as the new replaced flow label;

[0105] In step S38, the number of replacements is updated;

[0106] In step S39, it is determined whether the number of replacements is greater than or equal to a preset number threshold;

[0107] In step S40, if it is determined that the number of replacements is greater than or equal to the number threshold, the replaced flow label is discarded;

[0108] When it is determined that the number of replacements is less than the number threshold, the process returns to the step of determining whether the value of the main counter of the target unit before replacement is greater than the preset main counter threshold, that is, returns to step S28.

[0109] When it is determined that the value of the master counter of the target unit before replacement is greater than the master counter threshold, the replaced flow information can be processed, that is, step S30 to step S40. Figure 5 In the method shown, step S30 can be used to select another hash function corresponding to the target unit, and input the replaced flow label into the hash function to obtain the index of the bucket corresponding to the replaced flow label. Wherein, when the target unit belongs to the first bucket array, the hash function can be the hash function of the second bucket array; conversely, when the target unit belongs to the second bucket array, the hash function can be the hash function of the first bucket array.

[0110] Steps S31 to S33 can be used to determine whether there is an empty unit in the new search match set. If there is an empty unit, the flow information (which may also include the values ​​of the primary counter and the secondary counter in the original target unit) can be directly placed in the empty unit.

[0111] In the case that there is no vacant unit in the new search match set, the unit with the smallest value of the primary counter in the search match set needs to be used as the target unit, and then steps S35 to S37 are performed to determine whether replacement is required. In step S37, when the replaced flow label is replaced with the target unit, the values ​​of the primary counter and the secondary counter in the original target unit need to be simultaneously placed in the target unit.

[0112] Steps S38 to S40 can be used as termination conditions in the replacement iteration loop. When it is determined that the number of replacements is greater than or equal to the number threshold, it means that the number of replacements is too many, and the flow label can be directly discarded, thereby ending a round of the loop.

[0113] In addition, in order to regularly detect the recorded large flow information, in one embodiment of the present invention, the recorded large flow information can be downloaded and detected according to a preset time period, thereby achieving complete monitoring of the communication server port.

[0114] On the other hand, the present invention further provides a large flow detection system in a high-speed network scenario, the system comprising a processor, and the processor is configured to execute any of the above methods.

[0115] Through the above technical solution, the embodiment of the present invention provides a method and system for detecting a large flow in a high-speed network scenario. The method and system detect and filter each flow label in the data flow information by setting a first bucket array and a second bucket array in the server. When facing large flow information, it is dynamically updated in real time based on the recorded information of the first bucket array and the second bucket array. Compared with the detection method of the prior art, the method and system provided by the present invention have higher efficiency and accuracy due to the use of dynamic real-time update.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0122] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0123] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0124] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting a large flow in a high-speed network scenario, characterized in that: The method comprises: Constructing a first bucket array and a second bucket array for recording data flow information, wherein the first bucket array and the second bucket array each include a plurality of buckets and an independent hash function, and each bucket includes a plurality of units; Acquiring the data flow information; Calculate the index of the bucket corresponding to the flow label according to the flow label of the data flow information by using the hash function of the first bucket array and the second bucket array; Determine a set of units to be searched for the flow label according to the index; According to the flow label, a match search is performed in the set of units to be searched, and it is determined whether a target unit can be matched and searched in the set of units to be searched; In the case that the target unit can be matched and searched in the search unit set, the main counter and the sub-counter of the target unit are updated.

2. The method according to claim 1, characterized in that The method further comprises: In the case where it is determined that the target unit cannot be matched and searched in the search unit set, determining whether there is an empty unit in the search unit set; When it is determined that there are vacant units in the search unit set, one of the vacant units is used as a target unit, and a main counter and a sub-counter of the target unit are updated.

3. The method according to claim 2, characterized in that The method further comprises: In the case where it is determined that there is no vacant unit in the search unit set, selecting a unit with a minimum value of a main counter in the search unit set as a target unit; Determine whether the secondary counter of the target unit is 0; In the case where it is determined that the secondary counter of the target unit is not 0, the secondary counter of the selected target unit is decayed by using a preset first probability value; Determining whether the secondary counter of the target unit is decayed successfully; If it is determined that the secondary counter of the target unit is successfully decayed, the value of the secondary counter of the target unit is reduced by 1, and the flow information corresponding to the current flow label is discarded; When it is determined that the secondary counter of the target unit has not been successfully decayed, the flow information corresponding to the current flow label is discarded.

4. The method according to claim 3, characterized in that When it is determined that the secondary counter of the target unit is not 0, the secondary counter of the selected target unit is attenuated by using a preset first probability value, including: The first probability value is calculated according to formula (1): Among them, p1 is the first probability value, A i [h i (f)][l].fc is the value of the sub-counter.

5. The method according to claim 3, characterized in that: The method further comprises: When it is determined that the secondary counter of the target unit is 0, replacing the current flow label with the selected target unit according to a preset second probability value; Determine whether the current flow label is replaced successfully; If it is determined that the current flow label is not replaced successfully, discard the flow information corresponding to the flow label; If it is determined that the current flow label is replaced successfully, updating the secondary counter of the target unit; Determine whether the value of the main counter of the target unit before replacement is greater than a preset main counter threshold; When it is determined that the value of the main counter of the target unit before replacement is less than or equal to the main counter threshold, the replaced flow label is discarded.

6. The method according to claim 5, characterized in that When it is determined that the secondary counter of the target unit is 0, replacing the current flow label with the selected target unit according to a preset second probability value includes: The second probability value is calculated according to formula (2): Among them, p2 is the second probability value, A i [h i (f)][l].lc is the value of the main counter.

7. The method according to claim 5, characterized in that The method further comprises: When it is determined that the value of the main counter of the target unit before replacement is greater than the main counter threshold, another hash function corresponding to the target unit is selected, and the replaced flow label is input into the hash function to obtain an index of a bucket corresponding to the replaced flow label; Determine a new search match set according to the index of the bucket corresponding to the replaced flow label; Determine whether there is an empty cell in the search match set; When it is determined that there are vacant units in the search match set, the replaced flow label is stored in one of the vacant units, and the flow label, the primary counter and the secondary counter of the vacant unit are updated.

8. The method according to claim 7, characterized in that The method further comprises: In the case where it is determined that there is no vacant unit in the search matching set, taking the unit with the smallest value of the main counter in the search matching set as the target unit; Determine whether the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label; If it is determined that the value of the main counter of the target unit is greater than or equal to the value of the main counter corresponding to the replaced flow label, the replaced flow label is discarded; When it is determined that the value of the main counter of the target unit is less than the value of the main counter corresponding to the replaced flow label, the replaced flow label is replaced into the target unit, and the flow label in the target unit before the replacement is used as the new replaced flow label; Update replacement times; Determining whether the number of replacements is greater than or equal to a preset number threshold; If it is determined that the number of replacements is greater than or equal to the number threshold, discarding the replaced flow label; When it is determined that the number of replacements is less than the number threshold, the process returns to the step of determining whether the value of the main counter of the target unit before replacement is greater than a preset main counter threshold.

9. A high-speed network scenario large flow detection system, characterized in that: The system comprises a processor configured to execute the method according to any one of claims 1 to 8.

Images