White-box switch network measurement method, device, equipment and medium

Abstract

The invention discloses a white-box switch network measurement method and device, equipment and a medium. The method comprises the following steps: receiving input target traffic; mapping the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generating a first traffic flow value of the target traffic; and under the condition that the counter value of a large-flow vote counter is detected to be greater than a preset large-flow threshold value, mapping the target flow to a second hash list and a third hash list according to the flow identifier of the target flow. According to the method, the network flow can be effectively and rapidly measured in combination with the white-box switch, and the measurement precision of the large flow is improved through the large flow threshold value.

Description

Technical Field

[0001] The present invention relates to the technical field of network traffic management, and in particular to a white box switch network measurement method, device, equipment and medium. Background Art

[0002] With the promotion of technologies such as artificial intelligence and the Internet of Things, the Internet has penetrated into every corner of people's work and life. The scale and bandwidth of the network are constantly increasing, and the programmability and flexibility of the network are also increasing. As a result, traditional traffic measurement methods face the dual challenges of computing resources and memory resources. On the one hand, the processing speed of data packets cannot match the growing port speed. On the other hand, network devices such as switches and routers have limited memory and cannot accommodate huge network data. White box switches are a new type of network equipment that can cope with the explosive growth of network data volume, the rise of cloud computing and big data technologies, the demand for network programmability, the development of software-defined networks, the impact of open source code, and the need to reduce hardware costs and supply chain diversity. However, in the existing technology, there is a lack of network measurement and identification methods for white box switches based on white box switches. Summary of the invention

[0003] The present invention provides a white box switch network measurement method, device, equipment and medium to solve the technical problem that the prior art cannot measure large-volume network traffic.

[0004] According to one aspect of the present invention, a white-box switch network measurement method is provided, which is applied to a white-box switch and includes:

[0005] Receive input target traffic; wherein the target traffic includes traffic data and traffic identifier;

[0006] Map the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information and a large flow vote counter;

[0007] When it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

[0008] According to another aspect of the present invention, a white box switch network measurement device is provided, which is applied to a white box switch, and includes:

[0009] The flow receiving module is used to receive the input target flow; wherein the target flow includes flow data and flow identification;

[0010] A first hash module is used to map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information, and a large flow vote counter;

[0011] The large flow hash module is used to map the target flow to the second hash list and the third hash list respectively according to the flow identifier of the target flow when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold.

[0012] According to another aspect of the present invention, a white box switch is provided, the white box switch comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the white-box switch network measurement method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the white-box switch network measurement method described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention receives input target traffic; maps the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generates the first traffic flow value of the target traffic, so that the target traffic can be effectively identified and measured through the first hash list, the efficiency of identifying network traffic is improved, and the network traffic is effectively classified; when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier of the target traffic. Large flows can be effectively screened through the secondary hash list and the tertiary hash list, further improving the efficiency and accuracy of white box switch network measurement, solving the technical problem that the prior art cannot measure large flow network traffic, and can effectively and quickly measure network traffic in combination with white box switches, and improve the measurement accuracy of large flows through large flow thresholds.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a white box switch network measurement method is provided for an embodiment of the present invention;

[0021] Figure 2 A flowchart of another white box switch network measurement method provided by an embodiment of the present invention;

[0022] Figure 3 A flowchart of a white box switch network measurement method provided by an embodiment of the present invention;

[0023] Figure 4 A flowchart of a white box switch network measurement method provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of a white box switch network measurement device provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of the internal architecture of a white box switch disclosed in an embodiment of the present invention;

[0026] Figure 7 The present invention discloses a white box switch network traffic processing structure. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1 A flowchart of a white-box switch network measurement method is provided for an embodiment of the present invention. This embodiment is applicable to the case where a white-box switch inserts measurement into network traffic. The method can be performed by a white-box switch network measurement device. The white-box switch network measurement device can be implemented in the form of hardware and / or software. The white-box switch network measurement device can be configured in a white-box switch. Figure 1 As shown, the method includes:

[0030] S110: Receive input target traffic.

[0031] The target flow may be the network flow transmitted through the white box switch in the network. It should be noted that the network flow will change over time, and the size of the network flow will change at different times. In the network flow, the flow with a large amount of data is called a large flow, and the flow with a small amount of data is called a small flow. By distinguishing the large flow from the small flow of the network flow, the scheduling efficiency and accuracy of the white box switch can be improved.

[0032] Optionally, the target traffic includes traffic data and a traffic identifier. The traffic data may be data transmitted by the network traffic. The traffic identifier may be identification information of the network traffic. The traffic identifier is five-tuple information. The five-tuple information is usually composed of a source IP (Internet Protocol) address, a destination IP address, a source port number, a destination port number, and a protocol. The source IP address and the destination IP address identify the sender and receiver of the network traffic in the network communication. The source port number and the destination port number may be used to establish a network connection. The protocol may be a rule and format for network communication and may be used for different types of network communication.

[0033] S120. Map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value for the target traffic.

[0034] The first hash list may be a hash list constructed based on a Sketch (probabilistic data structure) data structure. The first hash list includes R rows and W columns, and there are R*W data buckets. Each row corresponds to a hash function h g , g=1,2,3......R; each network traffic can be mapped to a data bucket by the hash function. The data bucket of the first hash list also includes a traffic frequency counter, a large flow identification information and a large flow vote counter. The counter value of the traffic frequency counter can be used to represent the frequency of network traffic in the data bucket, the large flow identification information can be the identification of the large flow in the network traffic, and the counter value of the large flow vote counter can be the vote value of the large flow. Exemplarily, in the first hash list, the bucket position of the data bucket in the i-th row and j-th column in the first hash list can be represented by B[i][j], S i,j represents the frequency of all network traffic in the data bucket of row i and column j, F i,j Indicates the large flow identification information in the data bucket of row i and column j, C i,j Represents the vote value of the large flow vote counter in the data bucket of the i-th row and j-th column.

[0035] Among them, the first position bucket can be a data bucket storing the target traffic in the first hash list. It should be noted that the target traffic is stored in multiple data buckets in the first hash list, and there is a data bucket in each row as the first position bucket.

[0036] The first traffic flow value may be a traffic estimation value of the target traffic in the first hash list, and the traffic size of the target traffic can be evaluated through the first traffic flow value.

[0037] Optionally, a hash calculation is performed on a hash function corresponding to the first hash list of the target traffic, a first position bucket is determined, the target traffic is inserted into the first position bucket, and a first traffic flow value of the target traffic is evaluated.

[0038] S130: When it is detected that the counter value of the large flow vote counter is greater than a preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

[0039] Among them, the preset large flow threshold can be a value pre-set to evaluate whether the target flow is a large flow; the preset large flow threshold is compared with the counter value of the large flow vote counter of the first position bucket, if the counter value of the large flow vote counter is greater than the preset large flow threshold, the target flow is considered to be a large flow, and the target flow needs to be mapped to the second hash list and the third hash list; if the counter value of the large flow vote counter is less than the preset large flow threshold, the target flow is considered to be a small flow, and there is no need to map the target flow to the second hash list and the third hash list. Exemplarily, the preset large flow threshold can be represented by Tresold.

[0040] Among them, the second hash list can be a hash list built based on the Sketch data structure. In the second hash list, the hash list includes R rows and W columns, there are R*W data buckets, each row corresponds to a hash function, each large flow can be mapped to a data bucket by the hash function, and a traffic frequency counter is set in the data bucket of the second hash list. The counter value of the traffic frequency counter represents the frequency of the large flow in the data bucket.

[0041] Among them, the third hash list can be a hash list built based on the Sketch data structure. In the third hash list, the hash list includes 2 rows and W columns, there are 2*W data buckets, each row corresponds to a hash function, each large flow can be mapped to a data bucket by the hash function, and W+1 counters are set in the data bucket of the third hash list. The W+1 counters include a traffic frequency counter, including W binary counters, and the binary counters are used to represent the total number of 1 bits at each bit position of the binary representation of the traffic identifier. Exemplarily, 32 binary counters are used to store the source IP address. The most significant bit of the IP address is recorded in the 31st binary counter, the second most significant bit is stored in the 30th binary counter, and so on, and the binary representation of the traffic identifier is recorded.

[0042] Specifically, after the target traffic is inserted into the first hash list, the relationship between the counter value of the large flow vote counter and the preset large flow threshold is detected. If it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier; if it is detected that the counter value of the large flow vote counter is less than the preset large flow threshold, there is no need to map the target traffic to the second hash list and the third hash list.

[0043] The technical solution of the embodiment of the present invention receives input target traffic; maps the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generates the first traffic flow value of the target traffic, so that the target traffic can be effectively identified and measured through the first hash list, the efficiency of identifying network traffic is improved, and the network traffic is effectively classified; when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier of the target traffic. Large flows can be effectively screened through the secondary hash list and the tertiary hash list, further improving the efficiency and accuracy of white box switch network measurement, solving the technical problem that the prior art cannot measure large flow network traffic, and can effectively and quickly measure network traffic in combination with white box switches, and improve the measurement accuracy of large flows through large flow thresholds.

[0044] Figure 2 This is a flowchart of another white box switch network measurement method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is the specific method of mapping the target traffic to the first hash list. Figure 2 As shown, the method includes:

[0045] S210: Receive input target traffic.

[0046] S220. Perform hash calculation on the traffic identifier using the first-level hash function of the first hash list to determine a first hash number set.

[0047] The first-level hash function may be a hash function corresponding to each row in the first hash list.

[0048] The first hash number set may be a data set consisting of hash values ​​obtained by performing hash calculations on the traffic identifier of the target traffic and each hash function of the first hash list. It should be noted that the traffic identifier of the target traffic is hashed with each hash function of the first hash list to obtain a hash value corresponding to each hash function, and the data set consisting of all hash values ​​is used as the first hash number set.

[0049] Specifically, a hash calculation is performed on the primary hash function corresponding to the first hash list and the traffic identifier to obtain a first hash number set corresponding to the traffic identifier.

[0050] S230: Perform mapping position calculation according to the number of columns in the first hash list and the first hash number set to determine at least one first position bucket.

[0051] Specifically, after obtaining the first hash number set, each hash function of the first hash number set takes a remainder of the number of columns of the first hash list to determine the first position bucket of each row of the target traffic in the first hash list.

[0052] S240: Insert the target traffic into the first position bucket, and generate a first traffic flow value of the target traffic.

[0053] Specifically, the target traffic is inserted into each first position bucket, and after the target traffic is inserted into each first position bucket, a first traffic flow value corresponding to the target traffic in the first hash list is generated.

[0054] Optionally, in another optional embodiment of the present invention, inserting the target traffic into the first position bucket includes:

[0055] Inserting the flow data into the first location bucket;

[0056] Updating the counter value of the flow frequency counter of the first position bucket;

[0057] If the large flow identification information is a null value, the traffic identification is used as the large flow identification information; and the counter value of the large flow vote counter is updated;

[0058] If the large flow identification information is not a null value, the counter value of the large flow vote counter is updated according to the large flow identification information and the traffic identifier, and the large flow identification information is updated according to the counter value of the large flow vote counter.

[0059] Specifically, the target traffic is inserted into each first position bucket, and the insertion process is as follows: insert the traffic data into the first position bucket, add 1 to the counter value of the traffic frequency counter of the first position bucket, identify the large flow identification information of the first position bucket, if the large flow identification information is a null value, use the traffic identification of the target traffic as the large flow identification information of the first position bucket, and add 1 to the counter value of the large flow vote counter; if the large flow identification information is not a null value, obtain the large flow identification information, when the large flow identification information and the traffic identification are different, subtract 1 from the counter value of the large flow vote counter, when the large flow identification information and the traffic identification are the same, add 1 to the counter value of the large flow vote counter; update the large flow identification information according to the counter value of the large flow vote counter.

[0060] Optionally, in another optional embodiment of the present invention, updating the large flow identification information according to the counter value of the large flow vote counter includes:

[0061] If the counter value of the large flow vote counter is less than 0, the traffic identifier is used as the large flow identifier information; if the counter value of the large flow vote counter is not less than 0, the large flow identifier information is retained.

[0062] Specifically, when the counter value of the large flow vote counter is updated, the counter value of the large flow vote counter is identified. If the counter value of the large flow vote counter is less than 0, it means that the network traffic corresponding to the large flow identification information is not a large flow. The traffic identifier of the target traffic is used as the large flow identification information, and the counter value of the large flow vote counter of the large flow identification information is set to 1; if the counter value of the large flow vote counter is not less than 0, it means that the network traffic corresponding to the large flow identification information has the possibility of a large flow, and the large flow identification information will be retained.

[0063] Exemplarily, before mapping the target traffic to the first hash list, the traffic frequency counter, the large flow identification information and the large flow vote counter in the first hash list are set to 0, and the traffic identification of the target traffic is used for calculation, that is, the five-tuple information is used for calculation with a hash function corresponding to each row in the first hash list to obtain a hash value corresponding to each row of the five-tuple information, and for the hash value corresponding to each row of the five-tuple information in the first hash list, each hash value is taken the remainder with the number of columns in the first hash list in turn to calculate the first position bucket of the hash value corresponding to each row in the current row, and then the five-tuple information is mapped to the first position bucket of each row in the first hash list, the traffic data is inserted into the first position bucket, and the counter value of the traffic frequency counter of the first position bucket is increased by 1 to identify the large flow of the first position bucket. Identification information, if the large flow identification information is null, the flow identification of the target flow is used as the large flow identification information of the first position bucket, and the counter value of the large flow vote counter is increased by 1; if the large flow identification information is not null, the large flow identification information is obtained, and when the large flow identification information and the flow identification are different, the counter value of the large flow vote counter is reduced by 1, and when the large flow identification information and the flow identification are the same, the counter value of the large flow vote counter is increased by 1; the counter value of the large flow vote counter is identified, if the counter value of the large flow vote counter is less than 0, it means that the network flow corresponding to the large flow identification information is not a large flow, and the flow identification of the target flow is used as the large flow identification information; if the counter value of the large flow vote counter is not less than 0, it means that the network flow corresponding to the large flow identification information has the possibility of a large flow, and the large flow identification information will be retained.

[0064] Optionally, in another optional embodiment of the present invention, generating the first flow value of the target flow includes:

[0065] If the large flow identification information is the same as the flow identification, a flow value calculation is performed according to the counter value of the flow frequency counter and the counter value of the large flow vote counter by using a first preset flow value estimation algorithm to determine the first flow value;

[0066] If the large flow identification information and the traffic identification are different, a flow value is calculated according to the counter value of the traffic frequency counter and the counter value of the large flow vote counter by using a second preset flow value estimation algorithm to determine the first traffic flow value.

[0067] Optionally, after the target traffic is inserted into the first hash list, if the large flow identification information and the traffic identification are the same, there are two situations. The first situation is: in the first position bucket, the large flow identification information and the traffic identification are the same, then the first traffic flow value is determined by calculating the flow value according to the counter value of the traffic frequency counter and the counter value of the large flow vote counter through the first preset flow value estimation algorithm. Among them, the first preset flow value estimation algorithm can sum the counter value of the traffic frequency counter and the counter value of the large flow vote counter to obtain the sum value, and multiply the sum value by 2, and take the inverse proportion of 1 and the sum value multiplied by 2 as the first traffic value.

[0068] Optionally, the second situation is: in the first position bucket, the large flow identification information is different from the flow identification, and the flow value is calculated according to the counter value of the flow frequency counter and the counter value of the large flow vote counter by the second preset flow value estimation algorithm to determine the first flow flow value. Among them, the second preset flow value estimation algorithm can be based on the first preset flow value estimation algorithm corresponding to each first position bucket, which can sum the counter value of the flow frequency counter and the counter value of the large flow vote counter to obtain the sum value, and multiply the sum value by 2, and sum the inverse proportion of 1 and the sum value multiplied by 2 with the counter value of the large flow vote counter to obtain the calculated value of each first position bucket, and take the minimum value of the calculated values ​​of all first position buckets in the target flow as the first flow flow value.

[0069] S250: When it is detected that the counter value of the large flow vote counter is greater than a preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

[0070] The technical solution of the embodiment of the present invention receives input target traffic; maps the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generates the first traffic flow value of the target traffic, so that the target traffic can be effectively identified and measured through the first hash list, the efficiency of identifying network traffic is improved, and the network traffic is effectively classified; when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier of the target traffic. Large flows can be effectively screened through the secondary hash list and the tertiary hash list, further improving the efficiency and accuracy of white box switch network measurement, solving the technical problem that the prior art cannot measure large flow network traffic, and can effectively and quickly measure network traffic in combination with white box switches, and improve the measurement accuracy of large flows through large flow thresholds.

[0071] Figure 3 The present invention provides a flowchart of a white box switch network measurement method. The relationship between this embodiment and the above embodiment is the specific method of mapping the target traffic into the second hash list and the third hash list when the target traffic is a large flow. Figure 3 As shown, the method includes:

[0072] S310: Receive input target traffic.

[0073] S320. Map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value for the target traffic.

[0074] S330: When it is detected that the counter value of the large flow vote counter is greater than a preset large flow threshold, the flow identifier is identified through a preset Bloom filter to determine a target identification result.

[0075] The Bloom filter may be a probabilistic data structure pre-set to determine whether the target traffic already exists. The Bloom filter may perform a hash calculation on the traffic identifier of the target traffic to obtain an identification result of the target traffic.

[0076] The target identification result may be an identification result of the traffic identifier of the target traffic by the Bloom filter; illustratively, the target identification result may be non-inserted traffic and inserted traffic.

[0077] Specifically, when it is detected that the counter value of the large flow vote counter of the target flow is greater than the preset large flow threshold, the flow identifier is identified through a preset Bloom filter to determine the target identification result.

[0078] S340. Map the target traffic to the second hash list and the third hash list according to the target identification result.

[0079] Specifically, the target traffic is mapped to the second hash list and the third hash list according to the target identification result.

[0080] Optionally, in another optional embodiment of the present invention, mapping the target traffic to the second hash list according to the target identification result includes:

[0081] If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the second hash function of the second hash list to determine a second hash number set;

[0082] Perform mapping position calculation according to the number of columns of the second hash list and the second hash number set to determine at least one second position bucket; insert the target traffic into the second position bucket, and use the first traffic flow value as the counter value of the traffic frequency counter of the second position bucket.

[0083] The second hash function may be a hash function corresponding to each row in the second hash list.

[0084] The second hash number set may be a data set consisting of hash values ​​obtained by performing hash calculations on the traffic identifier of the target traffic and each hash function of the second hash list. It should be noted that the traffic identifier of the target traffic is hashed with each hash function of the second hash list to obtain a hash value corresponding to each hash function, and the data set consisting of all hash values ​​is used as the second hash number set.

[0085] The second location bucket may be a second data bucket corresponding to the target traffic in the second hash list. The second data bucket may be a data bucket in the second hash list.

[0086] Specifically, when it is identified that the target identification result is that no traffic is inserted, the traffic identifier is hashed according to the second hash function of the second hash list, a second hash number set is determined, a mapping position calculation is performed according to the number of columns of the second hash list and the second hash number set, at least one second position bucket is determined in the second data bucket, the target traffic is inserted into the second position bucket, and the first traffic flow value is used as the counter value of the traffic frequency counter of the second position bucket.

[0087] Optionally, in another optional embodiment of the present invention, mapping the target traffic to the second hash list according to the target identification result further includes:

[0088] If the target identification result is that traffic has been inserted, the second position bucket corresponding to the target traffic is obtained in the second hash list, and the counter value of the traffic frequency counter of the second position bucket is updated.

[0089] Specifically, when the target identification result is the inserted traffic of the second hash list, the second position bucket corresponding to the target traffic is directly obtained in the second hash list without inserting the target traffic into the second position bucket again, and the counter value of the traffic frequency counter of the second position bucket is changed.

[0090] Optionally, in another optional embodiment of the present invention, the third position bucket further includes a flow frequency counter and at least one binary counter;

[0091] Mapping the target traffic to the third hash list according to the target identification result includes:

[0092] If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the third hash function of the third hash list to determine a third hash number set;

[0093] Perform mapping position calculation according to the number of columns of the third hash list and the third hash number set to determine at least one third position bucket;

[0094] The target traffic is inserted into the third position bucket, the first traffic flow value is used as the counter value of the traffic frequency counter of the third position bucket, and the traffic identifier is updated to the binary counter.

[0095] The third hash function may be a hash function corresponding to each row in the third hash list.

[0096] The third hash number set may be a data set consisting of hash values ​​obtained by performing hash calculations on the traffic identifier of the target traffic and each hash function of the third hash list. It should be noted that the traffic identifier of the target traffic is hashed with each hash function of the third hash list to obtain a hash value corresponding to each hash function, and the data set consisting of all hash values ​​is used as the third hash number set.

[0097] The third position bucket may be a third data bucket corresponding to the target traffic in the third hash list. The third data bucket may be a data bucket in the third hash list.

[0098] Specifically, when it is identified that the target identification result is that no traffic is inserted, the traffic identifier is hashed according to the third hash function of the third hash list to determine the third hash number set, and a mapping position calculation is performed according to the number of columns of the third hash list and the third hash number set. Two third position buckets are determined in the third data bucket, and the target traffic is inserted into the two third position buckets. The first traffic flow value is used as the counter value of the traffic frequency counter of the third position bucket, and the traffic identifier is updated to the binary counter.

[0099] The technical solution of the embodiment of the present invention receives input target traffic; maps the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generates the first traffic flow value of the target traffic, so that the target traffic can be effectively identified and measured through the first hash list, the efficiency of identifying network traffic is improved, and the network traffic is effectively classified; when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier of the target traffic. Large flows can be effectively screened through the secondary hash list and the tertiary hash list, further improving the efficiency and accuracy of white box switch network measurement, solving the technical problem that the prior art cannot measure large flow network traffic, and can effectively and quickly measure network traffic in combination with white box switches, and improve the measurement accuracy of large flows through large flow thresholds.

[0100] Figure 4 The flowchart of a white box switch network measurement method provided by an embodiment of the present invention is a specific method for restoring network traffic through a three-level hash list. Figure 4 As shown, the method includes:

[0101] S410: Receive input target traffic.

[0102] S420. Map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value for the target traffic.

[0103] S430: When it is detected that the counter value of the large flow vote counter is greater than a preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

[0104] S440: Based on the query task of the network traffic, a traffic query is performed in a preset traffic queue.

[0105] The query task may be to query the large flow of data traffic in the three-level hash list. It should be noted that the three-level hash list is the first hash list, the second hash list and the third hash list.

[0106] The traffic queue may be a large flow queue pre-set for storing network traffic. It should be noted that the network traffic stored in the traffic queue is a large flow. If there is a large flow in the traffic queue, the large flow can be directly obtained from the traffic queue. If there is no large flow in the traffic queue, the large flow needs to be restored through the third hash list and the second hash list of the three-level hash list.

[0107] S450: If the traffic queue is empty, access the third data bucket of the third hash list.

[0108] Specifically, when there is no large flow in the traffic queue, a third data bucket in the third hash list is accessed.

[0109] S460. If the counter value of the traffic frequency counter of the third data bucket is greater than a preset access traffic threshold, the network traffic of the third data bucket is restored to obtain query network traffic.

[0110] Among them, the preset access traffic threshold can be a data value preset for judging whether there is a large flow in the third data bucket. It should be noted that when recovering the large flow in the third data bucket, it is necessary to judge in advance whether there is a large flow in the third data bucket, and then compare the counter value of the traffic frequency counter of the third data bucket with the preset access traffic threshold. If the counter value of the traffic frequency counter of the third data bucket is greater than the preset access traffic threshold, it means that there is a large flow in the third data bucket, and the large flow can be recovered in the third data bucket. If the counter value of the traffic frequency counter of the third data bucket is less than the preset access traffic threshold, it means that there is no large flow in the third data bucket, and then select the next third data bucket, and continue to judge the next third data bucket.

[0111] Optionally, in a third data bucket, if the counter value of the traffic frequency counter of the third data bucket is greater than a preset access traffic threshold, it is necessary to restore the network traffic of the third data bucket to determine the query network traffic. The query network traffic may be a large flow stored in the third data bucket.

[0112] Optionally, in another optional embodiment of the present invention, the recovering the network traffic of the third data bucket to obtain the query network traffic includes:

[0113] determining a bit rate of each of the binary counters according to a counter value of the traffic frequency counter of the third data bucket and a counter value of each of the binary counters;

[0114] updating the counter value of each of the binary counters according to the bit rate of the binary counter and a preset bit recovery threshold, and determining a recovery value of each of the binary counters;

[0115] The query network traffic is determined according to the restored value of each binary counter; wherein the query network traffic includes a query traffic identifier and query traffic data.

[0116] If the counter value of the traffic frequency counter of the third data bucket is greater than the preset access traffic threshold, the network traffic of the third data bucket is restored to obtain the query network traffic.

[0117] The bit rate of the binary counter may be used to detect the majority of large flows in the third data bucket. It should be noted that there is more than one large flow in the third data bucket, and the bit rate of the binary counter can be used to identify the majority of large flows in the third data bucket. The bit rate of the binary counter may be a ratio of a counter value of each binary counter to a counter value of a traffic frequency counter of the third data bucket, and the value range of the bit rate of each binary counter is between 1 and 0.

[0118] The bit recovery threshold may be a data value preset to identify the majority of large flows in the third data bucket.

[0119] Optionally, a recovery value of a binary counter is determined according to a bit recovery threshold, and the bit recovery threshold is compared with the bit rate of the binary counter. If the bit rate of the binary counter is greater than the bit recovery threshold, the recovery value of the binary counter is set to 1, and if the bit rate of the binary counter is less than the bit recovery threshold, the recovery value of the binary counter is set to 0. The recovery value of the binary counter can be used to recover the large flow of the third data bucket.

[0120] Optionally, for a third data bucket, the bit rate of each binary counter is determined based on the counter value of the traffic frequency counter of the third data bucket and the counter value of each binary counter, and the bit rate of each binary counter is numerically compared with the bit recovery threshold in turn to obtain the recovery value of each binary counter, and the recovery value of each binary counter is taken out in turn according to the position order of each binary counter in the third data bucket to form a binary number, the constructed binary number is identified, and a query traffic identifier for querying the network traffic is obtained, and the query traffic data is obtained based on the query traffic identifier.

[0121] Exemplarily, taking the recovered query traffic identifier as an IP address as an example, after calculating the bit rate of each binary counter, set the bit recovery threshold to 0.5, and compare the bit rate of each binary counter with the bit recovery threshold. If the bit rate of the binary counter is greater than 0.5, the recovery value of the binary counter is set to 1; if the bit rate of the binary counter is less than 0.5, the recovery value of the binary counter is set to 0 to obtain the bit value of each binary counter. According to the position order of each binary counter in the third data bucket, the recovery value of each binary counter is taken out in turn to form a binary number. The binary number is in the form of an IP address: 00001010.00000000.00000000.00000100, and the IP address can be obtained as 10.0.0.4.

[0122] Optionally, in another optional embodiment of the present invention, after the network traffic of the third data bucket is restored to obtain the query network traffic, the method further includes:

[0123] Acquire a second traffic flow value in the second hash list according to the query traffic identifier;

[0124] determining the flow identification value according to the counter value of the flow frequency counter of the third data bucket and the second flow estimation value;

[0125] If the traffic identification value is greater than a preset large flow identification threshold, the queried network traffic is considered to be large flow data, and threshold change information of the preset large flow threshold is obtained;

[0126] The preset large flow threshold is updated according to the threshold change information, and the query network flow is sent to the flow queue;

[0127] If the traffic identification value is less than a preset large flow identification threshold, the queried network traffic is considered to be small flow data, and a third traffic flow value is obtained in the first hash list according to the queried traffic identifier;

[0128] The preset large flow threshold is updated according to the third traffic flow value and the second traffic flow value, and threshold change information of the preset large flow threshold is recorded.

[0129] The second traffic flow value may be a traffic estimation value of the query network traffic in the second hash list. It should be noted that the second traffic flow value is estimated through the second hash list, by determining at least one second data bucket corresponding to the query traffic identifier in the second hash list, obtaining the counter value of the traffic frequency counter of each second data bucket, and taking the minimum counter value of the traffic frequency counter as the second traffic flow value.

[0130] The traffic identification value may be a data value used to detect whether the restored network traffic is a large flow. It should be noted that the ratio of the counter value of the traffic frequency counter of the third data bucket and the second traffic flow value is used as the traffic identification value.

[0131] The large flow identification threshold may be a data threshold pre-set for identifying the restored network flow. It should be noted that if the flow identification value is greater than the large flow index threshold, the restored network flow may be determined to be a large flow; if the flow identification value is less than the large flow index threshold, the restored network flow may be determined to be a small flow.

[0132] The threshold change information may be data recording the history of changes in the preset high-volume threshold. It should be noted that, in an embodiment of the present invention, the preset high-volume threshold may be adjusted dynamically in real time. When the preset high-volume threshold is adjusted, the adjustment information of the preset high-volume threshold is recorded in the threshold change information. Exemplarily, the threshold change information may include a timestamp and an adjustment change value for each adjustment of the preset high-volume threshold.

[0133] Optionally, based on the query traffic identifier of the query network traffic, at least one corresponding second data bucket is determined in the second hash list, the second traffic flow value is determined by the counter value of the traffic frequency counter in each second data bucket, the counter value of the traffic frequency counter of the query traffic identifier in the third data bucket is determined, and the traffic identification value is determined according to the counter value of the traffic frequency counter in the third data bucket and the second traffic flow value. If the traffic identification value is greater than a preset large flow identification threshold, the query network traffic is considered to be large flow data, and the threshold change information of the preset large flow threshold is obtained, the preset large flow threshold is updated based on the threshold change information, and the query network traffic is sent to the traffic queue.

[0134] Optionally, after the query network traffic is sent to the traffic queue, the query network traffic is deleted from the second hash list and the third hash list, and the second hash list and the third hash list are updated.

[0135] Optionally, after identifying that the queried network traffic is a large flow, the threshold change information of the preset large flow threshold in the first hash list is queried. If the threshold change information shows that the preset large flow threshold has not increased, the large flow is directly sent to the traffic queue; if the threshold change information shows that the preset large flow threshold has increased, the preset large flow threshold is reduced by 1.

[0136] Optionally, if the queried traffic identification value is less than the preset large flow identification threshold, the queried network traffic is considered to be small flow data, which means that the preset large flow threshold of the first hash list is too small, causing the small flow to enter the second hash list and the third hash list, then the third traffic flow value is obtained in the first hash list based on the query traffic identifier of the queried network traffic, and if the third traffic flow value is greater than the second traffic flow value, the preset large flow threshold is numerically increased, and the increased value is the difference between the third traffic flow value and the second traffic flow value; if the third traffic flow value is less than the second traffic flow value, the preset large flow threshold is directly increased by 1. The third traffic flow value can be the first traffic flow value of the query traffic identifier in the first hash list.

[0137] Optionally, in an embodiment of the present invention, there may be multiple large flows in the third data bucket. Therefore, after a large flow is identified in the third data bucket, the large flow is deleted from the third data bucket and the second data bucket, and the second flow value is subtracted from the counter value of the flow frequency counter corresponding to each of the third data bucket and the second data bucket, and the second flow value is also subtracted from the binary counter in the third data bucket to obtain the updated third data bucket and the second data bucket. If the counter value of the flow frequency counter of the third data bucket is greater than the preset access traffic threshold, the network traffic of the third data bucket is restored to obtain the next operation steps of querying the network traffic and updating the preset large flow threshold until there is no large flow in the third data bucket.

[0138] Optionally, after sending the query network traffic to the traffic queue, query whether the traffic queue is a full queue. If it is a full queue, return the first restored large flow. If the queue is not a full queue, return to execute if the counter value of the traffic frequency counter of the third data bucket is greater than the preset access traffic threshold, then restore the network traffic of the third data bucket, obtain the next query network traffic and update the preset large flow threshold, until the counter value of the traffic frequency counter of the third data bucket is less than the preset access traffic threshold or the traffic queue is a full queue; when the counter value of the traffic frequency counter of the third data bucket is less than the preset access traffic threshold, locate the next third data bucket, if the third data bucket is fully restored, check the traffic queue, if the traffic queue is not empty, return the first restored large flow; if the traffic queue is empty, report that all large flows in the third hash list have been restored.

[0139] The technical solution of the embodiment of the present invention receives input target traffic; maps the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generates the first traffic flow value of the target traffic, so that the target traffic can be effectively identified and measured through the first hash list, the efficiency of identifying network traffic is improved, and the network traffic is effectively classified; when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target traffic is mapped to the second hash list and the third hash list according to the traffic identifier of the target traffic. Large flows can be effectively screened through the secondary hash list and the tertiary hash list, further improving the efficiency and accuracy of white box switch network measurement, solving the technical problem that the prior art cannot measure large flow network traffic, and can effectively and quickly measure network traffic in combination with white box switches, and improve the measurement accuracy of large flows through large flow thresholds.

[0140] Figure 5 A schematic diagram of the structure of a white box switch network measurement device provided by an embodiment of the present invention. Figure 5 As shown, the device includes: a traffic receiving module 510, a first hash module 520 and a large flow hash module 530; wherein,

[0141] The flow receiving module 510 is used to receive input target flow; wherein the target flow includes flow data and flow identification;

[0142] A first hash module 520 is used to map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information, and a large flow vote counter;

[0143] The large flow hash module 530 is used to map the target flow to the second hash list and the third hash list respectively according to the flow identifier of the target flow when it is detected that the counter value of the large flow vote counter is greater than a preset large flow threshold.

[0144] Optionally, the first hash module is specifically used for:

[0145] Performing hash calculation on the traffic identifier by using the primary hash function of the first hash list to determine a first hash number set;

[0146] Perform mapping position calculation according to the number of columns in the first hash list and the first hash number set to determine at least one first position bucket;

[0147] The target flow is inserted into the first position bucket, and a first flow flow value of the target flow is generated.

[0148] Optionally, the first hash module is further used for:

[0149] Inserting the flow data into the first location bucket;

[0150] Updating the counter value of the flow frequency counter of the first position bucket;

[0151] If the large flow identification information is a null value, the traffic identification is used as the large flow identification information; and the counter value of the large flow vote counter is updated;

[0152] If the large flow identification information is not a null value, the counter value of the large flow vote counter is updated according to the large flow identification information and the traffic identifier, and the large flow identification information is updated according to the counter value of the large flow vote counter.

[0153] Optionally, the first hash module is further used for:

[0154] If the counter value of the large flow vote counter is less than 0, the flow identifier is used as the large flow identifier information;

[0155] If the counter value of the large flow vote counter is not less than 0, the large flow identification information will be retained.

[0156] Optionally, the first hash module is further used for:

[0157] If the large flow identification information is the same as the flow identification, a flow value calculation is performed according to the counter value of the flow frequency counter and the counter value of the large flow vote counter by using a first preset flow value estimation algorithm to determine the first flow value;

[0158] If the large flow identification information and the traffic identification are different, a flow value is calculated according to the counter value of the traffic frequency counter and the counter value of the large flow vote counter by using a second preset flow value estimation algorithm to determine the first traffic flow value.

[0159] Optionally, the large flow hash module is specifically used for:

[0160] Identify the traffic identifier through a preset Bloom filter to determine a target identification result;

[0161] The target traffic is mapped to the second Hash list and the third Hash list according to the target identification result.

[0162] Optionally, the large flow hash module is further used for:

[0163] If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the second hash function of the second hash list to determine a second hash number set;

[0164] Perform mapping position calculation according to the number of columns in the second hash list and the second hash number set to determine at least one second position bucket;

[0165] The target flow is inserted into the second position bucket, and the first flow value is used as the counter value of the flow frequency counter of the second position bucket.

[0166] Optionally, the large flow hash module is further used for:

[0167] If the target identification result is that traffic has been inserted, the second position bucket corresponding to the target traffic is obtained in the second hash list, and the counter value of the traffic frequency counter of the second position bucket is updated.

[0168] Optionally, the large flow hash module is further used for:

[0169] If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the third hash function of the third hash list to determine a third hash number set;

[0170] Perform mapping position calculation according to the number of columns of the third hash list and the third hash number set to determine at least one third position bucket;

[0171] The target traffic is inserted into the third position bucket, the first traffic flow value is used as the counter value of the traffic frequency counter of the third position bucket, and the traffic identifier is updated to the binary counter.

[0172] Optionally, the device further includes a traffic query module, a data bucket access module and a traffic recovery module:

[0173] A traffic query module is used to perform traffic query in a preset traffic queue based on a query task for network traffic;

[0174] A data bucket access module, configured to access the third data bucket of the third hash list if the traffic queue is empty;

[0175] The traffic recovery module is used to recover the network traffic of the third data bucket if the counter value of the traffic frequency counter of the third data bucket is greater than a preset access traffic threshold to obtain query network traffic.

[0176] Optionally, the traffic recovery module is specifically used to: determine the bit rate of each of the binary counters according to the counter value of the traffic frequency counter of the third data bucket and the counter value of each of the binary counters;

[0177] updating the counter value of each of the binary counters according to the bit rate of the binary counter and a preset bit recovery threshold, and determining a recovery value of each of the binary counters;

[0178] The query network traffic is determined according to the restored value of each binary counter; wherein the query network traffic includes a query traffic identifier and query traffic data.

[0179] Optionally, the traffic recovery module is further configured to: obtain a second traffic flow value in the second hash list according to the query traffic identifier;

[0180] determining the flow identification value according to the counter value of the flow frequency counter of the third data bucket and the second flow estimation value;

[0181] If the traffic identification value is greater than a preset large flow identification threshold, the queried network traffic is considered to be large flow data, and threshold change information of the preset large flow threshold is obtained;

[0182] The preset large flow threshold is updated according to the threshold change information, and the query network flow is sent to the flow queue;

[0183] If the traffic identification value is less than a preset large flow identification threshold, the queried network traffic is considered to be small flow data, and a third traffic flow value is obtained in the first hash list according to the queried traffic identifier;

[0184] The preset large flow threshold is updated according to the third traffic flow value and the second traffic flow value, and threshold change information of the preset large flow threshold is recorded.

[0185] The white-box switch network measurement device provided in the embodiment of the present invention can execute the white-box switch network measurement method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0186] Figure 6 A schematic diagram of the internal architecture of a white box switch disclosed in an embodiment of the present invention.

[0187] like Figure 6As shown, the white box switch includes a general processing layer and a switch pipeline; the general processing layer includes SRAM (Static Random-Access Memory), DRAM (Dynamic Random-Access Memory) and CPU (Central Processing Unit); the switch pipeline includes an inlet pipeline and an outlet pipeline, and the inlet pipeline and the outlet pipeline include a parser, a buffer and an inverse parser. In order to meet the line rate processing requirements, there are some limitations on the packet processing pipeline. For example, at each stage, network traffic can only access limited computing and memory resources. The design of each stage is the same, and the resource type is also the same. In order to provide flexible match-action operations, each stage has a matching table for matching network traffic with specific values, followed by an action unit that executes a simple instruction set, depending on the output of the matching unit.

[0188] Each pipeline stage has multiple hardware hash function calls that are used to calculate hash functions on network traffic fields or metadata to support operations such as load balancing and table lookups. Each pipeline stage also has a fixed amount of SRAM that can be used to maintain state, such as a counter array. The state ALU (SALU, arithmetic logic unit) is a hardware resource that allows one read and one write operation to the state object in SRAM. Each SALU can be used for counter update operations, such as counter increment or decrement. Finally, each pipeline stage is also equipped with a certain amount of ternary content addressable memory (TCAM) that can be used for wildcard matching on header fields. Overall, the amount of these resources is fixed at hardware design time and is finite.

[0189] The switch pipeline has the advantage of fast speed because it is hardware processing, but its disadvantage is that its flexibility is poorer than software processing. The characteristics of white box switches are that they have the hardware processing capabilities of the switch pipeline and the CPU to implement general software processing, combining flexibility and processing speed.

[0190] White-box switches have limited computational capabilities for processing flow identifiers larger than 32 bits. The switch's SALU currently only supports basic operations (such as addition and subtraction) on variables up to 32 bits, but the candidate flow identifiers in the update exceed the capabilities of the SALU because the size of the flow identifier is usually more than 32 bits (for example, a 5-tuple key has 104 bits). To address this problem, the long-bit flow identifier can be split into multiple sub-flow identifiers, and these sub-keys can be processed using multiple stages and SALUs. For example, a 104-bit 5-tuple flow flow identifier can be split into three 32-bit sub-flow identifiers and an 8-bit sub-flow identifier, and each sub-flow identifier can be accessed using a SALU in each stage.

[0191] The switch pipeline supports a limited memory access model. First, to ensure smooth execution of the pipeline, the time budget for each memory access is limited, and only one read-modify-write operation is allowed for each variable. In addition, each memory block can only be accessed within the stage to which it belongs, which means that the memory area can only be accessed once when a packet passes through the pipeline. However, in the process of updating the traffic frequency counter and the large flow vote counter of a hash list, the operations on the traffic frequency counter and the large flow vote counter are interdependent: the operation of writing the large flow vote counter depends on the traffic frequency counter, and the operation of writing the traffic frequency counter depends on the large flow vote counter. If you want to update the traffic frequency counter and the large flow vote counter in one stage, you need to perform multiple reads and writes sequentially, which will break the time budget of one stage, so two stages are needed to update the traffic frequency counter and the large flow vote counter.

[0192] In a software environment, the number of hash calls is not a bottleneck because these calls are executed on demand on each packet, but the hash situation in hardware is different. In a hardware switch, all hash calls that appear in the code need to be pre-allocated because the line rate of all possible execution paths must be guaranteed, even if these hash calls do not necessarily need to be executed. Multiple short hash calls can be merged by calling a long hash to intercept K hash values.

[0193] In order to ensure that the pipeline can be executed smoothly, it is necessary to consider the number of pipeline resources consumed by the data structure. For hash call resources, assuming that up to 32-bit hash is supported, hash calls are required for the data structure of the row, and the length of each hash call is, so at least hash call resources are used, and at most 10 hash calls are configured for each pipeline stage on the switch. For SALU resources, the five-tuple flow key is split into 4 sub-flow identifiers, and 4 SALUs are required. Each bucket has a counter. Assuming that the large flow identification information and the large flow vote counter in the bucket need to be updated, the SALU resources required are, and at most 10 hash calls and 10 SALUs are configured for each pipeline stage on a general switch. SRAM has a maximum of 10MB per stage, which can support the data structure designed by the present invention.

[0194] Figure 7 FIG. 1 is a schematic diagram of a white box switch network traffic processing structure disclosed in an embodiment of the present invention. Figure 7 As shown: the first level of the three-level hash structure of the present invention is deployed on the switch pipeline. Since the three-level hash structure has multiple counters, multiple SALUs will be used to update the counters, so it cannot be deployed on the pipeline layer, and the second-level structure mainly interacts with the third-level structure, so the second-level hash structure and the third-level hash structure are deployed on the general processing layer.

[0195] When a data packet reaches the switch, it will enter the pipeline layer for processing. Each stage performs the corresponding hash call and counter operation through the matching action table stored in TCAM. Each stage first locates the counter stored in the specified bucket in SRAM through the hash function call, and then the SALU reads the counter value in SRAM and performs the corresponding operation action inside the SALU according to the matching action table in TCAM. After the operation is completed, the result is written to SRAM. The flow value estimation action will be completed at a certain stage. After completion, it is compared with the threshold stored in the matching table. If it exceeds the threshold, the flow is considered to be a large flow. The large flow report action is executed to transmit the key and value of the large flow to the update queue in the general processing layer so as to update the second hash list and the third hash list. If the threshold is not exceeded or the report is completed, the measurement at the switch is completed, and the data packet can be forwarded to the next switch.

[0196] When processing a query task, first query the traffic queue. If the queue is not empty, directly take out the key. If it is empty, it is necessary to restore the traffic identifier of the large flow to the third hash list and return the result to the queue. When the third hash list is restored, it needs to query the second traffic flow value from the second hash list. The second hash list returns the second traffic flow value to the third hash list to verify whether the restored traffic is a large flow. If a preset large flow threshold adjustment is required, the dynamic threshold adjustment is performed by configuring the matching table in the switch.

[0197] The present invention designs a three-level structure deployed in a white box switch. The first hash list is deployed in the switch pipeline for quickly measuring the data flow, and the second and third hash lists are deployed in the general processing layer for processing large flow queries, while improving the efficiency of the first hash list. This query and measurement separation method can ensure that the large flow query task is met without affecting the measurement efficiency.

[0198] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the white-box switch network measurement method provided in any embodiment of the present invention are implemented. The method includes:

[0199] Receive input target traffic; wherein the target traffic includes traffic data and traffic identifier;

[0200] Map the target traffic to the first position bucket in the first hash list according to the traffic identifier, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information and a large flow vote counter;

[0201] When it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

[0202] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0203] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0204] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0205] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0206] It should be understood by those skilled in the art that the above modules or steps of the present invention can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0207] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0208] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A white box switch network measurement method, characterized in that: Applicable to white box switches, including: Receive input target flow; wherein the target flow includes flow data and flow identifier; Map the target traffic to the first position bucket in the first hash list according to the traffic identifier of the target traffic, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information and a large flow vote counter; When it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, the target flow is mapped to the second hash list and the third hash list respectively according to the flow identifier of the target flow.

2. The method according to claim 1, characterized in that The step of mapping the target traffic to a first position bucket in a first hash list according to the traffic identifier of the target traffic and generating a first traffic flow value of the target traffic includes: Performing hash calculation on the traffic identifier by using the primary hash function of the first hash list to determine a first hash number set; Perform mapping position calculation according to the number of columns in the first hash list and the first hash number set to determine at least one first position bucket; The target flow is inserted into the first position bucket, and a first flow flow value of the target flow is generated.

3. The method according to claim 2, characterized in that The inserting the target flow into the first position bucket includes: Inserting the flow data into the first location bucket; Updating the counter value of the flow frequency counter of the first position bucket; If the large flow identification information is a null value, the traffic identification is used as the large flow identification information; and the counter value of the large flow vote counter is updated; If the large flow identification information is not a null value, the counter value of the large flow vote counter is updated according to the large flow identification information and the traffic identifier, and the large flow identification information is updated according to the counter value of the large flow vote counter.

4. The method according to claim 3, characterized in that The updating of the large flow identification information according to the counter value of the large flow vote counter includes: If the counter value of the large flow vote counter is less than 0, the flow identifier is used as the large flow identifier information; If the counter value of the large flow vote counter is not less than 0, the large flow identification information will be retained.

5. The method according to claim 4, characterized in that The generating the first flow value of the target flow includes: If the large flow identification information is the same as the flow identification, a flow value calculation is performed according to the counter value of the flow frequency counter and the counter value of the large flow vote counter by using a first preset flow value estimation algorithm to determine the first flow value; If the large flow identification information and the traffic identification are different, a flow value is calculated according to the counter value of the traffic frequency counter and the counter value of the large flow vote counter by using a second preset flow value estimation algorithm to determine the first traffic flow value.

6. The method according to claim 5, characterized in that The mapping of the target traffic to the second hash list and the third hash list respectively according to the traffic identifier of the target traffic includes: Identify the traffic identifier through a preset Bloom filter to determine a target identification result; The target traffic is mapped to the second Hash list and the third Hash list respectively according to the target identification result.

7. The method according to claim 6, characterized in that The mapping the target traffic to the second hash list according to the target identification result includes: If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the second hash function of the second hash list to determine a second hash number set; Perform mapping position calculation according to the number of columns in the second hash list and the second hash number set to determine at least one second position bucket; wherein the second position bucket also includes a traffic frequency counter; The target flow is inserted into the second position bucket, and the first flow value is used as the counter value of the flow frequency counter of the second position bucket.

8. The method according to claim 7, characterized in that The mapping the target traffic to the second hash list according to the target identification result also includes: If the target identification result is that traffic has been inserted, the second position bucket corresponding to the target traffic is obtained in the second hash list, and the counter value of the traffic frequency counter of the second position bucket is updated.

9. The method according to claim 6, characterized in that Mapping the target traffic to the third hash list according to the target identification result includes: If the target identification result is that no traffic is inserted, performing hash calculation on the traffic identifier according to the third hash function of the third hash list to determine a third hash number set; Perform mapping position calculation according to the number of columns of the third hash list and the third hash number set to determine at least one third position bucket; wherein the third position bucket also includes a traffic frequency counter and at least one binary counter; The target traffic is inserted into the third position bucket, the first traffic flow value is used as the counter value of the traffic frequency counter of the third position bucket, and the traffic identifier is updated to the binary counter.

10. The method according to claim 9, characterized in that In the case where it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold, after mapping the target flow to the second hash list and the third hash list respectively according to the flow identifier of the target flow, it also includes: Based on the query task of network traffic, traffic query is performed in the preset traffic queue; If the traffic queue is empty, accessing the third data bucket of the third hash list; If the counter value of the traffic frequency counter of the third data bucket is greater than the preset access traffic threshold, the network traffic of the third data bucket is restored to obtain the query network traffic.

11. The method according to claim 10, characterized in that The recovering the network traffic of the third data bucket to obtain the query network traffic includes: determining a bit rate of each of the binary counters according to a counter value of the traffic frequency counter of the third data bucket and a counter value of each of the binary counters; updating the counter value of each of the binary counters according to the bit rate of the binary counter and a preset bit recovery threshold, and determining a recovery value of each of the binary counters; The query network traffic is determined according to the restored value of each binary counter; wherein the query network traffic includes a query traffic identifier and query traffic data.

12. The method according to claim 11, characterized in that After the network traffic of the third data bucket is recovered to obtain the query network traffic, the method further includes: Acquire a second traffic flow value in the second hash list according to the query traffic identifier; determining a flow identification value according to a counter value of a flow frequency counter of the third data bucket and the second flow estimation value; If the traffic identification value is greater than a preset large flow identification threshold, the queried network traffic is considered to be large flow data, and threshold change information of the preset large flow threshold is obtained; The preset large flow threshold is updated according to the threshold change information, and the query network flow is sent to the flow queue; If the traffic identification value is less than a preset large flow identification threshold, the queried network traffic is considered to be small flow data, and a third traffic flow value is obtained in the first hash list according to the queried traffic identifier; The preset large flow threshold is updated according to the third traffic flow value and the second traffic flow value, and threshold change information of the preset large flow threshold is recorded.

13. A white box switch network measurement device, characterized in that: Applicable to white box switches, including: The flow receiving module is used to receive the input target flow; wherein the target flow includes flow data and flow identification; A first hash module is used to map the target traffic to a first position bucket in a first hash list according to the traffic identifier, and generate a first traffic flow value of the target traffic; wherein the first position bucket includes a traffic frequency counter, a large flow identifier information, and a large flow vote counter; The large flow hash module is used to map the target flow to the second hash list and the third hash list respectively according to the flow identifier of the target flow when it is detected that the counter value of the large flow vote counter is greater than the preset large flow threshold.

14. A white box switch, characterized in that: The white box switch comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the white-box switch network measurement method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the white-box switch network measurement method according to any one of claims 1 to 12 when executed.

Images