Traffic statistical method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a flow statistical method and device, electronic equipment, a storage medium and a program product, and relates to the technical fields of cloud technology, cloud computing, smart traffic, big data and the like. In each sampling period, after a first message sampling rate for a message actually sent by a network card is obtained, the first message sampling rate continues to be corrected based on the number of original messages to be sent in the period and the number of the messages actually sent by the network card, so that a more accurate second message sampling rate adapting to the original messages is obtained; the original message is sampled by using the second message sampling rate, so that the accuracy of the sampling process can be greatly improved; moreover, traffic statistics is carried out by using the total sampling data volume obtained by sampling, so that the problem of inaccurate sampling process caused by access of the original message and the actually sent message is avoided, and the accuracy of traffic statistics is effectively improved.

Description

Technical Field

[0001] The present application relates to technical fields such as cloud technology, cloud computing, smart transportation, and big data. The present application relates to a traffic statistics method, device, electronic device, storage medium, and program product. Background Art

[0002] With the development of network technology, the amount of data transmitted on the network is also increasing. In some scenarios in this field, it is usually necessary to collect traffic statistics of the transmitted data to analyze network behavior and diagnose problems.

[0003] In the related art, a set sampling rate is usually used to sample the messages to be sent, and after sampling, the traffic statistics of the sent messages are performed in combination with the sampling rate.

[0004] However, due to the complexity and variability of actual application scenarios, the messages to be sent and the actual messages sent are often different, or even have large discrepancies, which makes the sampling rate inconsistent with the actual situation, and further leads to inaccurate traffic statistics based on the sampling rate, that is, the accuracy of traffic statistics is low. Summary of the Invention

[0005] The present application provides a traffic statistics method, device, electronic device, storage medium and program product, which can effectively improve the accuracy of traffic statistics.

[0006] In one aspect, an embodiment of the present application provides a traffic statistics method, the method comprising:

[0007] In the process of periodically sampling messages sent by the network card, for each sampling period, obtaining the number of original messages and the number of sent messages corresponding to the sampling period, wherein the number of original messages is the number of original messages to be sent within the sampling period, and the number of sent messages is the number of sent messages actually sent within the sampling period;

[0008] Based on the number of sent messages, obtaining a first message sampling rate corresponding to the sent messages;

[0009] Based on the number of original messages and the number of sent messages, the first message sampling rate is modified to obtain a second message sampling rate corresponding to the original message;

[0010] Sampling the original messages within the sampling period based on the second message sampling rate, and counting the total sampled data volume of the original messages obtained by sampling;

[0011] Based on the total sampled data volume of the original messages obtained by the sampling, traffic statistics are performed on the messages actually sent during the sampling period.

[0012] On the other hand, an embodiment of the present application provides a traffic statistics device, the device comprising:

[0013] a first acquisition module configured to, during periodic sampling of messages sent by the network card, acquire, for each sampling period, a number of original messages and a number of sent messages corresponding to the sampling period, wherein the number of original messages is the number of original messages to be sent within the sampling period, and the number of sent messages is the number of sent messages actually sent within the sampling period;

[0014] A second acquisition module is used to acquire a first message sampling rate corresponding to the sent messages based on the number of sent messages;

[0015] A correction module, configured to correct the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original messages;

[0016] a sampling module, configured to sample the original messages within the sampling period based on the second message sampling rate, and calculate a total sampled data volume of the original messages obtained by sampling;

[0017] The first statistics module is configured to perform traffic statistics on the outgoing messages actually sent during the sampling period based on the total sampled data volume of the original messages obtained by sampling.

[0018] In a possible implementation, the network card enables a function of splitting a message before sending it, and the sent message is obtained by splitting an original message to be sent within the sampling period;

[0019] The correction module, when correcting the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original messages, includes:

[0020] a split change rate determining unit, configured to determine a split change rate based on the number of original messages and the number of sent messages, the split change rate indicating a degree of message change in the sent messages compared to the original messages due to the split operation;

[0021] A correction unit is used to correct the first message sampling rate based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message.

[0022] In one possible implementation, the network card segments the message using multiple segmentation methods;

[0023] The correction unit, when correcting the first message sampling rate based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message, is specifically configured to:

[0024] Obtaining a target segmentation method corresponding to the original message to be sent within the sampling period;

[0025] Determining a target correction function associated with a target segmentation method based on an association relationship between a plurality of segmentation methods and a correction function corresponding to each segmentation method;

[0026] Each segmentation method corresponds to a correction function, and each correction function represents a mapping relationship of mapping the first message sampling rate to the second message sampling rate based on the segmentation change rate;

[0027] Based on the segmentation change rate, the first message sampling rate is corrected to the second message sampling rate according to the mapping relationship represented by the target correction function.

[0028] In one possible implementation, the apparatus further includes:

[0029] a second statistics module, configured to count the amount of sampled data belonging to each message type in the sampled original message according to at least one message type;

[0030] The first statistics module is specifically configured to, when performing traffic statistics on the outgoing messages actually sent during the sampling period based on the total sampled data volume of the original messages obtained by sampling,:

[0031] Determining a flow sampling rate for the original message based on the total sampled data volume of the original message obtained by the sampling and the total data volume of the message sent within the sampling period;

[0032] Based on the traffic sampling rate and the sampled data volume of each message type in the sampled original message, the sent data volume of each message type in the sent messages within the sampling period is obtained by statistics.

[0033] In one possible implementation, the apparatus further includes a third acquisition module, configured to acquire at least one message type;

[0034] The third acquisition module, when acquiring at least one message type, is specifically used for any of the following:

[0035] Performing statistics on at least one quintuple corresponding to the sampled original message to obtain the at least one message type;

[0036] Counting at least one service identifier corresponding to the sampled original message to obtain the at least one message type;

[0037] Statistics are collected on at least one quality of service (QoS) priority corresponding to the sampled original message to obtain the at least one message type.

[0038] In one possible implementation, the sampled original message includes multiple messages;

[0039] The second statistical module, when counting the amount of sampled data belonging to each message type in the sampled original message according to at least one message type, is specifically configured to:

[0040] Extracting a message type of a corresponding message from each message obtained by the sampling;

[0041] For each message obtained by sampling, based on the message type of each message and in accordance with the pre-configured association relationship between the message type and the sampled data amount, the data amount of each message is accumulated to the sampled data amount associated with the message type of each message.

[0042] In one possible implementation, the apparatus further includes a fourth acquisition module, configured to acquire a total data volume of sent messages within a sampling period;

[0043] The third acquisition module is specifically used for any of the following when acquiring the total data volume of the messages sent within the sampling period:

[0044] estimating a total data volume of messages sent within the sampling period based on the second message sampling rate and a total sampled data volume of the original messages obtained by sampling;

[0045] A statistical interface for the network card is called based on the sampling period to obtain a total data volume of messages sent by the network card during the sampling period.

[0046] In one possible implementation, the sampling module, when sampling the original messages within the sampling period based on the second message sampling rate and counting the total sampled data volume of the sampled original messages, is specifically configured to:

[0047] Sampling the IP headers of the original messages within the sampling period according to the second message sampling rate to obtain the sampled IP headers of the original messages;

[0048] Based on the amount of message data extracted from the IP header of the sampled original message, the total sampled data amount of the sampled original message is obtained by statistics.

[0049] On the other hand, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-mentioned traffic statistics method.

[0050] On the other hand, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned traffic statistics method is implemented.

[0051] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned traffic statistics method when executed by a processor.

[0052] The traffic statistics method provided in the present application obtains a first message sampling rate for messages actually sent by the network card in each sampling period, and then continues to correct the first message sampling rate based on the number of original messages to be sent in the period and the number of messages actually sent by the network card to obtain a second message sampling rate that is more accurate and suitable for the original messages; using the second message sampling rate to sample the original messages can greatly improve the accuracy of the sampling process; and using the total sampled data volume obtained by sampling to perform traffic statistics, thereby avoiding the problem of inaccurate sampling process caused by discrepancies between the original messages and the actually sent messages, thereby effectively improving the accuracy of traffic statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of an implementation environment for a traffic statistics method provided in an embodiment of the present application;

[0054] Figure 2 A flow chart of a traffic statistics method provided in an embodiment of the present application;

[0055] Figure 3 A flow chart of another method for calculating traffic statistics provided in an embodiment of the present application;

[0056] Figure 4 A flow chart of another method for calculating traffic statistics provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the execution flow of a traffic statistics process provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a flow statistics process provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the accuracy of a related technology provided in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of an error rate of a related technology provided in an embodiment of the present application;

[0061] Figure 9 A schematic diagram of experimental data of the present application provided in an embodiment of the present application;

[0062] Figure 10 A schematic diagram of the structure of a flow statistics device provided in an embodiment of the present application;

[0063] Figure 11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0065] Those skilled in the art will understand that, unless otherwise specified, the singular forms "a," "an," "the," and "the" used herein may also include the plural forms. The terms "including" and "comprising" used in the embodiments of this application mean that the corresponding features can be implemented as the presented features, information, data, steps, and operations, but do not exclude the implementation of other features, information, data, steps, operations, etc. supported by the technical field.

[0066] It is understandable that in the specific implementation of the present application, any user-related data such as the user's message (such as the original message to be sent, the actual message sent), the number of messages, the amount of data for traffic statistics of the message, the five-tuple in the message, the service identifier, the quality of service QoS priority, etc., when the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain user permission or consent, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if any user-related data is involved in the embodiments of the present application, these data need to be obtained with the authorization and consent of the user and in compliance with the relevant laws, regulations and standards of the country and region.

[0067] Figure 1 This is a schematic diagram of the implementation environment of a traffic statistics method provided by this application. Figure 1 As shown, the implementation environment includes: an electronic device 101, which can be a server, a terminal, or a cloud computing center device, etc.

[0068] The electronic device 101 may be configured with an application for executing the traffic statistics method of the present application. The application may be an independent application or a program plug-in installed in an independent application. For example, the application may be a management platform with a traffic statistics function. Of course, the management platform may also have functions such as traffic analysis and traffic billing. The present application does not limit this.

[0069] The electronic device 101 may adopt a Linux system, and the electronic device 101 includes a Linux-based kernel and a network card; wherein the Linux-based kernel may send a message to be sent to the network card, and the message is sent out via the network card.

[0070] In one possible scenario, the network card may be able to fragment packets before sending them. For example, the network card may support TSO (TCP Segmentation Offload). This TSO feature is designed to improve kernel protocol stack performance. The kernel directly sends large packets to the network card, which then fragments them based on the maximum transmission unit (MTU) to reduce the kernel protocol stack's fragmentation overhead.

[0071] The traffic statistics method provided in the present application can, when the TSO function of the network card is enabled, after obtaining the first message sampling rate for the messages actually sent by the network card, continue to correct the first message sampling rate based on the number of original messages to be sent and the number of messages actually sent by the network card to obtain a more accurate second message sampling rate corresponding to the original message; use the second message sampling rate to sample the original message, and use the total sampled data volume obtained by sampling to perform traffic statistics, thereby avoiding the problem of inaccurate sampling process caused by discrepancies between the original message and the actually sent message, thereby effectively improving the accuracy of traffic statistics.

[0072] It should be noted that a server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The above-mentioned networks may include, but are not limited to, wired networks and wireless networks, wherein the wired networks include local area networks, metropolitan area networks, and wide area networks, and the wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication.

[0073] The terminal can be a smart phone (such as an Android phone, iOS phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (Personal Digital Assistant), a desktop computer, a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal, a vehicle-mounted computer, etc.), a smart speaker, a smart watch, etc. The specific requirements can also be determined based on the actual application scenario and are not limited here.

[0074] Figure 2 This is a flow chart of a traffic statistics method provided in an embodiment of the present application. The execution subject of this method can be an electronic device. Figure 2 As shown, the method includes S201-S205.

[0075] S201. The electronic device obtains the number of original messages and the number of sent messages corresponding to each sampling period during the process of periodically sampling messages sent via the network card.

[0076] The number of original messages is the number of original messages to be sent within the sampling period, and the number of sent messages is the number of sent messages actually sent within the sampling period.

[0077] In this application, if there is an original message to be sent in the electronic device, the kernel of the electronic device can send the original message to the network card, and the network card can send it out. The electronic device can also sample the original message to use the sampling situation to calculate the flow statistics of the message actually sent by the network card.

[0078] When the kernel of the electronic device sends the original message to the network card, the electronic device may copy the original message passing through the network card to a buffer so as to sample the original message stored in the buffer.

[0079] In this step, for each sampling period, the electronic device can count the original messages stored in the buffer to obtain the original message number of the original messages within the sampling period; for example, the electronic device can, based on the timestamps of the original messages in the buffer and the sampling period, take the total number of original messages whose timestamps are within the sampling period as the number of original messages corresponding to the sampling period.

[0080] In this step, the electronic device may obtain the number of messages actually sent by the network card during the sampling period by calling a statistics interface. The statistics interface may be an interface for counting messages actually sent by the network card. For example, the statistics interface may be called based on the sampling period to obtain the number of messages sent within the sampling period, and the number of messages sent is used as the number of messages corresponding to the sampling period.

[0081] It should be noted that the sampling period is a time window used to limit the number of packets. At the end of each sampling period, the number of original packets and actual packets sent within that sampling period can be counted. For example, a sampling period can be 30ms long. For example, the first sampling period can be a time interval from 1ms to 30ms. At the end of 30ms, the number of original packets to be sent and the number of packets actually sent by the network card from 1ms to 30ms can be counted.

[0082] S202: The electronic device obtains a first message sampling rate corresponding to the sent messages based on the number of sent messages.

[0083] In this application, the message sampling rate refers to the sampling interval for each message collected; for example, if 1 sample is taken for every 5 messages, the message sampling rate is 5.

[0084] In one possible scenario, if the number of original messages to be sent differs from the number of outgoing messages actually sent by the network card, the first message sampling rate is obtained based on the number of outgoing messages actually sent by the network card, and therefore is a sampling rate applicable to outgoing messages. However, since actual sampling is performed on the original messages in the buffer, after obtaining the first message sampling rate, the present application further corrects the first message sampling rate to a second message sampling rate applicable to the original messages in S203.

[0085] In one possible approach, the electronic device may determine the first message sampling rate based on the number of sent messages and the target number of sampled messages. Exemplarily, the target number of sampled messages is the number of messages expected to be sampled corresponding to the sampling period.

[0086] For example, the electronic device may calculate the first message sampling rate based on the number of sent messages and the target number of sampled messages using the following formula 1:

[0087] Formula 1: R1=N / n;

[0088] Wherein, R1 represents the first message sampling rate, N represents the number of sent messages, and n represents the target sampling message number.

[0089] Exemplarily, the electronic device may determine the first message sampling rate based on the number of sent messages and a preconfigured number of sampled messages. That is, n may be a preconfigured specified value, for example, n may be a preconfigured default value, such as 300, 500, etc. If n = 300, then the first message sampling rate may be directly calculated based on R1 = N / 300 to obtain R1.

[0090] S203: The electronic device corrects the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original messages.

[0091] In this step, the electronic device may correct the first message sampling rate based on the relative size between the number of original messages and the number of sent messages to obtain a second message sampling rate.

[0092] For example, if the ratio of the number of outgoing messages to the number of original messages is greater than 1, that is, the number of outgoing messages is greater than the number of original messages, the first message sampling rate can be reduced to obtain a second message sampling rate that is less than the first message sampling rate. Based on this, the sampling interval for sampling the original messages can be shortened, thereby increasing the number of messages actually sampled. This allows the actual sampling rate and number of sampled messages to better adapt to situations where the number of messages actually sent by the network card is greater than the number of original messages.

[0093] If the ratio of the number of sent packets to the number of original packets is less than 1, that is, the number of sent packets is less than the number of original packets, the first packet sampling rate can be increased to obtain a second packet sampling rate that is greater than the first packet sampling rate. Based on this, the sampling interval for sampling the original packets can be increased, thereby reducing the number of packets actually sampled. This allows the actual sampling rate and the number of sampled packets to better cope with the situation where the number of packets actually sent by the network card is smaller than the number of original packets.

[0094] In a possible implementation, the network card enables the function of splitting the message before sending. The message actually sent by the network card is obtained by splitting the original message to be sent within the sampling period. Figure 3 As shown, S203 may include the following S2031-S2032:

[0095] S2031: The electronic device determines a segmentation change rate based on the number of original messages and the number of sent messages.

[0096] The split change rate indicates the degree of change in the sent message compared to the original message caused by the split operation.

[0097] For example, the electronic device may obtain the ratio between the number of sent messages and the number of original messages, and use the ratio as the split change rate. For example, the electronic device may calculate the split change rate using the following formula 2:

[0098] Formula 2: T = N / nCAP;

[0099] T represents the segmentation change rate, which can be the sampling error ratio caused by the NIC splitting packets before sending them. N represents the number of sent packets, which is the actual number of packets sent by the NIC. nCAP represents the number of original packets, which is the total number of original packets counted in the buffer during the current sampling period.

[0100] It should be noted that the segmentation change rate represents the overall segmentation degree of the original message within the sampling period. For example, the original message to be sent within the sampling period may include multiple messages. The network card can use the TSO function to segment the messages that meet the conditions among the multiple messages according to the pre-configured segmentation method.

[0101] In one example, the network card can adopt an equal splitting method. For multiple messages to be sent, the message to be split whose message data volume is greater than the target threshold can be evenly split into multiple small messages with equal data volume. Among them, the data volume of each small message obtained by splitting the message to be split is equal and less than the target threshold. For example, the target threshold can be the maximum length threshold of the data link layer for the data frame length limit, that is, the MTU (Maximum Transmission Unit). For example, if the MTU is 200 bytes and the message to be split is 300 bytes, the message to be split can be split into 2 small messages of 160 bytes.

[0102] In another example, the network card can adopt an unequal splitting method. For multiple messages to be sent, the message to be split whose message data volume is greater than the target threshold can be split into multiple small messages; wherein the data volume of the multiple small messages after splitting can be unequal. For example, the electronic device can randomly split the message to be split into multiple small messages whose data volume is less than the target threshold. For example, if the MTU is 200 bytes and the message to be split is 300 bytes, the message to be split can be split into one small message of 190 bytes and one small message of 130 bytes.

[0103] S2032: The electronic device corrects the first message sampling rate based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message.

[0104] For example, the electronic device may modify the first message sampling rate based on the segmentation change rate. If the segmentation change rate is greater than 1, the first message sampling rate is reduced; if the segmentation change rate is less than 1, the first message sampling rate is increased. The original message is then sampled at a sampling interval corresponding to the second message sampling rate, so that the number of sampled messages matches the number of messages actually sent.

[0105] For example, the electronic device may use the following formula 3 to calculate the second message sampling rate:

[0106] Formula 3: R2 = R1 / T;

[0107] Wherein, R2 represents the second message sampling rate; R1 represents the first message sampling rate; and T represents the segmentation change rate.

[0108] In one possible approach, the network card uses multiple segmentation methods to segment the message; the present application also supports selecting a correction method for the first message sampling rate based on the segmentation method. Accordingly, the implementation of S2032 may include S2032-1 to S2032-3:

[0109] S2032-1. Obtain the target segmentation method corresponding to the original message to be sent within the sampling period;

[0110] For example, the multiple splitting methods can be configured based on needs, and this application does not limit this. For example, the multiple splitting methods may include but are not limited to: an equal splitting method and an unequal splitting method. The target splitting method can be an equal splitting method or an unequal splitting method. These two splitting methods have been introduced in S2031 and will not be repeated here.

[0111] For another example, the multiple splitting methods may also include but are not limited to: splitting at the beginning of each sampling period, and splitting after a specified period of time has passed since the beginning of each sampling period; for example, for a period of 30ms, the first original message in the period may be split at the beginning of each period; or, the original message from the 5th ms in the period may be split 5ms after the beginning of each period, and the original messages before the 5th ms are not split.

[0112] Of course, the multiple segmentation methods may also include other methods, and various segmentation methods may be configured in combination with actual scene requirements. This application only uses the above examples for illustration, but does not limit the specific possible methods of the multiple segmentation methods.

[0113] S2032-2. Determine a target correction function associated with a target segmentation method based on the association relationship between the plurality of segmentation methods and the correction function corresponding to each segmentation method;

[0114] Each segmentation method corresponds to a correction function, and each correction function represents a mapping relationship of mapping the first message sampling rate to the second message sampling rate based on the segmentation change rate.

[0115] S2032-3. Based on the segmentation change rate, and in accordance with the mapping relationship represented by the target correction function, correct the first message sampling rate to the second message sampling rate.

[0116] In the present application, a correction function corresponding to each splitting method can be pre-configured, and the correction function represents the correction logic for correcting the first message sampling rate based on the splitting change rate; that is, a mapping relationship based on the splitting change rate from the first message sampling rate to a sampling rate that is more in line with the actual situation; the actual situation includes but is not limited to: the number of messages actually sent by the network card, the splitting method of the original message actually split by the network card, etc.

[0117] Exemplarily, taking equal segmentation and unequal segmentation as examples, the correlation between different segmentation methods and the correction function is illustrated.

[0118] For example, if the target segmentation method is equal segmentation, the associated target correction function can be expressed as the following formula 4:

[0119] Formula 4: R2 = (a / T)R1;

[0120] Where R2 represents the second message sampling rate; R1 represents the first message sampling rate; T represents the slicing change rate; and a represents the first error parameter. The parameter value of a can be configured based on the evaluation results of multiple historical sampling cycles. In Formula 3, a = 1.

[0121] In S2032-3, based on the mapping relationship of the correction function in Formula 4, the first message sampling rate may be corrected to the second message sampling rate.

[0122] It should be noted that if equal splitting is adopted, in fact, for each small message of equal size obtained after splitting the original message, an additional IP (Internet Protocol) header needs to be added to each small message actually sent, and during actual sampling, only one IP header of one original message corresponding to each small message will be sampled; for one original message corresponding to each small message, it is equivalent to sending multiple IP headers when it is actually sent, and the ratio of the additional IP headers in each small message to the data volume of the small message is fixed and equal; that is, the actual amount of data sent is relatively fixed relative to the data volume of the original message, and therefore, the correction function with a fixed ratio between R2 and R1 in formula 4 can be used to accurately correct the first message sampling rate.

[0123] For another example, if the target segmentation method is unequal segmentation, the associated target correction function can be expressed as the following formula 5:

[0124] Formula 5: R2 = (a / T)R1 + b;

[0125] Where R2 represents the second message sampling rate; R1 represents the first message sampling rate; T represents the slicing change rate; a represents the first error parameter; and b represents the second error parameter. The parameter values ​​of a and b can be configured based on the evaluation results of multiple historical sampling cycles. In Formula 3, a = 1 and b = 0.

[0126] In S2032-3, the first message sampling rate may be corrected to the second message sampling rate based on the mapping relationship of the correction function in Formula 5.

[0127] It should be noted that if unequal splitting is adopted, the small messages of different sizes obtained after splitting the original message will have different proportions of the additional IP header in the corresponding message; based on this, the second error parameter b in the linear function of formula 5 can be used to further compensate for the error in the case of unequal splitting, thereby effectively correcting the first message sampling rate to a sampling rate closer to the actual situation, thereby improving the accuracy of the sampling rate.

[0128] S204: The electronic device samples the original messages within the sampling period based on the second message sampling rate, and counts the total sampled data volume of the original messages obtained by sampling.

[0129] Exemplarily, the electronic device may sample original messages in the buffer whose timestamps fall within the sampling period according to the second message sampling rate. For example, if the first message sampling rate is 1 sample per 10 messages, and after correction, the second message sampling rate is 1 sample per 5 messages, the electronic device may sample 1 original message from the buffer for every 5 original messages.

[0130] In a possible implementation, the electronic device may sample the IP header of the original message and perform statistics on the sampled original message based on the information carried in the IP header. For example, S204 may include S2041 to S2042:

[0131] S2041. Sample the IP headers of the original messages within the sampling period according to the second message sampling rate to obtain sampled IP headers of the original messages.

[0132] S2042: Based on the packet data volume extracted from the IP header of the sampled original packet, obtain the total sampled data volume of the sampled original packet through statistics.

[0133] For example, whenever an original message is sampled, the electronic device may start collecting data of the IP header of the original message from the start bit of the original message, and the collection ends when the collection of the IP header data is completed.

[0134] During actual data collection, the electronic device can set a maximum data length for collection. The maximum sampled data length is the maximum amount of data that can be collected from a single original message. While collecting data from the IP header of the original message, the electronic device can count the length of data collected for the original message in real time. If the collected data length does not exceed the maximum data length, the electronic device will continue collecting data until the IP header data is collected, and then the collection for the original message will be terminated. If the collected data length exceeds the maximum data length, the collection will be terminated.

[0135] The electronic device may configure the maximum sampling data length as a target length threshold, and the target length threshold may be configured based on needs, which is not limited in this application.

[0136] In one possible scenario, a VxLAN (Virtual eXtensible Local Area Network) overlay network includes a Layer 2 IP header. Therefore, a target length threshold can be set to a value no less than the Layer 2 IP header's data size. For example, the target length threshold could be 135 bytes. This allows parsing of the VxLAN packet down to the inner transport layer, resulting in the VxLAN packet's quintuple.

[0137] The IP header includes the packet data volume of the original packet. For each sampled packet, the total packet data volume of each packet can be calculated, which is also the total sampled data volume.

[0138] S205: The electronic device performs traffic statistics on the messages actually sent during the sampling period based on the total sampled data volume of the original messages obtained through the sampling.

[0139] For example, the electronic device may further, based on the total sampled data volume, categorize and calculate traffic statistics for different types of messages actually sent during the sampling period. Alternatively, the electronic device may also, based on the total sampled data volume, calculate the total data volume of messages actually sent during the sampling period.

[0140] In a possible implementation, when the electronic device samples the original message based on the second message sampling rate, it can also statistically analyze the amount of sampled data corresponding to each message type obtained by sampling in different categories. Correspondingly, Figure 4 As shown, before S205, S3 is also included:

[0141] S3. The electronic device counts, according to at least one message type, the amount of sampled data belonging to each message type in the sampled original message;

[0142] Exemplarily, the original message is a general term for such unsegmented messages to be sent, relative to the actually sent messages. When the electronic device samples the original message, the sampled original messages contain multiple messages, that is, multiple unsegmented messages are sampled in the buffer; then, according to at least one message type, the amount of sampled data belonging to each message type in the sampled original message is counted, including:

[0143] Extracting a message type of a corresponding message from each message obtained by the sampling;

[0144] For each message obtained by sampling, based on the message type of each message and in accordance with the pre-configured association relationship between the message type and the sampled data volume, the data volume of each message is accumulated to the sampled data volume associated with the message type of each message.

[0145] For example, the message type may be extracted from the IP header of the sampled message.

[0146] For example, the electronic device may associate and store each message type and the corresponding sampled data volume for each message type in a map. Based on this, whenever the electronic device samples a message, it may search the map for the sampled data volume associated with the message type of the message, and add the message data volume of the message to the sampled data volume found.

[0147] It should be noted that the execution order of S3 and S2042 is not specific. S3 can be executed first and then S2042; S2042 can be executed first and then S3, or they can be executed simultaneously. Figure 4 Only one case is shown as an example, and this application does not limit this.

[0148] Accordingly, in S205, based on the sampled data volume of each message type, the traffic statistics of the actually sent messages can be further classified into different categories. Specifically, Figure 4 As shown, S205 may include S2051 to S2052:

[0149] S2051. The electronic device determines a flow sampling rate for the original message based on the total sampled data volume of the original message obtained by sampling and the total data volume of the messages sent within the sampling period;

[0150] For example, the traffic sampling rate indicates the data volume interval of each sampling unit of traffic. For example, if the traffic sampling rate is 50 bytes, it means that 1 byte of data is sampled every 50 bytes.

[0151] Exemplarily, the electronic device may calculate the traffic sampling rate based on the total sampled data volume and the total data volume of messages sent during the sampling period using the following formula 6:

[0152] Formula 6: Rb = Mb / mb;

[0153] Where Rb represents the traffic sampling rate; Mb represents the total amount of data in packets sent by the network card during the sampling period; and mb represents the total amount of sampled data in the original packets obtained by sampling.

[0154] Exemplarily, a method for obtaining the total data volume of messages sent during the sampling period includes any one of the following methods 1 and 2:

[0155] Method 1: based on the second message sampling rate and the total sampled data volume of the original message obtained by sampling, the total data volume of the message sent in the sampling period is estimated;

[0156] Method 2: Based on the sampling period, a statistical interface for the network card is called to obtain the total data volume of messages sent by the network card during the sampling period.

[0157] In mode 1, the electronic device may estimate the total data volume of the messages sent within the sampling period using the following formula 7 based on the second message sampling rate and the total sampled data volume of the original messages obtained by sampling:

[0158] Formula 7: S = R2*mb;

[0159] Wherein, S represents the total data volume of the messages sent by the network card in the sampling period; R2 represents the second message sampling rate; and mb represents the total sampled data volume of the original messages obtained by sampling.

[0160] In mode 2, the electronic device may obtain the total data volume by calling a statistical interface. For example, the electronic device may call a statistical interface based on the sampling period to obtain the total data volume of the messages sent within the sampling period.

[0161] S2052: The electronic device obtains, based on the traffic sampling rate and the sampled data volume of each message type in the original message obtained by sampling, the amount of sent data of each message type in the messages sent within the sampling period.

[0162] For example, for any message type, the electronic device may calculate the amount of data sent for the message type based on the traffic sampling rate and the amount of sampled data of the message type using the following formula 8:

[0163] Formula 8: S = Rb*Nb;

[0164] Where S represents the amount of data sent for this packet type, that is, the total amount of data of all packets of this packet type sent by the network card during the sampling period. Rb represents the traffic sampling rate. Nb represents the sampled data volume of this packet type, that is, the total amount of data of all packets of this packet type in the sampled original packets.

[0165] It should be noted that when performing traffic statistics, the traffic sampling rate is used instead of the packet sampling rate. Compared with the packet sampling rate, the traffic sampling rate can measure the sampling interval in a more fine-grained manner from the perspective of data volume, thereby making the traffic statistics results closer to the actual situation and further improving the accuracy of traffic statistics.

[0166] In one possible implementation, the message type may be determined by any one of a quintuple of the message, a service identifier, or a quality of service (QoS) priority; accordingly, a method for obtaining the at least one message type includes any one of the following methods 1 to 3:

[0167] Method 1: Count at least one quintuple corresponding to the sampled original message to obtain the at least one message type;

[0168] Method 2: Count at least one service identifier corresponding to the sampled original message to obtain the at least one message type;

[0169] Mode 3: Count at least one quality of service (QoS) priority corresponding to the sampled original message to obtain the at least one message type.

[0170] Exemplarily, the electronic device may also extract at least one of the five-tuple, service identifier, or QoS priority of the original message from the IP header of the sampled original message.

[0171] In method 1, the message type is determined by the quintuple of the original message. Different quintuples correspond to different message types, and each message type is represented by a quintuple. Each quintuple corresponding to each message in the sampled original message can be used as the message type. Based on this, the data volume of messages with different quintuples sent by the network card can be counted separately.

[0172] In method 2, the message type can be determined by the service identifier of the original message. Different service identifiers correspond to different message types, and each message type corresponds to a service identifier. Therefore, the service identifiers corresponding to each message in the sampled original message can be used as the message type. Based on this, the data volume of messages with different service identifiers sent by the network card can be counted separately.

[0173] In method 3, the packet type is determined by the QoS priority of the original packet. Different QoS priorities correspond to different packet types, and each packet type is associated with a specific QoS priority. Therefore, the QoS priorities corresponding to each packet in the sampled original packet can be used as the packet type. Based on this, the data volume of packets with different QoS priorities sent by the network card can be counted separately.

[0174] The following combination Figure 5 The execution flow shown here introduces the traffic statistics process of this application:

[0175] like Figure 5 As shown, the traffic statistics process of this application may include:

[0176] 1. Initialize sampling parameters:

[0177] Set the sampling filter mode to the outbound direction of the network card (outbound and (not broadcast and not multicast));

[0178] Set the maximum length of collected data to the target length threshold. For example, in a VxLAN network scenario based on an overlay network, set the maximum length of collected data to 135 bytes.

[0179] 2. Set the sampling timestamp accuracy, sampling timeout, and buffer size:

[0180] The sampling timestamp accuracy is set to microseconds; the sampling timeout is 10ms. The sampling timeout refers to the waiting time from the start time of the sampling cycle to the start of sampling the message within a sampling cycle. For example, if no original message appears after waiting for 10ms from 1ms to 30ms in a sampling cycle, that is, the message cannot be sampled, the waiting time is stopped and the sampling process corresponding to the sampling cycle ends. The buffer size is set to 30M.

[0181] 3. Set the maximum number of maps to be stored;

[0182] Taking the map storing five-tuple traffic statistics as an example, the maximum number of maps storing five-tuple traffic statistics is 4096;

[0183] 4. Get the statistics of the messages currently sent by the network card:

[0184] last_tx_packets and last_tx_bytes, respectively, indicate the sequence number of the last message in the previous sampling period and the number of bytes of the last message;

[0185] 5. Enter the periodic sampling loop:

[0186] First, obtain the start_time timestamp of this cycle; it is used to control the duration of a single sampling cycle to be within 30ms;

[0187] 6. Set the target number of sampling messages n:

[0188] Set n = 300;

[0189] 7. Initialize the total amount of sampled data collected in the current cycle:

[0190] Initialize the total amount of sampled data collected in the current cycle mb = 0;

[0191] 8. Get the statistics of the messages sent to the network card in the current cycle:

[0192] tx_packets and tx_bytes respectively represent the sequence number and byte count of the first message in the current sampling period, and are accumulated as messages are sent in the current sampling period.

[0193] At the end of the current sampling period, calculate the number of packets to be processed in the current period, where N = tx_packets – last_tx_packets for the number of packets actually sent by the network card; Mb = tx_bytes – last_tx_byte for the total amount of data in the packets actually sent by the network card; then, save the last statistics, making last_tx_packets equal to tx_packets and last_tx_bytes equal to tx_bytes;

[0194] 9. Calculate the first message sampling rate and the segmentation change rate, correct the first message sampling rate to the second message sampling rate based on the segmentation change rate, and perform sampling according to the second message sampling rate:

[0195] ①: Get the total number of original messages in the current cycle nCAP obtained from the statistics in the buffer; calculate the sampling rate R1 = N / n, T = N / nCAP; and make R2 = R1 / T;

[0196] ②: Start sampling the original messages in the buffer, that is, collect one original message every R2 original messages;

[0197] ③: Whenever an original message is collected, if n>1, set n=n-1 and parse the five-tuple of the sampled message;

[0198] ④: Find the sampled data volume Nb associated with the corresponding quintuple in the map, and add the data volume of the sampled message to the sampled data volume Nb associated with the quintuple;

[0199] ⑤: Add the data volume of the sampled message to the total sampled data volume mb corresponding to the current cycle;

[0200] ⑥: Compare the message timestamp with the current time. If the message timestamp is greater than or equal to the current time, it means that the current cycle has ended, and the sampling process of the current cycle ends;

[0201] 10. Calculate the traffic sampling rate: Rb = Mb / mb;

[0202] 11. Count the data volume of each five-tuple actually sent by the network card:

[0203] ①: Traverse the quintuple map, extract the correlation between each quintuple and the amount of sampled data, and calculate the total amount of data sent by each quintuple during the sampling period: S = Rb * Nb;

[0204] ②: Report to the analysis server: Use the quintuple as the key and the traffic flow S as the value, record a piece of data corresponding to each quintuple, and report each piece of data to the traffic analysis server;

[0205] 12. Control the current cycle duration within 30ms:

[0206] Get the timestamp end_time again;

[0207] Calculate the time consumption of this cycle tcap (ms) = end_time-start_time, sleep (30-tcap); based on this, the time consumption of packet capture in this cycle can be controlled within 30ms, thereby indirectly controlling the CPU consumption of this acquisition program at a low level.

[0208] 13. Repeat step 5.

[0209] Below is Figure 6 The flow chart shown in the figure further introduces the sampling process and circulation statistics process of this application. Figure 6 As shown, taking a 30ms sampling period as an example, for the current period, the total number of original packets, nCAP, and the actual number of outgoing packets, N, are obtained. The segmentation change rate, T, is then calculated. Based on the segmentation change rate, T, the first packet sampling rate is modified to achieve R2 = R1 / T. This makes the second packet sampling rate, R2, more suitable for the actual outgoing packets of the network card. Sampling is performed based on R2, and combined with the total sampled data volume based on R2, traffic statistics are generated for the actual outgoing packets sent in the current period. This effectively improves the accuracy of traffic statistics.

[0210] The following combination Figure 7 and Figure 8 , analyze the accuracy and error rate of related technologies. Figure 7 The accuracy of the traffic statistics method in the relevant technology; Figure 8 It is the error rate of traffic statistics using related technical methods.

[0211] like Figure 7As shown, in the related art, R=N / n is used to calculate the sampling rate, and sampling is performed according to R, where N is the number of packets actually sent and n is the number of sampled packets; S=R*mb is used to count the traffic, where mb is the collected traffic and S is the total traffic counted. However, when the TSO function is turned on for the Linux network card, the kernel collects large packets when collecting packets, while the network card sends small packets after segmentation, which makes R too large, resulting in the number of collected packets being much smaller than the number of packets actually sent by the network card; since the sampling rate R is too large due to the increase in N, after collecting one packet every R packets, the number of collected packets is too small; and the statistical traffic S is too large because R is too large. As Figure 7 As shown, in related technologies, the statistical traffic is much larger than the actual traffic in real situations, and the accuracy is only about 20% to 30%;

[0212] like Figure 8 As shown, in the related art, when the TSO function is enabled on the Linux network card, the sampling error rate is relatively large, and can reach the order of 100 at most.

[0213] Figure 9 The error rate of traffic statistics using the method of this application is as follows: Figure 9 As shown, after the TSO function is enabled on the Linux network card, when the method of the present application is used for sampling and traffic statistics, the accuracy of the kernel-based traffic collection is greatly improved, and the error rate is greatly reduced. The error rate has dropped from the original average error rate of about 50% to an average of less than 5%. The accuracy of the method of the present application is significantly improved, which meets the need for accurate traffic statistics in the scenario of massive server deployment without intruding on the business or modifying the network card configuration, and further saves costs while improving the accuracy as much as possible.

[0214] The traffic statistics method provided in the present application obtains a first message sampling rate for messages actually sent by the network card in each sampling period, and then continues to correct the first message sampling rate based on the number of original messages to be sent in the period and the number of messages actually sent by the network card to obtain a second message sampling rate that is more accurate and suitable for the original messages; using the second message sampling rate to sample the original messages can greatly improve the accuracy of the sampling process; and using the total sampled data volume obtained by sampling to perform traffic statistics, thereby avoiding the problem of inaccurate sampling process caused by discrepancies between the original messages and the actually sent messages, thereby effectively improving the accuracy of traffic statistics.

[0215] In addition, when performing traffic statistics, the traffic sampling rate can be further obtained based on the total sampled data volume. By using the traffic sampling rate instead of the message sampling rate for traffic statistics, compared with the message sampling rate, the traffic sampling rate can measure the sampling interval in a more fine-grained manner from the perspective of data volume, thereby making the traffic statistics results closer to the actual situation and further improving the accuracy of traffic statistics.

[0216] In addition, for different segmentation methods, a technical concept is further provided for correcting different correction functions corresponding to different segmentation methods; so that under different segmentation methods, a precise matching correction method can be used to correct the first message sampling rate, thereby effectively correcting the first message sampling rate to a sampling rate that is closer to the actual segmentation situation, thereby improving the accuracy of the actual sampling rate and improving the accuracy of the sampling process.

[0217] The traffic statistics method provided in this application involves technical fields such as cloud technology, cloud computing, cloud storage, and big data. For example, cloud computing technology is used to perform traffic statistics on messages sent by network cards in real time; for example, cloud storage technology can be used to store relevant data of sampled messages.

[0218] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to required resources over the internet. In a broader sense, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to required services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0219] Cloud computing has rapidly grown, driven by the internet, real-time data streams, the diversification of connected devices, and the growing demand for search services, social networks, mobile commerce, and open collaboration. Unlike previous parallel and distributed computing approaches, the emergence of cloud computing will fundamentally revolutionize the entire internet and enterprise management model.

[0220] Cloud storage is a new concept that has been extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0221] Currently, storage systems utilize a method for creating logical volumes. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may consist of disks on a specific storage device or several storage devices. When a client stores data on a logical volume, it stores the data on a file system. The file system divides the data into multiple parts, each of which is an object. An object contains not only the data but also additional information such as the data identifier (ID) of the data entity. The file system writes each object to the physical storage space of the logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can provide access to the data based on the storage location information of each object.

[0222] The storage system allocates physical storage space to logical volumes by pre-dividing the physical storage space into stripes based on the estimated capacity of the objects to be stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the Redundant Array of Independent Disks (RAID) groupings. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0223] Big data refers to collections of data that cannot be captured, managed, and processed within a specific timeframe using conventional software tools. These massive, rapidly growing, and diverse information assets require new processing models to enhance decision-making, insight discovery, and process optimization. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to efficiently process large amounts of time-sensitive data. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

[0224] Figure 10 This is a schematic diagram of the structure of a flow statistics device provided in an embodiment of the present application. Figure 10 As shown, the device includes:

[0225] A first acquisition module 1001 is configured to, during periodic sampling of messages sent by the network card, acquire, for each sampling period, a number of original messages and a number of sent messages corresponding to the sampling period, where the number of original messages is the number of original messages to be sent during the sampling period, and the number of sent messages is the number of sent messages actually sent during the sampling period;

[0226] A second acquisition module 1002 is configured to acquire a first message sampling rate corresponding to the sent messages based on the number of sent messages;

[0227] A correction module 1003 is configured to correct the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original message;

[0228] The sampling module 1004 is configured to sample the original messages within the sampling period based on the second message sampling rate, and to count the total sampled data volume of the original messages obtained by sampling;

[0229] The first statistics module 1005 is configured to perform traffic statistics of the outgoing messages actually sent during the sampling period based on the total sampled data volume of the original messages obtained by the sampling.

[0230] In a possible implementation, the network card enables a function of splitting a message before sending it, and the sent message is obtained by splitting the original message to be sent within the sampling period;

[0231] The correction module, when correcting the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original message, includes:

[0232] a split change rate determining unit, configured to determine a split change rate based on the number of original messages and the number of sent messages, the split change rate indicating a degree of message change in the sent messages compared to the original messages due to the split operation;

[0233] The correction unit is used to correct the first message sampling rate based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message.

[0234] In one possible implementation, the network card uses multiple segmentation methods to segment the message;

[0235] The correction unit, when correcting the first message sampling rate based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message, is specifically configured to:

[0236] Obtain the target segmentation method corresponding to the original message to be sent within the sampling period;

[0237] Determining a target correction function associated with a target segmentation method based on an association relationship between a plurality of segmentation methods and a correction function corresponding to each segmentation method;

[0238] Each segmentation method corresponds to a correction function, and each correction function represents a mapping relationship of mapping the first message sampling rate to the second message sampling rate based on the segmentation change rate;

[0239] Based on the segmentation change rate, the first message sampling rate is corrected to the second message sampling rate according to the mapping relationship represented by the target correction function.

[0240] In one possible implementation, the device further includes:

[0241] A second statistical module is used to count the amount of sampled data belonging to each message type in the original message obtained by sampling according to at least one message type;

[0242] The first statistics module is specifically configured to, when performing traffic statistics on the outgoing messages actually sent during the sampling period based on the total sampled data volume of the original messages obtained by the sampling,:

[0243] Determine a traffic sampling rate for the original message based on the total sampled data volume of the original message obtained by the sampling and the total data volume of the messages sent within the sampling period;

[0244] Based on the traffic sampling rate and the sampled data volume of each message type in the original message obtained by sampling, the sent data volume of each message type in the sent messages within the sampling period is obtained by statistics.

[0245] In one possible implementation, the apparatus further includes a third acquisition module, the third acquisition module being configured to acquire at least one message type;

[0246] The third acquisition module is specifically used for any of the following when acquiring at least one message type:

[0247] Performing statistics on at least one quintuple corresponding to the sampled original message to obtain the at least one message type;

[0248] Counting at least one service identifier corresponding to the sampled original message to obtain the at least one message type;

[0249] Statistics are collected on at least one quality of service (QoS) priority corresponding to the sampled original message to obtain the at least one message type.

[0250] In a possible implementation, the sampled original message includes multiple messages;

[0251] The second statistics module is specifically configured to, when counting the amount of sampled data belonging to each message type in the sampled original message according to at least one message type,:

[0252] Extracting a message type of a corresponding message from each message obtained by the sampling;

[0253] For each message obtained by sampling, based on the message type of each message and in accordance with the pre-configured association relationship between the message type and the sampled data volume, the data volume of each message is accumulated to the sampled data volume associated with the message type of each message.

[0254] In one possible implementation, the apparatus further includes a fourth acquisition module, the fourth acquisition module being configured to acquire a total data volume of sent messages within a sampling period;

[0255] The third acquisition module is specifically used for any of the following when acquiring the total data volume of the messages sent within the sampling period:

[0256] Based on the second message sampling rate and the total sampled data volume of the original message obtained by sampling, the total data volume of the message sent in the sampling period is estimated;

[0257] Based on the sampling period, a statistical interface for the network card is called to obtain a total data volume of messages sent by the network card during the sampling period.

[0258] In one possible implementation, the sampling module, when sampling the original messages within the sampling period based on the second message sampling rate and counting the total sampled data volume of the sampled original messages, is specifically configured to:

[0259] Sampling the IP headers of the original messages within the sampling period according to the second message sampling rate to obtain the sampled IP headers of the original messages;

[0260] Based on the amount of message data extracted from the IP header of the sampled original message, the total sampled data amount of the sampled original message is obtained by statistics.

[0261] The traffic statistics device provided by the present application obtains a first message sampling rate for messages actually sent by the network card in each sampling period, and then continues to correct the first message sampling rate based on the number of original messages to be sent within the period and the number of messages actually sent by the network card to obtain a second message sampling rate that is more accurate and suitable for the original messages; using the second message sampling rate to sample the original messages can greatly improve the accuracy of the sampling process; and using the total sampled data volume obtained by sampling to perform traffic statistics, thereby avoiding the problem of inaccurate sampling process caused by discrepancies between the original messages and the actually sent messages, thereby effectively improving the accuracy of traffic statistics.

[0262] In addition, when performing traffic statistics, the traffic sampling rate can be further obtained based on the total sampled data volume. By using the traffic sampling rate instead of the message sampling rate for traffic statistics, compared with the message sampling rate, the traffic sampling rate can measure the sampling interval in a more fine-grained manner from the perspective of data volume, thereby making the traffic statistics results closer to the actual situation and further improving the accuracy of traffic statistics.

[0263] In addition, for different segmentation methods, a technical concept is further provided for correcting different correction functions corresponding to different segmentation methods; so that under different segmentation methods, a precise matching correction method can be used to correct the first message sampling rate, thereby effectively correcting the first message sampling rate to a sampling rate that is closer to the actual segmentation situation, thereby improving the accuracy of the actual sampling rate and improving the accuracy of the sampling process.

[0264] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.

[0265] Figure 11 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 11 As shown, the electronic device includes: a memory, a processor, and a computer program stored in the memory. The processor executes the above computer program to implement the steps of the traffic statistics method. Compared with the related art, it can achieve:

[0266] The traffic statistics method provided in the present application obtains a first message sampling rate for messages actually sent by the network card in each sampling period, and then continues to correct the first message sampling rate based on the number of original messages to be sent in the period and the number of messages actually sent by the network card to obtain a second message sampling rate that is more accurate and suitable for the original messages; using the second message sampling rate to sample the original messages can greatly improve the accuracy of the sampling process; and using the total sampled data volume obtained by sampling to perform traffic statistics, thereby avoiding the problem of inaccurate sampling process caused by discrepancies between the original messages and the actually sent messages, thereby effectively improving the accuracy of traffic statistics.

[0267] In addition, when performing traffic statistics, the traffic sampling rate can be further obtained based on the total sampled data volume. By using the traffic sampling rate instead of the message sampling rate for traffic statistics, compared with the message sampling rate, the traffic sampling rate can measure the sampling interval in a more fine-grained manner from the perspective of data volume, thereby making the traffic statistics results closer to the actual situation and further improving the accuracy of traffic statistics.

[0268] In addition, for different segmentation methods, a technical concept is further provided for correcting different correction functions corresponding to different segmentation methods; so that under different segmentation methods, a precise matching correction method can be used to correct the first message sampling rate, thereby effectively correcting the first message sampling rate to a sampling rate that is closer to the actual segmentation situation, thereby improving the accuracy of the actual sampling rate and improving the accuracy of the sampling process.

[0269] In an alternative embodiment, an electronic device is provided, such as Figure 11 As shown, Figure 11 The electronic device 1100 shown includes: a processor 1101 and a memory 1103. The processor 1101 and the memory 1103 are connected, for example, via a bus 1102. Optionally, the electronic device 1100 may further include a transceiver 1104, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 1104 is not limited to one, and the structure of the electronic device 1100 does not constitute a limitation on the embodiments of the present application.

[0270] The processor 1101 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 1101 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0271] The bus 1102 may include a path for transmitting information between the above components. The bus 1102 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0272] The memory 1103 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media\other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0273] The memory 1103 is used to store the computer program for executing the embodiments of the present application, and the execution is controlled by the processor 1101. The processor 1101 is used to execute the computer program stored in the memory 1103 to implement the steps shown in the above method embodiments.

[0274] Among them, electronic equipment includes but is not limited to: servers, terminals or cloud computing center equipment, etc.

[0275] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0276] An embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0277] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0278] Those skilled in the art will understand that, unless otherwise specified, the singular forms "a," "an," "the," and "the" used herein may also include the plural forms. The terms "including" and "comprising" used in the embodiments of this application mean that the corresponding features can be implemented as the presented features, information, data, steps, and operations, but do not exclude the implementation of other features, information, data, steps, operations, etc. supported by the technical field.

[0279] The terms "first," "second," "third," "fourth," "1," "2," and the like (if any) in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than that shown or described in the drawings.

[0280] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the order of implementation of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times respectively. Under different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0281] The above description is only an optional implementation method for some implementation scenarios of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of this application, the use of other similar implementation methods based on the technical ideas of this application also falls within the protection scope of the embodiments of this application.

Claims

1. A traffic statistics method, characterized in that: The method comprises: In the process of periodically sampling messages sent by the network card, for each sampling period, obtaining the number of original messages and the number of sent messages corresponding to the sampling period, wherein the number of original messages is the number of original messages to be sent within the sampling period, and the number of sent messages is the number of sent messages actually sent within the sampling period; Based on the number of sent messages, obtaining a first message sampling rate corresponding to the sent messages; Based on the number of original messages and the number of sent messages, the first message sampling rate is modified to obtain a second message sampling rate corresponding to the original message; Sampling the original messages within the sampling period based on the second message sampling rate, and counting the total sampled data volume of the original messages obtained by sampling; Based on the total sampled data volume of the original messages obtained by the sampling, traffic statistics are performed on the messages actually sent during the sampling period.

2. The method according to claim 1, characterized in that The network card enables a function of splitting the message before sending, and the sent message is obtained by splitting the original message to be sent within the sampling period; The modifying the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original messages includes: Determining a split change rate based on the number of original messages and the number of outgoing messages, the split change rate indicating a degree of change in the outgoing messages compared to the original messages caused by the split operation; The first message sampling rate is corrected based on the segmentation change rate to obtain a second message sampling rate corresponding to the original message.

3. The method according to claim 2, characterized in that The network card uses multiple segmentation methods to segment the message; The correcting the first message sampling rate based on the slicing change rate to obtain a second message sampling rate corresponding to the original message includes: Obtaining a target segmentation method corresponding to the original message to be sent within the sampling period; Determining a target correction function associated with a target segmentation method based on an association relationship between a plurality of segmentation methods and a correction function corresponding to each segmentation method; Each segmentation method corresponds to a correction function, and each correction function represents a mapping relationship of mapping the first message sampling rate to the second message sampling rate based on the segmentation change rate; Based on the segmentation change rate, the first message sampling rate is corrected to the second message sampling rate according to the mapping relationship represented by the target correction function.

4. The method according to claim 1 or 2, characterized in that The method further comprises: According to at least one message type, counting the amount of sampled data belonging to each message type in the original message obtained by sampling; The total sampled data volume of the original messages obtained by sampling is used to perform traffic statistics on the messages actually sent during the sampling period, including: Determining a flow sampling rate for the original message based on the total sampled data volume of the original message obtained by the sampling and the total data volume of the message sent within the sampling period; Based on the traffic sampling rate and the sampled data volume of each message type in the sampled original message, the sent data volume of each message type in the sent messages within the sampling period is obtained by statistics.

5. The method according to claim 4, characterized in that The method for obtaining the at least one message type includes any one of the following: Performing statistics on at least one quintuple corresponding to the sampled original message to obtain the at least one message type; Counting at least one service identifier corresponding to the sampled original message to obtain the at least one message type; Statistics are collected on at least one quality of service (QoS) priority corresponding to the sampled original message to obtain the at least one message type.

6. The method according to claim 4, characterized in that The sampled original message includes multiple messages; The counting, according to at least one message type, of the amount of sampled data belonging to each message type in the sampled original messages includes: Extracting a message type of a corresponding message from each message obtained by the sampling; For each message obtained by sampling, based on the message type of each message and in accordance with the pre-configured association relationship between the message type and the sampled data amount, the data amount of each message is accumulated to the sampled data amount associated with the message type of each message.

7. The method according to claim 4, characterized in that The total amount of data of the messages sent during the sampling period may be obtained by any of the following methods: estimating a total data volume of messages sent within the sampling period based on the second message sampling rate and a total sampled data volume of the original messages obtained by sampling; A statistical interface for the network card is called based on the sampling period to obtain a total data volume of messages sent by the network card during the sampling period.

8. The method according to claim 1 or 2, characterized in that The sampling of the original messages within the sampling period based on the second message sampling rate and counting the total sampled data volume of the sampled original messages includes: Sampling the IP headers of the original messages within the sampling period according to the second message sampling rate to obtain the sampled IP headers of the original messages; Based on the amount of message data extracted from the IP header of the sampled original message, the total sampled data amount of the sampled original message is obtained by statistics.

9. A flow statistics device, characterized in that: The device comprises: a first acquisition module configured to, during periodic sampling of messages sent by the network card, acquire, for each sampling period, a number of original messages and a number of sent messages corresponding to the sampling period, wherein the number of original messages is the number of original messages to be sent within the sampling period, and the number of sent messages is the number of sent messages actually sent within the sampling period; A second acquisition module is used to acquire a first message sampling rate corresponding to the sent messages based on the number of sent messages; A correction module, configured to correct the first message sampling rate based on the number of original messages and the number of sent messages to obtain a second message sampling rate corresponding to the original messages; a sampling module, configured to sample the original messages within the sampling period based on the second message sampling rate, and calculate a total sampled data volume of the original messages obtained by sampling; The statistics module is used to perform traffic statistics on the messages actually sent out during the sampling period based on the total sampled data volume of the original messages obtained by the sampling.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the traffic statistics method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the traffic statistics method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the traffic statistics method according to any one of claims 1 to 8 is implemented.

Images