Traffic statistics method, device, system, equipment and medium

Abstract

The present disclosure provides a flow statistics method, device, system, equipment and medium, which relates to the field of communication technology. The method is applied to a flow statistics system, which is configured in a distributed cluster mode. The method includes: issuing flow collection tasks through a distributed cluster master node according to a preset time period; obtaining a first collection value of a routing device according to an object identifier OID library of the routing device and a simple network management protocol SNMP; performing data processing on the first collection value and a second collection value obtained by collecting from a third-party source through a public interface to obtain a flow collection value; and performing statistics on the flow collection value according to a circuit relationship model, a network type corresponding to the routing device and the timing requirements of flow statistics to generate flow statistics data. At least it can be used to collect and count the flow of each routing device, and by issuing tasks through a distributed cluster, the server cache mechanism can be reasonably utilized to improve efficiency.

Description

Background Art

[0002] With the advancement of technology, more and more enterprises and companies are working through the Internet. Enterprises can achieve one-stop access to global Internet resources, which requires the use of traffic. Therefore, the collection and statistics of traffic are essential as an important part of the service.

[0003] However, in the related art, there is currently no method that can collect and count the traffic of a large number of routing devices at the same time.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a traffic statistics method, device, system, equipment and medium, which can at least be used to collect and count the traffic of each routing device.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] In a first aspect, an embodiment of the present disclosure provides a traffic statistics method, which is applied to a traffic statistics system, wherein the traffic statistics system is configured in a distributed cluster manner; the method includes:

[0008] The traffic collection tasks are sent out through the master node of the distributed cluster according to the preset time period.

[0009] According to the object identifier OID library of the routing device and the simple network management protocol SNMP, the traffic of the routing device supporting the simple network management protocol is collected to obtain a first collection value of the routing device;

[0010] The first collected value and the second collected value are processed to obtain a flow collection value; the second collected value is obtained by collecting data from a third-party source through a public interface from a routing device that does not support the Simple Network Management Protocol SNMP;

[0011] According to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of the traffic statistics, the traffic collection values ​​are counted to generate traffic statistics data.

[0012] In a possible embodiment, the sending of the traffic collection task through the distributed cluster master node according to the preset time period includes:

[0013] According to the preset time period, the traffic collection task is divided into multiple traffic collection sub-tasks through the distributed cluster master node based on the collection device, and the traffic collection sub-tasks are distributed to the collection server, so that each collection server calls multiple threads to execute the traffic collection sub-task based on the circuit of the collection device to execute the traffic collection task.

[0014] In a possible embodiment, performing data processing on the first collected value and the second collected value to obtain a flow collection value includes:

[0015] Setting the data with abnormal collection in the first collected value and the second collected value to a blank flag and recording it as first error data;

[0016] Identify and correct the data reset by the counter to obtain corrected data;

[0017] Setting the data in the second collected value that does not conform to the preset rule to a blank flag and recording it as second error data;

[0018] The first collected value and the second collected value are processed according to the first error data, the second error data and the correction data to obtain a flow collection value; the flow collection value is physical flow data that complies with preset rules, and serves as the flow collection value.

[0019] In a possible embodiment, the network types include Internet Protocol versions IPV4 and IPV6; the timing requirements include collecting traffic statistics every 5 minutes, collecting traffic statistics every 30 minutes, collecting traffic statistics every 1 hour, collecting traffic statistics every 2 hours, and collecting traffic statistics every 24 hours.

[0020] In a possible embodiment, the circuit relationship model is used to collect statistics on traffic corresponding to the routing device;

[0021] Wherein, the circuit relationship model is a tree relationship model formed in the order of charging circuit, logic circuit layer, physical circuit layer and sub-circuit layer;

[0022] The sub-circuit layer includes a plurality of sub-ports connected to the circuit port, and the traffic of different directions of the circuit port is stored through different sub-ports;

[0023] The physical circuit layer is used to aggregate the flow information of each sub-port to form physical circuit flow information;

[0024] The logic circuit layer is used to shield the physical details of the physical circuit flow information to form logic circuit flow information;

[0025] The billing circuit is used to generate a traffic order according to the traffic information of the logic circuit.

[0026] In a possible embodiment, the method further includes:

[0027] If the flow of the routing device is collected according to the simple network management protocol SNMP and a known object identifier OID library, and there is a network anomaly when reading the value of the circuit flow counter, the average flow value is recorded as a blank mark.

[0028] In a possible embodiment, the method further includes:

[0029] Determine the Kth average flow value according to the Kth flow collection value, the Kth flow collection time, the K+1th flow collection value and the K+1th flow collection time;

[0030] The K-th average flow value is determined as follows:

[0031]

[0032] Among them, Q represents the average flow value of the Kth time; A represents the flow collection value of the K+1th time; B represents the flow collection value of the Kth time; T(A) represents the flow collection time of the K+1th time; T(B) represents the flow collection time of the Kth time.

[0033] In a possible embodiment, the method further includes:

[0034] If the type of the traffic collection value of the K+1th time is non-digital type, the traffic collection value of the K+1th time is recorded as a blank mark, and the average traffic value of the Kth time is recorded as a blank mark;

[0035] If the Kth flow collection value is a blank mark, a flow collection value that meets the preset conditions and is closest to the Kth flow collection value is used as the Kth flow collection value to determine the Kth average flow value;

[0036] If the Kth flow collection value is greater than the K+1th flow collection value, the Kth average flow value is determined as follows:

[0037]

[0038] Q ′ It indicates the average flow value of the Kth time when the flow collection value of the Kth time is greater than the flow collection value of the K+1th time; A indicates the flow collection value of the K+1th time; B indicates the flow collection value of the Kth time; T(A) indicates the flow collection time of the K+1th time; T(B) indicates the flow collection time of the Kth time.

[0039] In a possible embodiment, the method further includes:

[0040] The traffic statistics data is collected through the 95th statistical algorithm to generate original billing traffic data.

[0041] In a possible embodiment, after generating traffic statistics data, the method further includes:

[0042] Visualize and share traffic statistics

[0043] In a second aspect, an embodiment of the present disclosure provides a flow statistics device, including:

[0044] A sending unit is used to send the flow collection task to the circuit collection flow of the collection device through the distributed cluster master node according to the preset time period;

[0045] A collection unit, used for obtaining the flow collection value of the routing device according to the simple network management protocol SNMP and a third-party source through a public interface;

[0046] The statistical unit is used to generate traffic statistical data according to the network type, circuit relationship model and timing requirements corresponding to the traffic collection value.

[0047] In a third aspect, an embodiment of the present disclosure provides a traffic statistics system, which is configured in a distributed cluster manner; the traffic statistics system includes: a routing device, a traffic collection service module and a traffic statistics service module;

[0048] According to the preset time period as a cycle, the traffic statistics system sends the traffic collection task to the circuit collection traffic of the collection device through the distributed cluster master node;

[0049] The traffic collection service module is used to obtain the traffic collection value of the routing device according to the simple network management protocol SNMP and a third-party source through a public interface;

[0050] The traffic statistics service module is used to generate various types of traffic statistics data according to the network type, circuit relationship type and timing type corresponding to the traffic collection value.

[0051] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method described in the first aspect above by executing the executable instructions.

[0052] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described in the first aspect when executed by a processor.

[0053] In a sixth aspect, according to another aspect of the present disclosure, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the above-described methods.

[0054] A flow statistics method provided by an embodiment of the present disclosure is applied to a flow statistics system, and the flow statistics system is configured in a distributed cluster manner. The method includes: issuing flow collection tasks through a distributed cluster master node according to a preset time period; obtaining a first collection value of a routing device according to an object identifier (OID) library and a simple network management protocol (SNMP) of the routing device; performing data processing on the first collection value and a second collection value obtained by collecting a third-party source through a public interface to obtain a flow collection value; and performing statistics on the flow collection value according to a circuit relationship model, a network type corresponding to the routing device, and a timing requirement for flow statistics to generate flow statistics data. The flow statistics method provided by an embodiment of the present disclosure can not only collect and count flow, but also issue tasks through a distributed cluster master node, and can reasonably utilize a server cache mechanism to improve efficiency. In addition, by being able to automatically process routing devices that support SNMP, the method has strong adaptability and is convenient for subsequent addition of routing devices.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0057] Figure 1 A schematic diagram showing the structure of a traffic statistics system in an embodiment of the present disclosure is shown;

[0058] Figure 2 A schematic diagram showing the structure of another traffic statistics system in an embodiment of the present disclosure is shown;

[0059] Figure 3A flow chart showing a method for counting traffic in an embodiment of the present disclosure is shown;

[0060] Figure 4 A schematic diagram showing a circuit relationship model in an embodiment of the present disclosure is shown;

[0061] Figure 5 A schematic diagram showing a 5-minute flow chart in an embodiment of the present disclosure is shown;

[0062] Figure 6 A schematic diagram showing a monthly flow report in an embodiment of the present disclosure is shown;

[0063] Figure 7 A schematic diagram showing the structure of a flow statistics device in an embodiment of the present disclosure is shown;

[0064] Figure 8 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0066] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0067] In the related technology, there is currently no method that can collect and count the traffic of a large number of routing devices at the same time.

[0068] Therefore, the present disclosure provides a flow statistics method, which is applied to a flow statistics system, and the flow statistics system is configured in a distributed cluster manner; the method includes: issuing flow collection tasks through a distributed cluster master node according to a preset time period; obtaining a first collection value of a routing device according to an object identifier OID library of the routing device and a simple network management protocol SNMP; performing data processing on the first collection value and a second collection value obtained by collecting a third-party source through a public interface to obtain a flow collection value; and performing statistics on the flow collection value according to a circuit relationship model, a network type corresponding to the routing device, and a timing requirement for flow statistics to generate flow statistics data. It can at least be used to collect and count the flow of each routing device, and tasks can also be issued through the master node of the distributed cluster, and the server cache mechanism can be reasonably utilized to improve efficiency. In addition, the routing device that supports the protocol can be automatically processed through the simple network management protocol, which has strong adaptability and is convenient for adding routing devices later.

[0069] Figure 1 A schematic diagram of a traffic statistics system that can be applied to an embodiment of the present disclosure is shown.

[0070] like Figure 1 As shown, the traffic statistics system 100 may include a traffic collection service module 101 , a traffic collection service module 102 , a traffic statistics service module 103 and a routing device 104 , wherein the routing device 104 includes a first routing device 1041 and a second routing device 1042 .

[0071] Among them, the traffic statistics system 100 sends traffic collection tasks through the distributed cluster master node according to the preset time period as a cycle; the traffic collection service module 101 is used to collect the traffic of the routing device that supports the simple network management protocol according to the object identifier OID library of the routing device and the simple network management protocol SNMP, and obtain the first collection value of the routing device 104; the traffic analysis service module 102 processes the first collection value and the second collection value to obtain the traffic collection value; the second collection value is obtained by collecting the routing device that does not support the simple network management protocol SNMP through a third-party source through a public interface; the traffic statistics service module 103 is used to count the traffic collection value according to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of the traffic statistics, and generate traffic statistics data.

[0072] Furthermore, the first routing device 1031 is a routing device supporting the Simple Network Management Protocol, and the second routing device 1032 is a routing device not supporting the Simple Network Management Protocol. Both the first routing device 1031 and the second routing device 1032 may include multiple ones.

[0073] The traffic statistics service module 103 is further used to collect statistics on the traffic statistics data through a 95th statistics algorithm to generate original billing traffic data that meets billing requirements.

[0074] The traffic statistics system 100 can use easy-cluster middleware to implement distributed clusters and have resource scalability. When the server performance utilization rate is too high, a new collection service is started to expand capacity. When the server performance is too low for a long time, redundant collection services are closed to achieve energy saving.

[0075] Furthermore, the embodiment of the present disclosure also provides a traffic statistics system 200 , which may include a traffic collection service module 201 , a traffic analysis service module 202 , a traffic statistics service module 203 , a routing device 204 and a traffic management service module 205 .

[0076] Among them, the traffic collection service module 201, the traffic analysis service module 202, the traffic statistics service module 203, the routing device 204 and Figure 1 The traffic collection service module 101, the traffic collection service module 102, the traffic statistics service module 103 and the routing device 104 have the same function. The traffic management service module 205 is used to visualize and share traffic statistics data.

[0077] The traffic management service module 205 can realize data visualization functions such as traffic graphs and monthly billing reports based on traffic information and realize data sharing based on Webservice technology.

[0078] Furthermore, the above-mentioned traffic statistics system 200 may also include a traffic information library 206; the first collected value, the second collected value, the traffic collection value and the traffic statistics data may be stored in the traffic information library 206 for preservation.

[0079] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0080] First, a traffic statistics method is provided in an embodiment of the present disclosure, which is applied to a traffic statistics system. The traffic statistics system is configured in a distributed cluster manner. In the following embodiment, the traffic statistics system is taken as a server as an example.

[0081] Figure 3 A flow chart of a traffic statistics method according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the traffic statistics method provided in the embodiment of the present disclosure includes the following steps:

[0082] S302: Sending traffic collection tasks through the distributed cluster master node according to a preset time period.

[0083] In a possible embodiment, according to a preset time period as a cycle, the traffic collection task is divided into multiple traffic collection sub-tasks by the distributed cluster master node based on the collection device, and the traffic collection sub-tasks are distributed to the collection server, so that each collection server calls multiple threads to execute the traffic collection sub-tasks based on the circuit of the collection device to execute the traffic collection task.

[0084] For example, taking the preset time period of 5 minutes as an example, and starting a flow collection task every 5 minutes as an example, the cluster master node divides the flow collection task into multiple flow collection subtasks based on the collection device, and dispatches the collection subtasks to the collection server for collection; the collection server calls multiple threads to execute the collection subtasks based on the circuit of the collection device. After the task dispatching is optimized as above, the server cache mechanism can be reasonably utilized to improve efficiency; at the same time, it can avoid a large number of exception processing and time timeouts caused by abnormal equipment when tasks are randomly dispatched. In addition, the present disclosure can also collect, analyze and count the flow of a large number of circuits at the same time, with high timeliness.

[0085] After the traffic collection task is issued, step S304 may be executed according to the collected traffic.

[0086] S304: Collect the traffic of the routing device supporting the Simple Network Management Protocol (SNMP) according to the object identifier OID library of the routing device and the Simple Network Management Protocol (SNMP) to obtain a first collected value of the routing device.

[0087] In a possible embodiment, according to a routing device information library and a known object identifier OID library, the flow of the routing device is collected through a simple network management protocol SNMP to obtain a first collected value.

[0088] Specifically, the routing device that can collect data from the traffic statistics system can be understood as a routing device that supports the SNMP protocol. The first collected value is obtained by combining the OID library of different types of devices in different networks with the SNMP protocol.

[0089] S306: Process the first collected value and the second collected value to obtain a flow collection value.

[0090] The second collected value is obtained by collecting data from a third-party source through a public interface from a routing device that does not support the Simple Network Management Protocol (SNMP).

[0091] In a possible embodiment, for routing devices that cannot be collected by the traffic statistics system, that is, routing devices that do not support the SNMP protocol, data is collected from a third-party source through an open public interface to obtain a second collection value, and then the first collection value and the second collection value are processed to obtain a traffic collection value.

[0092] Exemplarily, the traffic statistics system sets a traffic collection task with a preset time period of 5 minutes, collects traffic through SNMP according to the routing device information library and the OID library, and records the traffic collection value, which is recorded as the first traffic collection value. After another 5 minutes, the traffic collection value is collected again, which can be recorded as the second traffic collection value; the traffic information is collected at a preset time period of every 5 minutes, and the traffic information is saved in the traffic information library for subsequent statistics. The statistical method can be explained here by an example, and the traffic statistics data of 5 minutes are obtained by subtracting the second traffic collection value from the first traffic collection value.

[0093] For a device that cannot be collected, third-party data is read as a second collected value through the circuit and third-party source relationship information of the collection device and the circuit and third-party source relationship information of the routing device.

[0094] The specific process of processing the first collected value and the second collected value may include data analysis and data cleaning, etc. Through the data processing process, the first collected value and the second collected value are transformed into compliant physical flow data. The details are as follows:

[0095] The data collected abnormally in the first collected value and the second collected value is set to a blank flag, which can be NULL, and recorded as the first error data; the data reset by the counter is identified and corrected to obtain the corrected data; the data that does not conform to the preset rules in the second collected value is set to a blank flag, and recorded as the second error data; according to the first error data, the second error data and the corrected data, the first collected value and the second collected value are processed to obtain the flow collection value; the flow collection value is the physical flow data that conforms to the preset rules, which is used as the flow collection value.

[0096] The above data processing process is explained. For example, the erroneous data caused by the collection exception is recorded as abnormal information by setting the mark to a blank, as the first erroneous data, to facilitate subsequent data cleaning.

[0097] For example, data analysis is performed on the collected first collected value and the second collected value to identify erroneous data caused by counter resetting, and the erroneous data caused by counter resetting is optimized and corrected in the same manner as before the counter is reset to obtain corrected data.

[0098] After processing the first collection value and the second collection value in the above manner, non-compliant data can be deleted, and corrected data can be used to replace erroneous data to obtain physical flow data that meets preset rules as flow collection values.

[0099] After a period of data collection, a large number of flow collection values ​​can be generated. The flow statistics system can verify and supplement the flow collection values ​​according to the set period to avoid the impact of empty data on subsequent data statistics. The set period is not limited and can be adjusted according to the length of the preset time period.

[0100] For network-reachable routing devices, automatic data collection can be achieved by configuring the routing device name, that is, the object identifier and SNMP community information of the routing device, that is, adding the routing device to SNMP.

[0101] Execute S308 according to the flow collection value obtained from the above content.

[0102] S308: Counting the traffic collection values ​​according to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of traffic statistics to generate traffic statistics data.

[0103] The network types corresponding to the routing device include Internet Protocol versions IPV4 and IPV6; the timing requirements include collecting traffic statistics every 5 minutes, collecting traffic statistics every 30 minutes, collecting traffic statistics every 1 hour, collecting traffic statistics every 2 hours, and collecting traffic statistics every 24 hours. The timing requirements can also be other set time periods, not limited to the above four situations.

[0104] The circuit relationship model may include a circuit port and sub-port relationship model, a physical circuit and logical circuit relationship model, a circuit merging relationship model, etc.

[0105] Specifically, the circuit relationship model is used to count the traffic corresponding to the routing device. The circuit relationship model can be a tree relationship model formed in the order of the billing circuit, the logical circuit layer, the physical circuit layer and the sub-circuit layer, such as Figure 4 shown.

[0106] Specifically, the sub-circuit layer includes multiple sub-ports connected to the circuit port, and the traffic of different directions of the circuit port is saved through different sub-ports, and the traffic can be collected by directly collecting the sub-ports. Among them, different traffic can flow to all parts of the world.

[0107] The physical circuit layer is used to aggregate the traffic information of each sub-port to form physical circuit traffic information. Aggregate the traffic information of each sub-port to form a complete physical circuit traffic information. The relationship between the physical circuit and the sub-port is established through the port. The sub-port can be named in the form of physical circuit prefix + serial number. For example, physical circuit 1, physical circuit 2, and physical circuit 3.

[0108] The logical circuit layer is used to shield the physical details of the physical circuit flow information and form the logical circuit flow information. Multiple physical circuits with the same merge number can be merged into one logical circuit. The logical circuit can shield the physical details and improve the convenience of use. For example, during the sales stage, customers can apply for work orders based on the logical circuit.

[0109] The billing circuit is used to generate traffic orders based on the traffic information of the logical circuit. To reduce the bill information processing during billing, multiple logical circuits with the same merge number can be merged into one billing circuit. An order is generated through the billing circuit, and the customer pays according to the order.

[0110] Based on the circuit relationship model, network type and timing requirements, physical to physical, physical to logical and other data statistics can be achieved to generate various types of traffic statistics.

[0111] After the traffic statistical data are generated, the traffic statistical data may be counted based on a 95th statistical algorithm to generate original billing traffic data.

[0112] Specifically, the 95 in the 95th statistical algorithm refers to the data in the given data set that exceeds 95% of the data in the data set. The data values ​​in the traffic statistical data are sorted in descending order, and the top 5% of the numerical values ​​are selected to obtain 95% of the traffic statistical data.

[0113] Exemplarily, a circuit in the traffic statistics system collects a traffic collection value every 5 minutes. There are 31 collection days in March. Taking the case where data is processed and the data is not cleaned, the total number of data sets corresponding to all the traffic statistics data is: N = (60*24*31) / 5 = 8928; according to the 95th statistical algorithm, the 95% position is determined (if there is a decimal point, move forward one place), N*5%=8928*0.05=446.4≈447, the traffic data set is sorted in descending order, and the traffic value at position 447 is obtained, and the original billing traffic data is determined by the traffic value after 447.

[0114] Further, according to actual needs, the circuit relationship model in the present disclosure can be Figure 4 A partial tree relationship model in, for example, a circuit relationship model may only include a tree relationship model consisting of a sub-circuit layer and circuit ports corresponding to the sub-circuit layer.

[0115] In a possible embodiment, the traffic statistics system also includes a traffic information library, which can save the original physical traffic for a preset time period, that is, the traffic collected directly at the circuit port; 5-minute physical traffic; 5-minute logical traffic; billing traffic and other types of traffic data.

[0116] The traffic statistics method disclosed in the present invention also includes:

[0117] (1) If the flow of the routing device is collected according to the simple network management protocol SNMP and the known object identifier OID library, and there is a network anomaly when reading the value of the circuit flow counter, the average flow value is recorded as a blank mark.

[0118] (2) Determine the Kth average flow value based on the Kth flow collection value, the Kth flow collection time, the K+1th flow collection value and the K+1th flow collection time.

[0119] The K-th average flow value is determined as follows:

[0120]

[0121] Among them, Q represents the average flow value of the Kth time; A represents the flow collection value of the K+1th time; B represents the flow collection value of the Kth time; T(A) represents the flow collection time of the K+1th time; T(B) represents the flow collection time of the Kth time.

[0122] Exemplarily, if the type of the K+1th traffic collection value is a non-digital type, the K+1th traffic collection value is recorded as a blank mark, and the Kth average traffic value is recorded as a blank mark.

[0123] Exemplarily, if the Kth flow collection value is marked as blank, the flow collection value that meets the preset conditions and is closest to the Kth flow collection value will be used as the Kth flow collection value, and will be used together with the K+1th flow collection value to determine the Kth average flow value.

[0124] The preset condition here can be a data of normal traffic. For example, the closest traffic collection value to the Kth traffic collection value is the K-1th traffic collection value, but the K-1th traffic collection value is also blank, which does not meet the preset condition. Then the K-2th traffic collection value, which is normal traffic data, is selected as the Kth traffic collection value.

[0125] For example, if the Kth flow collection value is greater than the K+1th flow collection value, the Kth average flow value is determined as follows:

[0126]

[0127] Q ′It indicates the average flow value of the Kth time when the flow collection value of the Kth time is greater than the flow collection value of the K+1th time; A indicates the flow collection value of the K+1th time; B indicates the flow collection value of the Kth time; T(A) indicates the flow collection time of the K+1th time; T(B) indicates the flow collection time of the Kth time.

[0128] After the calculation is completed in the above manner, the K+1th flow collection value can be saved in a copy and used after the K+2th flow collection value is collected to determine the K+1th average flow value.

[0129] Furthermore, data collected by third-party sources should be strictly verified before use to prove that the data is reliable. Data supplementation and collection analysis of data collected by third-party sources should be separated independently. The execution time of data supplementation tasks is later than that of collection analysis. The data with blank marks should be replaced by the collection time. Data anomalies and program anomalies generated by third-party sources should be set to blank marks.

[0130] Furthermore, the flow information and intermediate results are recorded with blank marks to distinguish the data when the normal collection result is 0, which can be used for alarm monitoring to detect abnormalities.

[0131] In a possible embodiment, the method of the present disclosure may also perform visualization processing on traffic statistics data, for example, generating a 5-minute traffic graph and a monthly billing report.

[0132] like Figure 5 As shown, 5-minute flow chart: can display the circuit 5-minute flow chart information in real time to understand the circuit flow usage trend; other types of flow charts can be selected on the right. Take the total flow, flow in area A and flow in area B internationally as examples. Figure 5 On the right, you can choose 30 minutes, 2 hours, and 1 day to display different traffic graph information according to different timing requirements, including inflow traffic (IN), i.e. downloading things, etc., and outflow traffic (OUT), i.e. uploading files, etc.

[0133] Figure 6 A schematic diagram of a monthly flow report is shown. According to the 95th statistical algorithm, the monthly billing flow of the circuit is finally obtained. On the right, you can choose to view the data of the most recent six months.

[0134] Based on the same inventive concept, the present disclosure also provides a flow statistics device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0135] Figure 7 A schematic diagram of the structure of a flow statistics device in an embodiment of the present disclosure is shown as follows: Figure 7As shown, the flow statistics device 70 includes: a sending unit 701, which is used to send flow collection tasks through a distributed cluster master node according to a preset time period; a collection unit 702, which is used to collect the flow of a routing device that supports a simple network management protocol according to an object identifier OID library of the routing device and a simple network management protocol SNMP, and obtain a first collection value of the routing device; an analysis unit 703, which is used to perform data processing on the first collection value and the second collection value to obtain a flow collection value; the second collection value is obtained by collecting the flow from a third-party source through a public interface for a routing device that does not support the simple network management protocol SNMP; a statistics unit 704, which is used to perform statistics on the flow collection value according to a circuit relationship model, a network type corresponding to the routing device, and a timing requirement for flow statistics, and generate flow statistics data.

[0136] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0137] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0138] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0139] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform any step of the above method embodiments.

[0140] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0141] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0142] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0143] The electronic device 800 may also communicate with one or more external devices 840 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0144] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0145] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the above embodiment.

[0146] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0147] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0149] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0150] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0151] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0152] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0153] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the example implementation modes described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation mode of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation mode of the present disclosure.

[0154] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A traffic statistics method, characterized in that: Applied to a traffic statistics system, the traffic statistics system is configured in a distributed cluster manner; the method includes: The traffic collection tasks are sent out through the master node of the distributed cluster according to the preset time period. According to the object identifier OID library of the routing device and the simple network management protocol SNMP, the traffic of the routing device supporting the simple network management protocol is collected to obtain a first collection value of the routing device; The first collected value and the second collected value are processed to obtain a flow collection value; the second collected value is obtained by collecting data from a third-party source through a public interface from a routing device that does not support the Simple Network Management Protocol SNMP; According to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of the traffic statistics, the traffic collection values ​​are counted to generate traffic statistics data; The circuit relationship model is used to count the traffic corresponding to the routing device; Wherein, the circuit relationship model is a tree relationship model formed in the order of charging circuit, logic circuit layer, physical circuit layer and sub-circuit layer; The sub-circuit layer includes a plurality of sub-ports connected to the circuit port, and the traffic of different directions of the circuit port is stored through different sub-ports; The physical circuit layer is used to aggregate the flow information of each sub-port to form physical circuit flow information; The logic circuit layer is used to shield the physical details of the physical circuit flow information to form logic circuit flow information; The charging circuit is used to generate a flow order according to the flow information of the logic circuit; The process of sending traffic collection tasks through the distributed cluster master node according to the preset time period includes: According to the preset time period, the traffic collection task is divided into multiple traffic collection sub-tasks through the distributed cluster master node based on the collection device, and the traffic collection sub-tasks are distributed to the collection server, so that each collection server calls multiple threads to execute the traffic collection sub-task based on the circuit of the collection device to execute the traffic collection task.

2. The traffic statistics method according to claim 1, characterized in that: The step of performing data processing on the first collected value and the second collected value to obtain a flow collected value includes: Setting the data with abnormal collection in the first collected value and the second collected value to a blank flag and recording it as first error data; Identify and correct the data reset by the counter to obtain corrected data; Setting the data in the second collected value that does not conform to the preset rule to a blank flag and recording it as second error data; The first collected value and the second collected value are processed according to the first error data, the second error data and the correction data to obtain a flow collection value; the flow collection value is physical flow data that complies with preset rules, and serves as the flow collection value.

3. The traffic statistics method according to claim 1, characterized in that: The network types include Internet Protocol versions IPV4 and IPV6; the timing requirements include collecting traffic statistics every 5 minutes, collecting traffic statistics every 30 minutes, collecting traffic statistics every 1 hour, collecting traffic statistics every 2 hours, and collecting traffic statistics every 24 hours.

4. The traffic statistics method according to claim 1, characterized in that: The method further comprises: If the flow of the routing device is collected according to the simple network management protocol SNMP and a known object identifier OID library, and there is a network anomaly when reading the value of the circuit flow counter, the average flow value is recorded as a blank mark.

5. The traffic statistics method according to claim 1, characterized in that: The method further comprises: Determine the Kth average flow value according to the Kth flow collection value, the Kth flow collection time, the K+1th flow collection value and the K+1th flow collection time; The K-th average flow value is determined as follows: Among them, Q represents the average flow value of the Kth time; A represents the flow collection value of the K+1th time; B represents the flow collection value of the Kth time; T(A) represents the flow collection time of the K+1th time; T(B) represents the flow collection time of the Kth time.

6. The traffic statistics method according to claim 1, characterized in that: The method further comprises: If the type of the traffic collection value of the K+1th time is non-digital type, the traffic collection value of the K+1th time is recorded as a blank mark, and the average traffic value of the Kth time is recorded as a blank mark; If the Kth flow collection value is a blank mark, a flow collection value that meets the preset conditions and is closest to the Kth flow collection value is used as the Kth flow collection value to determine the Kth average flow value; If the Kth flow collection value is greater than the K+1th flow collection value, the Kth average flow value is determined as follows: Q ′ It indicates the average flow value of the Kth time when the flow collection value of the Kth time is greater than the flow collection value of the K+1th time; A indicates the flow collection value of the K+1th time; B indicates the flow collection value of the Kth time; T(A) indicates the flow collection time of the K+1th time; T(B) indicates the flow collection time of the Kth time.

7. The traffic statistics method according to claim 1, characterized in that: The method further comprises: The traffic statistics data is collected through the 95th statistical algorithm to generate original billing traffic data.

8. The traffic statistics method according to claim 1, characterized in that: After generating traffic statistics data, the method further includes: The traffic statistics data are visualized and shared.

9. A flow statistics device, characterized in that: The device comprises: A sending unit is used to send traffic collection tasks through a distributed cluster master node according to a preset time period; A collection unit, configured to collect traffic of a routing device supporting the simple network management protocol (SNMP) according to an object identifier (OID) library of the routing device and a simple network management protocol (SNMP), and obtain a first collection value of the routing device; An analysis unit is used to process the first collected value and the second collected value to obtain a flow collection value; the second collected value is obtained by collecting data from a third-party source through a public interface from a routing device that does not support the Simple Network Management Protocol SNMP; A statistical unit, configured to collect statistics of the traffic collection values ​​according to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of the traffic statistics, and generate traffic statistical data; The circuit relationship model is used to count the traffic corresponding to the routing device; Wherein, the circuit relationship model is a tree relationship model formed in the order of charging circuit, logic circuit layer, physical circuit layer and sub-circuit layer; The sub-circuit layer includes a plurality of sub-ports connected to the circuit port, and the traffic of different directions of the circuit port is stored through different sub-ports; The physical circuit layer is used to aggregate the flow information of each sub-port to form physical circuit flow information; The logic circuit layer is used to shield the physical details of the physical circuit flow information to form logic circuit flow information; The charging circuit is used to generate a flow order according to the flow information of the logic circuit; The sending unit is used to divide the traffic collection task into multiple traffic collection subtasks according to the preset time period, through the distributed cluster master node and the collection device as the unit, and distribute the traffic collection subtasks to the collection server, so that each collection server calls multiple threads to execute the traffic collection subtasks based on the circuit of the collection device to execute the traffic collection task.

10. A traffic statistics system, characterized in that: The traffic statistics system is configured in a distributed cluster mode; the traffic statistics system includes: a routing device, a traffic collection service module and a traffic statistics service module; The traffic collection tasks are sent to the master node of the distributed cluster according to the preset time period. The traffic collection service module is used to collect the traffic of the routing device supporting the simple network management protocol according to the object identifier OID library of the routing device and the simple network management protocol SNMP, and obtain the first collection value of the routing device; The traffic analysis service module is used to process the first collected value and the second collected value to obtain a traffic collection value; the second collected value is obtained by collecting data from a third-party source through a public interface from a routing device that does not support the Simple Network Management Protocol SNMP. The traffic statistics service module is used to collect statistics of the traffic collection values ​​and generate traffic statistics data according to the circuit relationship model, the network type corresponding to the routing device and the timing requirements of traffic statistics; The circuit relationship model is used to count the traffic corresponding to the routing device; Wherein, the circuit relationship model is a tree relationship model formed in the order of charging circuit, logic circuit layer, physical circuit layer and sub-circuit layer; The sub-circuit layer includes a plurality of sub-ports connected to the circuit port, and the traffic of different directions of the circuit port is stored through different sub-ports; The physical circuit layer is used to aggregate the flow information of each sub-port to form physical circuit flow information; The logic circuit layer is used to shield the physical details of the physical circuit flow information to form logic circuit flow information; The charging circuit is used to generate a flow order according to the flow information of the logic circuit; The process of sending traffic collection tasks through the distributed cluster master node according to the preset time period includes: According to the preset time period, the traffic collection task is divided into multiple traffic collection sub-tasks through the distributed cluster master node based on the collection device, and the traffic collection sub-tasks are distributed to the collection server, so that each collection server calls multiple threads to execute the traffic collection sub-task based on the circuit of the collection device to execute the traffic collection task.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images