Multi-dimensional network traffic statistics method and device based on data integration

Abstract

The present invention provides a multi-dimensional network traffic statistics method and device based on data integration, which can be used in the field of big data technology. The method includes: according to a preset data acquisition strategy, obtaining the traffic and network card data on the leaf server under the switch; saving the traffic and network card data in a data file preset on the leaf server; according to a data acquisition request, sending the traffic and network card data in the data file to a transit server, so that the transit server saves the traffic and network card data to a preset data table, and performs traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table. The device is used to execute the above method. The multi-dimensional network traffic statistics method and device based on data integration provided by an embodiment of the present invention can automatically integrate the network card and traffic data of multiple leaf servers to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a multi-dimensional network traffic statistics method and device based on data integration. Background Art

[0002] Currently, network communication and traffic data are collected based on switches and stored in a mirrored form on a cache server for user use. However, due to the limited memory of the cache server, the collected data is stored for a short period of time, and the data of the leaf switch and its leaf servers are not collected in the cache server. For the network card and traffic data of the leaf servers under the LEAF switch, the general method is to rely on tools deployed on the server to collect and transmit them locally for display, such as NMON and other tools. However, the disadvantage of such tools is that the data granularity is fixed and can only display the network status of a single server. It is impossible to perform multi-dimensional aggregation display and cannot meet diversified analysis needs. Summary of the invention

[0003] In view of the problems in the prior art, an embodiment of the present invention provides a multi-dimensional network traffic statistics method and device based on data integration, which can at least partially solve the problems existing in the prior art.

[0004] In a first aspect, the present invention proposes a multi-dimensional network traffic statistics method based on data integration, comprising:

[0005] According to the preset data acquisition strategy, obtain the traffic and network card data on the leaf servers under the switch;

[0006] Saving the traffic and network card data to a data file preset on the leaf server;

[0007] According to the data acquisition request, the traffic and network card data in the data file are sent to the relay server, so that the relay server saves the traffic and network card data to a preset data table, and performs traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table.

[0008] Optionally, obtaining the traffic and network card data on the leaf server under the switch according to a preset data acquisition strategy includes:

[0009] Get the current time point traffic and network card data on the leaf server under the switch at preset time intervals.

[0010] Optionally, the method further includes:

[0011] The traffic and network card data in the data file are cleaned up according to the preset data cleaning strategy.

[0012] In a second aspect, the present invention proposes a multi-dimensional network traffic statistics method based on data integration, comprising:

[0013] Obtain the traffic and network card data in the data files of the leaf servers under each switch;

[0014] Saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least 1;

[0015] Traffic statistics are performed based on the traffic and network card data of each leaf server stored in the preset data table.

[0016] Optionally, the step of saving the traffic and network card data obtained from each leaf server into a preset data table includes:

[0017] Obtain the field values ​​of each field in the preset data table in each flow and network card data;

[0018] The field values ​​of each field obtained from each flow and network card data are correspondingly saved in the preset data table.

[0019] Optionally, performing traffic statistics according to the traffic and network card data of each leaf server stored in the preset data table includes:

[0020] According to the IP or network card, date, and time period requested in the daily traffic query request, search for corresponding network data in the preset data table;

[0021] Calculate the network traffic of the IP or network card requesting the query at preset intervals within the time period on the date according to the found network data; and / or

[0022] According to the IP or network card and date range requested in the traffic trend query request, search for corresponding network data in the preset data table;

[0023] According to the found network data, the daily network traffic of the IP or network card requesting the query within the date range is calculated.

[0024] In a third aspect, the present invention proposes a multi-dimensional network traffic statistics device based on data integration, comprising:

[0025] The acquisition module is used to obtain the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy;

[0026] A saving module, used for saving the traffic and network card data to a data file preset on the leaf server;

[0027] The sending module is used to send the traffic and network card data in the data file to the relay server according to the data acquisition request, so that the relay server saves the traffic and network card data in the preset data table, and performs traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table.

[0028] Optionally, the acquisition module is specifically used to: obtain the traffic and network card data of the leaf server under the switch at the current time point every preset time.

[0029] Optionally, the device further comprises:

[0030] The cleaning module is used to clean up the traffic and network card data in the data file according to a preset data cleaning strategy.

[0031] In a fourth aspect, the present invention proposes a multi-dimensional network traffic statistics device based on data integration, comprising:

[0032] The acquisition module is used to obtain the traffic and network card data in the data files of the leaf servers under each switch;

[0033] A saving module, used for saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least 1;

[0034] The statistics module is used to perform traffic statistics based on the traffic and network card data of each leaf server stored in the preset data table.

[0035] Optionally, the storage module is specifically used for:

[0036] Obtain the field values ​​of each field in the preset data table in each flow and network card data;

[0037] The field values ​​of each field obtained from each flow and network card data are correspondingly saved in the preset data table.

[0038] Optionally, the statistics module is specifically used for:

[0039] According to the IP or network card, date, and time period requested in the daily traffic query request, search for corresponding network data in the preset data table;

[0040] Calculate the network traffic of the IP or network card requesting the query at preset intervals within the time period on the date according to the found network data; and / or

[0041] According to the IP or network card and date range requested in the traffic trend query request, search for corresponding network data in the preset data table;

[0042] According to the found network data, the daily network traffic of the IP or network card requesting the query within the date range is calculated.

[0043] In a fifth aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the multi-dimensional network traffic statistics method based on data integration described in any of the above embodiments are implemented.

[0044] In a sixth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-dimensional network traffic statistics method based on data integration described in any of the above embodiments.

[0045] The multi-dimensional network traffic statistics method and device based on data integration provided by the embodiment of the present invention collects network card and traffic data by deploying data acquisition tools on the leaf servers under each switch; then the traffic and network data of each leaf server are uniformly transmitted to a transit server, and the traffic data and corresponding network card data are entered into a preset data table on the transit server; a network traffic analysis tool is deployed on the transit server, and multi-dimensional traffic statistics are performed based on the traffic and network card data of each leaf server stored in the preset data table. In this way, the network card and traffic data of multiple leaf servers can be automatically integrated to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0047] Figure 1 It is a flowchart of a multi-dimensional network traffic statistics method based on data integration provided by an embodiment of the present invention.

[0048] Figure 2 It is a flowchart of a multi-dimensional network traffic statistics method based on data integration provided by another embodiment of the present invention.

[0049] Figure 3 It is a schematic diagram of the front-end display interface for single-day traffic query provided by an embodiment of the present invention.

[0050] Figure 4It is a schematic diagram of the front-end display interface for traffic trend query provided by an embodiment of the present invention.

[0051] Figure 5 It is a structural diagram of a multi-dimensional network traffic statistics device based on data integration proposed in one embodiment of the present invention.

[0052] Figure 6 It is a structural diagram of a multi-dimensional network traffic statistics device based on data integration proposed in one embodiment of the present invention.

[0053] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily.

[0055] The terms “first”, “second”, etc. used in this document do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0056] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0057] As used herein, "and / or" includes any or all combinations of the items described.

[0058] The execution subject of the multi-dimensional network traffic statistics method based on data integration provided in the embodiment of the present invention includes but is not limited to a computer.

[0059] Figure 1 FIG. 1 is a flow chart of a multi-dimensional network traffic statistics method based on data integration provided by an embodiment of the present invention. Figure 1 As shown, the multi-dimensional network traffic statistics method based on data integration provided by the embodiment of the present invention can be applied to the leaf server under the switch, and the method includes:

[0060] S101. Obtain the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy;

[0061] In this step, a data acquisition strategy is preset for the network card and traffic data of the leaf servers under the LEAF switch, so as to obtain the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy; the preset data acquisition strategy can specifically be timed acquisition, etc., and a shell script can be deployed on each leaf server to obtain the traffic and network card data of this server (including current time point, IP, network card name, network card status, inflow, outflow, number of sent and received packets, etc.) according to the preset data acquisition strategy.

[0062] S102, saving the traffic and network card data to a data file preset on the leaf server;

[0063] In this step, the traffic and network card data obtained from each leaf server is saved in a preset data file on the server, such as a txt file.

[0064] S103. According to the data acquisition request, the traffic and network card data in the data file are sent to the relay server, so that the relay server saves the traffic and network card data into a preset data table, and performs traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table.

[0065] In this step, the data acquisition request may be sent by the transit server, which may specifically be an operation and maintenance server for integrating traffic data; each leaf server uniformly sends the traffic and network card data to the transit server, and specifically, remote transmission technology may be used to periodically transmit the traffic and network card data on each leaf server (such as a Linux server) to the transit server, which may be achieved by deploying timing and FTP scripts on the transit server.

[0066] The transit server can be deployed with a parsing program to enter the traffic data and the corresponding network card data into a preset data table. A network traffic analysis tool is deployed on the transit server to perform multi-dimensional traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table.

[0067] The multi-dimensional network traffic statistics method based on data integration provided by the embodiment of the present invention collects network card and traffic data by deploying data acquisition tools on the leaf servers under each switch; then the traffic and network data of each leaf server are uniformly transmitted to a transit server, and the traffic data and corresponding network card data are entered into a preset data table on the transit server; a network traffic analysis tool is deployed on the transit server, and multi-dimensional traffic statistics are performed based on the traffic and network card data of each leaf server stored in the preset data table. In this way, the network card and traffic data of multiple leaf servers can be automatically integrated to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics.

[0068] Optionally, obtaining the traffic and network card data on the leaf servers under the switch according to a preset data acquisition strategy includes: obtaining the traffic and network card data on the leaf servers under the switch at the current time point at preset time intervals.

[0069] In this embodiment, a crontab scheduled task can be added to the server to automatically execute the shell script every 1 minute to obtain the flow and network card data of the leaf server under the switch at the current time point. After a period of time, the corresponding flow and network card data will be recorded in the data file (such as a txt file) on the leaf server, one piece of data per minute.

[0070] Optionally, the method may further include: cleaning the traffic and network card data in the data file according to a preset data cleaning strategy.

[0071] In this embodiment, in order to avoid server memory problems caused by excessive data volume in data files (such as txt files) on leaf servers, shell scripts and crontab scheduled tasks can be deployed to execute the shell script once every 24 hours to clear the txt text content.

[0072] Figure 2 FIG. 1 is a flow chart of a multi-dimensional network traffic statistics method based on data integration provided by an embodiment of the present invention. Figure 2 As shown, the multi-dimensional network traffic statistics method based on data integration provided by the embodiment of the present invention can be applied to the transit server, and the method includes:

[0073] S201, obtaining the traffic and network card data in the data files of the leaf servers under each switch;

[0074] In this step, you can deploy a bat script on the transit server and use the FTP command to obtain the traffic and network card data from each leaf server, such as obtaining the text content of a txt file (i.e., traffic and network card data). Specifically, you can set a windows scheduled task on the transit server to execute the bat script once a minute to obtain the latest traffic and network card data on the server.

[0075] S202, saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least one;

[0076] In this step, a parsing tool can be deployed on the transit server to parse the downstream traffic and network card data and insert them into at least one preset data table, thus completing the network data integration of multiple leaf servers.

[0077] S203, performing traffic statistics according to the traffic and network card data of each leaf server stored in the preset data table.

[0078] In this step, you can deploy a network traffic analysis tool on the transit server to form a multi-dimensional, visual traffic analysis page. When the tool is executed, it will request data from the transit server and receive the return, and finally display the analysis results on the client. Specifically, you can deploy a python script on the transit server, including the front-end display code of the network traffic tool and the background data call and calculation code. The background calculation code is mainly used to achieve connection with the data table and perform data screening and calculation.

[0079] The multi-dimensional network traffic statistics method based on data integration provided by the embodiment of the present invention collects the traffic and network data of each leaf server, enters the collected traffic data and corresponding network card data into a preset data table, and uses the deployed network traffic analysis tool to perform multi-dimensional traffic statistics based on the traffic and network card data of each leaf server stored in the preset data table. In this way, the network card and traffic data of multiple leaf servers can be automatically integrated to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics.

[0080] Optionally, saving the traffic and network card data obtained from each leaf server into a preset data table includes: obtaining the field value of each field in the preset data table in each traffic and network card data; and saving the field value of each field obtained from each traffic and network card data into the preset data table accordingly.

[0081] In this embodiment, a parsing module can be deployed on the transit server to form a multi-type screening mode, and field combinations can be freely configured to parse, screen, and aggregate the original data collected from multiple leaf servers, decouple and recombine data elements, and achieve multi-dimensional data integration. The data are then written into data tables of different functional types to facilitate data management and improve the flexibility of subsequent network traffic pages.

[0082] For example, (1) First, create a new data table in the Oracle database to store the network data of each server, including time, IP, network card name, network card status, inflow, outflow, number of received / sent packets, and other fields. (2) Then deploy a Python script on the transit server to parse the source data transmitted by each leaf server: according to the text format of the source data, extract the corresponding values ​​of the IP, network card, traffic, and other fields, and insert the values ​​into the newly created data table in (1) one by one in the dictionary format. At this point, the newly created Oracle data table contains the summary of the network history data of each server.

[0083] Optionally, performing traffic statistics according to the traffic and network card data of each leaf server stored in the preset data table includes:

[0084] According to the IP or network card, date, and time period requested in the daily traffic query request, search for corresponding network data in the preset data table;

[0085] Calculate the network traffic of the IP or network card requesting the query at preset intervals within the time period on the date according to the found network data; and / or

[0086] According to the IP or network card and date range requested in the traffic trend query request, search for corresponding network data in the preset data table;

[0087] According to the found network data, the daily network traffic of the IP or network card requesting the query within the date range is calculated.

[0088] In this embodiment, a python script can be deployed on the transfer server, including the front-end display code of the network traffic analysis tool and the back-end data call and calculation code. The back-end calculation code is mainly used to realize the connection of the data table and perform data screening and calculation. Figure 3 and Figure 4 .

[0089] like Figure 3As shown, users can enter the IP or network card name, date, and time period they want to query through the front-end display interface, and click the query button to query the daily traffic. The back-end will call the tool code to access the Oracle data table, obtain and calculate the network data corresponding to the input IP, network card, date, etc., and return charts of network data such as inflow and outflow on the front-end. This supports the summary display of network data for multiple IPs or network cards. The back-end code will calculate and return the results for traffic statistics of clusters or multiple network cards. The result data on the "Single-Day Traffic Query" page can be in minutes, that is, one piece of data per minute, which is used to locate abnormal points.

[0090] like Figure 4 As shown, the result data of the "Traffic Trend Query" page can be in "days", that is, one piece of data per day, for overall trend analysis. The backend query principle is the same as "single-day traffic query", but the data of each day will be aggregated and calculated before returning the result. It also supports multi-IP and multi-NIC aggregate query.

[0091] According to the above introduction, the embodiment of the present invention deploys self-developed tools on multiple leaf servers (such as Linux servers) to collect network card and traffic data; then the traffic data of each server is uniformly transmitted to a transit server, and a parsing program is deployed on the transit server to enter the traffic data and the corresponding network card data into a preset data table; finally, a web tool is developed to display multi-dimensional traffic (including multi-IP or cluster overall traffic, time-sharing traffic trends and daily traffic trends) and abnormal comparison, which is intuitive and effective. The whole process involves three stages, including the data source end, the integration end, and the client end. The data of multiple data sources are aggregated to the integration end through integration technology, and after customized processing, they are distributed to multiple clients for use.

[0092] Figure 5 FIG. 1 is a schematic diagram of a multi-dimensional network traffic statistics device based on data integration proposed in one embodiment of the present invention. Figure 5 As shown, a multi-dimensional network traffic statistics device based on data integration proposed in an embodiment of the present invention includes:

[0093] The acquisition module 31 is used to acquire the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy;

[0094] A saving module 32, used for saving the traffic and network card data to a data file preset on the leaf server;

[0095] The sending module 33 is used to send the traffic and network card data in the data file to the relay server according to the data acquisition request, so that the relay server saves the traffic and network card data to a preset data table, and performs traffic statistics based on the traffic and network card data of each leaf server saved in the preset data table.

[0096] The multi-dimensional network traffic statistics device based on data integration provided by the embodiment of the present invention collects network card and traffic data by deploying data acquisition tools on the leaf servers under each switch; then the traffic and network data of each leaf server are uniformly transmitted to a transit server, and the traffic data and corresponding network card data are entered into a preset data table on the transit server; a network traffic analysis tool is deployed on the transit server, and multi-dimensional traffic statistics are performed based on the traffic and network card data of each leaf server stored in the preset data table. In this way, the network card and traffic data of multiple leaf servers can be automatically integrated to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics.

[0097] Optionally, the acquisition module is specifically used to: obtain the traffic and network card data of the leaf server under the switch at the current time point every preset time.

[0098] Optionally, the device further comprises:

[0099] The cleaning module is used to clean up the traffic and network card data in the data file according to a preset data cleaning strategy.

[0100] The embodiments of the apparatus provided in the embodiments of the present invention can be specifically used to execute the processing flow of the above-mentioned method embodiments, and its functions are not described in detail here, and reference can be made to the detailed description of the above-mentioned method embodiments.

[0101] Figure 6 FIG. 1 is a schematic diagram of a structure of a multi-dimensional network traffic statistics device based on data integration according to an embodiment of the present invention. Figure 6 As shown, the present invention proposes a multi-dimensional network traffic statistics device based on data integration, comprising:

[0102] An acquisition module 41 is used to acquire the traffic and network card data in the data files of the leaf servers under each switch;

[0103] A saving module 42, used for saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least 1;

[0104] The statistics module 43 is used to perform traffic statistics based on the traffic and network card data of each leaf server stored in the preset data table.

[0105] The multi-dimensional network traffic statistics device based on data integration provided by the embodiment of the present invention collects the traffic and network data of each leaf server, enters the collected traffic data and corresponding network card data into a preset data table, and uses the deployed network traffic analysis tool to perform multi-dimensional traffic statistics based on the traffic and network card data of each leaf server stored in the preset data table. In this way, the network card and traffic data of multiple leaf servers can be automatically integrated to perform multi-dimensional network traffic statistics, thereby realizing the diversification and automation of network traffic statistics.

[0106] Optionally, the storage module is specifically used for:

[0107] Obtain the field values ​​of each field in the preset data table in each flow and network card data;

[0108] The field values ​​of each field obtained from each flow and network card data are correspondingly saved in the preset data table.

[0109] Optionally, the statistics module is specifically used for:

[0110] According to the IP or network card, date, and time period requested in the daily traffic query request, search for corresponding network data in the preset data table;

[0111] Calculate the network traffic of the IP or network card requesting the query at preset intervals within the time period on the date according to the found network data; and / or

[0112] According to the IP or network card and date range requested in the traffic trend query request, search for corresponding network data in the preset data table;

[0113] According to the found network data, the daily network traffic of the IP or network card requesting the query within the date range is calculated.

[0114] The embodiments of the apparatus provided in the embodiments of the present invention can be specifically used to execute the processing flow of the above-mentioned method embodiments, and its functions are not described in detail here, and reference can be made to the detailed description of the above-mentioned method embodiments.

[0115] It should be noted that the multi-dimensional network traffic statistics method and device based on data integration provided in the embodiments of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiments of the present invention do not limit the application field of the multi-dimensional network traffic statistics method and device based on data integration.

[0116] Figure 7 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 7As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the logic instructions in the memory 503 to execute the method described in any of the above embodiments.

[0117] In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0118] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0119] This embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables the computer to execute the methods provided by the above-mentioned method embodiments.

[0120] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] In the description of this specification, the description with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0125] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-dimensional network traffic statistics method based on data integration, characterized in that: include: According to the preset data acquisition strategy, obtain the traffic and network card data on the leaf servers under the switch; Saving the traffic and network card data to a data file preset on the leaf server; According to the data acquisition request, the flow and network card data in the data file are sent to the transit server, so that the transit server saves the flow and network card data into a preset data table, and performs flow statistics according to the flow and network card data of each leaf server saved in the preset data table; The method of obtaining the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy includes: Deploy a shell script on each leaf server under the switch to obtain the server traffic and network card data according to the preset data acquisition strategy; The transfer server saves the traffic and network card data into a preset data table, including: A data table is established in the Oracle database to store the network data of each server, including time, IP, network card name, network card status, inflow, outflow, and received / sent packet number fields; a Python script is deployed on the transit server to parse the source data transmitted by each leaf server: according to the text format of the source data, the corresponding value of the target field is separated. The target field includes IP, network card and traffic fields, and the values ​​are inserted into the data table one by one in a dictionary format. At this point, the data table contains a summary of the network history data of each server.

2. The method according to claim 1, characterized in that The method of obtaining the traffic and network card data on the leaf server under the switch according to the preset data acquisition strategy includes: Get the current time point traffic and network card data on the leaf server under the switch at preset time intervals.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The traffic and network card data in the data file are cleaned up according to the preset data cleaning strategy.

4. A multi-dimensional network traffic statistics method based on data integration, characterized in that: include: Obtain the traffic and network card data in the data files of the leaf servers under each switch. Each leaf server under the switch is deployed with a shell script, and the shell script obtains the traffic and network card data of the server according to the preset data acquisition strategy; Saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least 1; Perform traffic statistics according to the traffic and network card data of each leaf server stored in the preset data table; The step of saving the traffic and network card data obtained from each leaf server into a preset data table includes: A data table is established in the Oracle database to store the network data of each server, including time, IP, network card name, network card status, inflow, outflow, and received / sent packet number fields; a Python script is deployed on the transit server to parse the source data transmitted by each leaf server: according to the text format of the source data, the corresponding value of the target field is separated. The target field includes IP, network card and traffic fields, and the values ​​are inserted into the data table one by one in a dictionary format. At this point, the data table contains a summary of the network history data of each server.

5. The method according to claim 4, characterized in that The flow statistics according to the flow and network card data of each leaf server stored in the preset data table include: According to the IP or network card, date, and time period requested in the daily traffic query request, search for corresponding network data in the preset data table; Calculate the network traffic of the IP or network card requesting the query at preset intervals within the time period on the date according to the found network data; and / or According to the IP or network card and date range requested in the traffic trend query request, search for corresponding network data in the preset data table; According to the found network data, the daily network traffic of the IP or network card requesting the query within the date range is calculated.

6. A multi-dimensional network traffic statistics device based on data integration, characterized in that: include: The acquisition module is used to obtain the traffic and network card data on the leaf servers under the switch according to the preset data acquisition strategy; A saving module, used for saving the traffic and network card data to a data file preset on the leaf server; A sending module, used for sending the flow and network card data in the data file to the transit server according to the data acquisition request, so that the transit server saves the flow and network card data into a preset data table, and performs flow statistics according to the flow and network card data of each leaf server saved in the preset data table; The acquisition module is specifically used for: Deploy a shell script on each leaf server under the switch to obtain the server traffic and network card data according to the preset data acquisition strategy; The transfer server saves the traffic and network card data into a preset data table, including: Create a data table in the Oracle database to store the network data of each server, including time, IP, network card name, network card status, inbound traffic, outbound traffic, and number of packets received / sent; Deploy a python script on the transit server to parse the source data transmitted from each leaf server: extract the corresponding value of the target field according to the text format of the source data. The target field includes IP, network card and traffic fields. Insert the values ​​into the data table one by one in dictionary format. At this point, the data table contains a summary of the network history data of each server.

7. A multi-dimensional network traffic statistics device based on data integration, characterized in that: include: The acquisition module is used to obtain the traffic and network card data in the data files of the leaf servers under each switch. Each leaf server under the switch is deployed with a shell script, and the shell script obtains the traffic and network card data of the server according to the preset data acquisition strategy; A saving module, used for saving the traffic and network card data obtained from each leaf server into a preset data table, where the number of the preset data tables is at least 1; A statistics module, used for performing traffic statistics according to the traffic and network card data of each leaf server stored in the preset data table; The storage module is specifically used for: A data table is established in the Oracle database to store the network data of each server, including time, IP, network card name, network card status, inflow, outflow, and received / sent packet number fields; a Python script is deployed on the transit server to parse the source data transmitted by each leaf server: according to the text format of the source data, the corresponding value of the target field is separated. The target field includes IP, network card, and traffic fields. The values ​​are inserted into the data table one by one in a dictionary format. At this point, the data table contains a summary of the network history data of each server.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 or 4 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 or 4 to 5 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can execute the method of any one of claims 1 to 3 or 4 to 5 above.

Images