A RoCE Network End-System Traffic Monitoring Method

Abstract

The RoCE network end-system traffic monitoring method provided by the present invention constructs a RoCE network end-system, which consists of a traffic data transmission and parsing module, a traffic data statistics module, and a traffic data display module; after parsing the work requests of the application program through the data transmission module, data transmission is realized to obtain the data transmission parameters corresponding to the work requests; after the transmission is completed, the traffic data statistics module statistically analyzes the transmitted data according to the data transmission parameters, and updates the statistically transmitted data after the statistics are completed; after the update is completed, the traffic data display module obtains the updated transmitted data according to the work requests and displays it. Compared with the prior art, the present invention can implement statistical monitoring in the RoCE network protocol stack to effectively monitor the RoCE network traffic by constructing a RoCE network end-system.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and particularly to a method for monitoring RoCE network end-system traffic. Background Art

[0002] The RoCE network is an Ethernet-based RDMA (Remote Direct Memory Access) network. The end-to-end delay, CPU resource utilization and other performance indicators of the RoCE network are comparable to those of the Infiniband network, but its deployment cost is much lower than that of the Infiniband network. Therefore, it has been widely deployed and applied in data center networks for application scenarios such as AI and supercomputing that have high requirements for network latency and CPU utilization.

[0003] Currently, research on network traffic monitoring mainly focuses on the field of Ethernet technology. Due to different network architectures and protocol stack technology implementations, network monitoring technologies applied to Ethernet cannot be used in the RoCE network; the method of distinguishing network traffic by quintuple in Ethernet also cannot be applied to the RoCE network. And real-time monitoring of network traffic is extremely important for network status maintenance, network management, and network fault diagnosis.

[0004] Therefore, it is an urgent problem for those skilled in the art to provide a method for monitoring RoCE network end-system traffic that can effectively statistically monitor RoCE network traffic in the RoCE network protocol stack. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for monitoring RoCE network end-system traffic, which has clear logic, is safe, effective, reliable and easy to operate, and can statistically monitor various elements to improve the monitoring accuracy.

[0006] Based on the above purpose, the technical solution provided by the present invention is as follows:

[0007] A method for monitoring RoCE network end-system traffic includes the following steps:

[0008] Construct a RoCE network traffic monitoring system composed of a traffic data transmission and parsing module, a traffic data statistics module, and a traffic data display module;

[0009] The traffic data transmission and parsing module parses the work requests of the application program and then realizes data transmission to obtain the data transmission parameters corresponding to the work requests;

[0010] The traffic data statistics module statistically analyzes the transmitted data according to the data transmission parameters, and updates the statistically transmitted data after the statistics are completed;

[0011] The traffic data display module obtains the updated transmission data according to the work request and displays it.

[0012] Preferably, the traffic data transmission and parsing module includes: a sending parsing sub-module and a receiving parsing sub-module;

[0013] The data transmission parameters include: sending parameters and receiving parameters;

[0014] The traffic data transmission and parsing module realizes data transmission after parsing the work request of the application program to obtain the data transmission parameters corresponding to the work request, specifically:

[0015] The sending parsing sub-module receives the work request of the application program and parses the work request to obtain the sending parameters;

[0016] The receiving parsing sub-module obtains the polling work completion instruction of the application program and parses the polling work completion instruction to obtain the receiving parameters.

[0017] Preferably, both the sending parsing sub-module and the receiving parsing sub-module parse the work request and the polling work completion instruction in the data format of the RoCE network protocol stack.

[0018] Preferably, the traffic data statistics module includes: a data statistics module, a data update module, and a statistics database;

[0019] The data statistics module completes the statistics of the sent data according to the sending parameters and sends the sent data to the data update module;

[0020] The data statistics module completes the statistics of the received data according to the receiving parameters and sends the received data to the data update module;

[0021] The data update module updates the statistics database according to the sent data and the received data.

[0022] Preferably, the sending parameters and the receiving parameters are specifically: process ID, sending end QPN, receiving end QPN, operation type, and data size.

[0023] Preferably, the statistics database includes: a temporary data sub-library and a long-term data sub-library;

[0024] The data update module updates the statistics database according to the sent data and the received data, specifically:

[0025] The data update module updates the temporary different process IDs, temporary different sender QPNs, temporary different receiver QPNs, the amount of sent data of different temporary operation types, and the amount of received data of different temporary operation types in the temporary data sub-library according to the sent data and the received data respectively;

[0026] According to a preset data storage period, the temporary data sub-library respectively converts the temporary different process IDs, the temporary different sender QPNs, the temporary different receiver QPNs, the amount of sent data of different temporary operation types, and the amount of received data of different temporary operation types into long-term different process IDs, long-term different sender QPNs, long-term different receiver QPNs, the amount of sent data of different long-term operation types, and the amount of received data of different long-term operation types and stores them in the long-term data sub-library.

[0027] Preferably, the RoCE network traffic monitoring system further includes: a timing processing module;

[0028] The timing processing module is respectively connected to the temporary data sub-library and the long-term data sub-library;

[0029] Before the traffic data transmission and parsing module realizes data transmission after parsing the work request of the application program to obtain the data transmission parameters corresponding to the work request, it further includes:

[0030] The timing processing module adds the temporary different process IDs, the temporary different sender QPNs, the temporary different receiver QPNs, the amount of sent data of different temporary operation types, and the amount of received data of different temporary operation types to the process ID, the sender QPN, the receiver QPN, the amount of sent data of different long-term operation types, and the amount of received data of different long-term operation types respectively according to a preset update processing period, and then clears the temporary data in the temporary data sub-library.

[0031] Preferably, the traffic data display module obtains and displays the updated transmission data according to the work request, specifically:

[0032] The traffic data display module obtains the corresponding data from the updated statistical database according to the work request and displays it in both picture or text format.

[0033] The RoCE network end-system traffic monitoring method provided by the present invention constructs a RoCE network end-system, which consists of a traffic data transmission and parsing module, a traffic data statistics module, and a traffic data display module; after parsing the work requests of the application program through the data transmission module to implement data transmission, the data transmission parameters corresponding to the work requests are obtained; after the transmission is completed, the traffic data statistics module statistically analyzes the transmitted data according to the data transmission parameters, and updates the statistically transmitted data after the statistics are completed; after the update is completed, the traffic data display module obtains and displays the updated transmitted data according to the work requests.

[0034] Compared with the prior art, the present invention can effectively monitor the RoCE network traffic in the RoCE network protocol stack by constructing a RoCE network end-system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0036] Figure 1 It is a flowchart of a RoCE network end-system traffic monitoring method provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic structural diagram of a RoCE network traffic monitoring system provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic structural diagram of a traffic data transmission and parsing module provided by an embodiment of the present invention;

[0039] Figure 4 It is a schematic structural diagram of a traffic data statistics module provided by an embodiment of the present invention;

[0040] Figure 5 It is a schematic structural diagram of a statistical database provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0042] The embodiments of the present invention are written in a progressive manner.

[0043] The embodiments of the present invention provide a method for monitoring the traffic of the RoCE network end system, mainly solving the technical problem in the prior art that it is not applicable to the RoCE network, resulting in the inability to effectively monitor the RoCE network traffic.

[0044] As Figure 1 and Figure 2 shown, a method for monitoring the traffic of the RoCE network end system includes the following steps:

[0045] S1. Construct a RoCE network traffic monitoring system composed of a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module;

[0046] S2. After the traffic data transmission parsing module parses the work request of the application program, data transmission is realized to obtain the data transmission parameters corresponding to the work request;

[0047] S3. The traffic data statistics module statistically analyzes the transmitted data according to the data transmission parameters, and updates the statistically transmitted data after the statistics are completed;

[0048] S4. The traffic data display module obtains and displays the updated transmission data according to the work request.

[0049] In step S1, a RoCE network traffic monitoring system is constructed. This system runs on the RoCE network end system and is deployed together with the RoCE network protocol stack. In this system, there are a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module;

[0050] In steps S2 to S4, the traffic data statistics module is respectively connected to the traffic data transmission parsing module and the traffic data display module; when receiving the work request of the application program, the traffic data transmission parsing module parses the work request and realizes data transmission, so as to obtain the data transmission parameters corresponding to the work request and transmit them to the traffic data statistics module; the traffic data statistics module statistically analyzes the data according to the data transmission parameters, updates the statistically transmitted data after the statistics are completed, and sends the updated transmission data to the traffic data display module; the traffic data display module obtains and displays the updated transmission data according to this work request.

[0051] As Figure 3 shown, preferably, the traffic data transmission parsing module includes: a sending parsing sub-module and a receiving parsing sub-module;

[0052] The data transmission parameters include: sending parameters and receiving parameters;

[0053] After parsing the work requests of the application program, the traffic data transmission parsing module realizes data transmission to obtain the data transmission parameters corresponding to the work requests. Specifically:

[0054] The sending parsing sub-module receives the work requests of the application program and parses the work requests to obtain the sending parameters;

[0055] The receiving parsing sub-module obtains the polling work completion instruction of the application program and parses the polling work completion instruction to obtain the receiving parameters.

[0056] In the actual operation process, the traffic data transmission parsing module includes: a sending parsing sub-module and a receiving parsing sub-module; the corresponding data transmission parameters include: sending parameters and receiving parameters; therefore, the parsing of the work requests by the corresponding traffic data transmission parsing module is divided into two types. One is that the sending parsing sub-module receives the work requests of the application program and parses the work requests to obtain the corresponding sending parameters; the other is that the receiving parsing sub-module obtains the polling work completion instruction of the application program and parses the polling work completion instruction to obtain the receiving parameters.

[0057] In this embodiment, the sending parsing sub-module intercepts the sending work requests sent by the application program, parses the sending work requests according to the RoCE network protocol stack data format, and sends the sending parameters to the data statistics module; the receiving parsing sub-module intercepts the completion of the polling work of the application program, identifies and parses the receiving completion according to the RoCE network protocol stack, and sends the receiving parameters to the data statistics module;

[0058] Preferably, both the sending parsing sub-module and the receiving parsing sub-module parse the work requests and the polling work completion instructions in correspondence with the RoCE network protocol stack data format.

[0059] In the actual operation process, both the sending parsing sub-module and the receiving parsing sub-module parse the work requests and the polling work completion instructions in correspondence with a specific data format;

[0060] In this embodiment, the selected specific data format is the RoCE network protocol stack data format;

[0061] Such as Figure 4 shown, preferably, the traffic data statistics module includes: a data statistics module, a data update module, and a statistical database;

[0062] The data statistics module completes the sending data statistics according to the sending parameters and sends the sending data to the data update module;

[0063] The data statistics module completes the receiving data statistics according to the receiving parameters and sends the receiving data to the data update module;

[0064] The data update module updates the statistical database according to the sending data and the receiving data.

[0065] In the actual operation process, a data statistics module, a data update module, and a statistical database are set in the traffic data statistics module; the data statistics module completes the statistics of sent data and received data according to the sent parameters and received parameters and sends the completed sent data and received data to the data update module, and the data update module updates the corresponding sent data and received data in the statistical database.

[0066] Preferably, the sent parameters and received parameters are specifically: process ID, sender QPN, receiver QPN, operation type, and data size.

[0067] In the actual operation process, the sent parameters and received parameters are specifically: process ID, sender QPN, receiver QPN, operation type, and data size;

[0068] In this embodiment, the QP number (Queue Pair number, QPN): Each QP pair on the RDMA node has a unique number QPN to identify it.

[0069] Such as Figure 5 As shown, preferably, the statistical database includes: a temporary data sub-library and a long-term data sub-library;

[0070] The data update module updates the statistical database according to the sent data and received data, specifically:

[0071] The data update module updates the sent data volume of different temporary process IDs, different temporary sender QPNs, different temporary receiver QPNs, and different temporary operation types and the received data volume of different temporary operation types in the temporary data sub-library according to the sent data and received data respectively;

[0072] According to the preset data storage period, the temporary data sub-library respectively converts the sent data volume of different temporary process IDs, different temporary sender QPNs, different temporary receiver QPNs, and different temporary operation types and the received data volume of different temporary operation types into the sent data volume of different long-term process IDs, different long-term sender QPNs, different long-term receiver QPNs, and different long-term operation types and stores them in the long-term data sub-library.

[0073] In the actual operation process, the statistical database is designed as required: a temporary data sub-library and a long-term data sub-library; the data update module updates the temporary data in the temporary data sub-library according to the sent data and the received data, including the temporary data of different process IDs, different sender QPNs, different receiver QPNs, the sent data volume of different operation types, and the received data volume of different operation types; after the update is completed, the temporary data is periodically converted into long-term data according to the set data storage period, specifically including the long-term data of different process IDs, different sender QPNs, different receiver QPNs, the sent data volume of different operation types, and the received data volume of different operation types, and after the conversion is completed, it is stored in the long-term data sub-library.

[0074] In this embodiment, the statistical database stores various types of long-term and temporary statistical data; the long-term statistical data includes but is not limited to the long-term sent data volume of different process IDs, different sender QPNs, different receiver QPNs, and different operation types, and the long-term received data volume of different process IDs, different sender QPNs, different receiver QPNs, and different operation types; the temporary statistical data includes but is not limited to the temporary sent data volume of different process IDs, different sender QPNs, different receiver QPNs, and different operation types, and the temporary received data volume of different process IDs, different sender QPNs, different receiver QPNs, and different operation types.

[0075] Preferably, the RoCE network traffic monitoring system further includes a timing processing module;

[0076] The timing processing module is respectively connected to the temporary data sub-library and the long-term data sub-library;

[0077] Before the traffic data transmission and parsing module parses the work request of the application program to implement data transmission to obtain the data transmission parameters corresponding to the work request, it further includes:

[0078] The timing processing module adds the temporary data of different process IDs, different sender QPNs, different receiver QPNs, the sent data volume of different temporary operation types, and the received data volume of different temporary operation types to the process ID, sender QPN, receiver QPN, the long-term sent data volume of different operation types, and the long-term received data volume of different operation types according to the preset update processing period, and then clears the temporary data in the temporary data sub-library.

[0079] In the actual operation process, a timing processing module is also set in the RoCE network end system; the timing processing module is respectively connected to the temporary data sub-library and the long-term data sub-library; before the traffic data transmission and parsing module parses, the timing processing module adds the temporary data: the data volumes of different process IDs, different sender QPNs, different receiver QPNs, different operation types for sending and different operation types for receiving to the process ID, sender QPN, receiver QPN, long-term data volumes of different operation types for sending and long-term data volumes of different operation types for receiving, and then clears the above temporary data in the temporary data sub-library according to a preset update period.

[0080] Preferably, the traffic data display module obtains and displays the updated transmission data according to the work request, specifically:

[0081] The traffic data display module obtains the corresponding data from the updated statistical database according to the work request and displays them in the form of pictures or texts.

[0082] In the actual operation process, the traffic data display module obtains the corresponding data from the updated statistical database according to the work request and displays them in the form of pictures or texts to realize the monitoring of the change of traffic data.

[0083] In the embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The system embodiments described above are only illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical or other forms.

[0084] In addition, in each embodiment of the present invention, each functional module can be all integrated in one processor, or each module can be separately used as a device, or two or more modules can be integrated in one device; each functional module in each embodiment of the present invention can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0085] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed through program instructions and related hardware. The foregoing program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above method embodiments are performed; and the foregoing storage medium includes: various media that can store program codes such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0086] It should be understood that in this application, if "system", "device", "unit" and / or "module" are used, they are only a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, that word can be replaced by other expressions.

[0087] As shown in this application and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. An element defined by the statement "comprising one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

[0088] If a flowchart is used in this application, the flowchart is used to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or after do not necessarily need to be executed precisely in sequence. On the contrary, the steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps of operations can be removed from these processes.

[0089] The above has introduced in detail a method for monitoring the traffic of a RoCE network end system provided by the present invention. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring the traffic of a RoCE network end system, characterized in that, It includes the following steps: Construct a RoCE network traffic monitoring system composed of a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module; The sending parsing sub-module receives the work request of the application program, parses the work request to obtain the sending parameters; The receiving parsing sub-module obtains the polling work completion instruction of the application program, parses the polling work completion instruction to obtain the receiving parameters; The traffic data statistics module statistically analyzes the transmitted data according to the data transmission parameters, and updates the statistically transmitted data after the statistics are completed; The traffic data display module obtains and displays the updated transmitted data according to the work request; Among them, the traffic data transmission parsing module includes: the sending parsing sub-module and the receiving parsing sub-module; The data transmission parameters include: the sending parameters and the receiving parameters; The traffic data statistics module includes: a data statistics module, a data update module, and a statistical database; The data statistics module completes the sending data statistics according to the sending parameters and sends the sending data to the data update module; The data statistics module completes the receiving data statistics according to the receiving parameters and sends the receiving data to the data update module; The data update module updates the statistical database according to the sending data and the receiving data; The sending parameters and the receiving parameters are specifically: process ID, sending end QPN, receiving end QPN, operation type, and data size; The statistical database includes: a temporary data sub-library and a long-term data sub-library; The data update module updates the statistical database according to the sending data and the receiving data, specifically: The data update module respectively updates the sending data volume of different temporary process IDs, different temporary sending end QPNs, different temporary receiving end QPNs, and different temporary operation types and the receiving data volume of different temporary operation types in the temporary data sub-library according to the sending data and the receiving data; According to a preset data storage period, the temporary data sub-library respectively converts the sending data volume of different temporary process IDs, different temporary sending end QPNs, different temporary receiving end QPNs, and different temporary operation types and the receiving data volume of different temporary operation types into the sending data volume of different long-term process IDs, different long-term sending end QPNs, different long-term receiving end QPNs, and different long-term operation types and stores them in the long-term data sub-library.

2. The RoCE network end system traffic monitoring method according to claim 1, wherein, Both the sending parsing sub-module and the receiving parsing sub-module parse the work request and the polling work completion instruction in the data format of the RoCE network protocol stack.

3. The RoCE network end-system traffic monitoring method according to claim 1, wherein The RoCE network traffic monitoring system further includes: a timing processing module; The timing processing module is respectively connected to the temporary data sub-library and the long-term data sub-library; Before the traffic data transmission parsing module parses the work request of the application program to implement data transmission and obtain the data transmission parameters corresponding to the work request, it further includes: The timing processing module adds the temporary different process IDs, the temporary different sender QPNs, the temporary different receiver QPNs, the transmission data volumes of the temporary different operation types, and the reception data volumes of the temporary different operation types to the process IDs, the sender QPNs, the receiver QPNs, the transmission data volumes of the long-term different operation types, and the reception data volumes of the long-term different operation types according to a preset update processing period, and then clears the temporary data in the temporary data sub-library.

4. The RoCE network end-system traffic monitoring method according to claim 1, characterized in that, The traffic data display module obtains and displays the updated transmission data according to the work request, specifically: The traffic data display module obtains the corresponding data from the updated statistical database according to the work request, and displays them in both picture or text format.

Images