RoCE network end system flow monitoring method

Abstract

According to the RoCE network end system flow monitoring method provided by the invention, the RoCE network end system is constructed, and the system is composed of a flow data transmission analysis module, a flow data statistics module and a flow data display module; the data transmission module analyzes the work request of the application program and then realizes data transmission to obtain a data transmission parameter corresponding to the work request; after the transmission is completed, counting the transmitted data according to the data transmission parameters through a flow data counting module, and updating the counted transmission data after the counting is completed; and after the updating is completed, acquiring the updated transmission data through a flow data display module according to the work request and displaying the updated transmission data. Compared with the prior art, by constructing the RoCE network end system, statistical monitoring can be realized in the RoCE network protocol stack so as to realize effective monitoring of the RoCE network flow.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a method for monitoring flow of a RoCE network end system. Background Art

[0002] RoCE network is an Ethernet-based RDMA (Remote Direct Memory Access) network. The performance indicators of RoCE network, such as end-to-end latency and CPU resource utilization, are comparable to those of Infiniband network, but its deployment cost is much lower than that of Infiniband network. Therefore, it is widely deployed and applied in data center networks for application scenarios such as AI and supercomputing that have high requirements for network latency and CPU utilization.

[0003] At present, the research on network traffic monitoring is mainly focused on the field of Ethernet technology. Due to the different network architecture and protocol stack technology implementation, the network monitoring technology applied to Ethernet cannot be used in RoCE networks; the method of distinguishing network traffic in Ethernet in the form of quintuples is also not applicable to RoCE networks. Real-time monitoring of network traffic is extremely important for network status maintenance, network management, and network fault diagnosis.

[0004] Therefore, it is an urgent problem to be solved by those skilled in the art to provide a RoCE network end system traffic monitoring method for realizing effective statistical monitoring of RoCE network traffic in a RoCE network protocol stack. Summary of the invention

[0005] The object of the present invention is to provide a RoCE network end system traffic monitoring method, which has clear logic, is safe, effective, reliable and easy to operate, and can perform statistical monitoring of multiple factors to improve the monitoring accuracy.

[0006] Based on the above objectives, the technical solution provided by the present invention is as follows: A method for monitoring RoCE network end system traffic, comprising the following steps: Construct a RoCE network traffic monitoring system consisting of a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module; The flow data transmission parsing module implements data transmission after parsing the work request of the application program to obtain data transmission parameters corresponding to the work request; The traffic data statistics module counts the transmitted data according to the data transmission parameters, and updates the counted transmission data after the statistics are completed; The traffic data display module obtains and displays the updated transmission data according to the work request.

[0007] Preferably, the flow data transmission parsing module comprises: a sending parsing submodule and a receiving parsing submodule; The data transmission parameters include: sending parameters and receiving parameters; The traffic data transmission parsing module implements data transmission after parsing the work request of the application program to obtain data transmission parameters corresponding to the work request, specifically: The sending and parsing submodule receives the work request of the application program, and parses the work request to obtain the sending parameter; The receiving and parsing submodule obtains an application polling work completion instruction, and parses the polling work completion instruction to obtain the receiving parameter.

[0008] Preferably, the sending parsing submodule and the receiving parsing submodule both parse the work request and the polling work completion instruction correspondingly in the RoCE network protocol stack data format.

[0009] Preferably, the traffic data statistics module includes: a data statistics module, a data update module and a statistics database; The data statistics module completes the statistics of the sent data according to the sending parameters, and sends the sent data to the data update module; The data statistics module completes the received data statistics according to the received parameters, and sends the received data to the data update module; The data updating module updates the statistical database according to the sending data and the receiving data.

[0010] Preferably, the sending parameters and the receiving parameters are specifically: process ID, sending end QPN, receiving end QPN, operation type and data size.

[0011] Preferably, the statistical database includes: a temporary data sub-database and a long-term data sub-database; The data updating module updates the statistical database according to the sent data and the received data, specifically: The data updating module updates the temporary different process ID, the temporary different sender QPN, the temporary different receiver QPN, the amount of sent data of the temporary different operation types and the amount of received data of the temporary different operation types in the temporary data sub-library according to the sent data and the received data; According to a preset data storage cycle, the temporary data sub-library converts the temporary different process IDs, the temporary different sender QPNs, the temporary different receiver QPNs, the sending data amounts of the temporary different operation types, and the receiving data amounts of the temporary different operation types into long-term different process IDs, long-term different sender QPNs, long-term different receiver QPNs, the sending data amounts of the long-term different operation types, and the receiving data amounts of the long-term different operation types, and stores them in the long-term data sub-library.

[0012] Preferably, the RoCE network traffic monitoring system further includes: a timing processing module; The timing processing module is connected to the temporary data sub-library and the long-term data sub-library respectively; Before the traffic data transmission parsing module implements data transmission after parsing the work request of the application program to obtain the data transmission parameters corresponding to the work request, it also includes: The timing processing module adds the temporary different process ID, the temporary different sending end QPN, the temporary different receiving end QPN, the sending data amount of the temporary different operation type and the receiving data amount of the temporary different operation type to the process ID, the sending end QPN, the receiving end QPN, the sending data amount of the long-term different operation type and the receiving data amount of the long-term different operation type according to the preset update processing cycle, and then clears the temporary data in the temporary data sub-library.

[0013] Preferably, the traffic data display module obtains and displays the updated transmission data according to the work request, specifically: The flow data display module obtains corresponding data from the updated statistical database according to the work request, and displays them in the form of pictures or text.

[0014] The RoCE network end system traffic monitoring method provided by the present invention is to construct a RoCE network end system, which is composed of a traffic data transmission parsing module, a traffic data statistics module and a traffic data display module; the data transmission is realized after the work request of the application is parsed by the data transmission module, and the data transmission parameters corresponding to the work request are obtained; after the transmission is completed, the transmitted data is counted according to the data transmission parameters by the traffic data statistics module, and the statistical transmission data is updated after the statistics are completed; after the update is completed, the updated transmission data is obtained and displayed according to the work request by the traffic data display module.

[0015] Compared with the prior art, the present invention can realize effective monitoring of RoCE network traffic in the RoCE network protocol stack by constructing a RoCE network end system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flow chart of a RoCE network end system traffic monitoring method provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a RoCE network traffic monitoring system provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a flow data transmission parsing module provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a traffic data statistics module provided in an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a statistical database provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The embodiments of the present invention are written in a progressive manner.

[0020] The embodiment of the present invention provides a method for monitoring the flow of a RoCE network end system, which mainly solves the technical problem in the prior art that the RoCE network is not applicable and thus the RoCE network flow cannot be effectively monitored.

[0021] like Figure 1 and Figure 2 As shown, a RoCE network end system traffic monitoring method includes the following steps: S1. Construct a RoCE network traffic monitoring system consisting of a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module; S2. The traffic data transmission parsing module implements data transmission after parsing the application's work request to obtain the data transmission parameters corresponding to the work request; S3. The traffic data statistics module performs statistics on the transmitted data according to the data transmission parameters, and updates the transmission data after the statistics are completed; S4. The traffic data display module obtains and displays the updated transmission data according to the work request.

[0022] In step S1, a RoCE network traffic monitoring system is constructed. The system runs on the RoCE network end system and is deployed together with the RoCE network protocol stack. The system is provided with a traffic data transmission parsing module, a traffic data statistics module and a traffic data display module; In step S2 to step S4, the traffic data statistics module is connected to the traffic data transmission parsing module and the traffic data display module respectively; when receiving a work request from an application, the traffic data transmission parsing module parses the work request and implements data transmission, thereby obtaining the data transmission parameters corresponding to the work request and transmitting them to the traffic data statistics module; the traffic data statistics module counts the data according to the data transmission parameters, updates the counted transmission data after the statistics are completed, and sends the updated transmission data to the traffic data display module; the traffic data display module obtains the updated transmission data according to the work request and displays it.

[0023] like Figure 3 As shown, preferably, the flow data transmission parsing module includes: a sending parsing submodule and a receiving parsing submodule; Data transmission parameters include: sending parameters and receiving parameters; The traffic data transmission parsing module parses the application's work request and implements data transmission to obtain the data transmission parameters corresponding to the work request, specifically: The sending parsing submodule receives the work request of the application and parses the work request to obtain the sending parameters; The receiving and parsing submodule obtains the application program polling work completion instruction, and parses the polling work completion instruction to obtain the receiving parameters.

[0024] During actual application, the traffic data transmission parsing module includes: a sending parsing submodule and a receiving parsing submodule; the corresponding data transmission parameters include: sending parameters and receiving parameters; therefore, the corresponding traffic data transmission parsing module parses work requests into two types, one is that the sending parsing submodule receives the application's work request, parses the work request, and obtains the corresponding sending parameters; the other is that the receiving parsing submodule obtains the application's polling work completion instruction, parses the polling work completion instruction, and obtains the receiving parameters.

[0025] In this embodiment, the sending parsing submodule intercepts the sending work request sent by the application, parses the sending work request according to the RoCE network protocol stack data format, and sends the sending parameters to the data statistics module; the receiving parsing submodule intercepts the work completion of the application polling, identifies and parses the receiving completion according to the RoCE network protocol stack, and sends the receiving parameters to the data statistics module; Preferably, the sending parsing submodule and the receiving parsing submodule both correspond to the parsing work request and the polling work completion instruction in the RoCE network protocol stack data format.

[0026] In actual use, both the sending parsing submodule and the receiving parsing submodule correspond to the parsing work request and the polling work completion instruction in a specific data format; In this embodiment, the specific data format selected is the RoCE network protocol stack data format; like Figure 4 As shown, preferably, the traffic data statistics module includes: a data statistics module, a data update module and a statistics database; The data statistics module completes the sent data statistics according to the sent parameters, and sends the sent data to the data update module; The data statistics module completes the received data statistics according to the received parameters and sends the received data to the data update module; The data update module updates the statistical database according to the sent data and received data.

[0027] During actual application, the traffic data statistics module is provided with a data statistics module, a data update module and a statistical database; the data statistics module completes the sending data statistics and the receiving data statistics according to the sending parameters and the receiving parameters and sends the completed sending data and the receiving data to the data update module, and the data update module updates the corresponding sending data and the receiving data in the statistical database.

[0028] Preferably, the sending parameters and receiving parameters are specifically: process ID, sending end QPN, receiving end QPN, operation type and data size.

[0029] In actual use, the sending parameters and receiving parameters are: process ID, sender QPN, receiver QPN, operation type and data size; In this embodiment, QP number (Queue Pair number, QPN): Each QP pair on the RDMA node has a unique number QPN to identify it.

[0030] like Figure 5 As shown, preferably, the statistical database includes: a temporary data sub-base and a long-term data sub-base; The data update module updates the statistical database according to the sent data and received data, specifically: The data update module updates the temporary different process ID, the temporary different sender QPN, the temporary different receiver QPN, the temporary different operation type send data volume and the temporary different operation type receive data volume in the temporary data sub-data base according to the send data and the receive data; According to the preset data storage cycle, the temporary data sub-library converts the temporary different process IDs, temporary different sender QPNs, temporary different receiver QPNs, the sending data volumes of temporary different operation types and the receiving data volumes of temporary different operation types into the long-term different process IDs, long-term different sender QPNs, long-term different receiver QPNs, the sending data volumes of long-term different operation types and the receiving data volumes of long-term different operation types and stores them in the long-term data sub-library.

[0031] During actual application, the statistical database is designed as needed: a temporary data sub-library and a long-term data sub-library; the data update module updates the temporary data sub-library according to the sending data and the receiving data, and the temporary data includes: the sending data volume of different process IDs, different sending end QPNs, different receiving end QPNs, different operation types, and the receiving data volume of different operation types; after the update is completed, the temporary data is periodically converted into long-term data according to the set data storage cycle, specifically: long-term data includes: the sending data volume of different process IDs, different sending end QPNs, different receiving end QPNs, different operation types, and the receiving data volume of different operation types, and after the conversion is completed, it is stored in the long-term data sub-library.

[0032] In this embodiment, the statistical database stores various types of long-term and temporary statistical data; the long-term statistical data include but are not limited to: the long-term amount of sent data with different process IDs, different sender QPNs, different receiver QPNs, and different operation types, and the long-term amount of received data with different process IDs, different sender QPNs, different receiver QPNs, and different operation types; the temporary statistical data include but are not limited to: the temporary amount of sent data with different process IDs, different sender QPNs, different receiver QPNs, and different operation types, and the temporary amount of received data with different process IDs, different sender QPNs, different receiver QPNs, and different operation types.

[0033] Preferably, the RoCE network traffic monitoring system further comprises: a timing processing module; The timing processing module is connected to the temporary data sub-database and the long-term data sub-database respectively; After the traffic data transmission parsing module parses the work request of the application program and implements data transmission to obtain the data transmission parameters corresponding to the work request, it also includes: The timing processing module adds the temporarily different process IDs, temporarily different sender QPNs, temporarily different receiver QPNs, the amount of sent data of temporarily different operation types, and the amount of received data of temporarily different operation types to the process ID, sender QPN, receiver QPN, the amount of sent data of long-term different operation types, and the amount of received data of long-term different operation types according to the preset update processing cycle, and then clears the temporary data in the temporary data sub-library.

[0034] During actual application, a timing processing module is also provided in the RoCE network end system; the timing processing module is connected to the temporary data sub-library and the long-term data sub-library respectively; before the traffic data transmission parsing module parses, the timing processing module adds the temporary data according to the preset update cycle: different process IDs, different sender QPNs, different receiver QPNs, the amount of sent data of different operation types and the amount of received data of different operation types to the process ID, sender QPN, receiver QPN, the amount of sent data of different operation types in the long term and the amount of received data of different operation types in the long term, and then clears the above temporary data in the temporary data sub-library.

[0035] Preferably, the traffic data display module obtains and displays the updated transmission data according to the work request, specifically: The traffic data display module obtains the corresponding data from the updated statistical database according to the work request, and displays them in picture or text format.

[0036] During actual use, the traffic data display module retrieves the corresponding data from the updated statistical database according to the work request, and displays it in picture or text format to realize the monitoring of traffic data changes.

[0037] In the embodiments provided in the present application, it should be understood that the disclosed methods can be implemented in other ways. The system embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0038] In addition, all functional modules in the embodiments of the present invention may be integrated into one processor, or each module may be a separate device, or two or more modules may be integrated into one device; each functional module in the embodiments of the present invention may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0039] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above method embodiment are executed; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.

[0040] It should be understood that the use of "system", "device", "unit" and / or "module" in this application is only a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0041] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0042] If a flow chart is used in the present application, the flow chart is used to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or a certain step or several steps of operations can be removed from these processes.

[0043] The above is a detailed introduction to a RoCE network end system traffic monitoring method provided by the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest scope consistent with the principles and novel features disclosed in this article.

Claims

1. A RoCE network end system traffic monitoring method, characterized in that: The steps include: Construct a RoCE network traffic monitoring system consisting of a traffic data transmission parsing module, a traffic data statistics module, and a traffic data display module; The flow data transmission parsing module implements data transmission after parsing the work request of the application program to obtain data transmission parameters corresponding to the work request; The traffic data statistics module counts the transmitted data according to the data transmission parameters, and updates the counted transmission data after the statistics are completed; The traffic data display module obtains and displays the updated transmission data according to the work request.

2. The RoCE network end system traffic monitoring method according to claim 1, characterized in that: The flow data transmission analysis module includes: a sending analysis submodule and a receiving analysis submodule; The data transmission parameters include: sending parameters and receiving parameters; The traffic data transmission parsing module implements data transmission after parsing the work request of the application program to obtain data transmission parameters corresponding to the work request, specifically: The sending and parsing submodule receives the work request of the application program, and parses the work request to obtain the sending parameter; The receiving and parsing submodule obtains an application polling work completion instruction, and parses the polling work completion instruction to obtain the receiving parameter.

3. The RoCE network end system traffic monitoring method as claimed in claim 2, characterized in that: The sending parsing submodule and the receiving parsing submodule both parse the work request and the polling work completion instruction in the RoCE network protocol stack data format.

4. The RoCE network end system traffic monitoring method as claimed in claim 2, characterized in that: The traffic data statistics module includes: a data statistics module, a data update module and a statistics database; The data statistics module completes the statistics of the sent data according to the sending parameters, and sends the sent data to the data update module; The data statistics module completes the received data statistics according to the received parameters, and sends the received data to the data update module; The data updating module updates the statistical database according to the sending data and the receiving data.

5. The RoCE network end system traffic monitoring method as claimed in claim 4, characterized in that: The sending parameters and the receiving parameters are specifically: process ID, sending end QPN, receiving end QPN, operation type and data size.

6. The RoCE network end system traffic monitoring method as claimed in claim 5, characterized in that: The statistical database includes: a temporary data sub-database and a long-term data sub-database; The data updating module updates the statistical database according to the sent data and the received data, specifically: The data updating module updates the temporary different process ID, the temporary different sender QPN, the temporary different receiver QPN, the amount of sent data of the temporary different operation types and the amount of received data of the temporary different operation types in the temporary data sub-library according to the sent data and the received data; According to a preset data storage cycle, the temporary data sub-library converts the temporary different process IDs, the temporary different sender QPNs, the temporary different receiver QPNs, the sending data amounts of the temporary different operation types, and the receiving data amounts of the temporary different operation types into long-term different process IDs, long-term different sender QPNs, long-term different receiver QPNs, the sending data amounts of the long-term different operation types, and the receiving data amounts of the long-term different operation types, and stores them in the long-term data sub-library.

7. The RoCE network end system traffic monitoring method according to claim 6, characterized in that: The RoCE network traffic monitoring system also includes: a timing processing module; The timing processing module is connected to the temporary data sub-library and the long-term data sub-library respectively; Before the traffic data transmission parsing module implements data transmission after parsing the work request of the application program to obtain the data transmission parameters corresponding to the work request, it also includes: The timing processing module adds the temporary different process ID, the temporary different sending end QPN, the temporary different receiving end QPN, the sending data amount of the temporary different operation type and the receiving data amount of the temporary different operation type to the process ID, the sending end QPN, the receiving end QPN, the sending data amount of the long-term different operation type and the receiving data amount of the long-term different operation type according to the preset update processing cycle, and then clears the temporary data in the temporary data sub-library.

8. The RoCE network end system traffic monitoring method as claimed in claim 4, characterized in that: The traffic data display module obtains and displays the updated transmission data according to the work request, specifically: The flow data display module obtains corresponding data from the updated statistical database according to the work request, and displays them in the form of pictures or text.

Images