An ORT management method and system

Abstract

An ORT management system provided by the present invention includes: a writing management module, a shared storage space scheduling module, and a request processing module; the shared storage space scheduling module is respectively connected to the writing management module and the request processing module; the writing management module is connected to the request processing module. Compared with the prior art, the dynamic ORT management system supporting RDMA reliable transmission in the present invention performs shared storage management on the ORs of all active reliable service transmission type QPs based on limited on-chip storage resources. The storage space occupied by each QP can be dynamically allocated and recycled as needed, thereby greatly reducing space waste. In addition, when the RDMA network card performs completion confirmation on the ORs of the reliable transmission service type QPs, there is no need to access off-chip storage or host memory again, which can significantly reduce the processing delay and bandwidth attenuation of the network card. This method has the same beneficial effects.

Description

Technical Field

[0001] This application relates to the technical field of remote direct memory access, and particularly to a dynamic ORT management method and system that support reliable RDMA transmission. Background Art

[0002] In recent years, with the rapid development of technologies such as artificial intelligence and cloud computing, RoCEv2 (RDMA over Converged Ethernet) technology has been widely applied in data centers. Its characteristics of high bandwidth, low latency, multi-link, and zero-copy enable it to perform more excellently in multi-node distributed computing. In the application scenarios of data centers, the number of links it needs to support often exceeds 10K, and the link bandwidth is also mostly in the order of hundreds of Gbps.

[0003] Based on the requirements of RDMA protocol technology implementation, RDMA network cards need to consume a large amount of on-chip storage resources for caching QPC (Queue Pair Context), CQC (Completion Queue Context), and ORT (Outstanding Request Table), etc. Among them, ORT caches the OR (Outstanding Request) of the reliable transmission service type QP (Queue Pair). The OR is obtained by the requester extracting some fields from the outstanding SQE (Send Queue Element). When the network card receives a response message, it reads the corresponding OR from the ORT to determine whether the corresponding SQE is completed and constructs the corresponding CQE (Completion Queue Element) for uploading. However, ORT often requires a large amount of storage resources. Taking a network card that supports 10K QPs, each QP caches 500 ORs, and each OR entry size is 32B for a simple estimate, it requires 160MB of on-chip storage. However, the total amount of on-chip RAM (Random Access Memory) storage resources of RDMA network cards implemented based on FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit) is difficult to reach hundreds of MB, and the cascading of a large number of RAM blocks makes layout, wiring, and timing convergence more difficult. Therefore, it is difficult to implement independent caching of ORs for each QP using such a large amount of storage resources.

[0004] For this problem, there are currently two common solutions. One method is that when the network card receives a response packet and needs to confirm whether it is completed, it reads the SQE from memory again to verify whether the confirmation is completed; the other method is to directly deploy the ORT in off-chip storage, such as DDR (Double Data Rate) memory. However, the former method requires occupying the PCIE bandwidth to read the SQE again, and the latter method requires the support of an external storage chip. Moreover, both of these methods also bring a relatively large delay overhead and performance loss. When the number of active QPs is large and the message length corresponding to the SQE is short, the access granularity to memory or external storage will be finer, and at this time, the performance degradation of the RDMA network card is particularly significant.

[0005] In view of this, it is an urgent technical problem for those skilled in the art to provide a method and system for dynamic ORT management based on limited on-chip storage resources to support RDMA reliable transmission. Summary of the Invention

[0006] To solve the above technical problems, the object of the present invention is to provide a dynamic ORT management method and system that support RDMA reliable transmission, and at the same time have the characteristics of only relying on limited on-chip storage resources without accessing off-chip storage or host memory, and can significantly reduce the processing delay and bandwidth attenuation of the network card.

[0007] The first object of the present invention is to provide a dynamic ORT management system that supports RDMA reliable transmission;

[0008] The technical solution provided by the present invention is as follows:

[0009] An ORT management system includes: a write management module, a shared storage space scheduling module, and a request processing module;

[0010] The shared storage space scheduling module is respectively connected to the write management module and the request processing module; the write management module is connected to the request processing module;

[0011] The write management module is used to maintain the status information of active QPs and control the OR of the active QPs to be written into the shared storage space of the shared storage space scheduling module according to the status information;

[0012] The shared storage space scheduling module is used to provide the shared storage space and realize the access of the OR of the active QPs by dynamically allocating and recycling the shared storage space;

[0013] The request processing module is used to obtain a response message, and based on the response message, determine whether the OR of the corresponding QP has been processed. If so, it notifies the shared storage space scheduling module to recycle the shared storage space occupied by the processed OR.

[0014] Preferably, the writing management module includes: a QP information management component and an OR writing control component;

[0015] The QP information management component is respectively connected to the OR writing control component and the request processing module; the OR writing control component is also connected to the shared storage space scheduling module;

[0016] The QP information management component is used to maintain the status information of active QPs;

[0017] The OR writing control component is used to control the OR of the active QP to be written into the shared storage space of the shared storage space scheduling module according to the status information.

[0018] Preferably, the shared storage space scheduling module includes: a shared OR storage component, a storage relationship maintenance component, an idle block management component, and a block space recycling component;

[0019] The shared OR storage component is connected to the OR writing control component and the request processing module; the storage relationship maintenance component is respectively connected to the OR writing control component and the request processing module; the idle management block component is respectively connected to the QP information management component and the block space recycling component; the block space recycling component is respectively connected to the QP information management component and the request processing module;

[0020] The shared OR storage component is used to provide the shared storage space for storing and reading the OR written by the OR writing control component. Among them, the shared storage space is composed of multiple storage blocks, and each storage block is used to store a preset number of ORs;

[0021] The storage relationship maintenance component is used to maintain the storage order relationship between the storage blocks in the shared storage space;

[0022] The idle block management component is used to manage the allocation and recycling of idle storage blocks in the shared storage space;

[0023] The block space recycling component is used to recycle the storage blocks of the shared storage space occupied by the processed OR.

[0024] Preferably, the status information includes: the next OR write address, the next OR read address, the number of occupied storage blocks, the maximum occupied storage block threshold, the retransmission waiting status flag, the SSN of the latest SQE sent by the requesting end, the SSN of the retransmission start SQE, and the OR cache address of the retransmission start SQE.

[0025] Preferably, when the OR write control component executes the control of the OR write of the active QP to the shared storage space of the shared storage space scheduling module according to the status information, it is specifically used for:

[0026] Judge whether the OR to be written is the OR of a new SQE according to the SSN of the latest SQE sent by the requesting end in the status information;

[0027] When the OR to be written is the OR of a new SQE, and it is judged according to the next OR write address in the status information, the number of occupied storage blocks in the status information, and the maximum occupied storage block threshold that the remaining space of the current storage block meets the minimum space condition for writing the OR in the new SQE, allocate storage block space for the OR in the new SQE, and write the OR into the storage block space;

[0028] When the OR to be written is the OR of a new SQE, and it is judged according to the next OR write address, the number of occupied storage blocks, and the maximum occupied storage block threshold that the remaining space of the current storage block does not meet the minimum space condition for writing the OR in the new SQE, pause processing the current new SQE, and wait for the free block management component to allocate free storage blocks and then continue to process the OR of the current new SQE;

[0029] When the OR to be written is the OR of a retransmitted SQE, and the SQE corresponding to the OR to be written is the first SQE of the retransmission, write the OR of the retransmitted SQE into the corresponding allocated storage block space;

[0030] When an OR is written to the shared storage space, notify the QP information management component to update the status information.

[0031] Preferably, when the free block management component executes the allocation and recycling of free storage blocks in the shared storage space, it is specifically used for:

[0032] Cache the numbers of free storage blocks in the shared storage space through a FIFO queue;

[0033] When the network card starts up, all storage blocks in the shared storage space are free storage blocks, and write the numbers of all storage blocks into the FIFO queue;

[0034] When the current storage block space written by the OR write control component is insufficient, allocate a storage block number from the FIFO queue and allocate it to the OR write control component for OR writing;

[0035] When an idle storage block is recycled by the block space recycling component, write the recycled storage block number into the FIFO queue.

[0036] Preferably, when the block space recycling component executes the recycling of the storage block occupying the shared storage space where the processed OR is located, it is specifically used for:

[0037] Independently count the number of completed ORs in each storage block of the shared storage space through a counter;

[0038] When the number of completed ORs in the current storage block reaches the preset number, clear the counter of the current storage block, and send the current storage block number to the idle block management component for recycling of the idle storage block.

[0039] Preferably, when the storage relationship maintenance component executes the maintenance of the storage order relationship between each storage block in the shared storage space, it is specifically used for:

[0040] Maintain the storage order link relationship between each storage block in the shared storage space through a linked list;

[0041] The linked list includes multiple entries, the address of each entry corresponds one-to-one with the storage block number of the shared storage space, and the content of the linked list is the next storage block number to be written by the corresponding QP after the current storage block space is used up;

[0042] The content of the linked list is updated after the write operation of the OR write control component is executed.

[0043] Preferably, when the request processing module executes obtaining a response message, and based on the response message, determines whether the OR corresponding to the QP is processed, and if so, notifies the shared storage space scheduling module to recycle the shared storage space occupied by the processed OR, it is specifically used for:

[0044] Obtain a response message, read the OR corresponding to the QP according to the QPN and the status information of the response message, and based on the information in the OR, determine whether the OR corresponding to the QP is processed. If so, generate a CQE according to the corresponding OR, and notify the shared storage space scheduling module to recycle the shared storage space occupied by the processed OR.

[0045] The second object of the present invention is to provide a dynamic ORT management method supporting RDMA reliable transmission;

[0046] The technical solution provided by the present invention is as follows:

[0047] An ORT management method is applied to an ORT management system. The ORT management system includes a write management module, a shared storage space scheduling module, and a request processing module. The method includes:

[0048] Using the write management module to maintain the status information of active QPs, and controlling the OR of the active QPs to write to the shared storage space of the shared storage space scheduling module according to the status information;

[0049] Using the request processing module to obtain a response message, and judging whether the OR of the corresponding QP is processed completed according to the response message. If so, notifying the shared storage space scheduling module to recycle the shared storage space occupied by the processed OR;

[0050] Using the shared storage space scheduling module to provide the shared storage space, and realizing the access of the OR of the active QPs by dynamically allocating and recycling the shared storage space.

[0051] An ORT management system provided by the present invention includes a write management module, a shared storage space scheduling module, and a request processing module; the shared storage space scheduling module is respectively connected to the write management module and the request processing module; the write management module is connected to the request processing module. Compared with the prior art, the dynamic ORT management system supporting RDMA reliable transmission according to the present invention performs shared storage management on the ORs of all active reliable service transmission type QPs based on limited on-chip storage resources. The storage space occupied by each QP can be dynamically allocated and recycled as needed, thus greatly reducing space waste. When the RDMA network card completes the confirmation of the OR of the reliable transmission service type QP, there is no need to access off-chip storage or host memory again, which can significantly reduce the processing delay and bandwidth attenuation of the network card. This method has the same beneficial effects. Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0053] Figure 1 It is a schematic structural diagram of an ORT management system in an embodiment of the present invention;

[0054] Figure 2Schematic diagram of another ORT management system in an embodiment of the present invention;

[0055] Figure 3 Flowchart of an ORT management method in an embodiment of the present invention. Detailed implementation manners

[0056] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0057] As Figure 1 shown, an embodiment of the present invention provides an ORT management system, including: a writing management module, a shared storage space scheduling module, and a request processing module;

[0058] The shared storage space scheduling module is respectively connected to the writing management module and the request processing module; the writing management module is connected to the request processing module;

[0059] The writing management module is used to maintain the status information of the active QP and control the OR of the active QP to be written into the shared storage space of the shared storage space scheduling module according to the status information;

[0060] The shared storage space scheduling module is used to provide a shared storage space and realize the access of the OR of the active QP by dynamically allocating and recycling the shared storage space;

[0061] The request processing module is used to obtain a response message and determine whether the OR of the corresponding QP has been processed according to the response message. If so, it notifies the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed.

[0062] It should be noted that the active QP is the QP for which an SQE is being processed by the network card; compared with allocating a dedicated OR cache space for each QP, the shared storage space is a storage area shared by all QPs for caching ORs. Only the active QP for which an SQE is being processed by the network card will occupy the shared storage space, that is, the shared storage space only needs to store the ORs corresponding to all the SQEs processed by the network card during the period from when the network card processes the first SQE and writes the OR to when the first SQE is completed and the storage space occupied by the corresponding OR is recycled. Based on this, the number of corresponding SQEs can be calculated according to the amount of data sent by the network card within the network round-trip time, and then the size of the shared storage space can be estimated. The size of the shared storage space is estimated with reference to the following formula:

[0063] ORT_Size = IO_Bandwidth * Avg_RTT / Avg_Message_Length * OR_Width

[0064] Among them, ORT_Size is the size of the shared storage space, IO_Bandwidth is the network card interface bandwidth, Avg_RTT is the average round-trip delay, Avg_Message_Length is the average message length of SQE, and OR_Width is the bit width of OR. Estimated with a network card interface bandwidth of 100 Gbps, an average round-trip delay of 200 us, an average message length of 1 KB for SQE, and an OR bit width of 32 B, ORT_Size is 80 KB. Storage resources of this magnitude can be easily deployed in FPGA or ASIC. Therefore, this solution only performs shared storage management on ORT through limited on-chip resources without relying on off-chip storage resources for storage.

[0065] Compared with the prior art, the dynamic ORT management system of the present invention that supports RDMA reliable transmission performs shared storage management on the ORs of all active reliable service transmission type QPs based on limited on-chip storage resources. The storage space occupied by each QP can be dynamically allocated and recycled as needed, thus greatly reducing space waste. When the RDMA network card completes the confirmation of the OR of the reliable transmission service type QP, there is no need to access off-chip storage or host memory again, which can significantly reduce the processing delay and bandwidth attenuation of the network card.

[0066] Such as Figure 2 , the write management module includes: a QP information management component and an OR write control component;

[0067] The QP information management component is respectively connected to the OR write control component and the request processing module; the OR write control component is also connected to the shared storage space scheduling module;

[0068] The QP information management component is used to maintain the status information of active QPs;

[0069] The OR write control component is used to control the writing of the ORs of active QPs to the shared storage space of the shared storage space scheduling module according to the status information.

[0070] It should be noted that the OR write control component is also used to update the status information of the QP information management component after the OR is written to the shared storage space of the shared storage space scheduling module.

[0071] Such as Figure 2 As shown, the shared storage space scheduling module includes: a shared OR storage component, a storage relationship maintenance component, an idle block management component, and a block space recycling component;

[0072] The shared OR storage component is connected to the OR write control component and the request processing module; the storage relationship maintenance component is respectively connected to the OR write control component and the request processing module; the free block management component is respectively connected to the QP information management component and the block space recycling component; the block space recycling component is respectively connected to the QP information management component and the request processing module;

[0073] The shared OR storage component is used to provide a shared storage space for storing and reading the ORs written by the OR write control component. The shared storage space is composed of multiple storage blocks, and each storage block is used to store a preset number of ORs;

[0074] The storage relationship maintenance component is used to maintain the storage order relationship between the storage blocks in the shared storage space;

[0075] The free block management component is used to manage the allocation and recycling of free storage blocks in the shared storage space;

[0076] The block space recycling component is used to recycle the storage blocks in the shared storage space occupied by the ORs that have completed processing.

[0077] In the actual application process, the shared storage space provided by the shared OR storage component is composed of on-chip RAM block cascading. The bit width of the OR is determined according to the estimation result of the application scenario. The shared storage space is divided into 2^m storage blocks with equal size and continuous addresses. Each storage block can cache 2^n ORs. Each storage block is assigned a number with a bit width of m, which is the high m bits of the access address of the shared OR space. The access address of the shared storage space consists of two parts. The high m bits are the storage block number, and the low n bits are the number of the OR cached in this storage block. Among them, the values of m and n are determined according to the specific implementation of the network card hardware.

[0078] As an implementation, taking the network card interface bandwidth of 100Gbps, the average network round-trip delay of 200us, the average message length of SQE of 1KB, and the OR bit width of 32B as an example, according to the estimation formula, the shared storage space needs to be more than 80KB. Then the bit width of the shared storage space is set to 32B and the depth is 4096. Accordingly, considering the pipelining processing of storage block access and the number of active QPs, etc., the value of n can be set to 4 and the value of m is 8. That is, the shared storage space is divided into 256 storage blocks, each of which can cache 16 ORs. Then the network card can support the parallel processing of SQEs of up to 256 QPs at most.

[0079] Preferably, the status information includes: the next OR write address, the next OR read address, the number of occupied storage blocks, the maximum threshold of occupied storage blocks, the retransmission waiting status flag, the SSN of the latest SQE sent by the requesting end, the SSN of the retransmission start SQE, and the OR cache address of the retransmission start SQE.

[0080] It should be noted that the next OR write address is the access address of the next OR write shared storage space for each active QP, and the next OR write address in the status information is updated when the OR write control component writes a new OR; the next OR read address is the access address for each active QP to read the next OR from the shared storage space, and the next OR read address is updated when the current OR of the request processing module is processed; the number of occupied storage blocks is used to count the number of storage blocks already occupied by the current QP. The number of occupied storage blocks is incremented by one when a new storage block is allocated to the corresponding QP and decremented by one when the storage block of the corresponding QP is recycled; the maximum threshold of occupied storage blocks is the maximum number of storage blocks that a QP can occupy. Combined with the number of occupied storage blocks of the corresponding QP, it can be used to determine whether the current QP can continue to be allocated a new free block to prevent the shared storage space from being completely occupied by a few QPs with abnormal transmissions. The maximum threshold of occupied storage blocks is configured during QP initialization; the retransmission waiting flag is used to support the retransmission operation of reliable transmission. Specifically, it is used to indicate that the current QP has triggered a retransmission operation and is waiting for the OR write of the first retransmission SQE. The retransmission waiting flag is set to valid when the retransmission operation is triggered and set to invalid when the OR of the first retransmission SQE of the QP is written or an OR of this QP is completed; the SSN of the latest SQE sent by the requesting end is used to detect whether the OR being written is a new OR, and it is updated to the SSN in the new OR after the new OR is written; the SSN of the retransmission start SQE is used to detect whether the SQE corresponding to the OR being written is the first SQE of this retransmission operation. The SSN of the retransmission start SQE is updated when the corresponding QP triggers a retransmission, and the updated value is the SSN in the OR that triggers the retransmission; the OR cache address of the retransmission start SQE is used to cache the OR of the first SQE of the retransmission operation, and it is updated when the corresponding QP triggers a retransmission, and the updated value is the cache address of the OR that triggers the retransmission in the shared storage space.

[0081] Preferably, when the OR write control component executes the shared storage space of the OR write shared storage space scheduling module for the active QP according to the status information, it is specifically used for:

[0082] Judge whether the OR to be written is an OR of a new SQE according to the SSN of the latest SQE sent by the requesting end in the status information;

[0083] When the OR to be written is the OR of the new SQE, and it is determined according to the next OR write address in the status information, the number of occupied storage blocks in the status information, and the maximum occupied storage block threshold that the remaining space of the current storage block meets the minimum space condition for writing the OR in the new SQE, allocate storage block space for the OR in the new SQE and write the OR into the storage block space;

[0084] When the OR to be written is the OR of the new SQE, and it is determined according to the next OR write address, the number of occupied storage blocks, and the maximum occupied storage block threshold that the remaining space of the current storage block does not meet the minimum space condition for writing the OR in the new SQE, suspend the processing of the current new SQE and wait for the free block management component to allocate a free storage block and then continue to process the OR of the current new SQE;

[0085] When the OR to be written is the OR of the retransmission SQE, and the SQE corresponding to the OR to be written is the first SQE of the retransmission, write the OR of the retransmission SQE into the corresponding allocated storage block space;

[0086] When an OR is written into the shared storage space, notify the QP information management component to update the status information.

[0087] It should be noted that when the OR of the new SQE or the OR corresponding to the first SQE of the retransmission operation is written into the shared storage space, the OR write control component notifies the QP information management component to update the next OR write address. The lower n bits of the next OR write address in the status information are incremented by one after the OR write control component writes a new OR corresponding to a QP. If the lower n bits of the next OR write address are all 0 and the OR write control component finds that there is a new OR to be written for the corresponding QP, the QP information management component then obtains a new free storage block number from the free block management component as the higher m bits of the next OR write address.

[0088] In the actual application process, when the OR write control component executes the shared storage space of the shared storage space scheduling module that controls the OR writing of the active QP according to the status information, it first obtains the next OR write address, the number of occupied storage blocks, and the maximum occupied storage block threshold of the QP corresponding to the new SQE from the QP information management component. If the lower n bits of the next OR write address are not all 0, it means that there is still space in the storage block corresponding to the OR write address to write the OR. Otherwise, it means that a new storage block is needed to write the OR of this SQE. Then, it is necessary to compare the number of occupied storage blocks of the corresponding QP with the maximum occupied storage block threshold. If the former is less than the latter, the corresponding QP can allocate a new storage block for use. Otherwise, there is no space in the shared storage space to support the writing of the OR, and it is necessary to suspend the processing of the new SQE of this QP.

[0089] If the SQE to be processed is a retransmitted one and it has been allocated corresponding storage block space in the shared OR storage component, the above judgment is not required. The OR write control component also obtains the SSN of the latest SQE sent by the request side, the SSN of the retransmission start SQE, the retransmission wait flag, and the OR cache address of the retransmission start SQE from the QP information management component, and compares the SSN of the OR to be written with the SSN of the latest SQE sent by the request side and the SSN of the retransmission start SQE respectively. If the SSN of the OR to be written is equal to the SSN of the latest SQE sent by the request side plus one, that is, the OR to be written is the OR of the new SQE, it is written into the shared storage space according to the next OR write address; if the OR to be written is equal to the SSN of the retransmission start SQE, check whether the retransmission wait flag is valid. If the retransmission wait flag is valid, it means that the originally allocated storage space of the current retransmission start OR in the shared OR storage component still remains, and this retransmission start OR can be rewritten into the originally allocated storage space, that is, the current OR is rewritten into the shared storage space of the shared OR storage component according to the above OR cache address of the retransmission start SQE. If the retransmission wait flag is invalid, it means that the retransmission start OR has been completed or rewritten, so the current OR does not need to be written; if the SSN of the OR to be written does not belong to the above two cases, this OR does not need to be written.

[0090] Preferably, the free block management component, when performing the allocation and recycling of free storage blocks in the shared storage space, is specifically used for:

[0091] Caching the free storage block numbers in the shared storage space through a FIFO queue;

[0092] When the network card is started, all storage blocks in the shared storage space are free storage blocks, and all storage block numbers are written into the FIFO queue;

[0093] When the current storage block space written by the OR write control component is insufficient, allocate a storage block number from the FIFO queue and allocate it to the OR write control component for OR writing;

[0094] When a free storage block is recycled by the block space recycling component, write the recycled storage block number into the FIFO queue.

[0095] It should be noted that the FIFO queue is not limited to the data structure of the traditional queue, but covers all storage structures that conform to the FIFO logic, and all storage structures that can implement the FIFO logic belong to the protection scope of the FIFO queue in the present invention; in the free block initialization stage, such as when the network card is started, all storage blocks in the shared storage space are free storage blocks, and all storage block numbers are written into the FIFO queue.

[0096] Preferably, when the block space recycling component executes the recycling of the storage blocks in the shared storage space occupied by the processed ORs, it is specifically configured to:

[0097] Independently count the number of completed ORs in the storage blocks of each shared storage space through a counter;

[0098] When the number of completed ORs in the current storage block reaches the preset number, clear the counter of the current storage block, and send the current storage block number to the free block management component for free storage block recycling.

[0099] In the actual application process, still taking the shared storage space being divided into 2^m storage blocks of equal size and consecutive addresses, and each storage block can cache 2^n ORs as an example, the block space recycling component maintains a counter for each storage block. Each time an OR is completed in the request processing module, the corresponding OR storage address and QPN are notified to the block space recycling component. The block space recycling component then uses the high m bits of the OR cache address as the storage block number, finds the corresponding counter and increments it by one; when the counter corresponding to the storage block accumulates to 2^n, the counter is cleared, the corresponding storage block number is handed over to the free block management component for recycling, and at the same time, the QPN to which the storage block belongs is sent to the QP information management component.

[0100] Preferably, when the storage relationship maintenance component executes the maintenance of the storage order relationship between the storage blocks in the shared storage space, it is specifically configured to:

[0101] Maintain the storage order link relationship between the storage blocks in the shared storage space through a linked list;

[0102] The linked list includes multiple entries, the address of each entry corresponds one-to-one with the storage block number of the shared storage space, and the content of the linked list is the next storage block number written by the corresponding QP after the current storage block space is used up;

[0103] The content of the linked list is updated after the write operation of the OR write control component is executed.

[0104] In the actual operation process, still taking the shared storage space divided into 2^m storage blocks of equal size and continuous addresses as an example, and each storage block can cache 2^n ORs. The storage relationship maintenance component is used to store the storage order link relationship of each storage block in the form of a linked list, so that after the request processing module reads the ORs of a storage block, it can obtain the number of the next storage block. The linked list contains 2^m entries and is stored using a on-chip RAM block with a depth of 2^m. The address of each entry corresponds one-to-one with the storage block number, and the cached content is the number of the next storage block used by the corresponding QP, that is, the bit width of the linked list is m bits. In the actual working process of the storage relationship maintenance component, when a certain QP of the QP information management component finishes using a storage block numbered a and is allocated a storage block numbered b, the OR write control component notifies the storage relationship maintenance component to update the storage order link relationship of the storage block, that is, write the content b into the space with address a in the linked list; then, when the request processing module reads the last OR in the storage block numbered a and still needs to continue to obtain the next OR of the corresponding QP, it needs to access the linked list with address a to obtain the content b therein as the high m bits of the next OR reading address, so that the request processing module can read the corresponding OR.

[0105] Preferably, when the request processing module executes to obtain a response message, and based on the response message, determines whether the ORs of the corresponding QP are processed completely, and if so, notifies the shared storage space scheduling module to recycle the shared storage space occupied by the processed ORs, it is specifically used for:

[0106] Obtain a response message, read the ORs of the corresponding QP according to the QPN and status information in the response message, and based on the information in the ORs, determine whether the ORs of the corresponding QP are processed completely. If so, generate a CQE according to the corresponding OR and notify the shared storage space scheduling module to recycle the shared storage space occupied by the processed ORs.

[0107] In the actual operation process, the request processing module obtains the next OR read address from the QP information management component according to the QPN (Queue Pair Number) of the response message, reads the OR from the shared OR storage component based on this, and then determines how to process the current OR according to the SSN, operation type in the OR and the MSN, Syndrome and other information in the response message based on the network card protocol requirements: If the current OR can be correctly completed, but the MSN in the response message does not correspond to the SSN of the OR, then it is necessary to continue reading the next OR and repeat the above operation of determining how to process the current OR; If an error that can be recovered by retransmission is found, the current OR will not be completed immediately, and the next OR read address will not be updated; If an error that cannot be recovered by retransmission is found, the OR is completed with an error, and the remaining ORs of this QP are actively read and completed with a refresh error. In addition, when the OR is correctly or incorrectly completed, the next OR read address is updated, and the corresponding CQE is generated according to the OR and sent to the host, and at the same time, the block space recycling component is notified to recycle the shared storage space occupied by the OR that has been processed.

[0108] As Figure 3 shown, an ORT management method is applied to an ORT management system. The ORT management system includes a write management module, a shared storage space scheduling module, and a request processing module. The method includes:

[0109] S1. Use the write management module to maintain the status information of active QPs and control the ORs of active QPs to be written into the shared storage space of the shared storage space scheduling module according to the status information;

[0110] S2. Use the request processing module to obtain the response message and determine whether the OR of the corresponding QP has been processed according to the response message. If so, notify the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed;

[0111] S3. Use the shared storage space scheduling module to provide the shared storage space and realize the access of the ORs of active QPs by dynamically allocating and recycling the shared storage space.

[0112] In steps S1 to S3, the write management module is used to maintain the status information of active QPs, and control the OR of the active QPs to write to the shared storage space of the shared storage space scheduling module according to the status information; the request processing module is used to obtain the response message, and judge whether the OR of the corresponding QP is processed completed according to the response message. If so, it notifies the shared storage space scheduling module to recycle the shared storage space occupied by the processed OR; the shared storage space scheduling module is used to provide the shared storage space, recycle the shared storage space when the OR is completed by the request processing module, and allocate the shared storage space when the current storage space written by the QP is insufficient, so as to realize the access of the OR of the active QP on limited on-chip resources. Compared with the prior art, the dynamic ORT management method supporting RDMA reliable transmission of the present invention performs shared storage management on the ORs of all active reliable service transmission type QPs based on limited on-chip storage resources. The storage space occupied by each QP can be dynamically allocated and recycled as needed, thus greatly reducing space waste. When the RDMA network card confirms the completion of the OR of the reliable transmission service type QP, there is no need to access off-chip storage or host memory again, which can significantly reduce the processing delay and bandwidth attenuation of the network card.

[0113] It should be noted that the ORT management method of the present invention does not allocate exclusive OR cache space for each QP, but provides a shared storage space for caching ORs for all QPs, and accesses the ORs of active QPs by dynamically recycling and reallocating the shared storage space, so as to realize the ORT management on limited on-chip resources and greatly reduce the space waste.

[0114] In the embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are only illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical or other forms.

[0115] In addition, each functional module in the embodiments of the present invention can be all integrated in one processor, or each module can be separately used as a device, or two or more modules can be integrated in one device; each functional module in the embodiments of the present invention can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0116] Those of ordinary skill in the art will understand that all or part of the steps of implementing the above method embodiments can be completed through program instructions and related hardware. The foregoing program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above method embodiments are performed. The foregoing storage medium includes various media that can store program codes, such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0117] It should be understood that in this application, if the terms "system", "device", "unit", and / or "module" are used, they are only a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, that term can be replaced by other expressions.

[0118] As shown in this application and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "including" and "comprising" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. A method or device may also include other steps or elements. An element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or device that includes the element.

[0119] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, the meanings of "a plurality of" and "several" are two or more, unless otherwise specifically and clearly defined.

[0120] If a flowchart is used in this application, the flowchart is used to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or after may not be executed precisely in order. On the contrary, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0121] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An ORT management system, characterized in that, Including: A write management module, a shared storage space scheduling module, and a request processing module; The shared storage space scheduling module is respectively connected to the write management module and the request processing module; the write management module and the request processing module are connected; The write management module is used to maintain the status information of the active QP, and control the OR of the active QP to be written into the shared storage space of the shared storage space scheduling module according to the status information; The shared storage space scheduling module is used to provide the shared storage space, and realize the access of the OR of the active QP by dynamically allocating and recycling the shared storage space; wherein, the shared storage space is a limited on-chip storage resource; The request processing module is used to obtain a response message, and judge whether the OR of the corresponding QP has been processed according to the response message. If so, it notifies the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed.

2. The ORT management system according to claim 1, characterized in that, The write management module includes: a QP information management component and an OR write control component; The QP information management component is respectively connected to the OR write control component and the request processing module; the OR write control component is also connected to the shared storage space scheduling module; The QP information management component is used to maintain the status information of the active QP; The OR write control component is used to control the OR of the active QP to be written into the shared storage space of the shared storage space scheduling module according to the status information.

3. The ORT management system according to claim 2, wherein, The shared storage space scheduling module includes: a shared OR storage component, a storage relationship maintenance component, a free block management component, and a block space recycling component; The shared OR storage component is connected to the OR write control component and the request processing module; the storage relationship maintenance component is respectively connected to the OR write control component and the request processing module; the free block management component is respectively connected to the QP information management component and the block space recycling component; the block space recycling component is respectively connected to the QP information management component and the request processing module; The shared OR storage component is used to provide the shared storage space to store and read the OR written by the OR write control component. The shared storage space is composed of multiple storage blocks, and each storage block is used to store a preset number of ORs; The storage relationship maintenance component is used to maintain the storage order relationship between the storage blocks in the shared storage space; The free block management component is used to manage the allocation and recycling of the free storage blocks in the shared storage space; The block space recycling component is used to recycle the storage blocks of the shared storage space occupied by the OR that has been processed.

4. The ORT management system according to claim 3, wherein The status information includes: The next OR write address, the next OR read address, the number of occupied storage blocks, the maximum occupied storage block threshold, the retransmission waiting status flag, the SSN of the latest SQE sent by the request end, the SSN of the retransmission start SQE, and the OR cache address of the retransmission start SQE.

5. The ORT management system according to claim 4, characterized in that, When the OR write control component controls the OR of the active QP to write to the shared storage space of the shared storage space scheduling module according to the status information, it is specifically used for: Judging whether the OR to be written is the OR of the new SQE according to the SSN of the latest SQE sent by the requesting end in the status information; When the OR to be written is the OR of the new SQE, and it is judged according to the next OR write address in the status information, the number of occupied storage blocks in the status information, and the maximum occupied storage block threshold in the status information that the remaining space of the current storage block meets the minimum space condition for writing the OR in the new SQE, allocate storage block space for the OR in the new SQE, and write the OR into the storage block space; When the OR to be written is the OR of the new SQE, and it is judged according to the next OR write address, the number of occupied storage blocks, and the maximum occupied storage block threshold that the remaining space of the current storage block does not meet the minimum space condition for writing the OR in the new SQE, pause processing the current new SQE, and wait for the free block management component to allocate a free storage block and then continue to process the OR of the current new SQE; When the OR to be written is the OR of the retransmitted SQE, and the SQE corresponding to the OR to be written is the first SQE of the retransmission, write the OR of the retransmitted SQE into the corresponding allocated storage block space; When an OR is written to the shared storage space, notify the QP information management component to update the status information.

6. The ORT management system according to claim 3, wherein When the free block management component manages the allocation and recycling of free storage blocks in the shared storage space, it is specifically used for: Cache the numbers of free storage blocks in the shared storage space through a FIFO queue; When the network card is started, all storage blocks in the shared storage space are free storage blocks, and write the numbers of all storage blocks into the FIFO queue; When the current storage block space written by the OR write control component is insufficient, allocate a storage block number from the FIFO queue and allocate it to the OR write control component for OR writing; When a free storage block is recycled by the block space recycling component, write the number of the recycled storage block into the FIFO queue.

7. The ORT management system according to claim 3, characterized in that, When the block space recycling component recycles the storage blocks in the shared storage space occupied by the OR that has completed processing, it is specifically used for: Independently count the number of completed ORs in each storage block of the shared storage space through a counter; When the number of completed ORs in the current storage block reaches the preset number, clear the counter of the current storage block, and send the current storage block number to the free block management component for free storage block recycling.

8. The ORT management system according to claim 6, characterized in that, When the storage relationship maintenance component maintains the storage order relationship between each storage block in the shared storage space, it is specifically used for: Maintain the storage order link relationship between each storage block in the shared storage space through a linked list; The linked list includes multiple entries, and the address of each entry corresponds one-to-one with the storage block number of the shared storage space. The content of the linked list is the next storage block number written by the corresponding QP after the current storage block space is used up; The content of the linked list is updated after the write operation of the OR write control component is executed.

9. The ORT management system according to claim 4, wherein When the request processing module executes to obtain a response message, and according to the response message, determines whether the OR of the corresponding QP is processed, and if so, notifies the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed, it is specifically used for: Obtain a response message, read the OR of the corresponding QP according to the QPN of the response message and the status information, and according to the information in the OR, determine whether the OR of the corresponding QP is processed. If so, generate a CQE according to the corresponding OR, and notify the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed.

10. An ORT management method, characterized in that, Applied to the ORT management system as described in claim 1, wherein the ORT management system includes a write management module, a shared storage space scheduling module, and a request processing module, and the method includes: Using the write management module, maintain the status information of the active QP, and control the OR of the active QP to write to the shared storage space of the shared storage space scheduling module according to the status information; Using the request processing module, obtain a response message, and according to the response message, determine whether the OR of the corresponding QP is processed. If so, notify the shared storage space scheduling module to recycle the shared storage space occupied by the OR that has been processed; Using the shared storage space scheduling module, provide the shared storage space, and through dynamic allocation and recycling of the shared storage space, to realize the access of the OR of the active QP.

Images