Rdma engine, method and apparatus for avoiding system deadlock, medium, chip
Patent Information
- Application Number
- CN202311082755.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-25
- Publication Date
- 2026-08-21
- Estimated Expiration
- 2043-08-25
AI Technical Summary
[0003]本申请的目的在于提出一种RDMA引擎、避免系统死锁方法及其装置、介质、芯片,以解决因RDMA引擎sq Doorbell引起的死锁问题
[0025]本申请实施例的RDMA引擎在先入先出模块已满状态下,丢弃sq Doorbell,并采用位图(bitmap)的方式来记录被丢弃的sq Doorbell,来避免sq Doorbell反压PCIE总线从而造成死锁,且采用位图方式在芯片设计方面相对简单,节省了芯片的制作成本;在后续需要恢复sq Doorbell时,根据位图记录的信息来找到对应的队列对编号(qpn),根据该队列对编号找到对应的队列对上下文(qpc),根据队列对上下文中记录的主机内存地址,从主机内存中读取被丢弃的sq Doorbell,其中,主机在将sq Doorbell下发给RDMA引擎之前,将sqDoorbell写入主机内存地址备份,并将主机内存地址记录在sq Doorbell对应的队列对上下文中。
Smart Images

Figure CN117112464B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Remote Direct Memory Access (RDMA) technology, specifically to RDMA engines, methods and devices for avoiding system deadlock, media, and chips. Background Technology
[0002] When the RDMA engine receives a sq Doorbell, it needs to update the sq producer pointer value carried by the sq Doorbell into the Queue Pair Context (QPC). Therefore, it needs to read the QPC. If the QPC is not in the RDMA engine's internal cache at this time, the RDMA engine needs to read the QPC from host memory via the PCIe bus. Specifically, as follows... Figure 1 As shown, the RDMA engine receives a large number of sq Doorbells from the host. These sq Doorbells are first stored in the Doorbell FIFO (a first-in, first-out module for storing Doorbells). When the Doorbell FIFO is full, it pushes back onto the PCIe bus. Specifically, when processing sq Doorbells, the RDMA engine first reads the QPC from its internal cache. If the QPC is found in the internal cache, the sq Doorbell content is updated in the QPC, preventing deadlock. However, if the QPC is not found in the internal cache, it needs to be read from the host memory. When the QPC is not in the internal cache, the RDMA engine initiates a read request for the QPC to the host via the PCIe bus. The host memory returns the read QPC. The return of the QPC (completion type) and the Doorbell (posted type) issued by the host are both via the PCIe bus. Since the RDMA engine is still waiting for the QPC to return before it can process the sq Doorbell, and the QPC returned by the host memory is blocked by the sq Doorbell backpressure of the Doorbell FIFO on the PCIe bus, the backpressure of the Doorbell FIFO to the PCIe bus cannot be canceled, and the RDMA engine cannot obtain the QPC returned by the host memory, thus causing a deadlock. Summary of the Invention
[0003] The purpose of this application is to propose an RDMA engine, a method for avoiding system deadlock, and the same apparatus, medium, and chip, to solve the deadlock problem caused by the RDMA engine sq Doorbell.
[0004] To achieve the above objectives, embodiments of this application provide an RDMA engine, including a processing module, a bitmap, and a first-in-first-out module;
[0005] The processing module is configured to respond to a received sq Doorbell from the host, determine whether to discard the sq Doorbell based on the state of the first-in-first-out (FIFO) module, and if the FIFO module is not full, store the sq Doorbell in the FIFO module; if the FIFO module is full, discard the sq Doorbell and record the queue pair context number corresponding to the sq Doorbell using the bitmap; wherein, before the host sends the sq Doorbell to the processing module, it writes the sq Doorbell to the host memory address for backup and records the host memory address in the queue pair context corresponding to the sq Doorbell;
[0006] The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
[0007] In some embodiments, each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1;
[0008] The processing module is specifically configured to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
[0009] In some embodiments, n = m1 / m2, where m1 is the size of a physical page of a queue pair context used to manage multiple consecutive sqs, and m2 is the size of a queue pair context.
[0010] In some embodiments, the processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
[0011] This application also provides a method for avoiding system deadlock, implemented based on the aforementioned RDMA engine, the method comprising:
[0012] The processing module receives the sq Doorbell from the host and determines whether to discard it based on the state of the FIFO module. If the FIFO module is not full, the sq Doorbell is stored in the FIFO module; if the FIFO module is full, the sq Doorbell is discarded, and the bitmap is used to record the queue pair context number corresponding to the sq Doorbell. Before sending the sq Doorbell to the processing module, the host writes the sq Doorbell to its memory address for backup and records the host memory address in the queue pair context corresponding to the sq Doorbell.
[0013] When it is necessary to recover the discarded sq Doorbell, the processing module obtains the queue pair context number according to the bitmap, reads the queue pair context according to the queue pair context number, obtains the host memory address according to the queue pair context, and reads the corresponding sq Doorbell from the host memory according to the host memory address.
[0014] In some embodiments, each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1;
[0015] The processing module is specifically configured to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
[0016] In some embodiments, n = m1 / m2, where m1 is the size of a physical page of a queue pair context used to manage multiple consecutive sqs, and m2 is the size of a queue pair context.
[0017] In some embodiments, the processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
[0018] This application embodiment also provides a device for avoiding system deadlock, including the RDMA engine, host, and host memory described above;
[0019] The host is used to write the sq Doorbell to be sent into the host memory address backup, and record the host memory address in the queue pair context corresponding to the sq Doorbell. After the recording is completed, the sq Doorbell is sent to the RDMA engine.
[0020] The processing module is used to respond to the received sq Doorbell from the host, and determine whether to discard the sq Doorbell according to the state of the first-in-first-out module. If the state of the first-in-first-out module is not full, the sq Doorbell is stored in the first-in-first-out module; if the state of the first-in-first-out module is full, the sq Doorbell is discarded, and the context number of the queue corresponding to the sq Doorbell is recorded using the bitmap.
[0021] The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
[0022] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0023] This application also provides a chip including the RDMA engine described above.
[0024] The RDMA engine, system deadlock avoidance method and apparatus, medium and chip provided in this application have the following beneficial effects:
[0025] In this embodiment of the RDMA engine, when the FIFO module is full, the sq Doorbell is discarded, and a bitmap is used to record the discarded sq Doorbell to avoid deadlock caused by the sq Doorbell back-pressure on the PCIe bus. The bitmap method is relatively simple in chip design and saves chip manufacturing costs. When the sq Doorbell needs to be recovered later, the corresponding queue pair number (qpn) is found according to the information recorded in the bitmap. The corresponding queue pair context (qpc) is found according to the queue pair number. The discarded sq Doorbell is read from the host memory according to the host memory address recorded in the queue pair context. Before the host sends the sq Doorbell to the RDMA engine, it writes the sq Doorbell to the host memory address for backup and records the host memory address in the queue pair context corresponding to the sq Doorbell. Attached Figure Description
[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic diagram of a conventional RDMA engine for processing sq Doorbells.
[0028] Figure 2 This is a simplified schematic diagram of the structure of an RDMA engine in one embodiment of this application.
[0029] Figure 3 This is a flowchart of a method for avoiding system deadlock in one embodiment of this application.
[0030] Figure 4 This is a simplified schematic diagram of the structure of a system deadlock avoidance device in one embodiment of this application. Detailed Implementation
[0031] The detailed description of the accompanying drawings is intended to illustrate some embodiments of this application and is not intended to represent only the forms in which this application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included within the spirit and scope of this application.
[0032] See Figure 2One embodiment of this application provides an RDMA engine, including a processing module, a bitmap (DoorbellBitmap), and a first-in-first-out module (Doorbell FIFO);
[0033] The processing module is used to respond to the received sq Doorbell (sq refers to the sending queue in the queue pair, which includes a sending queue and a receiving queue) from the host, and to determine whether to discard the sq Doorbell based on the state of the first-in-first-out (FIFO) module. If the FIFO module is not full, the sq Doorbell is stored in the FIFO module; if the FIFO module is full, the sq Doorbell is discarded, and the bitmap is used to record the context number (qpn) of the queue pair corresponding to the sq Doorbell.
[0034] The bitmap is used to record the discard event of the sq Doorbell; the storage space of the bitmap is provided by the on-chip cache of the RDMA engine;
[0035] The first-in-first-out module is used to store the sq Doorbell sent by the host;
[0036] Before sending the sq Doorbell to the processing module, the host writes the sq Doorbell to the host memory address for backup and records the host memory address in the queue pair context (qpc) corresponding to the sq Doorbell.
[0037] The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
[0038] Specifically, to avoid backpressure on the PCIe bus, in this embodiment, when the FIFO module is full, the RDMA engine discards the sq Doorbell. However, the RDMA engine still needs to obtain the latest sq Doorbell content. Because the bitmap method is relatively simple in chip design and saves chip manufacturing costs, this embodiment uses the bitmap to record the discarded sq Doorbell. When the sq Doorbell needs to be recovered later, the corresponding queue pair number is found according to the information recorded in the bitmap, the corresponding queue pair context is found according to the queue pair number, and the discarded sq Doorbell is read from the host memory according to the host memory address recorded in the queue pair context, thus realizing the recovery of the discarded sq Doorbell.
[0039] The RDMA engine typically processes sq doorbells in a First-Come, First-Served (FCFS) order, meaning they are processed according to the order in which they arrive at the RDMA engine. Therefore, when the RDMA engine receives sq doorbells, it can mark them to determine their order. Processing sq doorbells in FCFS order helps ensure the sequentiality of data transmission and avoids out-of-order transmission problems. This is crucial for maintaining data integrity and consistency, especially in applications requiring consistent data order. It's understandable that if the markings of the current sq doorbell output by the FIFO module are not consecutive with the markings of the previous sq doorbell, it means the intermediate marked sq doorbell was discarded and needs to be recovered. However, at this point, it's unknown which sq (transmission queue) the discarded sq doorbell needs to be recovered. The corresponding QPN needs to be obtained based on the corresponding bit information in the bitmap, where each bit in the bitmap corresponds to a specific QPN.
[0040] It should be noted that when the RDMA engine in this embodiment processes a sq Doorbell, it first reads the QPC from the internal cache. If the QPC is hit in the internal cache, the sq Doorbell content is updated in the QPC, and no deadlock will occur. However, if the QPC is not hit in the internal cache, it needs to be read from the host memory. When the QPC is not in the internal cache, the RDMA engine initiates a read request for the QPC to the host through the PCIe bus. The host memory returns the read QPC. Since there is no backpressure sq Doorbell on the PCIe bus, the QPC can be successfully passed to the RDMA engine for processing, and no deadlock will occur.
[0041] In some embodiments, each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1;
[0042] The processing module is specifically used to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
[0043] Specifically, if each sq uses 1 bit of the bitmap, i.e., when n equals 1, assuming the RDMA engine supports 1M sq, then a total of 1M bits of bitmap resources are needed, which occupies too much bitmap resources. The increase in on-chip resources will increase the chip cost. Therefore, this embodiment adopts the method of compressing the bitmap, merging n consecutive sq doorbells into 1 bit, that is, any one of these n consecutive sq doorbells that is discarded is recorded in this 1 bit. At this time, n is greater than or equal to 2.
[0044] In some embodiments, n = m1 / m2, where m1 is the size of a physical page of a queue pair context used to manage multiple consecutive sqs, and m2 is the size of a queue pair context.
[0045] Specifically, the value of n usually depends on the size of a single qpc and a physical page. The qpcs of consecutive sqdoorbells are usually managed in a 4KB physical page. Assuming the qpc is 256B, it is recommended to take the value of n as 4KB / 256B = 16. In this case, if a 1M sqdoorbell is supported, the required bitmap resources are 1M bit / 16, which effectively saves bitmap resources. Of course, the value of n can be taken to a larger value to further save bitmap resources.
[0046] In some embodiments, the processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
[0047] Specifically, the bitmap only records the discard event of the sq doorbell. The RDMA engine still needs to obtain the latest sq doorbell content. Before the host driver distributes the sq doorbell, it needs to copy the latest sq doorbell content and write it to the host's memory address, which is then recorded by the QPC. When the RDMA engine retrieves the discarded sq doorbell through the bitmap, it first obtains the corresponding queue pair number through the bit information of the bitmap. One bit of the bitmap may correspond to 1 to n QPNs. The number of QPNs depends on the sq to which the n consecutive sq doorbells belong. If the n consecutive sq doorbells come from the same sq, then one bit of the bitmap corresponds to 1 QPN. If the m consecutive sq doorbells come from the same sq, then one bit of the bitmap corresponds to 1 QPN. If the doorbell comes from m sq, then one bit of the bitmap corresponds to m qpn. For example, bit 0 of the bitmap corresponds to qpn0~15. Therefore, several queue pair numbers (qpn) can be obtained based on the bit information of the bitmap. Further, based on these queue pair numbers (qpn), n corresponding queue contexts (qpc) can be read. Based on these n queue contexts (qpc), the address of the sq doorbell in the host memory can be obtained. Then, the discarded sq doorbell is read through this address to obtain the content of the latest sq doorbell. Other sq doorbells that are not discarded are normally output by the first-in-first-out module.
[0048] As described in the above embodiments, the RDMA engine of this embodiment records doorbell drop events by merging multiple consecutive sq doorbells into a single bit, greatly saving bitmap resources. Using a compressed bitmap, the value of n can be determined based on the size of a single qpc and the physical page, further saving bitmap resources. Simultaneously, to obtain the latest sq doorbell content, a memory address is recorded through the qpc, and the latest sq doorbell content is written to this address before the host issues the sq doorbell. When the RDMA engine retrieves the doorbell through the bitmap, it can read the latest sq doorbell content through this address. This ensures that the RDMA engine always obtains the latest sq doorbell content. Therefore, the RDMA engine of this embodiment has good application prospects, can reduce chip costs, and improve RDMA engine performance.
[0049] See Figure 3 Another embodiment of this application also provides a method for avoiding system deadlock, implemented based on the RDMA engine described in the above embodiments, the method comprising the following steps:
[0050] Step S1: The processing module receives the sq Doorbell from the host and determines whether to discard the sq Doorbell based on the state of the FIFO module. If the FIFO module is not full, the sq Doorbell is stored in the FIFO module; if the FIFO module is full, the sq Doorbell is discarded, and the context number of the queue pair corresponding to the sq Doorbell is recorded using the bitmap. Before sending the sq Doorbell to the processing module, the host writes the sq Doorbell to its memory address for backup and records the host memory address in the context of the queue pair corresponding to the sq Doorbell.
[0051] Step S2: When it is necessary to recover the discarded sq Doorbell, the processing module obtains the queue pair context number according to the bitmap, reads the queue pair context according to the queue pair context number, obtains the host memory address according to the queue pair context, and reads the corresponding sq Doorbell from the host memory according to the host memory address.
[0052] In some embodiments, each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1;
[0053] The processing module is specifically configured to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
[0054] In some embodiments, n = m1 / m2, where m1 is the size of a physical page of a queue pair context used to manage multiple consecutive sqs, and m2 is the size of a queue pair context.
[0055] In some embodiments, the processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
[0056] It should be noted that the method in this embodiment corresponds to the RDMA engine in the above embodiments. Therefore, any details not described in this embodiment can be obtained by referring to the RDMA engine content in the above embodiments, and will not be elaborated here.
[0057] See Figure 4 Another embodiment of this application also provides a device for avoiding system deadlock, including the RDMA engine, host (CPU), and host memory of the above embodiments;
[0058] The host is used to write the sq Doorbell to be sent into the host memory address backup, and record the host memory address in the queue pair context corresponding to the sq Doorbell. After the recording is completed, the sq Doorbell is sent to the RDMA engine.
[0059] The processing module is used to respond to the received sq Doorbell from the host, and determine whether to discard the sq Doorbell according to the state of the first-in-first-out module. If the state of the first-in-first-out module is not full, the sq Doorbell is stored in the first-in-first-out module; if the state of the first-in-first-out module is full, the sq Doorbell is discarded, and the context number of the queue corresponding to the sq Doorbell is recorded using the bitmap.
[0060] The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
[0061] Furthermore, each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1;
[0062] The processing module is specifically used to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
[0063] Specifically, if each sq uses 1 bit of the bitmap, i.e., when n equals 1, assuming the RDMA engine supports 1M sq, then a total of 1M bits of bitmap resources are needed, which occupies too much bitmap resources. The increase in on-chip resources will increase the chip cost. Therefore, this embodiment adopts the method of compressing the bitmap, merging n consecutive sq doorbells into 1 bit, that is, any one of these n consecutive sq doorbells that is discarded is recorded in this 1 bit. At this time, n is greater than or equal to 2.
[0064] Furthermore, n = m1 / m2, where m1 is the size of a physical page in the context used to manage multiple consecutive sq queues, and m2 is the size of a queue in the context.
[0065] Specifically, the value of n usually depends on the size of a single qpc and a physical page. The qpcs of consecutive sqdoorbells are usually managed in a 4KB physical page. Assuming the qpc is 256B, it is recommended to take the value of n as 4KB / 256B = 16. In this case, if a 1M sqdoorbell is supported, the required bitmap resources are 1M bit / 16, which effectively saves bitmap resources. Of course, the value of n can be taken to a larger value to further save bitmap resources.
[0066] Further, the processing module is specifically used to obtain bit information of a bit of the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
[0067] Specifically, the bitmap only records the discard event of the sq doorbell. The RDMA engine still needs to obtain the latest sq doorbell content. Before the host driver sends the sq doorbell, it needs to copy the latest sq doorbell content and write it to the host's memory address. The QPC records this memory address. When the RDMA engine retrieves the discarded sq doorbell through the bitmap, it first obtains the corresponding queue pair number (qpn) through the bit information of the bitmap. Through the queue pair number (qpn), it can read the corresponding queue context (qpc). Based on the queue context (qpc), it can obtain the address of the sq doorbell in the host memory. Then, it reads the discarded sq doorbell through this address to obtain the latest sq doorbell content.
[0068] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the system deadlock avoidance method described in the above embodiments.
[0069] This application also provides a chip that includes the RDMA engine described in the above embodiments.
[0070] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. An RDMA engine, characterized in that, This includes a processing module, a bitmap, and a first-in-first-out (FIFO) module. The processing module is used to respond to the sq Doorbell sent by the host, and determine whether to discard the sq Doorbell according to the status of the first-in-first-out module. If the status of the first-in-first-out module is not full, the sq Doorbell is stored in the first-in-first-out module. If the first-in-first-out module is full, the sq Doorbell is discarded, and the bitmap is used to record the queue pair context number corresponding to the sq Doorbell; wherein, before the host sends the sq Doorbell to the processing module, the host writes the sq Doorbell to the host memory address for backup, and records the host memory address in the queue pair context corresponding to the sq Doorbell; The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
2. The RDMA engine according to claim 1, characterized in that, in, Each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1; The processing module is specifically configured to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
3. The RDMA engine according to claim 2, characterized in that, n = m1 / m2, where m1 is the size of a physical page in the context used to manage multiple consecutive sq queues, and m2 is the size of a queue in the context.
4. The RDMA engine according to claim 2, characterized in that, The processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sq Doorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
5. A method for avoiding system deadlock, characterized in that, Based on the RDMA engine implementation of claim 1, the method includes: The processing module receives the sq Doorbell from the host and determines whether to discard it based on the state of the FIFO module. If the FIFO module is not full, the sq Doorbell is stored in the FIFO module; if the FIFO module is full, the sq Doorbell is discarded, and the bitmap is used to record the queue pair context number corresponding to the sq Doorbell. Before sending the sq Doorbell to the processing module, the host writes the sq Doorbell to its memory address for backup and records the host memory address in the queue pair context corresponding to the sq Doorbell. When it is necessary to recover the discarded sq Doorbell, the processing module obtains the queue pair context number according to the bitmap, reads the queue pair context according to the queue pair context number, obtains the host memory address according to the queue pair context, and reads the corresponding sq Doorbell from the host memory according to the host memory address.
6. The method according to claim 5, characterized in that, in, Each bit of the bitmap corresponds to n consecutive sq Doorbells, where n is greater than or equal to 1; The processing module is specifically configured to set the value of a corresponding bit in the bitmap to 1 when any one of the n sq Doorbells is discarded; and to set the value of a corresponding bit in the bitmap to 0 when none of the n sq Doorbells are discarded.
7. The method according to claim 6, characterized in that, n = m1 / m2, where m1 is the size of a physical page in the context used to manage multiple consecutive sq queues, and m2 is the size of a queue in the context.
8. The method according to claim 6, characterized in that, The processing module is specifically used to obtain bit information of a bit in the bitmap corresponding to the sqDoorbell, obtain the corresponding queue pair number based on the bit information, read the corresponding queue pair context from the host memory based on the queue pair number, obtain the host memory address based on the queue pair context, and read the sq Doorbell from the host memory based on the host memory address.
9. A device for preventing system deadlock, characterized in that, Includes the RDMA engine, host, and host memory as described in any one of claims 1 to 4; The host is used to write the sq Doorbell to be sent into the host memory address backup, and record the host memory address in the queue pair context corresponding to the sq Doorbell. After the recording is completed, the sq Doorbell is sent to the RDMA engine. The processing module is used to respond to the sq Doorbell sent by the host, and determine whether to discard the sq Doorbell according to the status of the first-in-first-out module. If the status of the first-in-first-out module is not full, the sq Doorbell is stored in the first-in-first-out module. If the first-in-first-out module is full, the sq Doorbell is discarded, and the context number of the queue corresponding to the sq Doorbell is recorded using the bitmap; The processing module is further configured to, when it is necessary to recover the discarded sq Doorbell, obtain the queue pair context number according to the bitmap, read the queue pair context according to the queue pair context number, obtain the host memory address according to the queue pair context, and read the corresponding sq Doorbell from the host memory according to the host memory address.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 5 to 8.
11. A chip comprising the RDMA engine according to any one of claims 1 to 4.
Citation Information
Patent Citations
Doorbell backpressure avoidance mechanism on a host channel adapter
US20140181323A1