Data processing method, hardware accelerator and computing device
By setting buffers and response conditions in the RDMA device to control the sending order of request messages, the problem of reading data when the RDMA device has not finished writing is solved, ensuring data integrity and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PHYTIUM TECH CO LTD
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-19
AI Technical Summary
The problem arises when the host node attempts to read data before the RDMA device has finished writing the data, resulting in the host node being unable to read the complete data.
In the RDMA device, a first buffer is set up to store request entries corresponding to multiple request messages and send them to the memory in a certain order. The first entry type request message is sent only when the target response information is received, ensuring that the data is written before reading.
This avoids host node read errors caused by incomplete data writing, ensuring data integrity and reliability, and improving the accuracy of data transmission.
Smart Images
Figure CN122069246A_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of computer technology, specifically to remote direct memory access technology in the field of computer technology, and more specifically to a data processing method, hardware accelerator, and computing device. Background Technology
[0002] RDMA (Remote Direct Memory Access) technology is a high-performance, low-latency communication technology that allows a hardware device to directly access memory without going through the processor of another hardware device.
[0003] However, in some cases, the RDMA device may not have finished writing the data before the host node starts reading the data, resulting in the host node being unable to read the complete data. Summary of the Invention
[0004] This specification provides a data processing method, hardware accelerator, and computing device to solve the problem that when the host node reads data before the RDMA device has finished writing the data, the host node cannot read the complete data.
[0005] To achieve the above technical objectives, the embodiments of this specification provide the following technical solutions: In a first aspect, one embodiment of this specification provides a data processing method applied to a Remote Direct Memory Access (RDMA) device, wherein the RDMA device establishes communication connections with both a host node and a memory, and the RDMA device includes a first buffer and multiple data transmission queues; the data processing method includes: In response to the data processing task sent by the host node, multiple request messages are generated, and the multiple request messages correspond to one or more of the data sending queues; The request entries corresponding to each of the plurality of request messages are stored in the first buffer, and the request entries are used to describe the request messages corresponding to the request entries; According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, the request message corresponding to the request entry is sent to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
[0006] Secondly, one embodiment of this specification provides a hardware accelerator, comprising: a host node, a memory, and an RDMA device; wherein, The RDMA device establishes communication connections with both the host node and the memory, and the RDMA device includes a first buffer and multiple data transmission queues; The RDMA device is configured as follows: In response to the data processing task sent by the host node, multiple request messages are generated, and the multiple request messages correspond to one or more of the data sending queues; The request entries corresponding to each of the plurality of request messages are stored in the first buffer, and the request entries are used to describe the request messages corresponding to the request entries; According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, the request message corresponding to the request entry is sent to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
[0007] In conjunction with the second aspect, in some embodiments of the second aspect, the request message corresponding to the first entry type is used to set a descriptor status bit in the memory, and the set descriptor status bit is used to indicate that the data writing is complete.
[0008] In conjunction with the second aspect, in some embodiments of the second aspect, the order of the request entries is used to characterize the order of the request entries within the same type of request entries, and the request messages corresponding to the same type of request entries are sent from the same data transmission queue.
[0009] In conjunction with the second aspect, in some embodiments of the second aspect, the request entry includes a queue number, a request type, and a conditional unlock identifier; wherein, The queue number is used to identify the data transmission queue for the request message corresponding to the request entry; The request type is used to characterize whether the request entry is the first entry type; The condition unlocking flag is used to indicate whether the sending condition corresponding to the request entry has been met.
[0010] In conjunction with the second aspect, in some embodiments of the second aspect, the RDMA device further includes a second buffer; the second buffer is used to store a plurality of the request messages; The request entry further includes a unique identifier, which describes the storage location of the request message corresponding to the request entry in the second buffer.
[0011] In conjunction with the second aspect, in some embodiments of the second aspect, the RDMA device further includes: a write response buffer, the write response buffer being used to store a response message of the request message, the response message being used to indicate that the request message has been executed; The RDMA device is further configured to, upon receiving the response message, write the response message into the write response buffer and delete the request entries to be deleted from the first buffer, wherein the request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
[0012] In conjunction with the second aspect, in some embodiments of the second aspect, the RDMA device further includes a read pointer that points to the first request entry in the first buffer; After deleting the request entry to be deleted from the first buffer, the response processing module is further configured to: Move all other request entries in the first buffer forward.
[0013] In conjunction with the second aspect, in some embodiments of the second aspect, when the request entry is a second entry type, the sending condition corresponding to the request entry of the second entry type includes: the second entry type is the first in the order of arrangement in the first buffer.
[0014] In conjunction with the second aspect, in some embodiments of the second aspect, the hardware accelerator further includes: an on-chip network; The RDMA device establishes communication connections with the host node and the memory respectively through the on-chip network; The host node includes a processor, which includes at least one of a graphics processor and a neural network processor; The RDMA device includes a network interface card (NIC).
[0015] Thirdly, one embodiment of this specification also provides a computing device including a hardware accelerator as described in any of the above embodiments.
[0016] Fourthly, one embodiment of this specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data processing method described above.
[0017] Fifthly, embodiments of this specification provide a computer program product or computer program, the computer program product including a computer program stored in a computer-readable storage medium; a processor of a computer device reads the computer program from the computer-readable storage medium, and when the processor executes the computer program, it implements the steps of the data processing method described above. Optionally, the computer program may be stored in a computer-readable storage medium or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or in the cloud.
[0018] As can be seen from the above technical solutions, the data processing method provided in this specification is applied to an RDMA device including a first buffer and multiple data transmission queues. When the RDMA device receives a data processing task, it generates multiple request messages and stores the request entries corresponding to the request messages in the first buffer. Then, according to the order of the request entries in the first buffer, and when the transmission conditions corresponding to the request entries are met, the request message corresponding to the request entry is sent to the memory. When the request entry is a first entry type, the transmission conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose transmission order is before the transmission order of the request messages corresponding to the first entry type. Thus, by setting up a first buffer, request entries representing multiple request messages are stored therein and sent to the memory in a certain order, ensuring that multiple request messages can be sent to the memory in sequence. At the same time, for the request message corresponding to the first entry type, it can only be sent to the memory when the target response message is received. This prevents the request message corresponding to the first entry type from arriving at the memory and being executed earlier than the request messages sent before it during transmission. This ensures that the request message corresponding to the first entry type will not arrive at the memory earlier than the request messages sent before it, thus avoiding problems that may occur if the request message corresponding to the first entry type arrives at the memory earlier than the request messages sent before it (such as the problem of the host node reading data before the RDMA device has finished writing the data). Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this specification. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0020] Figure 1 This is a schematic diagram illustrating the connection relationship between an RDMA device, a host node, and a memory, etc., as provided in one embodiment of this specification.
[0021] Figure 2 This is a flowchart illustrating a data processing method provided for one embodiment of this specification.
[0022] Figure 3 This is a schematic diagram of an RDMA device provided for one embodiment of this specification.
[0023] Figure 4 This is a schematic diagram of a hardware accelerator provided for one embodiment of this specification.
[0024] Figure 5 This is a schematic diagram of the structure of a computing device provided for one embodiment of this specification. Detailed Implementation
[0025] Unless otherwise defined, the technical or scientific terms used in the embodiments of this specification shall have the ordinary meaning understood by one of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to avoid confusion of constituent elements.
[0026] Unless the context otherwise requires, throughout this specification, "a plurality of" means "at least two," and "including" is interpreted as open-ended or encompassing, that is, "including, but not limited to." In the description of this specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this specification. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example.
[0027] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0028] Overview RDMA devices can have multiple data transmission queues, which can simultaneously process different data streams or tasks, enabling parallel data transmission. For example, in a high-performance computing cluster, different computing nodes may simultaneously send data requests to the RDMA device. The RDMA device's multiple data transmission queues can process these requests concurrently, fully utilizing network bandwidth and device resources to improve overall data transmission efficiency. Furthermore, different application scenarios have varying requirements for data transmission priority, latency, and bandwidth. By setting up multiple data transmission queues, different types of data streams can be categorized and processed. For instance, in a mixed-load system, high-priority real-time data transmission tasks can be assigned to a dedicated high-priority data transmission queue, while low-priority batch data transmission tasks can be assigned to other data transmission queues, thus meeting the specific needs of various applications.
[0029] refer to Figure 1 RDMA devices can connect to host nodes via a Network of Chips (NoC) and communicate with one or more host nodes to meet the RDMA operation requirements of one or more host nodes.
[0030] Some on-chip networks may not guarantee the first-to-write, first-to-observe principle (this principle means that during the execution of a write operation, the first write operation should be observed by other read operations first, i.e., data written first should be read from memory or other storage devices before data written later). In this case, request messages from the RDMA device may not be transmitted to the memory or other hardware in the order they were sent. This can lead to out-of-order writes to the memory or other hardware, where the descriptor status bit representing the data write status is set before the data is fully written to the memory. This may cause the host node to be unable to read the complete data from the memory based on this descriptor status bit (the data write may not be complete). For example, if messages 1-10 need to be sent to the memory in sequence, where messages 1-9 request to write data A to the memory, data A will only be fully written to the memory after messages 1-9 have been executed; message 10 requests to set the descriptor status bit of data A in the memory, and the set descriptor status bit indicates that data A has been written completely. Messages 1 to 10 are sent to the memory in sequence. If the on-chip network cannot guarantee the principle of first-to-first-to-observe, message 10 may arrive at the memory before at least one of messages 1 to 9. In this case, data A may not be completely written, but its descriptor status bit has been set. The host node will then consider data A to be completely written and can perform data reading operations. However, the data A read by the host node at this time is not the complete data A, which may lead to unpredictable anomalies in the subsequent data processing.
[0031] To address this issue, a corresponding buffer space can be allocated to each data transmission queue. Request messages sent by each queue are buffered in their respective buffer spaces. Request messages in these buffers can be sent only after the response messages of previously sent request messages have been received, ensuring that request messages from each queue are sent to memory in the correct order, thus avoiding out-of-order writes. However, the inventors discovered that RDMA devices include multiple data transmission queues. At any given moment or within a given time period, not all queues have a need to send request messages. Allocating a separate buffer space for each queue might result in some buffer spaces being idle during certain periods, wasting buffer resources and placing higher demands on the RDMA device's buffer resources.
[0032] To address the newly discovered problems, the inventors conducted further research. As mentioned earlier, in the same data transmission queue, for messages requesting data writes, since these messages describe the address where the data needs to be written in memory, even if the memory does not execute in the order these messages are sent, it will not cause the data to be written to the wrong address. Therefore, it is not necessary to require that all request messages before it be responded to before it is sent; these request messages can be sent to the memory in sequence. However, for request messages requesting the setting of status bit descriptors, these request messages need to be sent only after all other request messages before them have been responded to, in order to avoid the situation where these request messages arrive at the memory before the request messages before them, thereby avoiding the problem of out-of-order writing and the potential problems that may arise from the host node setting the status bit of the descriptor in advance.
[0033] Based on this, the inventors propose a data processing method applied to an RDMA device including a first buffer and multiple data transmission queues. When the RDMA device receives a data processing task, it generates multiple request messages and stores the request entries corresponding to the request messages in the first buffer. Then, according to the order of the request entries in the first buffer, and provided that the transmission conditions corresponding to the request entries are met, the request messages corresponding to the request entries are sent to the memory. When the request entry is of a first entry type, the transmission conditions corresponding to the request entries of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose transmission order precedes the transmission order of the request messages corresponding to the first entry type. Thus, by setting up a first buffer, request entries representing multiple request messages are stored therein and sent to the memory in a certain order, ensuring that multiple request messages can be sent to the memory in sequence. At the same time, for the request message corresponding to the first entry type, it can only be sent to the memory when the target response message is received. This prevents the request message corresponding to the first entry type from arriving at the memory and being executed earlier than the request messages sent before it during transmission. This ensures that the request message corresponding to the first entry type will not arrive at the memory earlier than the request messages sent before it, thus avoiding problems that may occur if the request message corresponding to the first entry type arrives at the memory earlier than the request messages sent before it (such as the problem of the host node reading data before the RDMA device has finished writing the data).
[0034] Based on the above concept, this specification provides a data processing method. The data processing method provided by this specification will be described exemplarily below with reference to the accompanying drawings.
[0035] Exemplary methods Taking an RDMA device as an example, this specification provides a data processing method. The RDMA device establishes communication connections with both the host node and the memory. The RDMA device includes a first buffer and multiple data transmission queues. (See reference...) Figure 2 The method includes: S201: In response to the data processing task sent by the host node, generate multiple request messages, the multiple request messages corresponding to one or more of the data sending queues; S202: Store the request entries corresponding to each of the plurality of request messages into the first buffer, wherein the request entries are used to describe the request messages corresponding to the request entries; S203: According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, send the request message corresponding to the request entry to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
[0036] In some implementations, the host node can be a processor that establishes a communication connection with the RDMA device. These processors include, but are not limited to, at least one of a graphics processing unit (GPU) and a neural network processor. The host node can establish a communication connection with the RDMA device through an on-chip network. The on-chip network may not have the ability to maintain the first-write-first-observed principle, thereby reducing the requirements for the on-chip network while solving the out-of-order write problem. The host node can be encapsulated in the same hardware device (e.g., a hardware accelerator) as the RDMA device. The host node can receive tasks sent to it by external nodes and generate data processing tasks based on these tasks, which are then sent to the RDMA device for processing.
[0037] Multiple request messages generated by an RDMA device in response to data processing tasks can correspond to at least one data transmission queue. These request messages can all be stored in a first buffer. This eliminates the need for a separate buffer for each data transmission queue, allowing a single first buffer to meet the caching needs of multiple data transmission queues. Furthermore, the first buffer can store request entries describing the request messages instead of directly storing the messages themselves. A smaller first buffer is sufficient to meet the caching needs of multiple data transmission queues, ensuring the sequential transmission of request messages corresponding to the first entry type in multiple data transmission queues.
[0038] In some implementations, to improve the transmission efficiency of request messages in each data transmission queue, the order of the request entries is used to characterize the order of the request entries within the same type of request entries. Request messages corresponding to the same type of request entries are sent by the same data transmission queue. That is, the first buffer can store the corresponding request entries sequentially according to the order in which the request messages were generated or their IDs. To distinguish the request messages to be sent by different data transmission queues, the request entries corresponding to different data transmission queues can be sequentially arranged in the first buffer. For example, assuming there are data transmission queues A, B, and C, and data transmission queue A needs to send request messages 1-3, data transmission queue B needs to send request messages 4-5, and data transmission queue C needs to send request messages 6-10, then in the first buffer, the request entries corresponding to request messages 1-3 can be stored sequentially, the request entries corresponding to request messages 4-5 can be stored sequentially, and the request entries corresponding to request messages 6-10 can be stored sequentially. Requests of the same type can be sent sequentially, and requests of different types will not interfere with each other. That is, the request entries corresponding to request messages 1 to 3 can be sent in order, provided that the sending conditions corresponding to the request messages are met, without having to consider the sending of request messages 4 to 5 or request messages 6 to 10, which helps to improve the sending efficiency of request messages.
[0039] The request message corresponding to the first entry type can be a request message for which the sending order needs special attention. For example, in one embodiment, the request message corresponding to the first entry type is used to set a descriptor status bit in the memory. The set descriptor status bit is used to indicate that the data writing is complete. In this way, before the request message corresponding to the first entry type is sent, it can be guaranteed that other request messages whose sending order is earlier than that of the request message corresponding to the first entry type will be responded to by the memory, which can avoid the situation where the descriptor status bit is set before the data is written.
[0040] In summary, the data processing method is applied to an RDMA device including a first buffer and multiple data transmission queues. When the RDMA device receives a data processing task, it generates multiple request messages and stores the request entries corresponding to the request messages in the first buffer. Then, according to the order of the request entries in the first buffer, and provided that the transmission conditions corresponding to the request entries are met, the request message corresponding to the request entry is sent to the memory. When the request entry is of a first entry type, the transmission conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose transmission order precedes the transmission order of the request messages corresponding to the first entry type. Thus, by setting up a first buffer, request entries representing multiple request messages are stored therein and sent to the memory in a certain order, ensuring that multiple request messages can be sent to the memory in sequence. At the same time, for the request message corresponding to the first entry type, it can only be sent to the memory when the target response message is received. This prevents the request message corresponding to the first entry type from arriving at the memory and being executed earlier than the request messages sent before it during transmission. This ensures that the request message corresponding to the first entry type will not arrive at the memory earlier than the request messages sent before it. This avoids the problems that may occur if the request message corresponding to the first entry type arrives at the memory earlier than the request messages sent before it (such as the problem that the host node cannot read the complete data when the RDMA device has not finished writing the data).
[0041] In one embodiment of this specification, a feasible method for defining request entries is proposed, specifically, the request entry includes a queue number, a request type, and a conditional unlock identifier; wherein, The queue number is used to identify the data transmission queue for the request message corresponding to the request entry; The request type is used to characterize whether the request entry is the first entry type; The condition unlocking flag is used to indicate whether the sending condition corresponding to the request entry has been met.
[0042] In this embodiment, the queue number parameter marks the data transmission queue to which the request message corresponding to each request entry belongs, the request type parameter marks whether the request entry is the first entry type, and the condition unlocking flag parameter marks whether the transmission condition corresponding to the request entry is met. In this way, the transmission requirements of the request message corresponding to each request entry can be met by using these parameters.
[0043] In one embodiment, the RDMA device further includes a second buffer; the second buffer is used to store a plurality of the request messages; The request entry further includes a unique identifier, which describes the storage location of the request message corresponding to the request entry in the second buffer.
[0044] In this embodiment, the request message is stored in the second buffer. The request entry describes the storage location of the corresponding request message in the second buffer through a unique identifier, so as to achieve the purpose of finding the corresponding request message through the unique identifier.
[0045] In one embodiment, the RDMA device further includes: a write response buffer, the write response buffer being used to store a response message of the request message, the response message being used to indicate that the request message has been executed; The data processing method further includes: Upon receiving the response message, the response message is written to the write response buffer, and the request entries to be deleted are deleted from the first buffer. The request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
[0046] In this embodiment, a write response buffer is also provided in the RDMA device. After each request message is responded to by the memory, the response message returned by the memory to the RDMA device can be stored in the write response buffer. In this way, based on the response messages in the write response buffer, the request entries in the first buffer can be managed, avoiding the situation where request entries corresponding to completed request messages occupy the space of the first buffer for a long time.
[0047] In one embodiment, the RDMA device further includes a read pointer that points to the first request entry in the first buffer; After deleting the request entry to be deleted from the first buffer, the method further includes: Move all other request entries in the first buffer forward.
[0048] In this embodiment, a read pointer indicates the request entry to be read in the first buffer. Specifically, the value of the read pointer can be fixed at 0, so that the read pointer can point to the first request entry in the first buffer, satisfying the requirement that the request message corresponding to the request entry that enters the first buffer first is sent first. In one embodiment, if the order of the request entries is used to characterize the order of the request entries within the same type of request entries, then the number of read pointers can be multiple, with each read pointer pointing to a different type of request entry. In some embodiments, the RDMA device may also include a write pointer. The initial value of the write pointer can be 0, and the value of the write pointer can be incremented by 1 each time a request entry is written, thereby pointing to the position where the next request entry is to be written. The first buffer can be a circular buffer, and the write pointer will automatically return to the beginning of the first buffer when it reaches the end of the first buffer, forming a loop to avoid write pointer overflow. Similarly, if the order of the request entries is used to characterize the order of the request entries within the same type of request entries, then the number of write pointers can also be multiple. The depth of the first buffer can be configured according to the number of outstanding data items popped from each data sending queue. This number of outstanding data items can be determined based on factors such as network load and program communication mode. This specification does not limit this.
[0049] In one implementation, in order to improve the sending efficiency of request messages, when the request entry is of the second entry type, the sending condition corresponding to the request entry of the second entry type includes: the second entry type is the first in the order of arrangement in the first buffer.
[0050] The request message corresponding to the second entry type can be used to request the writing of data to the memory. As mentioned earlier, this type of request message carries information such as the address where the data needs to be written. If multiple second entry types arrive at the memory and are not responded to in the order of sending, it will not lead to problems such as data being written to the wrong address. Therefore, when the second entry type is first in the first buffer, the request message corresponding to the second entry type can be sent to the memory without waiting for the response message of the request message sent before the request message corresponding to the second entry type. This can improve the sending efficiency of the request message corresponding to the second entry type. Accordingly, if the order of the request entries is used to characterize the order of the request entries in the same type of request entries, the sending condition corresponding to the request entry of the second entry type may include: in the first buffer, the second entry type is first in the order of the same type of request entries.
[0051] refer to Figure 3 , Figure 3 A schematic diagram of a feasible structure for an RDMA device is shown. Figure 3In this context, RDMA devices can connect to an on-chip network via an AXI (Advanced eXtensible Interface) bus, thereby enabling communication with memory and host nodes through the on-chip network. An RDMA device may include an AXI bridge, which can be configured with a first buffer, a second buffer, and a write response buffer. The first buffer can be used to store the request entry corresponding to the request message (…). Figure 3 (referred to as "entry" in Chinese) Request entries from different data transmission queues can be arranged sequentially in the first buffer. Each request entry can include queue number Qn, request type Type, condition unlock identifier cq_vld, unique identifier Tag, and request index number Id. The second buffer can be used to store request messages. The unique identifier Tag of the request message can be used to find the message in the second buffer. The write response buffer is used to store response messages. The response message can have a request index number Id. When the request index number Id of the response message is the same as the request index number Id of the request entry, it indicates that the response message corresponds to the request entry.
[0052] In one embodiment of this specification, a feasible data processing method is provided, which may include: 1. The host node prepares the descriptors and receive buffer. This step is performed before initiating RDMA operation. A descriptor is a data structure that can contain key information required for RDMA operation, such as source address, destination address, and data length. The receive buffer is a buffer allocated in the host node's memory used to read data written to memory by the RDMA device.
[0053] 2. RDMA Descriptor Read: The RDMA device obtains relevant information for the current data write operation by reading the descriptor prepared by the host node. The RDMA device typically reads the descriptor's contents from the host node's memory using some mechanism (such as memory mapping, register access, etc.). Once the RDMA device successfully reads the descriptor, it understands key information such as where to read data, where to write data, and the data size, preparing for subsequent data transfer.
[0054] 3. The AXI request message (hereinafter referred to as the request message) sent by RDMA is stored in the second buffer, which can be located in a storage device such as SRAM. At the same time, the first entry type and the second entry type are stored in the first buffer. Each request entry can contain multiple fields such as queue number, request type, condition unlock flag, and unique identifier.
[0055] 4. The second entry type can be sent to the memory in sequence. The request message corresponding to the first entry type can be sent to the memory when the response messages of other request messages sent before this message have all been returned (that is, when the sending condition corresponding to the first entry type is met, the request message corresponding to the first entry type is sent).
[0056] 5. After the memory responds to the request message and completes the execution, it returns a response message to the RDMA device. The response message can be stored in a write response buffer. Each response message can also carry fields such as an index number to determine which request message it belongs to. By traversing the write response buffer and determining which request entry it belongs to based on the index number field, the request entry to be deleted can be removed from the first buffer. The request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
[0057] 6. The host node reads data from memory based on whether the descriptor status bit is set.
[0058] This data processing method ensures that the request message corresponding to the first entry type is sent only if all the request messages sent before it have been responded to. This avoids the situation where the descriptor status bit is set before the data is written, and avoids the problem of the host node not being able to read complete data.
[0059] Exemplary device In one exemplary embodiment of this specification, a data processing apparatus is also provided, applied to a remote direct memory read (RDMA) device, wherein the RDMA device establishes communication connections with both a host node and a memory, and the RDMA device includes a first buffer and multiple data transmission queues; the data processing apparatus includes: The request generation module is used to generate multiple request messages in response to the data processing task sent by the host node, and the multiple request messages correspond to one or more of the data sending queues; An entry storage module is used to store the request entries corresponding to each of the plurality of request messages into the first buffer, wherein the request entries are used to describe the request messages corresponding to the request entries; The request sending module is configured to send the request message corresponding to the request entry to the memory according to the order of the request entries in the first buffer, provided that the sending condition corresponding to the request entry is met. When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
[0060] Optionally, the request message corresponding to the first entry type is used to set the descriptor status bit in the memory, and the set descriptor status bit is used to indicate that the data writing is complete.
[0061] Optionally, the order of the request entries is used to characterize the order of the request entries within the same type of request entries, and the request messages corresponding to the same type of request entries are sent from the same data sending queue.
[0062] Optionally, the request entry includes a queue number, a request type, and a conditional unlock identifier; wherein, The queue number is used to identify the data transmission queue for the request message corresponding to the request entry; The request type is used to characterize whether the request entry is the first entry type; The condition unlocking flag is used to indicate whether the sending condition corresponding to the request entry has been met.
[0063] Optionally, the RDMA device further includes a second buffer; the second buffer is used to store multiple request messages; The request entry further includes a unique identifier, which describes the storage location of the request message corresponding to the request entry in the second buffer.
[0064] Optionally, the RDMA device further includes: a write response buffer, the write response buffer being used to store a response message of the request message, the response message being used to indicate that the request message has been executed; The data processing device further includes: The response processing module is configured to, upon receiving the response message, write the response message into the write response buffer and delete the request entries to be deleted from the first buffer, wherein the request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
[0065] Optionally, the RDMA device further includes a read pointer that points to the first request entry in the first buffer; After deleting the request entry to be deleted from the first buffer, the response processing module is further configured to: Move all other request entries in the first buffer forward.
[0066] Optionally, when the request entry is of the second entry type, the sending condition corresponding to the request entry of the second entry type includes: the second entry type is the first in the order of arrangement in the first buffer.
[0067] For specific limitations regarding the data processing device, please refer to the limitations regarding the data processing method above, which will not be repeated here. Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.
[0068] Exemplary accelerator One embodiment of this specification also provides a hardware accelerator, such as Figure 4 As shown, it includes: a host node, memory, and an RDMA device; wherein, The RDMA device establishes communication connections with both the host node and the memory, and the RDMA device includes a first buffer and multiple data transmission queues; The RDMA device is configured as follows: In response to the data processing task sent by the host node, multiple request messages are generated, and the multiple request messages correspond to one or more of the data sending queues; The request entries corresponding to each of the plurality of request messages are stored in the first buffer, and the request entries are used to describe the request messages corresponding to the request entries; According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, the request message corresponding to the request entry is sent to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
[0069] Optionally, the request message corresponding to the first entry type is used to set the descriptor status bit in the memory, and the set descriptor status bit is used to indicate that the data writing is complete.
[0070] Optionally, the order of the request entries is used to characterize the order of the request entries within the same type of request entries, and the request messages corresponding to the same type of request entries are sent from the same data sending queue.
[0071] Optionally, the request entry includes a queue number, a request type, and a conditional unlock identifier; wherein, The queue number is used to identify the data transmission queue for the request message corresponding to the request entry; The request type is used to characterize whether the request entry is the first entry type; The condition unlocking flag is used to indicate whether the sending condition corresponding to the request entry has been met.
[0072] Optionally, the RDMA device further includes a second buffer; the second buffer is used to store multiple request messages; The request entry further includes a unique identifier, which describes the storage location of the request message corresponding to the request entry in the second buffer.
[0073] Optionally, the RDMA device further includes: a write response buffer, the write response buffer being used to store a response message of the request message, the response message being used to indicate that the request message has been executed; The RDMA device is further configured to, upon receiving the response message, write the response message into the write response buffer and delete the request entries to be deleted from the first buffer, wherein the request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
[0074] Optionally, the RDMA device further includes a read pointer that points to the first request entry in the first buffer; After deleting the request entry to be deleted from the first buffer, the response processing module is further configured to: Move all other request entries in the first buffer forward.
[0075] Optionally, when the request entry is of the second entry type, the sending condition corresponding to the request entry of the second entry type includes: the second entry type is the first in the order of arrangement in the first buffer.
[0076] Optionally, the hardware accelerator further includes: an on-chip network; The RDMA device establishes communication connections with the host node and the memory respectively through the on-chip network; The host node includes a processor, which includes at least one of a graphics processor and a neural network processor; The RDMA device includes a network interface card (NIC).
[0077] Exemplary computing device Another embodiment of this application also proposes a computing device, see [link to relevant documentation] Figure 5 As shown, an exemplary embodiment of this specification also provides a computing device, including a hardware accelerator as described in any of the above embodiments.
[0078] The internal structure of the computing device can be as follows: Figure 5 As shown, in addition to the hardware accelerator, the computing device may also include a processor, memory, network interface, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection.
[0079] The processor may include the main processor, as well as baseband chips, modems, etc.
[0080] It is understood that the processor described in this specification can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method implementation can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor.
[0081] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0082] Input devices may include devices that receive data and information input by the user, such as keyboards, mice, cameras, scanners, light pens, voice input devices, touch screens, pedometers, or gravity sensors.
[0083] Output devices may include devices that allow information to be output to the user, such as displays, printers, speakers, etc.
[0084] The communication interface may include any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.
[0085] The computing device may also include a display component and a voice component. The display component may be a liquid crystal display screen or an e-ink display screen. The input device of the computing device may be a touch layer covering the display component, or a button, trackball or touchpad set on the casing of the computing device, or an external keyboard, touchpad or mouse, etc.
[0086] Those skilled in the art will understand that Figure 5 The structures shown are merely block diagrams of some structures related to the solutions in this specification and do not constitute a limitation on the computing devices on which the solutions in this specification are applied. Specific computing devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.
[0087] Exemplary computer program products and storage media In addition to the methods and devices described above, the data processing methods provided in the embodiments of this specification can also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data processing methods according to various embodiments of this specification as described in the "Exemplary Methods" section above.
[0088] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.
[0089] The computer program product described herein can be written in any combination of one or more programming languages to perform the operations of the embodiments described herein. These programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0090] Furthermore, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of the steps in the data processing methods according to various embodiments of this specification as described in the "Exemplary Methods" section above.
[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0093] The embodiments described above are merely illustrative of several implementation methods outlined in this specification. While the descriptions are specific and detailed, they should not be construed as limiting the scope of the solutions provided in this specification. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this specification, and these all fall within the scope of protection of this specification. Therefore, the scope of protection for this patent should be determined by the appended claims.
Claims
1. A data processing method, characterized in that, An application to a remote direct memory read (RDMA) device, wherein the RDMA device establishes communication connections with both the host node and the memory, and the RDMA device includes a first buffer and multiple data transmission queues; The data processing method includes: In response to the data processing task sent by the host node, multiple request messages are generated, and the multiple request messages correspond to one or more of the data sending queues; The request entries corresponding to each of the plurality of request messages are stored in the first buffer, and the request entries are used to describe the request messages corresponding to the request entries; According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, the request message corresponding to the request entry is sent to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
2. The method according to claim 1, characterized in that, The request message corresponding to the first entry type is used to set the descriptor status bit in the memory. The set descriptor status bit is used to indicate that the data writing is complete.
3. The method according to claim 1, characterized in that, The order of the request entries is used to characterize the order of the request entries within the same type of request entries, and the request messages corresponding to the same type of request entries are sent from the same data sending queue.
4. The method according to claim 3, characterized in that, The request entry includes a queue number, a request type, and a conditional unlock identifier; wherein... The queue number is used to identify the data transmission queue for the request message corresponding to the request entry; The request type is used to characterize the entry type of the request entry, and the entry type includes at least the first entry type; The condition unlocking flag is used to indicate whether the sending condition corresponding to the request entry has been met.
5. The method according to claim 4, characterized in that, The RDMA device further includes a second buffer; the second buffer is used to store multiple request messages; The request entry further includes a unique identifier, which describes the storage location of the request message corresponding to the request entry in the second buffer.
6. The method according to any one of claims 1 to 5, characterized in that, The RDMA device further includes: a write response buffer, which is used to store the response message of the request message, and the response message is used to indicate that the request message has been executed. The data processing method further includes: Upon receiving the response message, the response message is written to the write response buffer, and the request entries to be deleted are deleted from the first buffer. The request entries to be deleted include the request entries corresponding to the request message corresponding to the response message.
7. The method according to claim 6, characterized in that, The RDMA device also includes a read pointer that points to the first request entry in the first buffer; After deleting the request entry to be deleted from the first buffer, the method further includes: Move all other request entries in the first buffer forward.
8. The method according to any one of claims 1 to 5, characterized in that, When the request entry is of the second entry type, the sending conditions corresponding to the request entry of the second entry type include: the second entry type is the first in the order of arrangement in the first buffer.
9. A hardware accelerator, characterized in that, include: Host node, memory, and RDMA device; among which, The RDMA device establishes communication connections with both the host node and the memory, and the RDMA device includes a first buffer and multiple data transmission queues; The RDMA device is configured as follows: In response to the data processing task sent by the host node, multiple request messages are generated, and the multiple request messages correspond to one or more of the data sending queues; The request entries corresponding to each of the plurality of request messages are stored in the first buffer, and the request entries are used to describe the request messages corresponding to the request entries; According to the order of the request entries in the first buffer, if the sending condition corresponding to the request entry is met, the request message corresponding to the request entry is sent to the memory; When the request entry is of the first entry type, the sending conditions corresponding to the request entry of the first entry type include: receiving target response information, the target response information indicating that the target request message has been executed, and the target request message including other request messages whose sending order is before the sending order of the request messages corresponding to the first entry type.
10. The hardware accelerator according to claim 9, characterized in that, Also includes: On-screen network; The RDMA device establishes communication connections with the host node and the memory respectively through the on-chip network; The host node includes a processor, which includes at least one of a graphics processor and a neural network processor; The RDMA device includes a network interface card (NIC).
11. A computing device, characterized in that, include: The hardware accelerator as described in any one of claims 9 to 10.