Method and apparatus for DMA between accelerator cards, and accelerator card, acceleration platform and medium

Abstract

The present application discloses a method and apparatus for direct memory access (DMA) between accelerator cards, and an accelerator card, an acceleration platform and a nonvolatile readable storage medium in the technical field of computers. According to the present application, a first accelerator card can actively initiate DMA, directly writes data into a memory of a second accelerator card in a DMA mode on the basis of idle memory information of the second accelerator card of a data destination end recorded in the first accelerator card, without inquiring an idle memory address of the second accelerator card before DMA, and without waiting for the second accelerator card to initiate DMA. The scheme realizes a direct DMA write operation of different accelerator cards in a hardware level, and solves the problem of DMA operation between accelerator cards.

Description

DMA method, device, accelerator card, acceleration platform and medium between accelerator cards

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on October 7, 2023, with application number 202311279391.0, and application name "DMA method, device, accelerator card, acceleration platform and medium between accelerator cards", all contents of which are incorporated by reference in this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a DMA method and device between accelerator cards, an accelerator card, an acceleration platform, and a non-volatile readable storage medium. Background Art

[0004] In an acceleration architecture consisting of a host and an accelerator card, DMA (Direct Memory Access) requires the host's participation. This means the accelerator card can only perform DMA with the host. Furthermore, when performing DMA between the accelerator card and the host, the host must first determine the data's storage address and notify the accelerator card. The accelerator card then initiates DMA to the host, and only then can the data be sent to the accelerator card. This interaction is quite complex.

[0005] Therefore, how to simplify the DMA process and improve DMA efficiency is a problem that those skilled in the art need to solve.

[0006] Summary of the Invention

[0007] In view of this, the purpose of this application is to provide a DMA method, device, accelerator card, acceleration platform and non-volatile readable storage medium between accelerator cards to simplify the DMA process and improve DMA efficiency. The specific solution is as follows:

[0008] The present application provides a DMA method between accelerator cards, which is applied to a first accelerator card in an acceleration platform without a host, comprising: if an acceleration task is executed to obtain data to be transmitted, determining a second accelerator card in the acceleration platform that is configured to process the data to be transmitted based on the free memory capacity and the accelerator card function; each accelerator card in the acceleration platform is provided with a memory direct access controller; querying the memory free information of the second accelerator card in the acceleration platform; broadcasting the memory free information in the acceleration platform when the free memory capacity of the second accelerator card is greater than a preset threshold; and using the memory direct access controller and the memory free information in the second accelerator card to write the data to be transmitted into the memory of the second accelerator card in a memory direct access manner.

[0009] Optionally, determining a second accelerator card in the acceleration platform configured to process the data to be transmitted based on the free memory capacity and the accelerator card function includes: estimating the amount of data to be transmitted; and determining a second accelerator card that has the function of processing the data to be transmitted and has free memory capacity that is not less than the amount of data.

[0010] Optionally, determining a second accelerator card that has a function of processing data to be transmitted and has a free memory capacity that is not less than the size of the data includes: querying the data processing functions of other accelerator cards in the acceleration platform except the first accelerator card, and selecting a candidate accelerator card whose data processing function is consistent with the next processing function of the data to be transmitted; and determining at least one candidate accelerator card whose free memory capacity is not less than the size of the data as the second accelerator card.

[0011] Optionally, the method further includes: regularly broadcasting its own storage resource usage information to other acceleration cards in the acceleration platform except the first acceleration card.

[0012] Optionally, it also includes: estimating the amount of memory required to execute the acceleration task, and if the difference between its own current free memory capacity and the memory amount is greater than a preset threshold, broadcasting its own storage resource usage information to other acceleration cards in the acceleration platform except the first acceleration card.

[0013] Optionally, the method further includes: receiving storage resource usage information broadcasted by other acceleration cards in the acceleration platform except the first acceleration card; when the memory free capacity of the other acceleration cards is greater than a preset threshold, the other acceleration cards broadcast storage resource usage information including memory free information in the acceleration platform; and updating the memory free information of the other acceleration cards stored in the acceleration platform according to the storage resource usage information.

[0014] Optionally, the data to be transmitted is written into the memory of the second accelerator card in a memory direct access manner using the memory direct access controller and memory free information in the accelerator card itself, including: using the memory direct access controller in the accelerator card itself to construct descriptor information according to the memory free information and the memory address occupied by the data to be transmitted in the accelerator card itself; constructing the data to be transmitted into a first message according to the descriptor information; and sending the first message to the second accelerator card so that the second accelerator card writes the data to be transmitted in the first message into its own memory.

[0015] Optionally, the second accelerator card writes the data to be transmitted in the first message into its own memory, including: if the second accelerator card determines that the data write address of the first message falls within the memory address range, writing the data to be transmitted in the first message into its own memory.

[0016] Optionally, it also includes: receiving a second message and converting the second message into a target format; if it is determined that the data write address of the second message falls within the register address range, writing the data in the second message into its own register; using the data in the register to configure the table entries in the address mapping table, and / or modifying the broadcast period of the storage resource usage information.

[0017] Optionally, the method further includes: if it is determined that the data write address of the second message falls within the memory address range, writing the data in the second message into its own memory.

[0018] Optionally, the method further includes: receiving a memory direct access request sent by an external host; searching an address mapping table for a memory address corresponding to a virtual address in the memory direct access request; and completing the memory direct access request according to the memory address.

[0019] Optionally, the method further includes: if the data to be transmitted is the final result of the acceleration task, sending the data to be transmitted to an external host.

[0020] Optionally, the method further includes: receiving an acceleration task sent by an external host through a switch network.

[0021] Optionally, the downstream interface of the first accelerator card is a root device mode interface or a slave device mode interface that supports the serial computer expansion bus standard, and the bus address range of the downstream interface is not less than the bus address range of the downstream device to which it is connected.

[0022] Optionally, the configuration process of the downstream interface includes: receiving a configuration message of the downstream interface through the upstream interface of the first accelerator card; converting the configuration message into a second message that complies with the serial computer expansion bus standard; configuring the bus address range of the downstream device according to the second message, and configuring identification information for the downstream device; configuring the bus address of the root device core or the slave device core of the downstream interface according to the second message, so that the bus address range of the downstream interface is not less than the bus address range of the downstream device.

[0023] Optionally, before performing bus address configuration on the slave device core of the downstream interface according to the second message, the second message is converted into a data format that matches the slave device mode.

[0024] Optionally, there are multiple downstream interfaces; accordingly, it also includes: receiving a data packet sent by a downstream device connected to any downstream interface; if the destination address of the data packet is not within the network routing window, determining whether the destination address of the data packet is within the forwarding routing window of other downstream interfaces; if the destination address of the data packet is within the forwarding routing window of the destination interface, forwarding the data packet to the downstream device connected to the destination interface through the destination interface; the destination interface is at least one of the other downstream interfaces.

[0025] The present application provides a DMA device between accelerator cards, which is applied to a first accelerator card in an acceleration platform without a host, including: a determination module, configured to determine, if an acceleration task is executed to obtain data to be transmitted, a second accelerator card in the acceleration platform configured to process the data to be transmitted based on the free memory capacity and the function of the accelerator card; each accelerator card in the acceleration platform is provided with a memory direct access controller; a query module, configured to query the memory free information of the second accelerator card in itself; when the free memory capacity of the second accelerator card is greater than a preset threshold, the second accelerator card broadcasts the memory free information in the acceleration platform; and an access module, configured to use the direct memory access controller and the memory free information in itself to write the data to be transmitted into the memory of the second accelerator card in a memory direct access manner.

[0026] The present application provides an accelerator card, which is configured to implement any of the above methods. The accelerator card is provided with a memory direct access controller, a storage resource register, a selection module and a memory; the selection module determines other accelerator cards configured to process the data to be transmitted obtained by executing the acceleration task based on the memory free capacity and the accelerator card function; queries the memory free information of other accelerator cards; when the memory free capacity of the other accelerator cards is greater than a preset threshold, the other accelerator cards broadcast the memory free information in the acceleration platform; the storage resource register stores the memory free information of the other accelerator cards; and the memory direct access controller writes the data to be transmitted into the memory of the other accelerator cards in a memory direct access manner based on the memory free information.

[0027] Optionally, the acceleration card is an FPGA (Field Programmable Gate Array) acceleration card, an ASIC (Application Specific Integrated Circuit) acceleration card, or a multi-core processor.

[0028] The present application provides an acceleration platform, comprising: a plurality of the aforementioned acceleration cards, wherein different acceleration cards are connected via an expansion device; wherein each acceleration card is provided with a memory direct access controller, a storage resource register, a selection module, and a memory; the selection module determines, based on the free memory capacity and the function of the acceleration card, other acceleration cards configured to process data to be transmitted obtained by executing an acceleration task; queries the memory free information of other acceleration cards; when the free memory capacity of the other acceleration cards is greater than a preset threshold, the other acceleration cards broadcast the memory free information in the acceleration platform; the storage resource register stores the memory free information of the other acceleration cards; and the memory direct access controller writes the data to be transmitted into the memory of the other accelerator cards in a memory direct access manner based on the memory free information.

[0029] Optionally, the selection module also includes: an estimation unit and a selection unit; the estimation unit estimates the amount of data to be transmitted; the selection unit queries the data processing functions of other accelerator cards, and selects an accelerator card whose data processing function is consistent with the next processing function of the data to be transmitted and whose memory free capacity is not less than the data amount.

[0030] Optionally, each accelerator card is also provided with a resource sharing module, which estimates the amount of memory required to execute the acceleration task. If the difference between the memory free capacity and the memory amount is greater than a preset threshold, the memory storage resource usage information is broadcast to other accelerator cards; if it is in an idle state, when the memory free capacity is greater than a preset threshold, the memory storage resource usage information is broadcast to other accelerator cards; accordingly, the resource sharing module also receives the storage resource usage information including memory free information broadcast by other accelerator cards in the acceleration platform when the memory free capacity is greater than the preset threshold; and updates the memory free information of other accelerator cards stored in the storage resource register according to the storage resource usage information.

[0031] The present application provides a non-volatile readable storage medium configured to store a computer program, wherein the computer program implements any of the aforementioned methods when executed by a processor.

[0032] It can be seen from the above scheme that the present application provides a DMA method between accelerator cards, which is applied to the first accelerator card in a hostless acceleration platform, including: if the acceleration task is executed to obtain the data to be transmitted, the second accelerator card in the acceleration platform configured to process the data to be transmitted is determined based on the memory free capacity and the accelerator card function; each accelerator card in the acceleration platform is provided with a memory direct access controller; the memory free information of the second accelerator card is queried in itself; the memory free information of the second accelerator card is broadcast in the acceleration platform when the memory free capacity is greater than a preset threshold; the memory direct access controller and memory free information in itself are used to write the data to be transmitted into the memory of the second accelerator card in a memory direct access manner.

[0033] As can be seen, the beneficial effect of this application is that the first accelerator card, acting as the data source, actively initiates DMA and, based on the memory free space information of the second accelerator card at the data destination recorded in its own memory, directly writes the data to the memory of the second accelerator card via DMA. There is no need to query the second accelerator card's memory free address before DMA, nor to wait for the second accelerator card to initiate DMA. This solution implements direct DMA write operations between different accelerator cards at the hardware level, simplifying the DMA process and improving DMA efficiency.

[0034] Correspondingly, the DMA device between accelerator cards, accelerator card, acceleration platform and non-volatile readable storage medium provided in this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0036] FIG1 is a flow chart of a DMA method for accelerating inter-cards disclosed in this application;

[0037] FIG2 is a flow chart of a data transmission method disclosed in this application;

[0038] FIG3 is a schematic diagram of the DMA function of an FPGA accelerator card disclosed in this application;

[0039] FIG4 is a schematic diagram of message parsing and judgment disclosed in this application;

[0040] FIG5 is a schematic diagram of a DMA trigger and memory update disclosed in this application;

[0041] FIG6 is a schematic diagram of an acceleration system disclosed in this application;

[0042] FIG7 is a schematic diagram of the switch function of another FPGA accelerator card disclosed in this application;

[0043] FIG8 is a structural diagram of an accelerator card 1 provided in this application;

[0044] FIG9 is a structural diagram of an accelerator card 2 provided by the present application;

[0045] FIG10 is a schematic diagram of an acceleration platform disclosed in this application;

[0046] FIG11 is a structural diagram of another accelerator card provided by the present application;

[0047] FIG12 is a schematic diagram of another acceleration platform provided by this application;

[0048] FIG13 is a schematic diagram of another acceleration platform provided in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] At present, in the acceleration architecture composed of the host and the accelerator card, DMA (Direct Memory Access) must be participated by the host, that is, the accelerator card can only perform DMA with the host. In addition, when the accelerator card and the host use DMA, the host needs to first determine the storage address of the data in its own storage address, and then notify the accelerator card. After that, the accelerator card initiates DMA to the host, and then the data can be sent to the accelerator card. The interaction process is relatively cumbersome. To this end, the present application provides a DMA solution between accelerator cards, which can realize direct DMA write operations of different accelerator cards at the hardware level, simplifying the DMA process and improving DMA efficiency.

[0051] As shown in FIG1 , an embodiment of the present application discloses a DMA method between accelerator cards, which is applied to a first accelerator card in a hostless acceleration platform, including:

[0052] S101: If data to be transmitted is obtained by executing an acceleration task, a second acceleration card in the acceleration platform configured to process the data to be transmitted is determined according to free memory capacity and acceleration card functions.

[0053] In an embodiment of the present application, a hostless acceleration platform can receive acceleration tasks sent by an external host, such as image recognition tasks, encryption or decryption tasks, etc. Generally, the same acceleration task can be divided among different accelerator cards in the acceleration platform. For example, a first accelerator card performs the table lookup operation in the acceleration task, while a second accelerator card performs logical operations on the table lookup result to obtain the processing result of the entire task. Multiple first and second accelerator cards can be used.

[0054] It should be noted that the acceleration task is sent from the external host to the first accelerator card via the switch network. The first accelerator card may first perform the pre-operation of the acceleration task (such as the aforementioned table lookup operation) or the post-operation of the acceleration task (such as the aforementioned logical operation operation). If the first accelerator card performs the post-operation of the acceleration task, the data to be transmitted is the final result of the acceleration task. Therefore, the data to be transmitted is sent to the external host, eliminating the need for other accelerator cards to process the data to be transmitted.

[0055] In an optional embodiment, determining a second accelerator card in an acceleration platform configured to process data to be transmitted based on free memory capacity and accelerator card functions includes: estimating the size of the data to be transmitted; and determining a second accelerator card that has the function of processing the data to be transmitted and whose free memory capacity is not less than the size of the data. Determining a second accelerator card that has the function of processing the data to be transmitted and whose free memory capacity is not less than the size of the data includes: querying the data processing functions of other accelerator cards in the acceleration platform except the first accelerator card, and selecting a candidate accelerator card whose data processing function is consistent with the next processing function of the data to be transmitted; and determining at least one candidate accelerator card whose free memory capacity is not less than the size of the data as the second accelerator card. It can be seen from this that the second accelerator card is an accelerator card that has the function of processing the data to be transmitted and whose remaining memory capacity is large enough; if the next processing function of the data to be transmitted is encryption, then the accelerator card function is encryption, and the second accelerator card must be able to perform encryption operations. If an accelerator card can only perform decryption operations, then the accelerator card function is decryption, and the accelerator card will not be selected as a candidate accelerator card.

[0056] Each acceleration card in the acceleration platform is provided with a direct memory access controller (ie, a DMA controller).

[0057] S102: Query the memory free information of the second accelerator card in the accelerator.

[0058] The second acceleration card broadcasts memory idle information in the acceleration platform when the memory idle capacity is greater than a preset threshold.

[0059] It should be noted that each accelerator card in the acceleration platform records the storage resource usage information of other accelerator cards in the acceleration platform, and the storage resource usage information includes: free memory starting address, total memory capacity and other information. Therefore, in an optional embodiment, the first accelerator card periodically broadcasts its own storage resource usage information to other accelerator cards in the acceleration platform except the first accelerator card. The first accelerator card also receives the storage resource usage information broadcasted by other accelerator cards in the acceleration platform except the first accelerator card; and updates the memory free information of other accelerator cards stored in itself according to the storage resource usage information. The memory free information is specifically: free memory starting address. The second accelerator card or other accelerator cards can be: an accelerator card connected to the first accelerator card through an expansion device, or a downstream device connected to any downstream interface of the first accelerator card.

[0060] In one example, a first accelerator card can estimate the amount of memory required to execute an acceleration task. If the difference between its current free memory capacity and the total memory capacity is greater than a preset threshold, the first accelerator card broadcasts its own storage resource usage information to all other accelerator cards in the acceleration platform, excluding the first accelerator card. If the first accelerator card is in an idle state, the first accelerator card broadcasts its own storage resource usage information to all other accelerator cards in the acceleration platform, excluding the first accelerator card, when its current free memory capacity is greater than a preset threshold. Accordingly, the storage resource usage information broadcast by other accelerator cards and received by the first accelerator card is the information broadcast by the other accelerator cards when their free memory capacity is greater than the preset threshold, and the storage resource usage information includes memory free information.

[0061] S103: Using the memory direct access controller and memory idle information in the accelerator card, the data to be transmitted is written into the memory of the second accelerator card in a memory direct access manner.

[0062] In an optional embodiment, the data to be transmitted is written into the memory of the second accelerator card in a memory direct access manner using the memory direct access controller and memory free information in itself, including: using the memory direct access controller in itself to construct descriptor information according to the memory free information and the memory address occupied by the data to be transmitted in itself; constructing the data to be transmitted into a first message according to the descriptor information; sending the first message to the second accelerator card so that the second accelerator card writes the data to be transmitted in the first message into its own memory. Wherein, the second accelerator card writes the data to be transmitted in the first message into its own memory, including: if the second accelerator card determines that the data write address of the first message falls within the memory address range, then the data to be transmitted in the first message is written into its own memory. Wherein, the first message is specifically a TLP (Transaction Layer Packet) message, and different accelerator cards communicate through PCIE (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard). The memory direct access method is: DMA method.

[0063] In an embodiment of the present application, the first accelerator card further receives a second message (a message for configuring registers) and converts the second message into a target format (e.g., PCIE Stream, PCIE stream mode); if it is determined that the data write address of the second message falls within the register address range, the data in the second message is written into its own register; the data in the register is used to configure an entry in an address mapping table, and / or to modify the broadcast period of storage resource usage information. If it is determined that the data write address of the second message falls within the memory address range, the data in the second message is written into its own memory.

[0064] The entries in the address mapping table are mappings between virtual addresses used by the external host and memory addresses. Therefore, in an optional embodiment, the first accelerator card also receives a direct memory access request from the external host; queries the address mapping table for the memory address corresponding to the virtual address in the direct memory access request; and completes the direct memory access request according to the memory address. Therefore, any accelerator card in the embodiments of the present application can also perform direct memory access with the external host.

[0065] Referring to the process shown in Figure 1 , the corresponding acceleration platform can be seen in Figure 13 . In Figure 13 , the hostless acceleration platform includes a first accelerator card and a second accelerator card. Both the first accelerator card and the second accelerator card are equipped with a memory direct access controller and memory modules. The first accelerator card and the second accelerator card can directly perform DMA data transfer according to the process shown in Figure 1 , and can directly perform DMA write operations at the hardware level. The first accelerator card and the second accelerator card store each other's memory free information in a storage resource register. The first accelerator card and the second accelerator card can be connected via an expansion device.

[0066] As can be seen, in this embodiment of the application, the first accelerator card, acting as the data source, actively initiates DMA and, based on the memory free space information of the second accelerator card at the data destination recorded in its own memory, directly writes the data to the memory of the second accelerator card via DMA. There is no need to query the second accelerator card for the free memory address before DMA operation, nor to wait for the second accelerator card to initiate DMA. This solution implements direct DMA write operations between different accelerator cards at the hardware level, simplifying the DMA process and improving DMA efficiency.

[0067] The embodiment of the present application discloses another data transmission method, which is applied to any accelerator card, wherein the downstream interface of the accelerator card is a root device mode interface or a slave device mode interface that supports a serial computer expansion bus standard (such as PCIE), and the bus address range of the downstream interface is not less than the bus address range of the downstream device to which it is connected. The PCIE interface of the accelerator card has a root device mode (Root Port, RP) and a slave device mode (End Point, EP). The current root device mode and the current slave device mode only support one-to-one communication between the accelerator card and the device connected to its PCIE interface, and do not support communication between the upstream device of the accelerator card and the device connected to the PCIE interface of the accelerator card, nor support P2P (Peer-to-Peer, point-to-point) communication between devices connected to different PCIE interfaces of the accelerator card.

[0068] 2 , the data transmission method provided in the embodiment of the present application includes:

[0069] S201: Receive a data message sent by a downstream device connected to any downstream interface of an accelerator card.

[0070] S202. If the destination address of the data message is within the network routing window, the upstream transmission protocol is determined according to the current protocol information in the network register, the data message is converted into an upstream message that complies with the upstream transmission protocol, and the upstream message is sent to the upstream device through the upstream interface.

[0071] S203. If the destination address of the data message is within the forwarding routing window of the destination interface, forward the data message to the downstream device connected to the destination interface through the destination interface; the destination interface is at least one of the other downstream interfaces.

[0072] The other downstream interfaces and the downstream interface for sending data packets are different downstream interfaces belonging to the same accelerator card.

[0073] In the embodiment of the present application, step S202 enables the message sent by the downstream device to the accelerator card through the downstream interface to reach the upstream device connected to the upstream interface, thereby realizing a bottom-to-top data path. In the embodiment of the present application, step S203 enables the message sent by the downstream device to the accelerator card through a certain downstream interface to reach the downstream device connected to other downstream interfaces, thereby realizing a data path between downstream devices connected to different downstream interfaces of the same accelerator card. Accordingly, if the destination address of the data message is not within the network routing window and the destination address of the data message is not within the forwarding routing window of any downstream interface, the data message is discarded.

[0074] In an embodiment of the present application, an accelerator card has multiple downstream interfaces, and the configuration process of any downstream interface includes: receiving a configuration message of the downstream interface through the upstream interface of the first accelerator card; converting the configuration message into a second message that complies with the serial computer expansion bus standard; configuring the bus address range of the downstream device according to the second message, and configuring identification information for the downstream device; configuring the bus address of the root device core or the slave device core of the downstream interface according to the second message, so that the bus address range of the downstream interface is not less than the bus address range of the downstream device. Before configuring the bus address of the slave device core of the downstream interface according to the second message, the second message is converted into a data format that matches the slave device mode. The data format that matches the slave device mode is the type0 format, and the data format that matches the root device mode is the type1 format.

[0075] In one example, the address of the root device core or the slave device core of the downstream interface is configured according to the second message, including: configuring the bus register and the memory window register in the accelerator card according to the second message to configure the root device core of the downstream interface; or converting the second message into a data format that matches the slave device mode, and configuring the bus address obtained by converting the memory address of the downstream device to the slave device core according to the converted second message.

[0076] When configuring the IP address (Internet Protocol Address) for the downstream interface, common command registers such as the memory space enable register, IO (Input / Output) space register, and bus master register are configured; the maximum write payload length register, the maximum read request register, and the strong-order mode register are configured. When initializing the downstream interface, bus registers such as the root bus register, sub-bus register, and maximum bus register are configured; the memory window register is configured to set information such as the interface's prefetchable memory range, the window base address for non-prefetchable memory, and the window size; and the downstream device identification information (BDF value) is configured. This allows the downstream interface to implement switch functionality based on the built-in root device core. The RP's capability configuration space registers are typically located on the root port, while the switch's capability configuration registers are located on the switch. The BDF (Bus Device Function) value refers to the bus number, device number, and function number.

[0077] When configuring the slave core built into the downstream interface, first convert the TLP configuration message in type 0 format to type 1 format. Then configure the BDF register to identify the downstream device of the current interface. Then configure the BAR (bus) address (converted from the memory address of the downstream device) to enable the current interface to read the memory address of the downstream device. Then configure the common command registers such as the memory space enable register, the IO space register, and the bus master register. Also configure the maximum write payload length register, the maximum read request register, and the strong order mode register. At this point, the downstream interface can implement the switch function based on the built-in slave core.

[0078] In an optional embodiment, determining the upstream transmission protocol based on current protocol information in a network register includes: reading the current protocol information from the network register; and determining the protocol type corresponding to the current protocol information as the upstream transmission protocol. The network register stores protocol information for the upstream transmission protocol currently used by the upstream device, and the user can flexibly modify the protocol information in the network register. In one example, in response to a protocol configuration operation input by the user, the protocol information configured by the protocol configuration operation is written to the network register to change the upstream transmission protocol currently used by the upstream device. Upstream transmission protocols include TCP (Transmission Control Protocol, a transport layer communication protocol).

[0079] In an embodiment of the present application, the upstream interface and each downstream interface can be flexibly enabled and disabled. Therefore, before determining the upstream transmission protocol based on the current protocol information in the network register, the process also includes: enabling the upstream interface and disabling the other downstream interfaces so that the data message is sent through the upstream interface. Correspondingly, before forwarding the data message to the downstream device connected to the destination interface via the destination interface, the process also includes: enabling the destination interface and disabling the upstream interface and other downstream interfaces except the destination interface so that the data message is sent through the destination interface.

[0080] In an optional embodiment, the accelerator card further converts data packets received through the upstream interface into a second packet that complies with the serial computer expansion bus standard. If the destination address of the second packet is within the forwarding routing window of at least one downstream interface, the second packet is forwarded via the at least one downstream interface to the downstream device connected to the at least one downstream interface. This allows packets sent from the upstream device to the accelerator card via the upstream interface to reach the downstream device connected to the at least one downstream interface, thus establishing a top-to-bottom data path. If the destination address of the second packet is not within the forwarding routing window of any downstream interface, the second packet is discarded.

[0081] As can be seen, in the embodiments of the present application, any downstream interface of any accelerator card complies with the serial computer expansion bus standard and is configured as a root device mode interface or a slave device mode interface. It is also configured with a bus address range that is no less than that of the downstream device to which it is connected. This allows the accelerator card to implement: bottom-up data paths, top-down data paths, and data paths between downstream devices connected to different downstream interfaces of the accelerator card, without the need for expansion devices. Furthermore, the type of upstream transmission protocol can be flexibly determined based on scenario requirements.

[0082] Taking the FPGA accelerator card as an example, the following describes the specific function implementation that enables the FPGA to directly perform DMA writes at the hardware level. In the embodiment of the present application, the P2P DMA function is implemented internally in the FPGA. It can act as a PCIE Master (master device) to initiate DMA requests, and it can also act as a PCIE Slave (slave device) to receive P2P DMA requests initiated by other PCIE devices, expanding the usage scenarios. Since it operates at a purely hardware level, the throughput performance is close to the theoretical performance of the PCIE interface. Among them, the PCIE Master is the initiator of the DMA request, and the PCIE Slave is the receiver of the DMA request. In the P2P DMA scenario, any EP can act as a Master. EP refers to: accelerator card; P2P DMA means: DMA between one accelerator card and another accelerator card. PCIE P2P DMA is a direct access to the memory space data of two accelerator cards.

[0083] Among them, when the FPGA acts as a PCIE Master and performs DMA with a GPU (graphics processing unit) accelerator card, the FPGA needs to obtain the address of the GPU accelerator card, and the address of the GPU accelerator card is the physical address of the PCIE bus domain. When the FPGA performs DMA with the host, since the host uses a virtual address, it is necessary to convert the virtual address into the physical address of the PCIE bus domain through the address mapping module in the FPGA. If the FPGA acts as a PCIE Slave and another accelerator card GPU acts as a PCIE Master, it is necessary to obtain the physical address of the FPGA's PCIE bus domain. The FPGA maps its own memory space to its own PCIE bus, and then the GPU directly uses the FPGA's PCIE bus domain physical address when performing DMA operations.

[0084] In addition, as a PCIE Slave accelerator card, it is also necessary to be able to parse TLP messages, distinguish the register read and write addresses and DMA memory read and write addresses, and send them to the corresponding modules for processing.

[0085] As shown in Figure 3, an FPGA that can serve as both a PCIE Master and a PCIE Slave requires a DMA controller. The DMA controller includes an interface conversion module, a receiving module, a register control module, a memory control module, a conversion module, an address mapping module, a cache module, and a sending module.

[0086] Interface conversion module. Because the PCIE user interfaces provided by different FPGA manufacturers are not uniform, an interface conversion module can be set up within the FPGA to ensure that all messages sent to the FPGA are in PCIE ST format or PCIE AXIS format (PCIE Advanced eXtensible Interface Stream, PCIE AXIS stream mode). The FPGA's PCIE user interface bus is in PCIE ST format or PCIE AXIS format. The PCIE ST bus is the standard bus for the PCIE interface, so messages sent to the FPGA are prioritized to be in PCIE ST format.

[0087] The receiving module is configured to receive and parse TLP messages output by the PCIE EP core (the hard core included in the FPGA's downstream PCIE interface), discarding any abnormal messages. If the TLP message is for reading or writing to a non-volatile, readable storage medium, it determines the address based on the message. If the address range is within the register read / write address range, the TLP is forwarded to the register control module for processing. If the address range is within the device memory address range, the TLP is sent to the memory control module for processing. If the message is a completion message, it is sent to the conversion module for processing. See Figure 4 for the detailed process.

[0088] The register control module can convert the register read and write control interface.

[0089] The memory control module can control the read and write control signals of the conversion memory.

[0090] The conversion module can convert register write or memory write signals.

[0091] The address mapping module records the mapping of virtual addresses to physical addresses. This module is only valid when the host initiates a DMA operation. At other times, it is directly transmitted to the receiving module. The mapping table in the address mapping module is initialized and configured by the host before application acceleration.

[0092] The cache module is configured to cache FPGA memory usage and free memory status.

[0093] The sending module constructs a descriptor request based on remote network input or DMA trigger requests initiated by the FPGA kernel, the GPU memory resources of the peer accelerator card, and the FPGA memory status. It initiates a DMA request and sends the data to the GPU via the PCIE interface. This module also transmits FPGA memory usage and free memory status to the peer accelerator card GPU via the FPGA's PCIE interface. See Figure 5 for a detailed process.

[0094] As shown in Figure 3, in addition to the DMA controller, the following modules also need to be implemented: registers, master control module, network module, and management module.

[0095] The registers include: a DMA doorbell register; a register that stores resource information of the peer accelerator card, which is periodically written to record the remaining memory resources of the peer accelerator card.

[0096] The master control module can read and write to the FPGA's memory interface. The FPGA's memory is either DDR (Double Data Rate) or HBM (High Bandwidth Memory).

[0097] The network module is a network message processing and control module that parses and processes remotely input messages. Data acceleration calculation signals are sent to the kernel module, DMA data signals are sent to the address mapping module, and control signals are sent to the management module. The kernel is the acceleration unit within the FPGA board, capable of performing network acceleration, calculation acceleration, and storage acceleration.

[0098] The management module can monitor the memory resource usage within the board and detect the resource usage of the remote acceleration card through the network interface.

[0099] The following details the DMA between the FPGA and GPU, and how they work together to complete acceleration tasks, from a system perspective. Figure 6 shows an acceleration platform consisting of FPGA accelerator card 1, FPGA accelerator card 2, and a GPU accelerator card. These three accelerator cards are connected via an expansion device (PCIE switch), enabling direct DMA writes between them. This ensures low DMA transfer latency and decouples the host server from the DMA transfer process. The acceleration platform is connected to a host server via a switch network.

[0100] In this example, it is assumed that the host server uses an acceleration platform to accelerate a recommendation inference algorithm, which is accelerated by both FPGA and GPU accelerator cards. The FPGA accelerator card completes the embedded vector lookup table function, and the GPU accelerator card completes the DNN (Deep Neural Network) calculation acceleration.

[0101] The system's initialization process: The three downstream ports of the PCIE Switch respectively initialize the PCIE interfaces of two FPGA accelerator cards and one GPU accelerator card, completing the configuration of the configuration space and the allocation of the BAR (bus) base address. In the acceleration platform, the three accelerator cards periodically broadcast their own free memory information through the PCIE interface. The PCIE Switch usually includes one upstream port and multiple downstream ports, where the upstream port is connected to the CPU (Central Processing Unit) and the downstream port is connected to the PCIE device. Through the PCIE Switch, the CPU can communicate with multiple PCIe devices, and P2P communication can also be achieved between PCIe devices.

[0102] The DMA process between the accelerator cards of this system is as follows:

[0103] (1) The host server sends the processing data required for the recommendation inference algorithm to FPGA accelerator card 1 and FPGA accelerator card 2 via the network, allowing FPGA accelerator card 1 and FPGA accelerator card 2 to simultaneously perform embedded vector table lookup calculations and accelerate the search in parallel. The network modules of FPGA accelerator card 1 and FPGA accelerator card 2 receive network message information, process it, and then send it to the kernel module for embedded table lookup processing.

[0104] (2) After the table lookup is completed, the data is stored in the memory of FPGA accelerator card 1 and FPGA accelerator card 2. FPGA accelerator card 1 and FPGA accelerator card 2 then notify their respective sending modules to initiate a DMA write operation. In FPGA accelerator card 1 and FPGA accelerator card 2, the sending module receives the DMA doorbell request initiated by the kernel, reads the value of the register storing GPU memory information, and reads the local memory resource usage information to form a descriptor queue to initiate a DMA write operation. The data in the memory is then assembled into a TLP message and sent to the format conversion module.

[0105] (3) In FPGA accelerator card 1 and FPGA accelerator card 2, the TLP message is sent to the PCIE EP core through the format conversion module, and then reaches the PCIE switch through the PCIE interface. The PCIE switch forwards the TLP message to the GPU according to the destination address of the TLP message.

[0106] (4) The PCIE DMA controller inside the GPU receives the data sent by the two FPGA accelerator cards, sends the received data to its own video memory, and then the computing core module starts DNN computing processing.

[0107] (5) After the GPU calculation is completed, a DMA request is triggered. Since the processed data only requires one FPGA card to complete subsequent processing, it is sent to FPGA accelerator card 1 by default. The GPU directly performs DMA write to FPGA accelerator card 1 based on the free memory of FPGA accelerator card 1.

[0108] (6) After the PCIE Switch receives the DMA write request from the GPU to the FPGA accelerator card 1, it forwards it to the FPGA accelerator card 1 according to the destination address. The PCIE interface of the FPGA accelerator card 1 processes it and outputs the TLP message to the inside of the FPGA accelerator card 1.

[0109] (7) FPGA accelerator card 1 receives and parses the TLP message, stores it in its own memory through the memory control module, and then converts the data in the memory into a network message and sends it back to the host server to complete the acceleration processing of the recommended inference algorithm.

[0110] During the entire acceleration process described above, the host server does not need to perform any operations other than sending the original data and receiving the acceleration results. The FPGA acceleration card and GPU acceleration card in the acceleration platform independently complete the control and processing. In the two P2P DMA processes, the party that wants to send data actively initiates DMA writes, and no other interactions are required during the period, with low latency and high performance. Optionally, the P2P DMA provided in the embodiment of the present application reduces two information interactions compared to the existing DMA process; in the existing DMA process, the GPU first needs to inform the FPGA that the data is ready, and then the FPGA initiates a DMA read request to the GPU, and then the GPU data can be sent to the FPGA. The P2P DMA provided in the embodiment of the present application is initiated by the GPU directly to the FPGA. There is no need to inquire about the free memory address of the FPGA, nor is there any need to wait for the FPGA to initiate DMA. As a result, the FPGA can act as a PCIE Master to initiate memory writes to the GPU, and can also act as a PCIE Slave to complete the DMA writes initiated by the GPU. It has a wide range of usage scenarios and supports true P2P DMA functions.

[0111] As can be seen, the embodiments of the present application decouple the DMA process from the host server. By utilizing the DMA controller implemented within each accelerator card, each accelerator card can act as a PCIE Master to initiate DMA requests, and also as a PCIE Slave to receive DMA requests initiated by other accelerator cards, expanding DMA usage scenarios. Furthermore, because DMA is implemented purely through hardware addresses, DMA performance approaches the theoretical performance of the PCIE interface. Furthermore, each accelerator card can autonomously create a descriptor queue and trigger DMA operations, achieving optimal latency and throughput performance, reducing host server CPU usage and freeing up valuable CPU resources.

[0112] The following uses an FPGA accelerator card as an example to describe the communication process between the FPGA's upstream device and the device connected to the FPGA's PCIE interface, as well as the P2P communication process between devices connected to different PCIE interfaces of the FPGA, when the FPGA's PCIE interface is in root device mode.

[0113] Optionally, the embodiment of the present application implements the PCIE Switch function based on the PCIE RP hard core inside the FPGA, which can also improve the utilization rate of FPGA logic resources. When configuring the PCIE interface in RP mode, the PCIE Switch function can be implemented by simply changing the configuration space content.

[0114] As shown in Figure 7, the interface configuration module, downstream device configuration module, logic processing module, and network module are implemented using the PCIE RP hard core within the FPGA. The logic processing module includes a parsing module, a multiplexing module, and an arbitration module for the network module and the two downstream PCIE interfaces, respectively. The FPGA in Figure 7 has two downstream PCIE interfaces, each of which houses a PCIE RP hard core. Of course, an accelerator card can have more or fewer downstream PCIE interfaces, with each downstream PCIE interface connecting to one downstream device.

[0115] Generally, the PCIE RP hard core provides three sets of user interfaces, namely the user receive bus, the user transmit bus and the configuration space configuration bus. Among them, the user receive bus and the transmit bus are user interfaces for transmitting TLP messages. The configuration space configuration bus can be used to complete the configuration of the PCIE RP configuration space, and read the default register value of the configuration space and output it to the parsing module of the logic processing module to assist in parsing and processing. The downstream device configuration module completes the initialization configuration of the configuration space of the two processors through the initiated configuration TLP message. The BAR base addresses of the two processors are configured as BAR1 and BAR2 respectively. The interface configuration module completes the initialization configuration of the configuration space of the two PCIE interfaces of the FPGA through the cfg_mgnt bus (configuration management, control management bus). The BAR base addresses of the two PCIE interfaces are configured as BAR3 and BAR4 respectively, where the BAR3 address range is greater than or equal to the BAR1 address range, and the BAR4 address range is greater than or equal to the BAR2 address range.

[0116] As can be seen, the downstream device configuration module is configured to initialize the downstream EP device. The interface configuration module is configured to initialize and configure the PCIE interface. The logic processing module is configured to implement TLP message forwarding and control, with a parsing module, a multiplexing module, and an arbitration module for each port. The parsing module parses the TLP messages output by the PCIE RP's transmit bus and outputs arbitration requests based on routing information, while also processing abnormal TLP messages. The arbitration module receives arbitration requests and outputs arbitration signals to select the corresponding transmit path for the message. The multiplexing module receives data output by each path parsing module and selects the appropriate path based on the arbitration signal to send to the PCIE RP's receive bus.

[0117] The network module is for processing related to the network interface. In an embodiment of the present application, the network module is connected to the upstream port of the FPGA. The network module implements multiple network transmission protocols, such as RDMA (Remote Direct Memory Access), TCP / IP or a custom low-latency network protocol, etc., and selects one of the network protocols according to the mode configuration input by the user to output to the upstream device through the network interface. As mentioned above, the network module not only supports the PCIE protocol inside the FPGA, but also supports external protocols such as RDMA, TCP / IP, and can realize the mutual conversion between internal and external protocols. Please refer to Figure 7. When selecting an externally supported protocol, it can be dynamically configured through the mode selection switch. The user sends a switch register message for the configuration mode selection through the network interface. After receiving the message, the upstream port of the Switch parses the message and extracts the corresponding mode selection switch value to configure it into the mode selection register module, and the network protocol processing module processes the corresponding network protocol according to the value of the mode selection register.

[0118] In this example, the PCIE switch includes three lanes: two connected to the AI ​​processor and one to the network interface. The following describes communication between two processors connected to the two downstream interfaces of the FPGA. AI processor-1 initiates a memory access to AI processor-2, with the destination address being BAR2. AI processor-1's DMA controller initiates the operation and sends this message via the PCIE interface to PCIE RP IP-1, the downstream port corresponding to the FPGA PCIE switch. PCIE RP IP-1 then sends it to parsing module 1, which parses the TLP message and extracts information such as the destination address. If the destination address is within the routing window of PCIE IP-2, it issues a destination address arbitration request to the arbitration module on each port. In this example, the destination address is BAR2. Arbitration module 2 outputs a valid arbitration enable signal, while the other arbitration modules output invalid enable signals. Based on the valid arbitration signal, the TLP message is sent to the receive bus of PCIE RP IP-2 and then to AI processor-2 via the PCIE interface. The DMA controller of AI processor-2 extracts the data from the TLP message and stores it in the corresponding non-volatile readable storage medium. If the destination address of the TLP message is not within the routing window range (ie, not within the routing window of PCIE IP-2, nor within the routing window of the network path) or the TLP message is abnormal, the message is discarded.

[0119] It should be noted that, according to the embodiments of the present application, each PCIE interface of the FPGA can be configured so that the PCIE interface becomes the downstream interface of the PCIE switch. Of course, the network port of the FPGA can also be configured as the upstream interface of the switch accordingly. In this way, P2P communication between AI processors can be achieved by configuring the FPGA, and communication between the AI ​​processor and the upstream network interface of the FPGA can be achieved without the need for expansion devices or expansion cards.

[0120] It can be seen that the embodiment of the present application can implement the PCIE Switch function based on the PCIE RP IP hard core that comes with the FPGA, without the need for an additional PCIE Switch, with low cost, simple implementation, and low utilization of FPGA logic resources. At the same time, this method can be applied to almost all FPGAs and has strong portability. This method can also be extended to other PCIE processors. If its PCIE interface comes with a PCIE RP IP but does not support the PCIE Switch mode, the downstream port of the PCIE Switch can be implemented through the PCIE RP IP to achieve the Switch forwarding function. Referring to this implementation method, it is also possible to use the PCIE EP IP hard core that comes with the accelerator card to configure the PCIE upstream port of the PCIE Switch without adding a Switch; but it should be noted that in the EP mode, the configuration TLP message format for configuring the PCIE Switch needs to be converted from type0 format to type1 format. Other configuration contents can refer to the configuration of the RP mode provided in this application. The IP (Intellectual property) hard core is a device in the form of a hardware module.

[0121] The following introduces a DMA device between accelerator cards provided in an embodiment of the present application. The DMA device between accelerator cards described below can be referenced with other embodiments described in this document.

[0122] The embodiment of the present application discloses a DMA device between accelerator cards, which is applied to a first accelerator card in a hostless acceleration platform, including:

[0123] a determination module configured to, if the acceleration task is executed and data to be transmitted is obtained, determine, based on the free memory capacity and the accelerator card functions, a second accelerator card in the acceleration platform configured to process the data to be transmitted; each accelerator card in the acceleration platform is provided with a memory direct access controller;

[0124] The query module is configured to query the memory free information of the second accelerator card in itself; when the memory free capacity of the second accelerator card is greater than a preset threshold, the second accelerator card broadcasts the memory free information in the acceleration platform;

[0125] The access module is configured to use the memory direct access controller and memory idle information in the access module to write the data to be transmitted into the memory of the second accelerator card in a memory direct access manner.

[0126] In an optional implementation, the determination module is configured to:

[0127] Estimate the amount of data to be transferred;

[0128] Determine a second accelerator card that has the function of processing the data to be transmitted and has a free memory capacity that is not less than the amount of data.

[0129] In an optional implementation, the determination module is configured to:

[0130] querying the data processing functions of the accelerator cards other than the first accelerator card in the acceleration platform, and selecting a candidate accelerator card whose data processing function is consistent with the next processing function of the data to be transmitted;

[0131] At least one candidate accelerator card whose free memory capacity is not less than the data size is determined as the second accelerator card.

[0132] In an optional embodiment, the method further includes:

[0133] The broadcast module is configured to periodically broadcast its own storage resource usage information to other acceleration cards in the acceleration platform except the first acceleration card.

[0134] In an optional implementation, the broadcast module is further configured to:

[0135] Estimate the amount of memory required to execute the acceleration task. If the difference between the current free memory capacity and the memory capacity is greater than a preset threshold, broadcast the memory resource usage information to other accelerator cards in the acceleration platform except the first accelerator card.

[0136] If it is in an idle state, when its current free memory capacity is greater than a preset threshold, it broadcasts its own storage resource usage information to other acceleration cards in the acceleration platform except the first acceleration card.

[0137] In an optional embodiment, the method further includes:

[0138] The information update module is configured to receive storage resource usage information broadcast by other accelerator cards in the acceleration platform except the first accelerator card; when the memory free capacity of other accelerator cards is greater than a preset threshold, the other accelerator cards broadcast the storage resource usage information including the memory free information in the acceleration platform; and update the memory free information of other accelerator cards stored in the acceleration platform according to the storage resource usage information.

[0139] In an optional embodiment, the access module is configured to:

[0140] Use the memory in itself to directly access the controller and construct descriptor information based on the memory free information and the memory address occupied by the data to be transmitted in itself;

[0141] Constructing the data to be transmitted into a first message according to the descriptor information;

[0142] The first message is sent to the second accelerator card, so that the second accelerator card writes the data to be transmitted in the first message into its own memory.

[0143] In an optional implementation, if the second accelerator card determines that the data write address of the first message falls within the memory address range, the second accelerator card writes the data to be transmitted in the first message into its own memory.

[0144] In an optional embodiment, the method further includes:

[0145] The read / write judgment module is configured to receive the second message and convert the second message into a target format; if it is determined that the data write address of the second message falls within the register address range, the data in the second message is written into its own register; the data in the register is used to configure the table entries in the address mapping table, and / or, modify the broadcast period of the storage resource usage information.

[0146] In an optional implementation, the read / write determination module is further configured to:

[0147] If it is determined that the data write address of the second message falls within the memory address range, the data in the second message is written into the own memory.

[0148] In an optional embodiment, the method further includes:

[0149] The host response module is configured to receive a memory direct access request sent by an external host; query the address mapping table for a memory address corresponding to the virtual address in the memory direct access request; and complete the memory direct access request according to the memory address.

[0150] In an optional embodiment, the method further includes:

[0151] The sending module is configured to send the data to be transmitted to the external host if the data to be transmitted is the final result of the acceleration task.

[0152] In an optional embodiment, the method further includes:

[0153] The task receiving module is configured to receive acceleration tasks sent by an external host through a switch network.

[0154] In an optional embodiment, the downstream interface of the first accelerator card is a root device mode interface or a slave device mode interface that supports the serial computer expansion bus standard, and the bus address range of the downstream interface is not less than the bus address range of the downstream device to which it is connected.

[0155] In an optional implementation, the configuration process of the downstream interface includes:

[0156] receiving, via the upstream interface of the first accelerator card, a configuration message from the downstream interface;

[0157] Converting the configuration message into a second message that complies with the serial computer expansion bus standard;

[0158] configuring a bus address range of the downstream device according to the second message, and configuring identification information for the downstream device;

[0159] The bus address of the root device core or the slave device core of the downstream interface is configured according to the second message, so that the bus address range of the downstream interface is not smaller than the bus address range of the downstream device.

[0160] In an optional implementation, before performing bus address configuration on the slave device core of the downstream interface according to the second message, the second message is converted into a data format that matches the slave device mode.

[0161] In an optional embodiment, there are multiple downstream interfaces; accordingly, the method further includes:

[0162] Receive data packets sent by any downstream device connected to the downstream interface;

[0163] If the destination address of the data packet is not within the network routing window, determine whether the destination address of the data packet is within the forwarding routing window of other downstream interfaces;

[0164] If the destination address of the data message is within the forwarding routing window of the destination interface, the data message is forwarded to the downstream device connected to the destination interface through the destination interface; the destination interface is at least one of the other downstream interfaces.

[0165] Among them, for more specific working processes of each module and unit in the embodiments of the present application, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0166] It can be seen that the embodiment of the present application provides a DMA device between accelerator cards, which can realize direct DMA write operations between different accelerator cards at the hardware level, simplify the DMA process, and improve DMA efficiency.

[0167] The following introduces an accelerator card provided in an embodiment of the present application. The accelerator card described below can be referenced with other embodiments described in this document.

[0168] The embodiment of the present application discloses an accelerator card, comprising:

[0169] a memory configured to store a computer program;

[0170] The processor is configured to execute a computer program to implement the method disclosed in any of the above embodiments.

[0171] It can be seen that the accelerator card disclosed in the embodiments of the present application can implement the method of any of the above embodiments, and the accelerator card is provided with a memory direct access controller.

[0172] In an optional embodiment, the acceleration card is an FPGA acceleration card, an ASIC acceleration card, or a multi-core processor.

[0173] Optionally, an embodiment of the present application further provides an accelerator card. The accelerator card may be either accelerator card 1 as shown in FIG8 or accelerator card 2 as shown in FIG9 . FIG8 and FIG9 are both diagrams of accelerator card structures according to an exemplary embodiment, and the contents of the diagrams are not to be construed as limiting the scope of use of this application.

[0174] Figure 8 is a schematic diagram of the structure of an accelerator card 1 provided in an embodiment of the present application. The accelerator card 1 may include at least one processor, at least one memory, a power supply, a communication interface, input / output interfaces, and a communication bus. The memory is configured to store a computer program, which is loaded and executed by the processor to implement the relevant steps of the inter-accelerator card DMA disclosed in any of the aforementioned embodiments.

[0175] In the embodiment of the present application, the power supply is configured to provide operating voltage for each hardware device on the accelerator card 1; the communication interface can create an inter-accelerator card DMA channel between the accelerator card 1 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application and is not specifically limited here; the input and output interface is configured to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0176] In addition, the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include operating system, computer programs and data, etc. The storage method can be temporary storage or permanent storage.

[0177] The operating system is used to manage and control the hardware devices and computer programs on the accelerator card 1 to enable the processor to operate and process data in the memory. It can be Windows Server, NetWare, Unix, Linux, etc. In addition to computer programs capable of implementing the DMA method between accelerator cards disclosed in any of the aforementioned embodiments, computer programs can also include computer programs capable of performing other specific tasks. Data can include data such as application update information and application developer information.

[0178] FIG9 is a schematic diagram of the structure of an accelerator card 2 provided in an embodiment of the present application. The accelerator card 2 may include but is not limited to a smartphone, a tablet computer, a laptop computer, or a desktop computer.

[0179] Typically, the accelerator card 2 in the embodiment of the present application includes: a processor and a memory.

[0180] Among them, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA, and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor configured to process data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor configured to process data in a standby state. In an embodiment of the present application, the processor may be integrated with a GPU, and the GPU is configured to be responsible for rendering and drawing the content to be displayed on the display screen. In an embodiment of the present application, the processor may also include an AI (Artificial Intelligence) processor, which is configured to process computing operations related to machine learning.

[0181] The memory may include one or more computer non-volatile readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In an embodiment of the present application, the memory is at least configured to store the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps in the DMA method between accelerator cards performed by the accelerator card 2 side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to update information of the application.

[0182] In the embodiment of the present application, the acceleration card 2 may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0183] Those skilled in the art will appreciate that the structure shown in FIG. 9 does not limit the accelerator card 2 , and the accelerator card 2 may include more or fewer components than shown in the figure.

[0184] An acceleration platform provided in an embodiment of the present application is introduced below. The acceleration platform described below can be referenced with other embodiments described herein.

[0185] Please refer to Figure 10. An embodiment of the present application provides an acceleration platform, including: multiple acceleration cards provided by the aforementioned embodiments, different acceleration cards are connected through expansion devices, and DMA can be directly performed between different acceleration cards.

[0186] Referring to Figure 11 , each accelerator card in the acceleration platform provided in the embodiment of the present application is equipped with a memory direct access controller, a storage resource register, a selection module, and memory. The selection module determines other accelerator cards configured to process the data to be transmitted obtained by executing the acceleration task based on the free memory capacity and the accelerator card's functions. The memory free information of other accelerator cards is queried. When the free memory capacity of other accelerator cards exceeds a preset threshold, the other accelerator cards broadcast the memory free information in the acceleration platform. The storage resource register stores the memory free information of other accelerator cards. The memory direct access controller writes the data to be transmitted into the memory of other accelerator cards using memory direct access based on the free memory information. The accelerator card shown in Figure 11 can be used in conjunction with any of the accelerator cards mentioned in the above embodiments.

[0187] In an optional embodiment, the selection module also includes: an estimation unit and a selection unit; the estimation unit estimates the amount of data to be transmitted; the selection unit queries the data processing functions of other accelerator cards, and selects an accelerator card whose data processing function is consistent with the next processing function of the data to be transmitted and whose memory free capacity is not less than the data amount.

[0188] In an optional embodiment, each accelerator card is also provided with a resource sharing module, which estimates the amount of memory required to execute the acceleration task. If the difference between the memory free capacity and the memory amount is greater than a preset threshold, the memory storage resource usage information is broadcast to other accelerator cards; if it is in an idle state, when the memory free capacity is greater than the preset threshold, the memory storage resource usage information is broadcast to other accelerator cards; accordingly, the resource sharing module also receives the storage resource usage information including memory free information broadcast by other accelerator cards in the acceleration platform when the memory free capacity is greater than the preset threshold; and updates the memory free information of other accelerator cards stored in the storage resource register according to the storage resource usage information.

[0189] In an optional embodiment, any accelerator card uses its own memory to directly access the controller, constructs descriptor information based on memory free information and the memory address occupied by the data to be transmitted in its own memory; constructs the data to be transmitted into a first message according to the descriptor information; and sends the first message to other accelerator cards, so that the other accelerator cards write the data to be transmitted in the first message into their own memory after determining that the data write address of the first message falls within the memory address range.

[0190] In an optional embodiment, any accelerator card also receives a second message (a message for configuring a register) and converts the second message into a target format; if it is determined that the data write address of the second message falls within the register address range, the data in the second message is written into its own register; the data in the register is used to configure the table entries in the address mapping table, and / or the broadcast period of the storage resource usage information is modified. The register for the storage resource usage information is different from the register for the storage address mapping table. Optionally, after determining that the data write address of the second message falls within the memory address range, the accelerator card writes the data in the second message into its own memory.

[0191] In an optional embodiment, any accelerator card further receives a memory direct access request sent by an external host; queries the address mapping table for a memory address corresponding to the virtual address in the memory direct access request; and completes the memory direct access request according to the memory address.

[0192] In an optional implementation, if the data to be transmitted is the final result of the acceleration task, the acceleration card sends the data to be transmitted to the external host. The acceleration card receives the acceleration task sent by the external host through the switch network.

[0193] In an optional embodiment, the downstream interface of any accelerator card is a root device mode interface or a slave device mode interface that supports the serial computer expansion bus standard, and the bus address range of the downstream interface is not less than the bus address range of the downstream device to which it is connected. In one embodiment, the configuration process of the downstream interface includes: receiving a configuration message of the downstream interface through the upstream interface of the accelerator card; converting the configuration message into a second message that complies with the serial computer expansion bus standard; configuring the bus address range of the downstream device according to the second message, and configuring identification information for the downstream device; configuring the bus address of the root device core or the slave device core of the downstream interface according to the second message, so that the bus address range of the downstream interface is not less than the bus address range of the downstream device. In one embodiment, before configuring the bus address of the slave device core of the downstream interface according to the second message, the second message is converted into a data format that matches the slave device mode.

[0194] In an optional embodiment, there are multiple downstream interfaces; accordingly, it also includes: the acceleration card receives a data packet sent by a downstream device connected to any downstream interface; if the destination address of the data packet is not within the network routing window, then it is determined whether the destination address of the data packet is within the forwarding routing window of other downstream interfaces; if the destination address of the data packet is within the forwarding routing window of the destination interface, then the data packet is forwarded to the downstream device connected to the destination interface through the destination interface; the destination interface is at least one of the other downstream interfaces.

[0195] Referring to the process shown in FIG. 1 provided in the aforementioned embodiment, the corresponding acceleration platform can be seen in FIG. 12 . In FIG. 12 , the acceleration platform includes a first accelerator card and a second accelerator card. Both the first accelerator card and the second accelerator card are equipped with a memory direct access controller, a storage resource register, a selection module, and a memory. Both the first accelerator card and the second accelerator card can perform DMA data transmission according to the process shown in FIG. The first accelerator card and the second accelerator card are connected via an expansion device.

[0196] In an embodiment of the present application, the acceleration platform may include acceleration cards such as GPU, FPGA, NPU (Neural Processing Unit) and TPU (Tensor Processing Unit). GPU usually performs pure computing acceleration, and FPGA can complete application acceleration processing such as computing acceleration, network acceleration and storage acceleration.

[0197] As shown in Figure 6, the acceleration platform can communicate with host servers via a switched network. This allows the acceleration platform to add pooled high-performance computing capabilities to host servers in the data center without changing the existing data center topology. Furthermore, each accelerator card in the acceleration platform uses PCIE P2P DMA for inter-card DMA. Both GPU and FPGA accelerator cards can independently initiate DMA requests, achieving P2P DMA while decoupling from the CPU. Furthermore, each accelerator card can implement PCIE switch functionality on its own, eliminating the need for additional PCIE switches for connection.

[0198] A non-volatile readable storage medium provided in an embodiment of the present application is introduced below. The non-volatile readable storage medium described below can be referenced with other embodiments described herein.

[0199] A non-volatile readable storage medium is configured to store a computer program, wherein when executed by a processor, the computer program implements the DMA method between accelerator cards disclosed in the aforementioned embodiments. The non-volatile readable storage medium is a computer non-volatile readable storage medium, which, as a carrier for resource storage, can be a read-only memory, random access memory, magnetic disk, or optical disk, etc. The resources stored thereon include an operating system, computer program, and data, etc., and the storage method can be temporary or permanent.

[0200] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0201] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile readable storage medium known in the art.

[0202] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for accelerating direct memory access DMA between cards, characterized in that: The first accelerator card applied to a hostless acceleration platform includes: If the acceleration task is executed to obtain data to be transmitted, a second acceleration card in the acceleration platform configured to process the data to be transmitted is determined according to the free capacity of the memory and the function of the acceleration card; each acceleration card in the acceleration platform is provided with a memory direct access controller; querying the memory free information of the second accelerator card in the acceleration platform; when the memory free capacity of the second accelerator card is greater than a preset threshold, the second accelerator card broadcasts the memory free information in the acceleration platform; The memory direct access controller and the memory free information are used to write the data to be transmitted into the memory of the second acceleration card in a memory direct access manner.

2. The method according to claim 1, characterized in that The method of determining the second accelerator card in the acceleration platform configured to process the data to be transmitted according to the free capacity of the memory and the function of the accelerator card includes: Estimating the amount of data to be transmitted; A second accelerator card having a function of processing the data to be transmitted and having a memory free capacity not less than the data amount is determined.

3. The method according to claim 2, characterized in that The step of determining a second accelerator card that has a function of processing the data to be transmitted and has a memory free capacity that is not less than the data volume includes: querying data processing functions of other accelerator cards in the acceleration platform except the first accelerator card, and selecting a candidate accelerator card whose data processing function is consistent with the next step processing function of the data to be transmitted; At least one candidate accelerator card whose free memory capacity is not less than the data amount is determined as the second accelerator card.

4. The method according to claim 1, characterized in that: Also includes: Estimate the amount of memory required to execute the acceleration task, and if the difference between its own current free memory capacity and the memory amount is greater than a preset threshold, broadcast its own storage resource usage information to other acceleration cards in the acceleration platform except the first acceleration card.

5. The method according to claim 1, characterized in that Also includes: receiving storage resource usage information broadcasted by other acceleration cards in the acceleration platform except the first acceleration card; The other acceleration card broadcasts storage resource usage information including memory idle information in the acceleration platform when the memory idle capacity is greater than a preset threshold; Update the memory free information of other acceleration cards stored in itself according to the storage resource usage information.

6. The method according to claim 1, characterized in that The method of using the memory direct access controller in the accelerator card and the memory free information to write the data to be transmitted into the memory of the second accelerator card in a memory direct access manner includes: Using the memory in itself to directly access the controller, construct descriptor information according to the memory free information and the memory address occupied by the data to be transmitted in itself; Constructing the data to be transmitted into a first message according to the descriptor information; The first message is sent to the second acceleration card, so that the second acceleration card writes the data to be transmitted in the first message into its own memory.

7. The method according to claim 6, characterized in that The second accelerator card writes the data to be transmitted in the first message into its own memory, including: If the second accelerator card determines that the data write address of the first message falls within the memory address range, the second accelerator card writes the data to be transmitted in the first message into its own memory.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: receiving a second message and converting the second message into a target format; If it is determined that the data write address of the second message falls within the register address range, writing the data in the second message into its own register; The data in the register is used to configure the table entries in the address mapping table, and / or the broadcast period of the storage resource usage information is modified.

9. The method according to claim 8, characterized in that Also includes: If it is determined that the data write address of the second message falls within the memory address range, the data in the second message is written into the own memory.

10. The method according to claim 8, characterized in that Also includes: receiving a memory direct access request sent by an external host; Querying the memory address corresponding to the virtual address in the memory direct access request in the address mapping table; The memory direct access request is completed according to the memory address.

11. The method according to any one of claims 1 to 7, characterized in that: The downstream interface of the first acceleration card is a root device mode interface or a slave device mode interface that supports the serial computer expansion bus standard, and the bus address range of the downstream interface is not less than the bus address range of the downstream device to which it is connected.

12. The method according to claim 11, characterized in that The configuration process of the downstream interface includes: Receiving a configuration message of the downstream interface through the upstream interface of the first acceleration card; Converting the configuration message into a second message that complies with the serial computer expansion bus standard; configuring a bus address range of the downstream device according to the second message, and configuring identification information for the downstream device; According to the second message, the bus address of the root device core or the slave device core of the downstream interface is configured, so that the bus of the downstream interface The address range is not less than the bus address range of the downstream device.

13. The method according to claim 12, characterized in that Before performing bus address configuration on the slave device core of the downstream interface according to the second message, the second message is converted into a data format matching the slave device mode.

14. The method according to claim 11, characterized in that There are multiple downstream interfaces; Accordingly, it also includes: Receive data packets sent by any downstream device connected to the downstream interface; If the destination address of the data message is not within the network routing window, determining whether the destination address of the data message is within the forwarding routing window of other downstream interfaces; If the destination address of the data message is within the forwarding routing window of the destination interface, the data message is forwarded to the downstream device connected to the destination interface through the destination interface; the destination interface is at least one of the other downstream interfaces.

15. A DMA device for accelerating cards, characterized in that: The first accelerator card applied to a hostless acceleration platform includes: A determination module is configured to determine, if the acceleration task is executed to obtain data to be transmitted, a second acceleration card in the acceleration platform configured to process the data to be transmitted according to the free capacity of the memory and the function of the acceleration card; each acceleration card in the acceleration platform is provided with a memory direct access controller; A query module, configured to query the memory free information of the second accelerator card in itself; when the memory free capacity of the second accelerator card is greater than a preset threshold, broadcast the memory free information in the acceleration platform; The access module is configured to use the memory direct access controller in the module and the memory free information to write the data to be transmitted into the memory of the second acceleration card in a memory direct access manner.

16. An acceleration platform, characterized in that: include: Multiple accelerator cards, different accelerator cards are connected through expansion devices; Each accelerator card is provided with a memory direct access controller, a storage resource register, a selection module and a memory; The selection module determines other acceleration cards configured to process the data to be transmitted obtained by executing the acceleration task according to the free memory capacity and the function of the acceleration card; queries the free memory information of other acceleration cards; and when the free memory capacity of other acceleration cards is greater than a preset threshold, the free memory information is broadcasted in the acceleration platform; The storage resource register stores memory free information of other acceleration cards; The memory direct access controller writes the data to be transmitted into the memory of other acceleration cards in a memory direct access manner according to the memory free information.

17. The acceleration platform according to claim 16, characterized in that: The selection module further includes: an estimation unit and a selection unit; The estimating unit estimates the amount of data to be transmitted; The selection unit queries the data processing functions of other acceleration cards, and selects an acceleration card whose data processing function is consistent with the next processing function of the data to be transmitted and whose memory free capacity is not less than the data amount.

18. The acceleration platform according to claim 16, characterized in that: Each accelerator card is also provided with a resource sharing module, the resource sharing module estimates the amount of memory required to execute the acceleration task, and if the difference between the free memory capacity of the memory and the amount of memory is greater than a preset threshold, the storage resource usage information of the memory is broadcast to other accelerator cards; If it is in an idle state, when the free memory capacity of the memory is greater than a preset threshold, the storage resource usage information of the memory is broadcast to other accelerator cards; Correspondingly, the resource sharing module also receives storage resource usage information including memory idle information broadcasted in the acceleration platform by other acceleration cards when the memory idle capacity is greater than a preset threshold; and updates the memory idle information of other acceleration cards stored in the storage resource register according to the storage resource usage information.

19. An accelerator card, characterized in that: A memory direct access controller, a storage resource register, a selection module and a memory are provided; The selection module determines other acceleration cards configured to process the data to be transmitted obtained by executing the acceleration task according to the free memory capacity and the function of the acceleration card; and queries the free memory information of other acceleration cards; When the free memory capacity of other acceleration cards is greater than a preset threshold, the free memory information is broadcasted in the acceleration platform; The storage resource register stores memory free information of other acceleration cards; The memory direct access controller writes the data to be transmitted into the memory of other acceleration cards in a memory direct access manner according to the memory free information.

20. The accelerator card according to claim 19, characterized in that: The acceleration card is an FPGA acceleration card, an ASIC acceleration card or a multi-core processor.

21. A non-volatile readable storage medium, characterized in that: The device is configured to store a computer program, wherein the computer program implements the method according to any one of claims 1 to 14 when executed by a processor.

Images