Data transmission method, graphics processing unit and electronic device

Abstract

The present disclosure provides a data transmission method, a graphics processing unit, and an electronic device. The method comprises: obtaining a target task descriptor via a control unit; parsing the target task descriptor via the control unit to obtain a data source address of target data and a target graphics processing unit identifier; determining a target address corresponding to the target graphics processing unit identifier via the control unit, and sending the data source address and the target address to a first remote memory direct access module; obtaining target data from the data source address via the first remote memory direct access module, and transmitting the target data to the target address. The data transmission method disclosed herein can reduce the difficulty of interconnecting multiple graphics processing units using remote memory direct access technology.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a data transmission method, a graphics processing unit, and an electronic device. Background Art

[0002] With the development of artificial intelligence (AI), the scale of neural network models has become increasingly large, and the computing power required by AI technology has become increasingly demanding. A single graphics processing unit (GPU) can no longer meet the computing power requirements of neural network models. Existing technologies often interconnect multiple GPUs to complete neural network model training and inference tasks through multiple GPUs to meet the computing power requirements of neural network models.

[0003] Traditional interconnection technologies often use the PCIe bus to connect GPUs. This approach limits the communication latency and bandwidth between GPUs to the actual link capacity of the PCIe bus and requires significant CPU resources. Existing technologies can also interconnect multiple GPUs using Remote Direct Memory Access (RDMA) technology. This allows each GPU to bypass the CPU and directly access data in the memory of other GPUs, reducing CPU resource usage. However, this interconnection method often relies on a separate RDMA network card. The RDMA card itself is external and operates independently of the GPU. This requires additional programming based on the verbs API to control the RDMA card's behavior and enable collaboration between the RDMA card and the GPU. However, the verbs API itself is complex to program, making it difficult to interconnect multiple GPUs using RDMA technology. Summary of the Invention

[0004] The embodiments of the present disclosure provide a data transmission method, a graphics processing unit, and an electronic device, which can reduce the difficulty of interconnecting multiple graphics processing units using remote memory direct access technology.

[0005] According to one aspect of the present disclosure, a data transmission method is provided, which is applied to a first graphics processing unit. The first graphics processing unit includes a first data transmission module, a control unit, and a first memory module. The first data transmission module includes a first remote memory direct access module. The first graphics processing unit is communicatively connected to at least one second graphics processing unit via the first data transmission module. The method includes:

[0006] acquiring, by the control unit, a target task descriptor, wherein the target task descriptor is used to characterize the transmission of target data between the first graphics processing unit and a target graphics processing unit, wherein the target graphics processing unit is at least one of the second graphics processing units;

[0007] parsing the target task descriptor by the control unit to obtain a data source address of target data and a target graphics processing unit identifier, wherein the data source address is an address of the target data in the first memory module, and the target graphics processing unit identifier is a graphics processing unit identifier corresponding to the target graphics processing unit;

[0008] Determining, by the control unit, a target address corresponding to the target graphics processing unit identifier, and sending a data source address and the target address to the first remote memory direct access module;

[0009] The target data is obtained from the data source address through the first remote memory direct access module, and the target data is transmitted to the target address.

[0010] Optionally, before acquiring the target task descriptor by the control unit, the method further includes:

[0011] performing a topology self-discovery process between the first graphics processing unit and the second graphics processing unit by the control unit to determine each of the second graphics processing units communicatively connected to the first graphics processing unit;

[0012] A graphics processing unit identifier is allocated to the first graphics processing unit and each of the second graphics processing units, wherein the first graphics processing unit and each of the second graphics processing units correspond to different graphics processing unit identifiers.

[0013] Optionally, the first graphics processing unit and each of the second graphics processing units are respectively provided with a corresponding registration memory, and before determining the target address corresponding to the target graphics processing unit identifier, the method further includes:

[0014] performing unified addressing on the registration memory of the first graphics processing unit and the registration memories of each of the second graphics processing units to determine a physical address of each of the registration memories, wherein the physical addresses corresponding to each of the registration memories are different;

[0015] determining an address mapping table based on the physical addresses of the respective registration memories, the graphics processing unit identifiers of the first graphics processing units, and the graphics processing unit identifiers of the respective second graphics processing units;

[0016] The determining of the target address corresponding to the target graphics processing unit identifier includes:

[0017] A target address corresponding to the target GPU is determined based on the target GPU identifier and the address mapping table.

[0018] Optionally, the first memory module is provided with a data replication unit, the first remote memory direct access module is provided with a corresponding first registered memory, the first registered memory and the data replication unit are connected via a local bus, and the acquiring the target data from the data source address and transmitting the target data to the target address includes:

[0019] copying the target data from the first memory module based on the data source address and the data length of the target data by the data copying unit, and transmitting the copied data to the first registered memory through the local bus;

[0020] The first remote memory direct access module sends the starting address and data length of the target data in the first registered memory to the target graphics processing unit, so that the target graphics processing unit reads the target data from the first registered memory based on the starting address and the data length.

[0021] Optionally, the control unit is provided with a work queue. Before the data copying unit copies the target data from the first memory module based on the data source address and the data length of the target data, and transmits the copied data to the first registered memory through the local bus, the method further includes:

[0022] determining a data length of the target data based on the target task descriptor, and assigning a starting address to the target data, wherein the starting address is a first address occupied when the target data is written into the first registered memory;

[0023] Based on the start address, the data length and the target address, a target work queue element corresponding to the target task descriptor is written to the work queue.

[0024] Optionally, the method further comprises sending the starting address and data length of the target data in the first registered memory to the target graphics processing unit through the first remote memory direct access module:

[0025] In response to the data copying unit copying and transmitting the target data to the first registered memory, controlling the first remote memory direct access module to read the target work queue element corresponding to the target task descriptor from the work queue;

[0026] The first remote memory direct access module reads the target data from the first registered memory according to the start address and the data length based on the target work queue element, and transmits the target data to the target address.

[0027] Optionally, the first graphics processing unit further includes a command processor, a calculation module, and a signal synchronization module, and the first data transmission module is further provided with a task read / write interface. Before obtaining the target task descriptor issued by the application layer through the control unit, the method further includes:

[0028] Acquire, by the command processor, a first command packet sent by the application layer of the first graphics processing unit;

[0029] parsing the first command packet by the command processor, scheduling a first computing task obtained by parsing the first command packet to the computing module for execution, and writing the target task descriptor obtained by parsing the first command packet into a first task queue when the type of the first command packet is a target command packet type;

[0030] executing the first computing task through the computing module, and writing data obtained from executing the first computing task into the first memory module as the target data, and returning a synchronization completion signal to the signal synchronization module after completing executing the first computing task;

[0031] In response to writing the synchronization completion signal, the signal synchronization module controls the task read / write interface to read the target task descriptor from the first task queue and sends the target task descriptor to the control unit.

[0032] Optionally, the control unit is further provided with a completion queue, the completion queue including a target completion queue element corresponding to the target data, and after the target data is transmitted to the target address, the method further includes:

[0033] receiving, through the first remote memory direct access module, a first response signal corresponding to the target data and returned by the target graphics processing unit, wherein the first response signal is used to indicate that the target graphics processing unit has obtained the target data from the first graphics processing unit;

[0034] The first remote memory direct access module generates a target completion queue element corresponding to the target data in response to the first response signal, and writes the target completion queue element into the completion queue;

[0035] The command processor returns a second response signal to the application layer in response to the target completion queue element in the completion queue, so that the application layer determines that the first command packet has been executed.

[0036] According to one aspect of the present disclosure, there is provided a graphics processing unit, comprising:

[0037] a first data transmission module, a control unit, and a first memory module, wherein the first data transmission module includes a first remote memory direct access module, and the graphics processing unit is communicatively connected to at least one second graphics processing unit via the first data transmission module;

[0038] The control unit is used to obtain a target task descriptor, wherein the target task descriptor is used to represent the transmission of target data between the graphics processing unit and a target graphics processing unit, and the target graphics processing unit is a second graphics processing unit used to receive the target data;

[0039] The control unit is further configured to parse the target task descriptor to obtain a data source address of the target data and a target graphics processing unit identifier, wherein the data source address is an address of the target data in the first memory module, and the target graphics processing unit identifier is a graphics processing unit identifier corresponding to the target graphics processing unit;

[0040] The control unit is further configured to determine a target address corresponding to the target graphics processing unit identifier, and send the target address to the first remote memory direct access module;

[0041] The first remote memory direct access module is used to obtain the target data from the data source address and transmit the target data to the target address.

[0042] According to an aspect of the present disclosure, an electronic device is provided, comprising the graphics processing unit as described above.

[0043] The data transmission method, graphics processing unit, and electronic device proposed in the present disclosure are implemented by integrating a first data transmission module including a first remote memory direct access module into the first graphics processing unit, and integrating a control unit for driving the first data transmission module into the first graphics processing unit. Subsequently, a task descriptor is obtained by the control unit, and a target graphics processing unit identifier for indicating a target graphics processing unit is extracted from the task descriptor. Based on the target graphics processing unit identifier, a corresponding target address is determined and sent to the first remote memory direct access module. After obtaining the data source address and target address sent by the control unit, the first remote memory direct access module can transmit the target data to the corresponding target graphics processing unit through remote memory direct access. During the entire data transmission process, the behavior of the first remote memory direct access module is controlled by the control unit, and the control unit only needs to obtain from the outside the task descriptor used to indicate the source address of the target data and the target graphics processing unit identifier. The control unit can automatically parse the task descriptor and determine the corresponding data source address and target address to control the first remote memory direct access module to complete subsequent data transmission tasks. When programming, the user only needs to specify the source address of the target data to be transmitted and the target graphics processing unit identifier, thereby effectively simplifying the user's programming model and reducing the user's programming complexity, thereby reducing the difficulty of interconnecting multiple graphics processing units using remote memory direct access technology.

[0044] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present disclosure. The purposes and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation to the technical solution of the present disclosure.

[0046] Figure 1 is a diagram of the system architecture of a graphics processing unit to which an embodiment of the present disclosure is applied;

[0047] Figure 2 is a main flow chart of a data transmission method according to an embodiment of the present disclosure;

[0048] Figure 3 This is a flowchart of topology self-discovery according to an embodiment of the present disclosure;

[0049] Figure 4 is a flowchart of unified addressing according to an embodiment of the present disclosure;

[0050] Figure 5 is a detailed architectural diagram of a first graphics processing unit according to an embodiment of the present disclosure;

[0051] Figure 6 yes Figure 2 A sub-flowchart of step S204;

[0052] Figure 7 is a flowchart of creating a work queue element according to an embodiment of the present disclosure;

[0053] Figure 8 yes Figure 6 A sub-flowchart of step S601;

[0054] Figure 9 This is a flowchart of an embodiment of the present disclosure, which is used to issue a target task descriptor after calculating target data;

[0055] Figure 10 This is a flowchart of an embodiment of the present disclosure, which notifies the application layer that the target task descriptor has been executed. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not intended to limit the present disclosure.

[0057] Before further explaining the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are explained. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations:

[0058] Remote Direct Memory Access (RDMA): A network communication technology. This technology is often used to interconnect multiple devices. The multiple devices interconnected using RDMA technology are often slave devices in a master-slave system. The multiple slave devices are controlled by the master device to collaboratively complete the same data processing task. When traditional methods such as PCIe are used for interconnection, the data transmission process between slave devices is controlled by the master device. This interconnection method will occupy more resources of the master device; while RDMA allows slave devices to bypass the master device and directly access and operate the memory of other slave devices without the CPU having to exercise excessive control over the data transmission process, which can reduce the occupation of master device resources. In addition, compared to using the PCIe bus, using RDMA to interconnect graphics processing units often has lower latency and higher bandwidth, which can increase the data transmission rate between devices.

[0059] In related technologies, the implementation of RDMA technology often relies on independent external RDMA network cards. In this interconnection configuration, the main CPU (the CPU that controls multiple GPUs to collaboratively complete computing tasks) only needs to issue a data transfer task to the GPUs and RDMA network cards. There is no need to send additional control signals to the GPUs and RDMA network cards during the actual data transfer process. The GPUs and RDMA network cards can, to a certain extent, bypass the main CPU's control and directly access the memory of other GPUs. Because the RDMA network cards themselves are external to the GPUs and not directly controlled by the GPU, users need to perform additional programming on the main CPU using the verbs API for each RDMA network card to regulate its behavior during data transfers. The verbs API itself involves numerous concepts such as send queues, receive queues, and completion queues. Users need to manage the creation, use, and destruction of these concepts during programming to control the operation of the RDMA network cards. This makes programming based on the verbs API more complex and makes interconnecting GPUs using remote memory direct access (RDMA) more challenging.

[0060] Based on this, the embodiments of the present disclosure provide a data transmission method, a graphics processing unit, and an electronic device, which can reduce the difficulty of interconnecting graphics processing units using remote memory direct access technology.

[0061] Description of the system architecture used in the embodiments of the present disclosure

[0062] Figure 1 This is a system architecture diagram applied to the data transmission method of an embodiment of the present disclosure, which includes a first graphics processing unit and at least one second graphics processing unit. The first graphics processing unit includes a first data transmission module 110, a control unit 120 and a first memory module 130, wherein the first data transmission module 110 includes a first remote memory direct access module 111, the first data transmission module 110, the control unit 120 and the first memory module 130 are connected through a local bus, and the first graphics processing unit is connected to at least one second graphics processing unit through the first data transmission module 110.

[0063] The control unit 120 can be a control module written with firmware for driving and controlling the first remote memory direct access module. The control unit can receive a target task descriptor sent by an upper-layer application. The target task descriptor is used to represent sending target data to a target graphics processing unit. The target graphics processing unit is a second graphics processing unit for receiving target data.

[0064] It can be understood that the second graphics processing unit and the first graphics processing unit themselves can have graphics processing units with the same hardware structure. The first graphics processing unit and the second graphics processing unit are interchangeable. When a graphics processing unit acts as the initiator of a data transmission request, that is, as the sender of data, it is the first graphics processing unit; when a graphics processing unit acts as the receiver of data, it is the second graphics processing unit.

[0065] The control unit 120 is further configured to parse the target task descriptor to obtain a data source address of the target data and a target graphics processing unit identifier, where the data source address is an address of the target data in the first memory module 130 and the target graphics processing unit identifier is a graphics processing unit identifier corresponding to the target graphics processing unit.

[0066] The control unit 120 is further configured to determine a target address corresponding to the target graphics processing unit identifier and send the target address to the first remote memory direct access module 130;

[0067] The first remote memory direct access module 130 is configured to obtain target data from a data source address and transmit the target data to a target address.

[0068] Overall implementation of the data transmission method according to the embodiment of the present disclosure

[0069] The present disclosure provides a data transmission method for use in Figure 1 The first graphics processing unit shown, referring to Figure 2 , data transmission methods include:

[0070] Step S201, obtaining a target task descriptor through a control unit;

[0071] Step S202: The control unit parses the target task descriptor to obtain the data source address of the target data and the target graphics processing unit identifier;

[0072] Step S203: determining a target address corresponding to the target graphics processing unit identifier through the control unit, and sending the data source address and the target address to the first remote memory direct access module;

[0073] Step S204: Obtain target data from the data source address through the first remote memory direct access module, and transmit the target data to the target address.

[0074] In step S201, the first data transmission module can be a module integrated into the first graphics processing unit and dedicated to data transmission, for example, it can be an IO Die dedicated to communication between the graphics processing unit and the outside world; the first data transmission module is provided with a first remote memory direct access module, and the first remote memory direct access module encapsulates a communication protocol for performing RDMA communication. The first remote memory direct access module can be one of a RoCEv2 engine, an infiniBand engine, or an iWARP engine, that is, the first remote memory direct access module can operate based on one of the infiniBand protocol, the RoCE protocol, and the iWARP protocol, which is not limited in this embodiment. The control unit can be a dedicated processor for controlling the first data transmission module, or a general-purpose processor for controlling various hardware modules in the first graphics processing unit. The control unit is embedded with firmware required for controlling the first remote memory direct access module to complete data transmission tasks. The control unit controls the first remote memory direct access module to perform corresponding data transmission tasks by running the firmware. For example, the control unit runs the firmware to send a link establishment request to each second graphics processing unit through Ethernet when the first graphics processing unit and the second graphics processing unit are interconnected, thereby establishing a remote memory direct access connection between the first graphics processing unit and each second graphics processing unit; or the control unit runs the firmware to create and maintain queue pairs based on the obtained target task descriptor to manage the behavior of the first remote memory direct access module.

[0075] The target task descriptor is used to indicate that the first GPU needs to send target data to the target GPU. It can be generated based on the data processing task issued by the GPU's application layer and sent to the control unit. It should be noted that the application layer is a logical layer, not an actual hardware layer. It can be a logical layer in the GPU used to process upper-layer applications of the CPU. The target GPU is the second GPU that receives the target data transmitted by the first GPU.

[0076] In one possible embodiment, the first remote memory direct access module can be used to perform multiple different types of data transfer operations, and the target task descriptor can also carry the type of data transfer operation that the first remote memory direct access module needs to perform, such as Read, Write, Send / Receive operations, etc. These operation types indicate the manner in which the target data is operated between the first graphics processing unit and the target graphics processing unit.

[0077] The target data is data that needs to be transferred from the first graphics processing unit to the target graphics processing unit. It can be understood that the first graphics processing unit and each second graphics processing unit are used to collaboratively complete a certain computing task, for example, to complete the computing tasks corresponding to each operator in a neural network model, and the target data is a piece of intermediate data generated by the first graphics processing unit in the process but needs to be transferred to other graphics processing units for subsequent processing.

[0078] Specifically, when using a cluster of interconnected graphics processing units to collaboratively complete neural network models or other large-scale computing tasks, the data to be processed is often divided into multiple sub-data and then distributed to multiple graphics processing units for parallel computing in a pipeline parallel, data parallel, or tensor parallel manner. During the parallel computing process, after executing certain specific computing nodes, specific intermediate data needs to be communicated between different graphics processing units so that the graphics processing units can perform subsequent processing based on the data obtained after communication. For example, in the data parallel computing mode, the input of the neural network model will be decomposed into multiple sub-inputs and distributed to different graphics processing units. Each graphics processing unit is only responsible for performing the relevant processing of the neural network model on a part of the data, while the computing tasks corresponding to certain operators in the neural network model need to be calculated based on the intermediate data corresponding to the complete input. For example, when performing attention calculation, it is necessary to perform attention calculation based on the key matrix and value matrix corresponding to the complete input. At this time, it is necessary to communicate the key matrix block and value matrix block corresponding to each sub-input between multiple graphics processing units so that each graphics processing unit can obtain the key matrix and value matrix corresponding to the complete input. At this time, when it is detected that the target data to be transmitted is ready, a corresponding target task descriptor is sent to the control unit to instruct the first graphics processing unit to send the target data to the target graphics processing unit.

[0079] In one possible embodiment, the first graphics processing unit may need to send the same data to multiple graphics processing units respectively. For example, when the Allgather collective communication primitive is used to perform collective communication between multiple graphics processing units, each graphics processing unit needs to send the corresponding target data to all other graphics processing units. When the broadcast collective communication primitive is used to perform collective communication, the graphics processing unit that initiates the communication operation needs to broadcast the corresponding target data to each graphics processing unit interconnected with the graphics processing unit. Based on this, in this embodiment, the target graphics processing unit may include multiple second graphics processing units, and accordingly, the target task descriptor may also include multiple graphics processing unit identifiers corresponding to the multiple target graphics processing units. At this time, when the target data is subsequently transmitted, the same operation needs to be performed on each target graphics processing unit identifier to send the target data to each target graphics processing unit.

[0080] In step S202, the data source address is the physical address of the memory area in the first memory module used to store the target data; the target GPU identifier is the GPU identifier corresponding to the target GPU. Specifically, the target task descriptor indicates the target data to be transmitted and the second GPU to receive the target data. The control unit can determine the corresponding data address based on the target data, for example, by obtaining the address pointer of the target data to determine the corresponding data source address.

[0081] In another embodiment, the target task descriptor itself can carry the data source address corresponding to the target data and the target graphics processing unit identifier corresponding to the target graphics processing unit for receiving the target data. The data source address and the target graphics processing unit identifier serve as a specific field in the target task descriptor. At this time, the control unit parses the target task descriptor, that is, extracts the specific field from the target task descriptor to determine the data source address and the target graphics processing unit identifier.

[0082] In step S203, the target address is the physical address of the memory area of ​​the target GPU used to receive and store the target data. This target address may be the physical address corresponding to the registered memory of the target GPU. It is understood that the first remote memory direct access module, as the underlying hardware, can only understand hardware language, meaning that the first remote memory direct access module can only operate directly based on memory addresses. Therefore, in this embodiment, after parsing the target GPU identifier from the target task descriptor, the control unit must also convert the target GPU identifier into a corresponding memory address, enabling the first remote memory access module to perform data transmission based on the target address.

[0083] Specifically, when using remote memory direct access (RAMD) technology to interconnect GPUs, each GPU registers a specific memory area within its own memory as a memory area accessible and interconnected by other GPUs, known as a registered memory. The address corresponding to this registered memory is then broadcast to all other GPUs connected to it. Thus, after determining the target GPU ID, the address of the registered memory in the target GPU can be determined based on the target GPU ID.

[0084] Specifically, in one embodiment, when establishing a connection, the first GPU and each second GPU can register a memory space within their respective memories as registered memory and broadcast it to the other interconnected GPUs. In this case, the first GPU can store a pre-created address mapping table that represents the mapping relationship between the GPU identifiers corresponding to each second GPU and the corresponding target addresses. After determining the target GPU identifier, the target address corresponding to the target GPU identifier can be determined based on the address mapping table.

[0085] In another possible embodiment, the control unit may also send a signal to the corresponding target graphics processing unit based on the target graphics processing unit identifier to notify the target graphics processing unit that data needs to be transmitted with the first graphics processing unit in the future, so that the target graphics processing unit responds to the signal, registers a memory area in its own memory as a memory for receiving and storing target data, and returns the memory address corresponding to the registered memory area as the target address to the first graphics processing unit. Thus, the control unit can determine the corresponding target address based on the target graphics processing unit identifier.

[0086] It is understood that in this embodiment, the first GPU is simultaneously interconnected with multiple second GPUs. If the memory addresses corresponding to the memory areas registered in the memories of the multiple second GPUs overlap, that is, the same memory address corresponds to multiple GPU identifiers, then the first remote memory direct access module may easily transfer the target data to the wrong second GPU when performing data transmission based on the target address. Therefore, in this embodiment, the target addresses obtained based on different target GPU identifiers do not overlap.

[0087] In step S204, after determining the data source address and the target address, the first remote memory direct access module can perform data transmission operations based on the data source address and the target address. Specifically, the target address indicates the memory address of a memory area in the memory of the target graphics processing unit that can be operated by the first remote memory direct access module. Referring to the relevant description and control of the above steps, the target address is the address corresponding to the memory area in the target graphics processing unit that can be directly accessed and operated by other graphics processing units, that is, the first remote memory direct access module can operate on the memory area corresponding to the target address. Based on this, in one embodiment, the first remote memory direct access module can directly write the target data stored at the data source address to the memory area corresponding to the target address through a Write operation.

[0088] Furthermore, in another possible embodiment, the first remote memory direct access module may also transmit the target data to the target graphics processing unit through a Send / Recv operation.

[0089] In the embodiment disclosed in steps S201 to S204, the application layer only needs to pass a target task descriptor containing the data source address to be sent and the target graphics processing unit identifier corresponding to the graphics processing unit for receiving the data to be sent to the control unit. The control unit integrated in the graphics processing unit can automatically control the first data transmission module to complete the corresponding communication task according to the task descriptor. That is to say, when programming, the user only needs to specify the data source address corresponding to the target data to be sent and the graphics processing unit identifier corresponding to the target graphics processing unit for receiving the target data. This effectively simplifies the user's programming model and reduces the difficulty for the user to interconnect the graphics processing units using the remote memory access module.

[0090] Detailed implementation of establishing a connection between a first graphics processing unit and a second graphics processing unit

[0091] In one embodiment, referring to Figure 3 Before step S201, the method further includes:

[0092] Step S301: performing a topology self-discovery process between a first graphics processing unit and a second graphics processing unit by a control unit to determine each second graphics processing unit that is communicatively connected to the first graphics processing unit;

[0093] Step S302: assigning a graphics processing unit identifier to the first graphics processing unit and each second graphics processing unit, wherein the first graphics processing unit and each second graphics processing unit correspond to different graphics processing unit identifiers.

[0094] In step S301, topology self-discovery is performed to enable the first GPU and each second GPU to perceive the topology of the GPU cluster formed by themselves and the other GPUs. It will be appreciated that when interconnecting multiple GPUs, these GPUs can be connected via a dedicated network to form a GPU cluster, with each GPU acting as a node in the cluster. In response to the first GPU being connected to the corresponding dedicated network, the control unit in the first GPU sends an initialization signal to the dedicated network. This initialization signal is broadcast throughout the dedicated network, allowing each second GPU to perceive the presence of the first GPU in the GPU cluster. Similarly, as described in the above embodiments, each second GPU can have the same hardware structure as the first GPU, and each second GPU also integrates a corresponding second remote memory direct access module and control unit. The control unit of each second GPU also broadcasts a corresponding initialization signal to the dedicated network, allowing the other GPUs in the GPU cluster to confirm that the GPU cluster includes the corresponding second GPU. Thus, the first graphics processing unit can determine each second graphics processing unit interconnected with itself, and determine the network topology of the cluster formed by the first graphics processing unit and each second graphics processing unit.

[0095] In step S302, the GPU identifier is a global identifier. The GPU identifier corresponding to each GPU is unique, and the GPU identifiers corresponding to different GPUs are different. In this way, a unique target GPU can be found based on each target GPU.

[0096] Specifically, after the first graphics processing unit and each second graphics processing unit broadcasts the corresponding initialization signal to the dedicated network, the control unit of the first graphics processing unit and the control units of each second graphics processing unit will negotiate according to preset rules to assign corresponding graphics processing unit identifiers to the first graphics processing unit and each second graphics processing unit respectively.

[0097] Specifically, referring to the relevant description of the above step S201, the control unit runs firmware corresponding to the first remote memory direct access module, and the firmware may include a driver firmware for driving and initializing the first remote memory direct access module. When the first graphics processing unit and each second graphics processing unit are communicatively connected, the control unit of the first graphics processing unit and the control unit of each second graphics processing unit will run the driver firmware to automatically execute the above steps S301 and S302 to determine the graphics processing unit identifiers corresponding to the first graphics processing unit and each second graphics processing unit.

[0098] In the embodiment disclosed in steps S301 and S302, topology self-discovery is performed between a first GPU and multiple second GPUs to determine the respective second GPUs connected to the first GPU, thereby determining the network topology of the corresponding GPU cluster. Based on this, corresponding GPU identifiers are assigned to the first GPU and each second GPU as corresponding global identifiers. Thus, when programming, the user only needs to indicate the data source address of the target data and the GPU identifier corresponding to the target GPU through the corresponding code statements, thereby further simplifying the user's programming model and reducing the difficulty of interconnecting GPUs through remote memory direct access technology.

[0099] It is understandable that after topology self-discovery is performed between the first GPU and each second GPU and the GPU identifiers corresponding to each GPU are determined, a remote memory direct access connection is automatically established between the first GPU and each second GPU. Specifically, a three-way handshake can be performed between the remote memory direct access modules in the first GPU and each second GPU to establish a remote memory direct access connection between them. This can be referred to the RDMA network card link establishment process in related technologies and is not limited in this embodiment.

[0100] In one embodiment, the first graphics processing unit and each second graphics processing unit are provided with corresponding registration memory, referring to Figure 4 Before step S203, the method further includes:

[0101] Step S401: uniformly addressing the register memory of the first graphics processing unit and the register memories of each second graphics processing unit to determine the physical address of each register memory, wherein the physical address corresponding to each register memory is different;

[0102] Step S402, determining an address mapping table based on the physical addresses of the respective registered memories, the graphics processing unit identifier of the first graphics processing unit, and the graphics processing unit identifiers of the respective second graphics processing units;

[0103] Step S203 includes:

[0104] Step S403: determining a target address corresponding to the target GPU based on the target GPU identifier and the address mapping table.

[0105] In step S401, the registered memory can be a specific memory space within the first memory module of the corresponding GPU, or alternatively, a memory space independent of the first memory module. This registered memory can be directly accessed and operated by other interconnected GPUs. Specifically, in this embodiment, each GPU can designate a memory space within its own memory as the corresponding registered memory when establishing a connection with another GPU.

[0106] Unified addressing means treating the registered memory of the first graphics processing unit and the registered memories of each second graphics processing unit as a whole, and using this to physically address each registered memory. When addressing each registered memory, for the memory as a whole, the physical address corresponding to each memory area after addressing is different, so that multiple registered memories can correspond to different physical addresses, avoiding the same address corresponding to the memory areas of multiple graphics processing units. After the same addressing is performed on multiple registered memories, the physical addresses corresponding to the multiple registered memories can be continuous, so that it is more convenient to subsequently address based on the graphics processing unit identifier and determine the target address. Of course, the physical addresses of multiple registered memories can also be non-continuous. Specifically, whether the physical addresses corresponding to each registered memory are continuous needs to be determined according to the addressing method.

[0107] In step S402, the address mapping table is used to store the mapping relationship between the GPU identifier corresponding to each GPU and the physical address corresponding to the GPU's registered memory. Specifically, after uniformly addressing multiple registered memories to determine the physical address corresponding to each registered memory, a corresponding address mapping table is created based on the physical address corresponding to each registered memory and the GPU identifier of the GPU corresponding to each registered memory to determine the physical address corresponding to each GPU identifier.

[0108] In step S403, after the target GPU ID is determined, a search is performed in the address mapping table based on the target GPU ID to determine the physical address corresponding to the registered memory of the target GPU, and the physical address is used as the target address.

[0109] In the embodiment disclosed in steps S401 to S403, the registered memory of the first GPU and the registered memories of each second GPU are uniformly addressed so that each registered memory corresponds to a non-overlapping physical address. Then, an address mapping table is created based on the physical addresses corresponding to the registered memories of each GPU and the GPU identifiers corresponding to the GPUs. In this way, after determining the target GPU identifier corresponding to the target GPU, the target address corresponding to the registered memory of the target GPU can be found based on the address mapping table.

[0110] Detailed implementation of step S204

[0111] In one embodiment, referring to Figure 5 , Figure 5 This is a detailed architecture diagram of the first graphics processing unit of the present disclosure. The first memory module is provided with a data replication unit, the first remote memory direct access module is provided with a corresponding first registered memory, and the first registered memory and the data replication unit are connected via a local bus. Figure 6 Step S204 includes:

[0112] Step S601: copying target data from a first memory module based on a data source address and a data length of the target data by a data copying unit, and transmitting the copied data to a first registered memory via a local bus;

[0113] Step S602: the first remote memory direct access module sends the starting address and data length of the target data in the first registered memory to the target graphics processing unit, so that the target graphics processing unit reads the target data from the first registered memory based on the starting address and data length.

[0114] In this embodiment, the local bus can be a UCIe bus or an AXI bus. It is understandable that in the related art, since the RDMA network card is external, in this case, the graphics processing unit and the RDMA network card can often only be connected via the PCIe bus. In this embodiment, the first remote memory direct access module is integrated into the graphics processing unit. Compared with the method of connecting the first graphics processing unit and the external RDMA network card via the PCIe bus in the related art, using the UCIe bus or the AXI bus to connect the first remote memory direct access module and the first memory module can provide higher transmission bandwidth.

[0115] In step S601, the data copying unit is a dedicated hardware in the graphics processing unit for performing data copying operations. When the data copying unit detects that the target data is ready, it addresses the first memory module based on the data source address, and copies the data in a continuous memory address range starting from the data source address based on the data length of the target data, and transfers the copied data to the first registered memory via the local bus. Specifically, after receiving the target task descriptor, the control unit can pre-assign a starting address to the target data in the first registered memory. Thereafter, when the data copying unit copies and transfers the target data to the first registered memory, the target data can be written to a continuous memory space starting with the starting address.

[0116] In step S602, after the target data is copied to the first registered memory, as can be seen from the description of step S601, the control unit pre-assigns a starting address in the first registered memory to the target data. When the target data needs to be transmitted to the target GPU via the first remote memory direct access module, the first remote memory direct access module can obtain the starting address and corresponding data length pre-assigned by the control unit to the target data. The first remote memory access module can then send this starting address and data length to the target GPU, allowing the target GPU to directly read the target data from the first registered memory and write it to the target GPU's local memory through a Read operation based on the starting address and data length.

[0117] In the embodiment disclosed in steps S601 and S602, the data copying unit copies the target data from the first memory module to the first registered memory based on the data source address and data length. The first remote memory direct access module then directly transmits the starting address and data length to the target GPU, allowing the target GPU to directly perform a read operation on the first registered memory based on the starting address and data length to read the target data into the target GPU's memory. This allows the target GPU to directly read the target data from the first GPU's memory without requiring an additional processor to control the data transfer process between the first and target GPUs, thereby reducing processor resource usage.

[0118] In one embodiment, referring to Figure 5 , the control unit is provided with a work queue, refer to Figure 7 Before step S601, the method further includes:

[0119] Step S701, determining the data length of target data based on the target task descriptor, and assigning a starting address to the target data;

[0120] Step S702: Based on the start address, data length and target address, write the target work queue element corresponding to the target task descriptor into the work queue.

[0121] In this embodiment, a work queue (WQ) is a queue used to indicate data transfer tasks completed by the first remote memory access module. The work queue includes multiple work queue elements written by the control unit. Each work queue element indicates a data transfer operation to be performed by the first remote memory direct access module. The work queue can be a first-in-first-out (FIFO) queue. The first registered memory is a registered memory in the first graphics processing unit. The first registered memory allows each second graphics processing unit to directly access and operate on the first registered memory.

[0122] In step S701, the data length refers to the length of the memory area required to be occupied when the target data is written into the memory, and the starting address is the first address when the target data is written into the first registered memory. Specifically, the target task descriptor indicates the target data to be transmitted. The target data is often a tensor with a specific shape. Based on the shape of the tensor, the number of elements contained in the target data can be determined, thereby determining the size of the memory space required to be occupied by the target data and obtaining the corresponding data length. It can be understood that when the target data is written into the first registered memory, it is often written to a continuous memory address in the first registered memory. Based on this, a starting address in the first registered memory is assigned to the target data. In this way, based on the starting address and the data length, the address corresponding to the memory area occupied by the target data in the first registered memory can be determined.

[0123] In step S702, after determining the starting address and data length, a corresponding target work queue element is generated based on the starting address and data length and written into the work queue. This allows the first remote memory access module to subsequently read the target work queue element from the work queue to determine the target address to which the target data moved to the first registered memory should be transferred.

[0124] In one possible embodiment, the target task descriptor may also be used to indicate the type of operation to be performed when transmitting target data between the first GPU and the target GPU, such as Read, Write, Send / receive, and other operation types in remote memory direct access technology. After parsing the target task descriptor, the control unit may further determine the corresponding operation type and write the operation type into the work queue as part of the target work queue element. In this way, when the first remote memory direct access module reads the target work queue element from the work queue, the target data may be transmitted between the first GPU and the target GPU in accordance with the corresponding operation method based on the corresponding operation type.

[0125] It should be noted that, in this embodiment, the assignment of a starting address to the target data is only used to determine in advance the address when the target data is moved to the first registered memory, so as to create a corresponding target work queue element, and does not mean that the target data is immediately moved to the first registered memory. The steps of assigning a starting address and creating a target work queue element can be completed before the calculation module generates the target data. Of course, when the target data needs to be moved to the first registered memory later, it is still moved based on the starting address. That is, in this embodiment, the data length of the target data can be determined in advance based on the target task descriptor, and the corresponding starting address can be assigned to the target data, and then the corresponding target work queue element can be written to the work queue; afterwards, when the calculation module completes the corresponding calculation task and generates the target data, the target data is moved to the first registered memory based on the starting address.

[0126] In the embodiment disclosed in steps S701 to S702, after obtaining the target task descriptor, the data length of the target data to be transmitted is determined based on the target task descriptor, and a starting address is assigned to the target data as the first address when the target data is subsequently moved to the first registered memory. Then, based on the starting address, data length, and target address, a corresponding target work queue element is generated and written into the work queue. In this way, the first remote memory direct access module only needs to read the target work queue element from the work queue, and can transmit the target data based on the starting address, data length, and target address in the target work queue element.

[0127] In one embodiment, referring to Figure 5 , the first remote memory direct access module is provided with a doorbell module, referring to Figure 8 Step S601 includes:

[0128] Step S801, in response to the data copying unit copying and transmitting the target data to the first registered memory, controlling the first remote memory direct access module to read a target work queue element corresponding to the target task descriptor from the work queue;

[0129] Step S802: The first remote memory direct access module determines the starting address and data length of the target data to be transmitted in the first registered memory based on the target work queue element, and sends the starting address and data length to the target graphics processing unit.

[0130] In step S801, the details of the data copying unit can refer to the above steps and are not repeated here. Specifically, after the target data is copied and transmitted to the first registered memory by the data copying unit, this indicates that the data to be transmitted to the target graphics processing unit is ready, and the work queue has written a target work queue element consisting of the starting address of the target data in the first registered memory, the data length, and the target address for receiving the target data. Based on this, after detecting that the data copying unit has copied and written the target data into the first registered memory, the first remote memory direct access module is controlled to read the target work queue element from the work queue to determine the starting address, data length, and target address corresponding to the target data.

[0131] Specifically, after copying and transmitting the target data to the first registered memory, the data copying unit may send an enable signal to the control unit to notify the control unit that the target data has been copied to the first registered memory.

[0132] In addition, in another embodiment, after the data copying unit copies the target data to the first registered memory, the target task descriptor can be sent to the control unit. In this way, when the control unit receives the target task descriptor, it can determine that the target data has been written to the first registered memory. This will be explained in detail later and will not be repeated here.

[0133] After determining that the target data has been written to the first registered memory, the control unit can send an enable signal to the first remote memory direct access module, causing the first remote memory direct access module to read the target workqueue element from the workqueue. Referring to the description of step S701 above, it can be seen that the workqueue can be a first-in, first-out queue. It is not difficult to understand that the first graphics processing unit itself processes the commands issued by the application layer in an orderly manner. Therefore, the workqueue elements corresponding to the data transmission tasks obtained based on the orderly commands are also written to the workqueue in an orderly manner. The first graphics processing unit also executes the computing tasks obtained by parsing these commands in an orderly manner. That is, the order of the workqueue elements in the workqueue is consistent with the order in which the data to be transmitted corresponding to the target queue elements is generated. Referring to the above embodiment, it can be seen that when the data to be transmitted is calculated and copied to the first registered memory, the first remote memory direct access module reads a workqueue element from the workqueue. At this time, the first remote memory direct access module directly reads the first unexecuted workqueue element in the workqueue to obtain the target workqueue element corresponding to the target data.

[0134] In one possible embodiment, in the first graphics processing unit, each workqueue element is stored in memory space corresponding to a segment of continuous memory addresses. Ordered workqueue elements are sequentially stored in this segment of continuous memory addresses. The first remote memory direct access module maintains a workqueue element pointer, which points to an address in the memory space corresponding to the workqueue. Specifically, after the first remote memory access module determines that a workqueue element has been executed, it increments the workqueue element pointer. Since ordered workqueue elements are sequentially stored in this segment of continuous memory addresses, after incrementing the workqueue element pointer, the workqueue element pointer is made to point to the next workqueue element to be executed. At this point, the first remote memory direct access module can retrieve the target workqueue element from the workqueue based on the workqueue element pointer.

[0135] In step S802, after reading the target work queue element from the work queue, referring to the above steps S701 to S702, it can be seen that the target work queue element is generated based on the starting address and data length of the target data in the first registered memory, and the starting address and data length corresponding to the target data will be used as specific fields in the target queue element. Based on this, the first remote memory direct access module can extract the corresponding fields from the target work queue element, determine the starting address and data length, and send them to the target graphics processing unit.

[0136] In the embodiment disclosed in steps S801 to S802, after detecting that the data copying unit copies the target data to the first registered memory, the first remote memory direct access module is controlled to obtain the target work queue element from the work queue and extract the starting address and data length and send them to the target graphics processing unit. In this way, the target graphics processing unit can obtain the starting address and data length corresponding to the target data, and use them to directly read the target data from the registered memory of the first graphics processing unit.

[0137] Detailed implementation of combining data transfer process with calculation process

[0138] In one embodiment, referring to Figure 5 The first graphics processing unit also includes a command processor, a calculation module and a signal synchronization module. The first data transmission module is also provided with a task read and write interface. Figure 9 Before step S201, the method further includes:

[0139] Step S901: obtaining, through a command processor, a first command packet sent by an application layer of a first graphics processing unit;

[0140] Step S902: parsing the first command packet through the command processor, scheduling the first computing task obtained by parsing the first command packet to the computing module for execution, and writing the target task descriptor obtained by parsing the first command packet into the first task queue if the type of the first command packet is the target command packet type;

[0141] Step S903: executing the first computing task through the computing module, and writing the data obtained from executing the first computing task into the first memory module as the target data, and writing a synchronization completion signal to the signal synchronization module after completing the execution of the first computing task;

[0142] Step S904 : In response to the write synchronization completion signal, the signal synchronization module controls the task read / write interface to read the target task descriptor from the first task queue and sends the target task descriptor to the control unit.

[0143] In this embodiment, the command processor is a module for parsing the commands issued by the central processing unit into a form that can be understood and executed by the hardware module in the graphics processing unit. In one embodiment, the command processor can be the N310 processor. Figure 5The command processor may be integrated with a task descriptor engine, which is used to parse the command stream issued by the upper-layer application into at least one corresponding task descriptor and distribute the corresponding task descriptor to the corresponding hardware unit for execution. A computing module is a module within the graphics processing unit that performs calculations on data and may include multiple computing units. In one possible embodiment, each computing unit may be an in-memory computing module.

[0144] In step S901, a first command packet is a command packet sent from the application layer of the first graphics processing unit to the hardware layer. The first command packet may be determined based on the program code of the neural network model deployed on the first graphics processing unit, and is used to instruct the various hardware modules of the first graphics processing unit to perform at least one data processing task. These data processing tasks may include computing tasks that use computing modules to perform arithmetic and logical operations on corresponding data, data transmission tasks that use the first remote memory direct access module to transmit specific data to other graphics processing units, etc. For example, the first command packet may be a command packet used to instruct the first graphics processing unit to perform CCL collective communication of the results obtained by executing a computing task between the first graphics processing unit and multiple second graphics processing units after executing the computing task.

[0145] In step S902, the first computing task is a task that requires the computing module to perform a specific type of arithmetic and logical operation on the corresponding data. It can be generated based on an operator in the neural network model deployed to the first graphics processing unit. The first computing task indicates the data used by the computing module when performing the calculation and the type of operation performed on the data. For example, the first computing task can be used to instruct the computing module to perform a matrix multiplication operation on tensors A and B. The target command packet type is a command packet type used to indicate that after completing the first computing task, the first graphics processing unit needs to use the data obtained from executing the first computing task as the target data and transmit it to the target graphics processing unit. For example, the first command packet can instruct the first graphics processing unit to perform a matrix multiplication operation on tensors A and tensor B, and broadcast the tensor C obtained by the matrix multiplication, that is, broadcast matrix C to each second graphics processing unit interconnected with the first graphics processing unit.

[0146] It is understandable that the underlying hardware in the graphics processing unit can only understand machine language and cannot directly understand the program code of the neural network model written by the user. Based on this, in this embodiment, it is necessary to parse the first command packet sent by the application layer through the command processor to convert it into a form that can be understood and executed by the hardware in the graphics processing unit. It is understandable that the graphics processing unit itself is a device for performing large-scale parallel computing. The first command packet assigned to the first graphics processing unit for execution often contains at least one computing task. After the command processor parses the first command packet, a task descriptor corresponding to at least one first computing task can be obtained. By sending the task descriptor to the computing module, the corresponding first computing task can be scheduled to the computing module for execution.

[0147] In one embodiment, a first task queue is further provided in the task read / write interface. The first task queue can be a task first-in-first-out queue (Task FIFO) for caching task descriptors that need to be handed over to the first remote memory direct access module for execution. When the first command packet is of the target command packet type, it indicates that the first graphics processing unit needs to transfer the data obtained from executing the first computing task to other graphics processing units. After the command processor parses the first command packet, it not only obtains the task descriptor corresponding to the first computing task that needs to be assigned to the computing module for execution, but also obtains the target task descriptor that needs to be executed by the first remote memory direct access module. However, at this time, since the computing module has not completed the first computing task, that is, the target data that needs to be transmitted through the first remote memory direct access module is not ready, the target data cannot be transmitted through the first remote memory direct access module. Therefore, when the command processor publishes the first computing task to the computing module for execution, it can write the target task descriptor into the first task queue. After writing the target task descriptor to the first task queue, the task read and write interface will also register a synchronization wait signal corresponding to the target task descriptor in the signal synchronization module. The synchronization wait signal is used to indicate that the task read and write interface needs to be controlled to take out the target task descriptor from the first task queue and send it to the control unit after the signal synchronization module receives the synchronization completion signal returned by the computing module.

[0148] In step S903, upon receiving the task descriptor corresponding to the first computing task issued by the command processor, the computing module performs arithmetic and logical operations on the corresponding data based on the information indicated in the task descriptor to complete the first computing task. Upon completion of the first computing task, the computing module indicates that the target data to be transmitted to the target GPU is ready. At this point, the computing module returns a synchronization completion signal to the signal synchronization module, allowing the signal synchronization module to detect the completion of the target data preparation.

[0149] Furthermore, in another possible embodiment, after the target data is copied to the first registered memory via the data copying unit, the computing module may then return a synchronization completion signal to the signal synchronization module. In this manner, when the control unit receives the target task descriptor, the target data has already been written to the first registered memory. After receiving the target task descriptor, the control unit can directly control the first remote memory direct access module to transmit the target data without waiting for any external enable signal.

[0150] In addition, in one possible embodiment, the computing module can be provided with a task scheduler. Specifically, when the computing module receives the task descriptor corresponding to the first computing task, it has not yet completed executing other computing tasks received previously. Therefore, the computing module cannot start executing the first computing task immediately. At this time, the task descriptor corresponding to the first computing task can be written into the task queue corresponding to the task scheduler to wait for the computing module to complete executing other computing tasks received previously, and then read the task descriptor corresponding to the first computing task from the task queue to start executing the first computing task.

[0151] In step S904, when the signal synchronization module obtains the synchronization completion signal returned by the computing module, it can be determined that the first computing task has been completed, that is, the target data that needs to be transmitted from the first graphics processing unit to the target graphics processing unit is ready. At this time, the signal synchronization module can control the task read and write interface to read the target task descriptor from the first task queue, and send the target task descriptor to the control unit, so that the control unit and the first remote memory access module execute steps S201 to S204, and transmit the target data obtained by executing the first computing task to the target graphics processing unit.

[0152] Specifically, with reference to the relevant description of the above steps, it can be known that the first task queue is a first-in-first-out queue, and the execution order of the data transmission tasks corresponding to the various task descriptors in the first task queue is consistent with the execution order of the computing tasks corresponding to the various data transmission tasks. It is not difficult to understand that the first task descriptor that is not read out in the first task queue is the target task descriptor. Based on this, after receiving the synchronization completion signal written by the computing module, the signal synchronization module can send an enable signal to the task read-write interface so that the task read-write interface can obtain the target task descriptor from the first task descriptor in the first task queue, and then the target task descriptor can be transmitted to the control unit via the local bus.

[0153] In the embodiment disclosed in steps S901 to S904, the command processor first parses the first command packet sent by the application layer to extract the first computing task that the computing module needs to execute, and when the target task descriptor that needs to be executed by the first remote memory direct access module is parsed from the first command packet, the target task descriptor is written to the first task queue; then, the computing module executes the first computing task to obtain the target data to be transmitted, and after the computing module completes the execution of the first computing task, the computing module returns a synchronization completion signal to the signal synchronization module, so that the signal synchronization module responds to the synchronization completion signal, reads the target task descriptor from the first task queue and sends it to the control unit, thereby, when the computing module calculates the target data to be transmitted, it automatically calls the first remote memory access module to start transmitting the target data, without the user having to manually control when the first remote memory direct access module starts executing the target task descriptor through programming, further simplifying the user's programming model and reducing the difficulty of interconnecting graphics processing units using remote memory direct access technology.

[0154] In one embodiment, referring to Figure 5 The control unit is also equipped with a completion queue, see Figure 10 After transmitting the target data to the target address, the method further includes:

[0155] Step S1001: receiving a first response signal corresponding to target data returned by a target graphics processing unit through a first remote memory direct access module;

[0156] Step S1002: The first remote memory direct access module generates a target completion queue element corresponding to the target data in response to the first response signal, and writes the target completion queue element into the completion queue;

[0157] In step S1003, the command processor returns a second response signal to the application layer in response to the target completion queue element in the completion queue, so that the application layer determines that the first command packet has been executed.

[0158] In this embodiment, the completion queue is a queue corresponding to the work queue, and the elements in the completion queue and the work queue are in one-to-one correspondence.

[0159] In step S1001, the first response signal is a signal used to indicate that the target graphics processing unit has obtained the target data from the first graphics processing unit. Specifically, after receiving the target data from the first graphics processing unit, the target graphics processing unit will return a response signal to the first graphics processing unit. This response signal is the first response signal. After receiving the first response signal returned by the target graphics processing unit, the first graphics processing unit can confirm that the target data has been sent to the target graphics processing unit and the data transmission task corresponding to the target task descriptor has been completed.

[0160] In step S1002, the target completion queue element is a completion queue element (CQE) corresponding to the target data, which is used to indicate that the data transmission task corresponding to the target data has been completed, that is, the target work queue element in the work queue has been completed.

[0161] Specifically, referring to the description of steps S701 to S702, after obtaining the target task descriptor, the control unit will create a target work queue element corresponding to the target data. At this time, the control unit will generate a work request based on the target task descriptor and determine the corresponding work request identification number. Then, the control unit will generate a target work queue element based on the work request identification number and write it into the work queue, so as to indicate that the target work queue element is the work corresponding to the target data through the work request identification number. At this time, after receiving the first response signal, the first remote memory direct access module can generate a corresponding completion queue element based on the work request identification number corresponding to the target work queue element, thereby obtaining the target completion queue element.

[0162] Specifically, the first remote memory direct access module executes each work queue element in the work queue in an orderly manner. That is, the first remote memory direct access module executes the next work queue element in the work queue after completing one work queue element. It is not difficult to understand that the target work queue element is the last work queue element executed by the first remote memory direct access module before receiving the first response signal. In this case, the first remote memory direct access module only needs to generate a target completion queue element that can be used to indicate that the target data has been transmitted to the target graphics processing unit based on the work request identification number corresponding to the last work queue element executed before receiving the first response signal.

[0163] In step S1003, the second response signal is used to notify the application layer of the first graphics processing unit that the first command packet has been executed. It is understood that in order to ensure the orderly execution of each computing task in the graphics processing unit, the application layer of the graphics processing unit must confirm that the previous computing task has been completed and the data required for the next computing task has been prepared before issuing the next computing task. Based on this, after executing each command issued by the application layer, the hardware layer needs to return a signal to the application layer to notify the application layer of the completion of the previous command, so that the application layer can issue the next command to the hardware layer of the first graphics processing unit for execution.

[0164] Specifically, the control unit polls the completion queue to check each completion queue element written into the completion queue. When the control unit determines that there is a completion queue element in the completion queue corresponding to the same work request identification number as the target work queue element when polling the completion queue, it can be determined that the target data has been transmitted to the target graphics processing unit, that is, the previous command packet sent by the application layer has been executed. At this time, the control unit returns a second response signal to the application layer to notify the application layer that the computing module of the first graphics processing unit and the first remote memory direct access module have completed the execution of the previous command, and the application layer can continue to send the next command to the command processor.

[0165] In the embodiment disclosed in steps S1001 to S1003, when the first remote memory direct access module receives a first acknowledgment signal returned by the target GPU in response to the completion of the target data transfer, it can automatically generate a corresponding target completion queue element and write it to the completion queue in response to the first acknowledgment signal. Thus, when the control unit polls the completion queue and detects that the target completion queue element has been written to the completion queue, it can automatically control the command processor to return a second acknowledgment signal to the application layer to notify the application layer that the previous command packet has been executed. In this process, the completion queue elements in the completion queue are automatically completed by the hardware, such as the first remote memory access module, the control unit, and the command processor, in response to corresponding signals, without requiring the creation of completion queue elements based on user-programmed code. Consequently, users do not need to maintain the completion queue through corresponding code statements during programming, further simplifying the user's programming model and reducing the difficulty of interconnecting multiple GPUs using remote memory direct access technology.

[0166] The present disclosure also provides an electronic device, which includes the first graphics processing unit shown in the above embodiment. The electronic device can be used to execute the data transmission method provided in the above embodiment.

[0167] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein, for example, can be implemented in orders other than those illustrated or described herein. In addition, the terms "comprises" and "comprising," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0168] It should be understood that in the present disclosure, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0169] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0170] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0171] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0172] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0173] It should also be understood that the various implementations provided in the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0174] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A data transmission method, characterized in that: The method is applied to a first graphics processing unit, the first graphics processing unit including a first data transmission module, a control unit, and a first memory module, the first data transmission module including a first remote memory direct access module, the first graphics processing unit being communicatively connected to at least one second graphics processing unit via the first data transmission module, the first remote memory direct access module being provided with a corresponding first registered memory, the first registered memory and the first memory module being connected via a local bus; the method comprising: acquiring, by the control unit, a target task descriptor, wherein the target task descriptor is used to characterize the transmission of target data between the first graphics processing unit and a target graphics processing unit, wherein the target graphics processing unit is at least one of the second graphics processing units; parsing the target task descriptor by the control unit to obtain a data source address of target data and a target graphics processing unit identifier, wherein the data source address is an address of the target data in the first memory module, and the target graphics processing unit identifier is a graphics processing unit identifier corresponding to the target graphics processing unit; Determining, by the control unit, a target address corresponding to the target graphics processing unit identifier, and sending a data source address and the target address to the first remote memory direct access module; The starting address and data length of the target data in the first registered memory are sent to the target graphics processing unit through the first remote memory direct access module, so that the target graphics processing unit reads the target data from the first registered memory based on the starting address and the data length.

2. The data transmission method according to claim 1, wherein: Before acquiring the target task descriptor by the control unit, the method further includes: performing a topology self-discovery process between the first graphics processing unit and the second graphics processing unit by the control unit to determine each of the second graphics processing units communicatively connected to the first graphics processing unit; A graphics processing unit identifier is allocated to the first graphics processing unit and each of the second graphics processing units, wherein the first graphics processing unit and each of the second graphics processing units correspond to different graphics processing unit identifiers.

3. The data transmission method according to claim 2, wherein: The first graphics processing unit and each of the second graphics processing units are respectively provided with a corresponding registration memory. Before determining, by the control unit, a target address corresponding to the target graphics processing unit identifier, the method further includes: performing unified addressing on the registration memory of the first graphics processing unit and the registration memories of each of the second graphics processing units to determine a physical address of each of the registration memories, wherein the physical addresses corresponding to each of the registration memories are different; determining an address mapping table based on the physical addresses of the respective registration memories, the graphics processing unit identifiers of the first graphics processing units, and the graphics processing unit identifiers of the respective second graphics processing units; The determining of the target address corresponding to the target graphics processing unit identifier includes: A target address corresponding to the target GPU is determined based on the target GPU identifier and the address mapping table.

4. The data transmission method according to claim 1, wherein: The first memory module is provided with a data replication unit, the first registered memory and the data replication unit are connected via a local bus, and the data transmission method further includes: The target data is copied from the first memory module by the data copying unit based on the data source address and the data length of the target data, and the copied data is transmitted to the first registered memory through the local bus.

5. The data transmission method according to claim 4, characterized in that: The control unit is provided with a work queue. Before the data copying unit copies the target data from the first memory module based on the data source address and the data length of the target data, and transmits the copied data to the first registered memory through the local bus, the method further includes: Determine the data length of the target data based on the target task descriptor, and assign a starting address to the target data, where the starting address is the starting address when writing the target data into the first registered memory; Based on the start address, the data length and the target address, a target work queue element corresponding to the target task descriptor is written to the work queue.

6. The data transmission method according to claim 5, characterized in that: The step of sending the starting address and data length of the target data in the first registered memory to the target graphics processing unit through the first remote memory direct access module includes: In response to the data copying unit copying and transmitting the target data to the first registered memory, controlling the first remote memory direct access module to read the target work queue element corresponding to the target task descriptor from the work queue; The first remote memory direct access module reads the target data from the first registered memory according to the start address and the data length based on the target work queue element, and transmits the target data to the target address.

7. The data transmission method according to claim 1, wherein: The first graphics processing unit further includes a command processor, a calculation module, and a signal synchronization module. The first data transmission module is further provided with a task read and write interface. Before obtaining the target task descriptor through the control unit, the method further includes: Acquire, by the command processor, a first command packet sent by the application layer of the first graphics processing unit; parsing the first command packet by the command processor, scheduling a first computing task obtained by parsing the first command packet to the computing module for execution, and writing the target task descriptor obtained by parsing the first command packet into a first task queue when the type of the first command packet is a target command packet type; executing the first computing task through the computing module, and writing data obtained from executing the first computing task into the first memory module as the target data, and returning a synchronization completion signal to the signal synchronization module after completing executing the first computing task; In response to writing the synchronization completion signal, the signal synchronization module controls the task read / write interface to read the target task descriptor from the first task queue and sends the target task descriptor to the control unit.

8. The data transmission method according to claim 7, characterized in that: The control unit is further provided with a completion queue, wherein the completion queue includes a target completion queue element corresponding to the target data. After the target data is transmitted to the target address, the method further includes: receiving, through the first remote memory direct access module, a first response signal corresponding to the target data and returned by the target graphics processing unit, wherein the first response signal is used to indicate that the target graphics processing unit has obtained the target data from the first graphics processing unit; The first remote memory direct access module generates a target completion queue element corresponding to the target data in response to the first response signal, and writes the target completion queue element into the completion queue; The command processor returns a second response signal to the application layer in response to the target completion queue element in the completion queue, so that the application layer determines that the first command packet has been executed.

9. A graphics processing unit, characterized in that: include: a first data transmission module, a control unit, and a first memory module, wherein the first data transmission module includes a first remote memory direct access module, and the graphics processing unit is communicatively connected to at least one second graphics processing unit via the first data transmission module; the first remote memory direct access module is provided with a corresponding first registered memory, and the first registered memory and the first memory module are connected via a local bus; The control unit is used to obtain a target task descriptor, wherein the target task descriptor is used to represent the transmission of target data between the graphics processing unit and a target processing unit, and the target graphics processing unit is a second graphics processing unit used to receive the target data; The control unit is further configured to parse the target task descriptor to obtain a data source address of the target data and a target graphics processing unit identifier, wherein the data source address is an address of the target data in the first memory module, and the target graphics processing unit identifier is a graphics processing unit identifier corresponding to the target graphics processing unit; The control unit is further configured to determine a target address corresponding to the target graphics processing unit identifier, and send the target address to the first remote memory direct access module; The first remote memory direct access module is used to send the starting address and data length of the target data in the first registered memory to the target graphics processing unit, so that the target graphics processing unit reads the target data from the first registered memory based on the starting address and the data length.

10. An electronic device, characterized in that: The electronic device comprises the graphics processing unit according to claim 9.

Images