Data transmission method and device, computing equipment and storage medium

Abstract

The invention discloses a data transmission method and device, computing equipment and a storage medium, and relates to the technical field of computers.The method comprises the steps that the computing equipment obtains a shared memory address of target equipment from the storage medium in response to a received data transmission request from the target equipment, the shared memory address refers to a unique address of a shared memory in the target equipment; wherein the shared memory and the global memory in the target device both adopt a unified addressing technology, so that the shared memory is directly connected to the PCIe bus through the shared memory address. And the computing device initiates a data transmission operation to the shared memory address through the PCIe bus. In this manner, the GMEM and the SMEM use a unified address space, and the address of the SMEM is exposed to the outside. It means that both the CPU and the RDMA controller can access the two types of memories through a single address space. The technical problem of low data transmission efficiency in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data transmission method, apparatus, computing device and storage medium. Background Art

[0002] The memory areas in a computing system generally include: host memory, global memory (GMEM), shared memory (SMEM), registers, and various caches.

[0003] The hardware components inside the computing system are connected via PCIe (PCI Express), a universal serial connection standard suitable for connecting various memory areas for fast data transfer and sharing.

[0004] RDMA (Remote Direct Memory Access) is a high-performance network communication technology that allows a node in a network to directly access the memory of another node without the intervention of the operating system or CPU. It is particularly suitable for high-performance computing and data center scenarios that require large-scale data exchange.

[0005] GDR (GPU Direct RDMA) is a technology introduced by NVIDIA, which allows GPUs to exchange data directly with other RDMA-supported devices without going through the CPU. However, the memory currently operated by GDR is the GPU's global memory (GMEM), not the thread block's local shared memory (SMEM). For this reason, NVIDIA introduced a new feature TMA (Tensor Memory Accelerator) in the Hopper architecture to perform asynchronous memory replication between the GPU's GMEM and SMEM.

[0006] That is to say, in the traditional TMA architecture, data is first transferred to GMEM, and then transferred from GMEM to SMEM through asynchronous memory copy. The access speed of GMEM is slow, and the access speed of SMEM is fast, but because data must be transferred through GMEM, the data transmission efficiency is low. Summary of the invention

[0007] The embodiments of the present application provide a data transmission method, apparatus, computing device and storage medium to solve the technical problem of low efficiency of data transmission between devices in the prior art.

[0008] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions: In a first aspect, an embodiment of the present application provides a data transmission method, which is applied to a computing device, wherein the computing device includes a storage medium, and the method includes: In response to a data transmission request received from a target device, obtaining a shared memory address of the target device from the storage medium, wherein the shared memory address refers to a unique address of a shared memory in the target device, wherein the shared memory in the target device and the global memory both adopt a unified addressing technology so that the shared memory address is used for the shared memory to be directly connected to a PCIe bus; A data transmission operation is initiated to the shared memory address via the PCIe bus.

[0009] In combination with the first aspect, in a possible design manner, before obtaining the shared memory address of the target device from the storage medium, the method further includes: Obtain the shared memory address provided by the target device through a message request of the PCIe protocol; The shared memory address is written into the storage medium.

[0010] In combination with the first aspect, in a possible design, the computing device is a CPU or an RDMA controller, the target device is a computing device card, and the unified addressing technology includes: Build 2 64 Byte-sized address space; Allocating high 16 bits of the address in the address space to multiple nodes in the target cluster, wherein the address space of each node is 256 TB; Allocating the address space of each node to the CPU and computing device card corresponding to the node, the address space of the CPU being 128 TB, and the address space of the computing device card being 128 TB; Each of the nodes supports 64 computing device cards, each of the computing device cards is allocated with 2 TB of address space, and the shared memory and the global memory respectively occupy 1 TB of address space in the computing device card.

[0011] In combination with the first aspect, in a possible design manner, the acquiring the shared memory address provided by the target device through a message request of the PCIe protocol includes: Sending a message request to the target device; the message request is used to request to obtain the shared memory address of the target device; receiving response information; the response information indicates the address space of the shared memory of the target device, and the response information is sent by the target device in response to the message request; The shared memory address of the target device is obtained from the response information.

[0012] In combination with the first aspect, in a possible design manner, the target device includes an address register A, a register L, and a comparator, the register L indicates n high-significant bits of the address in the address register A, and after the data transmission operation is initiated to the shared memory address, the method further includes: The target device verifies the shared memory address through the comparator to obtain a verification result; If the verification result is true, the target device processes the data; if the verification result is false, the target device discards the data; The verification result indicates whether the Ln-bit address of the shared memory address is the same as the Ln-bit address in the address register A.

[0013] In combination with the first aspect, in a possible design manner, the initiating a data transmission operation to the shared memory address includes: When the computing device initiates a read operation instruction to the target device through the PCIe bus, the target device locks the shared memory corresponding to the shared memory address; The target device reads the data in the shared memory into a data buffer area based on the data start address and data length information in the read operation instruction; In the case where all the data are read from the shared memory to the data buffer area, the target device releases the shared memory; The target device sends the data to the PCIe bus based on the shared memory.

[0014] In combination with the first aspect, in a possible design manner, the initiating a data transmission operation to the shared memory address includes: When the computing device initiates a write operation instruction to the target device through the PCIe bus, the target device writes the data into the data buffer area based on the data start address and data length information in the write operation instruction; When all the data are written into the data buffer area, the target device locks the shared memory corresponding to the shared memory address; After the data is written from the data buffer area into the shared memory, the target device releases the shared memory and sends a signal to the PCIe bus, where the signal indicates that the write operation instruction is executed successfully.

[0015] In a second aspect, an embodiment of the present application provides a data transmission device, the device comprising: An address acquisition module is used to obtain a shared memory address of the target device from a storage medium of the computing device in response to a data transmission request received from the target device, wherein the shared memory address refers to a unique address of the shared memory in the target device, wherein the shared memory in the target device and the global memory both adopt a unified addressing technology so that the shared memory address is used for the shared memory to be directly connected to the PCIe bus; The data transmission module is used to initiate a data transmission operation to the shared memory address through the PCIe bus.

[0016] In a third aspect, an embodiment of the present application provides a computing device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method of the first aspect and possible design methods thereof.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the method of the first aspect and its possible design manner when running.

[0018] Compared with the prior art, the embodiment of the present application provides a data transmission method, device, computing device and storage medium. The computing device obtains the shared memory address of the target device from the storage medium in response to the data transmission request received from the target device. The shared memory address refers to the unique address of the shared memory in the target device; wherein the shared memory and the global memory in the target device both adopt a unified addressing technology so that the shared memory is directly connected to the PCIe bus through the shared memory address. The computing device initiates a data transmission operation to the shared memory address through the PCIe bus, thereby performing data transmission with the target device. GMEM and SMEM use a unified address space, and the address of SMEM is exposed to the external PCIe bus. In this way, the computing device skips the GMEM transfer in the traditional method, can directly access the shared memory through the PCIe bus, and transfer the data to SMEM at one time. This means that both the CPU and the RDMA controller can access the two types of memory, GMEM and SMEM, through a single address space. This method solves the problem of address conversion between GMEM and SMEM in the traditional architecture. Since the external can directly access SMEM and the SMEM has a fast read and write speed, the data transmission process is simplified and the data transmission efficiency is improved.

[0019] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A hardware structure block diagram of a computing device provided in an embodiment of the present application is shown; Figure 2 is a schematic diagram of the interconnection relationship between multiple devices provided in an embodiment of the present application; Figure 3 A flow chart of a data transmission method provided by an embodiment of the present application is shown; Figure 4 A flowchart of a unified addressing technology provided by an embodiment of the present application is shown; Figure 5 A flowchart of a shared memory address acquisition method provided by an embodiment of the present application is provided; Figure 6 A flow chart of a method for reading data using buffering provided in an embodiment of the present application is shown; Figure 7 A flow chart of a method for implementing data writing by using buffering provided in an embodiment of the present application is shown; Figure 8 A system framework diagram of a computing device based on unified addressing provided by an embodiment of the present application is shown; Fig. 9 A structural block diagram of a data transmission device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0022] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and apparatus, product or equipment comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or equipment. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0023] With the advancement of technology, PCIe, RDMA and GDR technologies have gradually emerged. PCIe is a universal serial connection standard, mainly used for connecting hardware components within a computing system, especially for connecting various memory areas for fast data transmission and sharing. RDMA is a high-performance network communication technology that allows a node in a network to directly access the memory of another node without the intervention of the operating system or CPU. It is particularly suitable for high-performance computing and data center scenarios that require large-scale data exchange. GDR is a technology launched by NVIDIA that allows GPUs to exchange data directly with other RDMA-supported devices without the need for CPU transfer. TMA is used for asynchronous memory replication between the GMEM and SMEM of the GPU.

[0024] In general, PCIe, RDMA and GDR are all important technologies aimed at improving communication performance. However, SMEM cannot be directly accessed in traditional technologies. When executing write operation instructions, data is first transferred to GMEM and then transferred from GMEM to SMEM through asynchronous memory copy. During the asynchronous transmission process, the PCIe bus cannot bypass GMEM to directly access SMEM, so it not only increases the delay of data transmission, but may also cause data transmission bottlenecks.

[0025] Based on this, the embodiment of the present application provides a data transmission method, which is applied to a computing device, and the computing device can transmit data with a target device. The computing device can obtain the unique address (shared memory address) of the shared memory in the target device, and store the shared memory address in the storage medium of the computing device. Then, when the computing device needs to read and write data to the target device, it can directly obtain the shared memory address from the storage medium to directly access the shared memory of the target device, and transfer the data to the shared memory at one time, skipping the GMEM transfer in the traditional method, simplifying the intermediate links of data transmission, and thus speeding up the transmission speed.

[0026] The method can be executed in a terminal, a computer or a similar computing system. For example, running on a computing device, Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device provided in an embodiment of the present application. Figure 1 As shown, the computing device may include one or more ( Figure 1 The computing device may further include a transmission device 106 and an input / output device 108 for communication functions. It will be appreciated by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above computing device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, to implement the above method. The memory 104 can be used to store data. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The embodiments of the present application can be applied to data transmission, and are particularly suitable for scenarios that require large amounts of data transmission, such as large model training, high performance computing (HPC), machine learning, etc. The speed and efficiency of data transmission in these scenarios directly affect the completion time and cost of computing tasks. The use of the technology provided by the embodiments of the present application can significantly improve the efficiency of data processing and accelerate the execution of computing tasks.

[0030] Figure 2 is a schematic diagram of the interconnection relationship between multiple devices provided in the embodiment of the present application, such as Figure 2 As shown, the cluster includes multiple computing nodes, such as computing node 0, computing node 1, computing node 2, and computing node 3. The computing nodes are interconnected through a high-speed network (RoCE or IB network). Each computing node includes a CPU and multiple computing device cards, which are connected within the computing node through PCIe, that is, data between devices in the computing node is transmitted through the PCIe interface. The computing device can be a CPU or an RDMA controller, and the target device can be a computing device card.

[0031] Figure 3 A flow chart of a data transmission method provided by an embodiment of the present application is shown. Figure 3 As shown, the method includes step S301 to step S302.

[0032] Step S301: In response to a data transmission request received from a target device, a computing device obtains a shared memory address of the target device from a storage medium, where the shared memory address refers to a unique address of a shared memory in the target device.

[0033] Among them, the shared memory and the global memory in the target device both adopt a unified addressing technology, so that the shared memory is directly connected to the PCIe bus through the shared memory address.

[0034] In this application, computing devices and target devices refer to devices with PCIe interfaces that follow the PCIe protocol. In traditional technology, only the address of GMEM is exposed to the external PCIe bus, so the computing device can only access GMEM through the PCIe bus first, and then read and write SMEM through GMEM. In this way, although the read and write speed of SMEM is very fast, it is affected by the slow read and write speed of GMEM, and the overall data transmission efficiency is low.

[0035] In this step, the unique address of SMEM (shared memory address) is stored in the storage medium by the computing device, so that when the computing device needs data transmission, it can directly access SMEM based on the shared memory address, thus skipping the step of accessing GMEM and directly transferring all data to SMEM at one time. Since SMEM has a fast read and write speed, connecting SMEM directly to the PCIe bus can improve transmission efficiency.

[0036] Before step S301, the method further includes: the computing device obtains the shared memory address provided by the target device through a message request of the PCIe protocol; and writes the shared memory address into a storage medium.

[0037] Specifically, before the computing device obtains the shared memory address of the target device, the computing device can also perform data transmission with the target device, such as initiating a message request through the PCIe protocol, so that the target device responds to the message request and sends data to the computing device. In this embodiment, the PCIe bus first accesses the GMEM, and then reads and writes the SMEM through the GMEM, thereby realizing the transmission of the shared memory address.

[0038] On the one hand, using the PCIe protocol to obtain the shared memory address helps ensure the compatibility of different computing devices with the communication content, compared with using other protocols to obtain the shared memory address, that is, it helps the computing device and the target device to accurately identify the communication content. For example, the communication content is a message request indicating the acquisition of the shared memory address, or a response message carrying the shared memory address. This not only helps the communication reliability, but also ensures the accuracy of the acquired address.

[0039] On the other hand, the shared memory address is obtained by using the message request mechanism of the PCIe protocol. Since the message request can customize the request content, compared with other requests using the PCIe protocol to obtain the shared memory address, it not only fully utilizes the scalability of the message request mechanism, but also ensures that the communication content complies with the provisions of the PCIe protocol and the feasibility of the address acquisition operation.

[0040] In some embodiments, the computing device also obtains the global memory address of the target device, which refers to the unique address of GMEM in the target device. That is, GMEM and SMEM use a unified address space, and the corresponding global memory or shared memory can be accessed based on the address space. This unified addressing design eliminates the need for address conversion between GMEM and SMEM, thereby simplifying the data transmission process.

[0041] In some of the embodiments, the shared memory and the global memory in the target device all use a unified addressing technology. Figure 4 A flowchart of a unified addressing technology provided by an embodiment of the present application is shown. Figure 4 As shown, the unified addressing technology includes steps S401 to S403.

[0042] Step S401: Build 2 64 Byte-sized address space.

[0043] This step uniformly addresses all devices in the cluster with a 64-bit address length, and the total address space size is 2 64 Bit.

[0044] Step S402: allocate the high 16 bits of the address in the address space to multiple nodes in the target cluster, wherein the address space of each node is 256 TB.

[0045] The high 16 bits of the address are used to calculate the addressing of different nodes in the cluster, supporting a total of 65,536 nodes.

[0046] In this step, the high 16-bit address refers to the highest 16 bits of the address (i.e., from bit 49 to bit 64). These high bits represent the part of the address space farthest from the processor and are used to allocate independent address spaces for different hardware devices or system resources.

[0047] Step S403: Allocate the address space of each node to the CPU and computing device card corresponding to the node, the address space of the CPU is 128 TB, and the address space of the computing device card is 128 TB.

[0048] In this step, the address space of each node is 256TB, which can be evenly distributed to the CPU and computing device card, 128TB each. Each node can support 64 computing device cards, with a maximum of 65536 nodes, so a total of 4194304 computing cards can be supported. The address space of the computing device card on each node is 128TB. The address space of each card is 2TB, of which 1TB is GMEM and 1TB is SMEM.

[0049] Therefore, GMEM and SMEM are uniformly addressed and obtain global memory addresses and shared memory addresses respectively. Computing devices (such as CPU) can directly access GMEM based on global memory addresses and directly access SMEM based on shared memory addresses.

[0050] In the embodiment of the present application, the GMEM and SMEM in the computing device may also be uniformly addressed using the method provided in steps S401 to S403 above, so that the target device can quickly access the SMEM in the computing device.

[0051] Figure 5 A flowchart of a method for obtaining a shared memory address provided in an embodiment of the present application is provided, such as Figure 5 As shown, in combination with the unified addressing technology described above, the process of obtaining the shared memory address provided by the target device through a message request of the PCIe protocol may further include steps S501 to S503.

[0052] Step S501: Send a message request to a target device; the message request is used to request to obtain a shared memory address of the target device.

[0053] Step S502, receiving response information; the response information indicates the address space of the shared memory of the target device, and the response information is sent by the target device in response to the message request.

[0054] Step S503: Obtain the shared memory address of the target device from the response information.

[0055] In the embodiment recorded in step S501 to step S503, after receiving the message request, if the target device supports P2P transmission, it will respond, such as returning response information based on receiving the message request. Otherwise, if it does not support P2P transmission, it will not respond.

[0056] Exemplarily, the message request includes a "PCIe Requester ID" which includes an ID of a computing device. Then the target device may return a response message according to the ID, that is, return a shared memory address to the device to which the ID indicated by the "PCIe Requester ID" belongs.

[0057] The target device may save the shared memory address in a specified field of the response information, and the computing device may obtain the shared memory address of the target device from the specified field (eg, Shared Memory Address) in the response information.

[0058] Through the above step S301, the SMEM of the target device is addressed and the address is exposed to the outside, so that the outside does not need to access the GMEM first and then access the SMEM, but can directly obtain the shared memory address from the storage medium, thereby executing step S302.

[0059] Step S302: The computing device initiates a data transmission operation to the shared memory address, thereby performing data transmission with the target device.

[0060] The data transmission operation includes data reading and writing operations, that is, writing data to SMEM or reading data from SMEM.

[0061] Since all SMEMs in each computing device card are individually connected to PCIe, each SMEM has a unique address, so read and write operations can be performed directly through PCIe.

[0062] In some embodiments, the target device verifies the shared memory address. Specifically, the target device includes an address register A, a register L, and a comparator. The register L indicates the n-bit high-significant bits of the address in the address register A. The comparator verifies the validity of the address. The address register A and the register L are read and written via the PCIe bus and are two programmable registers. Assume that the input shared memory address is b 63 b 62 b 61 ...b1b0, the value of address register A is a 63 a 62 a 61 …a1a0, and the value of register L is n. If b 63 b 62 b 61 …b 64-n Equal to a 63 a 62 a 61 …a 64-n , the comparator output is true, indicating that the address is selected, otherwise the output is false, indicating that the address is not selected. If the verification result is true, the target device processes the data, and if the verification result is false, the target device discards the data to improve security. In some embodiments, if the device supports multiple separated addresses, multiple unified addressing units need to be added, and each unified addressing unit performs the above comparison steps to compare and verify the multiple separated addresses at the same time.

[0063] In the embodiment of the present application, considering the speed difference between the PCIe bus and the internal bus of the computing device card, a read-write buffer queue is introduced in each SMEM to speed up the asynchronous transmission rate of data. The read-write buffer queue is implemented by hardware, which can receive read-write requests from PCIe and automatically complete the read-write of SMEM. Each SMEM adds a locking mechanism to lock the corresponding memory area when performing SMEM reading and writing to ensure that the read-write of the buffer queue does not conflict with the read-write of the computing kernel.

[0064] Specifically, Figure 6 A flow chart of a method for reading data using a buffer provided in an embodiment of the present application is shown. Figure 6 As shown, initiating a data transmission operation to a shared memory address includes steps S601 to S604.

[0065] Step S601: When the computing device initiates a read operation instruction to the target device through the PCIe bus, the target device locks the shared memory corresponding to the shared memory address.

[0066] Step S602: The target device reads the data in the shared memory into the data buffer area based on the data start address and data length information in the read operation instruction.

[0067] Step S603: When all the data is read from the shared memory to the data buffer area, the target device releases the shared memory.

[0068] Step S604: The target device sends data to the PCIe bus based on the shared memory.

[0069] In this embodiment, the read / write buffer queue includes a data buffer area, a data start address, and a data length. When the PCIe bus initiates a read operation, the read / write buffer queue receives the instruction and saves the read data start address and data length information, locks the corresponding SMEM block, and reads the data into the data buffer area; after completion, the locked SMEM block is released and the corresponding data is sent to the PCIe bus.

[0070] Figure 7 A flow chart of a method for implementing data writing by using a buffer provided in an embodiment of the present application is shown. Figure 7 As shown, initiating a data transmission operation to a shared memory address includes steps S701 to S703.

[0071] Step S701: When the computing device initiates a write operation instruction to the target device through the PCIe bus, the target device writes the data into the data buffer area based on the data start address and data length information in the write operation instruction.

[0072] Step S702: When all data are written into the data buffer area, the target device locks the shared memory corresponding to the shared memory address.

[0073] Step S703: After the data is written from the data buffer area to the shared memory, the target device releases the shared memory and sends a signal to the PCIe bus, the signal indicating that the write operation instruction is executed successfully.

[0074] In this embodiment, when the PCIe bus initiates a write operation, the read-write buffer queue receives the instruction and saves the write data starting address and data length information, and the data is written to the data cache area; then the corresponding SMEM block is locked and the data is written, and after completion, a successful execution signal is sent to the PCIe bus.

[0075] Therefore, by adding a locking mechanism in each SMEM, the corresponding shared memory area is locked when reading and writing SMEM to ensure that the buffer queue reading and writing does not conflict with the computing kernel reading and writing.

[0076] In the embodiment recorded in the above steps S301 to S302, the computing device obtains the shared memory address of the target device from the storage medium in response to the data transmission request received from the target device, and the shared memory address refers to the unique address of the shared memory in the target device; wherein the shared memory and the global memory in the target device both adopt a unified addressing technology so that the shared memory is directly connected to the PCIe bus through the shared memory address. The computing device initiates a data transmission operation to the shared memory address through the PCIe bus, thereby performing data transmission with the target device. GMEM and SMEM use a unified address space, and the address of SMEM is exposed to the external PCIe bus. In this way, the computing device skips the GMEM transfer in the traditional method, can directly access the shared memory through the PCIe bus, and transfer the data to SMEM at one time. This means that both the CPU and the RDMA controller can access both types of memory, GMEM and SMEM, through a single address space. This design eliminates the need for transfer and copy operations between GMEM and SMEM in the traditional architecture, thereby simplifying the data transmission process.

[0077] Secondly, by connecting SMEM directly to the PCIe bus, data can be transferred directly from the source to SMEM, thus skipping the intermediate step of GMEM. This direct connection significantly reduces the path length of data transmission and improves the efficiency of data transmission.

[0078] Since the data transmission rates of different computing devices may be inconsistent, a buffer queue is introduced on the bus and SMEM transmission. The buffer queue can temporarily store data until the receiving device is ready to receive the data. This mechanism effectively reduces the transmission waiting time caused by rate mismatch and ensures the continuity and efficiency of data transmission.

[0079] In addition, to ensure that the method is compatible with traditional computing devices, the computing core can still directly read and write SMEM and GMEM. This compatibility consideration allows existing systems to take advantage of the method without requiring large-scale hardware upgrades.

[0080] In general, the method provided in the embodiment of the present application has the advantages of less data copying, fast transmission speed, low latency, and compatibility with existing systems. It is a very effective and feasible method in data transmission for large model training, especially in RDMA transmission based on PCIe or high-speed network.

[0081] The method provided in the embodiment of the present application is further illustrated by a specific example below. Figure 8 FIG. 1 shows a system framework diagram of a computing device based on unified addressing provided by an embodiment of the present application. Figure 8 As shown in the figure, each node includes a CPU, a computing device card, and an RDMA controller (network card). The computing device card is connected to the CPU and the RDMA controller through PCIe. In the computing device card, the PCIe interface can directly access the global memory (GMEM) and the shared memory (SMEM) through a programmable unified address unit. Each shared memory is connected to the PCIe interface through a buffer queue. The buffer queue can temporarily store data until the receiving device (CPU, RDMA controller or SMEM) is ready to receive the data. The shared memory can transfer data with the computing core.

[0082] Fig. 9 A structural block diagram of a data transmission device provided in an embodiment of the present application is shown. Fig. 9 As shown, the device comprises: The address acquisition module 91 is used to obtain the shared memory address of the target device from the storage medium of the computing device in response to a data transmission request received from the target device, wherein the shared memory address refers to the unique address of the shared memory in the target device, wherein the shared memory in the target device and the global memory both adopt a unified addressing technology so that the shared memory address is used for the shared memory to be directly connected to the PCIe bus.

[0083] The data transmission module 92 is used to initiate a data transmission operation to the shared memory address through the PCIe bus.

[0084] In some of the embodiments, the address acquisition module 91 is further used to acquire the shared memory address provided by the target device through a message request of the PCIe protocol; and write the shared memory address into the storage medium.

[0085] In some of the embodiments, the address acquisition module 91 is also used to send a message request to the target device; the message request is used to request to obtain the shared memory address of the target device; receive response information; the response information indicates the address space of the shared memory of the target device, and the response information is sent by the target device in response to the message request; and the shared memory address of the target device is obtained from the response information.

[0086] In some embodiments, the address acquisition module 91 is further used to address the shared memory and the global memory in the target device using a unified addressing technology, and the unified addressing technology includes: constructing 2 64 The address space is a byte-sized address space; the high 16-bit addresses in the address space are allocated to multiple nodes in the target cluster, where the address space of each node is 256TB; the address space of each node is allocated to the computing device card corresponding to the node, the address space of the CPU is 128TB, and the address space of the computing device card is 128TB; each node supports 64 computing device cards, each computing device card is allocated with 2TB of address space, and the shared memory and global memory each occupy 1TB of the address space in the computing device card.

[0087] In some of the embodiments, the data transmission module 92 is also used for the target device to verify the shared memory address through a comparator to obtain a verification result; if the verification result is true, the target device processes the data; if the verification result is false, the target device discards the data; wherein the verification result indicates whether the Ln bit address of the shared memory address is the same as the Ln bit address in the address register A.

[0088] In some of the embodiments, the data transmission module 92 is also used for, when the computing device initiates a read operation instruction to the target device through the PCIe bus, the target device locks the shared memory corresponding to the shared memory address; the target device reads the data in the shared memory into the data cache based on the data starting address and data length information in the read operation instruction; when all the data is read from the shared memory to the data cache, the target device releases the shared memory; and the target device sends data to the PCIe bus based on the shared memory.

[0089] In some of the embodiments, the data transmission module 92 is also used for when the computing device initiates a write operation instruction to the target device through the PCIe bus, the target device writes the data into the data cache based on the data starting address and data length information in the write operation instruction; when all the data are written into the data cache, the target device locks the shared memory corresponding to the shared memory address; after the data is written from the data cache into the shared memory, the target device releases the shared memory and sends a signal to the PCIe bus, the signal indicating that the write operation instruction is executed successfully.

[0090] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0091] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0092] In addition, in combination with the method provided in the above embodiment, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the data transmission methods in the above embodiment is implemented.

[0093] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the processor in the above method embodiment.

[0094] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0095] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0096] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0097] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A data transmission method, characterized in that: Applied to a computing device, the computing device comprising a storage medium, the method comprising: In response to a data transmission request received from a target device, obtaining a shared memory address of the target device from the storage medium, wherein the shared memory address refers to a unique address of a shared memory in the target device, wherein the shared memory in the target device and the global memory both adopt a unified addressing technology so that the shared memory is directly connected to a PCIe bus through the shared memory address; A data transmission operation is initiated to the shared memory address via the PCIe bus.

2. The data transmission method according to claim 1, characterized in that: Before obtaining the shared memory address of the target device from the storage medium, the method further includes: Obtain the shared memory address provided by the target device through a message request of the PCIe protocol; The shared memory address is written into the storage medium.

3. The data transmission method according to claim 2, characterized in that: The computing device is a CPU or an RDMA controller, the target device is a computing device card, and the unified addressing technology includes: Build 2 64 Byte-sized address space; Allocating high 16 bits of the address in the address space to multiple nodes in the target cluster, wherein the address space of each node is 256 TB; Allocating the address space of each node to the CPU and computing device card corresponding to the node, the address space of the CPU being 128 TB, and the address space of the computing device card being 128 TB; Each of the nodes supports 64 computing device cards, each of the computing device cards is allocated with 2 TB of address space, and the shared memory and the global memory respectively occupy 1 TB of address space in the computing device card.

4. The data transmission method according to claim 3, characterized in that: The acquiring of the shared memory address provided by the target device through a message request of the PCIe protocol includes: Sending a message request to the target device; the message request is used to request to obtain the shared memory address of the target device; receiving response information; the response information indicates the address space of the shared memory of the target device, and the response information is sent by the target device in response to the message request; The shared memory address of the target device is obtained from the response information.

5. The data transmission method according to claim 3, characterized in that: The target device includes an address register A, a register L, and a comparator, wherein the register L indicates n high-significant bits of the address in the address register A. After initiating a data transmission operation to the shared memory address, the method further includes: The target device verifies the shared memory address through the comparator to obtain a verification result; If the verification result is true, the target device processes the data; if the verification result is false, the target device discards the data; The verification result indicates whether the Ln-bit address of the shared memory address is the same as the Ln-bit address in the address register A.

6. The data transmission method according to any one of claims 1 to 5, characterized in that: The initiating a data transmission operation to the shared memory address includes: When the computing device initiates a read operation instruction to the target device through the PCIe bus, the target device locks the shared memory corresponding to the shared memory address; The target device reads the data in the shared memory into a data buffer area based on the data starting address and data length information in the read operation instruction; In a case where all the data are read from the shared memory to the data buffer area, the target device releases the shared memory; The target device sends the data to the PCIe bus based on the shared memory.

7. The data transmission method according to any one of claims 1 to 5, characterized in that: The initiating a data transmission operation to the shared memory address includes: When the computing device initiates a write operation instruction to the target device through the PCIe bus, the target device writes the data into the data buffer area based on the data start address and data length information in the write operation instruction; When all the data are written into the data buffer area, the target device locks the shared memory corresponding to the shared memory address; After the data is written from the data buffer area into the shared memory, the target device releases the shared memory and sends a signal to the PCIe bus, where the signal indicates that the write operation instruction is executed successfully.

8. A data transmission device, characterized in that: The device comprises: An address acquisition module is used to obtain a shared memory address of the target device from a storage medium of the computing device in response to a data transmission request received from the target device, wherein the shared memory address refers to a unique address of the shared memory in the target device, wherein the shared memory in the target device and the global memory both adopt a unified addressing technology so that the shared memory address is used for the shared memory to be directly connected to the PCIe bus; The data transmission module is used to initiate a data transmission operation to the shared memory address through the PCIe bus.

9. A computing device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the data transmission method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the data transmission method according to any one of claims 1 to 7 when running.

Images