Inter-device p2p communication architecture, system and method based on universal address space

Abstract

The application discloses a kind of inter-device P2P communication architecture, system and method based on general address space, communication architecture includes data exchange device and multiple devices electrically connected with it, each device is configured with device ID and physical memory address, data exchange device includes management module and global address mapping table;Global address mapping table includes one-to-one correspondence device ID and DMA address, request device sends a data transmission request including request device ID and target device ID, management module determines the DMA address consistent with target device ID in global address mapping table as target DMA address;According to request device ID, target device ID determines the physical memory address corresponding to request device and target device;The communication between the physical memory address of request device and the physical memory address of target device is realized through target DMA address.The application can realize low delay, high efficiency P2P communication between devices.

Description

Technical Field

[0001] The present invention relates to the field of computer and communication technology, and in particular to a P2P communication architecture, system and method between devices based on a universal address space. Background Art

[0002] With the rapid development of artificial intelligence (AI), deep learning (DL), high-performance computing (HPC) and big data processing technologies, the demand for data transmission has exploded. Especially in high-bandwidth, low-latency data exchange scenarios, the traditional PCIe architecture faces many severe challenges.

[0003] (1) The bandwidth and latency requirements for multi-device parallel computing are constantly increasing.

[0004] Especially in areas like AI training, deep learning, and high-performance computing (HPC), large-scale parallel computing requires the collaboration of multiple devices (such as GPUs, FPGAs, and ASICs) to achieve higher computing performance and data throughput. However, under the existing PCIe architecture, data transmission speed and latency between devices become bottlenecks that restrict system performance. Each device often needs to transfer data through the CPU or main memory when exchanging data, resulting in lengthy data transmission paths, additional latency, and increased bandwidth contention.

[0005] With the continuous growth of artificial intelligence (AI) models and the rapid increase in training data volumes, traditional architectures are unable to meet the demand for extremely low latency and high throughput. In scenarios such as multi-GPU parallel training, GPU cluster computing, and data parallel computing, data exchange between devices becomes a system bottleneck. Bandwidth contention is particularly severe when multiple devices are accessing data simultaneously, impacting overall performance.

[0006] (2) Bandwidth bottleneck problem in traditional PCIe architecture.

[0007] PCIe, the most widely used high-speed data transmission standard, has significantly increased its bandwidth, but bandwidth sharing remains a challenge in large-scale systems. When multiple devices operate concurrently, the traditional PCIe architecture's bandwidth allocation method forces multiple devices to share limited transmission channels, resulting in a severe bandwidth shortage and further impacting data transmission efficiency.

[0008] The traditional PCIe architecture relies on the CPU for data scheduling and management. Data transmission between devices relies on the CPU-main memory-device path. This reliance not only makes the data transmission path lengthy but also leads to system bottlenecks due to increased CPU load. This is particularly prominent in GPU cluster computing or large-scale data parallel processing environments, where bandwidth competition becomes a major issue.

[0009] To alleviate this problem, existing technologies attempt to allocate bandwidth resources through multi-level switching architectures or direct device-to-device interconnection. However, these solutions often have complex architectures and cannot effectively solve the problems of low bandwidth utilization and high latency.

[0010] (3) Delay problem and lengthy data transmission path problem.

[0011] In the traditional PCIe architecture, communication between devices must pass through the CPU and main memory, resulting in a long data transmission path. Although PCIe technology itself has high bandwidth, the overall efficiency of data transmission is severely affected by latency, which is particularly prominent in scenarios involving massively parallel computing and high-performance data exchange.

[0012] For example, during deep learning training, data is constantly transferred between multiple GPUs. Traditional methods often rely on CPU and main memory scheduling, which increases data transmission latency and affects the overall efficiency of model training. For applications requiring low latency and high throughput, the traditional PCIe architecture is no longer suitable.

[0013] Furthermore, current architectures often struggle to provide effective direct communication between devices. Even with new switching architectures, traditional switches and network layers still increase the complexity and latency of communication paths. Therefore, reducing data transmission latency between devices has become an urgent challenge.

[0014] (4) The interconnection complexity in multi-device clusters is increasing.

[0015] With the development of data centers, hyperscale clusters, and cloud computing platforms, more and more devices require efficient interconnection within a unified architecture. However, traditional PCIe architecture design primarily focuses on connecting single-machine systems and lacks support for large-scale multi-device systems. This complicates the system topology and resource allocation strategies, especially when dealing with complex data exchange between multiple devices.

[0016] In a multi-device cluster, each device needs to access the data resources of other devices. The bandwidth limitations, latency issues, and shared resource management of the traditional PCIe architecture cannot meet these requirements. Especially when multiple devices are computing in parallel, topology design becomes extremely complex. The interconnection paths between devices must take into account factors such as scalability, fault tolerance, and bandwidth allocation, making it difficult to effectively implement flexible and efficient data exchange.

[0017] (5) PCIe's P2P data transmission lacks flexibility.

[0018] Although PCIe itself supports P2P (Peer-to-Peer) communication between devices, in traditional architectures, P2P data transmission between devices usually relies on the CPU and memory as intermediaries, which not only increases data transmission latency but also limits the flexibility and efficiency of direct communication between multiple devices.

[0019] The current PCIe interconnect architecture makes it difficult to efficiently support P2P data transmission between devices without CPU intervention, making direct communication between multiple devices impossible.

[0020] In summary, while existing interconnect technologies like NVLink and CCIX can partially address device communication latency and bandwidth issues, they suffer from poor hardware compatibility, high costs, complex network topologies, and incompatibility with the PCIe architecture. These limitations limit their application in large-scale clusters and heterogeneous computing environments. Improving the existing PCIe architecture, optimizing the P2P communication mechanism between devices, reducing latency, improving bandwidth utilization, and maintaining compatibility with existing hardware are pressing technical challenges.

[0021] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of the present application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above content has been disclosed before the filing date of the present application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the Invention

[0022] The purpose of the present invention is to provide a P2P communication architecture, system and method between devices based on a universal address space, which can achieve low-latency and high-efficiency P2P communication between devices.

[0023] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0024] A device-to-device P2P communication architecture based on a universal address space includes a data exchange device and a plurality of devices to be communicated electrically connected to the data exchange device, wherein the devices are configured with corresponding device IDs and physical memory addresses, each device being configured with a unique device ID and a unique physical memory address, and the device IDs and the physical memory addresses have a one-to-one correspondence;

[0025] The data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes a one-to-one correspondence between device IDs and DMA addresses; the requesting device and the target device in the device perform P2P communication in the following manner:

[0026] The requesting device sends a data transmission request to the data exchange device, wherein the data transmission request includes a requesting device ID and a target device ID;

[0027] The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address;

[0028] Determine the physical memory address of the requesting device according to the requesting device ID, and determine the physical memory address of the target device according to the target device ID;

[0029] Communication between the physical memory address of the requesting device and the physical memory address of the target device is achieved through the target DMA address.

[0030] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the management module is further configured to transfer data in one DMA address to another DMA address;

[0031] The DMA address corresponding to the same device ID communicates directly with the physical memory address, and the DMA addresses corresponding to different device IDs cannot communicate with the physical memory address;

[0032] Implementing communication between the physical memory address of the requesting device and the physical memory address of the target device through the target DMA address includes:

[0033] The physical memory address of the requesting device communicates with its corresponding requesting end DMA address;

[0034] The management module controls the communication between the requester DMA address and the target DMA address;

[0035] The target DMA address communicates with a target physical memory address corresponding to the target device requesting the device.

[0036] Further, based on any one of the technical solutions or a combination of multiple technical solutions described above, in response to receiving the data transfer request, the management module determines whether there is a DMA address in the global address mapping table that matches the target device ID; if so, determines the matching DMA address as the target DMA address, sends a communication approval signal to the requesting device based on the requesting device ID, sends the data transfer request to the target device based on the target device ID, and controls the requesting DMA address to communicate with the target DMA address;

[0037] In response to receiving the communication consent signal, the requesting device configures its physical memory address to communicate with its corresponding requesting end DMA address;

[0038] In response to receiving the data transfer request, the target device configures its physical memory address to communicate with the corresponding target DMA address.

[0039] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the data transmission request further includes a data transmission type, and the data transmission type includes writing data;

[0040] If the data transmission type is writing data, the requesting device and the target device perform P2P communication in the following manner:

[0041] Transferring the data in the physical memory address corresponding to the requesting device to the requesting end DMA address;

[0042] The management module transfers the data in the requesting end DMA address to the target DMA address;

[0043] The data in the target DMA address is transferred to the physical memory address corresponding to the target device.

[0044] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the data transmission request further includes a data transmission type, and the data transmission type includes reading data;

[0045] If the data transmission type is reading data, the requesting device and the target device perform P2P communication in the following manner:

[0046] Transferring the data in the physical memory address corresponding to the target device to the target DMA address;

[0047] The management module transfers the data in the target DMA address to the requesting end DMA address;

[0048] The data in the requesting end DMA address is transferred to the physical memory address corresponding to the requesting device.

[0049] Further, according to any one of the above technical solutions or a combination of multiple technical solutions, each of the DMA addresses is configured to communicate with any one of the physical memory addresses;

[0050] Implementing communication between the physical memory address of the requesting device and the physical memory address of the target device through the target DMA address includes:

[0051] The physical memory address of the requesting device communicates with the target DMA address;

[0052] The target DMA address communicates with a target physical memory address corresponding to the target device requesting the device.

[0053] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the system further includes a DMA controller corresponding to each of the devices, wherein the DMA controller corresponding to the device is configured to control the communication between the physical memory address of the device and the DMA address corresponding to the device;

[0054] or,

[0055] It also includes a DMA controller corresponding to each of the devices, and the DMA controller corresponding to the device is configured to control the communication between the physical memory address of the device and the target DMA address.

[0056] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, it further includes an uplink switch, wherein the uplink switch is electrically connected to each of the devices respectively, and the uplink switch is electrically connected to the CPU;

[0057] When the two devices need to perform P2P communication, the two devices do not perform P2P communication with the CPU through the uplink switch, but perform P2P communication through the data exchange device.

[0058] Furthermore, any one of the above technical solutions or a combination of multiple technical solutions includes a first device group and a second device group, wherein the first device group and the second device group each include a plurality of the devices;

[0059] The plurality of devices in the first device group are electrically connected to a first uplink switch, respectively; the plurality of devices in the second device group are electrically connected to a second uplink switch, respectively; and the first uplink switch and the second uplink switch are electrically connected to a CPU, respectively;

[0060] Each device in the first device group and the second device group is electrically connected to the data exchange device, and any two devices do not perform P2P communication through the first upstream switch, the second upstream switch, and the CPU, but perform P2P communication through the data exchange device;

[0061] The first uplink switch and the second uplink switch are electrically connected to the same CPU; or the first uplink switch and the second uplink switch are electrically connected to different CPUs.

[0062] Data exchange device The data exchange device further adopts any one of the above-mentioned technical solutions or a combination of multiple technical solutions, and the DMA addresses corresponding to the various devices do not overlap; and / or,

[0063] The sizes of the DMA addresses corresponding to the devices are equal; and / or,

[0064] The device is a PCIe device.

[0065] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the size of the DMA address corresponding to the device is determined by:

[0066] Determine the size of the physical memory address corresponding to each of the devices;

[0067] Determine the memory usage of the device, where the memory usage = the size of the physical memory address corresponding to the device / the sum of the sizes of the physical memory addresses corresponding to all devices;

[0068] The size of the DMA address corresponding to the device = the memory ratio × the total DMA resources.

[0069] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the size of the DMA address corresponding to the device is determined by the following method:

[0070] determining a communication frequency between each of the devices and other devices;

[0071] The greater the communication frequency, the larger the size of the DMA address corresponding to the device.

[0072] According to another aspect of the present invention, a communication system is provided, which includes the inter-device P2P communication architecture based on the universal address space as described in any one of the above technical solutions or a combination of multiple technical solutions.

[0073] According to another aspect of the present invention, a method for inter-device P2P communication based on a universal address space is provided, which is applicable to the inter-device P2P communication architecture based on a universal address space as described in any one of the above technical solutions or a combination of multiple technical solutions, comprising the following steps:

[0074] A data exchange device is configured to be electrically connected to a plurality of devices to be communicated, wherein the devices are configured with corresponding device IDs and physical memory addresses, and the data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes one-to-one correspondence between device IDs and DMA addresses;

[0075] A requesting device in the device sends a data transmission request to the data exchange apparatus to communicate with a target device, the target device being another device different from the requesting device, the data transmission request including a requesting device ID and a target device ID;

[0076] The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address;

[0077] Determine the physical memory address of the requesting device according to the requesting device ID, and determine the physical memory address of the target device according to the target device ID;

[0078] Communication between the physical memory address of the requesting device and the physical memory address of the target device is achieved through the target DMA address.

[0079] The beneficial effects brought about by the technical solution provided by the present invention are as follows:

[0080] a. This invention utilizes a data exchange device equipped with a management module and a global address mapping table. It allocates DMA resources to each device based on the UAS table and implements direct data transmission and bandwidth isolation between devices based on the target DMA address. This bypasses traditional CPU transfer and enables direct, low-latency, and highly efficient P2P communication between devices. Furthermore, inter-device communication no longer consumes bandwidth on the uplink path, avoiding bandwidth competition with other device communication paths, thereby improving overall system bandwidth utilization.

[0081] b. This invention proposes an inter-device P2P communication architecture and method that is applicable not only to single-system architectures but also to multi-system architectures. By completely decentralizing the inter-device P2P data transmission process, replacing the existing transit through the CPU and upstream switch, direct inter-device communication is achieved, significantly reducing data transmission latency. This makes it particularly suitable for low-latency applications such as AI training, HPC, and high-performance data exchange.

[0082] c. The present invention proposes multiple methods, such as uniformly allocating DMA resources to multiple devices and allocating DMA resources based on the physical memory size or communication frequency of the devices, which can meet various practical application needs. Among them, the uniform DMA allocation method is simple and easy to implement, while the allocation of DMA resources based on physical memory size or communication frequency is more in line with actual usage needs and can further improve data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0084] Figure 1 A schematic diagram comparing the existing PCIe device communication architecture and the inter-device P2P communication architecture of the present application;

[0085] Figure 2A general flow chart of a method for P2P communication between devices based on a universal address space is provided for an exemplary embodiment of the present invention;

[0086] Figure 3 A schematic diagram of the working principle of a first inter-device P2P communication architecture provided by an exemplary embodiment of the present invention;

[0087] Figure 4 A flowchart of initializing a P2P communication architecture between devices based on a universal address space is provided for an exemplary embodiment of the present invention;

[0088] Figure 5 A schematic diagram of a process of initializing a UAS to construct a UAS table provided by an exemplary embodiment of the present invention;

[0089] Figure 6 A schematic diagram of a process for allocating DMA resources to various devices provided by an exemplary embodiment of the present invention;

[0090] Figure 7 A flowchart of data transmission during P2P communication between devices provided by an exemplary embodiment of the present invention;

[0091] Figure 8 A schematic diagram of a process for implementing communication between ASIC X and ASIC 2 based on a UAS is provided as an exemplary embodiment of the present invention;

[0092] Figure 9 Flowchart of ASIC X writing data to ASIC 2 based on the first inter-device P2P communication architecture;

[0093] Figure 10 Flowchart of ASIC X reading data from ASIC 2 based on the first inter-device P2P communication architecture;

[0094] Figure 11 A schematic diagram of an inter-device P2P communication architecture with multiple uplink switches provided for an exemplary embodiment of the present invention;

[0095] Figure 12 A schematic diagram of a P2P communication architecture between devices with multiple CPUs is provided for an exemplary embodiment of the present invention;

[0096] Figure 13 A schematic diagram of the working principle of a first inter-device P2P communication architecture provided by an exemplary embodiment of the present invention;

[0097] Figure 14 Flowchart of ASIC X writing data to ASIC 2 based on the second inter-device P2P communication architecture;

[0098] Figure 15 The flowchart of ASIC X reading data from ASIC 2 based on the second inter-device P2P communication architecture is shown in FIG. DETAILED DESCRIPTION

[0099] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0100] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0101] Introduction to relevant terms involved in this application:

[0102] PCIe (PCI Express): is a widely used high-speed serial computer expansion bus standard that provides efficient data transmission and flexible expansion capabilities;

[0103] P2P (Peer-to-Peer) communication: This refers to peer-to-peer communication. Each node (Peer) has the same functions and status. Nodes can communicate directly with each other without going through a central server. Each node can both request and provide resources, and is therefore both a client and a server.

[0104] BAR (Base Address Register): used to define the memory mapping address or I / O address of the device;

[0105] DMA (Direct Memory Access): refers to the hardware resources and related mechanisms used for direct memory access within the switch. It is a technology that allows hardware devices to directly access system memory.

[0106] MMIO (Memory Mapping Input / Output) is a mechanism designed into the PCIe specification that maps a portion of physical addresses to PCIe devices. When the CPU initiates a read or write operation on an MMIO address, it is taken over by the PCIe management module and converted into an access request to the device connected to the PCIe bus.

[0107] To address the aforementioned issues with existing technologies, designing a low-latency, highly flexible, and efficient inter-device P2P communication architecture / method is a challenging task in the field of computer communication technology. For example, Chinese patent application publication number CN119597489A proposes a PCIe-NTB-based P2P communication method and system for I / O devices. By enabling direct P2P communication between I / O devices on multiple computing nodes using PCIe-NTB, this method avoids the protocol conversion overhead and main memory and I / O device memory transfer overhead required by traditional NIC-based RDMA communication, thereby reducing the latency of inter-device P2P communication. However, this technical solution extends the bus from a single computer (host) to multiple systems (hosts) using a non-transparent bridge (NTB), leaving the P2P communication method between multiple PCIe devices within a single host unchanged. Furthermore, this method uses NTBs to connect upstream switches in multiple hosts, but communication between PCIe devices still requires the upstream switch path, leading to bandwidth contention for inter-device communication.

[0108] In view of the shortcomings of the existing technology, the present invention introduces a new universal address space (UAS) and downstream switch (Switch) architecture, and proposes a P2P communication architecture and method between devices based on the universal address space. It can effectively solve the above-mentioned problems existing in the traditional PCIe architecture, especially the bandwidth competition and high latency problems in inter-device communication.

[0109] In one embodiment of the present invention, Figure 1 、 Figure 2 and Figure 3 As shown, a P2P communication architecture between devices based on a universal address space is provided, comprising a data exchange device and a plurality of devices to be communicated electrically connected to the data exchange device, wherein the devices are configured with corresponding device IDs and physical memory addresses;

[0110] The data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes a one-to-one correspondence between device IDs and DMA addresses; the requesting device and the target device in the device perform P2P communication in the following manner:

[0111] The requesting device sends a data transmission request to the data exchange device, wherein the data transmission request includes a requesting device ID and a target device ID;

[0112] The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address;

[0113] Determine the physical memory address of the requesting device according to the requesting device ID, and determine the physical memory address of the target device according to the target device ID;

[0114] Communication between the physical memory address of the requesting device and the physical memory address of the target device is achieved through the target DMA address.

[0115] In this embodiment, the inter-device P2P communication architecture based on the universal address space further includes an uplink switch, which is electrically connected to each of the devices and electrically connected to the CPU. When two devices need to perform P2P communication, the two devices do not communicate with the CPU through the uplink switch, but instead perform P2P communication through the data exchange device.

[0116] It should be noted that the traditional PCIe architecture is Figure 1 As shown in the center left portion, multiple devices communicate by sharing the same uplink bandwidth path. This creates significant bandwidth contention, especially in high-concurrency and large-scale computing environments, leading to reduced throughput. Furthermore, data transmission between devices often requires transit through the CPU and memory, which not only lengthens the data transmission path but also introduces significant latency. This latency can become a performance bottleneck, especially when multiple devices are performing parallel computing. In this PCIe architecture, data transmission competes with CPU communication for bandwidth, often failing to maximize bandwidth utilization.

[0117] The inter-device P2P communication architecture proposed in this application is as follows Figure 1 As shown in the right part of the figure, the UAS-based PCIeSwitch P2P communication architecture establishes a global unified DMA address space table, enabling each PCIe device to determine its own DMA resources and dedicated transmission path for communicating with the target device based on the UAS table, realizing direct data transmission between devices, bypassing the upstream switch and CPU path, thereby reducing data transmission latency and effectively improving bandwidth utilization.

[0118] Specifically, by introducing a dedicated data exchange device, which is equivalent to a downlink switch, the data exchange device in this application is different from the existing downlink switch in that it is equipped with a UAS table and a management module that can directly implement P2P communication between two devices by using the DMA resources allocated to each device in the UAS table. In this application, communication between devices no longer occupies the bandwidth of the uplink path, and data transmission between different devices uses independent paths. Therefore, bandwidth is effectively allocated and bandwidth waste is avoided. This bandwidth isolation mechanism enables each device to efficiently transmit data on its dedicated bandwidth path, thereby improving the bandwidth utilization of the entire system.

[0119] Therefore, the inter-device P2P communication architecture proposed in this application is based on the UAS table inside the data exchange device. Each device communicates with the data exchange device through an independent path. On the basis of achieving isolated communication between devices, it avoids bandwidth competition with other device communication paths, and can maximize bandwidth utilization. It is particularly suitable for high-bandwidth demand scenarios such as large-scale parallel computing and GPU clusters. Moreover, the inter-device P2P communication architecture proposed in this application is completely decentralized, bypassing the traditional CPU transit, achieving direct, low-latency inter-device communication, and significantly reducing the delay in the data transmission process. It is particularly suitable for low-latency application scenarios such as AI training, HPC, and high-performance data exchange.

[0120] In one embodiment of the present invention, the data exchange device functions as an independent downstream switch, equipped with multiple high-speed ports supporting high-bandwidth parallel communication. Each port can be electrically connected to a PCIe device, such as a GPU or ASIC. A management module and a global address mapping table are internally configured within the data exchange device, enabling peer-to-peer data transmission between devices.

[0121] First, if Figure 4 As shown, the P2P communication architecture between devices based on the universal address space is initialized to allocate memory and I / O ports globally to each device and build a global mapped address space (UAS), which specifically includes the following aspects.

[0122] Scanning and establishing a tree structure: The data switch (downstream PCIe switch) first performs a device scan, establishing a tree topology through the PCIe architecture. This process ensures the identification and initialization of each device, as well as the connection between the data switch and each device.

[0123] Configuring device space: After the tree topology is established, the data exchange device configures the memory and resources of each device. This step is the key to the initialization of the entire system and involves allocating memory, I / O ports, and MMIO (memory-mapped input and output) resources to each device.

[0124] Reading the BAR and allocating device resources: The data exchange device reads each device's Base Address Register (BAR) and allocates the device's required resources based on the BAR information. The BAR represents the device's memory address space, so reading the BAR is the first step in identifying and configuring the device's address space.

[0125] Configuring I / O and MMIO resources: Configure the I / O resources and MMIO (memory-mapped I / O) resources of each device. This part ensures that the device can communicate efficiently with other components in the system and provides the necessary memory mapping space and I / O addresses for the device.

[0126] Initializing the UAS Address Space: Finally, the data exchange device's internal management module, an embedded CPU, initializes the UAS (Universal Address Space) within the device. During this initialization process, the management module (embedded CPU) scans for connected devices and then establishes a Universal Address Space (UAS) based on the number of devices detected. This process ensures that each device has its own memory area within the unified global address space, avoiding conflicts. The UAS provides a unified address space for the entire system, allowing each device to directly transfer data and access resources within this address space.

[0127] On this basis, the UAS is further initialized to build the global address mapping table (UAS Table). The device's DMA resources are allocated according to the UAS mapping relationship and provide support for subsequent data transmission. The specific steps include:

[0128] Create a global address mapping table (UAS Table): Based on the UAS, a DMA address is configured for each device, enabling it to operate independently within the globally unified address space and directly transfer data. Based on the number of scanned devices, the management module assigns each device an independent physical address, that is, determines the physical memory address corresponding to each device, and allocates corresponding DMA resources to each device, including configuring a corresponding DMA address for the DMA resources allocated to each device, and configuring a corresponding DMA controller for the device based on the DMA address. Each device's DMA controller can directly access its physical memory address. Each device's DMA address is mapped to its independent physical memory address.

[0129] like Figure 5 As shown, a DMA address segment (DMA address space) for data transfer is initialized for each device. The DMA addresses of the four devices are superimposed to form a UAS table. This allows efficient and independent DMA data exchange between devices, avoiding CPU transfer in traditional architectures and improving system bandwidth utilization and low-latency performance.

[0130] When configuring the DMA addresses corresponding to the various devices, a uniform allocation method may be used, whereby the sizes of the DMA addresses allocated to the various devices are equal. This allocation method is relatively simple and easy to implement.

[0131] In addition, the size of the DMA address corresponding to the device can also be determined in the following manner: determine the size of the physical memory address corresponding to each of the devices; determine the memory share of the device, the memory share = the size of the physical memory address corresponding to the device / the sum of the sizes of the physical memory addresses corresponding to all devices; the size of the DMA address corresponding to the device = the memory share × total DMA resources.

[0132] Alternatively, the size of the DMA address corresponding to the device may be determined in the following manner: determining the communication frequency between each device and other devices; the greater the communication frequency, the larger the size of the DMA address corresponding to the device.

[0133] Although the latter two allocation methods of DMA addresses of each device are more complicated than the uniform allocation method, the latter two allocation methods are more in line with actual usage requirements and can further improve the efficiency of data transmission.

[0134] During the initialization process, the management module allocates DMA resources to each connected PCIe device and configures the device's DMA controller based on the UAS table created during initialization. This process ensures that each device can independently and efficiently perform data transfer tasks.

[0135] Find the device's DMA resources: According to the UAS table, each device's address and DMA resources have been configured in the global address space. The management module determines the DMA memory resources allocated to each device based on this table. Preferably, the DMA addresses of each device do not overlap to achieve communication isolation.

[0136] In the process of allocating DMA resources for each device, for each device (such as ASIC1, ASIC2, etc.), according to the address mapping in the UAS table, the CPU will allocate a memory space for the device (such as Figure 6 This memory area serves as the device's DMA address for subsequent transfers. During initialization, each device obtains the appropriate DMA resource from the UAS table based on its physical address and required memory size. Each device has independent DMA resources to ensure that memory access conflicts between devices are avoided.

[0137] Configure the DMA controller: After completing the memory allocation, the CPU also needs to configure the DMA controller of each device so that it can access the DMA address space just allocated. The DMA controller sets the target address to a physical address that the device can access based on the address in the UAS table. For example, for any one of the devices, the DMA controller is configured to only read and / or write to the DMA address corresponding to the device, and cannot read and / or write to the DMA addresses corresponding to other devices. Specifically, each DMA address is a range address from the starting address to the ending address, and the DMA controller can only access the DMA address range corresponding to the device ID it corresponds to, and cannot access the DMA address range corresponding to other device IDs.

[0138] Configure the data transfer type (such as read / write operation) and ensure that the device can be accessed according to the correct address during the data transfer process.

[0139] After initialization is complete and DMA resources are allocated to each device, the inter-device P2P communication architecture based on the universal address space enters the runtime phase, at which point each device can issue data transfer requests. This process ensures direct data transfer between devices, eliminating the traditional PCIe architecture's reliance on the CPU or upstream switch, thereby improving system bandwidth utilization and transmission efficiency.

[0140] The device can initiate a data transmission request during operation. The data transmission request usually includes the following important information:

[0141] Requesting device ID: identifies the device ID that sends the communication request, used to determine the source device that initiates the communication request;

[0142] Target device ID: identifies the device ID of the target communication, used to determine the target device with which the requesting device wishes to establish communication;

[0143] Target physical memory address: The target physical memory address is the physical memory address of the target device specified when the device initiates a request. It is used to identify the memory area of ​​the target device. It should be noted that the data transmission request may not include the target physical memory address, because the target device can determine the target physical memory address based on the mapping relationship between the target device ID and its physical memory address. Not configuring the target physical memory address in the data transmission request can improve the security of data transmission. Configuring the target physical memory address can eliminate the process of determining the target physical memory address based on the mapping relationship and improve data transmission efficiency.

[0144] Data transfer type (Memory Read / Write): includes read and write, and is used to specify the data operation type, identifying whether it is a read operation or a write operation, and the specific type of data.

[0145] like Figure 7 As shown, if the requesting device, ASIC X, in the device needs to communicate with the target device, ASIC Y, ASIC X sends a data transfer request to the data exchange device. After receiving the data transfer request, the management module within the data exchange device determines the DMA address in the global address mapping table that matches the target device ID as the target DMA address. The physical memory address of the requesting device directly communicates with the requesting DMA address corresponding to the requesting device; the requesting DMA address directly communicates with the target DMA address; and the target DMA address directly communicates with the target physical memory address.

[0146] Specifically, the DMA controller corresponding to the requesting device is used for data transmission between the physical memory address of the requesting device and the requesting-side DMA address corresponding to the requesting device; the DMA controller corresponding to the target device is used for data transmission between the target physical memory address and the target DMA address. The management module is used for data transmission between the requesting-side DMA address and the target DMA address.

[0147] For example, Figure 8 Figure 9 As shown, when a requesting device (ASIC X, where X is 0, 1, or 3) needs to send data and perform a write operation to a target device (ASIC 2), the data transmission request sent by the requesting device to the data exchange apparatus includes: ASIC X ID, ASIC 2 ID, write, and optionally includes the physical memory address of ASIC 2.

[0148] In response to receiving the data transfer request, the management module of the data exchange device determines whether there is a DMA address in the global address mapping table that matches the ASIC 2 ID. If so, the management module determines that the matching DMA address is the target DMA address, returns a communication approval signal to the requesting device, and sends the data transfer request to the target device, i.e., ASIC 2. If no matching DMA address is found, an error signal is returned to the requesting device.

[0149] In response to receiving the communication consent signal, ASIC X controls its corresponding DMA controller to move / transfer / write the data in the physical memory address of ASIC X to its corresponding ASIC X DMA address.

[0150] The management module matches the ASIC X DMA address corresponding to the ASIC X ID based on the ASIC X ID, and if the data transfer type is write, the management module moves / transfers / writes the data in the ASIC X DMA address to the target DMA address.

[0151] ASIC 2 receives the data transfer request. Since the data transfer type is write, the DMA controller corresponding to ASIC 2 writes the data in the target DMA address to the target physical memory address, i.e., the physical memory address of ASIC 2. In this way, P2P direct communication between ASIC X and ASIC 2 is implemented.

[0152] For example, Figure 10 As shown, when a requesting device (ASIC X, where X is 0, 1, or 3) needs to send data to a target device (ASIC 2) and perform a read operation, the data transmission request sent by the requesting device to the data exchange apparatus includes: ASICX ID, ASIC 2 ID, read, and optionally includes the physical memory address of ASIC 2.

[0153] In response to receiving the data transfer request, the management module of the data exchange device determines whether there is a DMA address in the global address mapping table that matches the ASIC 2 ID. If so, the management module determines that the matching DMA address is the target DMA address, returns a communication approval signal to the requesting device, and sends the data transfer request to the target device, i.e., ASIC 2. If no matching DMA address is found, the management module returns an error signal to the requesting device.

[0154] ASIC 2 receives the data transfer request. Since the data transfer type is read, its DMA controller transfers / reads the data in the target physical memory address, ie, the physical memory address of ASIC 2, to the target DMA address.

[0155] The management module matches the ASIC X DMA address corresponding to the ASIC X ID based on the ASIC X ID, and the data transmission type is read, and then the management module transmits / moves / reads the data in the target DMA address to the ASIC X DMA address.

[0156] In response to receiving the communication consent signal, the corresponding DMA controller of ASIC X transmits / moves / reads the data in the ASIC X DMA address to the physical memory address of ASIC X.

[0157] This technical solution can at least achieve the following technical effects:

[0158] Decentralized data transmission: Traditional data transmission between PCIe devices usually relies on the CPU and main memory. The present invention uses a downstream switch, i.e., a data exchange device, to achieve direct communication between devices, bypassing the CPU path and reducing data transmission delays.

[0159] High flexibility: Based on the UAS address mapping mechanism, a UAS Table is established in the data exchange device. Each device can map its DMA resources according to the global address space, enabling cross-device data access and improving the flexibility of inter-device communication.

[0160] Improved bandwidth utilization: By introducing a dedicated downlink switch, or data exchange device, for data transmission, the uplink path (the traditional data path from the CPU to the device) is avoided, thereby improving overall PCIe bandwidth utilization. This is particularly suitable for high-throughput scenarios such as AI, high-performance computing (HPC), and GPU clusters.

[0161] This paper proposes a PCIe switch P2P data transmission architecture based on a universal address space (UAS). This architecture provides an innovative solution for multi-device clusters through efficient inter-device DMA resource management and direct data transfer. With the development of fields such as artificial intelligence (AI), high-performance computing (HPC), and big data processing, this technical solution can effectively address the bandwidth bottlenecks and high latency issues in traditional PCIe architectures, and has broad application prospects.

[0162] This technical solution can be applied to high-performance computing (HPC) and supercomputing centers. This technology can significantly improve computing efficiency and is particularly suitable for AI training and large-scale parallel computing. It can optimize data transmission between devices, reduce latency, and maximize bandwidth utilization. It is also applicable to artificial intelligence and deep learning. In scenarios such as deep learning and GPU clusters, the present invention can provide efficient data transmission between devices, avoid bandwidth bottlenecks, and improve training speed and computing performance. It is also applicable to data centers and cloud computing platforms. This technology is applicable to data centers and cloud computing platforms and can improve the interconnection efficiency of storage and computing resources, reduce data transmission latency, and improve system scalability and resource utilization. It is also suitable for application scenarios with high bandwidth requirements, such as virtual reality (VR), augmented reality (AR), autonomous driving systems, and high-frequency trading, which require low latency and high bandwidth, to improve system response speed and stability.

[0163] In one embodiment of the present invention, a P2P communication architecture between devices based on a universal address space is proposed, such as Figure 11 As shown, the communication architecture includes a first device group and a second device group, each of which includes multiple devices. The multiple devices in the first device group, namely ASICs 0 to 3, are electrically connected to a first uplink switch, namely PCIe Switch 1. The multiple devices in the second device group, namely ASICs 4 to 7, are electrically connected to a second uplink switch, namely PCIe Switch 2. The first uplink switch and the second uplink switch are each electrically connected to the CPU via a root complex.

[0164] Each of the devices in the first device group and the second device group is electrically connected to the data exchange device respectively. The data exchange device is the same as the data exchange device in the above embodiment, and is internally configured with the management module, the global address mapping table and the DMA controller corresponding to each device.

[0165] In this embodiment, as in the above embodiment, any two devices do not perform P2P communication through the first uplink switch, the second uplink switch, and the CPU, but perform P2P communication through the data exchange device.

[0166] In one embodiment of the present invention, a P2P communication architecture between devices based on a universal address space is proposed, such as Figure 12 As shown, this embodiment is Figure 11 The difference between the shown embodiments is that Figure 11The embodiment shown is a single-HOST architecture, and this embodiment is a dual-HOST architecture. In this embodiment, the first uplink switch is electrically connected to CPU1, and the second uplink switch is electrically connected to CPU2.

[0167] In another embodiment of the present invention, a P2P communication architecture between devices based on a universal address space is provided, such as Figure 13 As shown, it includes a data exchange device and a plurality of devices to be communicated electrically connected to the data exchange device, and the devices are configured with corresponding device IDs and physical memory addresses;

[0168] The data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes a one-to-one correspondence between device IDs and DMA addresses;

[0169] The requesting device and the target device in the devices perform P2P communication in the following manner:

[0170] The requesting device sends a data transmission request to the data exchange device, wherein the data transmission request includes a target device ID;

[0171] The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address;

[0172] The physical memory address of the requesting device directly communicates with the target DMA address;

[0173] The target DMA address communicates with a target physical memory address corresponding to the target device.

[0174] Similarly, a corresponding DMA controller is configured for each of the devices, except that in this embodiment, the DMA controller is configured to perform direct data transfer between the physical memory address of the device and the target DMA address, and the DMA address corresponding to which device depends on which device the target device corresponding to the data transfer request is.

[0175] pass Figure 3 and Figure 13 By comparison, it can be seen that the difference between the inter-device P2P communication architecture provided in this embodiment and the above embodiment is that the communication between the requesting device and the target device is realized based on the target DMA address communication. Figure 3As shown, the DMA controller of each device can only access the physical memory address and DMA address corresponding to that device and cannot directly access the DMA addresses and physical memory addresses of other devices. Data transfer between different DMA addresses must be implemented through the management module. That is, in the above embodiment, the source address and destination address of each DMA controller are preset and remain unchanged.

[0176] In this embodiment, if Figure 13 As shown, the data transmission request includes a requesting device ID, a target device ID, and a data transmission type, and optionally includes a target device physical memory address.

[0177] In response to receiving the data transfer request, the management module in the data exchange device determines whether there is a DMA address in the global address mapping table that matches the target device ID. If so, the matching DMA address is determined to be the target DMA address, and a communication approval signal is returned to the requesting device and the data transfer request is sent to the target device, i.e., ASIC 2. If no matching DMA address is found, an error signal is returned to the requesting device.

[0178] In response to receiving the communication signal and based on the data transmission type, the requesting device controls its corresponding DMA controller to perform data transmission with one of the physical memory address corresponding to the requesting device and the target DMA address as the source address and the other as the target address.

[0179] In response to receiving the data transfer request, the target device controls its corresponding DMA controller to perform data transmission using one of the physical memory address corresponding to the target device and the target DMA address as the source address and the other as the target address according to the data transfer type.

[0180] Specifically, if Figure 14 As shown, if the data transfer type is write, the requesting device responds to receiving the consent communication signal and based on the data transfer type being write, its corresponding DMA controller directly writes / transfers / moves the physical memory address corresponding to the requesting device to the target DMA address.

[0181] In response to receiving the data transfer request and based on the data transfer type being write, the target device controls its corresponding DMA controller to transmit / write / move the data in the target DMA address to the physical memory address corresponding to the target device.

[0182] like Figure 15As shown, if the data transmission type is read, the target device responds to receiving the data transmission request and based on the data transmission type being read, controls its corresponding DMA controller to transfer / read / move the data in the physical memory address corresponding to the target device to the target DMA address.

[0183] In response to receiving the communication signal and based on the data transmission type being read, the requesting device controls its corresponding DMA controller to directly write / transfer / move the data in the target DMA address to the physical memory address corresponding to the requesting device.

[0184] In this embodiment, the management module of the data exchange device does not have the function of moving data from one DMA address to another DMA address. The main function of the management module is to determine whether data can be transmitted between the requesting device and the target device based on the data transmission request, and to determine the target DMA address for the requesting device to directly communicate with its physical memory address. In this embodiment, the initialization of the communication architecture, including configuring the device ID, physical memory address, UAS space configuration, establishment of the UAS table, and configuration of the DMA controller corresponding to each device, is the same as the above embodiment and will not be repeated.

[0185] In addition, in the process of configuring the UAS space and establishing the UAS table, the method of dividing the DMA addresses corresponding to each device is the same as the above embodiment and will not be repeated here.

[0186] In one embodiment of the present invention, a communication system is provided, which includes the inter-device P2P communication architecture based on the universal address space as described in any of the above embodiments.

[0187] In one embodiment of the present invention, a method for P2P communication between devices based on a universal address space is provided, comprising the following steps:

[0188] A data exchange device is configured to be electrically connected to a plurality of devices to be communicated, wherein the devices are configured with corresponding device IDs and physical memory addresses; the data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes one-to-one correspondence between device IDs and DMA addresses;

[0189] A requesting device in the device sends a data transmission request to the data exchange apparatus to communicate with a target device, the target device being another device different from the requesting device, the data transmission request including a requesting device ID, a target device ID and a data transmission type;

[0190] The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address;

[0191] Determine the physical memory address of the requesting device according to the requesting device ID, and determine the physical memory address of the target device according to the target device ID;

[0192] Communication between the physical memory address of the requesting device and the physical memory address of the target device is achieved through the target DMA address.

[0193] In the first method, based on the data transfer type, the physical memory address of the requesting device communicates with its corresponding requesting-side DMA address; based on the data transfer type, the management module controls the requesting-side DMA address to communicate with the target DMA address; and based on the data transfer type, the target DMA address communicates with the target physical memory address corresponding to the target device. The specific communication method is the same as in the above embodiment and will not be repeated here.

[0194] Second mode: Based on the data transfer type, the physical memory address of the requesting device communicates directly with the target DMA address; based on the data transfer type, the target DMA address communicates with the target physical memory address corresponding to the target device. The specific communication mode is the same as the above embodiment and will not be repeated here.

[0195] In one embodiment of the present invention, a data exchange device is provided, comprising a plurality of ports, a management module, and a DMA resource allocation table (equivalent to the UAS table in the above-described embodiment). The ports are high-speed communication ports that support high-bandwidth parallel communication. The management module is electrically connected to each of the ports, and the ports are configured to electrically connect to a device to be communicated with, such as a PCIe device.

[0196] Each port is configured with an independent and unique port address. When each device is connected to the data exchange device, the data exchange device will allocate an independent communication port to the device to ensure that communication between devices does not interfere. For example, the DMA resource allocation table within a four-port data exchange device is shown in Table 1.

[0197]

[0198] Based on the DMA resource allocation table, each port is configured with a corresponding DMA controller, and the DMA controller is configured to directly access the local cache of the device connected to its corresponding port, that is, the configured physical memory address, and access the corresponding DMA address, and complete the reading / writing of data between the memory resources corresponding to the port and the corresponding DMA address.

[0199] In this embodiment, the data exchange device can be used as an independent product to implement P2P communication between multiple devices. Specifically, for multiple devices to communicate, one device is electrically connected to one port of the data exchange device, so that one device corresponds to one port ID.

[0200] When the data exchange device receives a communication request, the communication request includes a requesting port ID, a target port ID, and a communication request type, where the communication request type includes read and write. For example, when a requesting device in the device needs to send (write) data to a target device, or when the requesting device needs to read data from a target device, the port electrically connected to the requesting device is the requesting port, and the port electrically connected to the target device is the target port.

[0201] The first data transmission method is as follows. When the management module receives the communication request, it determines the DMA address in the DMA resource allocation table that matches the target port ID as the target DMA address, returns a communication approval signal to the request port, transmits the communication request to the target port, and performs data transmission between the DMA address corresponding to the request port and the target DMA address based on the communication request type. Specifically, if the communication request type is write, the management module writes the data in the DMA address corresponding to the request port into the target DMA address; if the communication request type is read, the management module reads the data in the target DMA address into the DMA address corresponding to the request port.

[0202] In response to receiving the communication approval signal, the request port corresponding to the DMA controller performs data transfer between the physical memory address and the DMA address corresponding to the request port based on the communication request type. Specifically, if the communication request type is write, the DMA controller corresponding to the request port writes the data in the physical memory address of the device connected to the request port to the DMA address corresponding to the request port; if the communication request type is read, the DMA controller corresponding to the request port reads the data in the DMA address corresponding to the request port to the physical memory address of the device connected to the request port.

[0203] In response to receiving the communication request, the target port performs data transfer between its corresponding physical memory address and the target DMA address based on the communication request type. Specifically, if the communication request type is write, the DMA controller corresponding to the target port writes the data in the target DMA address to the physical memory address of the device connected to the target port; if the communication request type is read, the DMA controller corresponding to the target port reads the data in the physical memory address of the device connected to the target port into the target DMA address.

[0204] In this data transfer method, the DMA addresses accessed by each DMA controller are fixed, so initialization configuration and corresponding program settings are simpler. In addition, the data transfer process between different DMA addresses performed by the management module is a black box for the requesting port and the target port, so the data transmission security is better.

[0205] The second data transmission method is as follows: Upon receiving the communication request, the management module determines a DMA address in the DMA resource allocation table that matches the target port ID as the target DMA address and sends it to the DMA controller corresponding to the requesting port and the DMA controller corresponding to the target port. Furthermore, the management module returns a communication approval signal to the requesting port and sends the communication request to the target port.

[0206] In response to receiving the communication approval signal and the target DMA address, and based on the communication request type being write, the DMA controller corresponding to the request port uses the physical memory address of the requesting device as the source address and the target DMA address as the target address to move / transfer the data in the physical memory address of the requesting device to the target DMA address.

[0207] In response to receiving the communication request and the target DMA address, and based on the communication request type being write, the DMA controller corresponding to the target port moves / writes / transfers the data in the target DMA address to the physical memory address corresponding to the target device connected to the target port.

[0208] In response to receiving the communication approval signal and the target DMA address, and based on the communication request type being read, the DMA controller corresponding to the target port moves / writes / transfers the data in the physical memory address corresponding to the target device connected to the target port to the target DMA address.

[0209] The target port responds to receiving the communication request and the target DMA address, and based on the communication request type being read, the DMA controller corresponding to the request port uses the target DMA address as the source address and the physical memory address of the requesting device as the target address, and moves / transfers the data in the target DMA address to the physical memory address of the requesting device.

[0210] Compared to the aforementioned data transfer methods, this data transfer method requires more complex initialization configuration and program setup because the DMA addresses accessed by each DMA controller are not fixed. However, in this data transfer method, the management module of the data exchange device does not need to transfer data from one DMA address to another, thus saving the data transfer process between the two DMA addresses, achieving higher data transfer efficiency, and further reducing data transfer latency.

[0211] It should be noted that the communication system, the inter-device P2P communication method based on a universal address space, and the data exchange device embodiments provided by the present invention have the same inventive concept as the above-mentioned inter-device P2P communication architecture embodiment based on a universal address space. The entire contents of the inter-device P2P communication architecture embodiment based on a universal address space are incorporated into the communication system, the inter-device P2P communication method based on a universal address space, and the data exchange device embodiments by introduction.

[0212] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0213] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A P2P communication architecture between devices based on a universal address space, characterized in that: It includes a data exchange device and a plurality of devices to be communicated electrically connected to the data exchange device, wherein the devices are configured with corresponding device IDs and physical memory addresses; The data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes a one-to-one correspondence between a device ID and a DMA address; the requesting device and the target device in the device perform P2P communication in the following manner: the requesting device sends a data transmission request to the data exchange device, wherein the data transmission request includes a requesting device ID and a target device ID; the management module determines a DMA address in the global address mapping table that matches the target device ID as a target DMA address; determines a physical memory address of the requesting device based on the requesting device ID, and determines a physical memory address of the target device based on the target device ID; and realizes communication between the physical memory address of the requesting device and the physical memory address of the target device through the target DMA address; The management module is further configured to transfer data in one DMA address to another DMA address; The DMA address corresponding to the same device ID communicates directly with the physical memory address, and the DMA addresses corresponding to different device IDs cannot communicate with the physical memory address; Implementing communication between the physical memory address of the requesting device and the physical memory address of the target device through the target DMA address includes: The physical memory address of the requesting device communicates with its corresponding requesting end DMA address; The management module controls the communication between the requester DMA address and the target DMA address; The target DMA address communicates with a target physical memory address corresponding to the target device requesting the device.

2. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: In response to receiving the data transfer request, the management module determines whether there is a DMA address in the global address mapping table that matches the target device ID, and if so, determines the matching DMA address as the target DMA address, sends a communication approval signal to the requesting device based on the requesting device ID, sends the data transfer request to the target device based on the target device ID, and controls the requesting DMA address to communicate with the target DMA address; In response to receiving the communication consent signal, the requesting device configures its physical memory address to communicate with its corresponding requesting end DMA address; In response to receiving the data transfer request, the target device configures its physical memory address to communicate with the corresponding target DMA address.

3. The inter-device P2P communication architecture based on universal address space according to claim 2, characterized in that: The data transmission request further includes a data transmission type, and the data transmission type includes writing data; If the data transmission type is writing data, the requesting device and the target device perform P2P communication in the following manner: Transferring the data in the physical memory address corresponding to the requesting device to the requesting end DMA address; The management module transfers the data in the requester DMA address to the target DMA address; The data in the target DMA address is transferred to the physical memory address corresponding to the target device.

4. The inter-device P2P communication architecture based on universal address space according to claim 2, characterized in that: The data transmission request further includes a data transmission type, and the data transmission type includes reading data; If the data transmission type is reading data, the requesting device and the target device perform P2P communication in the following manner: Transferring the data in the physical memory address corresponding to the target device to the target DMA address; The management module transfers the data in the target DMA address to the requesting end DMA address; The data in the requesting end DMA address is transferred to the physical memory address corresponding to the requesting device.

5. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: Also included is a DMA controller corresponding to each of the devices, wherein the DMA controller corresponding to the device is configured to control communication between the physical memory address of the device and the DMA address corresponding to the device; or, It also includes a DMA controller corresponding to each of the devices, and the DMA controller corresponding to the device is configured to control the communication between the physical memory address of the device and the target DMA address.

6. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: It also includes an uplink switch, wherein the uplink switch is electrically connected to each of the devices, and the uplink switch is electrically connected to the CPU; When the two devices need to perform P2P communication, the two devices do not perform P2P communication with the CPU through the uplink switch, but perform P2P communication through the data exchange device.

7. The inter-device P2P communication architecture based on universal address space according to claim 6, characterized in that: comprising a first device group and a second device group, wherein the first device group and the second device group each comprise a plurality of the devices; The plurality of devices in the first device group are electrically connected to a first uplink switch, respectively; the plurality of devices in the second device group are electrically connected to a second uplink switch, respectively; and the first uplink switch and the second uplink switch are electrically connected to a CPU, respectively; Each device in the first device group and the second device group is electrically connected to the data exchange device, and any two devices do not perform P2P communication through the first upstream switch, the second upstream switch, and the CPU, but perform P2P communication through the data exchange device; The first uplink switch and the second uplink switch are electrically connected to the same CPU; or the first uplink switch and the second uplink switch are electrically connected to different CPUs.

8. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: The DMA addresses corresponding to the devices do not overlap; and / or, The sizes of the DMA addresses corresponding to the devices are equal; and / or, The device is a PCIe device.

9. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: The size of the DMA address corresponding to the device is determined by: Determine the size of the physical memory address corresponding to each of the devices; Determine the memory usage of the device, where the memory usage = the size of the physical memory address corresponding to the device / the sum of the sizes of the physical memory addresses corresponding to all devices; The size of the DMA address corresponding to the device = the memory ratio × the total DMA resources.

10. The inter-device P2P communication architecture based on universal address space according to claim 1, characterized in that: The size of the DMA address corresponding to the device is determined by: determining a communication frequency between each of the devices and other devices; The greater the communication frequency, the larger the size of the DMA address corresponding to the device.

11. A communication system, characterized in that: The communication system includes the inter-device P2P communication architecture based on a universal address space as described in any one of claims 1 to 10.

12. A method for P2P communication between devices based on a universal address space, characterized in that: The device-to-device P2P communication architecture based on a universal address space according to claim 1 comprises the following steps: A data exchange device is configured to be electrically connected to a plurality of devices to be communicated, wherein the devices are configured with corresponding device IDs and physical memory addresses, and the data exchange device includes a global address mapping table and a management module, wherein the global address mapping table includes one-to-one correspondence between device IDs and DMA addresses; A requesting device in the device sends a data transmission request to the data exchange apparatus to communicate with a target device, the target device being another device different from the requesting device, the data transmission request including a requesting device ID and a target device ID; The management module determines a DMA address in the global address mapping table that matches the target device ID as the target DMA address; Determine the physical memory address of the requesting device according to the requesting device ID, and determine the physical memory address of the target device according to the target device ID; Communication between the physical memory address of the requesting device and the physical memory address of the target device is achieved through the target DMA address.

Images