Computing systems and computing methods

Abstract

The application provides a computing system and a computing method, which can be applied to the field of distributed technology. The computing system comprises: a plurality of computing devices, wherein any computing device is directly connected to another two computing devices in the plurality of computing devices through a host direct link to form a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology transmit data through the host direct link between each other; a plurality of shared memory devices directly connected to each other through a memory direct link, and the plurality of shared memory devices are respectively directly connected to corresponding computing devices in the plurality of computing devices through a device direct link.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed technology, in particular to a computing system and a computing method. BACKGROUND

[0002] In the face of huge computing demand, the computing power and storage resources of a single device are difficult to bear, and distributed computing systems emerge as the times require. However, in the related art, the network architecture of the distributed computing system has low data transmission speed, which limits the efficiency of model training. SUMMARY

[0003] In view of the above problems, the present application provides a computing system and a computing method.

[0004] According to an aspect of the present application, a computing system is provided, comprising: a plurality of computing devices, wherein any computing device is directly connected to another two computing devices in the plurality of computing devices through a host direct link, to constitute a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology perform data transmission through the host direct link between each other; a plurality of shared memory devices directly connected to each other through a memory direct link, the plurality of shared memory devices are respectively directly connected to corresponding computing devices in the plurality of computing devices through a device direct link, wherein the plurality of computing devices comprise a first computing device and a second computing device which are spaced apart in the ring communication topology, the plurality of shared memory devices comprise a first shared memory device directly connected to the first computing device and a second shared memory device directly connected to the second computing device, the first computing device is configured to directly write first data for the second computing device to the first shared memory device through the device direct link, the first shared memory device is configured to directly write the first data to the second shared memory device through the memory direct link, and the second computing device is configured to directly read the first data from the second shared memory device through the device direct link.

[0005] According to another aspect of the present application, a computing method is provided, applied to a computing system, the computing method comprising: transmitting data among a plurality of computing devices in the computing system, and performing a data processing task based on the transmitted data to obtain a data processing result, wherein any computing device is directly connected to another two computing devices in the plurality of computing devices through a host direct link to form a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology transmit data through the host direct link between each other; the computing system further comprises a plurality of shared memory devices directly connected to each other through a memory direct link, and the plurality of shared memory devices are respectively directly connected to corresponding computing devices in the plurality of computing devices through a device direct link, wherein the plurality of computing devices comprise a first computing device and a second computing device which are spaced apart in the ring communication topology, and the plurality of shared memory devices comprise a first shared memory device directly connected to the first computing device and a second shared memory device directly connected to the second computing device, so that the first computing device directly writes first data for the second computing device to the first shared memory device through the device direct link, the first shared memory device directly writes the first data to the second shared memory device through the memory direct link, and the second computing device directly reads the first data from the second shared memory device through the device direct link.

[0006] According to the embodiments of the present application, for the first computing device and the second computing device which are spaced apart in the ring communication topology, in the case that the first computing device needs to write first data to the second computing device, the first computing device can directly write the first data to the first shared memory device through the device direct link between the first computing device and the first shared memory device, and then the first shared memory device can directly write the first data to the second shared memory device through the memory direct link, so that the second computing device reads the first data from the second shared memory device through the device direct link. In this way, on the basis of using the shared memory to perform data transmission between the computing devices which are spaced apart in the ring communication topology, the present application simplifies the communication path between the computing devices which are spaced apart in the ring communication topology to a hardware-level access path, and realizes the communication between the computing devices which are spaced apart in the ring communication topology in the form of memory transfer, thereby at least partially solving the situation of transmission delay caused by using a switch to perform data transmission, and improving the data transmission speed of the distributed computing system. BRIEF DESCRIPTION OF DRAWINGS

[0007] The above content and other purposes, features and advantages of the present application will be more apparent from the following description of the embodiments of the present application with reference to the accompanying drawings.

[0008] Figure 1 A schematic diagram of a computing system according to an embodiment of the present application is shown.

[0009] Figure 2 A schematic diagram of a computing system according to another embodiment of the application is shown.

[0010] Figure 3 A schematic diagram of a computing system according to another embodiment of the application is shown.

[0011] Figure 4A A schematic diagram of a computing system according to another embodiment of the application is shown.

[0012] Figure 4B A schematic diagram of a computing system according to another embodiment of the application is shown.

[0013] Figure 4C A schematic diagram of a computing system according to another embodiment of the application is shown.

[0014] Figure 5 A schematic diagram of a computing network based on a computing system according to an embodiment of the application is shown.

[0015] Figure 6 A flowchart of a computing method according to an embodiment of the application is shown. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, the description is merely exemplary of the present application, and therefore is not to be taken in a limiting sense. Throughout the description, for the purpose of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present application. It will be apparent, however, to one skilled in the art that embodiments of the present application can be practiced without some or all of these specific details. In other instances, well known structures and functions have not been described in detail in order not to unnecessarily obscure the concept of the present application.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, mean the term "comprises," "comprising," "comprises," "comprising," "comprises," "comprising," and so forth.

[0018] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined herein. It should be noted that the terms used herein are not intended to have any ideologically or overly formal meanings, but should be interpreted in a manner consistent with the context of the present specification.

[0019] In the case of using expressions such as "at least one of A, B, and C", it generally should be understood that such expression is used to indicate only one of A, or B, or C, or any combination of A, B, and C. For example, "a system having at least one of A, B, and C" shall be interpreted to be a system that has A alone, or B alone, or C alone, or a system that has A and B, or A and C, or B and C, or a system that has A, B, and C, etc.

[0020] It should be noted that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. The terms "mounting", "connecting", "connection" should be interpreted broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium; it can be the communication inside two elements. The terms "parallel", "perpendicular", "equal" include the described case and the approximate case similar to the described case, and the range of the approximate case is within the acceptable deviation range, wherein the acceptable deviation range is determined by the ordinary skilled in the art considering the measurement being discussed and the error related to the measurement of the specific quantity (i.e. the limitation of the measurement system). For example, "parallel" includes absolute parallel and approximate parallel, and the acceptable deviation range of approximate parallel can be, for example, within 5°; "perpendicular" includes absolute perpendicular and approximate perpendicular, and the acceptable deviation range of approximate perpendicular can also be, for example, within 5°. "Equal" includes absolute equality and approximate equality, and the acceptable deviation range of approximate equality can be, for example, that the difference between the two equalities is less than or equal to 5% of either. For the ordinary skilled in the art, the specific meaning of the above terms in the present application can be understood on a case-by-case basis.

[0021] In the face of huge computing needs, the computing power and storage resources of a single device are difficult to bear, and distributed computing systems have emerged. However, in the related art, the data transmission speed of the distributed computing system is low, specifically, the data transmission speed between nodes in the distributed computing system is low.

[0022] On one hand, in a distributed computing system, subtasks of the same data processing task have relatively low independence from each other. For example, in an iterative training process, nodes in the distributed computing system need to communicate periodically to achieve gradient synchronization among the nodes. The independence between tasks of an artificial intelligence (AI) service is extremely low, and periodic communication is needed in the iterative training process to achieve critical gradient synchronization.

[0023] On the other hand, the running of an AI service highly depends on units such as a graphics processing unit (GPU) and a neural processing unit (NPU), and the concurrency of direct communication between the units is high. Based on this, it is necessary to improve the computing power of the distributed computing system by innovating related technologies of units such as the GPU and the NPU.

[0024] On this basis, the technical innovation around units such as the GPU and the NPU has achieved remarkable results, and the computing power of the distributed computing system has been greatly improved. However, while the computing power bottleneck is gradually overcome, due to the strong synchronization of AI service communication, the final performance of any communication transaction depends on the slowest communication link, and the network link of data transmission and interaction becomes a new performance constraint point. In this case, the computing network topology structure of the distributed computing system in some solutions has been difficult to meet the efficient data transmission and low-latency interaction requirements of distributed machine learning.

[0025] In some solutions, a distributed computing system uses a switch to achieve large-scale expansion of a computing cluster, such as a Fat-tree architecture and a Spine-leaf architecture of a CLOS architecture, and each host is interconnected through a switch. However, in a high-frequency communication scenario, this architecture has obvious bottlenecks. Taking a model training task as an example, the low-latency and high-bandwidth communication requirements of the model training task are essentially contradictory to the routing mechanism of the switch: each data transmission needs to pass through multiple levels of switch forwarding, and the number of switch hops directly leads to a linear increase in network delay. The number of switch hops can refer to the number of switches that data passes through in the process of transmitting data. In a many-to-one communication mode, such as a distributed computing system for training model parameters, in the case of a large number of concurrent requests converging to a single receiving node, Incast (Incast) and other congestion problems are easily triggered. For example, when multiple computing nodes send data to the same computing node through a switch port, the data of the multiple computing nodes needs to be aggregated to the target port buffer area of the switch first, and then forwarded to the receiving computing node. If the concurrent traffic exceeds the capacity of the target port buffer area, data packet loss or retransmission may occur, which in turn causes a delay to skyrocket.

[0026] In some model training algorithms, such as an Allreduce algorithm, synchronization of model parameters can be achieved through multiple rounds of data aggregation and broadcasting. On this basis, each round of communication relies on the efficient forwarding of switches. When the scale of the computing cluster is expanded, the bandwidth competition of the switch port is intensified, and the queue blocking and ACK (Acknowledgement) backlog problems caused by the cache mechanism are further worsened, eventually resulting in a significant increase in synchronization delay in the training process. In addition, the depth of the cache area of the switch and the mismatch between the port rate of the switch and other hardware limitations have become key bottlenecks restricting the performance improvement of distributed computing systems.

[0027] For example, Ethernet communication needs to go through the entire network protocol stack, resulting in a delay of 2-4 microseconds. The network protocol stack may include, for example, an application layer, a Transmission Control Protocol / Internet Protocol (TCP / IP) layer, a network card driver, and a physical network card. Alternatively, RDMA communication can be achieved by bypassing the kernel protocol stack and using a host channel adapter. When using InfiniBand technology, the delay can be reduced to about 1 microsecond, and when using RoCEv2 technology, the delay can be reduced by about 1-2 microseconds.

[0028] Therefore, the present application provides a computing system. The computing devices of the computing system can directly read and write a shared memory device to perform data transmission via the shared memory device, so that the communication path can be simplified to hardware-level memory access, and the delay can be compressed to 300-400 nanoseconds. In some schemes, since the number of switch hops required by the data transmission path is large in the scheme using a switch, the communication delay is high. In contrast, the computing system of the present application can greatly reduce the communication delay. In the present application, the number of switch hops for data transmission between at least part or even all computing devices is 0. The computing system of the present application is described below in conjunction with the accompanying drawings.

[0029] Figure 1 A schematic diagram of a computing system according to an embodiment of the present application is shown.

[0030] As Figure 1As shown, the computing system of the embodiment can include a computing device and a shared memory device. It should be understood that the present application is not limited thereto, and in other embodiments of the present application, the computing system can further include other devices, such as a control device, etc., which are not limited by the present application.

[0031] The computing device can be configured to perform a computing task to obtain computing data. Further, in embodiments of the present application, the computing device can be multiple. In the multiple computing devices, any one of the computing devices is directly connected to two other computing devices of the multiple computing devices to form a ring communication topology based on the multiple computing devices, wherein adjacent computing devices in the ring communication topology perform data transmission through the direct connection therebetween.

[0032] Further, the computing device can include one or more computing nodes. The data transmitted by the multiple computing devices can be the computing data calculated by the computing nodes, but the present application is not limited thereto, and the multiple computing devices can also transmit other data.

[0033] In other embodiments of the present application, the data of the multiple computing devices can also be transmitted via other devices. For example, in embodiments of the present application, the computing data of the multiple computing devices can be transmitted through the shared memory device.

[0034] The shared memory device can be used as a common memory of the multiple computing devices. Specifically, any one of the computing devices can perform a data write operation or a data read operation on the shared memory device. However, embodiments of the present application are not limited thereto, and in other embodiments of the present application, the multiple computing devices can include a first computing device and a second computing device. The first computing device and the second computing device can refer to two computing devices that are spaced apart in the ring communication topology. On this basis, in the case that the first computing device writes data into the shared memory device, the second computing device can read the data written into the shared memory device by the first computing device from the shared memory device. In this way, data transmission between the first computing device and the second computing device can be achieved.

[0035] For example, the shared memory device can be implemented based on a Compute Express Link (CXL) multi-head device. In which, a "head" can refer to a port. Compared with the delay of hundreds of nanoseconds of the switch solution, the CXL multi-head device can compress the communication delay, so that the data transmission performance is improved by orders of magnitude. Specifically, the shared memory device can implement hardware-level memory access, and the data of the computing device is transmitted through the shared memory device, without involving the whole process of network forwarding on the switch link, so there is no problem of rate mismatch of the transmission port of the switch. For example, the switch as an existence independent of the computing device will cause the problem of rate mismatch. However, the CXL multi-head device as a memory expansion of the computing device can abstract the computing device and the corresponding CXL multi-head device as a whole hardware device connected through a bus, so that the rates are relatively matched. Thus, the data transmission congestion problem caused by the above-mentioned port rate mismatch of the switch can be avoided.

[0036] Further, in the embodiments of the present application, the shared memory device can be multiple. On this basis, the shared memory device can be connected with the corresponding computing device. In addition, the multiple shared memory devices can be connected with each other. For example, the multiple shared memory devices can be directly connected with each other through a memory direct connection link, and the multiple shared memory devices can also be directly connected to the corresponding computing device in the multiple computing devices through a device direct connection link respectively. In this way, the multiple computing devices and the multiple shared memory devices constitute the ring communication topology shown in the present application. Figure 1

[0037] ​On this basis, a first computing device in the plurality of computing devices can be connected to a first shared memory device in the plurality of shared memory devices. In this way, the first computing device can read and write data to the first shared memory device. Similarly, a second computing device in the plurality of computing devices can be connected to a second shared memory device in the plurality of shared memory devices. In this way, the second computing device can read and write data to the second shared memory device. However, embodiments of the present application are not limited thereto, and in embodiments of the present application, the first shared memory device can also read and write data to the second shared memory device to achieve data transmission between the first computing device and the second computing device. Similarly, the second shared memory device can also read and write data to the first shared memory device, which will not be described herein. For example, the shared memory device can be deployed with a corresponding controller, which can respond to a control instruction to control the shared memory device to output the stored data via the port to achieve data transmission, and the present application will be implemented according to the concept, which will not be described herein. For example, the first computing device can directly write first data for the second computing device to the first shared memory device through the device direct link, the first shared memory device can directly write the first data to the second shared memory device through the memory direct link, and the second computing device can directly read the first data from the second shared memory device through the device direct link. Wherein, the first data can be data that needs to be sent by the first computing device to the second computing device, etc.

[0038] Based on this, in the present application, for the first computing device and the second computing device that are spaced apart in the ring communication topology, in the case that the first computing device needs to transmit first data to the second computing device, the first computing device can directly write the first data to the first shared memory device through the device direct link between the first computing device and the first shared memory device, and then the first shared memory device can directly write the first data to the second shared memory device through the memory direct link, so that the second computing device reads the first data from the second shared memory device through the device direct link. In this way, on the basis of using the shared memory device to perform data transmission between the computing devices that are spaced apart in the ring communication topology, the present application simplifies the communication path between the computing devices that are spaced apart in the ring communication topology to a hardware-level access path, and realizes communication between the computing devices that are spaced apart in the ring communication topology in the form of memory transfer, at least partially solving the situation of transmission delay caused by using a switch to perform data transmission, and improving the data transmission speed of the distributed computing system.

[0039] Figure 2 A schematic diagram of a computing system according to another embodiment of the present application is shown. It is to be noted that, for ease of identification, lines with occlusions are shown in different shades of gray in Figure 2 the same below, which will not be described again.

[0040] As Figure 2 illustrated, in the embodiments of the present application, any computing device can include a plurality of computing nodes. For example, the computing device can be a host device such as a server, and the computing nodes can be graphics processing units. The plurality of computing nodes can be electrically connected through, for example, a bus. However, it should be understood that the present application is not limited thereto, and in other embodiments of the present application, the computing nodes can also be other units that can perform data processing tasks, such as neural network processing units, etc.

[0041] The plurality of computing devices can be communicatively connected to form a ring communication topology for data transmission of the plurality of computing devices. Further, in the ring communication topology, the computing devices adjacent to each other can be communicatively connected through network cards. The number of high-performance network cards of any computing device can be at least two, and the specific number can be determined according to the demand, the use of the network card, and the virtualization of the network card. In the ring communication topology, the nodes are the computing devices. The edges can be communication links formed by the network cards.

[0042] The shared memory devices can be multiple. For the plurality of shared memory devices, the shared memory devices can be communicatively connected through network cards, or can also be electrically connected through a bus. Since in the process of communication through the network card, it is necessary to perform operations such as encapsulation and analysis of data packets, in order to ensure the communication speed of the plurality of shared memory devices, it is preferred in the present application to use a bus to connect the plurality of shared memory devices.

[0043] Any shared memory device can be connected to a plurality of computing devices. For example, the corresponding computing devices and shared memory devices can be electrically connected through a bus.

[0044] Further, the storage space of any shared memory device can include a storage area. For example, the storage area can include a storage area for storing data of the computing device connected to any shared memory device. However, it should be understood that the embodiments of the present application are not limited thereto, and in another embodiment of the present application, the storage space of the shared memory device also includes a storage area for storing data of other shared memory devices and / or another computing device in the plurality of computing devices, except for the computing device corresponding to the shared memory device. Specifically, the storage area of any shared memory device can also include a storage area for storing data of a computing device that is not connected to the any shared memory device but is connected to other shared memory devices in the ring communication topology. In still another embodiment of the present application, the storage space of the shared memory device connected to any computing device includes a plurality of storage areas. The plurality of storage areas are respectively configured to store data of a plurality of computing nodes of any computing device.

[0045] In the embodiments of the present application, the computing device connected with any shared memory device or other shared memory device can read and write the storage area of any shared memory device according to the address of the storage area in any shared memory device. For example, the computing node connected with any shared memory device can read and write the storage area of any shared memory device according to the address of the storage area in any shared memory device. It should be understood that in the switch scheme, when multiple computing devices send data to the same target node through the switch port, all the data need to be aggregated to the target port buffer of the switch first, and then forwarded to the receiver. If the concurrent traffic exceeds the capacity of the port buffer, it will cause packet loss, retransmission, and then cause a delay surge.

[0046] Specifically, if the high-performance network card port rate (such as 100 Gbps) of the computing device is higher than the switch port rate (such as 40 Gbps) connected, the amount of data sent by the network card will exceed the receiving capacity of the switch port, and the excess data will be temporarily stored in the cache queue of the switch. When the cache queue is full, it will trigger the problem of “packet loss and retransmission” or “head-of-line blocking”, which will cause a sharp increase in data transmission delay. However, in the present application, the computing device as the sender directly writes data into the independent block of the shared memory device, and the receiver does not need to wait for the data to be forwarded to the local through a single switch port, but directly reads the data from the independent block of the shared memory device pool. This way achieves memory partitioning rather than port-level data aggregation. In this way, the present application can be applied to high-concurrency scenarios and can solve the above problems.

[0047] On this basis, in an embodiment of the present application, the first shared memory device includes a plurality of first storage areas, and the plurality of first storage areas are respectively associated with a plurality of shared memory devices. For example, the storage addresses of the plurality of first storage areas can be associated with the plurality of shared memory devices. In this way, the first shared memory device can directly write the first data into the second shared memory device in response to the first computing device writing the first data into the target storage area associated with the second shared memory device in the plurality of first storage areas. On this basis, the first shared memory device can directly write the data stored in any first storage area into another shared memory device associated with the first storage area, realizing the memory direct writing delivery of data and improving the data transmission speed.

[0048] In another embodiment of the present application, the first shared memory can include a first port and a second port. The first port is connected to the first computing device, and the second port is connected to the second shared memory device. The first port and the second port are associated with a target first storage area in the plurality of first storage areas. For example, the port identifier of each of the first port and the second port can be stored in association with the storage address of the target first storage area in the shared memory device. In this way, the first shared memory device can directly write the first data into the second shared memory device via the second port in response to the first computing device writing the first data into the target storage area via the first port. In this way, the first shared memory device can directly output the data in a storage area via another port associated with the storage area in the case that the data is written into the storage area via a certain port, so that the data can be directly written into another shared memory device, thereby realizing memory direct write transmission of the data and improving the data transmission speed. However, the embodiments of the present application are not limited thereto, and the first storage area can further include a plurality of sub-areas. Different sub-areas can be associated with different shared memory devices, so that the data stored in a sub-area can be directly written into the shared memory device associated with the sub-area in the case that the data is stored in the sub-area.

[0049] In addition, in another embodiment of the present application, at least two computing devices connected to any shared memory device are spaced apart from each other in the ring communication topology. For example, the computing devices connected to any shared memory device can be substantially uniformly spaced apart from each other in the ring communication topology, but in some embodiments, the computing devices connected to the shared memory device can also not be uniformly spaced apart in the ring communication topology, which is not limited in the present application. Further, any shared memory device includes a second storage area, and the second storage area is associated with at least two computing devices connected to the shared memory device. The association manner is similar to the foregoing description, which is not repeated here. In this way, in the case that one of the at least two computing devices writes second data for other computing devices in the at least two computing devices into the second storage area, the other computing devices read the second data from the second storage area. The second data can be data that needs to be sent by one of the at least two computing devices to the other computing devices, and the like.

[0050] In this way, the shared memory device can realize memory transmission between the at least two computing devices connected thereto, thereby improving the data transmission efficiency. However, the embodiments of the present application are not limited thereto, and the storage area can further include a plurality of sub-areas. One sub-area can be associated with at least two computing devices, so that the shared memory device can directly write data into another computing device associated with the sub-area in the case that one of the at least two computing devices writes the data into the sub-area. The computing devices associated with different sub-areas are different, which is not repeated here.

[0051] In this embodiment, the shared memory devices and computing devices can be connected according to the principle of uniform division. For example, any shared memory device has a predetermined number of ports, which correspond to the "heads" of the CXL multi-head device described above. Each port is used to connect to the corresponding computing device, and the number of shared memory devices is equal to the total number of computing devices divided by the predetermined number and rounded up. For example, in one embodiment of this application, if the number of ports of the shared memory devices is 3 and the total number of computing devices is 9, then the number of shared memory devices can be 3. However, it should be understood that this embodiment is not limited to this. In another embodiment of this application, if the number of ports of the shared memory devices is 3 and the total number of computing devices is 8, then the number of shared memory devices can be 3. It should be understood that when 3 is divided by 8, the remainder is 2, that is, there are 2 remaining computing devices. Therefore, a shared memory device with 3 ports should be added to connect the remaining 2 computing devices. In this way, the number of shared memory devices can still be 3, where 3 is the rounded-up value.

[0052] Specifically, multiple shared memory devices each have a port for connecting to a corresponding computing device, and each shared memory device has the same number of such ports, so that the multiple shared memory devices can connect to the same number of computing devices via the same number of ports. For example, in Figure 2 In this configuration, any shared memory device has multiple (e.g., three) ports, which are connected to multiple computing devices (e.g., three computing devices). The purpose is to ensure that any shared memory device is logically connected to the same number of computing devices. This simplifies the management of the ring communication topology. Furthermore, multiple shared memory devices may also have ports for connecting to each other, which will not be elaborated upon here.

[0053] Further, in the data processing task, a communication link between the plurality of computing devices can be constructed based on the plurality of shared memory devices. Since any shared memory device has a corresponding port to receive or send data of the computing device or the shared memory device, the data received via the port of any shared memory device can be directly stored to the storage area corresponding to the port, or the data of the corresponding storage area can be directly output based on the read request received via the port of any shared memory device. In this way, the Incast problem caused by using the switch for data transmission is at least partially avoided, the redundancy of the communication link between the plurality of computing devices is improved, and the data transmission bandwidth of the plurality of computing devices is improved. At the same time, the utilization of the plurality of ports of the plurality of shared memory devices can be parallel for data transmission. For example, the first computing device and the third computing device can write data to the first shared memory device in parallel. The first shared memory device can write the data from the first computing device and the data from the third computing device to the second shared memory device. The second computing device can read the data from the first computing device and the data from the third computing device from the second shared memory device. On this basis, the shared memory device can perform data transmission with two or more computing devices, thereby improving the utilization of the plurality of shared memory devices in high-bandwidth transmission. In this way, the execution efficiency of the data processing task is improved. On this basis, the centralized switch for data transmission is removed, and the distributed computing system is adopted to solve the congestion problem of the switch architecture in the many-to-one communication scenario in some solutions.

[0054] In addition, for the principle of "uniform division" described above, in the case where the number of computing devices is insufficient, the problem can be solved by adding virtual nodes. The following will be described in combination with Figure 3 .

[0055] Figure 3 A schematic diagram of a computing system according to another embodiment of the present application is shown.

[0056] As Figure 3 shown, in the embodiment of the present application, the plurality of computing devices can be communicatively connected to each other to constitute a ring communication topology for data transmission of the plurality of computing devices. Further, in the ring communication topology, the computing devices adjacent to each other can also be communicatively connected through network cards. Any computing device can include a plurality of connected computing nodes.

[0057] The shared memory device can be multiple. Any shared memory device can be connected to a plurality of computing devices. For example, the corresponding computing devices and shared memory devices can be electrically connected through a bus.

[0058] In the embodiment of the present application, the shared memory devices and the computing devices are still connected in accordance with the principle of uniform division. However, in this embodiment, the number of ports of the shared memory devices is 3, and the total number of the computing devices is 8, and then the number of the shared memory devices can be 3. It should be understood that in the case of 8 divided by 3, the remainder is 2, that is, the remaining computing devices are 2. In this case, one shared memory device with 3 ports should also be supplemented to connect the remaining 2 computing devices.

[0059] At this time, for the third shared memory device, there is still one port not connected to the computing device. In this case, the third port of the shared memory device can be configured to connect a virtual node. The virtual node is not the actual computing device described above. That is, in the case that the third shared memory device is configured to connect a virtual node, the above-mentioned port of the third shared memory device is still not actually connected to the computing device. In this case, the third shared memory device can be configured to connect a virtual node. Figure 3 In the ring communication topology shown, the computing devices located on both sides of the virtual node (or also referred to as a virtual device), that is, the two computing devices adjacent to the virtual device, can be directly connected in communication. In one way of configuring the virtual node in the present application, the data that should be written to the virtual node can be changed to be written to other computing devices, or the data that should be read from the virtual node can be changed to be read from other computing devices through logical configuration by software. However, it should be understood that the embodiments of the present application are not limited thereto, and other ways of adding virtual nodes can also be used in the present application, which will not be described here. In this way, the process of building a ring communication topology can be simplified.

[0060] In addition, in the embodiments of the present application, at least one of the following communication connections can be included: communication connection between at least two shared memory devices through a network card or a bus; communication connection between corresponding computing devices and shared memory devices through a bus; communication connection between computing devices through a network card; communication connection between computing nodes in the same computing device through a bus. Among them, the bus can support the CXL protocol.

[0061] On this basis, a plurality of shared memory devices are connected to each other through a memory direct link, and the memory direct link supports a memory interconnection protocol. A plurality of computing devices are connected to each other through a host direct link. The host direct link supports a protocol different from the memory interconnection protocol. For example, the host direct link can work cooperatively based on multiple layers of the TCP / IP protocol stack. Any computing device and the corresponding shared memory device are connected through a device direct link. The device direct link supports the memory interconnection protocol.

[0062] In this way, in the case that the communication between at least two shared memory devices is disconnected, the at least two shared memory devices disconnected in communication can communicate via another link, which will be described below. Figure 4A In this way, in the case that the communication between at least two shared memory devices is disconnected, the at least two shared memory devices disconnected in communication can communicate via another link, which will be described below.

[0063] Figure 4A A schematic diagram of a computing system according to another embodiment of the present application is shown. It is to be noted that in the present application, a line with intersecting short lines (e.g. "+", but not limited thereto, also can be "x") represents a faulty line, and an arrow represents the direction of data transmission, and the same applies hereinafter, which will not be repeated herein.

[0064] As Figure 4A shown, in the state of a faulty memory direct link between the first shared memory device and the second shared memory device, the first computing device and the second computing device are configured to transmit data between each other via a device direct link between the first computing device and the first shared memory device, another memory direct link between the first shared memory device and the third shared memory device, yet another memory direct link between the third shared memory device and the second shared memory device, and another device direct link between the second shared memory device and the second computing device.

[0065] For example, a computing node of the first computing device can send a read instruction to the first shared memory device. The first shared memory device can send an instruction to the third shared memory device under the control of the read instruction, so as to make the third shared memory device read data indicated by the instruction from the second shared memory device, and then write the data into the first shared memory device.

[0066] For another example, upon receiving a read instruction, the first shared memory device can determine whether the memory direct link between the first shared memory device and the second shared memory device is faulty based on a read address indicated by the read instruction, so as to send an instruction carrying a memory address of the second shared memory device to the third shared memory device in the case that the memory direct link between the first shared memory device and the second shared memory device is determined to be faulty, so as to make the third shared memory device perform data reading on the second shared memory device via the memory direct link between the second shared memory device and the third shared memory device. The process of data writing is similar, and a computing node of the first computing device can send a write instruction, which will not be repeated herein.

[0067] Conversely, a computing node of the first computing device can perform read and write on the first shared memory device via the above-mentioned yet another memory direct link between the third shared memory device and the second shared memory device, and the above-mentioned another memory direct link between the first shared memory device and the third shared memory device. The specific process is similar to the above, which will not be repeated herein.

[0068] Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device.

[0069] Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device. Figure 4B Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device.

[0070] Figure 4B A schematic diagram of a computing system according to another embodiment of the present application is shown.

[0071] As shown in FIG. 1, in this embodiment, in a state where the device direct link between the first computing device and the first shared memory device is faulty, the computing node of the first computing device reads and writes data to the first shared memory device via a host direct link between the first computing device and the third computing device, a device direct link between the third computing device and the corresponding third shared memory device, and a memory direct link between the third shared memory device and the first shared memory device. Figure 4B Specifically, the first computing device can send a read instruction to the third computing device via the host direct link between the first computing device and the third computing device, so that the third computing device sends a read instruction carrying a memory address of the first shared memory device to the third shared memory device via the device direct link between the third computing device and the corresponding third shared memory device, so that the third shared memory device reads data from the first shared memory device via the memory direct link between the third shared memory device and the first shared memory device. The data writing process of the first computing device to the first shared memory device is similar to this, and is not described herein.

[0072] Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device.

[0073] Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device. Figure 4C Further, embodiments of the present application are not limited thereto, and in another embodiment of the present application, the first computing device can also read and write data to the second shared memory device via a host direct link between the first computing device and the third computing device, and a device direct link between the third computing device and the third shared memory device.

[0074] Figure 4C A schematic diagram of a computing system according to another embodiment of the present application is shown.

[0075] As shown in FIG. 1, in this embodiment, in a state where the device direct link between the first computing device and the first shared memory device is faulty, the computing node of the first computing device reads and writes data to the first shared memory device via a host direct link between the first computing device and the third computing device, a device direct link between the third computing device and the corresponding third shared memory device, and a memory direct link between the third shared memory device and the first shared memory device. Figure 4CAs shown, in the state of the host direct link failure between the third computing device and the fourth computing device adjacent to each other in the ring communication topology in this embodiment, the third computing device and the fourth computing device perform data transmission between each other via the device direct link between the third computing device and the corresponding third shared memory device, the memory direct link between the third shared memory device and the corresponding fourth shared memory device of the fourth computing device, and another device direct link between the fourth shared memory device and the fourth computing device.

[0076] For example, the third computing device can send a data transmission instruction to the third shared memory device via the device direct link between the third computing device and the corresponding third shared memory device, so that the third shared memory device sends the data transmission instruction to the fourth shared memory device, and then the fourth shared memory device can transmit the data carried by the data transmission instruction to the fourth computing device. Conversely, the same applies, which is not described herein.

[0077] In addition, in the embodiments of the present application, any computing device can be equipped with multiple high-performance network cards. For example, the network cards can be 4, but are not limited thereto. Moreover, the network cards can be duplexed and support virtualization. On this basis, multiple host direct links can be configured between the computing devices adjacent to each other. The third computing device and the fourth computing device are configured to switch to other communication links in the multiple host direct links for communication in the case of failure of the current communication link in the multiple host direct links. The multiple host direct links can include a primary link and a backup link, wherein the primary link can be a link with the highest communication priority, used for daily communication. In the case of failure of the primary link used for communication in the multiple host direct links, the computing devices adjacent to each other can automatically switch to communication by another non-failed backup link.

[0078] It is additionally explained that the switching mode of the above-mentioned link of the present application is only as an example, and in fact, communication can also be based on the above-mentioned other links of the present application, which is not listed one by one. It is additionally explained that for the communication between the shared memory devices and the shared memory devices, and the communication between the computing devices and the corresponding shared memory devices, the bus link is preferred to be implemented to reduce the delay as much as possible. Moreover, in the present application, multiple computing devices are supported to read and write the shared memory device in parallel.

[0079] Based on the above, the ring communication topology of the present application has a rich fault-tolerant mechanism, which can ensure that in the case of a faulty communication link, other communication links automatically take over the tasks originally performed by the faulty communication link to at least partially ensure the continuity of the communication process and the continuity of the communication, thereby improving the fault-tolerability and reliability of the system. Compared with some schemes that use multiple switches to forward data, the scheme of the present application can reduce the overall communication delay of the computing system by 1-2 orders of magnitude.

[0080] In addition, the present application can also realize the elastic expansion of the transmission link of the ring communication topology by increasing the number of shared memory devices, thereby improving the link fault-tolerability. Specifically, the ring communication topology in the present application has high flexibility in structure and high scalability. In this way, by reasonably planning the direct link between the shared memory device and the computing device, the scale of the computing cluster can be easily expanded without significantly increasing the complexity and cost of the system.

[0081] The architecture of the computing system of the present application has been introduced above, and the data processing tasks performed by the computing system will be introduced below. For example, the data processing task can include a model training task. For example, the model training task can be implemented based on a 2D-Torus AllReduce (i.e., two-dimensional torus all-reduce) algorithm. The computing system is configured to perform the model training task, and each computing node of any computing device obtains its own gradient data in the model training task, which represents data for optimizing the model parameters of the model. The following will be described in conjunction with Figure 5 .

[0082] Figure 5 A schematic diagram of a computing network based on a computing system according to an embodiment of the present application is shown.

[0083] In Figure 5 , a computing network based on computing nodes and shared memory devices is shown. For example, the computing system can include 9 computing devices, and any computing device can include 4 computing nodes. On this basis, it can be known from Figure 5 that each row corresponds to a computing device, and each column corresponds to a group of computing nodes. In the present embodiment, the computing nodes spaced apart from each other can communicate via shared memory devices, and the computing nodes adjacent to each other can directly communicate.

[0084] Specifically, in the present embodiment, the gradient data on the multiple computing nodes of any computing device can be locally reduced to obtain local gradient data of each of the multiple computing devices. This process corresponds to the horizontal computing process of the computing network shown in Figure 5 .

[0085] Then, the local gradient data of each of the plurality of computing devices can be globally reduced among the plurality of computing devices, and the global gradient data is obtained on the plurality of computing devices, respectively, wherein the first computing device and the second computing device are spaced apart from each other in the ring communication topology, and the local gradient data of each of the first computing device and the second computing device is transmitted between the first computing device and the second computing device through the first shared memory device and the second shared memory device. For example, the first data described above can include the local gradient data of the first computing device. The process corresponds to the vertical computing process of the computing network shown in FIG. 1. Figure 5 The vertical computing process of the computing network shown in FIG. 1.

[0086] Specifically, the gradient data on any computing node is divided into a predetermined number of gradient blocks. For example, the data dimension ranges corresponding to the predetermined number of gradient blocks are different from each other. The plurality of computing nodes in the same computing device can correspond to different data dimension ranges. On this basis, the data dimension ranges of the local computing data reduced in the process of topology reduction are the same. It should be noted that the predetermined number here corresponds to the number of computing nodes, which is different from the predetermined number of ports of the shared memory device described above. In order to distinguish, they can be called first predetermined data or second predetermined number, respectively.

[0087] Further, the local reduction can include: passing the gradient blocks between the plurality of computing nodes in any computing device, so that any node has a local gradient block of the corresponding dimension, which can be used to calculate the corresponding local gradient data, and the same applies to others, which will not be repeated. For example, the same computing device can include computing node 11, computing node 12, computing node 13 and computing node 14. Specifically, computing node 11, computing node 12, computing node 13 and computing node 14 can be four computing nodes in the same row of the computing network in FIG. 1. Figure 5 The four computing nodes in the same row of the computing network in FIG. 1.

[0088] On this basis, the computing node 11 can send the gradient block of the 1st dimension to the computing node 12; the computing node 12 can perform computation on the received gradient block of the 1st dimension and the locally stored gradient block of the 1st dimension (in a preset manner, which is not limited here, and the same applies below), to obtain a computed gradient block of the 1st dimension, and then send the computed gradient block of the 1st dimension to the computing node 13; the computing node 13 can perform computation on the received gradient block of the 1st dimension and the locally stored gradient block of the 1st dimension, to obtain a computed gradient block of the 1st dimension, and then send the computed gradient block of the 1st dimension to the computing node 14; the computing node 14 can perform computation on the received gradient block of the 1st dimension and the locally stored gradient block of the 1st dimension, to obtain a computed gradient block of the 1st dimension, and then send the computed gradient block of the 1st dimension to the computing node 11. In this way, the computing node 11 can take the computed gradient block of the 1st dimension received from the computing node 14 as the local gradient block after local reduction. The gradient blocks of other dimensions of other computing nodes can also be computed in a similar manner, which is not described here. In an embodiment, after the gradient on the N computing nodes in the computing device is split into N small blocks and iteratively reduced, the gradient block on the last computing node after iteration will have a complete gradient of the same dimension. The gradient block of any dimension of any computing node will contain the sum of all gradients of the corresponding block in the N computing nodes. N is a positive integer greater than 1, corresponding to 4 in the above embodiment.

[0089] Further, the global reduction can include: passing the gradient blocks of the same dimension between the corresponding nodes having the gradient blocks of the same dimension in the respective computing nodes of the plurality of computing devices, so that any computing node in any computing device has a global gradient block of the corresponding dimension. For example, the 9 computing devices can respectively include the computing node 11, the computing node 21, the computing node 31, the computing node 41, the computing node 51, the computing node 61, the computing node 71, the computing node 81, and the computing node 91 (hereinafter referred to as “computing nodes 11 to 91” for simplicity, which is not described here). The computing nodes 11 to 91 can be 9 computing nodes in the same column of the computing network in the Figure 5 For example, the computing node 11 can send the local gradient block to other computing nodes, and pass and compute the gradient blocks among the computing nodes 11 to 91 in a similar manner as described above, so as to obtain the global gradient block of the 1st dimension, and then the global gradient block of the 1st dimension can be returned from the computing node 91 to the computing node 11. In this way, iteration can complete the reduction of the global gradient block of each data dimension range.

[0090] On this basis, for the respective nodes in the computing nodes of the plurality of computing devices that have the same dimension gradient block and are spaced apart from each other, the same dimension gradient block can be transmitted via the corresponding shared memory device, and for the respective nodes in the computing nodes of the plurality of computing devices that have the same dimension gradient block and are adjacent to each other, the same dimension gradient block can be directly transmitted, so that any computing node in any computing device has a global gradient block of the corresponding dimension. In this way, the data on the computing nodes of any computing device is vertically globally reduced within the cluster, so that the dimension gradient block in which any computing node is located includes the sum of all gradients corresponding to the block in the vertical computing node. Moreover, by using different communication modes (which can be referred to as a hybrid mode) according to the spacing relationship between the nodes as described above, the delay can be compressed to 300-400 nanoseconds.

[0091] After obtaining the global gradient block, the model training task can further include globally converging the global gradient blocks of the respective dimensions of the plurality of computing nodes of the computing device within any computing device to obtain final gradient information. Specifically, the plurality of computing nodes within the same computing device can broadcast the global gradient blocks of the corresponding dimensions within the computing device, so that the same computing node has global gradient blocks of multiple data dimension ranges, so that the global gradient block can be converged, and any computing node has final gradient information. In this way, the gradient on the dimension gradient block in which any computing node is located is copied to the corresponding dimension gradient block within the computing device for global aggregation. Based on this, the above model training task can be completed by repeatedly iterating the above process.

[0092] On this basis, for the above process, especially the above global reduction and global convergence processes with high concurrency and high bandwidth, since the computing nodes of the computing devices that are spaced apart from each other can communicate via the corresponding shared memory device, and the computing nodes of the computing devices that are adjacent to each other directly communicate, the above problems caused by switch data transmission can be at least partially avoided, the transmission path of the above distributed algorithm is optimized, the data synchronization delay and switch hop count of the above computing process are reduced, the communication efficiency is improved, and the algorithm convergence is accelerated. On this basis, the application breaks through the congestion bottleneck of the switch architecture, and the communication gain effect of the ring communication topology of the application has a significant improvement compared to the switch architecture.

[0093] On this basis, the shared memory device is applied to a rack-level server scheme, and is very suitable for small and medium-sized high-performance clusters, such as single-rack or inter-rack artificial intelligence and cloud computing memory workloads, and can maximize the transmission speed between computing devices. The shared memory device is used for direct interconnection between two computing devices spaced apart from each other, and a high-efficiency and low-cost efficient memory sharing scheme is constructed. The shared memory device serves as a data transmission path for cross-node data exchange, supports multiple computing device parallel reading and writing, and improves model training efficiency.

[0094] For small and medium-sized artificial intelligence training clusters in a single rack, 8 servers are equipped with 8 image processing units, or can be called computing cards or acceleration cards, and need to support distributed training of large language models, require low-latency gradient synchronization (such as the above-mentioned reduction operation and aggregation operation, etc.) and high-bandwidth data interaction, and avoid congestion problems caused by switches. On this basis, the computing system of the present application disperses concurrent requests in direct link and shared memory device, thereby meeting the above-mentioned requirements, and at least partially solving the Incast congestion problem caused by the switch. In the computing system of the present application, multiple computing devices can adopt a direct connection design, any computing device is equipped with 2 high-performance network cards, and a closed ring topology is formed through direct connection of the network cards, that is, any computing device is directly connected with the adjacent 2 computing devices, and “direct connection ring” communication between the computing devices without switches is realized.

[0095] For servers across racks (which can be applied to scenarios such as hybrid artificial intelligence training and cloud computing loads), it is necessary to support dynamic expansion of cluster size and have link fault tolerance capability, that is, a single link fault does not affect the overall communication. For this, the computing system of the present application also meets this kind of requirement.

[0096] In addition, for the distributed database cluster of the high-frequency trading system, millisecond-level data synchronization (such as order information, account balance) needs to be supported, and high-concurrency read-write and low-latency data consistency guarantee are required. To this end, the computing system of the present application also meets such a demand. For example, 6 computing devices can be directly connected in a ring through the respective deployed single high-performance network card, meeting the low-latency direct communication between nodes (for example, synchronizing the order data of adjacent computing devices). And 2 4-port CXL multi-head devices can be selected as a global shared memory pool. The multi-head device is connected with the computing device through the CXL bus, supporting parallel read-write of core data such as database logs and indexes. On this basis, real-time transaction data generated by order interaction between adjacent computing devices is directly transmitted through the direct connection ring of the computing device, and the delay can be controlled within microseconds. And in the process of cross-node transaction submission, global data consistency check can be completed through the shared memory based on the CXL multi-head device. After data is written to the shared memory device, all computing devices can directly read it, reducing the delay of the TCP / IP protocol stack, for example, the delay time of the present application is compressed from 2-4 microseconds to within 300 nanoseconds. On this basis, the present application can reduce the submission delay of distributed transactions, meet the real-time requirements of high-frequency trading, and there is no cache bottleneck of the switch, avoiding the head-of-line blocking problem caused by multiple node concurrent write requests.

[0097] Further, in some application switch schemes, the total delay of inter-node communication is composed of multi-stage switch forwarding delay, protocol stack delay, etc. As shown in the following formula (1):

[0098] T_switch scheme=N_hops×t_single-hop switch+t_protocol stack+t_congestion wait (1)

[0099] Wherein, T_switch scheme represents the total delay of inter-node communication. N_hops represents the number of switch stages that data transmission passes through, such as some architectures are usually designed as 2-3 hops. t_single-hop switch represents the forwarding delay of a single switch. t_protocol stack represents the network protocol stack processing delay. t_congestion wait represents the queuing delay caused by Incast problem, port competition, cache depth, etc., and significantly increases with the increase of cluster size.

[0100] On this basis, some schemes also involve a congestion model, as shown in the following formula (2):

[0101] P_congestion=f(M, B, C) (2)

[0102] Wherein, P_congestion represents the probability of switch congestion. M represents the number of concurrent requests of the switch, such as the many-to-one request in the parameter server architecture. B represents the port bandwidth of the switch. C represents the cache depth of the switch. f represents that M, B and C have a relationship with P_congestion.

[0103] On this basis, the application is relative to the above scheme:

[0104] For the "N_ hop number x t_ single-hop exchange" part, the application can reduce the number of hops in the data transmission process to 1-2 hops, and the data of the application passes through a multi-head device or a direct connection device that can support high concurrency during transmission. And the delay of the ring direct connection communication between the computing devices, without switches, only physical link delay, which is close to the hardware level delay. Among them, the physical link includes the link connected by the shared memory device.

[0105] For the "t_ protocol stack" part, the application reduces the transmission delay through the shared memory device without protocol stack forwarding. Thus, the application reduces the transmission delay.

[0106] For the "t_ congestion waiting" part, even if the port bandwidth of the shared device of the application is the same as that of the above-mentioned switch, the computing system of the application will reduce the probability of port competition in the scene of request concurrency. In addition, due to the limited cache depth of the switch, and due to the cost advantage and storage advantage of the shared memory device, the cache depth impact basically does not exist, so the congestion waiting probability and waiting time can be further reduced.

[0107] The improvement effect can be represented by the following formula (3):

[0108] T_Improved = T_Switch scheme-T_Application (3)

[0109] Wherein, T_Improved represents the delay shortening time of the application relative to the above-mentioned switch scheme. T_Switch scheme can be referred to the description in the foregoing. T_Application represents the delay time of data transmission of the application.

[0110] As the cluster size increases and the communication frequency increases, the switch scheme problem and the delay problem will be more prominent, and the improvement scheme gain will increase significantly. Specifically, in the distributed training, taking the above algorithm as an example, the total synchronization time is a key performance indicator. On this basis, assuming that the iteration round K of the algorithm, the overall improvement effect is KxT_Improved.

[0111] Based on this, in the model training task, the application can split the model or data to multiple nodes based on the Compute ExpressLink (CXL) technology, to quickly realize data cross-node replication by means of high-speed memory copy, so as to quickly copy parameters and other data from one node to other nodes, and reduce the waiting time in the training process.

[0112] Further, on the basis of giving full play to the communication delay compression advantage of CXL technology, the application adopts a CXL multi-head device memory transfer mechanism to build a distributed computing network scheme, and designs a ring communication topology structure suitable for the mechanism. As the core transmission link of data exchange between different nodes, the ring communication topology structure greatly improves the efficiency of overall data transmission operation through the memory direct transfer mode realized by the CXL multi-head device.

[0113] The core advantage of this architecture is that it completely gets rid of the additional delay caused by the inherent operations such as queue queuing and store-and-forward in the traditional switch-based network architecture. On the one hand, the direct communication path is formed by the direct connection between the hosts in a ring, and the data transmission does not need to pass through multiple levels of switch forwarding, which ensures the shortest path at the physical link layer. On the other hand, the CXL multi-head device realizes cross-node data interaction through a shared memory pool, and the communication path is simplified to hardware-level memory access, further compressing the transmission delay. The double design meets the strict requirements of real-time in the distributed computing scenario, especially providing a more efficient and low-delay underlying network solution for high-frequency communication scenarios such as large-scale distributed machine learning and cloud computing.

[0114] In this way, based on the CXL multi-head device topology scheme, the application can realize a switch-free computing network architecture. First, the communication delay is significantly reduced, and the data transmission efficiency is improved. By reducing the switch forwarding delay, combined with the advantages of direct connection and architecture design, the synchronization operation performance can be optimized in algorithm application (such as AllReduce, etc.); then, it breaks through the hardware limitations of traditional switches, solves the congestion and bottleneck problems of traditional architectures, and has good network topology scalability; in addition, the link redundancy design and flexible topology adaptation can be extended to improve system fault tolerance and stability, adapt to high-performance computing scenarios such as data center artificial intelligence training, and have a wide range of application.

[0115] Based on this, the application reduces the overall system delay through the memory transfer mode of the CXL multi-head device, and uses the shared memory pool of the multi-head device as the core path of cross-node data exchange, has greater cache capacity, supports multi-host parallel reading and writing of shared memory, realizes "hardware-level memory access" type data transmission, and replaces the forwarding mechanism of the traditional switch relying on the network protocol stack and limited cache. Moreover, based on the communication mode and fault tolerance optimization mechanism of the application, the ring direct communication between hosts (reducing intermediate media delay) ensures low-delay direct communication of adjacent nodes, the memory transfer communication of the multi-head device intermediary (avoiding switch delay and congestion, reducing protocol stack delay), and provides cross-device communication efficiency and link redundancy. Moreover, the full-connection design of the shared memory device provides rich links, ensures strong fault tolerance and high bandwidth utilization, supports path switching in case of failure, adapts to distributed algorithms such as 2D-Torus AllReduce, and improves the communication efficiency of global reduction, aggregation and other operations through topology optimization.

[0116] Based on this, any application that needs memory expansion and sharing, involving artificial intelligence, cloud computing, distributed databases, scientific computing, big data and other scenarios that need distributed high-performance data computing and transmission, can use the computing system of the application.

[0117] In addition, the application also provides a computing method based on the above computing system.

[0118] Figure 6 A flowchart of the computing method according to an embodiment of the application is shown.

[0119] As shown in Figure 6 , the computing method of this embodiment can include operations S610-S620.

[0120] In operation S610, data is transmitted between a plurality of computing devices.

[0121] In operation S620, a data processing task is performed based on the transmitted data to obtain a data processing result.

[0122] In the embodiments of the present application, any computing device is directly connected to another two computing devices in the plurality of computing devices to form a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology transmit data through direct connection between each other; the computing system further includes a plurality of shared memory devices directly connected to each other through a memory direct link, and the plurality of shared memory devices are respectively directly connected to corresponding computing devices in the plurality of computing devices through a device direct link. Among them, the plurality of computing devices includes a first computing device and a second computing device which are spaced apart in the ring communication topology, and the plurality of shared memory devices includes a first shared memory device connected to the first computing device and a second shared memory device connected to the second computing device, so that the first computing device directly writes first data for the second computing device to the first shared memory device through the device direct link, the first shared memory device directly writes the first data to the second shared memory device through the memory direct link, and the second computing device directly reads the first data from the second shared memory device through the device direct link. It should be understood that the computing system of the present application can also implement any of the methods described above, which will not be repeated here.

[0123] For example, the data processing task includes a model training task; the data processing result includes gradient data; the gradient data represents data of model parameters for optimizing the model; the plurality of computing devices transmit data via the plurality of shared memory devices and perform the data processing task based on the transmitted data to obtain the data processing result, including: any computing node of any computing device performs the model training task to obtain corresponding gradient data.

[0124] For example, any computing node of any computing device performs the model training task to obtain corresponding gradient data, including: performing local reduction on the gradient data on the plurality of computing nodes of the computing device within the computing device to obtain local gradient data of the plurality of computing devices respectively; performing global reduction on the local gradient data of the plurality of computing devices respectively to obtain global gradient data on the plurality of computing devices, wherein the first computing device and the second computing device in the plurality of computing devices are spaced apart from each other in the ring communication topology and transmit their respective local gradient data through the first shared memory device and the second shared memory device.

[0125] The computer program product of the first aspect can include a computer readable storage medium. The computer readable storage medium can be a tangible computer readable storage medium storing the program code. The computer readable storage medium can include, but is not limited to, magnetic storage, optical storage, and the like. The computer readable storage medium can be non-transitory. The computer readable storage medium can be a non-transitory computer readable medium. The above general description or the following detailed description can use "computer readable storage medium", "computer readable medium", or similar term. These terms are used interchangeably. These terms are used to broadly convey that the medium is tangible.

[0126] Those skilled in the art can understand that the features described in various embodiments of the present application can be combined and / or integrated in various combinations, even if such combinations or integrations are not expressly described in the present application. In particular, the features described in various embodiments of the present application can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present application. All such combinations and / or integrations are within the scope of the present application.

[0127] The above describes embodiments of the present application. However, these embodiments are for illustrative purposes only, and are not intended to limit the scope of the present application. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Those skilled in the art can make various alternatives and modifications without departing from the scope of the present application, and these alternatives and modifications shall fall within the scope of the present application.

Claims

1. A computing system, comprising: Comprising: a plurality of computing devices, any one of which is directly connected to two other computing devices of the plurality of computing devices through host direct links to form a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology communicate data through the host direct links between each other; a plurality of shared memory devices directly connected to each other through memory direct links, the plurality of shared memory devices being directly connected to respective computing devices of the plurality of computing devices through device direct links, respectively, wherein the plurality of computing devices comprises a first computing device and a second computing device that are spaced apart in the ring communication topology, and the plurality of shared memory devices comprises a first shared memory device directly connected to the first computing device and a second shared memory device directly connected to the second computing device, the first computing device is configured to directly write first data for the second computing device to the first shared memory device through a device direct link, the first shared memory device is configured to directly write the first data to the second shared memory device through a memory direct link, and the second computing device is configured to directly read the first data from the second shared memory device through a device direct link.

2. The computing system of claim 1, wherein, the first shared memory device comprises a plurality of first storage regions, the plurality of first storage regions being associated with the plurality of shared memory devices, respectively; the first shared memory device is further configured to directly write the first data to the second shared memory device in response to the first computing device writing the first data to a target first storage region of the plurality of first storage regions that is associated with the second shared memory device.

3. The computing system of claim 2, wherein, the first shared memory device comprises a first port and a second port, the first port being directly connected to the first computing device, and the second port being directly connected to the second shared memory device, the target first storage region being associated with the first port and the second port; the first shared memory device is further configured to directly write the first data to the second shared memory device via the second port in response to the first computing device writing the first data to the target first storage region via the first port.

4. The computing system of any of claims 1-3, wherein, at least two computing devices directly connected to any one of the shared memory devices are spaced apart from each other in the ring communication topology, the any one of the shared memory devices comprises a second storage region associated with the at least two computing devices directly connected to the any one of the shared memory devices to facilitate reading of second data for other computing devices of the at least two computing devices from the second storage region by the other computing devices in a case that one of the at least two computing devices writes the second data to the second storage region.

5. The computing system of any one of claims 1-3, wherein, the plurality of computing devices further comprises a third computing device connected to the first shared memory device; the first computing device and the third computing device are configured to write data to the first shared memory device in parallel; The first shared memory device is further configured to write data from the first computing device and data from the third computing device to the second shared memory device; The second computing device is configured to read data from the first computing device and data from the third computing device from the second shared memory device.

6. The computing system of any one of claims 1-3, wherein, The computing system is configured to perform a model training task, any computing device comprising a plurality of computing nodes, any computing node of any computing device obtaining corresponding gradient data in the model training task, the gradient data representing data for optimizing model parameters of a model.

7. The computing system of claim 6, wherein, the gradient data on the plurality of computing nodes of any computing device is locally reduced within the computing device to obtain local gradient data of the computing device respectively; the local gradient data of the plurality of computing devices is globally reduced among the plurality of computing devices to obtain global gradient data on the plurality of computing devices respectively, wherein the first data comprises the local gradient data of the first computing device.

8. The computing system of claim 7, wherein, the gradient data on any computing node is divided into a predetermined number of gradient blocks, and the local reduction comprises passing the gradient blocks among the plurality of computing nodes within any computing device so that any computing node has local gradient blocks of corresponding dimensions; the global reduction comprises passing the gradient blocks of the same dimensions among corresponding computing nodes having the gradient blocks of the same dimensions in the computing nodes of the plurality of computing devices so that any computing node within any computing device has global gradient blocks of corresponding dimensions; and the model training task further comprises globally aggregating the global gradient blocks of corresponding dimensions of the plurality of computing nodes of any computing device to obtain final gradient data.

9. The computing system of any of claims 1-3, wherein, in a state of failure of the memory direct link between the first shared memory device and the second shared memory device, the first shared memory device writes the first data to the second shared memory device via another memory direct link between the first shared memory device and a third shared memory device, and another memory direct link between the third shared memory device and the second shared memory device.

10. The computing system of any of claims 1-3, wherein, in a state of failure of the device direct link between the first computing device and the first shared memory device, the first computing device reads and writes data to the first shared memory device via a host direct link between the first computing device and a third computing device, a device direct link between the third computing device and a corresponding third shared memory device, and a memory direct link between the third shared memory device and the first shared memory device.

11. The computing system of any of claims 1-3, wherein, In a state of a host direct link failure between a third computing device and a fourth computing device adjacent to each other in the ring communication topology, the third computing device and the fourth computing device are configured to transmit data between each other via a device direct link between the third computing device and a corresponding third shared memory device, a memory direct link between the third shared memory device and a corresponding fourth shared memory device of the fourth computing device, and another device direct link between the fourth shared memory device and the fourth computing device.

12. The computing system of any one of claims 1-3, wherein, In a state of a host direct link failure between a third computing device and a fourth computing device adjacent to each other in the ring communication topology, the third computing device and the fourth computing device are configured to transmit data between each other via a device direct link between the third computing device and a corresponding third shared memory device, a memory direct link between the third shared memory device and a corresponding fourth shared memory device of the fourth computing device, and another device direct link between the fourth shared memory device and the fourth computing device.

13. The computing system of any of claims 1-3, wherein: the memory direct link comprises a connection through a network card or a bus; the device direct link comprises a connection through a bus; the host direct link comprises a connection through a network card; computing nodes within a same computing device are connected through a bus.

14. A computing method, comprising: The computing method is applied to a computing system, and the computing method comprises: transmitting data between a plurality of computing devices in the computing system and performing a data processing task based on the transmitted data to obtain a data processing result, wherein any computing device is directly connected to another two computing devices in the plurality of computing devices through a host direct link to form a ring communication topology based on the plurality of computing devices, wherein adjacent computing devices in the ring communication topology are directly connected through a host direct link between each other for data transmission; the computing system further comprises a plurality of shared memory devices directly connected to each other through a memory direct link, and the plurality of shared memory devices are respectively directly connected to corresponding computing devices in the plurality of computing devices through a device direct link, wherein the plurality of computing devices comprises a first computing device and a second computing device spaced apart in the ring communication topology, and the plurality of shared memory devices comprises a first shared memory device connected to the first computing device and a second shared memory device connected to the second computing device, so that the first computing device directly writes first data for the second computing device to the first shared memory device through a device direct link, the first shared memory device directly writes the first data to the second shared memory device through a memory direct link, and the second computing device directly reads the first data from the second shared memory device through a device direct link.

15. The computing method of claim 14, wherein, The data processing task comprises a model training task; the data processing result comprises gradient data; and the gradient data represents data for optimizing model parameters of a model. transmitting data between a plurality of computing devices in the computing system and performing a data processing task based on the transmitted data to obtain a data processing result, comprises: Any computing node of any of the computing devices performs the model training task to obtain respective gradient data.

Images