Computing device and cluster

Abstract

A computing device and a cluster. The computing device comprises a plurality of computing boards and at least one switching chip, with each computing board being used for integrating one or more computing chips, wherein a first computing chip and a second computing chip are electrically connected by means of signal lines, the first computing chip and the second computing chip are accelerator-type chips, the first computing chip and a third computing chip are electrically connected by means of a switching chip, and the first computing chip and the third computing chip are located on different computing boards among the plurality of computing boards.

Description

A computing device and cluster

[0001] The present application claims priority to the Chinese patent application No. 202411394984.6, filed on September 30, 2024, and entitled "A computing device and cluster", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of computer technology, and in particular to a computing device and cluster. BACKGROUND

[0003] At present, the growth rate of the computing power of a single computing chip has far failed to meet the growing demand for computing power. In this case, multiple computing chips can be interconnected to jointly form a computing device with stronger computing power, such as a rack, to accelerate computing.

[0004] Since multiple computing chips are integrated in a rack to jointly complete a same computing task, there is a huge data transmission demand between different computing chips. Currently, the time consumption for transmitting data between different computing chips has become one of the bottlenecks of system performance. How to improve the efficiency of transmitting data between different computing chips in a rack and reduce system latency under the constraint of cost has become a problem to be solved. SUMMARY

[0005] The present application provides a computing device and cluster. Through direct electrical connection between different computing chips in a same computing device via a signal line and indirect electrical connection between different computing chips via a switching chip, the communication efficiency can be effectively improved and the latency can be reduced with a smaller number of switching chips.

[0006] In a first aspect, the present application provides a computing device, which includes a plurality of computing boards and at least one switching chip. Each computing board is configured to integrate one or more computing chips. The first computing chip and the second computing chip are electrically connected via a signal line, and the first computing chip and the second computing chip are accelerator-type chips. The first computing chip and the third computing chip are electrically connected via the switching chip, and the first computing chip and the third computing chip are located on different computing boards in the plurality of computing boards.

[0007] It can be understood that the accelerator type computing chips are usually used to perform model training, inference and other tasks, that is, the amount of data to be transmitted between the accelerator type computing chips is much higher than the amount of data to be transmitted between the accelerator type computing chips and other type computing chips, and the computing chips with huge transmission demand are connected by signal lines, which can save the forwarding of the switching chip and reduce the delay; and the computing chips on different computing boards are connected through the switching chip, and the two computing chips only need to be transferred once by the switching chip, compared with the computing device without the switching chip and only part of the computing chips on different computing boards are directly connected by signal lines, the average transfer number of data transmission is less, the invalid transfer flow occupies less bandwidth, in other words, the effective bandwidth in the computing device is improved.

[0008] In a possible implementation, the first computing chip and the second computing chip are located in different computing boards of the plurality of computing boards.

[0009] It can be understood that the computing chips on different computing boards are connected by signal lines and / or through the switching chip, which is more flexible than using only one connection method, and fully utilizes the low latency advantage of signal line direct connection and the flexible routing advantage of the switching chip.

[0010] In a possible implementation, the first computing chip and the third computing chip are computing chips of different types, and the first computing chip is further configured to be connected to the third computing chip by signal lines.

[0011] It can be understood that the computing chips of different types on different computing boards are connected by signal lines, such as the direct connection of central processing units and graphic processing units, which can realize direct communication between two computing chips without forwarding, can significantly improve the communication efficiency and reduce the system latency.

[0012] In a possible implementation, the first computing chip is further configured to be connected to the second computing chip through the switching chip.

[0013] It can be understood that the accelerator type computing chips are connected by signal lines and switching chips, so that the transmission of data can be flexibly selected, and in the case of large amount of data to be transmitted, both communication methods can be used at the same time to accelerate transmission and improve communication efficiency.

[0014] In a possible implementation, the third computing chip is the computing chip connected to the most other computing chips on the computing board by signal lines; and / or, the first computing chip is the computing chip connected to the most other computing chips on the computing board by signal lines.

[0015] It can be understood that, in the same computing board, the computing chip connected directly to the most other chips is electrically connected to the computing chip on another computing board through the switching chip, and the computing chip connected directly and the switching chip are enabled to communicate with the computing chip on another computing board through forwarding of the computing chip and the switching chip, without each computing chip being connected to the switching chip, reducing the number of occupied communication ports of the switching chip, and thereby helping to reduce the number of switching chips to be deployed in the computing device.

[0016] In a possible implementation, the first computing chip is the computing chip with the highest frequency of change in communication relationship on the computing board; and / or, the third computing chip is the computing chip with the highest frequency of change in communication relationship on the computing board.

[0017] It can be understood that the frequency of change in communication relationship refers to the frequency of switching the object of communication of the computing chip, and since each wired connection mode occupies a communication port on the computing chip, and the number of communication ports on each computing chip is limited, the computing chip with the highest frequency of change in communication relationship is electrically connected to the switching chip, and the number of occupied communication ports is significantly reduced under the condition that the communication demand of the computing chip is met through the flexible routing capability of the switching chip.

[0018] In a possible implementation, the frequency of change in communication relationship is less than or equal to 100 milliseconds per time.

[0019] In a possible implementation, the switching chip and the third computing chip are located on the same computing board, or the switching chip and the first computing chip are located on the same computing board.

[0020] It can be understood that integrating the switching chip and the computing chip into the same computing board can improve the space utilization in the computing device, and help miniaturization of the computing device.

[0021] In a possible implementation, the computing device further includes a switching network board, and the switching chip is located on the switching network board.

[0022] In a possible implementation, the computing device further includes a backboard, and the switching network board and the plurality of computing boards are located on the backboard.

[0023] It can be understood that integrating the computing board and the switching network board into the backboard can shorten the wiring distance between the switching chip and the first computing chip, and enable the computing board and the switching network board to obtain stable support.

[0024] In a possible implementation, the computing device further includes that the switching network board is orthogonally connected to the plurality of computing boards.

[0025] It can be understood that integrating the switch chip into the switch network board connected with the computing board orthogonally can significantly shorten the wiring distance between the switch network board and the first computing chip, improve the communication efficiency, and at the same time, due to the orthogonal connection mode, the large backboard can be saved, so that the space utilization in the computing device can be improved, and the miniaturization of the computing device is helpful.

[0026] In a second aspect, the present application provides a computing device cluster, comprising a plurality of computing devices as described in the first aspect, and different computing devices are connected through switches.

[0027] These aspects or other aspects of the present application will be more apparent in the following description. BRIEF DESCRIPTION OF DRAWINGS

[0028] FIG. 1 is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application;

[0029] FIG. 2 is a structural schematic diagram of a cabinet related to the embodiment shown in FIG. 1;

[0030] FIG. 3 is a schematic diagram of an interconnection topology in a computing device provided by an embodiment of the present application;

[0031] FIG. 4 is a schematic diagram of an interconnection topology in a computing device related to the embodiment shown in FIG. 3;

[0032] FIG. 5 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 3;

[0033] FIG. 6 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 4;

[0034] FIG. 7 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 3;

[0035] FIG. 8 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 4;

[0036] FIG. 9 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 5;

[0037] FIG. 10 is a schematic diagram of an interconnection topology in another computing device related to the embodiment shown in FIG. 6;

[0038] FIG. 11 is a schematic diagram of a local direct connection topology in a computing device provided by an embodiment of the present application;

[0039] FIG. 12 is a schematic diagram of another local direct connection topology in a computing device provided by an embodiment of the present application;

[0040] FIG. 13 is a local schematic diagram of the position of a switch chip provided by an embodiment of the present application;

[0041] FIG. 14 is a partial schematic view of another position of a switch chip provided by an embodiment of the present application;

[0042] FIG. 15 is a partial schematic view of an internal structure of a computing device provided by an embodiment of the present application;

[0043] FIG. 16 is a partial schematic view of another internal structure of a computing device provided by an embodiment of the present application;

[0044] FIG. 17 is a schematic view of a computing device cluster provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions and advantages of embodiments of the present application clearer, the following will further describe the embodiments of the present application with reference to the accompanying drawings.

[0046] In the present document, “multiple” refers to two or more. “And / or” describes the association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character “ / ” generally represents an “or” relationship between the associated objects before and after it.

[0047] Also, in the description of embodiments of the present application, “multiple” refers to two or more, unless otherwise specified. “At least one of” or the like refers to any combination of the items, including any combination of single item or multiple items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0048] In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using “first”, “second”, etc. The skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. also do not necessarily mean different. At the same time, in the embodiments of the present application, the words “exemplary” or “for example” are used to represent as an example, illustration or description. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of “exemplary” or “for example” is intended to present the relevant concept in a specific manner, for understanding.

[0049] First, the application scenarios of embodiments of the present application are exemplarily introduced.

[0050] Taking an intelligent question-answering system based on a large language model as an example, the system faces challenges such as high concurrency demand and low latency, and almost every moment bears huge computing load, which cannot be met by the computing power of a single computing chip. In this case, the system can be selected to run in a computing cluster composed of multiple racks, as shown in FIG. 1, which is an exemplary application scenario provided by an embodiment of the present application. In the application scenario, the intelligent question-answering system needs to efficiently process the requests input by multiple users such as user 1-user 3, for example, to answer the questions input by the users in the form of text or voice, and the system runs in a computing cluster composed of four racks, namely, rack 1-rack 4. The four racks are interconnected through an external switch, and the computing power of the computing cluster far exceeds that of a single computing chip, so that the intelligent question-answering system can process the corresponding business at a faster speed.

[0051] Compared with a single computing chip, the rack shown in FIG. 1 is a larger and more complex computing device, and its structure is shown in FIG. 2, which is a structural diagram of a rack involved in the embodiment shown in FIG. 1. The rack shown in FIG. 2 can be one of rack 1-rack 4 in FIG. 1, and the rack shown in FIG. 2 at least includes a power supply and multiple computing boards such as computing board 1-computing board 3.

[0052] Specifically, the computing board shown in FIG. 2 can integrate one or more computing chips, as shown on the right side of FIG. 2. Computing board 1 integrates computing chip 1 and computing chip 2, computing board 2 integrates computing chip 3 and computing chip 4, and computing board 3 integrates computing chip 5. The types of these computing chips can be the same or different, for example, computing chip 1-computing chip 4 are all graphic processing units, and computing chip 5 is a central processing unit. The computing power of the rack composed of these computing chips is much greater than that of a single computing chip.

[0053] Optionally, the rack shown in FIG. 2 further includes a management board, a backup board, a switching network board, and other components or structures such as a wiring slot.

[0054] The management board can be used to run an operating system (OS) to manage the operation of the entire rack. The backup board can be used to integrate multiple storage devices (such as solid state disks) to store corresponding data. The switching network board can be used to integrate a switching chip. The wiring slot can be used to organize power lines, signal lines, and other wires.

[0055] Assuming that the data transmission volume corresponding to the bit computation volume of the computing task undertaken by the cabinet shown in FIG. 2 is constant, as the computing volume undertaken by a cabinet increases, the data transmission efficiency between different computing chips in the cabinet also increases, so as to avoid the time consumption of data transmission accounting for too large a proportion of the total time consumption of the entire computing task, and avoid reducing the efficiency of processing services, which requires that the different computing chips in the same cabinet be connected through reasonable interconnection to improve communication efficiency. Common connection modes between computing chips generally include two modes: 1. direct electrical connection (referred to as direct connection) through a signal line; and 2. electrical connection through a switching chip.

[0056] Generally, manufacturers without the ability to design switching chips can only use the first connection mode when designing the topology in the rack. In this connection mode, as the communication ports of each computing chip are limited, it is difficult to achieve direct connection between any two computing chips in a cabinet containing multiple computing chips, which further leads to the need for other computing chips to transit when the computing chips transmit data. The transit traffic will occupy the bandwidth of the computing chip performing the transit, reducing the effective bandwidth of the computing chip. In addition, each transit will introduce an additional time delay cost, so this mode is only suitable for scenarios where the communication objects of each computing chip are relatively fixed.

[0057] Some manufacturers with the ability to design switching chips need to ensure that any two computing chips in the rack can directly reach each other through only one forwarding of the switching chip and have full-bandwidth communication capability due to the limitation of the corresponding software development interface, so the second connection mode is used. In this connection mode, if each computing chip in the cabinet is interconnected through a switching chip, a large number of switching chips need to be deployed in the cabinet, which leads to greater overall power consumption of the computing device. In addition, the signal lines used when each switching chip and the computing chip are connected will occupy the limited capacity in the computing device, reduce the effective utilization of space, and are not conducive to heat dissipation. Moreover, this solution has a high cost.

[0058] Therefore, the embodiments of the present application propose a computing device including multiple computing boards. Computing chips located on the same or different computing boards can be directly electrically connected using signal lines or indirectly electrically connected through switching chips, that is, the computing device has a flexible hybrid topology design, so as to fully exert the routing capability of the switching chip and the low-latency advantage of direct connection while deploying a small number of switching chips, thereby ensuring high communication efficiency and reducing latency.

[0059] In some possible embodiments, the present application provides a computing device, comprising a plurality of computing boards and at least one switching chip, each of the computing boards is configured to integrate one or more computing chips, wherein the first computing chip and the second computing chip are electrically connected through a signal line, and the first computing chip and the second computing chip are accelerator type chips, the first computing chip and the third computing chip are electrically connected through the switching chip, and the first computing chip and the third computing chip are located on different computing boards of the plurality of computing boards. Since the accelerator type computing chips (which usually have a large amount of data transmission demand) are electrically connected through the signal line, the forwarding of the switching chip can be omitted, and the latency is reduced. The computing chips on different computing boards are connected through the switching chip, and the two computing chips only need to be transferred once by the switching chip. Compared with the computing device that does not contain the switching chip and only part of the computing chips on different computing boards are directly connected through the signal line, the average number of transfer times of data transmission is less, the invalid transfer flow occupies less bandwidth, in other words, the effective bandwidth in the computing device is improved.

[0060] It should be noted that the embodiments of the present application do not limit the material, shape, etc. of the shell of the computing device.

[0061] It should be noted that the embodiments of the present application do not limit the physical positions of the computing chips and the switching chip, in other words, the topological connection relationship involved in each of the following embodiments is a logical topological connection relationship.

[0062] It should be noted that the embodiments of the present application do not limit the way in which the computing chips and the switching chip are electrically connected, for example, the computing chips can also be electrically connected to the switching chip through a signal line.

[0063] Optionally, in the embodiments of the present application, the signal line can be a printed circuit board (PCB) trace, a copper cable or an optical fiber, etc.

[0064] Optionally, in the embodiments of the present application, the computing chips in the computing device can include a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), etc.

[0065] The computing device provided by the embodiments of the present application will be explained and described in detail below with reference to the accompanying drawings.

[0066] In the computing device provided in the embodiments of the present application, at least two computing boards are included, wherein at least one computing chip of an accelerator type (first computing chip) on one computing board is electrically connected (or directly connected) to another computing chip of an accelerator type (second computing chip) and is further electrically connected to a computing chip (third computing chip) on another computing board through an exchange chip.

[0067] Specifically, as shown in FIG. 3, which is a schematic diagram of an interconnection topology in a computing device provided in the embodiments of the present application, the computing device 1000 in FIG. 3 can be one of the No. 1-4 cabinets in FIG. 1, and the computing device 1000 includes multiple computing boards (only two are shown in the figure) such as the computing board 1100 and the computing board 1200, multiple exchange chips (only one is shown in the figure) such as the exchange chip 1310, and multiple computing chips (only six are shown in the figure). Specifically, the computing chips 1110-1130 are integrated on the computing board 1100, the computing chips 1210-1230 are integrated on the computing board 1200, and the exchange chip 1310 is integrated in the exchange network board 1300. Optionally, the computing device 1000 further includes a housing 1001.

[0068] For example, the computing chips shown in FIG. 3 are one of a GPU, a TPU, or an NPU, the computing board 1100, the computing board 1200, and the exchange network board 1300 are all PCB boards, the computing chip 1110 on the computing board 1100 is electrically connected to the computing chip 1120 and the computing chip 1130 through PCB wiring, and the computing chip 1110 is further electrically connected to the computing chip 1210 on the computing board 1200 through the exchange chip, and the computing chip 1210 on the computing board 1200 is electrically connected to the computing chip 1220 and the computing chip 1230 through PCB wiring.

[0069] Optionally, in the computing device provided in the embodiments of the present application, not only the computing chips on the same computing board can be electrically connected through a signal line, but also the computing chips on different computing boards can be electrically connected through a signal line.

[0070] As shown in FIG. 4, which is a schematic diagram of an interconnection topology in a computing device related to the embodiment shown in FIG. 3, compared with FIG. 3, FIG. 4 shows one signal line (dashed line in FIG. 4) that directly connects across the boards, specifically, the computing chip 1110 on the computing board 1100 in FIG. 4 is electrically connected to the computing chip 1210 on the computing board 1200 through a signal line. Optionally, the signal line can be an optical fiber.

[0071] Optionally, in the computing device provided in the embodiments of the present application, multiple types of computing chips are included.

[0072] For example, as shown in FIG. 5, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 3, the rectangular computing chips represent GPUs, and the oval computing chips represent CPUs. Compared with FIG. 3, the computing chip 1140 (CPU) on the computing board 1100 is electrically connected to the computing chip 1110 (GPU), the computing chip 1120 (GPU), and the computing chip 1130 (GPU) through signal lines, and the computing chip 1240 (CPU) on the computing board 1200 is electrically connected to the computing chip 1210 (GPU), the computing chip 1220 (GPU), and the computing chip 1230 (GPU) through signal lines, which facilitates the management of the GPUs by the CPUs.

[0073] For example, as shown in FIG. 6, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 4, the rectangular computing chips represent GPUs, and the oval computing chips represent CPUs. Compared with FIG. 4, the computing chip 1140 (CPU) on the computing board 1100 is electrically connected to the computing chip 1110 (GPU), the computing chip 1120 (GPU), and the computing chip 1130 (GPU) through signal lines, and the computing chip 1240 (CPU) on the computing board 1200 is electrically connected to the computing chip 1210 (GPU), the computing chip 1220 (GPU), and the computing chip 1230 (GPU) through signal lines, which facilitates the management of the GPUs by the CPUs.

[0074] Optionally, in the computing device provided in the embodiments of the present application, the accelerator-type computing chips electrically connected through signal lines can also be electrically connected through a switch chip.

[0075] For example, as shown in FIG. 7, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 3, compared with FIG. 3, the computing chip 1120 (NPU) in FIG. 7 is electrically connected to the switch chip 1310 (dashed line in FIG. 7), and the computing chip 1110 (NPU) is also electrically connected to the switch chip 1310. Therefore, the computing chip 1120 in FIG. 7 can be electrically connected to the computing chip 1110 through the switch chip, and the computing chip 1110 on the computing board 1100 in FIG. 7 is also electrically connected to the computing chip 1120 through signal lines. Thus, when the computing chip 1110 in FIG. 7 and the computing chip 1120 transmit data to each other, they can simultaneously transmit data in two communication modes, thereby achieving the superimposed use of the bandwidths of the two communication modes and improving the communication efficiency.

[0076] For example, as shown in FIG. 8, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 4, compared with FIG. 4, the computing chip 1110 (TPU) and the computing chip 1120 (TPU) in FIG. 8 can not only be electrically connected through the signal line, but also be electrically connected through the switch chip 1310, so that the computing chip 1110 in FIG. 8 is electrically connected with the computing chip 1120 and the computing chip 1130 on the same computing board and the computing chip 1210 on another computing board through the signal line, and the computing chip 1110 is also electrically connected with the computing chip 1120 and the computing chip 1210 on another computing board through the switch chip 1310, so that the computing chip 1110 is very flexible in communication and can communicate with other computing chips in different communication modes at the same time, improving the communication efficiency.

[0077] For example, as shown in FIG. 9, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 5, compared with FIG. 5, the computing chip 1120 (NPU) in FIG. 9 is electrically connected with the switch chip 1310 (dotted line in FIG. 9), and the computing chip 1110 (NPU) is also electrically connected with the switch chip 1310, so that the computing chip 1120 in FIG. 9 can be electrically connected with the computing chip 1110 through the switch chip, and the computing chip 1110 in FIG. 9 can also be electrically connected with the computing chip 1120 through the switch chip, so that when the computing chip 1110 and the computing chip 1120 transmit data to each other, two communication modes can be used at the same time, improving the communication efficiency.

[0078] For example, as shown in FIG. 10, which is a schematic diagram of an interconnection topology in another computing device involved in the embodiment shown in FIG. 6, compared with FIG. 6, the computing chip 1110 (TPU) and the computing chip 1120 (TPU) on the computing board 1100 in FIG. 10 are electrically connected through the signal line, and since the computing chip 1120 in FIG. 10 is also connected with the switch chip 1310 (dotted line in FIG. 10), the computing chip 1110 in FIG. 10 can also be electrically connected with the computing chip 1120 through the switch chip, so that when the computing chip 1110 and the computing chip 1120 transmit data to each other, two communication modes can be used at the same time, improving the communication efficiency.

[0079] Optionally, in the computing device provided in the embodiments of the present application, the number of other chips directly connected in the same computing board is the largest, and the computing chip on the same computing board is electrically connected with the computing chip on another computing board through the switch chip.

[0080] For example, as shown in FIG. 11, which is a schematic diagram of a local direct connection topology in a computing device according to an embodiment of the present application, in the computing board 2000 shown in FIG. 11, the computing board can be installed in the computing device 1000 shown in any one of FIGS. 3-10, wherein the computing chip 2010 is electrically connected to the computing chip 2020, the computing chip 2030, and the computing chip 2040 through signal lines, that is, the number of computing chips to which the computing chip 2010 is electrically connected through signal lines is 3, which is greater than the number of computing chips to which other computing chips in the computing board 2000 are electrically connected through signal lines (which is 1 in particular), so that the computing chip 2010 is electrically connected to the computing chips on other computing boards through the switching chip (see the foregoing, which is not described here again), so that the other three computing chips on the computing board 2000 can communicate with the computing chips on other computing boards through the computing chip 2010 and the switching chip. Further, the foregoing connection mode can be applied to each computing board in the computing device, so that more computing chips can communicate with each other while reducing the number of switching chips.

[0081] Optionally, in the computing device provided by an embodiment of the present application, the computing chip with the highest communication relationship change frequency in the same computing board is electrically connected to the computing chips on other computing boards through the switching chip.

[0082] The communication relationship change frequency refers to the frequency of switching communication objects of the computing chip, and the higher the communication relationship change frequency, the more frequently the computing chip needs to switch between communication objects. Electrically connecting such a computing chip to other computing chips through a switching chip can fully utilize the flexible routing capability of the switching chip, while avoiding occupying too many communication ports by electrically connecting to each communication object through multiple signal lines. In some feasible embodiments, the communication relationship change frequency can be evaluated according to the type, parameters, and corresponding application scenarios of the computing chip, which is not limited in the present application. Further, in an embodiment of the present application, the communication relationship change frequency is less than or equal to 100 milliseconds, and the computing chip is electrically connected to the computing chips on other computing boards through the switching chip each time, in other words, the computing chip that switches communication between 10 or more different computing chips within 1 second is electrically connected to the computing chips on other computing boards through the switching chip.

[0083] For example, the number of threads running on the CPU when performing a computing task is small but the logic is complex; while the GPU when performing a computing task often runs a large number of threads in parallel computing, and different GPUs often can perform the same computing task in parallel (such as distributed neural network training), which may cause different CPUs in the same computing device to not need to frequently switch communication objects, while different GPUs in the same computing device need to frequently communicate with each other, and the communication relationship change frequency can be 50 milliseconds. Therefore, the computing chip with the highest communication relationship change frequency and the switching chip are electrically connected, and through the flexible routing capability of the switching chip, in the case of limited communication ports of the computing chip, it is avoided that each communication object (other computing chips) is electrically connected through a signal line, thereby realizing that the number of occupied communication ports is significantly reduced while ensuring that the communication demand of the computing chip is met.

[0084] It should be noted that the topology structure formed by the plurality of computing chips in the computing device being electrically connected (directly connected) through the signal line in the embodiments of the present application is not limited, and specifically as shown in FIG. 12, FIG. 12 is a schematic diagram of another local direct connection topology in a computing device provided by the embodiments of the present application, wherein,

[0085] a) The computing chips 3110-3140 on the computing board 3100 are directly connected according to the topology structure of full connection (fullemsh) within the board.

[0086] b) The computing chips 3210-3240 on the computing board 3200 and the computing chips 3310-3340 on the computing board 3300 are directly connected according to the topology structure of full connection (fullemsh) within and between the boards.

[0087] c) The computing chips 3410-3440 on the computing board 3400 are directly connected according to the topology structure of 2D torus.

[0088] The following will be combined with FIG. 13-FIG. 16 to describe the plurality of possible implementation manners of the position of the switching chip in the computing device provided by the embodiments of the present application, including:

[0089] 1) The switching chip can be located on the computing board.

[0090] For example, as shown in FIG. 13, which is a partial schematic diagram of a location of a switching chip according to an embodiment of the present application, the computing board 4100 shown in FIG. 13 can be installed in the computing device 1000 shown in any one of FIGS. 3-10, wherein the switching chip 4110, the computing chip 4120, and the computing chip 4130 are integrated in the computing board 4100. In some possible embodiments, the computing chip 4120 and the computing chip 4130 can be electrically connected to the switching chip 4110 through PCB traces, and the switching chip 4110 can be electrically connected to the computing chips on other computing boards through signal lines.

[0091] 2) The switching chip can be located on a switching network board.

[0092] For example, as shown in FIG. 14, which is another partial schematic diagram of a location of a switching chip according to an embodiment of the present application, the switching network board 2100 shown in FIG. 14 can be installed in the computing device 1000 shown in any one of FIGS. 3-10, wherein the switching network board 2100 is integrated with multiple switching chips, such as the switching chip 2110, the switching chip 2120, and the switching chip 2130. In some possible embodiments, the switching chip 2110, the switching chip 2120, and the switching chip 2130 can be electrically connected through PCB traces, and the aforementioned three switching chips can be electrically connected to the computing chips integrated on the computing board through signal lines.

[0093] Further, the connection relationship between the switching network board and the computing board includes at least:

[0094] 2a) The switching network board and the computing board are integrated on a backboard.

[0095] For example, as shown in FIG. 15, which is a partial schematic diagram of an internal structure of a computing device according to an embodiment of the present application, the computing board 5100, the computing board 5200, and the computing board 5300 are integrated on the same backboard 5500, wherein each of the computing board 5100, the computing board 5200, and the computing board 5300 can be integrated with one or more computing chips, and the switching network board 5400 can be integrated with one or more switching chips. In this way, the computing chips on the aforementioned multiple computing boards and the switching chips integrated on the switching network board 5400 can be electrically connected through traces on the backboard 5500, and the backboard 5500 can provide stable support for the computing board 5100, the computing board 5200, the computing board 5300, and the switching network board 5400.

[0096] 2b) The switching network board and the computing board are connected orthogonally.

[0097] For example, as shown in FIG. 16, which is a schematic diagram of a partial internal structure of a computing device according to an embodiment of the present application, computing boards 6100 and 6200 are connected to switch boards 6300 and 6400 orthogonally, wherein each of the computing boards 6100-6200 can be integrated with one or more computing chips, and each of the switch boards 6300 and 6400 can be integrated with one or more switch chips. Such a connection mode can save the internal space of the computing device by eliminating the backplane structure. Optionally, an orthogonal connector can be arranged at the connection between the computing boards and the switch boards, and the orthogonal connector can be used to provide high-speed and stable connection between the computing chips integrated on the computing boards and the switch chips integrated on the switch boards.

[0098] In summary, the computing device according to the present application is described in detail in combination with the foregoing FIG. 3-FIG. 16. Based on the direct connection networking of the signal lines and the reasonable cooperation of the switch chips in the computing device according to the present application, the data transmission efficiency between the computing chips can be improved and the system latency can be reduced while the number of the switch chips is small.

[0099] The present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be any one of the computing devices shown in the foregoing FIG. 3-FIG. 16.

[0100] For example, as shown in FIG. 17, which is a schematic diagram of a computing device cluster according to an embodiment of the present application, the computing device cluster 100 includes computing devices 110-140, wherein the computing devices 110-140 can be any one of the computing devices shown in the foregoing FIG. 3-FIG. 16, and the computing devices 110-140 are connected to each other through an external switch 150 to realize communication between different computing devices. In the foregoing embodiments, the switch chips are built-in in the corresponding computing devices, while the switch 150 shown in FIG. 17 is a network device independent of the computing devices 110-140.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent features, and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A computing device, comprising: The computing device comprises: a plurality of computing boards and at least one switching chip, each of the computing boards is configured to integrate one or more computing chips, wherein, the first computing chip and the second computing chip are electrically connected through a signal line, the first computing chip and the second computing chip are accelerator type chips; the first computing chip and the third computing chip are electrically connected through the switching chip, the first computing chip and the third computing chip are located on different computing boards in the plurality of computing boards.

2. The computing device of claim 1, wherein, The first computing chip and the second computing chip are located on different computing boards in the plurality of computing boards.

3. The computing device of claim 1 or 2, wherein, The first computing chip and the third computing chip are different types of computing chips, and the first computing chip is further configured to be electrically connected with the third computing chip through a signal line.

4. The computing device of any of claims 1-3, wherein, The first computing chip is further configured to be electrically connected with the second computing chip through the switching chip.

5. The computing device of any of claims 1-4, wherein, The third computing chip is the one with the largest number of other computing chips connected through the signal line on the computing board; and / or, The first computing chip is the one with the largest number of other computing chips connected through the signal line on the computing board.

6. The computing device of any of claims 1-5, wherein, The first computing chip is the one with the highest frequency of communication relationship change on the computing board; and / or, The third computing chip is the one with the highest frequency of communication relationship change on the computing board.

7. The computing device of claim 6, wherein, The frequency of the communication relationship change is less than or equal to 100 milliseconds per time.

8. The computing device of any of claims 1-7, wherein, The switching chip and the third computing chip are located on the same computing board, or the switching chip and the first computing chip are located on the same computing board.

9. The computing device of any of claims 1-7, wherein, The computing device further comprises a switching network board, and the switching chip is located on the switching network board.

10. A cluster of computing devices, characterized in that, The computing device comprises a plurality of computing devices as claimed in any one of claims 1-9, and different computing devices are connected through a switch.

Images