Inter-chip communication system and method, exchange chip and chip product

Abstract

The embodiment of the invention provides an inter-chip communication system and method, a switching chip and a chip product, and the system comprises a plurality of switching networks; wherein each switching network comprises at least one processing chip and at least one switching chip, the switching chip in the switching network is interconnected with each processing chip in the switching network, and at least one pair of interconnected switching chips exists between different switching networks. Therefore, the communication delay can be reduced while the implementation cost of the system is reduced, so that the support for a large-scale network and low-delay communication is considered, and the overall expansibility and communication efficiency of the system are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the chip technical field, and in particular to an inter-chip communication system and method, a switch chip, and a chip product. BACKGROUND

[0002] With the increasing demand for high-performance computing, especially in large-scale artificial intelligence model training, efficient communication between multiple processing chips has become critical. The inter-chip communication system, as the basic framework supporting the collaborative work of multiple processing chips, is widely used in intelligent computing networks. In related technologies, the inter-chip communication system usually adopts a one-layer networking structure or a two-layer networking structure to build a symmetric interconnection network. For example, in the one-layer networking structure, each processing chip is fully interconnected with each switch chip; in the two-layer networking structure, each processing chip is connected through two levels of switch chips, forming a super node with stronger scalability. These solutions have improved communication performance to some extent, but still cannot balance the demand for low implementation cost and low communication delay, resulting in bottlenecks in cross-network communication and limiting the overall scalability and communication efficiency of the system. SUMMARY

[0003] Therefore, the embodiments of the present disclosure at least provide an inter-chip communication system and method, a switch chip, and a chip product.

[0004] The technical solutions of the embodiments of the present disclosure are implemented as follows: The embodiments of the present disclosure provide an inter-chip communication system, comprising: a plurality of switch networks; Each switch network includes at least one processing chip and at least one switch chip, the switch chip in the switch network is interconnected with each processing chip in the switch network, and there are at least one pair of interconnected switch chips between different switch networks.

[0005] The embodiments of the present disclosure provide an inter-chip communication method applied to the inter-chip communication system in the above embodiments, the method comprising: The first switch chip in the first switch network sends a first request to the second switch chip in the second switch network in response to a first request of the first processing chip in the first switch network; The second switch chip sends the first request to the second processing chip in the second switch network in response to the first request.

[0006] The embodiments of the present disclosure provide a switch chip applied to the inter-chip communication system in the above embodiments, the switch chip comprising: a receiving module configured to receive a first request sent by a first processing chip in a first switch network; the switch chip is distributed in the first switch network; The sending module is configured to send the first request to a second switch chip in a second switch network, so as to send the first request to a second processing chip in the second switch network through the second switch chip.

[0007] The chip product provided by the embodiment of the present disclosure comprises the inter-chip communication system in the above embodiment.

[0008] In the embodiment of the present disclosure, by setting multiple switch networks in the inter-chip communication system, the switch chips in the switch networks are fully interconnected with the processing chips in the switch networks, and the interconnection of at least one pair of switch chips is established between different switch networks. In this way, on the one hand, the switch chip only needs to be directly interconnected with the processing chips in the switch network where the switch chip is located, and does not need to be directly interconnected with the processing chips in other switch networks, so that the data transmission across the switch networks can be realized without significantly increasing the number of ports of a single switch chip, and the requirement for the data switching capability of the switch chip is reduced; on the other hand, the communication between two processing chips located in different switch networks only needs to pass through two switch chips, and the two switch chips are both first-level switch chips directly interconnected with the corresponding processing chips, compared with the inter-chip communication scheme realized by a two-layer networking structure in layers, the communication does not need to pass through a second-level switch chip, so that the number of communication hops is reduced, the communication delay is effectively reduced, and the communication efficiency in a large-scale cluster is improved. In this way, the communication delay can be reduced while the implementation cost is reduced, so that the support for large-scale networks and low-delay communication is taken into account, and the overall expansibility and communication efficiency of the system are improved.

[0009] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings incorporated in the specification and forming a part thereof illustrate embodiments consistent with the present disclosure and together with the description are used to explain the technical solutions of the present disclosure.

[0011] Figure 1 The composition structure of the inter-chip communication system provided by the embodiment of the present disclosure is shown in Figure 1 ; Figure 2 The composition structure of the inter-chip communication system provided by the embodiment of the present disclosure is shown in Figure 2 ; Figure 3 The implementation flowchart of the inter-chip communication method provided by the embodiment of the present disclosure is shown in Figure 4 The composition structure of the switch chip provided by the embodiment of the present disclosure is shown in Figure 5This is a schematic diagram of the composition structure of a chip product provided in an embodiment of the present disclosure; Figure 6 This is a schematic diagram of a single-layer network topology in a related technology. Figure 7 This is a schematic diagram of a two-layer network topology in related technologies. Figure 8 A schematic diagram of the composition structure of a layer 1 switching network provided in an embodiment of this disclosure; Figure 9 A schematic diagram of the composition structure of a cluster implemented using a 1.5-layer network, provided in an embodiment of this disclosure; Figure 10 A schematic diagram of the composition structure of an inter-chip communication system provided in this embodiment of the present disclosure. Figure 3 . Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0016] This disclosure provides an inter-chip communication system. Figure 1 A schematic diagram of the composition structure of an inter-chip communication system provided in this embodiment of the present disclosure. Figure 1 ,like Figure 2 As shown, the inter-chip communication system 100 includes: Multiple switching networks 10; Each of the switch networks 10 includes at least one processing chip 11 and at least one switch chip 12, the switch chip 12 in each switch network 10 is interconnected with each processing chip 11 in the switch network 10, and at least one pair of interconnected switch chips 12 exists between different switch networks 10.

[0017] Here, the processing chip can include, but is not limited to, at least one of a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU), etc. The processing chips in different switch networks can be of the same type or of different types.

[0018] The switch chip refers to an integrated circuit used for data packet forwarding and routing decision in network communication, and is a core communication component in the switch network, responsible for receiving data requests or data packets from the processing chip and forwarding the data requests or data packets to a target node, wherein the target node can include a processing chip interconnected with the switch chip and / or other switch chips interconnected. For example, the switch chip can include, but is not limited to, at least one of an Ethernet switch chip, a peripheral component interconnect express (PCIE) switch chip, etc.

[0019] In some embodiments, the switch chip has a multi-port capability and can process data exchange of multiple links at the same time, thereby improving communication efficiency. In the embodiments of the present disclosure, the switch chip supports communication forwarding functions within and between switch networks.

[0020] The switch network is a basic unit of the entire inter-chip communication system, and each switch network includes a plurality of processing chips and a plurality of switch chips. In the same switch network, the interconnection between the switch chip and the processing chip means that a bidirectional communication link is established between the switch chip and the processing chip. The interconnection between the switch chips in different switch networks means that a bidirectional communication link is established between two switch chips located in different switch networks.

[0021] Within a switch network, each processing chip is connected to each switch chip, i.e., each processing chip can communicate with another processing chip through a switch chip. Thus, a switch network can be regarded as a one-layer networking structure. Between switch networks, switch chips are connected according to a certain topology to realize inter-chip communication between switch chips. The number of processing chips and switch chips in each switch network can be one or more, and the number of processing chips in different switch networks can be the same or different, and the number of switch chips in different switch networks can be the same or different, which is not limited in the embodiments of the present disclosure.

[0022] In some embodiments, a switch network is composed of n processing chips and m switch chips. In a switch network, each processing chip is connected to all m switch chips, forming a full interconnection structure, where n and m are positive integers. The design of a switch network composed of n processing chips and m switch chips ensures that processing chips in the same switch network can directly communicate through any switch chip in the switch network without multiple-hop forwarding, thereby reducing the communication delay between processing chips in the switch network, reducing the processing overhead of intermediate nodes, improving data transmission efficiency, and improving bandwidth utilization. For example, in a certain application scenario, if n = 16 and m = 8, each processing chip will be connected to 8 switch chips, and each switch chip will also be connected to 16 processing chips, thereby forming a highly symmetrical interconnection structure within the same switch network.

[0023] The existence of at least one pair of interconnected switch chips between different switch networks means that at least one pair of switch chips in different switch networks are interconnected. For example, the first switch chip in the first switch network is interconnected with the second switch chip in the second switch network.

[0024] In some embodiments, each switch chip in the first switch network can be interconnected with part of the switch chips in the second switch network. In this way, the requirement for the data switching capability of the switch chip can be reduced, the hardware structure can be simplified, and the implementation cost can be reduced. For example, if there are 8 switch chips in the first switch network and 8 switch chips in the second network, each switch chip in the first switch network can be interconnected with 1 or 2 switch chips in the second switch network.

[0025] In some embodiments, one switch chip in the first switch network can be interconnected with all switch chips in the second switch network. In this way, the communication bandwidth between switch networks can be improved, thereby improving the data transmission capability between switch networks. For example, if there are 8 switch chips in the first switch network and 8 switch chips in the second network, each switch chip in the first switch network can be interconnected with 8 switch chips in the second switch network.

[0026] In some embodiments, there are at least one pair of interconnected switch chips between each of the exchange networks of the inter-chip communication system.

[0027] In the embodiments of the present disclosure, by setting multiple exchange networks in the inter-chip communication system, the switch chips in the exchange network are fully interconnected with each processing chip in the exchange network, and at least one pair of switch chips are interconnected between different exchange networks. In this way, on the one hand, the switch chip only needs to be directly interconnected with the processing chip in the exchange network where the switch chip is located, and does not need to be directly interconnected with each processing chip in other exchange networks, so as to realize data transmission across exchange networks without significantly increasing the number of ports of a single switch chip, and reduce the requirement for data exchange capability of the switch chip; on the other hand, the communication between two processing chips located in different exchange networks only needs to pass through two switch chips, and the two switch chips are both first-level switch chips directly interconnected with the corresponding processing chips, compared with the inter-chip communication scheme realized by two-layer networking structure in layers, without passing through second-level switch chips, thereby reducing the number of communication hops, effectively reducing the communication delay, and improving the communication efficiency in a large-scale cluster. In this way, the communication delay can be reduced while reducing the implementation cost, so as to balance the support for large-scale network and low-delay communication, and improve the overall expansibility and communication efficiency of the system.

[0028] In summary, the inter-chip communication system provided by the embodiments of the present disclosure can realize more efficient and flexible large-scale super-node construction without significantly increasing the hardware complexity of the system, and is suitable for high-performance computing scenarios such as large language model (LLM) training and inference.

[0029] In some embodiments, as shown in Figure 3 The first exchange network 10A and the second exchange network 10B are included in the multiple exchange networks 10, and at least one first switch chip 12A in the first exchange network 10A is one-to-one interconnected with at least one second switch chip 12B in the second exchange network 10B.

[0030] Here, the first exchange network and the second exchange network adopt a one-to-one interconnection mode of switch chips, that is, each first switch chip in the first exchange network is connected with a corresponding second switch chip in the second exchange network.

[0031] It should be noted that the first exchange network can be any suitable exchange network in the inter-chip communication system, and the second exchange network is an exchange network other than the first exchange network in the inter-chip communication system.

[0032] For example, in a system composed of two switching networks, if the first switching network contains 8 first switching chips and the second switching network contains 8 second switching chips, the 8 first switching chips in the first switching network are interconnected with the 8 second switching chips in the second switching network in a one-to-one correspondence. In this way, the multiple switching chips in the switching networks maintain consistent communication capabilities between the two switching networks, reducing the problem of uneven data transmission rates caused by bandwidth bottlenecks in the interconnection links between switching chips.

[0033] In the above embodiments, by interconnecting the switching chips in different switching networks in a one-to-one correspondence, the symmetry and scalability of the system are improved, the data transmission path is more balanced, the communication delay is reduced, and the overall network performance is improved.

[0034] In some embodiments, the switching chips 12 in the same switching network 10 have corresponding numbers, and the switching chips 12 with the same number in different switching networks 10 are interconnected.

[0035] Here, for each switching network, the switching chips in the switching network can be numbered in any suitable numbering manner.

[0036] In some embodiments, the switching chips in each switching network can be numbered respectively according to a target numbering manner. The target numbering manner can include but is not limited to at least one of sequential numbering, random allocation, etc. For example, when the switching chips in a switching network are numbered in a sequential numbering manner, if the switching network contains m switching chips, the switching chips can be numbered in a preset order as 0, 1, 2, …, m-1. For another example, when the switching chips in a switching network are numbered in a random allocation manner, if the switching network contains m switching chips, each number in the number set {0, 1, 2, …, m-1} can be randomly allocated to a switching chip in the switching network.

[0037] The switching chips between different switching networks can be interconnected according to the same number. For example, assuming that there are X switching networks, each switching network has m switching chips, and the m switching chips in each switching network are numbered as 0, 1, 2, …, m-1, respectively, then the 0th switching chip in each switching network can be interconnected, the 1st switching chip in each switching network can be interconnected, the 2nd switching chip in each switching network can be interconnected, …, and the (m-1)th switching chip in each switching network can be interconnected, thereby forming a communication channel across the switching networks. By using the same numbered switching chips for interconnection, the performance bottleneck caused by using a two-level switching chip as a central node in the traditional two-layer networking structure is reduced, and the number of communication hops is also reduced, so that point-to-point communication can be completed through only two switching chips.

[0038] In actual implementation, this interconnection mode not only improves communication efficiency, but also enhances the fault tolerance of the system. Because even if a link fails, the system can still complete communication through other links, thereby ensuring the stability of the entire system.

[0039] For example, in a system composed of two switching networks, if the first switching network contains 8 switching chips, the second switching network also contains 8 switching chips, and the first switching network and the second switching network are interconnected in order according to the number. This system design allows multiple switching chips in the same switching network to maintain consistent communication capabilities between the two switching networks, avoiding the problem of uneven data transmission caused by bandwidth bottlenecks in interconnection links between switching chips.

[0040] In the above embodiments, by assigning numbers to switching chips and interconnecting switching chips with the same numbers in different switching networks, a stable topology structure can be formed, which facilitates unified number management and improves the predictability of routing paths, thereby further simplifying the network topology structure, making the data forwarding path more clear and controllable, and thereby improving the maintainability and communication efficiency of the system.

[0041] In some embodiments, the first switching chip in the first switching network and the corresponding second switching chip in the second switching network have a first number of interconnection links, and the first number is greater than 1.

[0042] When the first number is greater than 1, it means that multiple independent interconnection links are configured between the first switching chip in the first switching network and the corresponding second switching chip in the second switching network. The configuration of these interconnection links improves bandwidth utilization and provides redundant communication capabilities. For example, in the case of multiple interconnection links between the first switching chip and the corresponding second switching chip, data can be transmitted simultaneously using multiple interconnection links at any time, thereby increasing bandwidth, reducing communication delay, and enhancing the fault tolerance of the data transmission process.

[0043] In the above embodiments, by setting multiple interconnection links between the interconnected two switching chips, efficient interconnection of switching chips between different switching networks is achieved, which can improve bandwidth while achieving redundant backup, reduce communication interruption caused by single point failure, thereby reducing communication delay, enhancing the reliability and fault tolerance of the system, and thereby supporting larger scale large model training and inference tasks.

[0044] In some embodiments, the number of second switching networks is a second number, and the first switching network includes a third number of first processing chips and a fourth number of first switching chips; the difference between the first product and the second product is less than a difference threshold, the first product is the product of the first number and the second number, and the second product is the product of the third number and the fourth number.

[0045] The second quantity refers to the total number of second switching networks in the entire inter-chip communication system. For example, in a supernode composed of multiple switching networks, the second switching network is the switching network other than the first switching network in the supernode. If there are 4 switching networks in the inter-chip communication system, and any one of the switching networks is taken as the first switching network, then the remaining 3 switching networks are all second switching networks, that is, the second quantity is 3.

[0046] The third quantity refers to the number of processing chips (i.e., first processing chips) included in the first switching network. In the switching network, the processing chip is used to perform data processing tasks (such as large-scale model training or inference tasks, etc.), and plays a role as a core processing unit, wherein the number of processing chips directly affects the total computing power of the system. The number of processing chips in each switching network can be the same or different, which is not limited in the embodiments of the present disclosure. For example, if the number of processing chips in each switching network is the same, then the number of processing chips included in each switching network in the inter-chip communication system is the third quantity.

[0047] In some embodiments, the number of first processing chips (i.e., the third quantity) in the first switching network can be set according to the computing power requirement of the first switching network.

[0048] The fourth quantity refers to the number of switching chips (i.e., first switching chips) included in the first switching network. The switching chip is responsible for connecting the processing chips and realizing the data transmission between the processing chips. The number of switching chips in the switching network directly affects the internal communication capability of the switching network, and further affects the communication delay and throughput of the entire inter-chip communication system. The number of switching chips in each switching network can be the same or different, which is not limited in the embodiments of the present disclosure. For example, if the number of switching chips in each switching network is the same, then the number of switching chips included in each switching network in the inter-chip communication system is the fourth quantity.

[0049] In some embodiments, the number of first switching chips (i.e., the fourth quantity) in the first switching network can be set according to the internal communication capability requirement of the first switching network.

[0050] It can be understood that the first product is equal to the number of interconnection links (the first quantity) between the first switching network and each second switching network multiplied by the number of second switching networks (the second quantity). Therefore, the first product reflects the total amount of links used by the first switching network for cross-group communication, and the first product is an important indicator for measuring the horizontal communication capability of the first switching network.

[0051] The second product is equal to the number of first processing chips in the first switching network (third number) multiplied by the number of first switching chips in the first switching network (fourth number). Since each first switching chip in the first switching network is fully interconnected with each first processing chip, the second product reflects the communication capability within the first switching network and is a key parameter for measuring the vertical communication efficiency of the first switching network.

[0052] The difference threshold is a preset value for judging the matching degree of the number of inter-switching network communication links and the number of intra-switching network communication links. When the difference between the first product and the second product is less than the difference threshold, it indicates that the inter-switching network communication capability and the intra-switching network communication capability are close, which can ensure balanced utilization of communication bandwidth to prevent communication in one direction from becoming a bottleneck, thereby improving the overall communication efficiency of the system, reducing communication delay, and further supporting larger-scale model training and inference tasks.

[0053] For example, the number of switching networks X, the number of switching chips in each switching network m, the number of processing chips in each switching network n, and the number of interconnection links between two switching chips interconnected between different two switching networks Y, that is, taking a switching network as the first switching network, the first number, the second number, the third number and the fourth number corresponding to the first switching network are Y, X-1, n, m respectively, where Y, X-1, n, m are positive integers, in order to improve the overall bandwidth utilization, the following formula (1) needs to be satisfied: (1); Where k is the difference threshold. It can be understood that under the condition of satisfying the above formula (1), Y×(X 1) and n×m are approximately equal, that is, the number of intra-switching network communication links and the number of inter-switching network communication links are matched as much as possible.

[0054] In the above embodiment, by controlling the number of intra-switching network and inter-switching network communication links to be close, the communication bandwidth of intra-switching network and inter-switching network is matched as much as possible, which helps to maximize the bandwidth utilization efficiency, reduce communication congestion, thereby reducing the decline of communication performance caused by link bottleneck, and improving the throughput and response speed of the entire system.

[0055] In some embodiments, the first product is equal to the second product. That is, the difference threshold is 0, so that by accurately setting the number of intra-switching network and inter-switching network communication links, the communication bandwidth of intra-switching network and inter-switching network is completely matched, which can maximize the communication efficiency and reduce resource waste, thereby further optimizing the communication performance of large-scale clusters.

[0056] For example, the first number, the second number, the third number and the fourth number corresponding to the first switching network are Y, X-1, n and m respectively, the first product is , the second product is , and Y, X-1, n and m satisfy the following formula (2): (2).

[0057] Based on the inter-chip communication system in the above embodiments, an inter-chip communication method is provided. Figure 3 An implementation flow diagram of the inter-chip communication method provided by the embodiments of the present disclosure is shown in Figure 4 , which includes the following steps S301 and S302: In step S301, the first switching chip in the first switching network sends a first request to the second switching chip in the second switching network in response to a first request of the first processing chip in the first switching network.

[0058] In step S302, the second switching chip sends the first request to the second processing chip in the second switching network in response to the first request.

[0059] In the communication process between the first processing chip and the second processing chip, the first processing chip first initiates a communication request (i.e., the first request), which is received by the first switching chip connected to the first processing chip. Then, the first switching chip determines the second switching chip in the second switching network according to the preset routing rule, and forwards the first request to the second switching chip. Next, the second switching chip forwards the first request to the second processing chip. The communication path between the first processing chip and the second processing chip is: first processing chip -> first switching chip -> second switching chip -> second processing chip, which only passes through two-hop switching chips. Compared with the three-hop communication required by the traditional two-layer networking result (e.g., starting processing chip -> first-level switching chip -> second-level switching chip -> first-level switching chip -> target processing chip), the communication path is shortened, the communication delay is significantly reduced, and the overall communication efficiency is improved.

[0060] The first switching chip and the second switching chip can be connected through a physical link (i.e., an interconnection link), which can be one or more to ensure the reliability and bandwidth of communication to meet the demand. In addition, the first switching chip may also need to parse the content of the first request before forwarding the request to determine the routing path of the first request.

[0061] In some embodiments, the first switch chip can determine whether the first request needs to be forwarded across the switch network according to a preset routing rule. If the first request needs to be forwarded across the switch network, the first switch chip can determine the second switch network, the second switch chip and / or the second processing chip according to the routing rule, and forward the first request to the corresponding first switch chip in the second switch network, so as to forward the first request to the second processing chip through the second switch chip, thereby completing the cross-network communication process. The routing rule can include at least one of random routing, load balancing and / or fixed path selection, but is not limited thereto.

[0062] In the embodiments of the present disclosure, by setting multiple switch networks in the inter-chip communication system, the switch chips in the switch network are fully interconnected with each processing chip in the switch network, and the interconnection of at least two switch chips is established between different switch networks. In this way, on the one hand, the switch chip does not need to be directly interconnected with each processing chip in each switch network, thereby realizing cross-network data transmission without significantly increasing the number of ports of a single switch chip, and reducing the requirement for data switching capability of the switch chip; on the other hand, the communication between two processing chips in different switch networks only needs to pass through two switch chips, and the two switch chips are both first-level switch chips directly interconnected with the corresponding processing chips, compared with the inter-chip communication scheme realized by a two-layer networking structure, the communication does not need to pass through a second-level switch chip, thereby reducing the number of communication hops and effectively reducing the communication delay, and improving the communication efficiency in a large-scale cluster. In this way, the communication delay can be reduced while reducing the implementation cost, thereby balancing the support for large-scale networks and low-delay communication, and improving the overall scalability and communication efficiency of the system.

[0063] In some embodiments, the switch chips in the same switch network have corresponding numbers, and the switch chips with the same number in different switch networks are interconnected.

[0064] The first switch chip in the first switch network sends the first request to the second switch chip in the second switch network in response to the first request of the first processing chip in the first switch network, which can include the following step S311: In step S311, the first switch chip receives the first request, determines the second switch network based on the first request, and sends the first request to the second switch chip with the same number as the first switch chip in the second switch network.

[0065] The switch chips with the same number in different switch networks are interconnected, that is, the switch chips with the same number establish direct interconnection links, so that routing can be realized according to the number of the switch chip to forward the first request to the second processing chip through the second switch chip.

[0066] In the above embodiments, when communicating across switching networks, there is no need to use complex routing tables or global address resolution. The location of the target switching chip can be determined simply by the number of the source switching chip, which can effectively reduce communication latency and routing overhead, thereby reducing routing complexity and improving communication efficiency.

[0067] This disclosure provides a switching chip applied to the inter-chip communication system described in the above embodiments. For example... Figure 5 As shown, the switching chip 400 includes: The receiving module 410 is used to receive a first request sent by a first processing chip in the first switching network; the switching chip is distributed in the first switching network. The sending module 420 is used to send the first request to the second switching chip in the second switching network, so that the first request can be sent to the second processing chip in the second switching network through the second switching chip.

[0068] In some embodiments, the receiving module is further configured to: receive a second request forwarded by the second processing chip through the second switching chip; the sending module is further configured to: send the second request to the first processing chip.

[0069] Here, the second request refers to a communication request across the switching network initiated by the second processing chip. When the second processing chip needs to access the services provided by the first processing chip, the second processing chip forwards the second request through the second switching chip connected to it. After receiving the second request, the receiving module of the first switching chip can forward the second request to the first processing chip, thereby realizing communication and cooperation between processing chips across the switching network.

[0070] This disclosure provides a chip product, such as... Figure 6 As shown, the chip product 500 includes the inter-chip communication system 100 in the above embodiments. Here, the chip product 500 can be an integrated chip, and the switching chips and processing chips in the inter-chip communication system 100 can be connected according to a set network topology and integrated into the integrated chip.

[0071] In some implementations, the application scenarios of this chip product may include, but are not limited to, at least one of high-performance computing, large-scale artificial intelligence training clusters, and data center interconnection.

[0072] The descriptions of the above method embodiments, switching chip embodiments, and chip product embodiments are similar to the descriptions of the above inter-chip communication system embodiments, and have similar beneficial effects as the inter-chip communication system embodiments. For technical details not disclosed in the method embodiments, switching chip embodiments, and chip product embodiments of this disclosure, please refer to the descriptions of the inter-chip communication system embodiments of this disclosure for understanding.

[0073] The following describes the application of the embodiments of this disclosure in a real-world scenario, which can be used to implement an inter-chip interconnect topology based on a switching chip.

[0074] Inter-chip interconnects in related technologies often employ single-layer or two-layer networking switching chips to construct flat, symmetrical scaling-up networks. For example, ... Figure 7 As shown, in a single-layer network topology, there are multiple switching chips 61 and multiple GPUs 62. Each switching chip 61 constitutes a data switching plane (hereinafter referred to as a plane), and each GPU 62 is connected to all planes. That is, each GPU 62 and each CPU 63 are interconnected with all switching chips 61. For example, ... Figure 8 As shown, in the two-layer network topology, each NPU63 and each CPU64 is connected by a first-level switching chip L1_Switch, and each first-level switching chip L1_Switch is connected by a second-level switching chip L2_Switch, forming a super node containing multiple NPUs / CPUs.

[0075] In building supernodes, node size and network communication latency affect the performance of data processing tasks (such as large model training and inference tasks). Supernode design should have larger scale and lower latency. When building such supernodes using a single-layer network topology, the radix requirement for the switching chip is high. For example, to implement a supernode containing 512 GPUs, a switching chip with a radix of 512 is needed, and a high radix increases the design complexity and cost of the switching chip. If a two-layer networking approach is used, it will lead to increased latency in point-to-point communication between GPUs. Compared to the single-hop point-to-point communication of a single-layer network (GPU->Switching Chip->GPU), the point-to-point communication of a two-layer network requires three hops (GPU->L1_switch->L2_switch->L1_switch->GPU).

[0076] In view of this, embodiments of this disclosure propose a compromise construction scheme for an inter-chip communication system that can reduce the switch radix requirement for large-scale supernodes while achieving lower latency than point-to-point communication in two-layer networking. The compromise network construction scheme proposed in this disclosure is implemented through a new communication path: GPU->L1_switch->L1_switch->GPU, referred to here as a Layer 1.5 network. The network construction scheme in this disclosure balances support for large-scale networks and low-latency communication.

[0077] The inter-chip communication system provided in this disclosure includes multiple layer-1 switching networks (corresponding to the switching networks in the above embodiments). Figure 8 This is a schematic diagram of the composition structure of a layer 1 switching network provided in an embodiment of the present disclosure, as shown below. Figure 9 As shown, a single-layer switching network 80 includes multiple groups of GPUs (e.g., GPU_Tray0~GPU_Tray17) and multiple groups of switching chips (e.g., Switch_Tray0~Switch_Tray7). Each GPU (e.g., GPU0~GPU(n-1)) is fully interconnected to a switching chip (e.g., Switch0~Switch(m-1)). Each GPU in the single-layer switching network has m interconnection links. The single-layer switching network 80 contains a total of n GPUs and m switching chips. Each switching chip uses n links to interconnect with each GPU. Each switching chip in each single-layer switching network 80 can be considered a plane.

[0078] For the entire cluster (i.e., the inter-chip communication system) implemented by a 1.5-layer network, X 1-layer switching networks can be configured. Figure 9 This is a schematic diagram of the composition structure of a cluster implemented using a 1.5-layer network, as provided in an embodiment of this disclosure. Figure 9 As shown, taking X=8 as an example, this cluster includes 8 layer-1 switching networks (such as Compute_Rack0~Compute_Rack7). Switching chips with the same number are interconnected between different layer-1 switching networks. For example, there are Y links between Switch0 in Compute_Rack0 and Switch0 in Compute_Rack1. Figure 10 Only one link is shown in the diagram. For each of the X layer-1 switching networks, each switching chip in the network is numbered 0, 1, ..., m-1. Switching chips with the same number belong to the same plane.

[0079] The connection method between the switching chips is as follows: each switching chip has A link is used for connection between switching chips in other switching networks; each switching chip is connected to another switching chip on the same plane by Y links for interconnection.

[0080] It should be noted that the values ​​of the four parameters m, n, X, and Y mentioned above can be designed by those skilled in the art according to the actual application scenario, and the embodiments disclosed herein do not limit this.

[0081] For example, m, n, X, and Y can be set to 1, 8, 4, and 1 respectively, such as... ​As shown, an integrated switching chip (corresponding to the inter-chip communication system or chip product in the aforementioned embodiments) can be constructed using four switching chips. This integrated switching chip includes X=4 switching networks, and the four switching networks include four switching chips. The four switching chips are divided into four groups (such as Group0, Group1, Group2, and Group3). Each switching network includes m=1 switching chip Switch0 and n=8 GPUs (such as GPU0~7, GPU8~15, GPU16~23, or GPU24~31). Each Switch0 is connected to the corresponding eight GPUs in the same switching network, and Y=1 links are set between each Switch0.

[0082] In summary, the 1.5-layer networking topology provided in this disclosure can effectively reduce the reliance on the high data exchange capacity of switches while maintaining low communication latency. It is suitable for the interconnection needs of large-scale GPU clusters, not only improving bandwidth utilization but also possessing good scalability, making it suitable for the construction of larger-scale intelligent computing networks in the future.

[0083] It should be noted that, in the embodiments of this disclosure, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0084] This disclosure provides a switching chip, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0085] This disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0086] This disclosure provides a computer program including computer-readable code. When the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0087] This disclosure provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0088] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the chip, storage medium, computer program, and computer program product embodiments above are similar to the descriptions of the inter-chip communication system embodiments above, and have similar beneficial effects to the inter-chip communication system embodiments. For technical details not disclosed in the chip, storage medium, computer program, and computer program product embodiments of this disclosure, please refer to the descriptions of the inter-chip communication system embodiments of this disclosure for understanding.

[0089] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0091] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0092] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may all be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in hardware or in a combination of hardware and software functional units.

[0093] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0094] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause the switching chip to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0095] The above are merely embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An inter-chip communication system, characterized in that, include: Multiple switching networks; Each of the switching networks includes at least one processing chip and at least one switching chip. The switching chip in the switching network is interconnected with each of the processing chips in the switching network, and there is at least one pair of interconnected switching chips between different switching networks.

2. The inter-chip communication system according to claim 1, characterized in that, Each switching chip in the same switching network has a corresponding number, and switching chips with the same number in different switching networks are interconnected.

3. The inter-chip communication system according to claim 1 or 2, characterized in that, The plurality of switching networks includes a first switching network and a second switching network, wherein at least one first switching chip in the first switching network is interconnected with at least one second switching chip in the second switching network in a one-to-one correspondence.

4. The inter-chip communication system according to claim 3, characterized in that, The first switching chip in the first switching network and the corresponding second switching chip in the second switching network have a first number of interconnection links, where the first number is greater than 1.

5. The inter-chip communication system according to claim 4, characterized in that, The number of the second switching network is the second number, and the first switching network includes a third number of first processing chips and a fourth number of first switching chips; The difference between the first product and the second product is less than the difference threshold. The first product is the product of the first quantity and the second quantity, and the second product is the product of the third quantity and the fourth quantity.

6. The inter-chip communication system according to claim 5, characterized in that, The first product is equal to the second product.

7. An inter-chip communication method, characterized in that, Applied to the inter-chip communication system of any one of claims 1 to 6, the method comprises: In response to a first request from a first processing chip in the first switching network, a first switching chip in the first switching network sends the first request to a second switching chip in the second switching network. In response to the first request, the second switching chip sends the first request to the second processing chip in the second switching network.

8. The inter-chip communication method according to claim 7, characterized in that, Each switching chip in the same switching network has a corresponding number, and switching chips with the same number in different switching networks are interconnected. In response to a first request from a first processing chip in the first switching network, a first switching chip in the first switching network sends the first request to a second switching chip in the second switching network, including: Upon receiving the first request, the first switching chip determines the second switching network based on the first request and sends the first request to the second switching chip in the second switching network that has the same number as the first switching chip.

9. A switching chip, characterized in that, The switching chip, used in any one of claims 1 to 6, comprises: A receiving module is used to receive a first request sent by a first processing chip in a first switching network; the switching chips are distributed in the first switching network. The sending module is used to send the first request to the second switching chip in the second switching network, so that the first request can be sent to the second processing chip in the second switching network through the second switching chip.

10. The switching chip according to claim 9, characterized in that, The receiving module is further configured to: receive a second request forwarded by the second processing chip through the second switching chip; The sending module is further configured to: send the second request to the first processing chip.

11. A chip product, characterized in that, Includes the inter-chip communication system as described in any one of claims 1 to 6.

Images