Computing systems, methods and media

Abstract

This disclosure relates to a computing system, method, and medium. The accelerator system includes multiple parallel processing units configured in one or more fully connected computing clusters. Input data is segmented based on the number of parallel processing units in a given fully connected computing cluster and loaded onto corresponding parallel processing units in the given fully connected computing cluster. Function computation is performed on the loaded input data using the given computing cluster.

Description

[0001] computing field

[0002] This disclosure relates to the field of chips, and more particularly to a computing system, method, and medium. Background Technology

[0003] Currently, parallel / distributed training methods for deep neural networks involve applying synchronous large and small batch stochastic gradient descent (SDG) processing across multiple distributed computing nodes to explore data parallelism-based acceleration. (See reference...) Figure 1 This illustrates an exemplary small-batch SDG processing run on the main CPU, including pseudocode. The processing is affected by a synchronization component, which acts as a bottleneck throughout the parallel acceleration process. Figure 2 As shown, in order to reduce bottlenecks, it is necessary to increase the bandwidth of the accelerator-side network and / or reduce the frequency of communication with the main accelerator.

[0004] Many synchronization algorithms have been proposed for mini-batch SDG processing. Reduce and All_Reduce functions are common communication patterns between computing nodes. (See reference...) Figure 3 The diagram illustrates the collapse function. In the collapse function, the value sets of multiple nodes, namely nodes 310 to 340, are passed to a given node 310. The given node 310 sums the corresponding values. The given node 310 stores the sum of the value sets. For example, the first node 310 receives the values ​​5, 2, 7, and 4 from nodes 310 to 340, sums these values, and stores the sum 18. The first node 310 also sums the values ​​1, 3, 8, and 2 and stores the sum 14. (See also...) Figure 4 The diagram illustrates a full collapse function. In this function, multiple nodes (nodes 410 to 440) are given their respective value sets to a given node 410, which then sums the corresponding values. The sum set is broadcast from the given node 410 to nodes 410 to 440, and each of these nodes stores the sum set. For example, the first node 410 sums the values ​​5, 2, 7, and 4 received from nodes 410 to 440. The first node 410 also sums the values ​​1, 3, 8, and 2. The first node 410 broadcasts the sum set of 18 and 14 to nodes 410 to 440, and each of these nodes stores the sum set. Figure 3 and Figure 4 As shown, the collapse function and the full collapse function are applied to a set of variables simultaneously.

[0005] However, improvements are needed in high-speed chip-to-chip serialization / deserialization (SerDes) interconnects to enable distributed systems capable of computing fully collapsed functions within chip clusters, rather than on distributed computers connected via slower Ethernet, InfiniBand, or similar communication links. Summary of the Invention

[0006] This disclosure can be better understood with reference to the following description and accompanying drawings, in which embodiments of this disclosure are illustrated in relation to hierarchical scaling of a plurality of fully connected parallel processing units (PPUs).

[0007] One objective of this disclosure is to enable distributed systems that can implement computationally collapsed or fully collapsed functions within a chip cluster.

[0008] In one embodiment, a computing system includes: a plurality of parallel processing units, wherein each parallel processing unit is coupled to each other parallel processing unit via a corresponding communication link, and the corresponding communication links of the plurality of parallel processing units are configured to one or more fully connected computing clusters based on specified computing parameters.

[0009] Optionally, the communication link includes a bidirectional communication link.

[0010] Optionally, the specified computation parameters include the number of parallel processing units in a given computing cluster.

[0011] Optionally, the specified calculation parameters include the size of the calculation processing bandwidth.

[0012] Optionally, each of the one or more fully connected computing clusters is configured to perform a corresponding collapse or full collapse function calculation on the corresponding input data.

[0013] Optionally, the multiple parallel processing units are configured in a fully connected computing cluster consisting of eight parallel processing units through corresponding communication links.

[0014] Optionally, the plurality of parallel processing units are configured in two fully connected computing clusters, each consisting of four parallel processing units, via corresponding communication links.

[0015] Optionally, the plurality of parallel processing units are configured in four fully connected computing clusters, each consisting of two parallel processing units, via corresponding communication links.

[0016] Optionally, the plurality of parallel processing units are configured via corresponding communication links in a fully connected computing cluster consisting of four parallel processing units and two fully connected computing clusters each consisting of two parallel processing units.

[0017] In one embodiment, a calculation method includes:

[0018] Based on specified computation parameters, the communication links of multiple parallel processing units are configured to one or more fully connected computing clusters;

[0019] Based on the number of parallel processing units in a given fully connected computing cluster, partition the input data; and

[0020] The input data is used to perform function computations through the given fully connected computing cluster.

[0021] Optionally, the specified computation parameters include the number of parallel processing units in a given computing cluster.

[0022] Optionally, the specified calculation parameters include the size of the calculation processing bandwidth.

[0023] Optionally, the plurality of parallel processing units are configured to a fully connected computing cluster consisting of eight parallel processing units via corresponding communication links; and the input data is divided into eight parts and loaded onto the corresponding parallel processing units among the eight parallel processing units.

[0024] Optionally, the calculation method further includes:

[0025] Based on the specified computational parameters, the communication links of the plurality of parallel processing units are configured to be distributed among the plurality of fully connected computing clusters. Specifically, the communication links between parallel processing units in a given fully connected computing cluster are configured to send data to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, the parallel processing units in other fully connected computing clusters are configured to relay data from the parallel processing units in the given fully connected computing cluster to other parallel processing units in the given fully connected computing cluster via the communication links between the parallel processing units in the given fully connected computing cluster and the parallel processing units in the other fully connected computing clusters.

[0026] The input data is computed using a collapse function or a full collapse function through the given fully connected computing cluster. Specifically, on the communication links between parallel processing units in the given fully connected computing cluster, the data of the collapse function or full collapse function is sent to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, on the communication links between a parallel processing unit in the given fully connected computing cluster and parallel processing units in other fully connected computing clusters, one or more parallel processing units in the other fully connected computing clusters relay the data of the collapse function or full collapse function.

[0027] Optionally, the plurality of parallel processing units are configured to two fully connected computing clusters, each consisting of four parallel processing units, via corresponding communication links; and the input data is divided into four parts and loaded onto the corresponding parallel processing units of the four parallel processing units in the given fully connected computing cluster.

[0028] Optionally, the plurality of parallel processing units are configured to four fully connected computing clusters, each consisting of two parallel processing units, via corresponding communication links; and the input data is divided into two parts and loaded onto the corresponding parallel processing units in the two parallel processing units of the given fully connected computing cluster.

[0029] Optionally, the communication link includes a bidirectional communication link.

[0030] In one embodiment, one or more computing device-readable media are provided having instructions stored thereon, which, when executed by one or more processing units, perform a method comprising the following steps:

[0031] Based on specified computation parameters, the communication links of multiple parallel processing units are configured to one or more fully connected computing clusters;

[0032] Input data is partitioned based on the number of parallel processing units in a given fully connected computing cluster; and

[0033] The input data is used to perform function computations through the given fully connected computing cluster.

[0034] Optionally, the plurality of parallel processing units are configured in a fully connected computing cluster consisting of eight parallel processing units via corresponding communication links; and

[0035] The input data is divided into eight parts and loaded onto the corresponding parallel processing units in the eight parallel processing units.

[0036] Optionally, the method further includes:

[0037] Based on the specified computational parameters, the communication links of the plurality of parallel processing units are configured to be distributed among the plurality of fully connected computing clusters. Specifically, the communication links between parallel processing units in a given fully connected computing cluster are configured to send data to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, the parallel processing units in other fully connected computing clusters are configured to relay data from the parallel processing units in the given fully connected computing cluster to other parallel processing units in the given fully connected computing cluster via the communication links between the parallel processing units in the given fully connected computing cluster and the parallel processing units in the other fully connected computing clusters.

[0038] The input data is computed using a collapse function or a full collapse function through the given fully connected computing cluster. Specifically, on the communication links between parallel processing units in the given fully connected computing cluster, the data of the collapse function or full collapse function is sent to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, on the communication links between a parallel processing unit in the given fully connected computing cluster and parallel processing units in other fully connected computing clusters, one or more parallel processing units in the other fully connected computing clusters relay the data of the collapse function or full collapse function.

[0039] The above approach enables the implementation of distributed systems that compute collapse or full collapse functions within a chip cluster.

[0040] The summary section is provided to briefly introduce the inventive concept, which will be further described in the following detailed description. The purpose of this summary section is not to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0041] In the accompanying drawings, embodiments of the present disclosure are shown by way of example rather than limitation, wherein similar reference numerals denote similar elements, and wherein:

[0042] Figure 1 An exemplary small-batch SDG processing method according to the prior art is shown.

[0043] Figure 2 This illustrates another exemplary small-batch SDG processing method based on existing technology.

[0044] Figure 3 The collapse function calculation based on existing technology is shown.

[0045] Figure 4 The calculation of the full collapse function according to the prior art is shown.

[0046] Figure 5 Multiple parallel processing units with fully connected hierarchical scaling are shown according to embodiments of the present disclosure.

[0047] Figure 6 The diagram illustrates the computation of a collapse or full collapse function over multiple parallel processing units that are fully connected and provide hierarchical scaling, according to embodiments of the present disclosure.

[0048] Figure 7 A hierarchical scaling method for a plurality of fully connected parallel processing units is illustrated according to an embodiment of the present disclosure.

[0049] Figure 8 A computing cluster of multiple parallel processing units with hierarchical scaling is shown according to an embodiment of the present disclosure.

[0050] Figure 9 The diagram illustrates the computation of a collapsed or fully collapsed function on a computing cluster of multiple parallel processing units that are fully connected and provide hierarchical scaling, according to embodiments of the present disclosure.

[0051] Figure 10 A computing cluster of multiple parallel processing units with hierarchical scaling is shown according to an embodiment of the present disclosure.

[0052] Figure 11 This illustration shows a plurality of parallel processing units that are segmented and fully connected, providing hierarchical scaling according to embodiments of the present disclosure.

[0053] Figure 12 An enhanced hierarchical scaling method for a plurality of fully connected parallel processing units is illustrated according to an embodiment of the present disclosure.

[0054] Figure 13 A computing cluster of multiple parallel processing units with hierarchical scaling is shown according to an embodiment of the present disclosure.

[0055] Figure 14 A computing cluster of multiple parallel processing units with hierarchical scaling is shown according to an embodiment of the present disclosure.

[0056] Figure 15 A computing cluster of multiple parallel processing units with hierarchical scaling is shown according to an embodiment of the present disclosure.

[0057] Figure 16 An exemplary computing system comprising multiple parallel processing units is shown according to an embodiment of the present disclosure.

[0058] Figure 17 An exemplary parallel processing unit according to an embodiment of this disclosure is shown. Specific Implementation

[0059] Embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. Although this disclosure will be described in conjunction with embodiments, it should be understood that it is not intended to limit this disclosure to these embodiments. Rather, this disclosure is intended to cover alternatives, modifications, and equivalents within the scope of this disclosure as defined by the appended claims. Furthermore, numerous specific details are set forth in the following detailed description of embodiments of this disclosure in order to provide a thorough understanding of this disclosure. However, it should be understood that this disclosure can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid obscuring aspects of this disclosure.

[0060] The embodiments of this disclosure described below are presented in the form of routines, modules, logic blocks, and other symbolic forms of operations on data within one or more electronic devices. These descriptions and forms are the manner adopted by those skilled in the art to most effectively convey the substance of the work to them. In this document, routines, modules, logic blocks, and / or the like are generally considered to be a self-consistent sequence of processes or instructions that lead to a desired result. These processes include physical operations on physical quantities. Although not strictly necessary, these physical operations are typically in the form of electrical or magnetic signals capable of being stored, transmitted, compared, and otherwise manipulated within an electronic device. For convenience, and in reference to common usage, with reference to embodiments of this disclosure, these signals are referred to as data, bits, values, elements, symbols, characters, terms, numbers, strings, etc.

[0061] However, it should be remembered that these terms will be interpreted as references to physical operations and physical quantities, and are merely convenient notations, and will be further interpreted according to terminology commonly used in the art. Unless explicitly stated in the discussion below, it should be understood that throughout the discussion of embodiments of this disclosure, the use of terms such as “receiving” refers to the actions or processes of an electronic device, such as an electronic computing device that manipulates and converts data. Data is represented as physical (e.g., electronic) quantities within the logic circuits, registers, memory, etc., of the electronic device, and is converted into other data similarly represented as physical quantities within the electronic device.

[0062] In this disclosure, the use of disjunctive terms is intended to include conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, references to “the” object or “an” object are also intended to indicate one of a plurality of such objects that may exist. The use of terms “comprising,” “including,” etc., specifies the presence of the said element but does not exclude the presence or addition of one or more other elements and / or groups of elements. It should also be understood that although the terms first, second, etc., may be used to describe different elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the embodiments. It should also be understood that when an element is referred to as “coupled” to another element, it may be directly or indirectly connected to the other element, or an intermediate element may be present. Conversely, when an element is referred to as “directly connected” to another element, no intermediate element is present. It should also be understood that the term “and / or” includes any and all combinations of one or more associated elements. It should also be understood that the phrases and terms used here are for descriptive purposes and should not be considered restrictive.

[0063] Now refer to Figure 5 , Figure 5 A plurality of fully connected parallel processing units providing hierarchical scaling is illustrated according to embodiments of the present disclosure. The plurality of parallel processing units can be configured to be coupled to each other via multiple communication links. For example, the fully connected plurality of parallel processing units includes eight parallel processing units, each including seven communication ports. Each of the eight parallel processing units can be directly connected to each of the other seven parallel processing units via its own seven configurable communication links. In one embodiment, each communication link is a bidirectional communication link.

[0064] The fully connected parallel processing units can be used to compute different functions such as collapse and full collapse. For example, to compute the collapse function, the input data can be divided into eight parts and loaded into each of the eight fully connected parallel processing units. Figure 6As shown, the first parallel processing unit 610 can receive corresponding data values ​​from each of the parallel processing units 610 to 680 on the corresponding communication links. The first parallel processing unit 610 can add the corresponding received data values ​​and store the sum in its buffer. Similarly, the other seven parallel processing units (parallel processing units 620 to 680) can also receive corresponding data values ​​from each of the parallel processing units 610 to 680. Each of the other parallel processing units can add its respective received data values ​​and store its respective sum in its buffer. In another example, the fully collapsed function can be computed by dividing the input data into eight parts and loading the divided parts of the input data into each of the eight fully connected parallel processing units. Figure 6 As shown, the first parallel processing unit 610 can receive corresponding data values ​​from each of the parallel processing units 610 to 680 on the corresponding communication link. The first parallel processing unit 610 can add the corresponding received data values ​​together. Figure 6 As shown, the first parallel processing unit 610 can then broadcast the corresponding sum value to parallel processing units 610 to 680 on the respective communication links. Similarly, the other seven parallel processing units (parallel processing units 620 to 680) can also receive corresponding data values ​​from each of the parallel processing units 610 to 680. Each other parallel processing unit can add its respective received data values. Each other parallel processing unit can also broadcast the corresponding sum value to the corresponding parallel processing unit on the respective communication link.

[0065] Reference Figure 7 Further explanation of the hierarchical scaling of parallel processing units, Figure 7 A hierarchical scaling method for a plurality of fully connected parallel processing units according to an embodiment of the present disclosure is illustrated. In step 710, communication links for the plurality of parallel processing units are configured to one or more fully connected computing clusters based on specified computational parameters. In one embodiment, the computational parameters may be the number of parallel processing units in a given computing cluster, for example, eight, four, or two parallel processing units in a given computing cluster. In another embodiment, the computational parameters may be the size of the computational processing bandwidth. The computational processing bandwidth may be mapped to a given number of parallel processing units. In one embodiment, such as Figure 5As shown, multiple parallel processing units can be configured in a computing cluster of fully connected parallel processing units that are communicatively coupled via bidirectional communication links. In some cases, applications may not require a computing cluster comprising all the multiple parallel processing units to compute collapsed, fully collapsed, or other similar functions. In other cases, such as cloud computing services, customers may want to choose whether to pay for the computing bandwidth of eight, four, or two parallel processing units.

[0066] Accordingly, in another embodiment, such as Figure 8 As shown, parallel processing units 505 to 540 can be configured in two computing clusters, each consisting of four parallel processing units, namely computing cluster 805 and computing cluster 810. Computing cluster 805 consists of parallel processing units 505 to 520, and computing cluster 810 consists of parallel processing units 525 to 540. Communication links can be configured by enabling a given subset of communication links interconnecting the parallel processing units in each respective computing cluster and disabling other communication links between parallel processing units in different computing clusters. For example, a subset of communication links between parallel processing units 505 to 520 in a fully connected first computing cluster 805 can be enabled. Similarly, communication links between parallel processing units 525 to 540 in a fully connected second computing cluster 810 can be enabled. However, communication links between parallel processing units in the first computing cluster 805 and the second computing cluster 810 can be disabled. In another embodiment, as... Figure 10 As shown, parallel processing units 505 to 540 can be configured in four computing clusters, each consisting of two parallel processing units, namely computing cluster 1005, computing cluster 1010, computing cluster 1015, and computing cluster 1020. Computing cluster 1005 consists of parallel processing units 505 and 510. Computing cluster 1010 consists of parallel processing units 515 and 520. Computing cluster 1015 consists of parallel processing units 525 and 530. Computing cluster 1020 consists of parallel processing units 535 and 540. In another embodiment, as... Figure 11 As shown, parallel processing units 505 to 540 can be configured in two computing clusters each consisting of two parallel processing units and one computing cluster consisting of four parallel processing units, namely computing cluster 1105, computing cluster 1110, and computing cluster 1115. Computing cluster 1105 consists of parallel processing units 505 and 510. Computing cluster 1110 consists of parallel processing units 515 and 520. Computing cluster 1115 consists of parallel processing units 525 to 540. Figure 6 , Figure 8 , Figure 10 and Figure 11 The exemplary configurations shown are just some possible configurations for configuring multiple parallel processing units in one or more computing clusters.

[0067] Refer again Figure 7 The hierarchical scaling method for multiple parallel processing units in a fully connected computing cluster further includes: in step 720, to compute a collapse, full collapse, or similar function on a given fully connected computing cluster, segmenting the input data and loading the corresponding segments onto the corresponding parallel processing units of the given fully connected computing cluster. For example, in a computing cluster with eight parallel processing units, the input data can be divided into eight parts and loaded onto the corresponding parallel processing units of the given fully connected computing cluster. For a computing cluster with four parallel processing units, the input data can be divided into four parts and loaded onto the corresponding parallel processing units of the given fully connected computing cluster. For a computing cluster with two parallel processing units, the input data can be divided into two parts and loaded onto the corresponding parallel processing units of the given fully connected computing cluster.

[0068] In step 730, the input data is computed using a collapsed, fully collapsed, or similar function on a given fully connected computing cluster. For example, in a fully connected computing cluster with eight parallel processing units, refer to the above... Figure 6 The calculation of the collapse or full collapse function is performed as described above.

[0069] like Figure 9 As shown, in the example of a fully connected computing cluster with four parallel processing units for calculating the collapse function, the first parallel processing unit 505 can receive corresponding data values ​​from each of the parallel processing units 505 to 520 of the given computing cluster 805 on the corresponding communication link. The first parallel processing unit 505 can add the corresponding received data values ​​and store the sum in its buffer. Similarly, the other parallel processing units of the given computing cluster 805, namely parallel processing units 510 to 520, can receive other corresponding data values ​​from the parallel processing units of the given computing cluster 805. Each other parallel processing unit can add its respective corresponding received data values ​​and store the sum in the buffer of the corresponding parallel processing unit. Figure 9As shown, in the example of four parallel processing units computing a fully collapsed function, the first parallel processing unit 505 can receive corresponding data values ​​from each of the parallel processing units 505 to 520 of the given computing cluster 805 on the corresponding communication link. The first parallel processing unit 505 can sum the corresponding received data values. Then, the first parallel processing unit 505 can broadcast the corresponding sum to the parallel processing units 505 to 520 of the given computing cluster 805 on the corresponding communication link. Similarly, the other three parallel processing units, namely parallel processing units 510 to 520, can receive other corresponding data values ​​from each of the parallel processing units 505 to 520. Each other parallel processing unit can sum its respective received data values. Each other parallel processing unit can also broadcast the corresponding sum to the corresponding parallel processing unit on the corresponding communication link. It should also be understood that the computing cluster 810 of the other four parallel processing units can be configured to compute another function for the same application, another application with the same client, different applications, etc.

[0070] like Figure 10 As shown, in a fully connected computing cluster with two parallel processing units for calculating the collapse function, the first parallel processing unit 505 can receive corresponding data values ​​from other parallel processing units 510 of the given computing cluster 1005 on the corresponding communication link. The first parallel processing unit 505 can add the corresponding received data values ​​and store the sum in its cache. Similarly, other parallel processing units 510 of the given computing cluster 1005 can also receive corresponding data values ​​from the first parallel processing unit 505 of the given computing cluster 1005. The other parallel processing units 510 can add the corresponding received data values ​​and store the sum in their caches. Figure 10As shown, in the example of two parallel processing units computing a fully collapsed function, the first parallel processing unit 505 can receive corresponding data values ​​from other parallel processing units 510 of a given computing cluster 1005 on the corresponding communication link. The first parallel processing unit 505 can add the corresponding received data values. Then, the first parallel processing unit 505 can broadcast the corresponding sum to the other parallel processing units 510 of the given computing cluster 1005 on the corresponding communication link. Similarly, the other parallel processing units 510 can also receive corresponding data values ​​from the first parallel processing unit 505. The other parallel processing units 510 can add the corresponding received data values. The other parallel processing units can also broadcast the corresponding sum to the first parallel processing unit 505 on the corresponding communication link. It should be understood that for the same application, another application with the same client, different applications, etc., the computing clusters of the other two parallel processing units, namely computing cluster 1010, computing cluster 1015, and computing cluster 1020, can be configured to compute another function.

[0071] In some cases, communication links within a given computing cluster of fully connected, parallel processing units can become a processing bottleneck. Furthermore, communication links within another computing cluster may be underutilized. Figure 12 An enhanced hierarchical scaling method for a fully connected plurality of parallel processing units is illustrated according to an embodiment of the present disclosure. (Refer to...) Figures 13-15This will further explain the enhanced hierarchical scaling of multiple parallel processing units in a fully connected computing cluster. The enhanced hierarchical scaling method for multiple parallel processing units in a fully connected computing cluster includes, in step 1210, configuring communication links for multiple parallel processing units across multiple fully connected computing clusters based on specified computational parameters. In one embodiment, the computational parameter is the number of parallel processing units in a given computing cluster, such as eight, four, or two parallel processing units. In another embodiment, the computational parameter is the size of the computational processing bandwidth. In one embodiment, each communication link can be a bidirectional communication link. Communication links between parallel processing units in a given fully connected computing cluster can be configured to send data to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, parallel processing units in other fully connected computing clusters can be configured to relay data from parallel processing units in a given fully connected computing cluster to other parallel processing units in the given fully connected computing cluster on communication links between parallel processing units in the given fully connected computing cluster and other parallel processing units in other fully connected computing clusters. Parallel processing units in other fully connected computing clusters can be used to provide single-hop transmissions to relay data, thereby increasing communication bandwidth. Typically, data relaying between parallel processing units can be performed with little or no utilization of the relayed parallel processing unit's data processing bandwidth. Therefore, the available communication bandwidth on communication links between parallel processing units in different computing clusters can be used to increase the communication bandwidth between parallel processing units in the same computing cluster.

[0072] In one embodiment, such as Figure 13As shown, the plurality of parallel processing units includes eight parallel processing units, namely parallel processing units 505 to 540. These eight parallel processing units can be configured in two fully connected computing clusters, each consisting of four parallel processing units: fully connected computing cluster 1305 and fully connected computing cluster 1310. Fully connected computing cluster 1305 consists of parallel processing units 505 to 520. Fully connected computing cluster 1310 consists of parallel processing units 525 to 540. The communication link (shown as a solid line) between parallel processing units 505 to 520 in the first fully connected computing cluster 1305 can be configured to send data to the corresponding parallel processing unit in the first fully connected computing cluster 1305. Furthermore, the parallel processing units 525 to 540 in the second fully connected computing cluster 1310 can be configured to relay data from the parallel processing units 505 to 520 in the first fully connected computing cluster 1305 to other parallel processing units in the second fully connected computing cluster 1310 via a communication link (as shown by the dashed line). When using a parallel processing unit in another fully connected computing cluster to relay data, a given parallel processing unit 520 in the first fully connected computing cluster 1305 can randomly select any parallel processing unit 525 to 540 in the second fully connected computing cluster 1310 and the corresponding communication link to relay data. In another embodiment, a given parallel processing unit 510 in the first fully connected computing cluster 1305 can select a corresponding given parallel processing unit 525 in the second fully connected computing cluster 1310 and the corresponding communication link to relay data.

[0073] In another example, such as Figure 14As shown, the plurality of parallel processing units includes eight parallel processing units, namely parallel processing units 505 to 540. These eight parallel processing units can be configured in four fully connected computing clusters, each consisting of two parallel processing units: fully connected computing cluster 1405, fully connected computing cluster 1410, fully connected computing cluster 1415, and fully connected computing cluster 1420. Fully connected computing cluster 1405 consists of parallel processing units 505 and 510. Fully connected computing cluster 1410 consists of parallel processing units 515 and 520. Fully connected computing cluster 1415 consists of parallel processing units 525 and 530. Fully connected computing cluster 1420 consists of parallel processing units 535 and 540. The communication link (shown as a solid line) between parallel processing units 505 and 510 in the first fully connected computing cluster 1405 can be configured to send data to other parallel processing units in the first fully connected computing cluster 1405. Furthermore, the parallel processing units 515 to 540 in other fully connected computing clusters, namely fully connected computing clusters 1410 to 1420, can be configured to relay data from parallel processing units in the first fully connected computing cluster 1405 to other parallel processing units in the first fully connected computing cluster 1405 via the communication links (as shown by the dashed lines) between parallel processing units 505 to 510 in the first fully connected computing cluster 1405 and parallel processing units 515 to 540 in the fully connected computing clusters 1410 to 1420. When using parallel processing units in another fully connected computing cluster to relay data, a given parallel processing unit 510 in the first fully connected computing cluster 1405 can randomly select any parallel processing unit 515 to 540 in the fully connected computing clusters 1410 to 1420 and the corresponding communication link to relay data. In another embodiment, a given parallel processing unit 510 in the first fully connected computing cluster 1405 may, for example, select a corresponding given parallel processing unit 535 in the third computing cluster 1420 and a corresponding communication link to relay data.

[0074] In yet another example, such as Figure 15As shown, the plurality of parallel processing units includes eight parallel processing units, namely parallel processing units 505 to 540. These eight parallel processing units can be configured in three fully connected computing clusters, including a fully connected computing cluster 1505 consisting of four parallel processing units and two fully connected computing clusters each consisting of two parallel processing units, namely fully connected computing cluster 1510 and fully connected computing cluster 1515. Fully connected computing cluster 1505 consists of parallel processing units 505 to 520. Fully connected computing cluster 1510 consists of parallel processing units 525 and 530. Fully connected computing cluster 1515 consists of parallel processing units 535 and 540. The communication link (shown as a solid line) between parallel processing units 505 to 520 in the first fully connected computing cluster 1505 can be configured to send data to other parallel processing units in the first fully connected computing cluster 1505. Furthermore, the parallel processing units 525 to 540 in other fully connected computing clusters, namely fully connected computing clusters 1510 to 1515, can be configured to relay data from parallel processing units in the first fully connected computing cluster 1505 to other parallel processing units in the first fully connected computing cluster 1505 via the communication link between parallel processing units 505 to 520 in the first fully connected computing cluster 1405 and parallel processing units 525 to 540 in the fully connected computing clusters 1510 to 1515 (as shown by the dashed line). When using parallel processing units in another fully connected computing cluster to relay data, a given parallel processing unit 515 in the first fully connected computing cluster 1505 can randomly select any parallel processing unit from the parallel processing units 525 to 540 in the fully connected computing clusters 1510 to 1515 and the corresponding communication link to relay data. In another embodiment, a given parallel processing unit 515 in the first fully connected computing cluster 1505 may select a corresponding given parallel processing unit 530 and a corresponding communication link in another fully connected computing cluster 1510 to relay data. It should be understood that, as Figures 13-15 The exemplary computing cluster and communication link configurations shown are just a few of many possible configurations.

[0075] Refer again Figure 12 The enhanced hierarchical scaling method for multiple parallel processing units in a fully connected computing cluster further includes: in step 1220, in order to compute a collapsed, fully collapsed, or similar function on a given fully connected computing cluster, segmenting the input data and loading the corresponding portions onto the corresponding parallel processing units of the given fully connected computing cluster. For example, in Figure 13In the fully connected computing cluster consisting of four parallel processing units, the input data is divided into four parts and loaded onto the corresponding parallel processing units of the given fully connected computing cluster. Figure 14 The fully connected computing cluster, consisting of two parallel processing units, splits the input data into two parts and loads the corresponding parts into the corresponding parallel processing units of the given fully connected computing cluster.

[0076] In step 1230, the input data is computed using a collapsed, fully collapsed, or similar function on a given fully connected computing cluster. For example, as... Figure 13 As shown, in a fully connected computing cluster 1305 with four parallel processing units for calculating the collapse function, a first parallel processing unit 505 receives at least a portion of the corresponding data value from each of the parallel processing units 505 to 520 of the given fully connected computing cluster 1305 on corresponding communication links 1315 to 1325. Furthermore, a portion of the corresponding data value can be relayed from the parallel processing units 510 to 520 of the given computing cluster 1305 by corresponding parallel processing units 525 to 540 in other computing clusters 1310 on corresponding communication links 1330 to 1345. The first parallel processing unit 505 can add the corresponding received data values ​​and store the sum in a cache within the first parallel processing unit 505. Similarly, other parallel processing units of the given computing cluster 1305, namely parallel processing units 510 to 520, can also receive corresponding data values ​​directly on their respective communication links or via single-hop transmission on other corresponding communication links from the parallel processing units of the given computing cluster 1305. Each other parallel processing unit can add its respective received data values ​​and store the sum in its respective buffer. Likewise, as... Figure 13As shown, in the example of four parallel processing units computing the fully collapsed function, the first parallel processing unit 505 receives at least a portion of the corresponding data value from each of the parallel processing units 505 to 520 of the given computing cluster 1305 on corresponding communication links 1315 to 1325. Furthermore, a portion of the corresponding data value can be relayed from the parallel processing units 510 to 520 of the given computing cluster 1305 via corresponding parallel processing units 525 to 540 in other computing clusters 1310 on corresponding communication links 1330 to 1345. The first parallel processing unit 505 can add the corresponding received data values. Then, the first parallel processing unit 505 can broadcast at least a portion of the corresponding sum value to the parallel processing units 505 to 520 of the given computing cluster 1305 on the corresponding communication links. Furthermore, a portion of the corresponding sum can be relayed from the first parallel processing unit 505 of the given computing cluster 1305 to other parallel processing units, namely parallel processing units 510 to 520, via corresponding parallel processing units 540 in other computing clusters 1310 on the corresponding communication links. Similarly, the other three parallel processing units, namely parallel processing units 510 to 520, can also receive corresponding data values ​​directly from each of the parallel processing units 505 to 520 on the corresponding communication links or via single-hop transmission on other corresponding communication links. Each other parallel processing unit can add its respective received data values. Each other parallel processing unit can broadcast the corresponding sum to the corresponding parallel processing unit directly on the corresponding communication link or via single-hop transmission on other corresponding communication links. It should be understood that the computing cluster 1310 of the other four parallel processing units can be configured to compute another function for the same application, another application with the same client, different applications, etc.

[0077] Reference Figure 16This illustration shows an exemplary computing system including multiple parallel processing units according to embodiments of the present disclosure. The exemplary computing system 1600 may include multiple parallel processing units 1610 and 1620 coupled together via one or more high-bandwidth inter-chip networks 1630. The multiple parallel processing units 1610 and 1620 include multiple neural processing accelerators. The multiple parallel processing units 1610 and 1620 may also be coupled to multiple main processing units 1640 and 1650 via one or more communication buses 1660 and 1670. The one or more communication buses 1660 and 1670 include one or more peripheral component interface fast (PCIe) buses. The one or more main processing units 1640 and 1650 may be coupled to one or more host-side networks 1680 via one or more network interface cards (NICs) 1690 and 1695.

[0078] Reference Figure 17This illustration shows an exemplary parallel processing unit according to an embodiment of the present disclosure. The parallel processing unit 1700 includes multiple compute cores 1705 and 1710, multiple inter-chip links (ICLs) 1715 and 1720, one or more high-bandwidth memory interfaces (HBM I / F) 1725 and 1730, one or more communication processors 1735, one or more direct memory access (DMA) controllers 1740 and 1745, one or more command processors (CPs) 1750, one or more on-chip networks (NoCs) 1755, shared memory 1760, and one or more high-bandwidth memory (HBM) 1765 and 1770. The parallel processing unit 1700 also includes one or more Joint Test Action Group (JTAG) engines 1775, one or more inter-integrated circuit (I2C) interfaces and / or serial peripheral interfaces (SPIs) 1780, one or more peripheral component interface fast (PCIe) interfaces 1785, one or more codecs (CoDecs) 1790, etc. In one embodiment, multiple computing cores 1705 and 1710, multiple inter-chip links 1715 and 1720, one or more high-bandwidth memory interfaces 1725 and 1730, one or more communication processors 1735, one or more direct memory access controllers 1740 and 1745, one or more command processors 1750, one or more on-chip networks 1755, shared memory 1760, one or more high-bandwidth memories 1765 and 1770, one or more joint test action group engines 1775, one or more inter-integrated circuit interfaces and / or serial peripheral interfaces 1780, one or more peripheral component interface fast interfaces 1785, one or more codecs 1790, etc., can be fabricated in a single monolithic integrated circuit (IC).

[0079] Inter-chip links 1715 and 1720 can be used for chip-to-chip communication between multiple parallel processing units. In one embodiment, parallel processing unit 1700 may include seven inter-chip links 1715 and 1720. Communication processor 1735, direct memory access engine 1740, and direct memory access engine 1745 can be used to coordinate the transmission and reception of data via inter-chip links 1715 and 1720. On-chip network 1755 can be used to coordinate data movement between computing core 1705, computing core 1710, and shared memory 1760. Communication processor 1735, direct memory access engine 1740, direct memory access engine 1745, on-chip network 1755, high-bandwidth memory interface 1725, and high-bandwidth memory interface 1730 can be used to coordinate data movement between high-bandwidth memory 1765, high-bandwidth memory 1770, shared memory 1760, and inter-chip links 1715 and 1720. The command processor 1750 can be used as an interface between the parallel processing unit 1700 and one or more main processing units. (Refer to the above...) Figure 5-15 As described above, multiple parallel processing units 1700 can be used to efficiently compute collapse, full collapse, or other similar functions.

[0080] According to embodiments of this disclosure, tiered scaling allows multiple parallel processing units to be configured in one or more computing clusters coupled by a corresponding number of parallel communication rings. Tiered scaling of the multiple parallel processing units is effective when an application requires a portion of the computing resources of the multiple parallel processing units that is less than the computing resources that a subset of the computing clusters can provide. Similarly, tiered scaling can be effectively employed in cloud computing platforms, enabling clients to purchase the computing bandwidth of a computing cluster of parallel processing units, rather than the computing bandwidth of all parallel processing units.

[0081] The foregoing description of embodiments of this disclosure is presented for purposes of illustration and description. The foregoing description is not intended to exclude or limit this disclosure to the precise forms disclosed, and many modifications and variations will be apparent from the teachings above. The embodiments were chosen and described in order to best explain the principles of this disclosure and its practical application, thereby enabling others skilled in the art to best utilize this disclosure and various embodiments with suitable modifications for the intended particular purpose. The scope of this disclosure is defined by the appended claims and their equivalents.

Claims

1. A computing system, comprising: Multiple parallel processing units, wherein each parallel processing unit is coupled to each other parallel processing unit via a corresponding communication link, and, based on specified computing parameters, by enabling a given subset of communication links that interconnect parallel processing units in each corresponding computing cluster, and disabling other communication links between parallel processing units in different computing clusters, the corresponding communication links of the multiple parallel processing units are configured to one or more fully connected computing clusters. Each parallel processing unit includes multiple communication ports, and each of the multiple parallel processing units is directly connected to each other parallel processing unit through its own multiple configurable communication links. Each of the one or more fully connected computing clusters is configured to perform a corresponding collapse or full collapse function calculation on the corresponding input data.

2. The computing system according to claim 1, wherein, The communication link includes a bidirectional communication link.

3. The computing system according to claim 1, wherein, The specified computation parameters include the number of parallel processing units in a given computing cluster.

4. The computing system according to claim 1, wherein, The specified calculation parameters include the size of the calculation processing bandwidth.

5. The computing system according to claim 1, wherein, The multiple parallel processing units are configured in a fully connected computing cluster consisting of eight parallel processing units through corresponding communication links.

6. The computing system according to claim 1, wherein, The multiple parallel processing units are configured in two fully connected computing clusters, each consisting of four parallel processing units, via corresponding communication links.

7. The computing system according to claim 1, wherein, The multiple parallel processing units are configured in four fully connected computing clusters, each consisting of two parallel processing units, via corresponding communication links.

8. The computing system according to claim 1, wherein, The multiple parallel processing units are configured through corresponding communication links in a fully connected computing cluster consisting of four parallel processing units and two fully connected computing clusters each consisting of two parallel processing units.

9. A calculation method, comprising: Based on specified computational parameters, the communication links of multiple parallel processing units are configured into one or more fully connected computing clusters by enabling a given subset of communication links that interconnect parallel processing units in each corresponding computing cluster and disabling other communication links between parallel processing units in different computing clusters. Each parallel processing unit includes multiple communication ports, and each of the multiple parallel processing units is directly connected to each other parallel processing unit through its own multiple configurable communication links. Based on the number of parallel processing units in a given fully connected computing cluster, partition the input data; and The input data is computed using the given fully connected computing cluster, wherein the function computation is a collapsed or fully collapsed function computation.

10. The calculation method according to claim 9, wherein, The specified computation parameters include the number of parallel processing units in a given computing cluster.

11. The calculation method according to claim 9, wherein, The specified calculation parameters include the size of the calculation processing bandwidth.

12. The calculation method according to claim 9, further comprising: Based on the specified computational parameters, the communication links of the plurality of parallel processing units are configured to be distributed among the plurality of fully connected computing clusters. Specifically, the communication links between parallel processing units in a given fully connected computing cluster are configured to send data to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, the parallel processing units in other fully connected computing clusters are configured to relay data from the parallel processing units in the given fully connected computing cluster to other parallel processing units in the given fully connected computing cluster via the communication links between the parallel processing units in the given fully connected computing cluster and the parallel processing units in the other fully connected computing clusters. The input data is computed using a collapse function or a full collapse function through the given fully connected computing cluster. Specifically, on the communication links between parallel processing units in the given fully connected computing cluster, the data of the collapse function or full collapse function is sent to other corresponding parallel processing units in the given fully connected computing cluster. Furthermore, on the communication links between a parallel processing unit in the given fully connected computing cluster and parallel processing units in other fully connected computing clusters, one or more parallel processing units in the other fully connected computing clusters relay the data of the collapse function or full collapse function.

13. One or more computing device-readable media having instructions stored thereon, which, when executed by one or more processing units, perform a method comprising the following steps: Based on specified computational parameters, by enabling a given subset of communication links that interconnect parallel processing units in each corresponding computing cluster, and disabling other communication links between parallel processing units in different computing clusters, the communication links of multiple parallel processing units are configured to be connected to one or more fully connected computing clusters. Each parallel processing unit includes multiple communication ports, and each of the multiple parallel processing units is directly connected to each other parallel processing unit through its own multiple configurable communication links. The input data is segmented based on the number of parallel processing units in a given fully connected computing cluster. as well as The input data is computed using the given fully connected computing cluster, wherein the function computation is a collapsed or fully collapsed function computation.

Images