Parallel processing virtualization method and computing system

Abstract

Parallel processing virtualization methods and computing systems are disclosed. A virtualization method includes determining a virtual function routing table for a virtual parallel processing unit (PPU) according to a logical topology of virtual functions. A first mapping of the virtual PPU to a first set of physical PPU is generated. The virtualization method can also include generating a first set of physical function routing tables for the first set of physical PPU based on the virtual function routing table and the first mapping of the virtual PPU to physical PPU. By generating a second mapping of the virtual PPU to a second set of physical PPU, an application can be migrated from the first set of physical PPU to the second set of physical PPU. A second set of physical function routing tables can be generated for the second set of physical PPU based on the virtual function routing table and the second mapping of the virtual PPU to physical PPU.

Description

Background Technology

[0001] Currently, distributed parallel training methods for deep neural networks involve synchronously executing large-batch stochastic gradient descent (SGD) processes on many distributed computing nodes to explore data parallelism-based acceleration. Referring to Figure 1, Figure 1 illustrates an exemplary mini-batch SGD process (including pseudocode) running on a CPU host. This SGD process is constrained by the synchronization portion, which hinders the overall parallel acceleration. As shown in Figure 2, to reduce this constraint, it is necessary to increase the network bandwidth at the accelerator end and / or reduce the frequency of communication between the host and the accelerator.

[0002] Mini-batch SGD processing has many synchronization algorithms. Some common functions for implementing inter-node communication are the Reduce and All_Reduce functions. As shown in Figure 3, in the Reduce function, a set of values ​​from each of multiple nodes 310-340 is passed to a designated node 310 among the multiple nodes 310-340, and node 310 adds the corresponding values ​​together. The sum of the set of values ​​is stored by the designated node 310. For example, the first node 310 receives the values ​​5, 2, 7, and 4 from multiple nodes 310-340, adds the received values ​​5, 2, 7, and 4 together, and stores the resulting sum of 18. The first node 310 also adds the values ​​1, 3, 8, and 2 together and stores the resulting sum of 14. As shown in Figure 4, in the All_Reduce function, a set of values ​​from each of multiple nodes 410-440 is passed to a designated node 410 among the multiple nodes 410-440, and node 410 adds the corresponding values ​​together. The resulting sum is broadcast by node 410 to multiple nodes 410-440, and these nodes store the sum. For example, the first node 410 adds the values ​​5, 2, 7, and 4 received from nodes 410-440. The first node 410 also adds the values ​​1, 3, 8, and 2. The first node 410 broadcasts the sum of 18 and 14 to these nodes, and each node in the 410-440 stores the sum. As shown in the diagram, the Reduce and All_Reduce functions are applied simultaneously to a group of variables.

[0003] While the simple topology implementations of the Reduce and All_Reduce functions are tree-based, ring-based implementations can achieve higher bandwidth utilization and efficiency. Referring now to Figure 5, which illustrates a conventional ring-based All_Reduce implementation on a distributed computing system. In the All_Reduce function, each of the N nodes in the distributed computing system communicates with its two peer nodes 2*(N-1) times. During communication, a node sends and receives array values. In the first (N-1) iterations, the received values ​​are added to the values ​​in the respective node's buffer. In the second (N-1) iterations, the received values ​​replace the values ​​held in the respective node's buffer. For example, 510 of Figure 5 shows three nodes (N=3), each buffering a set of its own input values. In the first iteration 520, the first node passes the first set of input values ​​to the second node. The second node adds the set of input values ​​received from the first node to the corresponding input values ​​held by the second node. The first node also receives a third set of input values ​​from the third node. The first node adds the set of input values ​​received from the third node to the corresponding value held by the first node. The second and third nodes also pass and add the corresponding sets of values ​​in the first iteration 520. In the second iteration 530, the first node passes the third set of input values ​​to the second node, which adds it to the corresponding value held by the second node. The first node also receives a second set of values ​​from the third node and adds it to the corresponding value held by the first node. The second and third nodes again pass and add the corresponding sets of values ​​in the second iteration 530. In the third iteration 540, the first node passes the second set of sums to the second node, which stores the sum. The first node also receives a first set of sums from the third node and stores the sum. The second and third nodes also pass and store the sums of their respective sets. In the fourth iteration 550, the first node passes the first set of sums to the second node, which stores the sum. The first node also receives a third set of sums from the third node and stores the sum. The second and third nodes also pass and store the sums of their respective sets. After the fourth iteration, each node has this series of sums. If the buffer is large enough, the ring-based All_Reduce function shown in Figure 5 can optimally utilize the available network of the distributed computing system.

[0004] For systems that compute the Reduce and All_Reduce functions mentioned above, as well as many other functions, the parallel execution of multiple processing units can benefit from virtualization and migration techniques. However, virtualization and migration techniques need continuous improvement for use in computing systems employing parallel processing topologies. Summary of the Invention

[0005] Embodiments of this disclosure can be well understood by referring to the following description and accompanying drawings, which illustrate embodiments of this disclosure relating to the virtualization and migration of parallel processing units.

[0006] In one embodiment, a parallel processing virtualization method may include determining virtual parallel processing units used by virtual functions. Based on the logical topology of the virtual functions, a virtual function routing table can be determined for the virtual parallel processing units. The virtual parallel processing units may be mapped to a first group of physical parallel processing units. Based on the virtual function routing table and the mapping from the virtual parallel processing units to the first group of physical parallel processing units, a first set of physical function routing tables can be generated for the first group of physical parallel processing units. The method may further include receiving a migration event. In response to the migration event, the virtual parallel processing units may be mapped to a second group of physical parallel processing units. Based on the virtual function routing table and the mapping from the virtual parallel processing units to the second group of physical parallel processing units, a second set of physical function routing tables can be generated for the second group of physical parallel processing units. Data from the first group of physical parallel processing units may be migrated to the second group of physical parallel processing units based on the mapping from the virtual parallel processing units to the first group of physical parallel processing units and the mapping from the virtual parallel processing units to the second group of physical parallel processing units.

[0007] In another embodiment, the parallel processing virtualization method may include determining virtual parallel processing units used by virtual functions. Based on the logical topology of the virtual functions, a virtual function routing table can be determined for the virtual parallel processing units. A first partition of multiple physical parallel processing units is identified, and the virtual parallel processing units can be mapped to a first group of physical parallel processing units within the first partition. Based on the virtual function routing table and the mapping from the virtual parallel processing units to the first group of physical parallel processing units, a first set of physical function routing tables, including in-use communication link identifiers, can be determined for the first group of physical parallel processing units. Subsequently, using the mapping from the virtual parallel processing units to the first group of physical parallel processing units and the first set of physical function routing tables including in-use communication link identifiers, an application can be executed on the first group of physical parallel processing units. In response to a migration event, a second partition of multiple parallel processing units is identified. The virtual parallel processing units can be mapped to a second group of physical parallel processing units within the second partition. Based on the virtual function routing table and the mapping from the virtual parallel processing units to the second group of physical parallel processing units, a second set of physical function routing tables, including in-use communication link identifiers, can be generated for the second group of physical parallel processing units. Then, based on the mapping from the virtual parallel processing unit to the first group of physical parallel processing units and the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, data can be migrated from the first group of physical parallel processing units to the second group of physical parallel processing units. Subsequently, using the mapping from the virtual parallel processing unit to the second group of physical parallel processing units and the second group of physical function routing tables including communication link identifiers, the application can be executed on the second group of physical parallel processing units.

[0008] The above summary is provided to introduce the selected concepts in a simplified form, which will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0009] Embodiments of this disclosure are illustrated by way of example rather than limitation in the accompanying drawings, wherein like reference numerals refer to like elements, and wherein:

[0010] Figure 1 illustrates an exemplary small-batch SDG processing according to the prior art;

[0011] Figure 2 illustrates another exemplary small-batch SDG processing according to the prior art;

[0012] Figure 3 illustrates the computation of the Reduce function according to the prior art;

[0013] Figure 4 illustrates the computation of the All_Reduce function according to the prior art;

[0014] Figure 5 illustrates the ring-based All_Reduce computation according to the prior art;

[0015] Figure 6 A topology of an exemplary plurality of PPUs according to an embodiment of the present disclosure is shown;

[0016] Figure 7 An exemplary virtual PPU topology according to an embodiment of the present disclosure is shown;

[0017] Figure 8 Another exemplary virtual PPU topology according to an embodiment of this disclosure is shown;

[0018] Figure 9 Another exemplary virtual PPU topology according to an embodiment of this disclosure is shown;

[0019] Figure 10 PPU virtualization according to an embodiment of this disclosure is illustrated;

[0020] Figure 11 PPU virtualization according to an embodiment of this disclosure is illustrated;

[0021] Figure 12 PPU virtualization according to an embodiment of this disclosure is illustrated;

[0022] Figure 13 PPU virtualization according to an embodiment of this disclosure is illustrated;

[0023] Figure 14A and 14B A PPU virtualization method for migration according to an embodiment of this disclosure is illustrated;

[0024] Figure 15 A virtual PPU is shown according to an embodiment of this disclosure;

[0025] Figure 16 A first PPU according to an embodiment of this disclosure is shown;

[0026] Figure 17 A second PPU according to an embodiment of this disclosure is shown;

[0027] Figure 18A and 18B A method for virtualizing parallel processing units for migration according to an embodiment of the present disclosure is illustrated;

[0028] Figure 19A and 19BA method for virtualizing parallel processing units for migration according to an embodiment of the present disclosure is illustrated;

[0029] Figure 20 An exemplary computing system including multiple parallel processing units according to an embodiment of the present disclosure is shown;

[0030] Figure 21 An exemplary parallel processing unit according to an embodiment of the present disclosure is shown. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. While this disclosure will be described together with these embodiments, it is to be understood that they are not intended to limit this disclosure to these embodiments. Rather, this disclosure is intended to cover alternatives, modifications, and equivalents included within the scope of protection defined by the appended claims. Furthermore, in the following detailed description of this disclosure, numerous specific details will be set forth to provide a thorough understanding of the disclosure. However, it should be understood that this disclosure can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid unnecessarily obscuring the embodiments of this disclosure.

[0032] The following embodiments of this disclosure are presented in the form of routines, modules, logic blocks, and other symbolic representations of data operations in one or more electronic devices. Description and representation are methods used by those skilled in the art to most effectively convey the essence of their work to others skilled in the art. Routines, modules, logic blocks, etc., as described herein are generally considered to be a series of self-consistent processes or instructions that lead to expected results. These processes include physical operations on physical quantities. Typically, although not necessarily, these physical operations are manifested as electrical or magnetic signals and can be stored, transmitted, compared, and otherwise manipulated in an electronic device. For convenience and with reference to common usage, according to embodiments of this disclosure, these signals are referred to as data, bits, values, elements, symbols, characters, terms, numbers, strings, etc.

[0033] However, it should be remembered that these terms should be understood as designations for physical operations or quantities, and are merely convenient approximations, and should be further interpreted according to the terminology commonly used in the art. Unless otherwise expressly stated in the following discussion, it should be understood that, through the discussion of this disclosure, the use of terms such as "receive" refers to the actions and processes of electronic devices, such as the operation and conversion of data by electronic computing devices. Data is represented as physical (e.g., electronic) quantities in the logic circuits, registers, memories, etc., of electronic devices, and is converted into other data in the electronic devices that are also represented as physical quantities.

[0034] In this application, the use of disjunctive conjunctions is intended to include connection. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, the labeling of “the” object or “a” object is also intended to indicate one of a possible plurality of such objects. The use of terms such as “comprising,” “including,” “having,” etc., indicates the presence of the said element, but does not exclude the presence or addition of one or more other elements and / or combinations thereof. It should also be understood that although various elements are described herein with the terms first, second, etc., these elements should not be limited by these terms. These terms are used herein to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the embodiments. It should also be understood that when an element is referred to as “coupled” to another element, it may be directly or indirectly connected to the other element, or there may be an intermediate element. Conversely, when an element is referred to as “directly connected” to another element, there is no intermediate element. It should also be understood that the term “and / or” includes any and all combinations of one or more related elements. It should also be understood that the wording and terminology used in this article are for descriptive purposes and should not be considered as limiting.

[0035] Now refer to Figure 6 , Figure 6 A topology of an exemplary plurality of parallel processing units (PPUs) according to an embodiment of the present disclosure is shown. The plurality of PPUs includes one or more sets of eight PPUs, each PPU including seven communication ports. The eight PPUs in a set can be organized into a first subset of four PPUs and a second subset of four PPUs. Each PPU can be configurably coupled to two nearest neighbor PPUs in the same subset via two communication links. Each PPU can also be configurably coupled to a farthest neighbor PPU in the same subset via one communication link. Each PPU can also be configurably coupled to corresponding PPUs in other subsets via two communication links. In one embodiment, the PPUs can be coupled via configurable bidirectional communication links. The configurable coupled communication links can be configured as up to three communication rings that couple the eight PPUs together.

[0036] However, applications are typically independent of the PPU's data structure topology and the configuration of the communication links between them.

[0037] For example, the All_Reduce function can be coded as shown in Table 1.

[0038]

[0039] Table 1

[0040] In one example, as the application shows, the topology could be a ring with 8 PPUs, which are composed of, for example, Figure 7 The three communication rings shown are coupled.

[0041] In another example, the All_Reduce function in the application can be coded as shown in Table 2.

[0042]

[0043] Table 2

[0044] Applications can be written to be independent of processing units. Virtual function (VF) drivers can utilize topologies based on virtual parallel processing unit (vPPU) identifiers. Hardware (HW) and physical function (PF) drivers can utilize topologies based on physical parallel processing unit (pPPU) identifiers. Now refer to Figure 8 , Figure 8 An exemplary topology of eight virtual PPUs 805-840 coupled by three communication rings 845-855 is shown. For virtual function drivers using virtual PPU identifiers, the current virtual PPU 815 can be coupled to the previous virtual PPU 810 and the next virtual PPU 820 via the first communication ring 845 (solid line). The current virtual PPU 815 can also be coupled to the previous virtual PPU 810 and the next virtual PPU 820 via the second communication ring 850 (dashed line). The current virtual PPU 815 can also be coupled to the previous virtual PPU 825 and the next virtual PPU 830 via the third communication ring 855 (dotted-dash line). Physical functions and hardware can also use corresponding physical PPU identifiers, such as... Figure 9 As shown.

[0045] Now refer to Figure 10 , Figure 10 An exemplary virtual PPU topology is shown. This virtual PPU topology may include four virtual PPUs 1005-1020 coupled by two communication rings 1025 and 1030. (Refer to...) Figure 11According to embodiments of this disclosure, PPU virtualization can be used to migrate applications to the same system with the same configuration or the same system with the same topology. In one embodiment, a mapping from virtual PPU identifiers to physical PPU identifiers can be used to migrate to the same system with the same configuration or the same topology, as shown in Table 3.

[0046] vPPU pPPU vPPU 1 pPPU 1 vPPU 2 pPPU 2 vPPU 3 pPPU 3 vPPU 4 pPPU 4

[0047] Table 3

[0048] Reference Figure 12 According to embodiments of this disclosure, PPU virtualization can also be used to migrate applications to the same system with different topologies. Alternatively, a mapping from virtual PPU identifiers to physical PPU identifiers can be used to migrate applications to the same system with different topologies, as shown in Table 4.

[0049] vPPU pPPU vPPU 1 pPPU 1 vPPU 2 pPPU 2 vPPU 3 pPPU 7 vPPU 4 pPPU 8

[0050] Table 4

[0051] Now refer to Figure 13 According to embodiments of this disclosure, PPU virtualization can also be used in some cases to migrate applications to a compatible system. As shown in Tables 5A and 5B, the mapping from virtual PPU identifiers to physical PPU identifiers for each communication ring can also be used to migrate applications to a compatible system.

[0052]

[0053] Now refer to Figure 14A and 14B , Figure 14A and 14B A method for PPU virtualization and migration according to an embodiment of this disclosure is illustrated. Reference will be made below. Figure 15-17 The method for virtualizing and migrating the PPU will be explained further. Figure 15 This shows the configuration of a set of virtual PPUs. Figure 16 This shows the first configuration of a set of physical PPUs. Figure 17 A second configuration of a set of physical PPUs is shown. The method includes a deployment phase and a migration phase. In the deployment phase, in step 1405, the virtual PPUs used by the virtual function driver can be determined. For example, it can be determined that the VF driver uses four virtual PPUs. In step 1410, a virtual function routing table can be determined for the virtual PPUs based on the logical topology of the virtual functions. For example, routing tables, as shown in Tables 6A, 6B, 6C, and 6D respectively, can be generated to draw communication links between the virtual PPUs used by vPPU 1 to vPPU 4.

[0054] vPPU1 routing table a b c d e f g vPPU 1 vPPU 2 1 1 vPPU 3 vPPU 4 1 1

[0055] Table 6A

[0056] Table 6A indicates that vPPU 1 is coupled to vPPU 2 through two communication links d and e, and vPPP 1 is coupled to vPPU 4 through two communication links f and g.

[0057] vPPU 2 routing table a b c d e f g vPPU 1 1 1 vPPU 2 vPPU 3 1 1 vPPU 4

[0058] Table 6B

[0059] Table 6B indicates that vPPU 2 is coupled to vPPU 1 through two communication links d and e, and is coupled to vPPU 3 through two communication links f and g.

[0060] vPPU 3 routing table a b c d e f g vPPU 1 vPPU 2 1 1 vPPU 3 vPPU 4 1 1

[0061] Table 6C

[0062] Table 6C indicates that vPPU 3 is coupled to vPPU 2 through two communication links f and g, and to vPPU 4 through two communication links d and e.

[0063] vPPU 4 routing table a b c d e f g vPPU 1 1 1 vPPU 2 vPPU 3 1 1 vPPU 4

[0064] Table 6D

[0065] Table 6D indicates that vPPU 4 is coupled to vPPU 3 through two communication links d and e, and is coupled to vPPU 1 through two communication links f and g.

[0066] In step 1415, virtual PPUs can be mapped to a first group of multiple physical PPUs. For example, the multiple physical PPUs may include pPPU 0 to pPPU 7. vPPU 1 to vPPU 4 can be mapped to pPPU 0 to pPPU 3, as shown in Table 7.

[0067] vPPU 1 pPPU 0 vPPU 2 pPPU 1 vPPU 3 pPPU 3 vPPU 4 pPPU 2

[0068] Table 7

[0069] In step 1420, a first set of physical function routing tables can be generated for the first set of physical PPUs based on the virtual function routing table and the first vPPU-pPPU mapping. For example, routing tables can be generated as shown in Tables 8A, 8B, 8C, and 8D, respectively, to draw the communication links between the physical PPUs used by pPPU 0 to pPPU 3.

[0070] pPPU 0 routing table a b c d e f g pPPU 0 pPPU 1 1 1 pPPU 2 1 1 pPPU 3

[0071] Table 8A

[0072] Table 8A indicates that pPPU 0 is coupled to pPPU 1 through communication links d and e, and pPPP 0 is coupled to pPPU 2 through communication links f and g.

[0073] pPPU 1 routing table a b c d e f g pPPU 0 1 1 pPPU 1 pPPU 2 pPPU 3 1 1

[0074] Table 8B

[0075] Table 8B indicates that pPPU 1 is coupled to pPPU3 through communication links f and g, and to pPPU0 through communication links d and e.

[0076] pPPU 2 routing table a b c d e f g pPPU 0 1 1 pPPU 1 pPPU 2 pPPU 3 1 1

[0077] Table 8C

[0078] Table 8C indicates that pPPU 2 is coupled to pPPU 0 through communication links f and g, and to pPPU 3 through communication links d and e.

[0079] pPPU 3 routing table a b c d e f g pPPU 0 pPPU 1 1 1 pPPU 2 1 1 pPPU 3

[0080] Table 8D

[0081] Table 8D indicates that pPPU 3 is coupled to pPPU 2 via communication links d and e, and to pPPU 1 via communication links f and g. Subsequently, this first vPPU-pPPU mapping can be used to execute applications.

[0082] In step 1425, a migration event may be received. In response to the migration event, in step 1430, the virtual PPU may be mapped to a second group of multiple physical PPUs. The second group of physical PPUs may be in the same set of physical PPUs or in new sets of physical PPUs. For example, vPPU 1 to vPPU 4 may be mapped to pPPU 0, pPPU 1, pPPU 7, and pPPU 6 of new sets of parallel processing units, as shown in Table 9.

[0083] vPPU 1 pPPU 0 vPPU 2 pPPU 1 vPPU 3 pPPU 7 vPPU 4 pPPU 6

[0084] Table 9

[0085] In step 1435, a second set of physical function routing tables can be generated for the second set of physical PPUs based on the virtual function routing table and the second vPPU-pPPU mapping. For example, routing tables that plot the communication links between physical PPUs, as shown in Tables 10A, 10B, 10C, and 10D, can be generated respectively.

[0086] pPPU 0 routing table a b c d e f g pPPU 0 pPPU 1 1 1 pPPU 7 pPPU 6 1 1

[0087] Table 10A

[0088] Table 10A indicates that pPPU 0 is coupled to pPPU 1 through communication links d and e, and pPPP 0 is coupled to pPPU 6 through communication links a and b.

[0089] pPPU 1 routing table a b c d e f g pPPU 0 1 1 pPPU 1 pPPU 7 1 1 pPPU 6

[0090] Table 10B

[0091] Table 10B indicates that pPPU 1 is coupled to pPPU 7 through communication links a and b, and is coupled to pPPU 0 through communication links d and e.

[0092] pPPU 7 routing table a b c d e f g pPPU 0 pPPU 1 1 1 pPPU 7 pPPU 6 1 1

[0093] Table 10C

[0094] Table 10C indicates that pPPU 7 is coupled to pPPU 1 via communication links a and b, and to pPPU 6 via communication links d and e.

[0095] pPPU 6 routing table a b c d e f g pPPU 0 1 1 pPPU 1 pPPU 7 1 1 pPPU 6

[0096] Table 10D

[0097] Table 10D indicates that pPPU 6 is coupled to pPPU 0 through communication links a and b, and to pPPU 7 through communication links d and e.

[0098] In step 1440, data from the first set of physical PPUs can be migrated to the second set of physical PPUs based on the first vPPU-pPPU mapping and the second vPPU-pPPU mapping. Thereafter, the virtual function driver can use the second vPPU-pPPU mapping to execute the application.

[0099] Now refer to Figure 18A and 18B According to embodiments of this disclosure, Figure 18A and 18B Another PPU virtualization method for migration is shown below. Figure 15-17 The PPU virtualization method used for migration is further explained. During the deployment phase, in step 1805, the virtual PPUs used by the virtual function driver can be determined. For example, based on the virtual functions, it can be determined that VF uses four virtual PPUs. In step 1810, a virtual function routing table can be determined for the virtual PPUs based on the logical topology of the virtual functions. For example, a routing table drawing the communication links between the virtual PPUs used by vPPU 1 to vPPU 4 can be generated as shown in Tables 6A, 6B, 6C, and 6D, respectively.

[0100] In step 1815, the virtual PPU can be mapped to a first group of multiple physical PPUs. For example, the multiple physical PPUs may include pPPU 0 to pPPU 7. vPPU 1 to vPPU 4 can be mapped to pPPP 0 to pPPP 3, as shown in Table 7.

[0101] In step 1820, a first set of physical function routing tables can be generated for the first group of physical PPUs based on the virtual function routing table and the first vPPU-pPPU mapping. For example, physical function routing tables, as shown in Tables 8A, 8B, 8C, and 8D respectively, can be generated to plot the communication links between the physical PPUs used by pPPU 0 to pPPU 3. In one embodiment, the physical function routing tables can be stored in hardware registers. In step 1825, a first set of in-use routing tables can be generated for the physical PPUs. For example, in-use routing tables describing the in-use communication links between the physical PPUs used by pPPU 0 to pPPU 3 can be generated as shown in Tables 11A, 11B, 11C, and 11D respectively.

[0102] pPPU 0 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 1 pPPU 2 1 pPPU 3

[0103] Table 11A

[0104] Table 11A indicates that pPPU 0 is coupled to pPPU 1 via communication link e, and to pPPU 2 via communication link f.

[0105] pPPU 1 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 1 pPPU 1 pPPU 2 pPPU 3 1

[0106] Table 11B

[0107] Table 11B indicates that pPPU 1 is coupled to pPPU 3 via communication link f, and to pPPU 0 via communication link e.

[0108] pPPU 2 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 1 pPPU 1 pPPU 2 pPPU 3 1

[0109] Table 11C

[0110] Table 11C indicates that pPPU 2 is coupled to pPPU 0 via communication link f, and to pPPU 3 via communication link e.

[0111] pPPU 3 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 1 pPPU 2 1 pPPU 3

[0112] Table 11D

[0113] Table 11D indicates that pPPU 3 is coupled to pPPU 2 via communication link e and to pPPU 1 via communication link f. In one embodiment, the first set of in-use routing tables may be stored in physical hubs (PHUBs) and switches in the hardware associated with the multiple PPUs. In step 1825, the application can be executed using the first vPPU-pPPU mapping. In one embodiment, the routing table within the hardware is based on physical PPU identifiers, while the routing table based on virtual PPU identifiers is exposed to the virtual function driver. For runtime execution, the first vPPU-pPPU mapping can be looked up. In one embodiment, storage space (e.g., page tables) is based on vPPU identifiers. Therefore, the application is exposed, but data structures are mapped to vPPU identifiers.

[0114] In step 1830, a migration event may be received. In response to the migration event, in step 1835, the virtual PPU may be mapped to a second group of multiple physical PPUs. The second group of physical PPUs may be in the same set of physical PPUs or in new sets of physical PPUs. In one example, vPPU 1 through vPPU 4 may be mapped to pPPU 0, pPPU 1, pPPU 7, and pPPU 6 of the new set of parallel processing units, as shown in Table 9.

[0115] In step 1840, a second set of physical function routing tables can be generated for the physical PPUs based on the virtual function routing table and the first vPPU-pPPU mapping. For example, physical function routing tables, as shown in Tables 8A, 8B, 8C, and 8D respectively, can be generated to plot the communication links between the physical PPUs used by pPPU 0, pPPU 1, pPPU 6, and pPPU 7. In one embodiment, the physical function routing tables can be stored in hardware registers. In step 1845, a second set of in-use routing tables can be generated for the physical PPUs. For example, in-use routing tables, as shown in Tables 12A, 12B, 12C, and 12D respectively, can be generated to plot the in-use communication links between the physical PPUs used by pPPU 0, pPPU 1, pPPU 6, and pPPU 7.

[0116] pPPU 0 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 1 pPPU 6 1 pPPU 7

[0117] Table 12A

[0118] Table 12A indicates that pPPU 0 is coupled to pPPU 1 via communication link e, and to pPPU 6 via communication link a.

[0119] pPPU 1 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 1 pPPU 1 pPPU 6 pPPU 7 1

[0120] Table 12B

[0121] Table 12B indicates that pPPU 1 is coupled to pPPU 7 via communication link a, and to pPPU 0 via communication link e.

[0122]

[0123]

[0124] Table 12C

[0125] Table 12C indicates that pPPU 7 is coupled to pPPU 1 via communication link a, and to pPPU 6 via communication link e.

[0126] pPPU 6 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 1 pPPU 1 pPPU 6 pPPU 7 1

[0127] Table 12D

[0128] Table 12D indicates that pPPU 6 is coupled to pPPU 7 via communication link e and to pPPU 0 via communication link a. In one embodiment, the second set of in-use routing tables can be stored in physical hubs (PHUBs) and switches in the hardware associated with the multiple PPUs. In step 1850, data from the first set of physical PPUs can be migrated to the second set of physical PPUs based on the first vPPU-pPPU mapping and the second vPPU-pPPU mapping. Subsequently, in step 1855, the virtual function driver can execute the application using the second vPPU-pPPU mapping. In one embodiment, the routing table within the hardware is based on physical PPU identifiers, while the routing table based on virtual PPU identifiers is exposed to the virtual function driver. For runtime execution, the first vPPU-pPPU mapping can be looked up. In one embodiment, storage space (e.g., page tables) is based on virtual vPPU identifiers. Therefore, the application is exposed, but data structures are mapped to virtual PPU identifiers.

[0129] Now refer to Figure 19A and 19B , Figure 19A and 19B Another method for virtualizing parallel processing units for migration according to an embodiment of this disclosure is illustrated. Reference will be made again below. Figure 15-17The PPU virtualization method used for migration is further explained. During the deployment phase, in step 1905, the virtual PPUs used by the virtual function driver can be determined. For example, it can be determined that the VF driver uses four virtual PPUs. In one embodiment, the logical topology of the virtual PPUs used by the VF can be stored in system memory. In step 1910, a virtual function routing table can be determined based on the logical topology of the virtual functions. For example, routing tables, as shown in Tables 6A, 6B, 6C, and 6D respectively, can be generated to plot the communication links between the virtual PPUs used by vPPU 1 to vPPU 4. In one embodiment, the virtual function routing table can be stored in system memory.

[0130] In step 1915, a first partition of the multiple physical PPUs can be identified. For example, available partitions of the complete topology of the physical PPUs can be identified. In one embodiment, the physical function can query the hardware to construct a system topology based on the multiple physical PPUs, and this system topology can be stored in system memory. Subsystem partition information can also be stored for each VF. For a bare-metal PF implementation subsystem, partition information including in-use communication link information can be stored. In step 1920, virtual PPUs can be mapped to a first group of physical PPUs in the first partition of the multiple physical PPUs. For example, the multiple physical PPUs can include pPPU 0 to pPPU 7. vPPU 1 to vPPU 4 can be mapped to pPPU 0 to pPPU 3, as shown in Table 7. The vPPU-pPPU mapping can be exposed to the virtual function.

[0131] In step 1925, based on the virtual function routing table and the first vPPU-pPPU mapping, a first set of physical function routing tables can be generated for the physical PPUs. In this routing table, physical PPU partitions can be restricted to a single VF. For example, a physical function routing table can be generated that plots the communication links between physical PPUs used by pPPU 0 to pPPU 3. In step 1930, the in-use communication links used by the first set of physical function routing tables can be identified. For example, a routing table can be generated as shown in Tables 13A, 13B, 13C, and 13D, plotting the communication links between physical PPUs and identifying the communication links used by pPPU 0 to pPPU 3.

[0132] pPPU 0 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 10 11 pPPU 2 11 10 pPPU 3

[0133] Table 13A

[0134] As shown in Table 13A, the first bit indicates that pPPU 0 is coupled to pPPU 1 via communication links d and e, and to pPPU 2 via communication links f and g. The second bit indicates that communication link e is used to couple pPPU 0 to pPPU 1, and communication link f is used to couple pPPU 0 to pPPU 2.

[0135] pPPU 1 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 10 11 pPPU 1 pPPU 2 pPPU 3 11 10

[0136] Table 13B

[0137] Table 13B indicates that pPPU 1 is coupled to pPPU 3 via communication links f and g, and to pPPU 0 via communication links d and e. Table 13B also indicates that pPPU 1 is coupled to pPPU 3 via communication link f, and to pPPU 0 via communication link e.

[0138] pPPU 2 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 11 10 pPPU 1 pPPU 2 pPPU 3 10 11

[0139] Table 13C

[0140] Table 13C indicates that pPPU 2 is coupled to pPPU 0 via communication links f and g, and to pPPU 3 via communication links d and e. Table 13C also indicates that pPPU 2 is coupled to pPPU 0 via communication link f, and to pPPU 3 via communication link e.

[0141] pPPU 3 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 11 10 pPPU 2 10 11 pPPU 3

[0142] Table 13D

[0143] Table 13D indicates that pPPU 3 is coupled to pPPU 2 via communication links d and e, and to pPPU 1 via communication links f and g. Table 13D also indicates that pPPU 3 is coupled to pPPU 2 via communication link e, and to pPPU 1 via communication link f. In one embodiment, the physical function routing table including the communication link information may be stored in a physical register or in a physical hub (PHUB) / switch in the hardware associated with the multiple PPUs. In step 1935, during operation, the VF / PF driver may execute on the PPU partition using the first vPPU-pPPU mapping and the first set of physical function routing tables including the communication link identifiers.

[0144] In step 1940, a migration event may be received. In 1945, in response to the migration event, a second partition of multiple physical PPUs may be identified. For example, a new set of available physical PPUs may be identified as the target partition. In one embodiment, a physical function may query the hardware to construct a system topology for multiple physical PPUs, which is stored in system memory. In step 1950, virtual PPUs may be mapped to a second set of physical PPUs. The second set of physical PPUs may be in the same set of physical PPUs or in a new set of physical PPUs. In one example, vPPU 1 through vPPU 4 may be mapped to pPPU 0, pPPU 1, pPPU 7, and pPPU 6 of a new set of parallel processing units, as shown in Table 9.

[0145] In step 1955, a second set of physical function routing tables can be generated based on the first set of virtual function routing tables and the second vPPU-pPPU mapping. For example, a physical function routing table that plots the communication links between physical PPUs can be generated. In one embodiment, the physical function routing table can be stored in a hardware register. In step 1960, in-use communication links used by the second set of physical function routing tables can be identified. For example, in-use routing tables that plot the in-use communication links between physical PPUs used by pPPU 0, pPPU 1, pPPU 7, and pPPU 6, as shown in Tables 14A, 14B, 14C, and 14D respectively, can be generated.

[0146] pPPU 0 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 10 11 pPPU 7 pPPU 6 11 10

[0147] Table 14A

[0148] As shown in Table 14A, the first bit indicates that pPPU 0 is coupled to pPPU 1 via communication links d and e, and to pPPU 6 via communication links a and b. The second bit indicates that communication link e is used to couple pPPU 0 to pPPU 1, and communication link a is used to couple pPPU 0 to pPPU 6.

[0149] pPPU 1 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 10 11 pPPU 1 pPPU 7 11 10 pPPU 6

[0150] Table 14B

[0151] Table 14B indicates that pPPU 1 is coupled to pPPU 0 via communication links d and e, and to pPPU 7 via communication links a and b. Table 14B also indicates that pPPU 1 is coupled to pPPU 0 via communication link e, and to pPPU 7 via communication link a.

[0152] pPPU 7 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 pPPU 1 11 10 pPPU 7 pPPU 6 10 11

[0153] Table 14C

[0154] Table 14C indicates that pPPU 7 is coupled to pPPU 1 via communication links a and b, and to pPPU 6 via communication links d and e. Table 14C also indicates that pPPU 7 is coupled to pPPU 1 via communication link a, and to pPPU 6 via communication link e.

[0155] pPPU 6 routing table Pa Pb Pc Pd Pe Pf Pg pPPU 0 11 10 pPPU 1 pPPU 7 10 11 pPPU 6

[0156] Table 14D

[0157] Table 14D indicates that pPPU 6 is coupled to pPPU 7 via communication links d and e, and to pPPU 0 via communication links a and b. Table 14D also indicates that pPPU 6 is coupled to pPPU 7 via communication link e, and to pPPU 0 via communication link a. In one embodiment, the physical function routing table including the communication link information may be stored in a hardware register or physical hub (PHUB) / switch of the hardware associated with the multiple PPUs.

[0158] In step 1965, the software of the source VF is copied to the target VF. In one embodiment, the program of the source VF on the current virtual PPU topology can be copied to the target VF on the new virtual PPU topology. In step 1970, state information and data from the first set of physical PPUs can be migrated to the second set of physical PPUs according to the first vPPU-pPPU mapping and the second vPPU-pPPU mapping. In step 1975, during runtime, the VF / PF driver can utilize the second vPPU-pPPU mapping and the second set of physical function routing tables including communication link identifiers to execute on the PPU partition.

[0159] refer to Figure 20 , Figure 20An exemplary computing system comprising multiple parallel processing units according to an embodiment of the present disclosure is illustrated. The exemplary computing system 2000 may include multiple PPUs 2010, 2020 coupled together via one or more high-bandwidth inter-chip networks 2030. The multiple PPUs 2010, 2020 may be, but are not limited to, multiple neural processing accelerators. The PPUs 2010-2020 may also be coupled to multiple host processing units 2040, 2050 via one or more communication buses 2060, 2070. The one or more communication buses 2060, 2070 may be, but are not limited to, one or more PCIe (peripheral component interface express) buses. The one or more host processing units 2040, 2050 may be coupled to one or more host-side networks 2080 via one or more network interface cards (NICs) 2090, 2095.

[0160] Reference Figure 21 , Figure 21 An exemplary parallel processing unit according to an embodiment of the present disclosure is illustrated. The PPU 2100 may include multiple compute cores 2105, 2110, multiple inter-chip links (ICLs) 2115, 2120, one or more high-bandwidth memory interfaces (HBM I / Fs) 2125, 2130, one or more communication processors 2135, one or more direct memory access (DMA) controllers 2140, 2145, one or more command processors (CPs) 2150, one or more networks-on-chip (NoC) 2155, shared memory 2160, and one or more high-bandwidth memory (HBM) 2165, 2170. The PPU 2100 may also include one or more JTAG (joint test action group) engines 2175, one or more inter-integrated circuits (I... 2C) Interface and / or Serial Peripheral Interface (SPI) 2180, one or more PCIe (Peripheral Component Interface Express) interfaces 2185, one or more codecs (CoDec) 2190, etc. In one embodiment, multiple computing cores 2105, 2110, multiple inter-chip links (ICL) 2115, 2120, one or more high-bandwidth memory interfaces (HBMI / F) 2125, 2130, one or more communication processors 2135, one or more direct memory access (DMA) controllers 2140, 2145, one or more command processors (CP) 2150, one or more on-chip networks (NoC) 2155, shared memory 2160, one or more high-bandwidth memory (HBM) 2165, 2170, one or more JTAG engines 2175, one or more integrated circuits (I 2 C) Interfaces and / or Serial Peripheral Interface (SPI) 2180, one or more PCIe interfaces 2185, one or more codecs (CoDec) 2190, and similar devices can be fabricated in a single integrated circuit (IC).

[0161] ICLs 2115 and 2120 can be configured for inter-chip communication between multiple PPUs. In one embodiment, PPU 2100 may include seven ICLs 2115 and 2120. Communication processor 2135 and direct memory access controllers 2140 and 2145 can be configured to coordinate the transmission and reception of data via ICLs 2115 and 2120. On-chip network (NoC) 2155 can be configured to coordinate data movement between compute cores 2105 and 2110 and shared memory 2160. Communication processor 2135, direct memory access controllers 2140 and 2145, on-chip network 2155, and high-bandwidth memory interface (HBM I / F) 2125 and 2130 can be configured to coordinate data movement between high-bandwidth memories 2165 and 2170, shared memory 2160, and ICLs 2115 and 2120. Command processor 2150 can be configured to act as an interface between PPU 2100 and one or more host processing units. (See above reference.) Figure 6-13 As described in 14A-14B, 15-17, 18A-18B and 19A-19B, the multiple PPU 2100s can be well used for the virtualization and migration of computation functions.

[0162] Specific embodiments of this disclosure have been described above for illustrative and descriptive purposes. They are not intended to be exhaustive or to limit the disclosure to the exact forms disclosed, and it will be apparent that many modifications and variations are possible in accordance with the foregoing teachings. The embodiments were chosen and described in order to best explain the disclosure and its practical application, thereby enabling others skilled in the art to best utilize the disclosure and its various embodiments with various modifications suitable for the particular purpose contemplated. The scope of this disclosure is to be defined by the claims appended herein and their equivalents.

Claims

1. A parallel processing virtualization method, comprising: Determine the virtual parallel processing unit used by the virtual function; The virtual function routing table is determined for the virtual parallel processing unit based on the logical topology of the virtual function; Map the virtual parallel processing unit to the first group of physical parallel processing units; as well as Based on the virtual function routing table and the mapping from the virtual parallel processing unit to the first group of physical parallel processing units, a first group of physical function routing tables is generated for the first group of physical parallel processing units.

2. The parallel processing virtualization method according to claim 1 further includes: Based on the logical topology of the virtual function, a first set of in-use routing tables is generated for the first set of physical parallel processing units.

3. The parallel processing virtualization method according to claim 2 further includes: The application is executed on the first group of physical parallel processing units using the mapping from the virtual parallel processing unit to the first group of physical parallel processing units, the first group of physical function routing tables, and the first group of in-use routing tables.

4. The parallel processing virtualization method according to claim 1 further includes: Receive migration events; Map the virtual parallel processing unit to the second group of physical parallel processing units; Based on the virtual function routing table and the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, a second set of physical function routing tables is generated for the second group of physical parallel processing units; and, Based on the mapping from the virtual parallel processing unit to the first group of physical parallel processing units and the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, data is migrated from the first group of physical parallel processing units to the second group of physical parallel processing units.

5. The parallel processing virtualization method according to claim 4 further includes: Based on the logical topology of the virtual function, a second set of in-use routing tables is generated for the second set of physical parallel processing units.

6. The parallel processing virtualization method according to claim 5 further includes: The application is executed on the second group of physical parallel processing units using the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, the second group of physical function routing tables, and the second group of in-use routing tables.

7. A computing system that performs the parallel processing virtualization method according to claim 1, wherein, Multiple physical parallel processing units are organized into multiple subsets of parallel processing units, wherein each physical parallel processing unit in each subset is coupled to each other physical parallel processing unit in the same subset via two bidirectional communication links, and each physical parallel processing unit is coupled to the corresponding physical parallel processing unit in the corresponding other subset of the same subset via one bidirectional communication link.

8. The computing system according to claim 7, wherein: The plurality of physical parallel processing units includes eight physical parallel processing units. The plurality of physical parallel processing units are organized into two subsets, each having four physical parallel processing units. Two bidirectional communication links couple each physical parallel processing unit to three other physical parallel processing units in the same subset of parallel processing units; and, A bidirectional communication link couples each physical parallel processing unit to the corresponding physical parallel processing unit of another subset of parallel processing units.

9. A parallel processing virtualization method, comprising: Determine the virtual parallel processing unit used by the virtual function; Based on the logical topology of the virtual function, a virtual function routing table is determined for the virtual parallel processing unit; Identify the first partition of multiple physically parallel processing units; Map the virtual parallel processing unit to the first group of physical parallel processing units in the first partition; as well as Based on the virtual function routing table and the mapping from the virtual parallel processing unit to the first group of physical parallel processing units, a first group of physical function routing tables, including in-use communication link identifiers, is generated for the first group of physical parallel processing units.

10. The parallel processing virtualization method according to claim 9, further comprising: The application is executed on the first group of physical parallel processing units using the mapping from the virtual parallel processing unit to the first group of physical parallel processing units and the first group of physical function routing tables including communication link identifiers.

11. The parallel processing virtualization method according to claim 9, further comprising: Receive migration events; Identify the second partition of multiple physically parallel processing units; Map the virtual parallel processing unit to the second group of physical parallel processing units in the second partition; Based on the virtual function routing table and the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, a second group of physical function routing tables, including in-use communication link identifiers, is generated for the second group of physical parallel processing units. Based on the mapping from the virtual parallel processing unit to the first group of physical parallel processing units and the mapping from the virtual parallel processing unit to the second group of physical parallel processing units, data is migrated from the first group of physical parallel processing units to the second group of physical parallel processing units.

12. The parallel processing virtualization method according to claim 11, further comprising: The application is executed on the second set of physical parallel processing units using the mapping from the virtual parallel processing unit to the second set of physical parallel processing units and the second set of physical function routing tables including communication link identifiers.

13. The parallel processing virtualization method according to claim 11, further comprising: Copy the source virtual function driver to use as the target virtual function driver.

14. The parallel processing virtualization method according to claim 11, wherein, The mappings from the virtual parallel processing unit to the first group of physical parallel processing units and the mappings from the virtual parallel processing unit to the second group of physical parallel processing units are exposed to the virtual function driver.

15. The parallel processing virtualization method according to claim 11, wherein, The virtual function routing table is stored in the system memory.

16. The parallel processing virtualization method according to claim 11, wherein, The storage space partition of the application running on the first set of physical parallel processing units is associated with the identifier of the virtual parallel processing unit.

17. The parallel processing virtualization method according to claim 11, wherein, The first group of physical parallel processing units and the second group of physical parallel processing units are located in the same plurality of physical parallel processing units.

18. The parallel processing virtualization method according to claim 11, wherein, The first group of physical parallel processing units and the second group of physical parallel processing units are different multiple physical parallel processing units.

19. The parallel processing virtualization method according to claim 11, wherein, The first group of physical parallel processing units and the second group of physical parallel processing units are in the same system and have the same configuration, the same topology, or different topologies.

20. The parallel processing virtualization method according to claim 11, wherein, The first group of physical parallel processing units and the second group of physical parallel processing units are in their respective compatible systems.

Images