Data center liquid cooling with intelligent refrigeration assistance

Abstract

A cooling system for a data center is disclosed. The data center cooling system includes a refrigerant cooling loop for absorbing heat from an auxiliary cooling loop located within the data center, or providing supplemental cooling for one or more components of the data center coupled to the auxiliary cooling loop.

Description

[0001] Cross Reference to Related Applications

[0002] This is a PCT application of U.S. Patent Application No. 16 / 885,105, filed May 27, 2020. The disclosure of this application is incorporated herein by reference in its entirety for all purposes. TECHNICAL FIELD

[0003] At least one embodiment relates to cooling systems for data centers. In at least one embodiment, a refrigerant cooling loop is provided to absorb heat from an auxiliary cooling loop located within a data center, or to provide supplemental cooling to one or more components of a data center coupled to the auxiliary cooling loop. BACKGROUND

[0004] Data center cooling systems use fans to circulate a gas through server components. Certain supercomputers or other high-capacity computers can use water or other cooling systems than gas cooling systems to draw heat from server components or racks of a data center to an area outside of the data center. The cooling system can include a chiller within an area of the data center. The area outside of the data center can be a cooling tower or other external heat exchanger that receives heated coolant from the data center and dissipates heat into the environment (or external cooling medium) through forced air or other means before the cooled coolant is recirculated back to the data center. In one example, the chiller and cooling tower, along with a pump set, form a cooling plant that is responsive to a temperature measured by an external device applied to the data center. Air cooling systems are unable to absorb enough heat to support efficient or effective cooling of a data center, while liquid cooling systems can cause significant damage to server components or racks due to shorting, flooding, or other issues. BRIEF DESCRIPTION OF DRAWINGS

[0005] Various embodiments according to the present disclosure will be described with reference to the drawings, in which:

[0006] Figure 1 is a block diagram of an example data center having improved cooling systems described in at least one embodiment;

[0007] Figure 2A is a block diagram illustrating server-level features incorporating refrigerant auxiliary cooling in a data center, according to at least one embodiment;

[0008] Figure 2B is another block diagram illustrating server-level features incorporating refrigerant auxiliary cooling in a data center, according to at least one embodiment;

[0009] Figure 3AFIG. 2 is another block diagram illustrating server-level features in a data center incorporating refrigerant-assisted cooling, according to at least one embodiment;

[0010] Figure 3B FIG. 3 is yet another block diagram illustrating server-level features in a data center incorporating refrigerant-assisted cooling, according to at least one embodiment;

[0011] Figure 4A FIG. 4 is an illustration of rack-level features of a data center incorporating refrigerant-assisted cooling, according to at least one embodiment;

[0012] Figure 4B FIG. 5 is an illustration of a data center incorporating refrigerant-assisted cooling with room-level or row-level features and with external features, according to at least one embodiment;

[0013] Figure 5 FIG. 6 is a process flow of steps of a method that can be used to use or manufacture a cooling system according to at least one embodiment; Figures 2A-4B Figures 6A-17D

[0014] Figure 6A FIG. 7 illustrates an example data center in which at least one embodiment from Figures 2A-5 may be used;

[0015] Figure 6B Figure 6C FIG. 8 illustrates reasoning and / or training logic to implement and / or support a refrigerant-assisted cooling system according to various embodiments, such as the reasoning and / or training logic used in Figure 6A and at least one embodiment of the present disclosure;

[0016] Figure 7A FIG. 9 is a block diagram illustrating an example computer system that can be a system with interconnected devices and components, a system-on-a-chip (SOC) or some combination thereof formed with a processor that can include execution units to execute instructions to support and / or implement the refrigerant-assisted cooling system described herein, according to at least one embodiment;

[0017] Figure 7B FIG. 10 is a block diagram illustrating an electronic device for utilizing a processor to support and / or implement the refrigerant-assisted cooling system described herein, according to at least one embodiment;

[0018] Figure 7C FIG. 11 is a block diagram illustrating an electronic device for utilizing a processor to support and / or implement the refrigerant-assisted cooling system described herein, according to at least one embodiment;

[0019] Figure 8 ​​​Another exemplary computer system for implementing various processes and methods described throughout this disclosure for a refrigerant- assisted cooling system is shown in accordance with at least one embodiment;

[0020] Figure 9A An exemplary architecture is shown in accordance with at least one embodiment of the present disclosure in which a GPU is communicatively coupled to a multi-core processor over a high-speed link to implement and / or support a refrigerant-assisted cooling system;

[0021] Figure 9B Additional details of an interconnect between a multi-core processor and a graphics acceleration module in accordance with one exemplary embodiment are shown;

[0022] Figure 9C Another exemplary embodiment is shown in accordance with at least one embodiment of the present disclosure in which an accelerator integration circuit is integrated within a processor to implement and / or support a refrigerant-assisted cooling system;

[0023] Figure 9D An exemplary accelerator integration tile 990 for implementing and / or supporting a refrigerant-assisted cooling system in accordance with at least one embodiment of the present disclosure is shown;

[0024] Figure 9E Additional details of one exemplary embodiment of a shared model for implementing and / or supporting a refrigerant-assisted cooling system in accordance with at least one embodiment of the present disclosure is shown;

[0025] Figure 9F Additional details of one exemplary embodiment of a unified memory that is addressable via a common virtual memory address space for accessing physical processor memory and GPU memory to implement and / or support a refrigerant-assisted cooling system in accordance with at least one embodiment of the present disclosure is shown;

[0026] Figure 10A An exemplary integrated circuit and associated graphics processor for a refrigerant-assisted cooling system in accordance with embodiments described herein is shown;

[0027] Figures 10B-10C An exemplary integrated circuit and associated graphics processor for supporting and / or implementing a refrigerant-assisted cooling system in accordance with at least one embodiment is shown;

[0028] Figures 10D-10E Additional exemplary graphics processor logic for supporting and / or implementing a refrigerant-assisted cooling system in accordance with at least one embodiment is shown;

[0029] Figure 11A is a block diagram showing a computing system for supporting and / or implementing a refrigerant-assisted cooling system in accordance with at least one embodiment;

[0030] Figure 11B A parallel processor to support and / or implement a refrigerant- assisted cooling system is shown in accordance with at least one embodiment;

[0031] Figure 11C is a block diagram of a partition unit in accordance with at least one embodiment;

[0032] Figure 11D A graphics multiprocessor for a refrigerant-assisted cooling system is shown in accordance with at least one embodiment;

[0033] Figure 11E A graphics multiprocessor is shown in accordance with at least one embodiment;

[0034] Figure 12A A multi-GPU computing system is shown in accordance with at least one embodiment;

[0035] Figure 12B is a block diagram of a graphics processor in accordance with at least one embodiment;

[0036] Figure 13 is a block diagram showing a micro-architecture for a processor that can include logic circuits to execute instructions in accordance with at least one embodiment;

[0037] Figure 14 A deep learning application processor is shown in accordance with at least one embodiment;

[0038] Figure 15 is a block diagram of a neuromorphic processor in accordance with at least one embodiment;

[0039] Figure 16A is a block diagram of a processing system in accordance with at least one embodiment;

[0040] Figure 16B is a block diagram of a processor with one or more processor cores, an integrated memory controller, and an integrated graphics processor in accordance with at least one embodiment;

[0041] Figure 16C is a block diagram of hardware logic for a graphics processor core in accordance with at least one embodiment;

[0042] Figures 16D-16E Thread execution logic including an array of processing elements of a graphics processor core is shown in accordance with at least one embodiment;

[0043] Figure 17A A parallel processing unit is shown in accordance with at least one embodiment;

[0044] Figure 17B A general processing cluster is shown in accordance with at least one embodiment;

[0045] Figure 17C A memory partition unit of a parallel processing unit is shown in accordance with at least one embodiment; and

[0046] Figure 17D A streaming multiprocessor is shown in accordance with at least one embodiment. DETAILED DESCRIPTION

[0047] In view of the high heat demand caused by today's computing components, air cooling of high density servers is inefficient and ineffective. Accordingly, the present disclosure seeks a liquid coolant and related system for cooling computing components, such as graphics processing units (GPUs), central processing units (CPUs), or switch components. These computing components are used in servers assembled in server trays on racks in data centers. As technology advances miniaturize computing components, server trays and racks accommodate more and more computing components, thus requiring to dissipate more heat generated per component than in past systems. One problem addressed by the present disclosure is the inability to provide on-demand cooling during periods of adverse seasonal effects on data centers.

[0048] Further, data center liquid cooling systems are supported by chiller equipment or systems that can be expensive as they are designed to have an over-provisioning demand. The over-provisioning demand can be due to a lack of proper control over the removal of heat from servers, such as from one or more components (e.g., GPUs, CPUs, etc.), from a collective of components in a server chassis, or from a collective of components in a rack. The efficient use of high heat capacity liquid cooling systems can be masked by many intermediary features in the cooling system, including liquid chillers, pumps, coolant distribution units (CDUs), and heat exchangers. Refrigerant assisted or refrigeration assisted cooling systems supplement high temperature utility water that can be used as a coolant in a primary cooling loop and an auxiliary cooling loop for liquid cooling of high heat components and systems. Accordingly, refrigerant assisted cooling systems supplement high peak regulation and remove extreme heat due to computing loads or environmental conditions. Accordingly, the problem is partially addressed by a refrigerant cooling loop that supplements an existing auxiliary cooling loop to absorb heat from an auxiliary cooling loop located within a data center, or to provide supplemental cooling to one or more components of a data center coupled to an auxiliary cooling loop.

[0049] As the power consumption and heat dissipation of GPU, CPU, and memory components that can be associated with servers and racks increase, liquid cooling becomes more effective for data centers that house such racks and servers. However, cooling through the cooling equipment and the primary cooling loop and the secondary cooling loop can be relatively weak, for example, due to over-heated environmental conditions. A direct expansion refrigeration cooling system that forms a refrigeration cooling loop, and a conventional coolant for the secondary cooling loop that circulates the hot coolant to the primary cooling loop, provide an effective method to address sudden cooling demands in a data center liquid cooling system. In one example, the sudden cooling demand stems from a high peak operating condition or an over-heated condition, when additional and emergency cooling is needed.

[0050] In at least one embodiment, the evaporator coil associated with the condensing unit is provided outside of the data center. In at least one embodiment, the condensing unit is as close to the secondary cooling loop as possible. In at least one embodiment, the condensing unit is located in a roof area of the data center and has direct access to a house manifold or row manifold to provide refrigerant in the refrigeration cooling loop to meet the immediate cooling demand. In at least one embodiment, the evaporator coil enables direct expansion refrigerant for the refrigeration cooling loop and coexists with the secondary cooling loop that carries the coolant, also known as a warm temperature liquid cooling system. The refrigeration cooling loop provides auxiliary cooling to the secondary cooling loop under peak conditions by directly cooling one or more components that need cooling, or by exchanging heat with the primary cooling loop or the secondary cooling loop. In at least one embodiment, the deployment of the refrigerant enables immediate cooling in one or more portions of the secondary cooling loop without the need to cool the coolant throughout the secondary cooling loop. The deployment of the coexisting refrigeration cooling system can be directly to one or more components through a cold plate with dual channels and associated with one or more components, such as a GPU, switch, CPU, or memory device similar to a dual in-line memory module (DIMM). The refrigeration cooling loop enters a portion of a server chassis in a rack through a server manifold.

[0051] The server chassis can include the components described above, but multiple server chassis at the rack level or multiple racks that benefit from liquid cooling can benefit from the present refrigerant auxiliary cooling system for a data center. The refrigeration or refrigerant auxiliary cooling system provides additional cooling closest to the heat source for extreme conditions, for example, allowing access to one or more components of a server chassis or a secondary cooling loop, thereby allowing the data center liquid cooling system, such as the secondary cooling loop, to operate efficiently in warm temperatures. This eliminates the additional cost of operating the data center cooler at a higher efficiency at lower or nominal ambient temperatures. In at least one embodiment, a micro-pump or other flow control device forms one or more flow controllers that are sensitive to the thermal requirements of at least one or more sensors in a server or rack.

[0052] Figure 1 is a block diagram of an example data center 100 having a cooling system subject to improvements described in at least one embodiment. The data center 100 can be one or more rooms 102 having racks 110 and ancillary equipment for housing one or more servers on one or more server trays. The data center 100 is supported by a cooling tower 104 located outside of the data center 100. The cooling tower 104 dissipates heat within the data center 100 by acting on a primary cooling loop 106. Further, a cooling distribution unit (CDU) 112 is used between the primary cooling loop 106 and a secondary or auxiliary cooling loop 108 to enable heat to be absorbed from the secondary or auxiliary cooling loop 108 to the primary cooling loop 106. In one aspect, the auxiliary cooling loop 108 is able to access all of the piping in the server trays as needed. The loops 106, 108 are illustrated as line drawings, but one of ordinary skill will recognize that one or more piping features can be used. In one example, flexible polyvinyl chloride (PVC) tubing can be used with associated piping to move fluid along each loop 106, 108. In at least one embodiment, one or more coolant pumps can be used to maintain a pressure differential within the loops 106, 108 to move coolant based on temperature sensors at different locations, including within the room, within one or more racks 110, and / or within server enclosures or server trays within the racks 110.

[0053] In at least one embodiment, the coolant in the primary cooling loop 106 and the secondary cooling loop 108 can be at least water and an additive, such as ethylene or propylene glycol. In operation, each primary and secondary cooling loop has its own coolant. In one aspect, the coolant in the secondary cooling loop can be dedicated to the needs of the components in the server trays or racks 110. The CDU 112 is capable of complex control of the coolant in the loops 106, 108 independently or simultaneously. For example, the CDU can be adapted to control flow rates in order to appropriately distribute one or more coolants to extract heat generated within the racks 110. In addition, more flexible tubing 114 is provided from the secondary cooling loop 108 to enter each server tray and provide coolant to the electrical and / or computing components. In this disclosure, electrical and / or computing components are used interchangeably to refer to heat generating components that benefit from the present data center cooling system. The tubing 118 that makes up a portion of the secondary cooling loop 108 can be referred to as a row manifold. In addition, the tubing 116 that extends from the tubing 118 can also be a part of the secondary cooling loop 108, but can be referred to as a column manifold. The tubing 114 that enters the racks as part of the secondary cooling loop 108, but can be referred to as a rack cooling manifold. In addition, the column manifold 116 extends along a column to all of the racks in the data center 100. The tubing of the secondary cooling loop 108, including the manifolds 118, 116, and 114, can be improved by at least one embodiment of the present disclosure. An optional chiller 120 can be provided in the primary cooling loop within the data center 102 to support cooling prior to the cooling tower. In the event that an additional loop is present in the primary control loop, one of ordinary skill in the art, upon reading the present disclosure, will recognize that the additional loop provides cooling outside of the racks and outside of the secondary cooling loop; and can be used with the primary cooling loop of the present disclosure.

[0054] In at least one embodiment, in operation, heat generated within the server trays of the rack 110 can be transported through the flexible tubing of the row manifold 114 of the secondary cooling loop 108 to the coolant exiting the rack 110. Accordingly, the secondary coolant from the CDU 112 (in the secondary cooling loop 108) for cooling the rack 110 moves to the rack 110. The secondary coolant from the CDU 112 is passed from one side of the house manifold with tubing 118 to one side of the rack 110 via the row manifold 116 and through the tubing 114 to one side of the server trays. The spent secondary coolant (or exiting secondary coolant carrying heat of computing components) exits from the other side of the server trays (e.g., into the left side of the rack and then exits the server trays from the right side of the rack after circulating through the server trays or components on the server trays). The spent secondary coolant exiting from the server trays or rack 110 flows out from a different side of the tubing 114 (e.g., on the exit side) and moves to a parallel but also exit side of the row manifold 116. The spent secondary coolant from the row manifold 116 moves in the parallel portion of the house manifold 118, traveling in the opposite direction and towards the CDU 112 from the incoming secondary coolant (which can also be refreshed secondary coolant).

[0055] In at least one embodiment, the spent secondary coolant exchanges its heat with the primary coolant in the primary cooling loop 106 via the CDU 112. The spent secondary coolant is refreshed (e.g., cooled down relatively compared to the temperature of the spent secondary coolant stage) and ready to be circulated back to the computing components through the secondary cooling loop 108. Various flow and temperature control features in the CDU 112 enable controlling the heat exchanged from the spent secondary coolant or the flow of the secondary coolant into and out of the CDU 112. The CDU 112 is also able to control the flow of the primary coolant in the primary cooling loop 106. Thus, some components within the servers and racks can not receive the required coolant level as the secondary or auxiliary loop portion provides coolant based on the temperature sensors that can be within the servers and racks with their default temperature properties. In addition, additional cooling loops can be provided to exchange heat with the secondary cooling loop, but are subject to the same usage of coolant as in the case of the primary or secondary cooling loop.

[0056] Figure 2A 、 Figure 2B 、 Figure 3A and Figure 3B are block diagrams illustrating server level features 200; 250; 300; 350 in conjunction with refrigerant secondary cooling in a data center, according to at least one embodiment. In Figure 2AIn at least one embodiment, server-level feature 200 is a server tray or server chassis. Thus, reference can be made to a server tray or server chassis, rather than server-level feature 200 which can be a component within a server. Server tray 200 can include one or more cold plates 220A-D (also referred to as cold plates). In at least one embodiment, server-level feature 200 is a server tray or chassis with cold plates that are detachable or integrated (e.g., monolithic structure) with the server tray or chassis structure. Server manifold 202A can be disposed within or proximate to server tray 200 and can include a server manifold cooling distribution unit (CDU) 202B. CDU 202B receives or includes at least one inlet pipe 204 that provides coolant from an auxiliary cooling loop (e.g., in example data center 100) to server-level feature 200. CDU 202B also provides or includes at least one outlet pipe 204 for returning coolant (spent coolant) to the auxiliary cooling loop. For example, coolant described elsewhere in this disclosure can be facility water.

[0057] Figure 2A Also shown are one or more component-level inlet pipes 212A; 214A for providing coolant associated with a component and being cooled by CDU 202B. One or more component-level outlet pipes 212C; 214C are provided for coolant associated with a component to exit back to CDU 202B. Inlet pipes can be reversed from outlet pipes so that coolant takes the shortest path to the cold plates. In at least one embodiment, intermediate pipes 212B; 214B are provided at a component level for enabling adaptation or configuration for serial cooling. In at least one embodiment, intermediate pipes can support serial or parallel coupling of cold plates, and can not be used when a CDU is directly associated with one cold plate. One or more cold plates 220A-D are integrally coupled or detachably coupled to components such as GPUs, CPUs, switches, and memory modules.

[0058] One or more flow controllers 222A, B can be provided to direct or facilitate cooling of one or more components associated with one or more cold plates 220A-D. In at least one embodiment, one or more flow controllers 222A are located within a server tray of a server-level feature 200 to facilitate movement of coolant associated with an auxiliary cooling loop to cool components associated with one or more cold plates 220A-D within a server. Cooling can be responsive to temperature sensors within a server or rack. In at least one embodiment, cooling is continuous and unimpeded by one or more flow controllers 222A. In at least one embodiment, the coolant is a component coolant in a component-level cooling loop that is part of an auxiliary cooling loop. An auxiliary coolant of the auxiliary cooling loop can be in thermal exchange with the component coolant of the component-level cooling loop via a CDU 202B. In at least one embodiment, the auxiliary coolant is also the component coolant that extends from the auxiliary control loop to components with or without a CDU 202B.

[0059] In at least one embodiment, refrigerant-assisted cooling can be provided to supplement cooling from or associated with an auxiliary cooling loop, such as a component-level cooling loop that carries a coolant. In at least one embodiment, one or more cooling plates 220A-D of a server-level feature 200 are adapted or configured to receive and output two different cooling loops. In at least one embodiment, a server manifold 202A can include a server manifold cooling distribution unit (CDU) 202B that supports or is adapted or configured to receive and output two different cooling loops. In addition to at least one inlet pipe 204 for a coolant associated with an auxiliary cooling loop (such as in example data center 100), the CDU 202B receives or includes at least one second inlet pipe 208 for a refrigerant of a refrigerant cooling loop. The at least one second inlet pipe 208 can be coupled to one or more server-level features 200. In addition to at least one outlet pipe 204 for returning a coolant associated with an auxiliary cooling loop (spent coolant), the CDU 202B provides or includes at least one second outlet pipe 206 for returning spent refrigerant to a refrigerant cooling loop. Unlike the coolant described elsewhere in this disclosure, the refrigerant can be, for example, R-134A, R-22, or R-407C.

[0060] Figure 2AOne or more component-level inlet conduits 216A; 218A are also shown for providing refrigerant associated with the components and being cooled by an external condenser unit (relative to the data center) via the CDU 202B. One or more component-level outlet conduits 216C; 218C are provided for the refrigerant associated with the components to exit back to the CDU 202B. The inlet conduits can be reversed from the outlet conduits so that the coolant uses the shortest path to the cold plates. In at least one embodiment, intermediate conduits 216B; 218B are provided at the component level to enable adaptation or configuration of serial cooling. In at least one embodiment, the intermediate conduits can support serial or parallel coupling of cold plates, and can not be used when the CDU is directly associated with one cold plate. As previously mentioned, one or more cold plates 220A-D are integrally coupled or detachably coupled to components such as GPUs, CPUs, switches, and memory modules.

[0061] One or more flow controllers 222B can be provided for directing or facilitating cooling of one or more components associated with one or more cold plates 220A-D of the refrigerant cooling loop, which is distinct from the coolant associated with the supplemental cooling loop. In at least one embodiment, one or more flow controllers 222B are located within the server trays of the server-level feature 200 for facilitating movement of refrigerant associated with the refrigerant cooling loop to provide supplemental cooling for components associated with one or more cold plates 220A-D within the server. The cooling can be responsive to a temperature sensor within the server or rack that indicates that sufficient cooling is not being achieved by the coolant associated with the supplemental cooling loop, or that supplemental cooling is needed regardless of the cooling by the coolant. In at least one embodiment, the refrigerant cooling is supplemental and can be modified or controlled by adjusting its flow rate via the controller 222B, while the coolant cooling is continuous and unimpeded by the controller 222A. In at least one embodiment, one or more flow controllers 222A, B can be located at the inlet and / or outlet sides of the inlet and outlet conduits of the respective cooling loops.

[0062] Temperature sensors associated with the server chassis provide temperature values ​​to enable control associated with flow controllers 222A and 222B. Therefore, the flow controllers are partially controlled based on sensed temperatures of one or more components associated with cold plates 220A-D. In at least one embodiment, when the temperature sensor indicates a high ambient temperature, and when the device controller (having a memory with instructions and a processor to execute those instructions) determines that the coolant has reached its maximum capacity, flow controller 222B can then be activated to provide refrigerant for supplemental cooling of the components associated with cold plates 220A-D. This causes flow controller 222B, associated with inlet pipe 216A, to facilitate (e.g., pump) more refrigerant from the refrigerant cooling loop into server feature 200. This... Figure 2B The example further illustrates this. However, the coolant flow rate can be kept constant and can be set by, for example, the CDU202B or the flow controller 222A associated with the auxiliary control loop.

[0063] In at least one embodiment, Figure 2B This illustrates another server-grade feature 250 that incorporates refrigerant-assisted cooling in a data center. Figure 2B Server-level feature 250 can be compared with Figure 2A Server-level feature 200 may be used in conjunction with or separately from component 266. Server-level feature 250 illustrates a cooling plate 264 (or other similar heat exchanger) integrated with or coupled to component 266 (which may be a CPU, GPU, DIMM, or switch). Other components requiring cooling may also be used with cooling plate 264. In at least one embodiment, server-level feature 250 includes channels, conduits, or other similar features 256, 258 for receiving and propelling a coolant different from the refrigerant. This enables cooling plate 264 to support supplemental cooling in addition to auxiliary cooling circuits. In at least one embodiment, cooling plate 264 includes receiving inlets 252, 254, which are different from each of the coolant and refrigerant cooling circuits. Cooling plate 264 may also include return outlets 260, 262 for returning the coolant or refrigerant to their respective cooling circuits. A heat transfer layer 268 formed of thermal paste or adhesive thermal features may be provided to couple cooling plate 264 and component 266 for efficient heat transfer.

[0064] Figure 3A and Figure 3B This is a block diagram illustrating server-level features 300 and 350 incorporating refrigerant-assisted cooling in a data center according to at least one embodiment. Figure 3B In at least one embodiment, server-level feature 300 is a server tray or server chassis, such as in Figure 2Aserver tray or server chassis, rather than server-level features 300, which can be components within a server. The server tray 300 can include one or more cold plates 320A-D (also referred to as cold plates). In at least one embodiment, the server-level features 300 are server trays or chassis with cold plates that are detachable or integrated with the server tray or chassis structure (e.g., monolithic structure). A server manifold 302A can be disposed within or proximate to the server tray 300 and can include a server manifold cold distribution unit (CDU) 302B. The CDU 302B receives or includes at least one inlet conduit 304 that provides coolant to the server-level features 300 from an auxiliary cooling loop (e.g., in the example data center 100). The CDU 302B also provides or includes at least one outlet conduit 304 for returning coolant (used coolant) to the auxiliary cooling loop. For example, the coolant described elsewhere in this disclosure can be facility water.

[0065] Figure 3A One or more component-level inlet conduits 312A; 314A are also shown for providing coolant associated with a component that is cooled via the CDU 302B. One or more component-level outlet conduits 312C; 314C are provided for the coolant associated with the component to exit back to the CDU 302B. The inlet conduits can be reversed from the outlet conduits so that the coolant uses the shortest path to the cold plates. In at least one embodiment, intermediate conduits 312B; 314B are provided at the component level for enabling adaptation or configuration for serial cooling. In at least one embodiment, the intermediate conduits can support serial or parallel coupling of cold plates, and can not be used when the CDU is directly associated with one cold plate. One or more cold plates 320A-D are integrally coupled or detachably coupled to components such as GPUs, CPUs, switches, and memory modules.

[0066] One or more flow controllers 322A, B can be provided for directing or facilitating cooling of one or more components associated with one or more cold plates 320A-D. In at least one embodiment, one or more flow controllers 322A are located within a server tray of a server-level feature 300 for facilitating movement of a coolant associated with an auxiliary cooling loop to cool components associated with one or more cold plates 320A-D within a server. Cooling can be in response to a temperature sensor within a server or rack. In at least one embodiment, cooling is continuous and unimpeded by the one or more flow controllers 322A. In at least one embodiment, the coolant is a component coolant in a component-level cooling loop that is part of an auxiliary cooling loop. An auxiliary coolant of the auxiliary cooling loop can be in heat exchange with the component coolant of the component-level cooling loop via a CDU 302B. In at least one embodiment, the auxiliary coolant is also the component coolant that extends from the auxiliary control loop to components with or without a CDU 302B.

[0067] In at least one embodiment, refrigerant-assisted cooling can be provided to supplement cooling from or associated with an auxiliary cooling loop, such as a component-level cooling loop that carries a coolant. In at least one embodiment, the refrigerant-assisted cooling can be provided in addition to or together with the server-level feature 200 of Figure 2A In at least one embodiment that is different from or used together with the embodiment of the server-level feature 200 in FIG. 1, a server manifold 302A is configured to receive two different cooling loops and use one of the two cooling loops to cool or exchange heat from the other of the two cooling loops. In at least one embodiment, the server manifold 302A can include a server manifold cooling distribution unit (CDU) 302B that supports or is adapted or configured to receive the two different cooling loops and is adapted or configured to allow heat exchange between the two cooling loops. In addition to at least one inlet conduit 304 for a coolant associated with an auxiliary cooling loop (such as in the example data center 100), the CDU 302B receives or includes at least one second inlet conduit 308 for a refrigerant of a refrigerant cooling loop. The at least one second inlet conduit 308 extends the length of the CDU 302B and exits from at least one second outlet conduit 306 that returns used refrigerant to the refrigerant cooling loop. Used refrigerant is used to describe refrigerant that is expanded based in part on heat removed from the coolant associated with the auxiliary cooling loop. As Figure 2AIn the case of at least one embodiment, the refrigerant can be different than the coolant and is described elsewhere in this disclosure, for example, it can be R-134A, R-22, or R-407C. In at least one embodiment, at least one second inlet conduit for the refrigerant extends the length of the CDU 302B in a zigzag pattern in the x-axis (across the width of the CDU) or the y-axis (across the length of the CDU) to maximize heat exchange from the coolant to the refrigerant.

[0068] Figure 3A Also shown in at least one embodiment are intermediate conduits 312B, 314B, which can support serial or parallel coupling of the cooling plates 320A-D, and can not be used when the CDU 302B is directly associated with one cooling plate. As previously described, one or more cold plates 320A-D are integrally coupled or detachably coupled to components such as GPUs, CPUs, switches, and memory modules.

[0069] One or more flow controllers 322B can be provided to direct or facilitate cooling of the coolant associated with the auxiliary cooling loop using the refrigerant cooling loop. In at least one embodiment, the one or more flow controllers 322B are located within the server tray of the server-level feature 300 to facilitate movement of the refrigerant associated with the refrigerant cooling loop to provide supplemental cooling to components associated with one or more cold plates 320A-D within the server. Even though the refrigerant cooling loop is not directly associated with the cooling plates 320A-D, supplemental cooling from the refrigerant cooling loop can be controlled in part due to temperatures sensed from within the server tray. In at least one embodiment, supplemental cooling from the refrigerant cooling loop can be controlled in part due to sensed temperatures of the coolant in the auxiliary cooling loop or from sensors associated with the auxiliary cooling loop.

[0070] In at least one embodiment, cooling can be responsive to temperature sensors within the server or rack that indicate that the coolant associated with the auxiliary cooling loop is not achieving sufficient cooling or that supplemental cooling is needed regardless of the cooling of the coolant. In at least one embodiment, the refrigerant cooling is supplemental and can be modified or controlled by adjusting its flow rate via the controller 322B, while the coolant cooling is continuous and unimpeded by the controller 322A. In at least one embodiment, one or more flow controllers 322A, B can be located at the inlet and / or outlet side of the inlet and outlet conduits of the respective cooling loops.

[0071] The temperature sensors associated with the server chassis provide temperature values to enable control associated with the flow controllers 322A, B. Thus, the flow controllers are controlled based in part on the sensed temperature of one or more components associated with the cold plates 320A-D. In at least one embodiment, when the temperature sensors indicate that the ambient temperature is high, and when the equipment controller (having memory with instructions and having a processor to execute the instructions) determines that the coolant has reached a maximum capacity, then the flow controller 322B can be activated to provide refrigerant for assisting in cooling the coolant associated with the secondary cooling loop, which is then able to absorb more heat from the cold plates 320A-D. In at least one embodiment, the refrigerant cooling loop is aligned proximate to the inlet piping associated with the coolant so that the coolant is further cooled to a lower temperature than that cooled by the cooling facility before reaching the cold plates 320A-D.

[0072] In at least one embodiment, the flow controller 322B associated with the inlet piping 312A is caused to facilitate (e.g., pump) more coolant associated with the secondary cooling loop into the server feature 200. This enables faster cooling of components associated with the cold plates and transfers supplemental cooling from the refrigerant to the components. This is further illustrated in the example of FIG. 3. However, the flow rate of the coolant can remain constant and can be set by, for example, the CDU 302B or the flow controller 322A associated with the secondary control loop. Figure 3B

[0073] Figure 3B is a block diagram illustrating a server-level feature 350 incorporating refrigerant-assisted cooling in a data center, in accordance with at least one embodiment. The server-level feature 350 can be a cross-section of a manifold heat exchanger 352. In at least one embodiment, the refrigerant piping or tubing within the manifold heat exchanger 352 is an evaporator coil for absorbing heat from the coolant of the secondary cooling loop. In at least one embodiment, the manifold heat exchanger 352 can be a server manifold that includes the CDU 302B. In at least one embodiment, the CDU 302B can be integrated with the server manifold 302A, and the manifold heat exchanger 352 is located within the CDU 302B. Thus, in at least one embodiment, the manifold heat exchanger 352 is a server-level manifold heat exchanger and can include one or more CDUs for operating the coolant portion of the refrigerant-assisted cooling system.

[0074] ​Manifold heat exchanger 352 includes one or more inlets 354, 360 for transporting coolant associated with the secondary cooling loop through manifold heat exchanger 352 via one or more conduits 356, 362. In at least one embodiment, manifold heat exchanger 352 includes outlets 358, 364 for passing coolant from the server (after the cold plate associated with the component) back to the secondary cooling loop (or component level cooling loop), or passing coolant associated with the secondary cooling loop back to the cold plate. Manifold heat exchanger 352 includes at least one second inlet 366 (serving as a refrigerant inlet) and at least one second outlet 370 (serving as a refrigerant outlet), both of which constitute part of a refrigerant cooling loop distinct from the secondary cooling loop or any component level cooling loop. Refrigerant compressed from an external refrigerant condensing unit is passed to refrigerant inlet 366 via conduit 368 provided, and then passed out from refrigerant outlet 370. As shown with reference to Figure 3A

[0075] Thus, manifold heat exchanger 352 enables supplemental cooling of the coolant as it is provided to or discharged from the cold plate. Manifold heat exchanger 352 can be directed to cool components of an adjacent server or rack, and in at least one embodiment, this is achieved by supplementally cooling the coolant as it exits the cold plate and server tray to be passed to the adjacent server or rack. This enables the coolant to effectively cool multiple servers or racks without having to be directed to a cooling device for cooling prior to being circulated back to the data center.

[0076] Figure 4A is an illustration of a rack level feature of a data center incorporating refrigerant assisted cooling in accordance with at least one embodiment. Figure 4A ​A rack 402 is shown having server trays or enclosures 410 and rack manifolds 404A, B. As with the case of the manifold heat exchanger 352, the rack manifolds 404A, B can be rack level heat exchangers and can include one or more CDUs for operating the coolant portion of the refrigerant assisted cooling system. The CDUs can be distinct from the heat exchanger features or can incorporate portions of the heat exchanger features. Thus, the rack level manifold heat exchangers 404A, B can be functionally similar to the server level heat exchangers 352. With reference to both, the server level and rack level heat exchangers can be referred to as manifold heat exchangers and their functions can be interrelated or used across both features of the refrigerant assisted cooling system.

[0077] The rack level manifold heat exchangers 404A, B include one or more inlets 402A, 402B for transporting coolant associated with the assisted cooling loop through the manifold heat exchangers 404A, B to one or more server trays, enclosures or cold plates (within the server trays or enclosures 410) via one or more tubes, as shown. In at least one embodiment, the manifold heat exchangers 404A, B include outlets 404A, B for passing the coolant from the servers (after the cold plates associated with the components) back to the assisted cooling loop (e.g., the row manifold or the room manifold of the assisted cooling loop) or to the cold plates associated with the assisted cooling loop.

[0078] The manifold heat exchangers 404A, B include at least one second inlet 406A; 406B for use as a refrigerant inlet and at least one second outlet 408A; 408B for use as a refrigerant outlet, both of which form part of a refrigerant cooling loop distinct from the assisted cooling loop. Refrigerant compressed from an external refrigeration condensing unit enters the refrigerant inlet 406A; 406B, passes through the provided tubes and is passed out from the refrigerant outlet 408A; 408B. As shown with reference to Figure 3A The tubes can be zig-zagged and designed to transport the refrigerant or cooling to the greatest extent within the manifold heat exchangers 404A; 404B, as shown. In at least one embodiment, the tubes can be bare copper and fans can be provided within the manifold heat exchangers 404A; 404B to enable the refrigerant to cool the locations within the manifold heat exchangers 404A; 404B where the tubes associated with the coolant pass through. Thus, the coolant is further cooled by the refrigerant within the manifold heat exchangers 404A; 404B.

[0079] Accordingly, the manifold heat exchangers 404A, B can effect supplemental cooling of the coolant as it is provided to the cold plates or server trays, or as it exits the cold plates or server trays. The manifold heat exchangers 404A; 404B can be directed to cool components of adjacent racks, and in at least one embodiment, by effecting supplemental cooling of the coolant as it exits the server trays to be delivered to the adjacent racks. This enables the coolant to effectively cool multiple servers or racks without having to be directed to a cooling device for cooling prior to being circulated back to the data center. The direction of the coolant and refrigerant in their respective cooling loops can be effectuated by flow controllers, as described elsewhere in other embodiments, but also apply here.

[0080] Figure 4B is an illustration of a data center 450 having room level or row level features, and having external features that incorporate refrigerant assisted cooling, according to at least one embodiment. A refrigeration condensing unit 452 can be located proximate to the feature of the data center room 464 that requires immediate cooling. In at least one embodiment, the refrigeration condensing unit 452 is located on the roof of the data center room 464, but can also be located on the ground adjacent to the data center room 464. The refrigeration condensing unit 452 is shown with a refrigerant inlet 456A for receiving refrigerant that can be in a heated vapor stage, and is shown with a condenser coil 466 for condensing the refrigerant to a liquid prior to delivering the refrigerant back to the data center room 464 via a refrigerant outlet 454A. The refrigeration condensing unit 452 is the beginning and end of the refrigerant cooling loop discussed elsewhere in at least one embodiment herein. Refrigerant is delivered from the refrigeration condensing unit 452 to an inlet of a room level or row level manifold heat exchanger 458.

[0081] Figure 4B Also shown are racks 460A-N, which have server trays or enclosures, and are cooled by coolant from the room level or row level manifold heat exchanger 458. As with the manifold heat exchangers 352, 404A, B, the room level or row level manifold heat exchanger 458 can be a heat exchanger, and can include one or more CDUs for operating the coolant portion of the refrigerant assisted cooling system. The CDUs can be distinct from the heat exchanger feature, or incorporate portions of the heat exchanger feature. Accordingly, the room level or row level manifold 458 can be functionally similar to the heat exchangers 352, 404A, B discussed herein with respect to other embodiments. Room level or row level, server level, and rack level heat exchangers can be referenced by reference to the manifold heat exchangers, the functionality of which can be interrelated or used across two features of the refrigerant assisted cooling system.

[0082] The row-level or row manifold heat exchanger 458 includes one or more outlets 468A-N for delivering coolant associated with the secondary cooling loop through the manifold heat exchanger 458 to one or more racks (and their respective server trays or enclosures) via one or more lines. In at least one embodiment, the manifold heat exchanger 458 includes an inlet 470A-N for transferring coolant from the racks back to the secondary cooling loop.

[0083] The manifold heat exchanger 458 includes at least one second inlet 462A for transferring coolant from the manifold heat exchanger 458 to a CDU of the primary cooling loop and at least one second outlet 462B for receiving coolant from the CDU of the primary cooling loop. This allows for heat exchange from the secondary cooling loop to the primary cooling loop for processing in the cooling facility. The manifold heat exchanger 458 includes a third inlet 454B (serving as a refrigerant inlet) and at least one second outlet 456B (serving as a refrigerant outlet), which make up part of a refrigerant cooling loop that is distinct from the secondary cooling loop. Refrigerant compressed from the external refrigeration condensing unit 452 is transferred to the refrigerant inlet 454B, through the provided tubing, and then out from the refrigerant outlet 456N. As shown in reference to FIGS. 4A and 4B, the tubing can be zig-zagged and designed for maximum delivery of refrigerant or cooling within the manifold heat exchanger 458. In at least one embodiment, the tubing can be bare copper, and a fan can be provided within the manifold heat exchanger 458 to enable the refrigerant to cool the location within the manifold heat exchanger 458 where the tubing associated with the coolant passes through. Thus, the coolant is further cooled by the refrigerant within the manifold heat exchanger 458. Figure 3A and Figure 4A

[0084] Thus, the manifold heat exchanger 458 enables supplemental cooling of the coolant as it is provided to a rack, or as the coolant exits a rack to go to an adjacent rack. The manifold heat exchanger 458 can be directed to cool components of an adjacent rack, and in at least one embodiment, this is done by supplementally cooling the coolant as it exits a server tray to be transferred to an adjacent rack. This enables the coolant to effectively cool multiple servers or racks without having to be directed to a cooling device for cooling before being circulated back to the data center. The direction of the coolant and refrigerant in their respective cooling loops can be achieved through flow controllers, as described elsewhere in other embodiments, but also apply here.

[0085] Figure 5 are adapted for use in or manufacture of a system for cooling data centers in accordance with at least one embodiment Figures 2A-4B and Figures 6A-17D ​The process flow 500 of the steps of the method of cooling a system. The sub-process 502 provides one or more flow controllers associated with a refrigerant cooling loop. The sub-process 504 facilitates movement of coolant in an auxiliary cooling loop. In at least one embodiment, this can be achieved through a flow controller such as a micro-pump associated with an inlet and / or outlet of the auxiliary cooling loop. The sub-process 506 enables one or more flow controllers of the refrigerant cooling loop to receive an input associated with a temperature of the auxiliary cooling loop or one or more components cooled by the auxiliary cooling loop. In at least one embodiment, the one or more components are associated with a cold plate, a server chassis or tray, or a rack of a data center. In at least one embodiment, the input is provided from a device controller that controls the one or more flow controllers of the refrigerant cooling loop and is associated with temperature monitoring from one or more sensors of the rack, server chassis or tray, or components associated with the auxiliary cooling loop. The device controller provides the input to modify the flow of refrigerant in the refrigerant cooling loop.

[0086] In at least one embodiment, the decision sub-process 508 is implemented to verify whether the one or more flow controllers of the refrigerant cooling loop have received the input. In one example, the coolant of the auxiliary cooling loop is constantly flowing through the loop, but the refrigerant of the refrigerant cooling loop is modified to flow only when supplemental cooling is needed. The supplemental cooling can be of the coolant itself or of the components associated with the auxiliary cooling loop that has received the coolant but needs supplemental cooling according to the temperature monitoring from the temperature sensors in the server chassis or tray or the rack that hosts the server chassis or tray. The sub-process 510 enables the refrigerant of the refrigerant cooling loop to provide supplemental cooling to the coolant or one or more components. When the input is not received, the refrigerant can be made to flow slowly or not at all until supplemental cooling is needed. Thus, the refrigerant condensing unit can be turned on or off according to the supplemental cooling needed in one or more data center rooms of the data center.

[0087] In at least one embodiment, the flow controller can be provided on the outlet side of the pipe in any data center feature. Further, the flow controller can be provided on the inlet side and the outlet side of the pipe of any data center feature. Thus, when the flow controller is located on the outlet side, it performs a suction action rather than a push action. When two flow controllers work in series, there is both a suction action and a push action. Higher flow rates can be achieved by the series flow controllers. For example, the configuration or adaptation of the flow controllers can be determined by the requirements of the components.

[0088] In at least one embodiment, if the secondary cooling loop is directly available to a component, the secondary cooling loop facilitates a default or standard movement of the coolant or second coolant in the component-level cooling loop. In at least one embodiment, the learning subsystem can be implemented via a deep learning application processor (e.g., processor 1400 in Figure 14 FIG. 1) and can use neurons 1502 and components thereof implemented using circuitry or logic, including one or more arithmetic logic units (ALUs) as described in Figure 15 FIG. 2. As such, the learning subsystem includes at least one processor to evaluate a temperature of a rack or server at a flow rate associated with one or more flow controllers of the refrigerant cooling loop. For example, a flow rate for cooling of the coolant used to implement the secondary cooling loop or for rack or server cooling of a particular component can be indexed for use in training the learning subsystem.

[0089] Once trained, the learning subsystem is able to provide an output related to a flow rate for facilitating movement of the refrigerant by controlling the one or more flow controllers of the refrigerant cooling loop while the coolant of the secondary cooling loop continues its normal operation as described with reference to Figures 2A-5 In at least one embodiment, temperatures used in the learning subsystem can be values inferred from voltage and / or current values output by the various components, but the voltage and / or current values themselves and the requisite flow rate of the refrigerant needed to bring the coolant or component to that temperature can be used to train the learning subsystem. Alternatively, the temperature reflects a temperature at the requisite flow rate of the refrigerant needed to cool the server, rack, or component that exceeds the temperature already provided by the coolant, and the subsequent temperature can be used by the learning subsystem to reflect the temperature when the requisite flow rate is no longer needed. Alternatively, the temperature difference of the coolant or server or rack and the subsequent temperature, and the requisite flow rate, are used to train the learning system to identify when to activate and deactivate the associated flow controllers of the refrigerant cooling loop. Once trained, the learning subsystem is able to provide one or more outputs to control the associated flow controllers of the refrigerant cooling loop in response to temperatures received from temperature sensors associated with the server, rack, or with the secondary cooling loop via a device controller described elsewhere in the disclosure.

[0090] Additionally, the learning subsystem executes a machine learning model that processes temperatures collected from previous applications (possibly in a test environment) to control temperatures in a coolant of a server, rack, or auxiliary cooling loop. The collected temperatures can include (a) temperatures obtained for certain flow rates of refrigerant; (b) temperature differentials obtained for certain flow rates of refrigerant over a particular time period; and (c) initial temperatures and corresponding flow rates of refrigerant that were applied to maintain the server, rack, or coolant in an optimal operating condition. In at least one embodiment, the collected information can include references to a coolant type or a region type (e.g., server or rack) that reflect a level of cooling required on the spot.

[0091] In at least one embodiment, aspects of the processing of the deep learning subsystem can use collected information processed according to the references Figure 14 , Figure 15 discussed. In one example, the processing uses multiple neuron levels of a machine learning model that are loaded with one or more of the collected temperature features described above and corresponding flow rates of refrigerant in the refrigerant cooling loop. The learning subsystem executes a training that can be represented as evaluating temperature changes associated with previous flow rates (or changes in flow rates) of refrigerant according to adjustments made to one or more flow controllers associated with the refrigerant cooling loop. The neuron levels can store values associated with the evaluation process and can represent an association or correlation between temperature changes and flow rates. The learning subsystem, once the training is complete, is able to determine, in an application, a flow rate of refrigerant required to achieve supplemental cooling of a temperature (or a change in temperature, such as a temperature reduction) of, for example, a server, rack, or coolant. The learning subsystem can use the collected temperatures and previous associated flow rates of refrigerant used to achieve the collected temperatures (or, for example, temperature differentials) to provide an output associated with a required flow rate of refrigerant to achieve supplemental cooling reflected by a temperature value (e.g., representing a temperature reduction) compared to an existing temperature of a rack, server, or coolant in one or more portions of an auxiliary cooling loop.

[0092] In at least one embodiment, the learning subsystem, in response to a result from a sensed temperature of a rack, server, or coolant, is an output to one or more flow controllers of a refrigerant cooling loop that modifies a flow rate of refrigerant associated with the refrigerant cooling loop. The modification of the flow rate enables a determined flow of refrigerant to reach a region of a rack, server, or coolant that requires supplemental cooling. The modified flow rate can be maintained until a temperature of the region reaches a temperature associated with the determined flow rate of refrigerant known to the learning subsystem. Alternatively, the modified flow rate can be maintained until the temperature of the region changes by a determined value. Alternatively, the modified flow rate can be maintained until the temperature of the region reaches a rated temperature of the coolant, server, or rack.

[0093] In at least one embodiment, the equipment controller includes a processor having at least one logic unit to control one or more flow controllers associated with a refrigerant cooling loop. In at least one embodiment, the equipment controller can be a processor of a data center, such as processor 702 of Figure 7A In at least one embodiment, the flow controllers facilitate movement of refrigerant associated with the refrigerant cooling loop to effect supplemental cooling of an area in the data center in response to a sensed temperature in the area. In at least one embodiment, the processor is a processor core of a multi-core processor, such as multi-core processor 905, 906 in Figure 9A In at least one embodiment, the at least one logic unit can be adapted to receive temperature values from a temperature sensor of a server, in-rack, or associated with an auxiliary cooling loop, and can be adapted to facilitate movement of refrigerant associated with the refrigerant cooling loop to further cool the server, rack, or coolant associated with the auxiliary cooling loop. The temperature sensor can sense a temperature external to a component within the server, and can cause the equipment controller to provide additional cooling through the refrigerant cooling loop.

[0094] In at least one embodiment, the processor can be one processor core of multi-core processor 905, 906 in Figure 9A and can include a learning subsystem. The learning subsystem is to evaluate a temperature of a server, rack, coolant, or even a component within the server, and a flow rate associated with one or more flow controllers of the refrigerant cooling loop. The learning subsystem provides an output associated with the flow rate to facilitate movement of the refrigerant by controlling the one or more flow controllers associated with the refrigerant cooling loop. In at least one embodiment, the learning subsystem executes a machine learning model to process the temperature using a plurality of neuron levels of the machine learning model having previously associated flow rates of the temperature and the refrigerant. The machine learning model can be implemented using the neuron structure described in Figure 15 and the deep learning processor described in Figure 14 The machine learning model provides the output associated with the flow rate to the one or more flow controllers in accordance with the evaluation of the previously associated flow rates. Further, an instruction output of the processor, such as a pin of a connector bus or a ball of a ball grid array, causes the output to be communicated to the one or more flow controllers to modify the flow rate of the refrigerant associated with the refrigerant cooling loop and cause the flow rate of the refrigerant to be determined in response to the output.

[0095] In at least one embodiment, the present disclosure relates to at least one processor for a cooling system. The at least one processor includes at least one logic unit to train a neural network having a hidden layer of neurons. The neural network is trained to evaluate a temperature associated with an area of a data center (e.g., a server, a component within a server, a rack, or a coolant of an auxiliary cooling loop) and a previously associated flow rate of a refrigerant to supplement cooling of the area. As described elsewhere in the present disclosure, and with reference to Figure 3A , Figure 4A , Figure 4B , Figure 14 and Figure 15 , the training can be performed by a layer of neurons provided with inputs of temperatures of one or more areas of a data center, possibly in a test environment, and inputs of associated flow rates of refrigerants from previous applications. The temperatures can include a starting temperature (and an associated flow rate of a refrigerant applied to the starting temperature to bring the temperature to a nominal temperature of the one or more areas); a final temperature (after a period of time in which a refrigerant has been applied to the coolant of the area at a certain flow rate); or a temperature difference obtained for a flow rate of a refrigerant and a period of time of application of the refrigerant. One or more of these features can be used to train the neural network to determine when to apply a flow of refrigerant and / or when to stop a flow of refrigerant for supplemental cooling - for example, when a temperature of a certain area is sensed to reach a starting temperature, when a temperature of the area is sensed to reach a final temperature, and / or when a sensed temperature of the area reflects a temperature difference suitable for the area (e.g., a temperature decrease to a nominal temperature).

[0096] In at least one embodiment, the at least one processor includes at least one logic unit configured or adapted to train the neural network. The processor can be a multi-core processor. In at least one embodiment, the at least one logic unit can be located within one processor core that is operable to evaluate a temperature of a certain area of a data center according to the neural network and output instructions to facilitate supplemental cooling of the area in need of such supplemental cooling. Thus, even if the area has received coolant associated with an auxiliary cooling loop, it is able to receive refrigerant for supplemental cooling and control the temperature of the area through a request or instruction for refrigerant auxiliary cooling. In response to the request or instruction, the refrigerant can be provided at a flow rate determined for a current temperature or a target temperature of the area. In at least one embodiment, the processor of the device controller has an output of instructions in the form of a pin or a ball for communication with one or more flow controllers associated with a refrigerant cooling loop. Thus, the one or more flow controllers modify a flow rate of the associated refrigerant. The modification is thus a response to the output, facilitating cooling of one or more areas of a data center. In at least one embodiment, the at least one logic unit is adapted to receive temperature values from temperature sensors associated with the device controller.

[0097] In at least one embodiment, a data center management system (DMS) houses at least one processor of a device controller. In at least one embodiment, the DMS is a distributed system that communicates from the device controller to sub-controllers directly associated with flow controllers of a refrigerant cooling loop. In at least one embodiment, the device controller is a distributed network of processors and memory having instructions executed by the processors. Each processor in the distributed network is located proximate to and controls a respective flow controller or plurality of flow controllers. In at least one embodiment, one or more processors are adapted to train and execute neural networks discussed elsewhere in this disclosure. Thus, there can be at least one or more neural networks actively executing within one or more processors at any time, while one or more neural networks on standby are trained to actively associate information and actively output speculative flow rate information that can be used to control flow controllers of a refrigerant cooling loop.

[0098] In at least one embodiment, a compute-intensive operation can require that a refrigerant cooling loop be first started for a data center or a portion of a data center prior to starting the compute-intensive operation. In at least one embodiment, the startup process is a computer-intensive process because many associated computing components are brought online. There can be temperature spikes for many components, especially processing components. The temperature spikes require cooling at least until the temperature stabilizes. Thus, in at least one embodiment, a refrigerant cooling loop is activated first for a server, component, or rack associated with the startup process. Then, an auxiliary cooling loop can be activated to maintain the temperature of the server, component, or rack. In at least one embodiment, once the temperature stabilizes with only the refrigerant cooling loop, the auxiliary cooling loop is activated.

[0099] In at least one embodiment, the refrigerant cooling loop is inactive when it is not circulating refrigerant. The same applies to the supplemental cooling loop. In at least one embodiment, the refrigerant cooling loop is adapted to provide supplemental cooling to one or more components of the data center during a startup process of the one or more components. The supplemental cooling loop is adapted to remain inactive during the startup process and be activated upon receiving an indication of temperature stabilization associated with the one or more components, where the temperature does not peak and remains at least within an intermediate temperature variation range for a period of time after the startup process or cycle is complete. In at least one embodiment, the supplemental cooling loop is activated a predetermined time after the refrigerant cooling loop is activated. In at least one embodiment, even though the refrigerant cooling loop is activated first and no coolant is flowing through the supplemental cooling loop to the one or more components, the refrigerant cooling loop is still providing supplemental cooling because the supplemental cooling loop is activated later, but can remain activated once the temperature associated with the one or more components stabilizes. Thus, the refrigerant provides an increase in cooling demand, whether or not coolant is flowing through the supplemental cooling loop, which is a supplement.

[0100] data center

[0101] Figure 6A An example data center 600 that can use at least one embodiment from FIGS. 2-5 is shown. In at least one embodiment, the data center 600 includes a data center infrastructure layer 610, a framework layer 620, a software layer 630, and an application layer 640. In at least one embodiment, features from the components 204-214, for example at least one embodiment described in relation to FIG. 2, can be performed within or in cooperation with the example data center 600. In at least one embodiment, the infrastructure layer 610, the framework layer 620, the software layer 630, and the application layer 640 can be provided in part or in whole by computing components located on server trays in the racks 210 of the data center 200. This enables the cooling systems of the present disclosure to directly cool certain of the computing components in an efficient and effective manner. Moreover, various aspects of the data center, including the data center infrastructure layer 610, the framework layer 620, the software layer 630, and the application layer 640, can be used to support the refrigerant supplemental cooling discussed above at least with reference to Figures 2A-5 the discussion. Thus, reference to Figures 6A-17D the discussion can be understood to apply to hardware and software features needed to implement or support a refrigerant supplemental cooling system for a data center such as Figure 2A , Figure 3A , Figure 4A and Figure 4B .

[0102] In at least one embodiment, as Figure 6AAs shown, the data center infrastructure layer 610 can include a resource orchestrator 612, grouped computing resources 614, and node computing resources (“node C.R.s”) 616(1)-616(N), where “N” represents any whole, positive integer. In at least one embodiment, the node C.R.s 616(1)-616(N) can include, without limitation, any number of central processing units (“CPUs” or “processors”), including accelerators, field programmable gate arrays (FPGAs), graphics processors, and / or the like, memory devices (e.g., dynamic random access memory), storage devices (e.g., solid state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, cooling modules, and / or the like. In at least one embodiment, one or more node C.R.s of the node C.R.s 616(1)-616(N) can be a server having one or more of the above-described computing resources.

[0103] In at least one embodiment, the grouped computing resources 614 can include individual groups of node C.R.s housed within one or more racks (not shown), or housed within a number of racks (also not shown) within various geographic locations. An individual group of node C.R.s within the grouped computing resources 614 can include a grouped computing, network, memory, or storage resource that can be configured or allocated to support one or more workloads. In at least one embodiment, several node C.R.s including CPUs or processors can be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks can also include any number and combination of power modules, cooling modules, and network switches.

[0104] In at least one embodiment, the resource orchestrator 612 can configure or otherwise control the one or more node C.R.s 616(1)-616(N) and / or the grouped computing resources 614. In at least one embodiment, the resource orchestrator 612 can include a software design infrastructure (“SDI”) management entity for the data center 600. In at least one embodiment, the resource orchestrator can comprise hardware, software, or some combination thereof.

[0105] In at least one embodiment, as Figure 6AAs shown, the framework layer 620 includes a job scheduler 622, a configuration manager 624, a resource manager 626, and a distributed file system 628. In at least one embodiment, the framework layer 620 can include a framework that supports software 632 of a software layer 630 and / or one or more applications 642 of an application layer 640. In at least one embodiment, software 632 or applications 642 can include web-based service software or applications, respectively, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 620 can be, but is not limited to, a type of free and open-source software web application framework such as Apache Spark™ (hereinafter “Spark”) that can utilize the distributed file system 628 for large-scale data processing (e.g., “big data”). In at least one embodiment, the job scheduler 622 can include a Spark driver to facilitate scheduling workloads supported by various layers of the data center 600. In at least one embodiment, the configuration manager 624 can be capable of configuring different layers, such as the software layer 630 and the framework layer 620 including Spark and the distributed file system 628 for supporting large-scale data processing. In at least one embodiment, the resource manager 626 can be capable of managing clustered or grouped computing resources mapped to or allocated for supporting the distributed file system 628 and the job scheduler 622. In at least one embodiment, the clustered or grouped computing resources can include the grouped computing resources 614 on the data center infrastructure layer 610. In at least one embodiment, the resource manager 626 can coordinate with the resource orchestrator 612 to manage these mapped or allocated computing resources.

[0106] In at least one embodiment, the software 632 included in the software layer 630 can include software used by at least a portion of the node C.R.s 616(1)-616(N), the grouped computing resources 614, and / or the distributed file system 628 of the framework layer 620. One or more types of software can include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0107] In at least one embodiment, one or more applications 642 included in application layer 640 can include one or more types of applications used by at least portions of node C.R.s 616(1)-616(N), grouped computing resources 614, and / or distributed file system 628 of framework layer 620. One or more types of applications can include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications including training or inferencing software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0108] In at least one embodiment, any of configuration manager 624, resource manager 626, and resource orchestrator 612 can implement any number and type of self-modifying actions based on any number and type of data acquired in any technically feasible fashion. In at least one embodiment, self-modifying actions can relieve data center operators of data center 600 from making possibly poor configuration decisions and can avoid underutilized and / or poorly performing portions of a data center.

[0109] In at least one embodiment, data center 600 can include tools, services, software, or other resources to train one or more machine learning models or use one or more machine learning models to predict or infer information in accordance with one or more embodiments herein. In at least one embodiment, a machine learning model can be trained by computing weight parameters according to a neural network architecture using software and computing resources described above with respect to data center 600. In at least one embodiment, using weight parameters computed through one or more training techniques herein, a trained machine learning model corresponding to one or more neural networks can be used to infer or predict information using resources described above with respect to data center 600. As previously discussed, deep learning techniques can be used to support intelligent control of flow controllers in refrigerant assisted cooling by monitoring regional temperatures of a data center. Deep learning can be advanced using any appropriate learning network and computing functionality of data center 600. Thus, in this manner, a deep neural network (DNN), recurrent neural network (RNN), or convolutional neural network (CNN) can be supported simultaneously or concurrently using hardware in a data center. For example, once a network has been trained and successfully evaluated to identify data in a subset or slice, the trained network can provide similar representative data for use with collected data.

[0110] In at least one embodiment, data center 600 can use CPUs, application specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware to perform training and / or inferencing using resources described above. Moreover, one or more software and / or hardware resources described above can be configured as a service to allow users to train or perform inferencing on information, such as pressure, flow rate, temperature, and location information, or other artificial intelligence services.

[0111] Inference and training logic

[0112] Inference and / or training logic 615 can be used to perform inferencing and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 615 can be used in a system Figure 6A on which neural network training operations are performed to infer or predict operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases herein. In at least one embodiment, inference and / or training logic 615 can include, without limitation, hardware logic in which computational resources are dedicated or otherwise uniquely used in conjunction with weight values or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, inference and / or training logic 615 can be used in conjunction with an application specific integrated circuit (ASIC), such as Tensorflow® Processing Unit (TPU) from Google, an inference processing unit (IPU) from Graphcore processing units, an inference processing unit (IPU) from Graphcore TM or a GPGPU (“Lake Crest”) processor from Intel Corp.

[0113] In at least one embodiment, inference and / or training logic 615 can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware such as field programmable gate arrays (FPGAs). In at least one embodiment, inference and / or training logic 615 includes, without limitation, code and / or data storage models that can be used to store code (e.g., graph code), weight values, and / or other information including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. In at least one embodiment, each code and / or data storage model is associated with a dedicated computational resource. In at least one embodiment, a dedicated computational resource includes computational hardware that further includes one or more ALUs that only perform mathematical functions (e.g., linear algebra functions) on information stored in a code and / or data storage model and stores results from which in an active storage model of inference and / or training logic 615.

[0114] Figure 6B 、 Figure 6CInference and / or training logic 615 are used in the Figure 6A at least one embodiment of this disclosure. The inference and / or training logic 615 are used to perform inferencing and / or training operations associated with at least one embodiment. Details regarding the inference and / or training logic 615 are provided below in conjunction with FIGS. 6A and / or 6B. Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided below in conjunction with FIGS. 6A and / or 6B. Inference and / or training logic 615, used in the Figure 6B and Figure 6C of the inference and / or training logic 615. In at least one embodiment, each of the compute hardware 602 and the compute hardware 606 includes one or more ALUs that perform mathematical functions (e.g., linear algebraic functions) only on information stored in the code and / or data memory 601 and the code and / or data memory 605, respectively, whose results are stored in the activation memory 620. As such, unless otherwise specified, the Figure 6B and Figure 6C may be alternative and can be used interchangeably.

[0115] In at least one embodiment, the inference and / or training logic 615 can include, without limitation, code and / or data storage 601 to store forward and / or output weights and / or input / output data and / or other parameters for neurons or layers of a neural network trained and / or used for inferencing in at least one embodiment. In at least one embodiment, the training logic 615 can include code and / or data storage 601 to include or couple to logic to store graph code or other software to control timing and / or order, where weight and / or other parameter information is loaded to configure the logic, including integer and / or floating point units (collectively, arithmetic logic unit(s) (ALUs)). In at least one embodiment, code, such as graph code, loads weight or other parameter information into processor ALUs based on an architecture of a neural network to which the code corresponds. In at least one embodiment, code and / or data storage 601 stores weight parameters and / or input / output data for each layer of a neural network trained or used in conjunction with at least one embodiment during forward propagation of input / output data and / or weight parameters during training and / or inferencing using aspects of at least one embodiment. In at least one embodiment, any portion of code and / or data storage 601 can be included with other on-chip or off-chip data storage, including a processor’s Ll, L2, or L3 cache or system memory.

[0116] In at least one embodiment, any portions of code and / or data storage 601 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or code and / or data storage 601 can be cache memory, dynamic random addressable memory (“DRAM”), static random addressable memory (“SRAM”), non-volatile memory (e.g., Flash), or other storage. In at least one embodiment, a choice of whether code and / or data storage 601 is internal or external to a processor, e.g., or comprises DRAM, SRAM, Flash, or some other storage type, can depend on available storage space on-chip or off-chip, latency requirements of training and / or inferencing functions being performed, batch size of data used in inferencing and / or training of a neural network, or some combination of these factors.

[0117] In at least one embodiment, inferencing and / or training logic 615 can include, without limitation, code and / or data storage 605 to store backward and / or output weight and / or input / output data corresponding to neurons or layers of a neural network trained and / or used for inferencing in at least one embodiment. In at least one embodiment, code and / or data storage 605 stores weight parameters and / or input / output data for each layer of a neural network trained or used in conjunction with at least one embodiment during backward propagation of input / output data and / or weight parameters during training and / or inferencing using at least one embodiment. In at least one embodiment, training logic 615 can include or be coupled to code and / or data storage 605 to store graph code or other software to control timing and / or order, wherein weight and / or other parameter information is loaded to configure logic, including integer and / or floating point units (collectively, arithmetic logic units (ALUs)).

[0118] In at least one embodiment, code such as graph code loads weight or other parameter information into processor ALUs based on an architecture of a neural network to which code corresponds. In at least one embodiment, any portion of code and / or data storage 605 can be included with other on-chip or off-chip data storage, including a processor’s LI, L2, or L3 cache or system memory. In at least one embodiment, any portion of code and / or data storage 605 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or data storage 605 can be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, a choice of whether code and / or data storage 605 is internal or external to a processor, e.g., including DRAM, SRAM, flash memory, or some other storage type, depends on whether available storage is on-chip or off-chip, latency requirements of training and / or inferencing functions being performed, batch size of data being used in inferencing and / or training of a neural network, or some combination of these factors.

[0119] In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 can be separate storage structures. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 can be the same storage structure. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 can be partially the same storage structure and partially separate storage structures. In at least one embodiment, any portion of code and / or data storage 601 and code and / or data storage 605 can be included with other on-chip or off-chip data storage, including a processor’s LI, L2, or L3 cache or system memory.

[0120] In at least one embodiment, inference and / or training logic 615 can include, without limitation, one or more arithmetic logic unit(s) (“ALUs”) 610 including integer and / or floating point units for performing computations and / or mathematical operations based, at least in part, on training and / or inference code (e.g., graph code) or instructions therefrom. In at least one embodiment, results of computations and / or mathematical operations performed by ALU(s) 610 can produce, as output, activations stored in activation storage 620 that are functions of input / output and / or weight parameter data stored in code and / or data storage 601 and / or code and / or data storage 605. In at least one embodiment, activations stored in activation storage 620 are generated by ALU(s) 610 in response to executing instructions or other code, where weight values stored in code and / or data storage 605 and / or code and / or data storage 601 are used as operands along with other values such as bias values, gradient information, momentum values, or other parameters or hyperparameters, any or all of which can be stored in code and / or data storage 605 and / or code and / or data storage 601 or other on-chip or off-chip storage.

[0121] In at least one embodiment, one or more ALUs 610 are included in one or more processors or other hardware logic devices or circuits, while in another embodiment one or more ALUs 610 can be external to a processor or other hardware logic device or circuit using them (e.g., a co-processor). In at least one embodiment, one or more ALUs 610 can be included within execution units of a processor, or otherwise included in a group of ALUs accessible by execution units of a processor, which can be within a same processor or distributed between different types of processors (e.g., central processing units, graphics processing units, fixed function units, etc.). In at least one embodiment, code and / or data storage 601, code and / or data storage 605, and activation storage 620 can be on a same processor or other hardware logic device or circuit, while in another embodiment they can be in different processors or other hardware logic devices or circuits or some combination of same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of activation storage 620 can be included with other on-chip or off-chip data storage, including L1, L2, or L3 cache(s) of a processor or system memory. Moreover, inference and / or training code can be stored with other code accessible to a processor or other hardware logic or circuitry and can be fetched and / or processed using fetch, decode, schedule, execute, exit, and / or other logic circuitry of a processor.

[0122] In at least one embodiment, the active memory 620 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the active memory 620 may be wholly or partially located inside or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the active memory 620 is internal to or external to the processor may depend on the available on-chip or off-chip storage, the latency requirements for training and / or inference functions, the batch size of data used in inference and / or training the neural network, or some combination of these factors. For example, it may include DRAM, SRAM, flash memory, or other storage types. In at least one embodiment, Figure 6B The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as those from Google. Processing unit, from Graphcore TM Inference processing units (IPUs) or from Intel Corp. (e.g., "Lake Crest") processor. In at least one embodiment, Figure 6B The inference and / or training logic 615 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware or other hardware (such as field programmable gate array (“FPGA”)).

[0123] In at least one embodiment, Figure 6C An inference and / or training logic 615 according to at least one of the various embodiments is shown, which may include, but is not limited to, hardware logic, wherein computational resources are dedicated or otherwise uniquely used in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, Figure 6C The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (ASIC), such as those from Google. Processing unit, from Graphcore TM Inference processing unit (IPU) or from Intel Corp. (e.g., "Lake Crest") processor. In at least one embodiment, Figure 6CThe inference and / or training logic 615 illustrated in FIG. 12 can be used in combination with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as field programmable gate arrays (FPGAs). In at least one embodiment, the inference and / or training logic 615 includes, without limitation, code and / or data storage 601 and code and / or data storage 605, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. In Figure 6C In at least one embodiment, each of code and / or data storage 601 and code and / or data storage 605 is associated with a dedicated computing resource, such as computing hardware 602 and computing hardware 606, respectively.

[0124] In at least one embodiment, each of code and / or data storage 601 and 605 and corresponding computing hardware 602 and 606 respectively correspond to different layers of a neural network, such that activations resulting from one “storage / computing pair 601 / 602” of code and / or data storage 601 and computing hardware 602 are provided as input to the next “storage / computing pair 605 / 606” of code and / or data storage 605 and computing hardware 606 in order to reflect the conceptual organization of a neural network. In at least one embodiment, each storage / computing pair 601 / 602 and 605 / 606 can correspond to more than one neural network layer. In at least one embodiment, additional storage / computing pairs (not shown) can be included in inference and / or training logic 615 after or in parallel with storage computing pairs 601 / 602 and 605 / 606.

[0125] Computer system

[0126] Figure 7A A block diagram of an exemplary computer system 700A is shown according to at least one embodiment, which can be a system with interconnecting devices and components, a system-on-a-chip (SOC), or some combination thereof, that can include a processor that can include execution units to execute instructions to support and / or implement the intelligent control of refrigerant-assisted cooling described herein. In at least one embodiment, a computer system 700A according to the present disclosure, such as an embodiment herein, can include, without limitation, components such as a processor 702 to execute an algorithm that can include execution units to execute instructions to support and / or implement processes described herein. In at least one embodiment, a computer system 700A can include a processor, such as a processor from the Intel Pentium, Pentium® XScale™, and / or StrongARM™, Celeron TM or Nervana TM The system can use a microprocessor, but other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors can also be used. In at least one embodiment, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces can also be used, the computer system 700B can execute a version of the Windows operating system available from Microsoft Corporation in Redmond, Washington.

[0127] In at least one embodiment, the exemplary computer system 700A may incorporate components 110-116 (from... Figure 1 One or more of these can be used to support processing aspects for intelligent control of refrigerant-assisted cooling systems. At least for this reason, in one embodiment, Figure 7A A system including interconnected hardware devices or "chips" is shown, while in other embodiments, Figure 7A An exemplary system-on-a-chip (SoC) can be illustrated. In at least one embodiment, Figure 7A The devices shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of the computer system 700B are interconnected using a Compute Fast Link (CXL) interconnect. Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments, such as those previously described. Figure 6A The discussion below is based on -C. Figure 6A -C provides details about the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be in the system Figure 7A The operation is used to infer or predict the operation based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0128] The embodiments can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip (SoC), a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system that can execute one or more instructions according to at least one embodiment.

[0129] In at least one embodiment, computer system 700A can include, without limitation, a processor 702 that can include, without limitation, one or more execution units 708 to perform, e.g., machine learning model training and / or inferencing, in accordance with techniques described herein. In at least one embodiment, computer system 700A is a single processor desktop or server system, but in another embodiment, computer system 700A can be a multiprocessor system. In at least one embodiment, processor 702 can include, without limitation, a complex instruction set computer (“CISC”) microprocessor, a reduced instruction set computing (“RISC”) microprocessor, a very long instruction word (“VLIW”) microprocessor, a processor implementing a combo of instruction sets, or any other processor device, such as a digital signal processor. In at least one embodiment, processor 702 can be coupled to a processor bus 710 that can transmit data signals between processor 702 and other components in computer system 700A.

[0130] In at least one embodiment, processor 702 can include, without limitation, a level 1 (“L1”) internal cache memory (“cache”) 704. In at least one embodiment, processor 702 can have a single internal cache or multiple levels of internal caches. In at least one embodiment, cache memory can reside in the processor 702’s external. Other embodiments can also include a combination of internal and external caches depending on the specific implementation and requirements. In at least one embodiment, register file 706 can store different types of data, including, without limitation, integer registers, floating point registers, status registers, and instruction pointer registers, in various registers.

[0131] In at least one embodiment, execution unit(s) 708, including, without limitation, logic to perform integer and floating point operations, also reside in processor 702. In at least one embodiment, processor 702 can also include a microcode (“ucode”) read-only memory (“ROM”) that stores microcode for certain macro instructions. In at least one embodiment, execution unit(s) 708 can include logic to handle a packed instruction set 709. In at least one embodiment, by including the packed instruction set 709 in a general-purpose processor’s instruction set, along with associated circuitry to execute the instructions, a wide variety of multimedia applications can be executed using the general-purpose processor 702. In one or more embodiments, by using the full width of a processor’s data bus most often used by general computing applications to fetch data, many multimedia applications can execute efficiently on general-purpose microprocessors. In one or more embodiments, using a processor’s data bus to fetch data on packed data elements can speed and more efficiently execute many multimedia applications that otherwise are executed on digital signal processors (DSPs) that have separate data and instruction buses.

[0132] In at least one embodiment, execution unit 708 can also include, without limitation, integer and floating point logic to perform integer and floating point operations respectively. In at least one embodiment, execution unit 708 can also include, without limitation, logic to execute a media BLAS (Basic Linear Algebra with Streaming SIMD Extensions) for media operations and calculations. In at least one embodiment, execution unit 708 can also include, without limitation, logic to execute AI (Artificial Intelligence) functions and operations.

[0133] In at least one embodiment, system logic chip can be coupled to processor bus 710 and memory 720. In at least one embodiment, system logic chip can include, without limitation, a memory controller hub (“MCH”) 716, and processor 702 can communicate with MCH 716 via processor bus 710. In at least one embodiment, MCH 716 can provide a high bandwidth memory path 718 to memory 720 for instruction and data storage and for storage of graphics commands, data, and textures. In at least one embodiment, MCH 716 can direct data signals between processor 702, memory 720, and other components in computer system 700A, and can

[0134] In at least one embodiment, computer system 700A can use system I / O 722 as a proprietary hub interface bus to couple MCH 716 to I / O controller hub (“ICH”) 730. In at least one embodiment, ICH 730 can provide direct connections to some I / O devices and a high-speed I / O bus to connect to other devices. In at least one embodiment, the high-speed I / O bus can include, without limitation, a PCI Express bus or a revved version thereof. Examples can include, without limitation, audio controllers 729, firmware hub (“flash BIOS”) 728, wireless transceivers 726, data storage 724, legacy I / O controllers 723 containing user input and keyboard interfaces, serial expansion ports 727, such as Universal Serial Bus (“USB”) ports, and network controllers 734. Data storage 724 can include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0135] Figure 7B is a block diagram illustrating an electronic device 700B for utilizing processor 710 to support and / or implement intelligent control of a refrigerant-assisted cooling system, in accordance with at least one embodiment. In at least one embodiment, electronic device 700B can be, for example and without limitation, a notebook, a tower server, a rack server, a blade server, a laptop, a desktop, a tablet, a mobile device, a phone, an embedded computer, or any other suitable electronic device. In at least one embodiment, example electronic device 700B can incorporate one or more of the components that support processing aspects of a refrigerant-assisted cooling system.

[0136] In at least one embodiment, system 700B can include, without limitation, processor 710 communicatively coupled to any suitable number or kind of components, peripherals, modules, or devices. In at least one embodiment, processor 710 is coupled using a bus or interface, such as an I2C bus, a System Management Bus (“SMBus”), a Low Pin Count (LPC) bus, a Serial Peripheral Interface (“SPI”), a High Definition Audio (“HDA”) bus, a Serial Advanced Technology Attachment (“SATA”) bus, a Universal Serial Bus (“USB”) (versions 1, 2, 3), or a Universal Asynchronous Receiver / Transmitter (“UART”) bus. In at least one embodiment, processor 710 can be a single core Figure 7B A system is shown that includes interconnected hardware devices or “chips,” while in other embodiments, Figure 7B An example System on a Chip (“SoC”) can be shown. In at least one embodiment, Figure 7B Devices shown in FIG. 7A can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment,Figure 7B One or more components interconnect using a compute express link (CXL) interconnect.

[0137] In at least one embodiment, Figure 7B may include a display 724, a touch screen 725, a touch pad 730, a near field communication unit (“NFC”) 745, a sensor hub 740, a thermal sensor 746, an express chipset (“EC”) 735, a trusted platform module (“TPM”) 738, a BIOS / firmware / flash memory (“BIOS, FW Flash”) 722, a DSP 760, a drive 720 (e.g., a solid state disk (“SSD”) or a hard disk drive (“HDD”)), a wireless local area network unit (“WLAN”) 750, a Bluetooth unit 752, a wireless wide area network unit (“WWAN”) 756, a global positioning system (GPS) unit 755, a camera (“USB 3.0 camera”) 754 (e.g., a USB 3.0 camera), and / or a low power double data rate (“LPDDR”) memory unit (“LPDDR3”) 715 implemented in, e.g., LPDDR3 standard. These components can each be implemented in any suitable manner.

[0138] In at least one embodiment, other components can be communicatively coupled to processor 710 by components as follows. In at least one embodiment, an accelerometer 741, an ambient light sensor (“ALS”) 742, a compass 743, and a gyroscope 744 can be communicatively coupled to sensor hub 740. In at least one embodiment, a thermal sensor 739, a fan 737, a keyboard 746, and a touch pad 730 can be communicatively coupled to EC 735. In at least one embodiment, a speaker 763, a headphone 764, and a microphone (“mic”) 765 can be communicatively coupled to an audio unit (“audio codec and class D amplifier”) 762, which can in turn be communicatively coupled to DSP 760. In at least one embodiment, audio unit 764 can include, without limitation, an audio coder / decoder (“codec”) and a class D amplifier. In at least one embodiment, a SIM card (“SIM”) 757 can be communicatively coupled to WWAN unit 756. In at least one embodiment, components such as WLAN unit 750 and Bluetooth unit 752, and WWAN unit 756 can be implemented as a next generation form factor (NGFF).

[0139] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, inference and / or training logic 615 can be used in system FIG. 6A and / or any other system Figure 7BIn use, for inferencing or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures, or neural network use cases herein.

[0140] Figure 7C A computer system 700C that is used to support and / or implement intelligent control of refrigerant-assisted cooling systems described herein is shown in accordance with at least one embodiment. In at least one embodiment, the computer system 700C includes, without limitation, a computer 771 and a USB stick 770. In at least one embodiment, the computer 771 can include, without limitation, any number and type of processor (not shown) and memory (not shown). In at least one embodiment, the computer 771 includes, without limitation, a server, a cloud instance, a laptop computer, and a desktop computer.

[0141] In at least one embodiment, the USB stick 770 includes, without limitation, a processing unit 772, a USB interface 774, and USB interface logic 773. In at least one embodiment, the processing unit 772 can be any instruction execution system, apparatus, or device capable of executing instructions. In at least one embodiment, the processing unit 772 can include, without limitation, any number and type of processing core (not shown). In at least one embodiment, the processing unit or core 772 includes an application-specific integrated circuit (“ASIC”) that is optimized to perform any number and type of operations associated with machine learning. For example, in at least one embodiment, the processing core 772 is a tensor processing unit (“TPC”) that is optimized to perform machine learning inferencing operations. In at least one embodiment, the processing core 772 is a vision processing unit (“VPU”) that is optimized to perform machine vision and machine learning inferencing operations.

[0142] In at least one embodiment, the USB interface 774 can be any type of USB connector or USB receptacle. For example, in at least one embodiment, the USB interface 774 is a USB 3.0 Type-C receptacle for data and power. In at least one embodiment, the USB interface 774 is a USB 3.0 Type-A connector. In at least one embodiment, the USB interface logic 773 can include any number and type of logic that enables the processing unit 772 to interface with a device (e.g., the computer 771) via the USB connector 774.

[0143] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 615 can be used in Figure 6B and Figure 6C are described below in conjunction with FIGS. 6A and / or 6B. In at least one embodiment, inference and / or training logic 615 can be used in Figure 6B and Figure 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 can be used for Figure 7C In the system, the operation is inferred or predicted based at least in part on the weight parameters calculated using the neural network training operation, neural network function and / or architecture, or neural network use case described herein.

[0144] Figure 8 Further exemplary computer system 800 for implementing various processes and methods throughout the refrigerant-assisted cooling system described herein, according to at least one embodiment, is illustrated. In at least one embodiment, computer system 800 includes, but is not limited to, at least one central processing unit (“CPU”) 802 connected to a communication bus 810 implemented using any suitable protocol, such as PCI (“Peripheral Device Interconnect”), Peripheral Component Interconnect Express (“PCI-Express”), AGP (“Accelerated Graphics Port”), HyperTransport, or any other bus or point-to-point communication protocol. In at least one embodiment, computer system 800 includes, but is not limited to, main memory 804 and control logic (e.g., implemented in hardware, software, or a combination thereof), and data may be stored in main memory 804 in the form of random access memory (“RAM”). In at least one embodiment, a network interface subsystem (“Network Interface”) 822 provides an interface to other computing devices and networks for receiving data from computer system 800 and transferring data to other systems.

[0145] In at least one embodiment, the computer system 800 includes, but is not limited to, an input device 808, a parallel processing system 812, and a display device 806, which may be implemented using a cathode ray tube (“CRT”), a liquid crystal display (“LCD”), a light-emitting diode (“LED”), a plasma display, or other suitable display technologies. In at least one embodiment, user input is received from the input device 808 (such as a keyboard, mouse, touchpad, microphone, and more). In at least one embodiment, each of the above-described modules may reside on a single semiconductor platform to form the processing system.

[0146] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments, such as those previously mentioned. Figure 6A The discussion below is based on -C. Figure 6A -C provides details about the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be in the system Figure 8The inference and / or training logic 615 can be used in systems Figure 8 to perform inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or the neural network use cases herein. In at least one embodiment, inference and / or training logic 615 can be used in system

[0147] Figure 9A An exemplary architecture is shown in which a plurality of GPUs 910-913 are communicatively coupled to a plurality of multi-core processors 940-943 over high-speed links 905-906 (e.g., buses, point-to-point interconnects, etc.). In one embodiment, high-speed links 940-943 support a communication throughput of 4GB / s, 30GB / s, 80GB / s or higher. Various interconnect protocols can be used including, but not limited to, PCIe 4.0 or 5.0 and NVLink 2.0.

[0148] Further, in one embodiment, two or more of GPUs 910-913 are interconnected over high-speed links 929-930, which can be implemented using the same or different protocol / link as used by high-speed links 940-943. Similarly, two or more of multi-core processors 905-906 can be connected by interconnect 928, which can be a symmetric multi-processor (SMP) bus that runs at 20GB / s, 30GB / s, 120GB / s or higher. Alternatively, all communication between various system components illustrated in FIG. 9 can be accomplished by Figure 9A using the same protocol / link (e.g., over a common interconnect structure).

[0149] In one embodiment, each multi-core processor 905-906 is communicatively coupled to processor memories 901-902 via memory interconnects 926-927, respectively, and each GPU 910-913 is communicatively coupled to GPU memories 920-923 through GPU memory interconnects 950-953, respectively. Memory interconnects 926-927 and 950-953 can utilize the same or different memory access technologies. As an example and not by way of limitation, processor memories 901-902 and GPU memories 920-923 can be volatile memories such as dynamic random access memory (DRAM) (including stacked DRAM), graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6), or high bandwidth memory (HBM), and / or can be non-volatile memories such as 3D XPoint or Nano-Ram. In one embodiment, certain portions of processor memories 901-902 can be volatile memories while another portion can be non-volatile memories (e.g., using a two-level memory (2LM) hierarchy).

[0150] As follows, although various processors 905-906 and GPUs 910-913 can be physically coupled to particular memories 901-902, 920-923, respectively, a unified memory architecture can be implemented in which a virtual system address space (also referred to as an “effective address” space) is distributed among the various physical memories. In at least one embodiment, processor memories 901-902 can each include 64 GB of system memory address space, and GPU memories 920-923 can each include 32 GB of system memory address space (resulting in a total of 256 GB of addressable memory size in this example).

[0151] As discussed elsewhere in this disclosure, at least flow rate and associated temperature can be established for an intelligent learning system such as a neural network system. Since the first level represents prior data, it also represents a smaller subset of data that can be used to improve the system by retraining the system. Testing and training can be performed in parallel using multiple processing units so that the intelligent learning system is robust. For example, a neural network system can be implemented using the architecture shown. When the intelligent learning system achieves convergence, the data in the data points and the number of data points that caused the convergence are recorded. The data and data points can be used to control reference, for example, to the refrigerant assisted cooling system discussed. Figure 9A Figures 2A-5

[0152] Figure 9B ​​Additional details are shown for an interconnect between the multi-core processor 907 and the graphics acceleration module 946, in accordance with one example embodiment. The graphics acceleration module 946 can include one or more GPU chips integrated on a line card that is coupled with the processor 907 via a high-speed link 940. Alternatively, the graphics acceleration module 946 can be integrated on the same package or chip as the processor 907.

[0153] In at least one embodiment, the processor 907 shown includes multiple cores 960A-960D, each with a translation lookaside buffer 961 A-961 D and one or more caches 962A-962D. In at least one embodiment, the cores 960A-960D can include various other components not shown, for executing instructions and processing data. The caches 962A-962D can include level 1 (LI) and level 2 (L2) caches. In addition, one or more shared caches 956 can be included in the caches 962A-962D and shared by the groups of cores 960A-960D. In at least one embodiment, one embodiment of the processor 907 includes 24 cores, each with its own LI cache, twelve shared L2 caches, and twelve shared L3 caches. In this embodiment, two adjacent cores share one or more L2 and L3 caches. The processor 907 and graphics acceleration module 946 are connected with system memory 914, which can include processor memory 901-902 in the memory hub 905. Figure 9A

[0154] Consistency for data and instructions stored in the individual caches 962A-962D, 956 and system memory 914 is maintained by inter-core communication over the coherence bus 964. In at least one embodiment, each cache can have cache coherency logic / circuitry associated with it to communicate over the coherence bus 964 in response to detecting a read or write to a particular cache line. In one implementation, a cache snoop protocol is implemented over the coherence bus 964 to snoop cache accesses.

[0155] In at least one embodiment, the agent circuit 925 communicatively couples the graphics acceleration module 946 to the coherence bus 964, allowing the graphics acceleration module 946 to participate in the cache coherency protocol as a peer to the cores 960A-960D. In particular, in at least one embodiment, the interface 935 provides connectivity over the high-speed link 940 (e.g., a PCIe bus, NVLink, etc.) to the agent circuit 925, and the interface 937 connects the graphics acceleration module 946 to the link 940.

[0156] ​In one implementation, accelerator integration circuit 936 provides caching, memory access, context management, and interrupt management services on behalf of graphics processing engines 931, 932, N of graphics processing module 906. Graphics processing engines 931, 932, N can each comprise a separate graphics processing unit (GPU). In at least one embodiment, graphics processing engines 931, 932, N alternatively can comprise different types of graphics processing engines within a GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, graphics acceleration module 946 can be a GPU with a plurality of graphics processing engines 931-932, N, or graphics processing engines 931-932, N can be individual GPUs integrated on a common package, line card, or chip. Figure 9B The above determination of the reconstruction parameters and reconstruction algorithm can be performed in GPU 931-N.

[0157] In one embodiment, accelerator integration circuit 936 includes a memory management unit (MMU) 939 to provide memory management for graphics processing engines 931-932, N. In at least one embodiment, MMU 939 includes memory partitions to enable process isolation for each user / process context. In at least one embodiment, one or more operating systems run on system core 950, and each has private memory. In at least one embodiment, memory management unit 939 also includes a translation lookaside buffer (TLB) to cache page table translations, and provides exceptions handling

[0158] A set of registers 945 store context data for threads executed by graphics processing engines 931-932, N, and context management circuit 948 manages thread contexts. In at least one embodiment, context management circuit 948 can perform save and restore operations to save and restore context of individual threads during context switches (e.g., where a first thread is saved and a second thread is stored so that the second thread can be executed by a graphics processing engine). In at least one embodiment, context management circuit 948 can store current register values to a designated area in memory (e.g., identified by a context pointer) upon a context switch. The register values can then be restored when returning to a context. In one embodiment, interrupt management circuit 947 receives and processes interrupts received from system devices.

[0159] In one implementation, MMU 939 translates virtual / effective addresses from graphics processing engines 931 to real / physical addresses in system memory 914. One embodiment of accelerator integration circuit 936 supports multiple (e.g., 4, 8, 16) graphics processor modules 946 and / or other accelerator devices. Graphics processor module 946 can be dedicated to a single application executing on processor 907, or can be shared between multiple applications. In one embodiment, a virtualized graphics execution environment is presented in which resources of graphics processing engines 931-932, N are shared between multiple applications or virtual machines (VMs). In at least one embodiment, resources can be subdivided into “slices” that are allocated to different VMs and / or applications based on processing requirements and priority associated with VMs and / or applications.

[0160] In at least one embodiment, accelerator integration circuit 936 performs as a bridge to the system for graphics processor 946, and provides address translation and system memory cache management services. In addition, accelerator integration circuit 936 can provide virtualization facilities to manage virtualization of graphics processing engines 931-932, N, interrupts, and memory management.

[0161] Since hardware resources of graphics processing engines 931-932, N are explicitly mapped to real address space seen by the host processor 907, any host processor can directly address these resources using effective address values. In at least one embodiment, a function of accelerator integration circuit 936 is to physically separate graphics processing engines 931-932, N so that they appear as independent units to the system.

[0162] In at least one embodiment, one or more graphics memory 933-934, M is coupled to each graphics processing engines 931-932, N, respectively. Graphics memory 933-934, M stores instructions and data for processing by each of the graphics processing engines 931-932, N. Graphics memory 933-934, M can be a volatile memory, such as DRAM (including stacked DRAM), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and / or can be a non-volatile memory, such as 3D XPoint or Nano-Ram.

[0163] In one embodiment, to reduce data traffic on link 940, biasing techniques are used to ensure that data stored in graphics memory 933-934, M is that most frequently used by graphics processing engines 931-932, N and that data that can not be used by cores 960A-960D. Similarly, biasing mechanisms attempt to keep data needed by cores (and can not be graphics processing engines 931-932, N) in caches 962A-962D, core 956, and system memory 914.

[0164] Figure 9C Another exemplary embodiment is shown in which accelerator integration circuit 936 is integrated in processor 907 for implementing and / or supporting intelligent control of a refrigerant-assisted cooling system in accordance with at least one embodiment disclosed herein. In at least this embodiment, graphics processing engines 931-932, N communicate directly with accelerator integration circuit 936 via interface 937 and interface 935 (again, any form of bus or interface protocol can be utilized) over high-speed link 940. Accelerator integration circuit 936 can perform a similar operation to that described with respect to Figure 9B described operations. But due to its close proximity to coherence bus 964 and caches 962A-962D, 956, can have higher throughput. At least one embodiment supports different programming models including a dedicated process programming model (no graphics acceleration module virtualization) and a shared programming model (with virtualization), which can include programming models controlled by accelerator integration circuit 936 and programming models controlled by graphics acceleration module 946.

[0165] In at least one embodiment, graphics processing engines 931-932, N are dedicated to a single application or process under a single operating system. In at least one embodiment, a single application can funnel other application requests to graphics processing engines 931-932, N, providing virtualization within a VM / partition.

[0166] In at least one embodiment, graphics processing engines 931-932, N can be shared by multiple VM / application partitions. In at least one embodiment, a shared model can use a hypervisor to virtualize graphics processing engines 931-932, N to allow access by each operating system. For a single-partition system without a hypervisor, an operating system owns graphics processing engines 931-932, N. In at least one embodiment, an operating system can virtualize graphics processing engines 931-932, N to provide access to each process or application.

[0167] In at least one embodiment, graphics acceleration module 946 or individual graphics processing engines 931-932, N use a process handle to select a process element. In at least one embodiment, process elements are stored in system memory 914 and are addressable using effective-to-real address translation techniques herein. In at least one embodiment, a process handle can be an implementation-specific value provided to a host process when it registers its context with graphics processing engines 931-932, N (i.e., calls system software to add a process element to a process element linked list). In at least one embodiment, lower 16 bits of a process handle can be an offset into a process element linked list for a process element.

[0168] Figure 9D An exemplary accelerator integration slice 990 to implement and / or support intelligent control of a refrigerant-assisted cooling system is shown in accordance with at least one embodiment disclosed herein. As used herein, a “slice” comprises a specified portion of processing resources of accelerator integration circuit 936. An application is an effective address space 982 in system memory 914 that stores a process element 983. In at least one embodiment, a process element 983 is stored in response to a GPU call 981 from an application 980 executing on processor(s) 907. A process element 983 contains process state for a respective application 980. A work descriptor (WD) 984 contained in process element 983 can be a single job requested by an application or can contain a pointer to a queue of jobs. In at least one embodiment, WD 984 is a pointer to a job request queue in an application’s address space 982.

[0169] Graphics acceleration module 946 and / or individual graphics processing engines 931-932, N can be shared by all or a subset of processes in a system. In at least one embodiment, can include infrastructure for setting up process state and sending a WD 984 to graphics acceleration module 946 to start a job in a virtualized environment.

[0170] In at least one embodiment, a dedicated process programming model is implementation specific. In this model, a single process owns a graphics acceleration module 946 or individual graphics processing engines 931. When a graphics acceleration module 946 is owned by a single process, a hypervisor initializes accelerator integration circuit 936 for the owned partition, and an operating system initializes accelerator integration circuit 936 for the owned process when a graphics acceleration module 946 is assigned.

[0171] In operation, a WD fetch unit 991 in accelerator integration slice 990 fetches a next WD 984, which includes an indication of work to be completed by one or more graphics processing engines of graphics acceleration module 946. Data from WD 984 can be stored in registers 945 and used by MMU 939, interrupt management circuit 947, and / or context management circuit 948, as shown. In at least one embodiment, one embodiment of MMU 939 includes segment / page walk circuitry for accessing segment / page tables 986 within OS virtual address space 985. Interrupt management circuit 947 can handle interrupt events 992 received from graphics acceleration module 946. In at least one embodiment, effective addresses 993 generated by graphics processing engines 931-932, N when performing graphics operations are translated to real addresses by MMU 939.

[0172] In one embodiment, the same set of registers 945 is replicated for each graphics processing engine 931-932, N and / or graphics acceleration module 946, and the same set of registers 945 can be initialized by a hypervisor or operating system. Each of these replicated registers can be included in accelerator integration slice 990. Exemplary registers that can be initialized by a hypervisor are shown in Table 1.

[0173] Table 1 - Hypervisor-Initialized Registers

[0174] 1 Slice Control Register 2 Real Address (RA) Supervisor Region Pointer 3 Privilege Mask Override Register 4 Interrupt Vector Table Entry Offset 5 Interrupt Vector Table Entry Limit 6 Status Register 7 Logical Partition Identification 8 Real Address (RA) Hypervisor Accelerator Utilization Record Pointer 9 Storage Descriptor Register

[0175] Exemplary registers that can be initialized by an operating system are shown in Table 2.

[0176] Table 2 - Operating System-Initialized Registers

[0177] 1 Process and Thread Identification 2 Effective Address (EA) Context Save / Restore Pointer 3 Virtual Address (VA) Accelerator Utilization Record Pointer 4 Virtual Address (VA) Storage Segment Table Pointer 5 Privilege Mask 6 Work Descriptor

[0178] In at least one embodiment, each WD 984 is specific to a particular graphics acceleration module 946 and / or graphics processing engine 931-932, N. It contains all information needed by the graphics processing engines 931-932, N to complete work, or it can be a pointer to a memory location where an application has set up a command queue of work to be completed.

[0179] Figure 9E Additional details of one exemplary embodiment of the shared model are shown. This embodiment includes a hypervisor real address space 998 in which a list of process elements 999 is stored. The hypervisor real address space 998 is accessible via a hypervisor 996 that virtualizes the graphics acceleration module engine for the operating system 995.

[0180] In at least one embodiment, the shared programming model allows all processes or a subset of processes from all partitions or a subset of partitions in the system to use the graphics acceleration module 946. There are two programming models in which the graphics acceleration module 946 is shared by multiple processes and partitions, i.e., time-sliced sharing and graphics-directed sharing.

[0181] In this model, the system hypervisor 996 owns the graphics acceleration module 946 and makes its functionality available to all operating systems 995. For the graphics acceleration module 946 to support virtualization through the system hypervisor 996, the graphics acceleration module 946 can be required to: (1) application job requests must be autonomous (i.e., no state needs to be maintained between jobs), or the graphics acceleration module 946 must provide a context save and restore mechanism, (2) the graphics acceleration module 946 guarantees that an application's job request will complete within a specified amount of time, including any translation faults, or the graphics acceleration module 946 provides the ability to preempt job processing, and (3) fairness between graphics acceleration module 946 processes must be ensured when operating in a directed shared programming model.

[0182] In one embodiment, the application 980 needs to make an operating system 995 system call using a graphics acceleration module type, a work descriptor (WD), an authority mask register (AMR) value, and a context save / restore area pointer (CSRP). The graphics acceleration module type describes the target acceleration function for the system call. In at least one embodiment, the graphics acceleration module type can be a system-specific value. In at least one embodiment, the WD is formatted specifically for the graphics acceleration module 946 and can take the form of a graphics acceleration module 946 command, a valid address pointer to a user-defined structure, a valid address pointer to a command queue, or any other data structure describing work to be done by the graphics acceleration module 946. In at least one embodiment, the AMR value is the AMR state for the current process. In at least one embodiment, the value passed to the operating system is similar to how an application program sets the AMR. If the accelerator integration circuit 936 and graphics acceleration module 946 implementation does not support a user authority mask override register (UAMOR), the operating system can apply the current UAMOR value to the AMR value before passing the AMR in the hypervisor call. In at least one embodiment, the hypervisor 996 can apply the current authority mask override register (AMOR) value before placing the AMR in the process element 983. In at least one embodiment, the CSRP is one of the registers 945 containing a valid address of an area in the application’s effective address space 982 for the graphics acceleration module 946 to save and restore context state. This pointer is used in at least one embodiment, if state does not need to be saved between jobs or when a job is preempted. In at least one embodiment, the context save / restore area can be a fixed system memory.

[0183] Upon receiving the system call, the operating system 995 can verify that the application 980 is registered and has been granted authority to use the graphics acceleration module 946. The operating system 995 then invokes the hypervisor 996 using the information shown in Table 3.

[0184] Table 3 - OS to hypervisor call parameters

[0185] 1 Work Descriptor (WD) 2 Privilege Mask Register (AMR) Value (Potentially Masked) 3 Effective Address (EA) Context Save / Restore Region Pointer (CSRP) 4 Process ID (PID) and Optional Thread ID (TID) 5 Virtual Address (VA) Accelerator Utilization Record Pointer (AURP) 6 Virtual Address of Storage Segment Table Pointer (SSTP) 7 Logical Interrupt Service Number (LISN)

[0186] Upon receiving the hypervisor call, the hypervisor 996 verifies that the operating system 995 is registered and has been granted authority to use the graphics acceleration module 946. The hypervisor 996 then places the process element 983 in the process element linked list for the corresponding graphics acceleration module 946 type. The process element can include the information shown in Table 4.

[0187] Table 4 - process element information

[0188]

[0189]

[0190] In at least one embodiment, the management program initializes multiple accelerator integration slice 990 registers 945.

[0191] like Figure 9F As shown, in at least one embodiment, a unified memory is used, which can be addressed via a common virtual memory address space for accessing physical processor memories 901-902 and GPU memories 920-923. In this implementation, operations performed on GPUs 910-913 utilize the same virtual / effective memory address space to access processor memories 901-902, and vice versa, thereby simplifying programmability. In one embodiment, a first portion of the virtual / effective address space is allocated to processor memory 901, a second portion to second processor memory 902, a third portion to GPU memory 920, and so on. In at least one embodiment, the entire virtual / effective memory space (sometimes referred to as the effective address space) is thus distributed across each of processor memories 901-902 and GPU memories 920-923, thereby allowing any processor or GPU to access that memory using a virtual address mapped to any physical memory.

[0192] In one embodiment, the bias / coherence management circuitry 994A-994E within one or more MMUs 939A-939E ensures cache coherence between the caches of one or more host processors (e.g., 905) and the GPUs 910-913, and implements biasing techniques that indicate the physical memory in which certain types of data should be stored. While in Figure 9F Several instances of bias / coherence management circuitry 994A-994E are shown, but bias / coherence circuitry can be implemented within the MMU of one or more host processors 905 and / or within accelerator integrated circuits 936.

[0193] One embodiment allows GPU-attached memory 920-923 to be mapped as part of system memory and accessed using shared virtual memory (SVM) technology, but without suffering the performance penalties associated with full system cache coherency. In at least one embodiment, the ability to access GPU-attached memory 920-923 as system memory without the heavy cache coherency overhead provides a favorable operating environment for GPU offload. This arrangement allows host processor 905 software to set operands and access computation results without the overhead of traditional I / O DMA data copies. Such traditional copies include driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, which are all less efficient than simple memory accesses. In at least one embodiment, the ability to access GPU-attached memory 920-923 without cache coherency overhead can be critical to the execution time of offloaded computations. For example, in cases with a large amount of streaming write memory traffic, cache coherency overhead can significantly reduce the effective write bandwidth seen by GPU 910-913. In at least one embodiment, the efficiency of operand setup, the efficiency of result access, and the efficiency of GPU computation can all play a role in determining the effectiveness of GPU offload.

[0194] In at least one embodiment, the selection of GPU bias and host processor bias is driven by a bias tracker data structure. For example, a bias table can be used, which can be a page-granular structure (e.g., controlled at the granularity of a memory page) that includes a per-GPU-attached memory page 1 or 2 bits. In at least one embodiment, with or without a bias cache in GPU 910-913 (e.g., to cache frequently / recently used entries of the bias table), the bias table can be implemented in the stolen memory range of one or more GPU-attached memories 920-923. Alternatively, the entire bias table can be maintained within the GPU.

[0195] In at least one embodiment, prior to actually accessing GPU memory, the bias table entry associated with each access to GPU-attached memory 920-923 is accessed, causing the following operations. Local requests from GPUs 910-913 that find their pages in GPU bias are forwarded directly to corresponding GPU memory 920-923. Local requests from GPUs that find their pages in host bias are forwarded to processor 905 (e.g., over a high-speed link as above). In one embodiment, requests from processor 905 that find requested pages in host processor bias complete requests similar to normal memory reads. Alternatively, requests that point to GPU-biased pages can be forwarded to GPUs 910-913. In at least one embodiment, if a page is not currently in use by a GPU, the GPU can then migrate the page to host processor bias. In at least one embodiment, the bias state of a page can be changed by software-based mechanisms, hardware-assisted software-based mechanisms, or in limited cases purely hardware-based mechanisms.

[0196] One mechanism for changing bias state employs an API call (e.g., OpenCL), which in turn invokes a device driver of a GPU, which in turn sends a message (or causes a command descriptor to be enqueued) to the GPU, directing the GPU to change bias state, and in certain migrations to perform a cache flush operation in the host. In at least one embodiment, the cache flush operation is used for migrations from host processor 905 bias to GPU bias, but not for the reverse.

[0197] In one embodiment, cache coherency is maintained by temporarily rendering GPU-biased pages that cannot be cached by host processor 905. To access these pages, processor 905 can request access from GPU 910, which can or can not grant access immediately. Thus, to reduce communication between processor 905 and GPU 910, it is beneficial to ensure that GPU-biased pages are pages that are needed by the GPU but not by host processor 905, and vice versa.

[0198] Inference and / or training logic 615 are used to perform one or more embodiments. Conjunctive use of the inference and / or training logic 615 in the following may Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided with respect to FIG. 6.

[0199] Figure 10AExemplary integrated circuits and associated graphics processors in accordance with various embodiments herein are shown, which can be fabricated using one or more IP cores to support or implement a refrigerant- assisted cooling system. Other logic and circuitry can be included in addition to what is shown, including additional graphics processors / cores, peripheral interface controllers or general-purpose processor cores, in at least one embodiment.

[0200] Figure 10A is a block diagram illustrating an exemplary system on a chip integrated circuit 1000A that can be fabricated using one or more IP cores, in accordance with at least one embodiment. In at least one embodiment, integrated circuit 1000A includes one or more application processor(s) 1005 (e.g., CPUs), at least one graphics processor 1010, and can additionally include an image processor 1015 and / or a video processor 1020, any of which can be a modular IP core. In at least one embodiment, integrated circuit 1000A includes peripheral or bus logic including a USB controller 1025, a UART controller 1030, an SPI / SDIO controller 1035, and an I2S / I2C controller 1040. In at least one embodiment, integrated circuit 1000A can include a display device 1045 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1050 and a mobile industry processor interface (MIPI) display interface 1055. In at least one embodiment, storage can be provided by a flash memory subsystem 1060, including flash memory and a flash memory controller. Memory interface can be provided via a memory controller 1065 for access to SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits also include an embedded security engine 1070. 2 S / I 2 Ccontroller 1040. In at least one embodiment, integrated circuit 1000A can include a display device 1045 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1050 and a mobile industry processor interface (MIPI) display interface 1055. In at least one embodiment, storage can be provided by a flash memory subsystem 1060, including flash memory and a flash memory controller. Memory interface can be provided via a memory controller 1065 for access to SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits also include an embedded security engine 1070.

[0201] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 615 can be used in Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6A and / or 6B. In at least one embodiment, inference and / or training logic 615 can be used in integrated circuit 1000A for inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0202] Figures 10B-10CExemplary integrated circuits and associated graphics processors in accordance with various embodiments herein are shown, which can be fabricated using one or more IP cores, to support and / or implement a refrigerant-assisted cooling system. Additional logic and circuitry can be included in at least one embodiment, including additional graphics processors / cores, peripheral interface controllers or general-purpose processor cores.

[0203] Figures 10B-10C is a block diagram illustrating an exemplary graphics processor for use within a SoC in accordance with embodiments described herein to support and / or implement a refrigerant-assisted cooling system. In one example, the graphics processor can be used for intelligent control of a refrigerant-assisted cooling system, as existing math engines have the ability to process faster with multi-level neural networks. Figure 10B An exemplary graphics processor 1010 of a system on a chip integrated circuit in accordance with at least one embodiment is shown, which can be fabricated using one or more IP cores. Figure 10C Another exemplary graphics processor 1040 of a system on a chip integrated circuit in accordance with at least one embodiment is shown, which can be fabricated using one or more IP cores. In at least one embodiment, Figure 10B The graphics processor 1010 of is a low-power graphics processor core. In at least one embodiment, Figure 10C The graphics processor 1040 of is a higher performance graphics processor core. In at least one embodiment, each graphics processor 1010, 1040 can be Figure 10A Variations of the graphics processor 1010 of.

[0204] In at least one embodiment, graphics processor 1010 includes a vertex processor 1005 and one or more fragment processor(s) 1015A-1015N (e.g., 1015A, 1015B, 1015C, 1015D up to 1015N-1 and 1015N). In at least one embodiment, graphics processor 1010 can execute different shader programs via separate logic for vertex processing and / or for fragment / pixel processing. In at least one embodiment, vertex processor 1005 is optimized to execute operations for vertex shader programs, while one or more fragment processor(s) 1015A-1015N may be optimized to execute fragment or pixel shader programs. In at least one embodiment, vertex processor 1005 processes vertex data that describes a vertices of 3D objects in a scene, and generates vertex data that describes the coordinates of the vertices of the mesh that represents a 3D surface for objects in the scene. In at least one embodiment, one or more fragment processor(s) 1015A-1015N process the vertex data generated by vertex processor 1005 and graphics processor 1010 generates a bitmap of an image representing a 2D or 3D surface of objects on a 2D screen. In at least one embodiment, a rasterization stage (not shown) of graphics processor 1010 decompresses the bitmaps generated by one or more fragment processor(s) 1015A-1015N into a 2D bitmap that is displayed on a display device.

[0205] In at least one embodiment, graphics processor 1010 additionally includes one or more memory management units (MMUs) 1020A-1020B, one or more caches 1025A-1025B, and one or more circuit interconnects 1030A-1030B. In at least one embodiment, one or more MMUs 1020A-1020B provide for virtual to physical address mapping for graphics processor 1010, including for vertex processor 1005 and / or fragment processor(s) 1015A-1015N, which can reference vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in one or more caches 1025A-1025B. In at least one embodiment, one or more MMUs 1020A-1020B can be synchronized with one or more MMUs within Figure 10A one or more application processor(s) 1005, image processors 1015, and / or video processors 1020 within a system, such that each processor 1005-1020 can participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 1030A-1030B enable graphics processor 1010 to interface with other IP cores within a SoC, via an internal bus.

[0206] In at least one embodiment, graphics processor 1040 includes Figure 10AThe graphics processor 1010 includes one or more MMUs 1020A-1020B, caches 1025A-1025B, and circuit interconnects 1030A-1030B. In at least one embodiment, the graphics processor 1040 includes one or more shader cores 1055A-1055N (e.g., 1055A, 1055B, 1055C, 1055D, 1055E, 1055F to 1055N-1 and 1055N), such as Figure 10B As shown, it provides a unified shader core architecture, where a single core or type or core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores can vary. In at least one embodiment, the graphics processor 1040 includes an inter-core task manager 1045, which acts as a thread dispatcher to assign execution threads to one or more shader cores 1055A-1055N and a tile unit 1058 to accelerate tile-based rendering operations, where scene rendering operations are subdivided in image space, for example, to take advantage of local spatial consistency within the scene or optimize the use of internal caches.

[0207] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 may be integrated into an integrated circuit. Figure 10A and / or Figure 10B The method is used to perform inference or prediction operations based at least in part on weight parameters calculated using neural network training operations, neural network functions or architectures, or the neural network use cases described herein.

[0208] Figures 10D-10E Additional exemplary graphics processor logic according to embodiments described herein is illustrated to support and / or implement a refrigerant-assisted cooling system. In at least one embodiment, Figure 10D It shows that it can be included in Figure 10A The graphics core 1000D within the graphics processor 1010, and in at least one embodiment, may be as follows: Figure 10C The unified shader cores shown are 1055A-1055N. Figure 10B A highly parallel general-purpose graphics processing unit (“GPGPU”) 1030 suitable for deployment on a multi-chip module is shown in at least one embodiment.

[0209] In at least one embodiment, the graphics core 1000D may include multiple slices 1001A-1001N or partitions of each core, and the graphics processor may include multiple instances of the graphics core 1000D. In at least one embodiment, slices 1001A-1001N may include supporting logic, including local instruction caches 1004A-1004N, thread schedulers 1006A-1006N, thread dispatchers 1008A-1008N, and a set of registers 1010A-1010N. In at least one embodiment, slices 1001A-1001N may include a set of additional functional units (AFU 1012A-1012N), floating-point units (FPU 1014A-1014N), integer arithmetic logic units (ALU 109A-109N), address calculation units (ACU 1013A-1013N), double-precision floating-point units (DPFPU 1015A-1015N), and matrix processing units (MPU 1017A-1017N).

[0210] In at least one embodiment, the FPU 1014A-1014N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while the DPFPU 1015A-1015N performs double-precision (64-bit) floating-point operations. In at least one embodiment, the ALU 1016A-1016N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision, and can be configured for mixed-precision operations. In at least one embodiment, the MPU 1017A-1017N can also be configured for mixed-precision matrix operations, including half-precision floating-point operations and 8-bit integer operations. In at least one embodiment, the MPU 1017A-1010N can perform various matrix operations to accelerate machine learning application frameworks, including enabling support for accelerated generalized matrix-to-matrix multiplication (GEMM). In at least one embodiment, the AFU 1012A-1012N can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0211] As discussed elsewhere in this disclosure, reasoning and / or training logic 615 (at least in Figure 6B , Figure 6C (Referencing) is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided. In at least one embodiment, inference and / or training logic 615 may be used in the graphics core 1000D to infer or predict operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases herein.

[0212] Figure 11A A block diagram of a computer system 1100A is shown, in accordance with at least one embodiment. In at least one embodiment, computer system 1100A includes a processing subsystem 1101 with one or more processors 1102 and a system memory 1104 communicating via an interconnection path 1105 that can include a memory hub 1105. In at least one embodiment, memory hub 1105 can be a separate component coupled with one or more processors 1102 via communication links 1106 to perform memory access operations; alternatively, memory hub 1105 can be integrated into one or more processors 1102, communication links 1106 used between memory hub 1105 and one or more processors 1102 can be implemented using established computer processing techniques; in at least one embodiment, communication links 1106 are implemented using inter-integrated circuit (I2C), advanced microcontroller bus architecture (AMBA) advanced microcontroler bus architecture (AMBA) advanced high-performance bus (AHB), AMBA advanced

[0213] In at least one embodiment, processing subsystem 1101 includes one or more parallel processor(s) 1112 coupled to memory hub 1105 via a bus or other communication link 1113. In at least one embodiment, communication link 1113 can use any one of a number of standard bus protocols, such as PCI, or be a vendor specific communications interface or communications structure. In at least one embodiment, one or more parallel processor(s) 1112 form a computationally intensive, parallel or vector processing system that can include a number of processing cores, and / or processing clusters, such as a multi-integrated core (MIC) processor. In at least one embodiment, one or more parallel processor(s) 1112 form a graphics processing subsystem that can output pixels to one or more display device(s) 1110A coupled via I / O hub 1107. In at least one embodiment, one or more parallel processor(s) 1112 can also include a display controller and display interface (not shown) to enable a direct connection to one or more display device(s) 1110B.

[0214] In at least one embodiment, system storage 1114 can connect to I / O hub 1107 to provide storage mechanisms for computing system 1100A. In at least one embodiment, I / O switch 1116 can be used to provide an interface mechanism to enable connections between I / O hub 1107 and other components, such as network adapter 1118 and / or wireless network adapter 1119 that can be integrated into one or more platforms, as well as various other devices that can be added via one or more add-in devices 1120. In at least one embodiment, network adapter 1118 can be an Ethernet adapter or another wired network adapter. In at least one embodiment, wireless network adapter 1119 can include one or more of Wi-Fi, Bluetooth, Near Field Communication (NFC), or other network devices including one or more radio bands.

[0215] In at least one embodiment, computing system 1100A can include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, and the like, which can also be connected to I / O hub 1107. In at least one embodiment, interconnection of the components of computing system 1100A can be achieved by using any suitable protocols, including PCI- based protocols (e.g., PCI-Express), or other bus or point-to-point communication interfaces and / or protocols, such as NV-Link high-speed interconnect, or interconnect protocols. Figure 11A

[0216] In at least one embodiment, parallel processor(s) 1112 include circuitry optimized for graphics and video processing, including video output circuitry, and are configured for use in a gaming console, a mobile phone, a personal computer, or other application. In at least one embodiment, parallel processor(s) 1112 incorporate circuitry optimized for general use computational processing, which allows for much more flexibility in applications. In at least one embodiment, components of computing system 1100A can be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, parallel processor(s) 1112, memory hub 1105, processor(s) 1102, and I / O hub 1107 can be integrated together into a system on a chip (SoC) integrated circuit. In at least one embodiment, components of computing system 1100A can be integrated into a single package to form a system in a package (SIP) configuration. In at least one embodiment, at least a portion of components of computing system 1100A can be integrated into a multi-chip module (MCM), which can be interconnected with other multi-chip modules to form a modular computing platform.

[0217] ​Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6 A, 6B, and 6C. Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6A, 6B, and 6C. In at least one embodiment, inference and / or training logic 615 can be used in a system that uses neural network training operations, neural network functions and / or architectures, or neural network use cases described herein. Figure 11A Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6A, 6B, and 6C. In at least one embodiment, inference and / or training logic 615 can be used in a system that uses neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0218] Processor

[0219] Figure 11B A parallel processor 1100B according to at least one embodiment is shown. In at least one embodiment, various components of parallel processor 1100B can be implemented using one or more integrated circuits, which can be programmable integrated circuits, application specific integrated circuits, or field programmable gate arrays. In at least one embodiment, parallel processor 1100B is a graphics processor that includes one or more graphics processing cores. Figure 11B Variations of one or more parallel processors 1112 are shown.

[0220] In at least one embodiment, parallel processor 1100B includes a parallel processing unit 1102. In at least one embodiment, parallel processing unit 1102 includes an I / O unit 1104 that enables communication with other devices, including other instances of parallel processing unit 1102. In at least one embodiment, I / O unit 1104 can be directly connected to other devices. In at least one embodiment, I / O unit 1104 connects with other devices via use of a hub or switch interface, such as memory hub 1105. In at least one embodiment, connections between memory hub 1105 and I / O unit 1104 form a communication link 1113. In at least one embodiment, I / O unit 1104 connects with a host interface 1106 and a memory crossbar 1116, where host interface 1106 receives commands directed to processing operations and memory crossbar 1116 receives commands directed to memory operations.

[0221] In at least one embodiment, when host interface 1106 receives a command buffer via I / O unit 1104, host interface 1106 can direct a work operation to execute those commands to front end 1108. In at least one embodiment, front end 1108 is coupled with scheduler 1110, which is configured to assign commands or other work items to processing cluster array 1112. In at least one embodiment, scheduler 1110 ensures that processing cluster array 1112 is properly configured and in an active state before assigning tasks to processing cluster array 1112. In at least one embodiment, scheduler 1110 is implemented by firmware logic executing on a microcontroller. In at least one embodiment, microcontroller- implemented scheduler 1110 can be configured to perform complex scheduling and work distribution operations with both coarse and fine grain, enabling fast preemption and context switching of threads executing on processing array 1112. In at least one embodiment, host software can prove a workload for scheduling on processing array 1112 over one of more graphics processing doorbells. In at least one embodiment, workload can then be automatically distributed on processing array 1112 by scheduler 1110 logic within microcontroller that includes scheduler 1110.

[0222] In at least one embodiment, processing cluster array 1112 can include up to “N” processing clusters (e.g., cluster 1114A, cluster 1114B, through cluster 1114N). In at least one embodiment, each cluster 1114A-1114N of processing cluster array 1112 can execute a large number of concurrent threads. In at least one embodiment, scheduler 1110 can use various scheduling and / or work distribution algorithms to assign work to clusters 1114A-1114N of processing cluster array 1112, which can vary depending on workload produced by each type of program or computation. In at least one embodiment, scheduling can be handled by scheduler 1110 dynamically, or can be assisted in part by compiler logic during compilation of program logic configured to be executed by processing cluster array 1112. In at least one embodiment, different clusters 1114A-1114N of processing cluster array 1112 can be allocated for processing different types of programs or for performing different types of computations.

[0223] In at least one embodiment, processing cluster array 1112 can be configured to perform various types of parallel processing operations. In at least one embodiment, processing cluster array 1112 is configured to perform general-purpose parallel compute operations. In at least one embodiment, processing cluster array 1112 can include logic to perform processing tasks including filtering of video and / or audio data, performing modeling operations, including physics operations, and performing data transformations.

[0224] In at least one embodiment, processing cluster array 1112 is configured to perform parallel graph processing operations. In at least one embodiment, processing cluster array 1112 can include additional logic to support performance of such graph processing operations, including but not limited to texture mapping logic to perform texture operations, and tessellation logic, and other vertex processing logic. In at least one embodiment, processing cluster array 1112 can be configured to execute shader programs related to graphics processing, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel (or fragment) shaders. In at least one embodiment, parallel processing unit 1102 can transfer data to be processed from a system memory over I / O unit 1104. In at least one embodiment, data being transferred can be stored to on-chip memory (e.g., parallel processor memory 1122) during processing, and then written back to system memory.

[0225] In at least one embodiment, when parallel processing unit 1102 is used to perform graphics processing, scheduler 1110 can be configured to divide the processing workload into approximately equal sized tasks, to better enable distribution of the graphics processing operations across multiple clusters 1114A-1114N of processing cluster array 1112. In at least one embodiment, portions of processing cluster array 1112 can be configured to perform different types of processing. In at least one embodiment, a first portion can be configured to perform vertex shading and topology generation, a second portion can be configured to perform tessellation and geometry shading, and a third portion can be configured to perform pixel shading or other screen space operations, in order to produce a rendered image for display, if needed to simulate valve control of a refrigerant assisted cooling system. In at least one embodiment, intermediate data produced by one or more of clusters 1114A-1114N can be stored in buffers to allow transmission of the intermediate data between clusters 1114A-1114N for further processing.

[0226] In at least one embodiment, processing cluster array 1112 can receive processing tasks to be executed via scheduler 1110, which receives commands defining the processing tasks from front end 1108. In at least one embodiment, a processing task can include an index to data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). In at least one embodiment, scheduler 1110 can be configured to fetch the index corresponding to a task, or can receive the index from front end 1108. In at least one embodiment, front end 1108 can be configured to ensure that processing cluster array 1112 is configured in an effective state before launching a workload specified by an incoming command buffer (e.g., a batch-buffer, a push buffer, etc.).

[0227] In at least one embodiment, each of one or more instances of parallel processing unit 1102 can be coupled with a parallel processor memory 1122. In at least one embodiment, parallel processor memory 1122 can be accessed by the memory crossbar 1116, which can receive memory requests from the processing cluster array 1112 and I / O units 1104. In at least one embodiment, memory crossbar 1116 can access parallel processor memory 1122 via a memory interface 1118. In at least one embodiment, memory interface 1118 can include a number of memory partitions (e.g., memory partition 1120A, memory partition 1120B, through memory partition 1120N), which can each be coupled to a portion (e.g., memory unit) of parallel processor memory 1122. In at least one embodiment, number of memory partitions 1120A-1120N is configured to be equal to the number of memory units, such that first memory partition 1120A has a corresponding first memory unit 1124A, second memory partition 1120B has a corresponding second memory unit 1124B, and nthmemory partition 1120N has a corresponding nthmemory unit 1124N. In at least one embodiment, number of memory partitions 1120A-1120N can not equal the number of memory devices.

[0228] In at least one embodiment, memory units 1124A-1124N can include various types of memory devices including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory units 1124A-1124N can also include 3D stacked memory including, but not limited to, high bandwidth memory (HBM). In at least one embodiment, rendering targets such as frame buffers or texture maps can be stored across memory units 1124A-1124N, allowing partition units 1120A-1120N to write portions of each rendering target in parallel to effectively use available bandwidth of parallel processor memory 1122. In at least one embodiment, local instances of parallel processor memory 1122 can be excluded in favor of a unified memory design that utilizes system memory in combination with local cache memory.

[0229] In at least one embodiment, any of clusters 1114A-1114N of processing cluster array 1112 can process data that is to be written into any of memory units 1124A-1124N within parallel processor memory 1122. In at least one embodiment, memory crossbar 1116 can be configured to transmit outputs of each cluster 1114A-1114N to any partition unit 1120A-1120N or another cluster 1114A-1114N, which can perform other processing operations on the outputs. In at least one embodiment, each cluster 1114A-1114N can communicate with memory interface 1118 through memory crossbar 1116 to read from or write to various external memory devices. In at least one embodiment, memory crossbar 1116 has a connection to memory interface 1118 to communicate with I / O unit 1104, as well as a local instance of parallel processor memory 1102, to enable processing clusters 1114A-1114N within different processing clusters 1114A-1114N to communicate with system memory or other memories not local to parallel processor 1102. In at least one embodiment, memory crossbar 1116 can use virtual channels to separate traffic streams between clusters 1114A-1114N and partition units 1120A-1120N.

[0230] In at least one embodiment, multiple instances of parallel processor 1102 can be provided on a single add-in card, or multiple add-in cards can be interconnected. In at least one embodiment, different instances of parallel processor 1102 can be configured to operate together as a single parallel processor 1102, even if the different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. In at least one embodiment, some instances of parallel processor 1102 can include more high-precision floating point units relative to other instances. In at least one embodiment, a system incorporating one or more instances of parallel processor 1102 or parallel processor 1100B can be implemented in a variety of configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0231] Figure 11C is a block diagram of a partition unit 1120 in accordance with at least one embodiment. In at least one embodiment, partition unit 1120 is a Figure 11Bone of the partition units 1120A-1120N of FIG. 11. In at least one embodiment, partition unit 1120 includes an L2 cache 1121, a frame buffer interface 1125, and a ROP 1126 (Raster Operations unit). L2 cache 1121 is a read / write cache that is configured to perform load and store operations received from memory crossbar 1116 and ROP 1126. In at least one embodiment, L2 cache 1121 outputs read misses and urgent write-backs to frame buffer interface 1125 for processing. In at least one embodiment, updates can also be sent to a frame buffer via frame buffer interface 1125 for processing. In at least one embodiment, frame buffer interface 1125 interacts with one of memory units 1124A-1124N (e.g., within parallel processor memory 1122) in parallel processor memory. Figure 11B

[0232] In at least one embodiment, ROP 1126 is a processing unit that performs raster operations including, for example, fill, line, ellipse, triangle, and / or the like. In at least one embodiment, ROP 1126 is configured to execute shaders stored in, for example, memory location 1123 of FIG. 11. In at least one embodiment, ROP 1126 includes one or more surface engines to process 3D

[0233] In at least one embodiment, ROP 1126 includes, for example, one or more processors to process tasks including, for example, third party graphics acceleration software, video encoding, and / or the like. In at least one embodiment, ROP 1126 includes hardware and / or software to perform memory bandwidth scrubbing. Figure 11B In at least one embodiment, ROP 1126 is included in each processing cluster 1114A-1114N within parallel processor 1100A of FIG. 11 rather than in partition unit 1120. In at least one embodiment, read and write requests for pixel data are transmitted over memory crossbar 1116 instead of pixel fragment data by partition unit 1120. In at least one embodiment, processed graphics data can be displayed on display device(s) 1110, routed to a Figure 11A Figure 11B

[0234] FIG. 11D FIG. 12 is a block diagram of a processing cluster 1114 within a parallel processing unit in accordance with at least one embodiment. In at least one embodiment, processing cluster 1114 is a FIG. 11B ​​​one of the processing clusters 1114A-1114N. In at least one embodiment, one or more processing clusters 1114 can be configured to execute many threads in parallel, where a “thread” refers to an instance of a particular program executing on a particular set of input data. In at least one embodiment, Single Instruction Multiple Data (SIMD) instruction issue techniques are used to support parallel execution of a large number of threads without providing multiple independent instruction units to apply instructions to each thread. In at least one embodiment, Single Instruction Multiple Thread (SIMT) techniques are used to support parallel execution of a large number of synchronized threads, using a common instruction unit configured to issue instructions to a group of processing engines within each processing cluster.

[0235] In at least one embodiment, operation of processing cluster 1114 can be controlled by a pipeline manager 1132 that allocates processing tasks to SIMT parallel processor. In at least one embodiment, pipeline manager 1132 receives instructions from scheduler 1110, and manages execution of those instructions via graphics multiprocessor 1134 and / or texture unit 1136. In at least one embodiment, graphics multiprocessor 1134 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, various types of SIMT parallel processors of differing architectures can be included within processing cluster 1114. In at least one embodiment, one or more instances of graphics multiprocessor 1134 can be included within a processing cluster 1114. In at least one embodiment, graphics multiprocessor 1134 can process data, and a data crossbar 1140 can be used to distribute processed data to one of a number of possible destinations, including other shader units. In at least one embodiment, pipeline manager 1132 can facilitate distribution by specifying destinations for processed data as a function of its origin. FIG. 11B

[0236] In at least one embodiment, each graphics multiprocessor 1134 within processing cluster 1114 can include an identical set of functional execution logic (e.g., arithmetic logic, load store units, etc.). In at least one embodiment, functional execution logic can be configured in a pipelined manner, where new instructions can be issued before previous instructions are complete. In at least one embodiment, functional execution logic supports a variety of operations including integer and floating point arithmetic, comparison operations, Boolean operations, shift operations, and a multitude of algebraic functions. In at least one embodiment, same functional-unit hardware can be leveraged to perform different operations using different settings of control bits in those instructions. Any combination of

[0237] ​In at least one embodiment, instructions delivered to processing cluster 1114 constitute a thread. In at least one embodiment, a set of threads executing across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a program on different input data. In at least one embodiment, each thread within a thread group can be assigned to a different processing engine within graphics multiprocessor 1134. In at least one embodiment, a thread group can include fewer threads than are present in a plurality of processing engines. In at least one embodiment, when a thread group includes fewer threads than the number of processing engines present in graphics multiprocessor 1134, one or more of the processing engines can be idle during the execution of the threads in the thread group. In at least one embodiment, a thread group can also include more threads than are present in a plurality of processing engines. In at least one embodiment, when a thread group includes more threads than the number of processing engines present in graphics multiprocessor 1134, multiple threads in a thread group can be executed concurrently on multiple processing engines of graphics multiprocessor 1134. In at least one embodiment, parallel thread execution can be instructed up from a higher level paradigm, for example, a model thread executing on a CPU.

[0238] In at least one embodiment, graphics multiprocessor 1134 includes internal cache memory, to perform load and store operations. In at least one embodiment, graphics multiprocessor 1134 can discard internal cache and use cache memory within processing cluster 1114 (e.g., Ll cache 1148). In at least one embodiment, each graphics multiprocessor 1134 can also have access to L2 Cache within a partition unit (e.g., partition units 1120A-1120N) that is shared among all processing clusters 1114 and can be used to transfer data between threads. In at least one embodiment, graphics multiprocessor 1134 can also have access to off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory external to parallel processor 1102 can be used as global memory. In at least one embodiment, processing cluster 1114 includes multiple instances of graphics multiprocessor 1134, which can share common instructions and data stored in Ll cache 1148. FIG. 11B

[0239] In at least one embodiment, each processing cluster 1114 can include a memory management unit (MMU) 1145 to map virtual addresses into physical addresses as is known in the art. In at least one embodiment, one or more instances of MMU 1145 can reside in FIG. 11B ​In at least one embodiment, MMU 1145 includes a set of page table entries (PTEs) used to map virtual addresses into physical addresses for task computation and, in at least one embodiment, into cache line indices for tasks. In at least one embodiment, MMU 1145 can include address translation lookaside buffer (TLB) or cache, which can reside in graphics processing cluster 1134 or in Ll cache 1148 or processing cluster 1114. In at least one embodiment, processing physical addresses to allocate surface data access locality for efficient request interleaving between partition units. In at least one embodiment, cache line indices can be used to determine whether a request for a cache line is a hit or miss.

[0240] In at least one embodiment, processing cluster 1114 can be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 for performing texture mapping operations, address translation, and filtering operations in connection with textures accessed by the shaders, which can include vertex and geometry shaders. In at least one embodiment, texture data can be read from an internal texture Ll cache (not shown) or from an L2 cache, a local parallel processor memory, or system memory, as needed. Each graphics multiprocessor 1134 outputs processed tasks to data crossbar 1140 in order to provide processed tasks to another processing cluster 1114 for further processing or to store processed task data in an L2 cache, a local parallel processor memory, or system memory via memory crossbar 1116. FIG. 11B In at least one embodiment, preROP 1142 is configured to receive data from graphics multiprocessors 1134, direct the data to ROP unit, which can be located physically adjacent to the partition units (e.g., partition units 1120A-1120N of FIG. 11) in at least one embodiment. In at least one embodiment, PreROP 1142 unit can perform optimizations related to color blending, organize pixel color data, and perform address translation.

[0241] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGs. 6A and / or 6B. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGs. 6A and / or 6B.

[0242] FIG. 11E A graphics processing unit 1134 according to at least one embodiment is shown. In at least one embodiment, graphics processing unit 1134 is coupled with a pipeline manager 1132 of processing cluster 1114. In at least one embodiment, graphics processing unit 1134 has a thread execution pipeline that includes, without limitation, an instruction cache 1152, an instruction unit 1154, an address mapping unit 1156, a register file 1158, one or more general-purpose GPU (GPGPU) cores 1162, and one or more load / store units 1166. One or more GPGPU cores 1162 and one or more load / store units 1166 are coupled with cache memory 1172 and shared memory 1170 via a memory and cache interconnect 1168.

[0243] In at least one embodiment, instruction cache 1152 receives a stream of instructions 1150 to be executed by graphics processing unit 1134 from pipeline manager 1132. In at least one embodiment, instructions are cached in instruction cache 1152 and dispatched for execution by instruction unit 1154. In one embodiment, instruction unit 1154 can dispatch instructions as a thread group (e.g., a warp) and assign each thread of the thread group to a different execution unit within one or more GPGPU cores 1162. In at least one embodiment, instructions can access any of a number of different address spaces through the use of addresses specified in the instructions. In at least one embodiment, address mapping unit 1156 can be used to translate addresses used by each thread of a thread group into a different memory address usable by one or more load / store units 1166.

[0244] In at least one embodiment, register file 1158 provides a set of registers for the functional units of graphics processing unit 1134. In at least one embodiment, register file 1158 provides temporary storage for operands of the data

[0245] In at least one embodiment, GPGPU cores 1162 can each include floating point units (FPUs) and / or integer arithmetic logic units (ALUs) that are capable of performing instructions defined by an extension of the SIMD instruction set implemented in GPGPU cores 1162. In at least one embodiment, GPGPU cores 1162 can also include fixed function and / or special-purpose processing units to perform specific functions in a hardware efficient manner. In at least one embodiment, one or more of GPGPU cores 1162 can also include fixed or special-purpose logic.

[0246] In at least one embodiment, GPGPU cores 1162 include SIMD logic capable of performing a single - or multiple -threaded instruction on multiple sets of data with each thread in a single -threaded instruction being processed by a separate SIMD unit. In at least one embodiment, GPGPU cores 1162 can implement a SIMD instruction set that includes SIMD Fused-Multiply Add (FMA) instructions. In at least one embodiment, GPGPU cores 1162 can also include fixed or special-purpose logic to support the execution of certain number-crunching algorithms that are used by applications in the fields of finance, data mining, and video image processing.

[0247] In at least one embodiment, memory and cache interconnect 1168 is an interconnect network that connects each functional unit of graphics multiprocessor 1134 to register file 1158 and shared memory 1170. In at least one embodiment, memory and cache interconnect 1168 is a crossbar interconnect that allows load / store units 1166 to implement load and store operations between shared memory 1170 and register file 1158. In at least one embodiment, register file 1158 can operate at same frequency as GPGPU cores 1162, such that latency for data transfers between GPGPU cores 1162 and register file 1158 is very low. In at least one embodiment, shared memory 1170 can be used to enable communication between threads executing on functional units within graphics multiprocessor 1134. In at least one embodiment, cache memory 1172 can be used to cache texture data communicated between texture unit 1136 and functional units. In at least one embodiment, shared memory 1170 can also be used as a program managed cache. In at least one embodiment, in addition to automatically cached data stored in cache memory 1172, a thread executing on GPGPU core 1162 can store data in shared memory in a programmed manner.

[0248] In at least one embodiment, a parallel processor or GPGPU as described herein is communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general purpose GPU (GPGPU) functions. In at least one embodiment, GPU can be communicatively coupled to host processor / cores by a bus or other interconnect (e.g., a high speed

[0249] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, inference and / or training logic 615 is used in conjunction with components of system 600, for example, to perform inferencing operations, training operations, and / or similar operations. FIG. 6B and / or FIG. 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 can be used in graphics multiprocessor 1134 for inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0250] 12A illustrates a multi-GPU computing system 1200A, according to at least one embodiment. In at least one embodiment, multi-GPU computing system 1200A can include a processor 1202 coupled to a plurality of general purpose graphics processing units (GPGPUs) 1206A-D via a host interface switch 1204. In at least one embodiment, host interface switch 1204 is a PCI Express switch device that couples processor 1202 to a PCI Express bus by which processor 1202 can communicate with GPGPUs 1206A-D. GPGPUs 1206A-D can be interconnected via a set of high-speed P2P GPU-to-GPU links 1216. In at least one embodiment, GPU-to-GPU links 1216 connect to each of GPGPUs 1206A-D via a dedicated GPU link. In at least one embodiment, P2P GPU links 1216 enable direct communication between each GPGPU 1206A-D without having to communicate through host interface bus 1204 to which processor 1202 is connected. In at least one embodiment, host interface bus 1204 remains available for system memory access or communication with other instances of multi-GPU computing system 1200A via one or more network devices, in case of GPU-to-GPU traffic directed to P2P GPU links 1216. While in at least one embodiment GPGPUs 1206A-D are connected to processor 1202 via host interface switch 1204, in at least one embodiment processor 1202 includes direct support for P2P GPU links 1216 and can connect directly to GPGPUs 1206A-D.

[0251] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 615 can be used in graphics multiprocessor 1134 for inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein. FIG. 6B and / or FIG. 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 can be used in multi-GPU computing system 1200A for inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0252] FIG. 12B is a block diagram of graphics processor 1200B in accordance with at least one embodiment. In at least one embodiment, graphics processor 1200B includes ring interconnect 1202, front-end pipeline 1204, media engine 1237, and graphics cores 1280A-1280N. In at least one embodiment, ring interconnect 1202 couples graphics processor 1200B to other processing units including other graphics processors or one or more general-purpose processor cores. In at least one embodiment, graphics processor 1200B is one of a number of processors integrated within a multi-core processing system.

[0253] In at least one embodiment, graphics processor 1200B receives batches of commands via ring interconnect 1202. In at least one embodiment, incoming commands are interpreted by a command streamer 1203 in pipeline front-end 1204. In at least one embodiment, graphics processor 1200B includes scalable execution logic to perform 3D geometry processing and media processing via the graphics cores 1280A-1280N. In at least one embodiment, for 3D geometry processing commands, command streamer 1203 supplies commands to geometry pipeline 1236. In at least one embodiment, for at least some media processing commands, command streamer 1203 supplies commands to a video front end 1234, which couples with media engine 1237. In at least one embodiment, media engine 1237 includes a video quality engine (VQE) 1230 for video and image post-processing, and a multi-format encode / decode (MFX) 1233 engine to

[0254] In at least one embodiment, graphics processor 1200B includes a scalable thread execution resource including a module core featuring a number of core slices 1280A-1280N (sometimes referred to as core tile). In at least one embodiment, graphics processor 1200B can have any number of graphics cores 1280A. In at least one embodiment, graphics processor 1200B includes graphics core 1280A having at least a first sub-core 1250A and a second sub-core 1260A. In at least one embodiment, graphics processor 1200B is a low power processor with a single sub-core (e.g., 1250A). In at least one embodiment, graphics processor 1200B includes multiple graphics cores 1280A-1280N each including a set of first sub-cores 1250A-1250N and a set of second sub-cores 1260A-1260N. In at least one embodiment, each sub-core in first sub-cores 1250A-1250N includes at least a first set of execution units 1252A-1252N and a media / texture samplers 1254A-1254N. In at least one embodiment, each sub-core in second sub-cores 1260A-1260N includes at least a second set of execution units 1262A-1262N and samplers 1264A-1264N. In at least one embodiment, each sub-core 1250A-1250N, 1260A-1260N shares a set of shared resources 1270A-1270N. In at least one embodiment, shared resources include shared cache memory and pixel operation logic.

[0255] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, inference and / or training logic 615 can be used in graphics processor 1200B for inferencing or predicting operations based, at least in part, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0256] FIG. 13is a block diagram illustrating microarchitecture for a processor 1300 that can include logic circuits to execute instructions, in accordance with at least one embodiment. In at least one embodiment, processor 1300 can execute instructions including x86 instructions, ARM instructions, specialized instructions for application specific integrated circuits (ASICs), and the like. In at least one embodiment, processor 1300 can include registers to store packed data, such as 64-bit wide MMX™ registers in microprocessors enabled with MMX technology by Intel Corporation of Santa Clara, California. In at least one embodiment, MMX registers available in integer and floating point form can operate with packed data elements that accompany single instruction multiple data (“SIMD”) and streaming SIMD extensions (“SSE”) instructions. In at least one embodiment, 128-bit wide XMM registers related to SSE2, SSE3, SSE4, AVX, or higher (generically referred to as “SSEx”) technology can hold such packed data operands. In at least one embodiment, processor 1300 can execute instructions to accelerate machine learning or deep learning algorithms, training, or inference.

[0257] In at least one embodiment, processor 1300 includes an in-order front-end (“front-end”) 1301 to fetch instructions to be executed and to prepare instructions for execution by other pipelines. In at least one embodiment, front-end 1301 can include several units. In at least one embodiment, instruction prefetcher 1326 fetches instructions from memory and provides pre-fetched instructions to instruction decoder 1328 which, in turn, decodes or interprets instructions. In at least one embodiment, instruction decoder 1328 decodes a received instruction as one or more operations on

[0258] In at least one embodiment, some instructions can be converted into a single micro- operation, while others can require several micro-operations to complete. In at least one embodiment, if more than four micro-instructions are needed to complete a single instruction, then instruction decoder 1328 can access microcode ROM 1332 to execute the instruction. In at least one embodiment, instructions can be decoded into a small number of micro-instructions to be processed at instruction decoder 1328. In at least one embodiment, if an instruction requires multiple micro-instructions to complete an operation, then the instruction can be stored in microcode ROM 1332. In at least one embodiment, a trace cache 1330 references an entry point programmable logic array (“PLA”) to determine a correct micro-instruction pointer for reading a microcode sequence from microcode ROM 1332 to complete one or more instructions, in accordance with at least one embodiment. In at least one embodiment, after microcode ROM 1332 completes sequencing of micro-operations for an instruction, a front end 1301 of a machine can resume fetching micro-operations from trace cache 1330.

[0259] In at least one embodiment, out-of-order execution engine (“out-of-order engine”) 1303 can prepare instructions for execution. In at least one embodiment, out-of-order execution logic has a number of buffers to smooth and reorder instruction flow to optimize performance as instructions are pipelined down and dispatched for execution. In at least one embodiment, out-of-order execution engine 1303 includes, without limitation, an allocator / register renamer 1340, a memory micro instruction queue 1342, an integer / float micro instruction queue 1344, a memory scheduler 1346, a fast scheduler 1302, a slow / general floating point scheduler (“slow / general FP scheduler”) 1304, and a simple floating point scheduler (“simple FP scheduler”) 1306. In at least one embodiment, fast scheduler 1302, slow / general floating point scheduler 1304, and simple floating point scheduler 1306 are also collectively referred to as “micro instruction schedulers 1302, 1304, 1306.” In at least one embodiment, allocator / register renamer 1340 allocates machine buffers and resources needed by each micro instruction to execute in sequence. In at least one embodiment, allocator / register renamer 1340 renames logical registers to entries in a register file. In at least one embodiment, allocator / register renamer 1340 also allocates entries for each micro instruction in one of two micro instruction queues, memory micro instruction queue 1342 for memory operations and integer / float micro instruction queue 1344 for non-memory operations, in front of memory scheduler 1346 and micro instruction schedulers 1302, 1304, 1306. In at least one embodiment, micro instruction schedulers 1302, 1304, 1306 determine when micro instructions are ready to execute based on readiness of their dependent input register operand sources and availability of execution resources micro instructions need to complete. In at least one embodiment, fast scheduler 1302 of at least one embodiment can schedule on every half of a main clock cycle, while slow / general floating point scheduler 1304 and simple floating point scheduler 1306 can schedule once per main processor clock cycle. In at least one embodiment, micro instruction schedulers 1302, 1304, 1306 arbitrate for a dispatch port to dispatch micro instructions for execution.

[0260] In at least one embodiment, execution block 1311 includes, without limitation, integer register file / bypass network 1308, floating point register file / bypass network (“FP register file / bypass network”) 1310, address generation units (“AGUs”) 1312 and 1314, fast ALUs 1316 and 1318, slow ALUs 1320, floating point ALUs (“FPs”) 1322, and floating point move unit (“FP move”) 1324. In at least one embodiment, integer register file / bypass network 1308 and floating point register file / bypass network 1310 are also referred to herein as “register files 1308, 1310.” In at least one embodiment, AGUs 1312 and 1314, fast ALUs 1316 and 1318, slow ALUs 1320, floating point ALUs 1322, and floating point move unit 1324 are also referred to herein as “execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324.” In at least one embodiment, execution block 1311 can include, without limitation, any number (including zero) and type of register files, bypass networks, address generation units, and execution units (in any combination).

[0261] In at least one embodiment, register networks 1308, 1310 can be arranged between micro-instruction schedulers 1302, 1304, 1306 and execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324. In at least one embodiment, integer register file / bypass network 1308 performs integer operations. In at least one embodiment, floating point register file / bypass network 1310 performs floating point operations. In at least one embodiment, each of register networks 1308, 1310 can include, without limitation, a bypass network that can bypass or forward a just-completed result that has not yet been written into a register file to a new dependee. In at least one embodiment, register networks 1308, 1310 can communicate data with each other. In at least one embodiment, integer register file / bypass network 1308 can include, without limitation, two separate register files, one for low order 32 bits data, a second for high order 32 bits data. In at least one embodiment, floating point register file / bypass network 1310 can include, without limitation, 128 bit-wide entries, as floating point instructions typically have operand sizes of 64 to 128 bits.

[0262] In at least one embodiment, execution units 1312, 1314, 1316, 1318, 1320, 1322, 1324 can execute instructions. In at least one embodiment, register files 1308, 1310 store integer and floating point data operand values upon which microinstructions require execution. In at least one embodiment, processor 1300 can include, without limitation, any number and combination of execution units 1312, 1314, 1316, 1318, 1320, 1322, 1324. In at least one embodiment, floating point ALU 1322 and floating point move unit 1324 can execute floating point, MMX, SIMD, AVX and SSE, or other operations, including specialized machine learning instructions. In at least one embodiment, floating point ALU 1322 can include, without limitation, a 64 bit by 64 bit floating point divider to execute divide, square root, and remainder micro-ops. In at least one embodiment, instructions for dealing with floating point values can be handled with floating point hardware. In at least one embodiment, ALU operations can be passed to fast ALUs 1316, 1318. In at least one embodiment, fast ALUs 1316, 1318 can execute fast operations with an effective latency of half a clock cycle. In at least one embodiment, most complex integer operations enter slow ALU 1320 as slow ALU 1320 can include, without limitation, integer execution hardware for long latency type operations such as

[0263] In at least one embodiment, micro-instruction scheduler 1302, 1304, 1306 schedules dependent operations prior to completion of parent load execution. In at least one embodiment, because micro-instructions can be speculatively scheduled and executed in processor 1300, processor 1300 can also include logic to handle memory misses. In at least one embodiment, if a data load in a data cache misses, there can be a dependent operation running in a pipeline that causes the scheduler to temporarily have incorrect data. In at least one embodiment, a replay mechanism tracks and re-executes instructions that use incorrect data. In at least one embodiment, dependent operations can need to be replayed and independent operations can be allowed to complete. In at least one embodiment, a scheduler and replay mechanism of at least one embodiment of a processor can also be designed to capture instruction sequences for text string compare operations.

[0264] In at least one embodiment, a “register” can refer to an on-board processor storage location that can be used as part of an instruction that identifies an operand. In at least one embodiment, a register can be one that can be used from outside of a processor (from a programmer’s perspective). In at least one embodiment, a register can not be limited to a particular type of circuit. Rather, in at least one embodiment, a register can store data, provide data, and perform functions described herein. In at least one embodiment, registers described herein can be implemented by circuitry within a processor using a variety of different techniques, such as dedicated physical registers, physical registers dynamically allocated using register renaming, a combination of dedicated and dynamically allocated physical registers, etc. In at least one embodiment, an integer register stores 32-bit integer data. A register file of at least one embodiment also contains eight multimedia SIMD registers for packing data.

[0265] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, inference and / or training logic 615 can be used in FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, portions or all of inference and / or training logic 615 can be incorporated into execution block 1311 and other memory or registers shown or not shown. For example, in at least one embodiment, training and / or inferencing techniques described herein can use one or more ALUs shown in execution block 1311. Further, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not shown) that configure ALUs of execution block 1311 to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0266] FIG. 14A deep learning application processor 1400 is shown, in accordance with at least one embodiment. In at least one embodiment, deep learning application processor 1400 uses instructions that, if executed by deep learning application processor 1400, cause deep learning application processor 1400 to perform some or all of the processes and techniques described throughout this disclosure. In at least one embodiment, deep learning application processor 1400 is an application specific integrated circuit (ASIC). In at least one embodiment, application processor 1400 performs matrix multiplication operations or is “hardwired” into hardware as a result of executing one or more instructions or both. In at least one embodiment, deep learning application processor 1400 includes, without limitation, processing clusters 1410(1)-1410(12), inter-chip links (“ICLs”) 1420(1)-1420(12), inter-chip controllers (“ICCs”) 1430(1)-1430(2), memory controllers (“Mem Ctrlrs”) 1442(1)-1442(4), high bandwidth memory physical layers (“HBM PHYs”) 1444(1)-1444(4), management controller central processing units (“management controller CPUs”) 1450, serial peripheral interfaces, internal integrated circuits, and general purpose input / output blocks (“SPI, I2C, GPIO”), peripheral component interconnect express controllers and direct memory access blocks (“PCIe controllers and DMA”) 1470, and sixteen lane peripheral component interconnect express ports (“PCI Express x 16”) 1480.

[0267] In at least one embodiment, processing clusters 1410 can perform deep learning operations, including inferencing or prediction operations based on weight parameters calculated based on one or more training techniques, including those techniques herein. In at least one embodiment, each processing cluster 1410 can include, without limitation, any number and type of processors. In at least one embodiment, deep learning application processor 1400 can include any number and type of processing clusters 1400. In at least one embodiment, inter-chip links 1420 are bidirectional. In at least one embodiment, inter-chip links 1420 and inter-chip controllers 1430 enable multiple deep learning application processors 1400 to exchange information, including activation information resulting from execution of one or more machine learning algorithms embodied in one or more neural networks. In at least one embodiment, deep learning application processor 1400 can include any number (including zero) and type of ICLs 1420 and ICCs 1430.

[0268] In at least one embodiment, HBM2 1440 provides a total of 32 GB of memory. HBM2 1440(i) is associated with both a memory controller 1442(i) and an HBM PHY 1444(i). In at least one embodiment, any number of HBM2 1440 can provide any type and total amount of high bandwidth memory and can be associated with any number (including zero) and type of memory controllers 1442 and HBM PHYs 1444. In at least one embodiment, SPI, I2C, GPIO 3360, PCIe controller 1460, and DMA 1470 and / or PCIe 1480 can be replaced with any number and type of blocks to implement any number and type of communication standards in any technically feasible manner.

[0269] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, deep learning application processor is used to train a machine learning model (e.g., neural network) to predict or infer information provided to deep learning application processor 1400. In at least one embodiment, deep learning application processor 1400 is used to infer or predict information based on a trained machine learning model (e.g., neural network) that has been trained by another processor or system or by deep learning application processor 1400. In at least one embodiment, processor 1400 can be used to perform one or more neural network use cases described herein.

[0270] FIG. 15 is a block diagram of a neuromorphic processor 1500, in accordance with at least one embodiment. In at least one embodiment, neuromorphic processor 1500 can receive one or more inputs from a source external to neuromorphic processor 1500. In at least one embodiment, these inputs can be transmitted to one or more neurons 1502 within neuromorphic processor 1500. In at least one embodiment, neurons 1502 and components thereof can be implemented using circuitry or logic including one or more arithmetic logic units (ALUs). In at least one embodiment, neuromorphic processor 1500 can include, without limitation, thousands or millions of instances of neurons 1502, but any suitable number of neurons 1502 can be used. In at least one embodiment, each instance of neuron 1502 can include neuron input 1504 and neuron output 1506. In at least one embodiment, neurons 1502 can generate outputs that can be transmitted to inputs of other instances of neurons 1502. In at least one embodiment, neuron inputs 1504 and neuron outputs 1506 can be interconnected via synapses 1508.

[0271] In at least one embodiment, neurons 1502 and synapses 1508 can be interconnected such that neuromorphic processor 1500 operates to process or analyze information received by neuromorphic processor 1500. In at least one embodiment, a neuron 1502 can send an output spike (or “spike” or “peak”) when input received through neuron input 1504 exceeds a threshold value. In at least one embodiment, neuron 1502 can sum or integrate signals received at neuron input 1504. For example, in at least one embodiment, neuron 1502 can be implemented as a leaky integrate-and-fire neuron, where neuron 1502 can produce an output (or “spike”) using a transfer function such as a sigmoid or threshold function if a sum (referred to as “membrane potential”) exceeds a threshold value. In at least one embodiment, a leaky integrate-and-fire neuron can sum signals received at neuron input 1504 into a membrane potential, and can apply a program decay factor (or leak) to reduce the membrane potential. In at least one embodiment, a leaky integrate-and-fire neuron can spike if multiple input signals are received at neuron input 1504 fast enough to exceed a threshold value before membrane potential decays too low to spike (in at least one embodiment). In at least one embodiment, neuron 1502 can be implemented using circuitry or logic that receives input, integrates input into a membrane potential, and decays the membrane potential. In at least one embodiment, input can be averaged, or any other suitable transfer function can be used. Furthermore, in at least one embodiment, neuron 1502 can include, without limitation, comparator circuitry or logic that produces an output spike at neuron output 1506 when a result of applying a transfer function to neuron input 1504 exceeds a threshold value. In at least one embodiment, once neuron 1502 spikes, it can ignore previously received input information by, for example, resetting the membrane potential to 0 or another suitable default value. In at least one embodiment, once membrane potential is reset to 0, neuron 1502 can resume normal operation after a suitable period of time (or refractory period).

[0272] In at least one embodiment, neurons 1502 can be interconnected by synapses 1508. In at least one embodiment, synapses 1508 can operate to transmit a signal from an output of a first neuron 1502 to an input of a second neuron 1502. In at least one embodiment, a neuron 1502 can transmit information over more than one instance of a synapse 1508. In at least one embodiment, one or more instances of neuron output 1506 can be connected through an instance of synapse 1508 to an instance of neuron input 1504 in the same neuron 1502. In at least one embodiment, an instance of neuron 1502 that produces an output to be transmitted over an instance of synapse 1508 can be referred to as a “presynaptic neuron” with respect to that instance of synapse 1508. In at least one embodiment, an instance of neuron 1502 that receives input transmitted over an instance of synapse 1508 can be referred to as a “postsynaptic neuron” with respect to that instance of synapse 1508. In at least one embodiment, with respect to various instances of synapse 1508, because an instance of neuron 1502 can receive input from one or more instances of synapse 1508 and can also transmit output through one or more instances of synapse 1508, a single instance of neuron 1502 can be both a “presynaptic neuron” and a “postsynaptic neuron”.

[0273] In at least one embodiment, neurons 1502 can be organized into one or more layers. Each instance of neuron 1502 can have one neuron output 1506 that can fan out to one or more neuron inputs 1504 through one or more synapses 1508. In at least one embodiment, neuron outputs 1506 of neurons 1502 in a first layer 1510 can be connected to neuron inputs 1504 of neurons 1502 in a second layer 1512. In at least one embodiment, layer 1510 can be referred to as a “feedforward layer”. In at least one embodiment, each instance of neuron 1502 in an instance of first layer 1510 can fan out to each instance of neuron 1502 in a second layer 1512. In at least one embodiment, first layer 1510 can be referred to as a “fully connected feedforward layer”. In at least one embodiment, each instance of neuron 1502 in each instance of second layer 1512 fans out to fewer than all instances of neuron 1502 in a third layer 1514. In at least one embodiment, second layer 1512 can be referred to as a “sparsely connected feedforward layer”. In at least one embodiment, neurons 1502 in (same) second layer 1512 can fan out to neurons 1502 in multiple other layers, including to neurons 1502 in second layer 1512. In at least one embodiment, second layer 1512 can be referred to as a “recurrent layer”. In at least one embodiment, neuromorphic processor 1500 can include, without limitation, any suitable combination of recurrent and feedforward layers, including without limitation sparsely connected feedforward layers and fully connected feedforward layers.

[0274] In at least one embodiment, neuromorphic processor 1500 can include, without limitation, a reconfigurable interconnect architecture or a dedicated hardwired interconnect to connect synapses 1508 to neurons 1502. In at least one embodiment, neuromorphic processor 1500 can include, without limitation, circuitry or logic that, depending on a neural network topology and neuron fan-in / fan-out, allows synapses to be allocated to different neurons 1502 as needed. For example, in at least one embodiment, synapses 1508 can be connected to neurons 1502 using an interconnect structure such as a network-on-chip or through dedicated connections. In at least one embodiment, synapse interconnects and components thereof can be implemented using circuitry or logic.

[0275] FIG. 16AA processing system is shown in accordance with at least one embodiment. In at least one embodiment, system 1600A includes one or more processor(s) 1602 and one or more graphics processor(s) 1608, and can be a single processor desktop system, a multiprocessor workstation system, or a server system having many processor(s) 1602 or processor core(s) 1607. In at least one embodiment, system 1600A is a processing platform incorporated within a system on a chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices.

[0276] In at least one embodiment, system 1600A can include or be incorporated within a server-based gaming platform, including a game console, a gaming console having game and media functions, a mobile gaming console, a handheld game console, or an online game console. In at least one embodiment, system 1600A is a mobile phone, a smart phone, a tablet device, or a web appliance. In at least one embodiment, processing system 1600A can also include or be coupled with a wearable device, such as a smart watch wearable device, smart glasses device, augmented reality device, or virtual reality device. In at least one embodiment, processing system 1600A is a television or set-top box device having one or more processor(s) 1602, as well as a graphical interface generated by one or more graphics processor(s) 1608.

[0277] In at least one embodiment, one or more processor(s) 1602 each include one or more processor cores 1607 to process instructions which, when executed, implement the operations for system and user software. In at least one embodiment, each of the one or more processor cores 1607 is configured to process a specific instruction set 1609. In at least one embodiment, instruction set 1609 can facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computing via a Very Long Instruction Word (VLIW). In at least one embodiment, processor cores 1607 can each process a different instruction set 1609, which can include instructions to facilitate emulation of other instruction sets. In at least one embodiment, processor core(s) 1607 can also include other processing devices, such as a digital signal processor (DSP).

[0278] In at least one embodiment, processor 1602 includes cache memory 1604. In at least one embodiment, processor 1602 can have single-level or multi-level internal caches. In at least one embodiment, cache memory is shared among various components of processor 1602. In at least one embodiment, processor 1602 also uses an external cache (e.g., a level three (L3) cache or last level cache (LLC)) (not shown), which can be shared between processor cores 1607 using known cache coherency techniques. In at least one embodiment, additionally included in processor 1602 are register file 1606, which processor can include different types of registers such as integer registers, floating point registers, status registers, and instruction pointer registers to name a few.

[0279] In at least one embodiment, one or more processor(s) 1602 are coupled with one or more interface bus(es) 1610 for communicating data between processor 1602 and other components of system 1600A, such as address, data, or control signals. In at least one embodiment, interface bus 1610 can be a version of a processor bus such as a direct media interface (DMI) bus in at least one embodiment. In at least one embodiment, interface bus 1610 is not limited to DMI bus, and can include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, processor 1602 includes integrated memory controller 1616 and platform controller hub 1630. In at least one embodiment, memory controller 1616 facilitates communication between memory devices and other components of processing system 1600A, while platform controller hub (PCH) 1630 provides connections to input / output (I / O) devices via local I / O bus.

[0280] In at least one embodiment, memory device 1620 can be a Dynamic Random Access Memory (DRAM) device, a Static Random Access Memory (SRAM) device, a flash memory device, or a phase change memory device, among others. In at least one embodiment, memory device 1620 can be a system memory of processing system 1600A, to store data 1622 and instructions 1621 for use when one or more processors 1602 executes an application or process. In at least one embodiment, memory controller 1616 also couples with an external graphics processor 1612, which can communicate with one or more graphics processors 1608 in processing system 1600A to perform graphics and media operations. In at least one embodiment, a display device 1611 can connect to processing system 1600A. In at least one embodiment, display device 1611 can include one or more of an internal display device, as in a mobile electronic device or a laptop device or an external display device attached via a display interface (e.g., DisplayPort, etc.). In at least one embodiment, display device 1611 can include a head mounted display (HMD) such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0281] In at least one embodiment, platform controller hub 1630 enables peripherals coupled to bridge 1642 to interact with the processor 1602. In at least one embodiment, platform controller hub 1630 can enable peripherals coupled to bridge 1642 to interact with a system firmware, such as a unified extensible firmware (UEFI). In at least one embodiment, memory controller 1616 is coupled to platform controller hub 1630. In at least one embodiment, platform controller hub 1630 can include a storage interface coupled to one or more non-volatile memory devices (e.g., NAND flash) via storage bus 1622.

[0282] In at least one embodiment, memory controller 1616 and instances of platform controller hub 1630 can be integrated into a discrete external graphics processor, such as external graphics processor 1612. In at least one embodiment, platform controller hub 1630 and / or memory controller 1616 can be external to one or more processor(s) 1602. In at least one embodiment, system 1600A can include an external memory controller 1616 and platform controller hub 1630, which can be configured as a memory hub and peripheral hub in a system-on-a-chip (SoC) configuration.

[0283] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are described in more detail below. FIG. 6B and / or FIG. 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, some or all of inference and / or training logic 615 can be incorporated with graphics processor 1600A. In at least one embodiment, the training and / or inference techniques described herein can use one or more ALUs embodied in graphics processor 1612. Moreover, in at least one embodiment, the inference and / or training operations described herein can be accomplished with logic other than that illustrated in FIG. 6B and / or FIG. 6C illustrated. In at least one embodiment, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not) that configure ALUs of graphics processor 1600A to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0284] FIG. 16B is a block diagram of a processor 1600B having one or more processor cores 1602A-1602N, an integrated memory controller 1614, and an integrated graphics processor 1608, in accordance with at least one embodiment. In at least one embodiment, processor 1600B can contain additional cores, up to and including the additional core 1602N represented by the dashed line. In at least one embodiment, each processor core 1602A-1602N includes one or more internal cache units 1604A-1604N. In at least one embodiment, each processor core can also access one or more shared cache units 1606.

[0285] In at least one embodiment, internal cache units 1604A-1604N and shared cache unit 1606 represent a cache memory hierarchy within processor 1600B. In at least one embodiment, cache memory units 1604A-1604N can include at least one level of instruction and data caches per processor core and one or more shared level caches, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level cache, where the highest level of cache prior to main memory is classified as an LLC. In at least one embodiment, cache coherence logic maintains coherence between various cache units 1606 and 1604A-1604N.

[0286] In at least one embodiment, processor 1600B also includes a set of one or more bus controller units 1616 and a system agent core 1610. In at least one embodiment, one or more bus controller units 1616 manage a set of peripheral buses, such as one or more PCI or PCIe buses. In at least one embodiment, system agent core 1610 provides management functionality for various processor components. In at least one embodiment, system agent core 1610 includes one or more integrated memory controllers 1614 to manage access to various external memory devices (not shown), including support for data bus protocols such as DDR DRAM.

[0287] In at least one embodiment, one or more processor cores 1602A-1602N include support to run in multiple threads concurrently. In at least one embodiment, system agent core 1610 includes components for coordination and operation of cores 1602A-1602N during multi-threaded processing. In at least one embodiment, system agent core 1610 can additionally include a power control unit (PCU), including logic and components to govern one or more power states of processor cores 1602A-1602N and graphics processor 1608.

[0288] In at least one embodiment, processor 1600B also includes graphics processor 1608 to execute graphics processing operations. In at least one embodiment, graphics processor 1608 couples with shared cache unit 1606, and system agent core 1610, including one or more integrated memory controllers 1614. In at least one embodiment, system agent core 1610 also includes a display controller 1611 to drive graphics processor output to one or more coupled displays. In at least one embodiment, display controller 1611 can also be a separate module coupled with graphics processor 1608 via at least one interconnect, or can be integrated within graphics processor 1608.

[0289] In at least one embodiment, ring based interconnect unit 1612 is used to couple internal components of processor 1600B. In at least one embodiment, alternative interconnects can be used, such as a point-to-point interconnect, a switched interconnect, or other technology. In at least one embodiment, graphics processor 1608 couples with ring interconnect 1612 via I / O link 1613.

[0290] In at least one embodiment, I / O link 1613 represents at least one of a variety of I / O interconnects, including a package I / O interconnect that facilitates communication between various processor components and a high performance embedded memory module 1618 (e.g., an eDRAM module). In at least one embodiment, each of processor cores 1602A-1602N and graphics processor 1608 uses embedded memory module 1618 as a shared last level cache.

[0291] In at least one embodiment, processor cores 1602A-1602N are homogeneous cores that execute a common instruction set architecture. In at least one embodiment, processor cores 1602A-1602N are heterogeneous with respect to instruction set architecture (ISA) in that one or more processor cores 1602A-1602N execute a common instruction set while one or more other processor cores 1602A-1602N execute a subset or a different instruction set. In at least one embodiment, processor cores 1602A-1602N are heterogeneous with respect to microarchitecture in that one or more cores have a relatively higher power consumption and one or more power cores have a lower power consumption. In at least one embodiment, processor 1600B can be implemented on one or more chips or as a SoC integrated circuit.

[0292] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, inference and / or training logic 615 can be used in graphics processor 1600B. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, portions or all of inference and / or training logic 615 can be incorporated in processor 1600B. For example, in at least one embodiment, training and / or inference techniques described herein can be performed using one or more ALUs, which are embodied in graphics core(s) 1612, processor core(s) 1602A-1602N, or other component(s) of FIG. 16A In at least one embodiment, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not) that configure ALUs of graphics processor 1600B to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein. FIG. 6B and / or FIG. 6C In at least one embodiment, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not) that configure ALUs of graphics processor 1600B to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0293] FIG. 16Cis a block diagram of hardware logic of a graphics processor core 1600C in accordance with at least one embodiment herein. In at least one embodiment, graphics processor core 1600C is included in a graphics core array. In at least one embodiment, graphics processor core 1600C (sometimes referred to as a core slice) can be one or more graphics cores within a modular graphics processor. In at least one embodiment, graphics processor core 1600C is an example of one graphics core slice and graphics processors herein can include multiple graphics core slices based on target power and performance envelopes. In at least one embodiment, each graphics core 1600C can include fixed function block 1630, also referred to as a sub slice, coupled with a plurality of sub-cores 1601A-1601F that include modular blocks of general- and fixed- function logic.

[0294] In at least one embodiment, fixed function block 1630 includes geometry fixed function pipeline 1636, for example, that can be shared by all sub-cores in graphics processor 1600C in lower performance and / or lower power graphics processor implementations. In at least one embodiment, geometry and fixed function pipeline 1636 includes a 3D fixed function pipeline, a video front-end unit, a thread generator and thread dispatcher, and a unified return buffer manager that manages a unified return buffer.

[0295] In at least one embodiment, fixed function block 1630 also includes a graphics SoC interface 1637, a graphics microcontroller 1638, and a media pipeline 1639. In at least one embodiment, graphics SoC interface 1637 provides an interface between graphics core 1600C and other processor cores within a system on a chip integrated circuit. In at least one embodiment, graphics microcontroller 1638 is a programmable sub-processor that is configurable to manage various functions of graphics processor 1600C, including thread dispatch, scheduling, and pre-emption. In at least one embodiment, media pipeline 1639 includes logic to facilitate decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. In at least one embodiment, media pipeline 1639 implements media operations via requests to compute or sample logic within sub-cores 1601-1601F.

[0296] In at least one embodiment, SoC interface 1637 enables graphics core 1600C to communicate with general application processor cores (e.g., CPUs) and / or other components within the SoC, including memory hierarchy elements such as shared last level cache, system RAM, and / or embedded on-chip or package DRAM. In at least one embodiment, SoC interface 1637 can also enable communication with fixed function devices within the SoC, such as a camera image signal processor (ISP) and / or a video codec, and to use and / or implement global memory atoms that can be shared between graphics core 1600C and a CPU within the SoC. In at least one embodiment, SoC interface 1637 can also implement power management controls for graphics core 1600C and enable an interface between a clock domain of graphics core 1600C and other clock domains within the SoC. In at least one embodiment, SoC interface 1637 enables receiving command buffers from command streamer and global thread dispatcher, which are configured to provide commands and instructions to each of one or more graphics cores within a graphics processor. In at least one embodiment, commands and instructions can be dispatched to a media pipeline 1639 when media operations are to be performed, or to a geometry and fixed function pipeline (e.g., geometry and fixed function pipeline 1636, geometry and fixed function pipeline 1614) when graphics processing operations are to be performed.

[0297] In at least one embodiment, graphics microcontroller 1638 can be configured to perform various scheduling and management tasks for graphics core 1600C. In at least one embodiment, graphics microcontroller 1638 can perform graphics and / or compute workload scheduling on various graphics processing engines within execution unit (EU) arrays 1602A-1602F, 1604A-1604F in sub-cores 1601A-1601F. In at least one embodiment, host software executing on a CPU core of a SoC including graphics core 1600C can submit workloads for one of graphics processing units that invoke scheduling operations on appropriate graphics engines. In at least one embodiment, scheduling operations include determining which workload to run next, submitting a workload to a command streamer, pre-empting existing workloads running on an engine, monitoring progress of a workload, and notifying host software when a workload completes. In at least one embodiment, graphics microcontroller 1638 can also facilitate low power or idle states for graphics core 1600C, providing the ability to save and restore registers across low power state transitions independent of operating system and / or graphics driver software on the system.

[0298] In at least one embodiment, graphics core 1600C can have up to N more or less modular sub-cores than shown, each sub-core having the same design. For each set of N sub-cores, graphics core 1600C can also include shared function logic 1610, shared and / or cache memory 1612, geometry / fixed function pipeline 1614, and additional fixed function logic 1616 to accelerate various graphics and compute operations. In at least one embodiment, shared function logic 1610 can include logic units (e.g., samplers, math, and / or inter-thread communication logic) that are shared by each N sub-core within graphics core 1600C. In at least one embodiment, fixed, shared, and / or cache memory 1612 can be a last level cache memory for N sub-cores 1601A-1601F within graphics core 1600C and can also be used as shared memory that can be accessed by multiple sub-cores. In at least one embodiment, geometry / fixed function pipeline 1614 can be included instead of geometry / fixed function pipeline 1636 within fixed function block 1630, and can include similar logic units.

[0299] In at least one embodiment, graphics core 1600C includes additional fixed function logic 1616 that can include various fixed function acceleration logic used by graphics core 1600C. In at least one embodiment, additional fixed function logic 1616 includes an additional geometry pipeline used in position only shading. In position only shading, there are at least two geometry pipelines, while in full geometry and fixed function pipeline 1614, 1636, and cull pipeline, which is a trimmed down version of full geometry pipeline that can be included in additional fixed function logic 1616. In at least one embodiment, cull pipeline is a trimmed down version of full geometry pipeline. In at least one embodiment, full and cull pipelines can execute different instances of an application, each instance having a separate context. In at least one embodiment, position only shading can hide long cull runs of triangles that are discarded, which can complete shading earlier in some cases. In at least one embodiment, cull pipeline logic in additional fixed function logic 1616 can execute position shaders in parallel with a main application and generate critical results faster than full pipeline because cull pipeline takes and shades position attributes of vertices without performing rasterization and rendering pixels to a frame buffer. In at least one embodiment, cull pipeline can use generated critical results to compute visibility information for all triangles, regardless of whether those triangles are culled or not. In at least one embodiment, full pipeline, which can be referred to as replay pipeline in this case, can consume visibility information to skip culled triangles to only shade visible triangles that are ultimately passed to rasterization stage.

[0300] In at least one embodiment, additional fixed function logic 1616 can also include machine learning acceleration logic, such as fixed function matrix multiplication logic for implementing optimizations including for machine learning training or inferencing.

[0301] In at least one embodiment, within each graphics sub-core 1601A-1601F includes a set of execution resources, which can be used to perform graphics, media, and compute operations in response to requests by graphics pipeline, media pipeline, or shader program. In at least one embodiment, graphics sub-cores 1601A-1601F include multiple arrays of execution units 1602A-1602F, 1604A-1604F, thread dispatch and inter-thread communication (TD / IC) logic 1603A-1603F, 3D (e.g., texture) samplers 1605A-1605F, media samplers 1606A-1606F, shader processors 1607A-1607F, and shared local memory (SLM) 1608A-1608F. Arrays of execution units 1602A-1602F, 1604A-1604F each include multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations including graphics, media, and compute operations in response to shader programs. In at least one embodiment, TD / IC logic 1603A-1603F performs local thread dispatch and thread control operations for execution units within a sub-core and facilitate

[0302] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. In at least one embodiment, inferencing and / or training operations include machine learning training operations or machine learning inferencing operations, or both. In at least one embodiment, inference and / or training logic 615 include, without limitation, hardware and / or software. In at least one embodiment, inference and / or training logic 615 include, without limitation, one or more of application specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), math co-processors (MCPs), central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), artificial intelligence processing units (AIPUs), and / or similar. FIG. 6B and / or FIG. 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, portions of inference and / or training logic 615 can be incorporated into graphics processor 1610. In at least one embodiment, the training and / or inference techniques described herein can be implemented using one or more ALUs embodied in graphics processor 1612, graphics microcontroller 1638, geometry and fixed function pipeline 1614 and 1636, or other logic in FIG. 16B In at least one embodiment, the inference and / or training operations described herein can be accomplished by logic other than that illustrated in FIG. 6B and / or FIG. 6C In at least one embodiment, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not) that configure ALUs of graphics processor 1600C to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0303] FIG. 16D-16E Thread execution logic 1600D including an array of processing elements that each include a graphics processor core is shown in accordance with at least one embodiment. FIG. 16D At least one embodiment is shown in which thread execution logic 1600D is used. FIG. 16E Exemplary internal details of a processing element are shown in accordance with at least one embodiment.

[0304] As FIG. 16DAs shown in FIG. 16D, in at least one embodiment, thread execution logic 1600D includes a shader processor 1602, a thread dispatcher 1604, an instruction cache 1606, a scalable execution unit array including a plurality of execution units 1608A-1608N, one or more samplers 1610, a data cache 1612, and a data port 1614. In at least one embodiment, the scalable execution unit array can be dynamically scalable, to permit varying numbers of execution units (e.g., included as part of a processor core) to be used as found in different embodiments of central processing units (CPUs), graphics processing units (GPUs), and the like, to permit varying amounts of parallel processing to be used on different processing threads, to permit different numbers of threads to be processed simultaneously, and the like. In at least one embodiment, the scalable execution unit array includes any number of execution units 1608A-1608N. In at least one embodiment, execution units 1608A-1608N are microprocessors accessed via a link to an interconnect structure that is used to communicate with other processing and input / output resources.

[0305] In at least one embodiment, execution units 1608A-1608N are primarily used for executing shader programs. In at least one embodiment, shader processor 1602 can process various shader programs and dispatch execution threads associated with shader programs via thread dispatcher 1604. In at least one embodiment, thread dispatcher 1604 includes logic to arbitrate thread initiation requests from graphics and media pipelines and instantiate requested threads on one or more execution units 1608A-1608N. In at least one embodiment, in at least one embodiment, a geometry pipeline can dispatch vertex, tessellation or geometry shaders to thread execution logic for processing. In at least one embodiment, thread dispatcher 1604 can also handle runtime thread spawning instructions from an executing shader program.

[0306] In at least one embodiment, execution units 1608A-1608N support a wide variety of instructions that are capable of performing a wide variety of integer and floating point operations. In at least one embodiment, execution units 1608A-1608N can also include specialized support (e.g., built-in functions) for audio, video, and multimedia, image processing, statistics and financial applications, and Emotion Engine applications.

[0307] In at least one embodiment, each of execution units 1608A-1608N operate on arrays of data elements. In at least one embodiment, a number of data elements is a “execution size,” or number of channels for the instruction. In at least one embodiment, an execution channel refers to a logical unit of execution for data element access, masking, and flow control within instructions. In at least one embodiment, a number of channels can map to a number of physical Arithmetic Logic Units (ALUs) or Floating Point Units (FPUs) for a particular graphics processor. In at least one embodiment, execution units 1608A-1608N support integer and floating-point data types.

[0308] In at least one embodiment, the execution unit instruction set includes SIMD instructions. In at least one embodiment, various data elements can be stored as a packed data type within a register and the execution unit will process the various elements within the packed data type together. In at least one embodiment, when processing packed data elements, the execution unit can support parallel execution (e.g., warp execution) of various types of instructions - vector (e.g., SIMD) and scalar. The parallel execution can allow different codes to be executed in parallel on different data elements within the packed data type. In at least one embodiment, the parallel execution can be used to process the packed data elements in parallel.

[0309] In at least one embodiment, one or more execution units can be combined into a fused execution unit 1609A-1609N having thread control logic (1607A-1607N) for fused EU. In at least one embodiment, multiple EU can be combined into a group of EU. In at least one embodiment, each EU in a fused group of EU can be configured to execute a separate SIMD hardware thread. The number of EU in a fused group of EU can vary from one embodiment to another. In at least one embodiment, each EU can execute a variety of SIMD widths including, but not limited to, SIMD8, SIMD16, and SIMD32. In at least one embodiment, each fused graphics execution unit 1609A-1609N includes at least two execution units. In at least one embodiment, fused execution unit 1609A includes first EU 1608A, second EU 1608B, and thread control logic 1607A common to both first EU 1608A and second EU 1608B. In at least one embodiment, thread control logic 1607A controls threads executing on fused graphics execution unit 1609A, allowing each EU within fused execution units 1609A-1609N to use a common instruction pointer register for execution.

[0310] In at least one embodiment, one or more internal instruction caches (e.g., 1606) are included in thread execution logic 1600D to cache thread instructions for execution units. In at least one embodiment, one or more data caches (e.g., 1612) are included to cache thread data during thread execution. In at least one embodiment, a sampler 1610 is included to provide texture sampling for 3D operations and media sampling for media operations. In at least one embodiment, sampler 1610 includes specialized texture or media sampling functionality to process texture or media

[0311] During execution, in at least one embodiment, graphics and media pipelines send thread initiation requests to thread execution logic 1600D through thread spawn and dispatch logic. In at least one embodiment, once a set of geometric objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within shader processor 1602 is invoked to further compute output information and cause resulting values to be written to an output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In at least one embodiment, pixel or fragment shaders compute values of various vertex attributes to be interpolated across a rasterized object. In at least one embodiment, pixel processor logic within shader processor 1602 then executes a pixel or fragment shader program provided by an application program interface (API). In at least one embodiment, to execute a shader program, shader processor 1602 dispatches threads to execution units (e.g., 1608A) via thread dispatcher 1604. In at least one embodiment, shader processor 1602 uses texture sampling logic in sampler 1610 to access texture data stored in a texture map in memory. In at least one embodiment, arithmetic operations on texture data and input geometry data compute pixel color data for each geometric fragment, or discard one or more pixels for further processing.

[0312] In at least one embodiment, data port 1614 provides a memory access mechanism for thread execution logic 1600D to output processed data to memory for further processing on a graphics processor output pipeline. In at least one embodiment, data port 1614 includes or couples to one or more cache memories (e.g., data cache 1612) to cache data for memory access via data port.

[0313] As FIG. 16EAs shown, in at least one embodiment, graphics processing unit 1608 can include an instruction fetch unit 1637, a general register file array (GRF) 1624, an architectural register file array (ARF) 1626, a thread arbiter 1622, an issue unit 1630, a branch unit 1632, a set of SIMD floating point units (FPUs) 1634, and in at least one embodiment, a set of specialized integer SIMD ALUs 1635. In at least one embodiment, GRF 1624 and ARF 1626 include a set of general and architectural register files associated with each simultaneous hardware thread that can be active in graphics processing unit 1608. In at least one embodiment, each thread architectural state is maintained in ARF 1626, while data used during thread execution is stored in GRF 1624. In at least one embodiment, each thread’s execution state, including each thread’s instruction pointer, can be saved in thread-specific registers in ARF 1626.

[0314] In at least one embodiment, graphics processing unit 1608 has an architecture that is a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). In at least one embodiment, the architecture has a modular configuration that can be fine-tuned at design time based on a target number of simultaneous threads and a number of registers per execution unit, where execution unit resources are logically allocated for execution of multiple simultaneous threads.

[0315] In at least one embodiment, graphics processing unit 1608 can issue multiple instructions collectively, each of which can be different instructions. In at least one embodiment, thread arbiter 1622 of graphics processing unit thread 1608 can dispatch an instruction to one of issue unit 1630, branch unit 1632, or SIMD FPU 1632 for execution. In at least one embodiment, each execution thread can have access to 128 general registers in GRF 1624, where each register can store 32 bytes, which can be accessed as a SIMD 8-element vector of 32-bit data elements. In at least one embodiment, each execution unit thread can have access to 4 KB in GRF 1624, although embodiments are not limited thereto, and more or less register resources can be provided in other embodiments. In at least one embodiment, although the number of threads per execution unit can also vary according to embodiments, up to seven threads can be executed simultaneously. In at least one embodiment in which seven threads have access to 4 KB, GRF 1624 can store a total of 28 KB. In at least one embodiment, flexible addressing modes can allow registers to be addressed together to effectively establish wider registers or rectangular block data structures that represent strides.

[0316] In at least one embodiment, memory operations, sampler operations, and other longer-latency system communications are dispatched via a “send” instruction executed by message passing send unit 1630. In at least one embodiment, branch instructions are dispatched to a dedicated branch unit 1632 to facilitate SIMD divergence and eventual convergence.

[0317] In at least one embodiment, graphics execution unit 1608 includes one or more SIMD floating point units (FPU) 1634 to perform floating point operations in SIMD execution threads. In at least one embodiment, one or more FPU 1634 also support integer computation. In at least one embodiment, one or more FPU 1634 can SIMD execute up to M 32-bit floating point or integer operations, or up to 2M 16-bit integer or 16-bit floating point operations. In at least one embodiment, at least one of FPUs 1634 provides extended math capability to support high throughput of math functions and double precision 64-bit floating point. In at least one embodiment, there is also a set of 8-bit integer SIMD ALUs 1635 which can be specially optimized to accelerate machine learning training / inference applications, for example.

[0318] In at least one embodiment, an array of instances of graphics execution unit 1608 can be instantiated in a graphics sub-core group (e.g., a sub-slice) as described herein. In at least one embodiment, execution unit 1608 can execute instructions by threads that are split into different execution lanes.

[0319] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6I and / or 6J. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6I and / or 6J. In at least one embodiment, portions or all of inference and / or training logic 615 can be incorporated in graphics processing cluster 2110D. Moreover, in at least one embodiment, logic other than that shown in FIGS. 6I and / or 6J can be used to perform the inferencing and / or training operations described herein. FIG. 6B and / or FIG. 6C Inference and / or training operations described herein can be embodied solely in hardware, or can utilize both software and hardware, including e.g. programmable logic device (PLD), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or digital signal processor (DSP). In at least one embodiment, weight parameters can be stored in on-chip or off-chip memory and / or registers (shown or not) that configure ALUs of execution logic 1600D to perform one or more machine learning algorithms, neural network architectures, use cases, or training techniques introduced herein.

[0320] FIG. 17AA parallel processing unit (“PPU”) 1700A, in accordance with at least one embodiment, is shown. In at least one embodiment, PPU 1700A is configured with machine-readable code that, if executed by PPU 1700A, causes PPU 1700A to perform some or all of the processes and techniques described throughout this disclosure. In at least one embodiment, PPU 1700A is a multi-threaded processor implemented on one or more integrated circuit devices and utilizes multi-threading as a latency-hiding technique designed to process computer-readable instructions (also referred to as machine-readable instructions or simply instructions) that are executed in parallel on multiple threads. In at least one embodiment, a thread refers to an execution thread and is an instance of a set of instructions configured to be executed by PPU 1700A. In at least one embodiment, PPU 1700A is a graphics processing unit (“GPU”) configured to implement a graphics rendering pipeline for processing three-dimensional (“3D”) graphics data in order to generate two-dimensional (“2D”) image data for display on a display device such as a liquid crystal display (“LCD”) device. In at least one embodiment, PPU 1700A is used to perform computations such as linear algebraic operations and machine learning operations. FIG. 17A Example parallel processors are shown for illustrative purposes only and are to be construed as non-limiting examples of processor architectures contemplated within the scope of the present disclosure, and any suitable processor can be employed in addition to and / or in place of the same.

[0321] In at least one embodiment, one or more PPUs 1700A are configured to accelerate high performance computing (“HPC”), data center, and machine learning applications. In at least one embodiment, PPU 1700A is configured to accelerate deep learning systems and applications including the following non-limiting examples: autonomous vehicle platforms, deep learning, high-precision voice, image, and text recognition systems, intelligent video analytics, molecular simulations, drug discovery, disease diagnosis, weather forecasting, big data analytics, astronomy, molecular dynamics simulations, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations, among others.

[0322] In at least one embodiment, PPU 1700A includes, without limitation, input / output (“I / O”) units 1706, front-end units 1710, scheduler units 1712, work distribution units 1714, hub 1716, crossbar (“Xbar”) 1720, one or more general processing clusters (“GPCs”) 1718, and one or more partition units (“memory partition units”) 1722. In at least one embodiment, PPU 1700A connects with one or more host processors, or other PPUs 1700A, via one or more high-speed GPU interconnects (“GPU interconnects”) 1708. In at least one embodiment, PPU 1700A connects to a host processor or other peripherals via interconnect 1702. In at least one embodiment, PPU 1700A connects to a local memory comprising one or more memory devices (“memory”) 1704. In at least one embodiment, memory devices 1704 include, without limitation, one or more dynamic random access memory (“DRAM”) devices. In at least one embodiment, one or more DRAM devices are configured and / or configurable as high bandwidth memory (“HBM”) subsystems, with multiple DRAM dies stacked

[0323] In at least one embodiment, high-speed GPU interconnect 1708 can refer to a link-based parallel computer bus that systems use to scale, and includes one or more PPUs 1700A in conjunction with one or more central processing units (“CPUs”), supports cache coherence between PPUs 1700A and CPUs, and CPU mastering. In at least one embodiment, high-speed GPU interconnect 1708 transports data and / or commands through hub 1716 to other units of PPU 1700A, such as one or more copy engines, video encoders, video decoders, power management units, and / or other components not explicitly shown in FIG. 17A

[0324] In at least one embodiment, I / O unit 1706 is configured to facilitate communication between PPU 1700A and one or more other devices, such as host processor, other PPUs 1700A, and / or one or more of memory devices 1704. In at least one embodiment, I / O unit 1706 is configured to perform FIG. 17A ​The I / O units 1706 send and receive communications (e.g., commands, data) to and from the interconnect 1702. In at least one embodiment, the I / O units 1706 communicate directly with the host processor(s) over the interconnect 1702 or through one or more intermediate devices such as a memory hub. In at least one embodiment, the I / O units 1706 can communicate with one or more other processors (e.g., one or more PPUs 1700A) via the interconnect 1702. In at least one embodiment, the I / O units 1706 implement a Peripheral Component Interconnect Express (“PCIe”) interface for communications over a PCIe bus. In at least one embodiment, the I / O units 1706 implement an interface for communications with external devices.

[0325] In at least one embodiment, the I / O units 1706 decode packets received via the interconnect 1702. In at least one embodiment, at least some packets represent commands configured to cause the PPU 1700A to perform various operations. In at least one embodiment, the I / O units 1706 send decoded commands to various other units of the PPU 1700A as specified by the commands. In at least one embodiment, commands are sent to the front-end unit 1710 and / or to the hub 1716 or other units of the PPU 1700A such as one or more copy engines, video encoders, video decoders, power management units, etc. FIG. 17A In at least one embodiment, the I / O units 1706 are not explicitly shown in FIG. 17. In at least one embodiment, the I / O units 1706 are configured to route communications between various logical units of the PPU 1700A.

[0326] In at least one embodiment, a program executed by the host processor encodes a command stream in a buffer that provides a workload to the PPU 1700A for processing. In at least one embodiment, the workload includes instructions and data to be processed by those instructions. In at least one embodiment, the buffer is a region in memory that is accessible (e.g., read / write) by both the host processor and the PPU 1700A - the host interface unit can be configured to access memory requests transmitted via the I / O units 1706 over the interconnect 1702 to connect to the buffer in system memory. In at least one embodiment, the host processor writes the command stream to the buffer and then sends a pointer to the start of the command stream to the PPU 1700A, causing the front-end unit 1710 to receive the one or more command stream pointers and manage the one or more command streams, reading commands from the command stream and forwarding the commands to various units of the PPU 1700A.

[0327] In at least one embodiment, front-end unit 1710 is coupled to a scheduler unit 1712 which configures various GPCs 1718 to process tasks defined by one or more command streams. In at least one embodiment, scheduler unit 1712 is configured to track state information related to various tasks managed by scheduler unit 1712, where state information can indicate which task is assigned to which GPC 1718, whether a task is active or inactive, a priority associated with a task, and so forth. In at least one embodiment, scheduler unit 1712 manages multiple tasks that are executed on one or more GPCs 1718.

[0328] In at least one embodiment, scheduler unit 1712 is coupled to a work distribution unit 1714, which is configured to apportion tasks to be processed by GPCs 1718. In at least one embodiment, work distribution unit 1714 tracks a number of tasks received for processing by scheduler unit 1712 and manages a pending task pool and an active task pool for each GPC 1718. In at least one embodiment, the pending task pool includes a number of slots (e.g., 16 slots) that hold tasks assigned to be processed by a particular GPC 1718; the active task pool can include a number of slots (e.g., 4 slots) for tasks that are actively being processed by a GPC 1718, such that as one task completes processing in a GPC 1718, that task is evicted from the active task pool for that GPC 1718 and one of the other tasks from the pending task pool is selected and scheduled for processing on the GPC 1718. In at least one embodiment, if an active task is idle, for example while waiting for a data dependency to resolve, the active task is evicted from the GPC 1718 and returned to the pending task pool while another task is selected from the pending task pool and scheduled for processing on the GPC 1718.

[0329] In at least one embodiment, work distribution unit 1714 communicates with one or more GPCs 1718 via XBar 1720. In at least one embodiment, XBar 1720 is an interconnect network that couples many units of PPU 1700A to other units of PPU 1700A and can be configured to couple work distribution unit 1714 to a particular GPC 1718. In at least one embodiment, other units of one or more PPU 1700A can also be connected to XBar 1716 via hub 1716.

[0330] In at least one embodiment, tasks are managed by a scheduler unit 1712 and dispatched to one of GPCs 1718 by a work distribution unit 1714. GPCs 1718 are configured to process tasks and generate results. In at least one embodiment, results can be consumed by other tasks within GPC 1718, routed to different GPCs 1718 via XBar 1716 or stored in memory 1704. In at least one embodiment, results can be written to memory 1704 via a partition unit 1722, which implements a memory interface for reading and writing data to memory 1704. In at least one embodiment, results can be transmitted to another PPU 1704 or CPU via a high-speed GPU interconnect 1708. In at least one embodiment, PPU 1700A includes, without limitation, U partition units 1722, which is equal to a number of separate and distinct memory devices 1704 coupled to PPU 1700A. In at least one embodiment, partition units 1722 are described in more detail below in conjunction with FIG. 17C Partition units 1722 are described in more detail.

[0331] In at least one embodiment, a host processor executes a driver core that implements an application programming interface (API) that enables one or more applications executing on a host processor to schedule operations to be performed on PPU 1700A. In one embodiment, multiple compute applications are executed simultaneously by PPU 1700A and PPU 1700A provides isolation, quality of service (“QoS”), and independent address spaces for multiple compute applications. In at least one embodiment, an application generates instructions (e.g., in the form of API calls) that cause a driver core to generate one or more tasks for execution by PPU 1700A and driver core outputs tasks to one or more streams processed by PPU 1700A. In at least one embodiment, each task includes one or more related thread groups, which can be referred to as warps. In at least one embodiment, a warp includes multiple related threads (e.g., 32 threads) that can be executed in parallel. In at least one embodiment, cooperating threads can refer to multiple threads that include instructions for performing a task and exchanging data via shared memory, in conjunction with FIG. 17C Threads and cooperating threads are described in more detail in accordance with at least one embodiment.

[0332] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Inferences can be made by one or more of inference and / or training logic 615 as described below in conjunction with FIG. 6B and / or FIG. 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, a deep learning application processor is used to train a machine learning model, such as a neural network, to predict or infer information provided to PPU 1700A. In at least one embodiment, PPU 1700A is used to infer or predict information based on a trained machine learning model (e.g., neural network) that has been trained by another processor or system, or by PPU 1700A. In at least one embodiment, PPU 1700A can be used to perform one or more neural network use cases described herein.

[0333] FIG. 17B A general processing cluster (“GPC”) 1700B, according to at least one embodiment, is shown. In at least one embodiment, GPC 1700B is a GPC 1718 of GPU 1701. In at least one embodiment, each GPC 1700B includes, without limitation, a number of hardware units for processing tasks and each GPC 1700B includes, without limitation, a pipeline manager 1702, a pre-raster operations unit (“PROP”) 1704, a raster engine 1708, a work distribution crossbar (“WDX”) 1716, a memory management unit (“MMU”) 1718, one or more data processing clusters (“DPCs”) 1706, and any suitable combination of such components. FIG. 17A

[0334] In at least one embodiment, operation of GPC 1700B is controlled by pipeline manager 1702. In at least one embodiment, pipeline manager 1702 manages configuration of one or more DPCs 1706 to process tasks assigned to GPC 1700B. In at least one embodiment, pipeline manager 1702 configures at least one of one or more DPCs 1706 to implement at least a portion of a graphics rendering pipeline. In at least one embodiment, DPC 1706 is configured to execute vertex shader programs on a programmable streaming multi-processor (“SM”) 1714. In at least one embodiment, pipeline manager 1702 is configured to route packets received from a work distribution unit to appropriate logical units within GPC 1700B, and in at least one embodiment, some packets can be routed to fixed function hardware units in PROP 1704 and / or raster engine 1708 while other packets can be routed to DPCs 1706 for processing by a primitive engine 1712 or SM 1714. In at least one embodiment, pipeline manager 1702 configures at least one of DPCs 1706 to implement a neural network model and / or compute pipeline.

[0335] ​In at least one embodiment, PROP unit 1704 is configured to route data generated by raster engine 1708 and DPC 1706 to a raster operations (“ROP”) unit in partition unit 1722, in at least one embodiment, in accordance with FIG. 17A In more detail, in at least one embodiment, PROP unit 1704 is configured to perform optimizations for color blending, organize pixel data, perform address translations, and so forth. In at least one embodiment, raster engine 1708 includes, without limitation, a number of fixed function hardware units configured to perform various raster operations, and in at least one embodiment, includes, without limitation, a setup engine, a coarse raster engine, a cull engine, a clip engine, a fine raster engine, a primitive assembly engine, and any suitable combination thereof. In at least one embodiment, setup engine receives transformed vertices and generates plane equations associated with geometric primitives defined by the vertices; plane equations are passed to coarse raster engine to generate coverage information (e.g., x, y coverage masks for tiles) for the primitive; output from coarse raster engine is passed to cull engine where primitives associated with z-test failures are culled, and to clip engine where fragments that are outside of a viewing frustum are clipped. In at least one embodiment, clipped and culled fragments are passed to fine raster engine to generate attributes of pixel fragments based on plane equations generated by setup engine. In at least one embodiment, output from raster engine 1708 includes fragments to be processed by any suitable entity, such as by a fragment shader implemented within DPC 1706.

[0336] In at least one embodiment, each DPC 1706 included in GPC 1700B includes, without limitation, a M-Pipe Controller (“MPC”) 1710; a primitive engine 1712; one or more SMs 1714; and any suitable combination thereof. In at least one embodiment, MPC 1710 controls operation of DPC 1706, routing received packets from pipeline manager 1702 to appropriate units in DPC 1706. In at least one embodiment, packets associated with vertices are routed to primitive engine 1712, which is configured to fetch vertex attributes associated with the vertices from memory; in contrast, packets associated with a shader program can be transmitted to SM 1714.

[0337] In at least one embodiment, SM 1714 comprises, without limitation, a programmable streaming processor configured to process tasks represented by a plurality of threads. In at least one embodiment, SM 1714 is multi-threaded and configured to execute a number of threads (e.g., 32 threads) from a particular group of threads concurrently and implements a single instruction, multiple data (“SIMD”) architecture wherein each thread in a group of threads is configured to process a different data set based on same set of instructions. In at least one embodiment, all threads in a group of threads execute same instructions. In at least one embodiment, SM 1714 implements a single instruction, multiple thread (“SIMT”) architecture wherein each thread of a group of threads is configured to process a different data set based on same set of instructions, but where individual threads of group of threads are allowed to diverge during execution. In at least one embodiment, program counter, call stack, and execution state are maintained for each thread block, enabling concurrency between thread blocks and serial execution within thread blocks as threads within a thread block diverge. In another embodiment, program counter, call stack, and execution state are maintained for each individual thread, enabling equal concurrency between all threads within and across thread blocks. In at least one embodiment, execution state is maintained for each individual thread and threads that converge upon same instruction can be executed in parallel for better efficiency. At least one embodiment of SM 1714 is described in greater detail below.

[0338] In at least one embodiment, MMU 1718 interfaces with GPC 1700B and a memory partition unit (e.g., partition unit 1722) to provide virtual address to physical address translations, memory protections, and arbitration of memory requests. In at least one embodiment, MMU 1718 provides one or more translation lookaside buffers (TLBs) for performing translation of virtual addresses to physical addresses in memory. FIG. 17A

[0339] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and / or 6M. In at least one embodiment, deep learning application processor is used to train machine learning models, such as neural networks, to predict or infer information provided to GPC 1700B. In at least one embodiment, GPC 1700B is used to infer or predict information based on a machine learning model (e.g., neural network) that has been trained by another processor or system or by GPC 1700B. In at least one embodiment, GPC 1700B can be used to perform one or more neural network use cases described herein. ​

[0340] FIG. 17C A memory partition unit 1700C of a parallel processing unit (“PPU”) is shown in accordance with at least one embodiment. In at least one embodiment, memory partition unit 1700C includes, without limitation, a raster operations (“ROP”) unit 1702; a level two (“L2”) cache 1704; a memory interface 1706; and any suitable combination thereof. In at least one embodiment, memory interface 1706 is coupled to memory. In at least one embodiment, memory interface 1706 can implement a 32-, 64-, 128-, 1024-bit data bus, or similar implementation for high-speed data transfer. In at least one embodiment, a PPU includes U memory interfaces 1706, one memory interface 1706 per pair of partition units 1700C, with each pair of partition units 1700C connected to a corresponding memory device. In at least one embodiment, a PPU can be connected to up to Y memory devices, such as a high bandwidth memory stack or graphics double data rate version 5 synchronous dynamic random access memory (“GDDR5 SDRAM”), in at least one embodiment.

[0341] In at least one embodiment, memory interface 1706 implements a high bandwidth memory second generation (“HBM2”) memory interface, and Y is equal to half of U. In at least one embodiment, HBM2 memory stacks are located on a same physical package as a PPU, which can provide a large amount of power and save area compared to a GDDR5 SDRAM system. In at least one embodiment, each HBM2 stack includes, without limitation, four memory dies, and Y = 4, with each HBM2 stack including two 128-bit channels per die for a total of 8 channels and a 1024-bit data bus width. In at least one embodiment, memory supports a single error correction double error detection (“SECDED”) error correcting code (“ECC”) to protect data. In at least one embodiment, ECC provides higher reliability for compute applications that are sensitive to data corruption.

[0342] In at least one embodiment, a PPU implements a multi-level memory hierarchy. In at least one embodiment, memory partition unit 1700C supports a unified memory to provide a single unified virtual address space for central processing units (“CPUs”) and PPU memory, enabling data sharing between functional units of different processors, including a CPU and a PPU. In at least one embodiment, a frequency of access to memory locations by a PPU is tracked in order to ensure that memory pages frequently accessed by a PPU are moved to physical memory to which the PPU has direct access. In at least one embodiment, a high-speed GPU interconnect 1708 supports address translation services allowing a PPU to directly access a page table resident in memory and providing full access to a CPU’s memory space. In at least one embodiment, a PPU includes, without limitation, a 16 complex ALU 1710, a 16 texture ALU 1712, a 16 special functional unit 1714, and a thread intercept unit (“TIU”) 1716.

[0343] In at least one embodiment, the replication engine transfers data between multiple PPUs or between a PPU and a CPU. In at least one embodiment, the replication engine can generate page faults for addresses not mapped to page tables, and the memory partitioning unit 1700C then servicees the page faults, mapping the addresses to page tables, after which the replication engine performs the transfer. In at least one embodiment, multiple replication engines operating on fixed (in at least one embodiment, non-pageable) memory across multiple processors substantially reduce available memory. In at least one embodiment, in the event of a hardware page fault, an address can be passed to the replication engine regardless of whether a memory page resides, and the replication process is transparent.

[0344] According to at least one embodiment, from FIG. 17A Data from memory 1704 or other system memory is retrieved by memory partitioning unit 1700C and stored in L2 cache 1704, which is located on-chip and shared among various GPCs. In at least one embodiment, each memory partitioning unit 1700C includes, but is not limited to, at least a portion of the L2 cache associated with the corresponding memory device. In at least one embodiment, lower-level caches are implemented in various units within a GPC. In at least one embodiment, each SM 1714 may implement a Level 1 (“L1”) cache, wherein the L1 cache is a private memory dedicated to a specific SM 1714, and data is retrieved from L2 cache 1704 and stored in each L1 cache for processing within the functional units of the SM 1714. In at least one embodiment, L2 cache 1704 is coupled to memory interface 1706 and XBar 1720.

[0345] In at least one embodiment, ROP unit 1702 performs graphics raster operations associated with pixel colors such as color compression, pixel blending, and the like. In at least one embodiment, ROP unit 1702 performs depth tests in conjunction with raster engine 1708, receiving depth of a sample location associated with a pixel fragment from a culling engine of raster engine 1708. In at least one embodiment, a depth test is performed against a respective depth in a depth buffer at a sample location associated with a fragment. In at least one embodiment, if a fragment passes a depth test for a sample location, ROP unit 1702 updates the depth buffer and sends a result of the depth test to raster engine 1708. It will be appreciated that a number of partition units 1700C can be different than a number of GPCs, and thus, in at least one embodiment, each ROP unit 1702 can be coupled to each GPC. In at least one embodiment, ROP unit 1702 tracks groups received from different GPCs and determines to which result a result generated by ROP unit 1702 is routed through XBar 1720.

[0346] FIG. 17D A streaming multi-processor (“SM”) 1700D is shown in accordance with at least one embodiment. In at least one embodiment, SM 1700D is a FIG. 17BSM. In at least one embodiment, SM 1700D includes, without limitation, an instruction cache 1702; one or more scheduler units 1704; a register file 1708; one or more processing cores (“cores”) 1710; one or more special-function units (“SFUs”) 1712; one or more load / store units (“LSUs”) 1714; an interconnect network 1716; shared memory / level-one (“LI”) cache 1718; and any suitable combination thereof. In at least one embodiment, a work distribution unit dispatches tasks for execution on general processing clusters (“GPCs”) of parallel processing units (“PPUs”) and each task is assigned a specific data processing cluster (“DPC”) within a GPC and, if task is associated with a shader program, to one of SMs 1700D. In at least one embodiment, scheduler units 1704 receive tasks from work distribution unit and manage dispatch of instructions to one or more thread blocks for execution on SM 1700D. In at least one embodiment, scheduler units 1704 schedule thread blocks to be executed to be dispatched as thread warps of parallel threads, with each thread block being assigned at least one thread warp. In at least one embodiment, each thread warp executes a thread. In at least one embodiment, scheduler units 1704 manage a plurality of different thread blocks, assign thread warps to different thread blocks, and then dispatch instructions from different ones of the plurality of different thread blocks to various functional units (e.g., processing cores 1710, SFUs 1712, and LSUs 1714) in each clock cycle.

[0347] In at least one embodiment, a cooperative group can refer to a programming model for organizing groups of communication threads that allows developers to express the granularity at which threads are communicating, enabling richer, more efficient parallel decomposition. In at least one embodiment, a cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. In at least one embodiment, an application program of the programming model provides a single, simple construct for synchronizing cooperative threads: a barrier across all threads of a thread block (e.g., a syncthreads() function). However, in at least one embodiment, a programmer can define thread groups at less than a thread block granularity and synchronize within defined groups to achieve higher performance, design flexibility, and software reuse in the form of collective group-wide function interfaces. In at least one embodiment, cooperative groups enable programmers to explicitly define thread groups at sub-block (down to a single thread in at least one embodiment) and multi-block granularity and perform collective operations, such as synchronizing threads in a cooperative group. In at least one embodiment, a programming model supports clean composition across software boundaries, so that library and utility functions can safely synchronize in their local environment without having to make assumptions about convergence. In at least one embodiment, cooperative group primitives enable new patterns of cooperative parallelism, including but not limited to producer-consumer parallelism, opportunistic parallelism, and global synchronization across a grid of thread blocks.

[0348] In at least one embodiment, a scheduler unit 1706 is configured to send instructions to one or more of the functional units, and a scheduler unit 1704 includes, without limitation, two scheduler units 1706 that enable two different instructions from the same warp to be scheduled in each clock cycle. In at least one embodiment, each scheduler unit 1704 includes a single scheduler unit 1706 or an additional scheduler unit 1706.

[0349] In at least one embodiment, each SM 1700D includes, without limitation, a register file 1708 that provides a set of registers for functional units of the SM 1700D. In at least one embodiment, register file 1708 is split between functional units as is, with a dedicated portion of the register file 1708 for each functional unit. In at least one embodiment, register file 1708 is split between different thread blocks executed by the SM 1700D, with the register file 1708 providing temporary storage for operands connected to data paths of the functional units. In at least one embodiment, each SM 1700D includes, without limitation, a plurality L of processing cores 1710. In at least one embodiment, SM 1700D includes, without limitation, a large number (e.g., 128 or more) of different processing cores 1710. In at least one embodiment, each processing core 1710 includes, without limitation, a full-pipe, single precision, double precision, and / or mixed precision processing unit including, without limitation, a floating point arithmetic logic unit and an integer arithmetic logic unit. In at least one embodiment, floating point arithmetic logic units implement IEEE 754-2008 standards for floating point arithmetic. In at least one embodiment, processing cores 1710 include, without limitation, 64 single-precision (32-bit) floating point cores, 64 integer cores, 32 double-precision (64-bit) floating point cores, and 8 tensor cores.

[0350] In at least one embodiment, one or more tensor cores are included in processing cores 1710. In at least one embodiment, tensor cores are configured to perform deep learning matrix arithmetic, such as convolution operations for neural network training and inferencing. In at least one embodiment, each tensor core operates on 4x4 matrices and performs matrix multiplication and accumulation operations D = A x B + C, where A, B, C, and D are 4x4 matrices.

[0351] In at least one embodiment, matrix multiplication inputs A and B are 16-bit floating point matrices, and accumulation matrices C and D are 16-bit floating point or 32-bit floating point matrices. In at least one embodiment, a tensor core performs 32-bit floating point accumulation operations on 16-bit floating point input data. In at least one embodiment, 16-bit floating point multiplication uses 64 operations and results in a full precision product, which is then accumulated with other intermediate products using 32-bit floating point addition for 4x4x4 matrix multiplication. In at least one embodiment, tensor cores are used to perform larger two-dimensional or higher dimensional matrix operations composed of these smaller elements. In at least one embodiment, an API such as CUDA 9 C++ API exposes specialized matrix load, matrix multiply and accumulate, and matrix store operations to efficiently use tensor cores from a CUDA-C++ program. In at least one embodiment, at CUDA level, a warp level interface assumes a 16x16 size matrix across all 32 warp threads.

[0352] In at least one embodiment, each SM 1700D includes, without limitation, M SFUs 1712 to perform special functions (e.g., certain math functions, exponentials, logarithms, etc.). In at least one embodiment, SFUs 1712 include, without limitation, tree traversal units configured to traverse a hierarchical tree data structure. In at least one embodiment, SFUs 1712 include, without limitation, texture units configured to perform texture mapping filtering operations. In at least one embodiment, texture units are configured to load a texture map (e.g., a 2D array of texture pixels) from memory and sample the texture map to produce sampled texture values for use in a shader program executed by SM 1700D. In at least one embodiment, texture maps are stored in shared memory / L1 cache 1718. In at least one embodiment, texture units use mip-maps (e.g., different levels of detail for a texture map) to perform texture operations such as filtering operations according to at least one embodiment. In at least one embodiment, each SM 1700D includes, without limitation, two texture units.

[0353] In at least one embodiment, each SM 1700D includes, without limitation, N LSUs 1714 that implement load and store operations between shared memory / L1 cache 1718 and register file 1708. In at least one embodiment, interconnect network 1716 connects each of the functional units to register file 1708, and LSUs 1714 to register file 1708 and shared memory / L1 cache 1718. In at least one embodiment, interconnect network 1716 is a cross-bar switch that can be configured to connect any of the functional units to any of the registers in register file 1708, and to connect LSUs 1714 to registers in register file 1708 and to memory locations in shared memory / L1 cache 1718.

[0354] In at least one embodiment, shared memory / L1 cache 1718 is an array of on-chip memory that, in at least one embodiment, allows data storage and communication between SMs 1700D and graphics engines, and between threads within SM 1700D. In at least one embodiment, shared memory / L1 cache 1718 includes, without limitation, 128 KB of storage and is located on a path from SM 1700D to partition units. In at least one embodiment, shared memory / L1 cache 1718 is used for caching reads and writes, in at least one embodiment. In at least one embodiment, one or more of shared memory / L1 cache 1718, L2 cache, and memory are backing stores.

[0355] In at least one embodiment, combining data cache and shared memory functionality into a single memory block provides improved performance for both types of memory accesses. In at least one embodiment, capacity is used by programs that do not use shared memory or used as a cache, e.g., if shared memory is configured to use half of capacity, and textures and load / store operations can use remaining capacity. According to at least one embodiment, integration within shared memory / L1 cache 1718 enables shared memory / L1 cache 1718 to be used as a high-throughput pipeline for streaming data, while providing high bandwidth and low latency access to frequently reused data. In at least one embodiment, when configured for general purpose parallel computation, a simpler configuration can be used compared to graphics processing. In at least one embodiment, fixed function graphics processing units are bypassed, creating a more straightforward programming model. In at least one embodiment, in a general purpose parallel computation configuration, work distribution unit allocates and distributes blocks of threads directly to DPCs. In at least one embodiment, threads in a block execute a general purpose program, using a unique thread ID in calculations to ensure that each thread generates a unique result, using SM 1700D to execute the program and perform calculations, using shared memory / L1 cache 1718 to communicate between threads, and using LSU 1714 to read and write global memory through shared memory / L1 cache 1718 and memory partition unit. In at least one embodiment, when configured for general purpose parallel computation, SM 1700D writes to scheduler unit 1704 commands that can be used to launch new work on DPCs.

[0356] In at least one embodiment, PPU is included in a desktop computer, laptop computer, tablet computer, server computer, supercomputer, smart- phone (e.g., a wireless, hand-held device), personal digital assistant (“PDA”), digital camera, vehicle, head mounted display, hand-held electronic device, etc. or is coupled to such devices. In at least one embodiment, PPU is implemented on a single semiconductor

[0357] In at least one embodiment, PPU can be included on a graphics card that includes one or more memory devices. In at least one embodiment, graphics card can be configured to interface with a PCIe slot on a motherboard of a desktop computer. In at least one embodiment, PPU can be an integrated graphics processing unit (“iGPU”) included in a chipset of a motherboard.

[0358] Inference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and 6M. FIG. 6B and / or FIG. 6C Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6L and 6M. In at least one embodiment, deep learning application processor is used to train machine learning models, such as neural networks, to predict or infer information provided to SM 1700D. In at least one embodiment, SM 1700D is used to infer or predict information based on a machine learning model (e.g., neural network) that has been trained by another processor or system or by SM 1700D. In at least one embodiment, SM 1700D can be used to perform one or more neural network use cases described herein.

[0359] In at least one embodiment, a single semiconductor platform can refer to a sole unitary integrated circuit or chipset that can contain one or more processors, memory, input / output circuitry, and various other components, depending on its specifications. In at least one embodiment, a multicore processor can be used that

[0360] In at least one embodiment, computer programs in form of executable code or computer control logic algorithms in machine readable, readable executable code or computer control logic algorithms are stored in main memory 4ee04 and / or secondary storage. In at least one embodiment, if executed by one or more processors, the computer programs enable system 4ee00 to perform various functions. In at least one embodiment, memory 4ee04, storage, and / or any other storage is a possible example of computer readable media. In at least one embodiment, secondary storage can refer to any suitable storage device or system for storing data, such as hard disk drives and / or removable storage drives, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, a digital versatile disk (“DVD”) drive, a recording device, a universal serial bus (“USB”) flash memory, etc. In at least one embodiment, architecture and / or functionality of various previous figures are implemented in an environment of a CPU 4ee02; parallel processing system 4ee12; integrated circuit capable of at least part of capabilities of two CPUs 4ee02; parallel processing system 4ee12; chipset (e.g., a group of integrated circuits designed to work together as a unit and sold as a unit, etc.); and any suitable combination of integrated circuits.

[0361] In at least one embodiment, the architecture and / or functionality of the various previous figures is implemented in the environment of a general purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, an application-specific system, etc. In at least one embodiment, computer system 4ee00 can take the form of a desktop computer, laptop computer, tablet computer, server, supercomputer, smart telephone (e.g., wireless, hand-held device), personal digital assistant (“PDA”), digital camera, vehicle, head mounted display, hand-held electronic device, mobile telephone device, television, workstation, game console, embedded system, and / or any other type of logic.

[0362] In at least one embodiment, parallel processing system 4ee12 includes, without limitation, a plurality of parallel processing units (“PPUs”) 4ee14 and associated memory 4ee16. In at least one embodiment, PPUs 4ee14 are connected to a host processor or other peripheral devices via an interconnect 4ee18 and switch 4ee20 or multiplexer. In at least one embodiment, parallel processing system 4ee12 allocates computational tasks to PPUs 4ee14 that can be parallelized, for example, as part of a distribution of computational tasks across multiple graphics processing unit (“GPU”) thread blocks. In at least one embodiment, memory is shared and accessed by some or all of PPUs 4ee14 (e.g., for read and / or write access), although such shared memory can incur a performance penalty relative to using local memory and registers resident on PPUs 4ee4ee. In at least one embodiment, operations of PPUs 4ee14 are synchronized by using commands such as __syncthreads(), where all threads in a block (e.g., executing across multiple PPUs 4ee14) reach a certain code execution point before proceeding.

[0363] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure, as defined in the appended claims.

[0364] Unless otherwise indicated or contradicted by context, the use of the terms "a" and "one" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated or contradicted by context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (meaning "including, but not limited to") unless otherwise noted or contradicted by context. The term "connected" (when used without modification) is to be construed as partly or wholly encompassed in, attached to, or joined together, even if there are some intervening items. Unless otherwise indicated herein, the reference herein to a range of values is intended merely as a shorthand way of referring individually to each numerical value falling within the range, and each individual value is incorporated into the specification as if it were individually recited herein. Unless otherwise indicated or contradicted by context, the use of the term "set" (e.g., "set of items") or "subset" is to be construed as a non-empty set of one or more members. Also, unless otherwise indicated or contradicted by context, a "subset" of a corresponding set does not necessarily represent a proper subset of the corresponding set, but rather the subset and the corresponding set can be equal.

[0365] Unless explicitly indicated otherwise or contradicted by context, conjunction phrases such as "at least one of A, B, and C" or "at least one of A, B, and C" are to be construed in context as generally used to mean items, clauses, etc., that can be A or B or C, or any non-empty subset of the set of A and B and C. For example, in the illustrative example of a set having three members, the conjunction phrases "at least one of A, B, and C" and "at least one of A, B, and C" refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunction language can not be intended to imply that at least one of A, at least one of B, and at least one of C are required to exist. Additionally, unless otherwise indicated or contradicted by context, "plurality" denotes a state of plurality (e.g., "a plurality of items" denotes a plurality of items). A plurality is at least two items, but can be more if explicitly indicated or indicated by context. Furthermore, unless otherwise indicated or clear from context, "based on" means "based at least in part on" rather than "based only on."

[0366] The operations of a process described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In at least one embodiment, a process, such as those processes outlined herein (or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions (e.g., computer programs, one or more applications, or one or more programs) to carry out the operations of the process, and the processes can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) running on a hardware of the one or more computer systems, collectively. In at least one embodiment, the code is stored on a computer-readable storage medium, e.g., in the form of a computer program, which is executable by one or more processors of the computer system. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues). In at least one embodiment, code (e.g., executable or source code) is stored on a set of one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) having stored thereon executable instructions that, as a result of being executed by one or more processors of a computer system (in at least one embodiment, as a result of being executed by one or more processors as a consequence of being executed by a computer system), cause the computer system to perform operations described herein. In at least one embodiment, a set of non-transitory computer-readable storage media includes multiple non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media of the multiple non-transitory computer-readable storage media lack all of the code, with the multiple non-transitory computer-readable storage media col...

Claims

1. A data center cooling system comprising: a supplemental refrigerant cooling loop such that heat is absorbed from one or more server components of a data center and an auxiliary cooling loop under peak operating conditions, the one or more server components coupled to the auxiliary cooling loop and the supplemental refrigerant cooling loop located within the data center, and the auxiliary cooling loop having a different piping than the supplemental refrigerant cooling loop.

2. The data center cooling system of claim 1, further comprising: a learning subsystem including at least one processor, the learning subsystem to evaluate a temperature within one or more servers or racks and provide an output associated with the at least one temperature to facilitate movement of a refrigerant in the supplemental refrigerant cooling loop by controlling one or more flow controllers associated with the supplemental refrigerant cooling loop.

3. The data center cooling system of claim 2, further comprising: a cold plate associated with the one or more server components; the one or more flow controllers to facilitate movement of the refrigerant through the cold plate; and the learning subsystem to execute a machine learning model to: process the temperature using a plurality of neuron levels of the machine learning model having the temperature and having a previous associated flow rate of the refrigerant; and provide the output associated with a flow rate of the refrigerant to the one or more flow controllers based on an evaluation of the previous associated flow rate.

4. The data center cooling system of claim 1, further comprising: one or more first flow controllers to modify a first flow rate of a coolant associated with the auxiliary cooling loop; and one or more second flow controllers to modify a second flow rate of a refrigerant in the supplemental refrigerant cooling loop.

5. The data center cooling system of claim 1, further comprising: one or more first flow controllers to maintain a coolant at a first flow rate using the auxiliary cooling loop; one or more second flow controllers to modify a refrigerant to one or more flow rates within the supplemental refrigerant cooling loop in response to a temperature sensed within one or more servers or racks of the data center such that heat is absorbed from the auxiliary cooling loop.

6. The data center cooling system of claim 1, further comprising: an individual cold plate associated with the one or more server components and having a first path for a refrigerant of the supplemental refrigerant cooling loop and a second path for a coolant associated with the auxiliary cooling loop.

7. The data center cooling system of claim 1, further comprising: the supplemental refrigerant cooling loop having direct or indirect contact with the auxiliary cooling loop in a manifold such that heat is absorbed from the auxiliary cooling loop.

8. The data center cooling system of claim 1, further comprising: ​ ​ the supplemental refrigerant cooling loop to absorb heat from one or more server components of a data center during a startup process of the one or more server components; and the auxiliary cooling loop to remain inactive during the startup process and to activate upon receiving an indication of temperature stabilization associated with the one or more server components.

9. At least one processor for use in a cooling system, comprising: at least one logic unit to control one or more flow controllers associated with a supplemental refrigerant cooling loop to cause heat to be absorbed from one or more server components of a data center and an auxiliary cooling loop during peak operating conditions, the one or more server components coupled to the auxiliary cooling loop and the supplemental refrigerant cooling loop located within the data center, and the auxiliary cooling loop having a different piping than the supplemental refrigerant cooling loop.

10. The at least one processor of claim 9, further comprising: a learning subsystem to evaluate a temperature within one or more servers or racks and to provide an output associated with at least one temperature to facilitate movement of a refrigerant in the supplemental refrigerant cooling loop by controlling the one or more flow controllers associated with the supplemental refrigerant cooling loop.

11. The at least one processor of claim 9, further comprising: a learning subsystem to execute a machine learning model to: process a temperature within one or more servers or racks using a plurality of neuron levels of the machine learning model having a prior associated flow rate of a temperature and a refrigerant; and provide an output associated with a flow rate of the refrigerant to one or more flow controllers associated with the supplemental refrigerant cooling loop based on an evaluation of the prior associated flow rate.

12. The at least one processor of claim 11, further comprising: an instruction output to communicate the output to the one or more flow controllers to modify a first flow rate of the refrigerant while maintaining a second flow rate of a coolant associated with the auxiliary cooling loop.

13. The at least one processor of claim 9, further comprising: the at least one logic unit adapted to receive a temperature value from a temperature sensor associated with the auxiliary cooling loop and to facilitate movement of a refrigerant within the supplemental refrigerant cooling loop to cool the one or more server components or a coolant associated with the auxiliary cooling loop.

14. A method for cooling a data center, comprising: providing a supplemental refrigerant cooling loop to cause heat to be absorbed from one or more server components of a data center and an auxiliary cooling loop during peak operating conditions, the one or more server components coupled to the auxiliary cooling loop and the supplemental refrigerant cooling loop located within the data center, and the auxiliary cooling loop having a different piping than the supplemental refrigerant cooling loop.

15. The method of claim 14, further comprising: enabling the one or more flow controllers to receive input from at least one temperature sensor associated with the one or more server components within the data center, or associated with the supplemental refrigerant cooling loop; and enabling the one or more flow controllers to facilitate movement of refrigerant in the supplemental refrigerant cooling loop to absorb heat from the one or more server components.

16. The method of claim 14, wherein the supplemental refrigerant cooling loop absorbs heat from the one or more server components by simultaneously cooling the one or more components using coolant associated with the supplemental cooling loop and refrigerant from the refrigerant cooling loop.

17. The method of claim 16, wherein the supplemental refrigerant cooling loop absorbs heat from the one or more server components by simultaneously cooling the one or more components using coolant associated with the supplemental cooling loop and cooling the coolant of the supplemental cooling loop by circulating the refrigerant in the refrigerant cooling loop proximate to at least one portion of the supplemental cooling loop.

18. The method of claim 14, further comprising: using a learning subsystem comprising at least one processor to evaluate a temperature of the one or more server components within the data center using a flow rate associated with refrigerant in the supplemental refrigerant cooling loop; providing an output associated with a flow rate; and controlling one or more flow controllers to facilitate movement of the refrigerant in the supplemental refrigerant cooling loop.

19. The method of claim 18, further comprising: associating a cold plate with the one or more server components; facilitating movement of the refrigerant through the cold plate using the one or more flow controllers; and executing a machine learning model of the learning subsystem to: process the temperature using a plurality of neuron levels of the machine learning model having a previous associated flow rate of the refrigerant and the temperature; and provide an output associated with the flow rate of the refrigerant to the one or more flow controllers based on an evaluation of the previous associated flow rate.

20. The method of claim 18, further comprising: modifying a second flow rate of the refrigerant in response to the output to facilitate movement of the refrigerant in the refrigerant cooling loop according to the flow rate.

21. The method of claim 14, further comprising: modifying a first flow rate of coolant associated with the supplemental cooling loop using one or more first flow controllers; and modifying a second flow rate of refrigerant in the supplemental refrigerant cooling loop using one or more second flow controllers.

22. The method of claim 14, further comprising: maintaining coolant at a first flow rate using the supplemental cooling loop using one or more first flow controllers; and ​ ​ In response to a sensed temperature from the auxiliary cooling loop or the one or more server components, one or more second flow controllers are used to modify the refrigerant to be at one or more flow rates within the supplemental refrigerant cooling loop to cause heat to be absorbed from the one or more server components.

Images