Intelligent server-level testing of data center cooling systems

Abstract

A heat load system for testing a data center liquid cooling system is disclosed. The system includes a server chassis having at least one thermal feature associated with at least one cooling feature and at least one flow controller, wherein the at least one thermal feature and the at least one flow controller are adjustable to generate cooling stress on the data center liquid cooling system.

Description

[0001] Cross-references to related applications

[0002] This application is a PCT application of U.S. Patent Application No. 16 / 984,650, filed on August 4, 2020. The disclosure of that application is incorporated herein by reference in its entirety for all purposes. Technical Field

[0003] At least one embodiment relates to the testing of a data center liquid cooling system. In at least one embodiment, a heat load system for testing a data center liquid cooling system includes a server chassis, having at least one thermal feature associated with at least one cooling feature and at least one flow controller, wherein the at least one thermal feature and the at least one flow controller are adjustable to generate cooling stress on the data center liquid cooling system. Background Technology

[0004] Data center cooling systems typically use fans to circulate air through server components. Some supercomputers or other high-capacity computers may use water or other cooling systems besides air cooling to extract heat from server components or racks within the data center to areas outside the data center. A cooling system may include coolers within the data center area, which may extend to areas outside the data center. These external areas may include cooling towers or other external heat exchangers that receive heated coolant from the data center and disperse the heat into the environment (or external cooling medium) via forced air or other means before the cooled coolant is recirculated back into the data center. In one example, coolers and cooling towers together form a cooling facility with pumps that respond to temperatures applied to the data center as measured by external devices. Air cooling systems alone may not extract enough heat to support effective or efficient cooling of a data center, and liquid cooling systems may not be economical for the needs of a data center. Attached Figure Description

[0005] Various embodiments according to this disclosure will be described with reference to the accompanying drawings, in which:

[0006] Figure 1 It is a block diagram of an example data center with a cooling system that requires improvement, as described in at least one embodiment;

[0007] Figure 2 This is a block diagram illustrating server-level characteristics of a heat load system for testing a data center liquid cooling system according to at least one embodiment;

[0008] Figure 3 This is a block diagram illustrating the component-level characteristics of a heat load system for testing a data center liquid cooling system according to at least one embodiment;

[0009] Figure 4 This is a block diagram illustrating rack-level characteristics of a heat load system for testing a data center liquid cooling system according to at least one embodiment;

[0010] Figure 5 It is available for use or manufacture according to at least one embodiment. Figures 2-4 and Figures 6A-17D The process flow of the cooling system method;

[0011] Figure 6A An example data center is shown, in which data can be used Figures 2-5 At least one embodiment;

[0012] Figure 6B , Figure 6C The inference and / or training logic according to various embodiments is shown, for example... Figure 6A The heat load system used in at least one embodiment of this disclosure for enabling and / or supporting testing of data center liquid cooling systems;

[0013] Figure 7A This is a block diagram illustrating an exemplary computer system, which may be a system of interconnected devices and components formed by a processor, a system-on-a-chip (SOC), or some combination thereof. The processor may include an execution unit to execute instructions according to at least one embodiment to support and / or enable the heat load system described herein for testing a data center liquid cooling system.

[0014] Figure 7B This is a block diagram illustrating an electronic device that utilizes a processor to support and / or enable a heat load system for testing a data center liquid cooling system, according to at least one embodiment.

[0015] Figure 7C This is a block diagram illustrating an electronic device that utilizes a processor to support and / or enable a heat load system for testing a data center liquid cooling system, according to at least one embodiment.

[0016] Figure 8 Another example computer system according to at least one embodiment is shown to implement various processes and methods described throughout the disclosure for testing the heat load system of a data center liquid cooling system;

[0017] Figure 9A An exemplary architecture is shown in which a GPU is communicatively coupled to a multi-core processor via a high-speed link to enable and / or support a heat load system for testing a data center liquid cooling system according to at least one embodiment disclosed herein.

[0018] Figure 9BAdditional details are shown regarding the interconnection between a multi-core processor and a graphics acceleration module according to an exemplary embodiment;

[0019] Figure 9C Another exemplary embodiment according to at least one embodiment disclosed herein is shown, wherein an accelerator integrated circuit is integrated in a processor for enabling and / or supporting a thermal load system for testing a data center liquid cooling system;

[0020] Figure 9D An exemplary accelerator integrated circuit chip 990 is shown according to at least one embodiment disclosed herein for enabling and / or supporting a heat load system for testing a data center liquid cooling system;

[0021] Figure 9E Additional details are shown for an exemplary embodiment of a shared model according to at least one embodiment disclosed herein, to enable and / or support a heat load system for testing a data center liquid cooling system;

[0022] Figure 9F Additional details are shown for an exemplary embodiment of a unified memory according to at least one embodiment disclosed herein, which can be addressed via a common virtual memory address space for accessing physical processor memory and GPU memory to enable and / or support thermal load systems for testing data center liquid cooling systems;

[0023] Figure 10A Exemplary integrated circuits and associated graphics processors according to embodiments described herein are shown for testing the heat load system of a data center liquid cooling system;

[0024] Figure 10B-10C An exemplary integrated circuit and associated graphics processor according to at least one embodiment are shown to support and / or enable a heat load system for testing a data center liquid cooling system;

[0025] Figures 10D-10E Additional exemplary graphics processor logic according to at least one embodiment is shown to support and / or enable a heat load system for testing a data center liquid cooling system;

[0026] Figure 11A A block diagram of a computing system according to at least one embodiment is shown, the computing system being used to support and / or enable a heat load system for testing a data center liquid cooling system;

[0027] Figure 11B A parallel processor according to at least one embodiment is shown, the parallel processor being used to support and / or enable a heat load system for testing a data center liquid cooling system;

[0028] Figure 11C It is a block diagram of a partitioning unit according to at least one embodiment;

[0029] Figure 11D A graphics multiprocessor according to at least one embodiment is shown for testing the heat load system of a data center liquid cooling system;

[0030] Figure 11E A graphics multiprocessor according to at least one embodiment is shown;

[0031] Figure 12A A multi-GPU computing system according to at least one embodiment is shown;

[0032] Figure 12B It is a block diagram of a graphics processor according to at least one embodiment;

[0033] Figure 13 This is a block diagram illustrating a microarchitecture for a processor according to at least one embodiment, the processor including logic circuitry for executing instructions;

[0034] Figure 14 A deep learning application processor according to at least one embodiment is shown;

[0035] Figure 15 It is a block diagram of a neuromorphic processor according to at least one embodiment;

[0036] Figure 16A This is a block diagram of a processing system according to at least one embodiment;

[0037] Figure 16B It is a block diagram of a processor having one or more processor cores, an integrated memory controller and an integrated graphics processor according to at least one embodiment;

[0038] Figure 16C It is a block diagram of the hardware logic of a graphics processor core according to at least one embodiment;

[0039] Figure 16D-16E The diagram illustrates thread execution logic according to at least one embodiment, which includes an array of processing elements of a graphics processor core.

[0040] Figure 17A A parallel processing unit according to at least one embodiment is shown;

[0041] Figure 17B A general processing cluster is shown according to at least one embodiment;

[0042] Figure 17C A memory partition unit of a parallel processing unit according to at least one embodiment is shown; and

[0043] Figure 17D A streaming multiprocessor according to at least one embodiment is shown. Detailed Implementation

[0044] Air cooling for high-density servers may not be efficient, or may be ineffective due to sudden high heat demands caused by changes in the computing load in existing computing components. However, because the requirements depend on variations or tend to vary from minimum to maximum different cooling requirements, suitable cooling systems must be used to meet these requirements economically. For medium to high cooling requirements, liquid cooling systems can be used. Different cooling requirements also reflect different thermal characteristics of the data center. In at least one embodiment, the heat generated from components, servers, and racks is collectively referred to as thermal characteristics or cooling requirements, because cooling requirements must comprehensively address thermal characteristics. In at least one embodiment, the thermal characteristics or cooling requirements of the cooling system are the heat or cooling requirements generated by components, servers, or racks associated with the cooling system, and may be part of the components, servers, and racks within the data center.

[0045] The process of commissioning a liquid cooling system for a data center may include testing the entire available load of the liquid cooling system. This testing can be conducted in a test environment using data center equipment prior to effective data center operation. The test can be designed to handle the heat load of the entire data center. However, such a process cannot be applied to individual heat loads within components, servers, and racks. The heat load of the entire data center cannot be traced down to individual heat loads of components, servers, and racks. Furthermore, hot-swapping of components, servers, or racks may impose additional heat loads on the liquid cooling system that were not previously handled or were unintentional, and this may not be determined by studying the heat load of the entire data center. Moreover, specific requirements for cooling servers and cold plates using liquid cooling systems are still not budgeted; therefore, associated data centers or computing equipment within data centers (such as graphics processing units (GPUs), central processing units (CPUs), and switches) may benefit, even when hot-swapping operations involve exchanging each of these components with different capacity components, resulting in the new components generating less or more heat than expected. Even after such hot-swapping of components, it will be expected to remain economical.

[0046] In at least one embodiment, a heat load system for testing a data center liquid cooling system is disclosed. The heat load system includes a server chassis having at least one thermal feature associated with at least one cooling feature and at least one flow controller. The at least one thermal feature and the at least one flow controller are adjustable to generate cooling stress on the data center liquid cooling system. In at least one embodiment, cooling stress represents the amount (flow rate or volume) of coolant or working fluid required to handle the maximum or variation of heat generated by data center equipment incorporated into the data center. In at least one embodiment, the heat load system is capable of replicating the maximum or variation of heat generated by data center equipment incorporated into the data center. In at least one embodiment, the heat load system is a liquid heat load bundle (LTLB). The LTLB supports liquid flow regulation and replicates or emulates control features of an actual operating server with computing components on it. The computing components may be GPUs, CPUs, switches, or other computing components that benefit from liquid cooling (e.g., via coolant). Liquid cooling allows coolant to reach the computing components directly. Servers incorporating such liquid cooling are part of a rack with hoses and quick disconnects, which allows fluid to be pushed into the server.

[0047] In at least one embodiment, the shape and size of the LTLB allow for a liquid connection between the LTLB within the data center rack and a manifold present in the rack. This liquid connection allows working fluid (e.g., coolant for a secondary cooling loop) to flow into the LTLB. In at least one embodiment, each LTLB is designed to provide a specifically designed thermal load that translates into cooling stress on the data center liquid cooling system and provides designed flow resistance that increases the cooling stress. Cooling stress can cause temperature rises and pressure drops equivalent to those of an actual server with computing components such as the aforementioned GPU, CPU, and switches, which will generate heat and can cause temperature rises and pressure drops in the manner of the LTLB device. In at least one embodiment, the LTLB supports existing working fluids used in hot-plug testing, which can be the same as the working fluids in an actual server. Therefore, after testing for an actual server and the results showing the expected cooling effect as predicted by the LTLB, the LTLB can be swapped out.

[0048] In at least one embodiment, the LTLB of this disclosure enables testing of liquid cooling systems before deploying or commissioning servers and racks in existing data centers. While there may be rack-sized load groups for testing liquid cooling systems across an entire data center, and while there may be load groups for testing air-cooled systems, in at least one embodiment, the LTLB of this invention satisfies at least a specific requirement of testing server-level liquid cooling requirements and testing these server-level liquid cooling requirements in a hot-swappable environment. Furthermore, in at least one embodiment, the LTLB of this invention enables modular rack-level testing via multiple LTLB server trays forming an entire rack.

[0049] In at least one embodiment, this disclosure provides a form factor-based LTLB, either a server chassis or a tray, whose assembly can be used to replicate rack-level load groups, but adaptability from the server level and above through this modular design is an added-value aspect. In at least one embodiment, the LTLB has thermal pads (also referred to as thermal features) and associated cold plates (also referred to as cooling features), wherein the thermal pads are activated or deactivated, and on the server side, the cooling capacity of the liquid cooling system is stress-tested. The stress test is also referred to as asserting cooling stress on the liquid cooling system. In at least one embodiment, the LTLB includes at least one controller that supports features for controlling or regulating the liquid flow of the liquid cooling system. The control or regulation of liquid flow replicates the practical use of plug-and-play (hot-swappable) aspects. In at least one embodiment, the hot-swapping, insertion, or removal of replication components (e.g., GPUs / CPUs / switches associated with racks having hoses and quick connectors, which alter or affect the cooling medium entering the rack server) is controlled or regulated. Therefore, by connecting and disconnecting the hose, and simply by controlling or regulating the same or similar coolant flowing through the LTLB, similar pressure, temperature, or flow rate changes of the coolant within an actual server can be affected. Thus, the LTLB applies pressure to the cooling system to replicate these scenarios in order to predict problems that may arise during the commissioning of the cooling system in an online data center. In at least one embodiment, the LTLB is designed to provide the design heat load and to provide design flow resistance that produces temperature rise and pressure drop equivalent to those in an actual electronic server.

[0050] Figure 1This is a block diagram of an example data center 100 described in at least one embodiment, the data center 100 having a cooling system to be improved. The data center 100 may be one or more rooms 102 with racks 110 and auxiliary equipment to house one or more servers located on one or more server trays. The data center 100 is supported by a cooling tower 104 located outside the data center 100. The cooling tower 104 dissipates heat from within the data center 100 by acting on a main cooling circuit 106. Furthermore, a cooling distribution unit (CDU) 112 is used between the main cooling circuit 106 and a second or auxiliary cooling circuit 108 to enable heat extraction from the second or auxiliary cooling circuit 108 to the main cooling circuit 106. In one aspect, the auxiliary cooling circuit 108 can be connected to various pipes as needed, all the way to the server trays. Circuits 106, 108 are shown in the line drawing, but those skilled in the art will recognize that one or more pipework features can be used. In one example, flexible polyvinyl chloride (PVC) pipes may be used with the associated pipes to allow fluid to travel forward in each of the circuits 106, 108. In at least one embodiment, one or more coolant pumps may be used to maintain a pressure difference within circuits 106, 108 based on temperature sensors located in various locations including a room, one or more racks 110, and / or in server cases or server trays within the racks 110, to enable the coolant to move.

[0051] In at least one embodiment, the coolant in the primary cooling circuit 106 and the secondary cooling circuit 108 may be at least water and additives (e.g., ethylene glycol or propylene glycol). In operation, each of the primary and secondary cooling circuits has its own coolant. In one aspect, the coolant in the secondary cooling circuit may be dedicated to the requirements of components in the server trays or racks 110. The CDU 112 is capable of precise control of the coolant independently or concurrently in the circuits 106, 108. For example, the CDU may be adapted to control the flow rate to properly distribute one or more coolants to extract heat generated within the rack 110. Furthermore, additional flexible conduits 114 are provided from the secondary cooling circuit 108 to enter each server tray and supply coolant to electrical and / or computing components. In this disclosure, "electrical and / or computing components" is used interchangeably to refer to heat-generating components that benefit from current data center cooling systems. A conduit 118 forming part of the secondary cooling circuit 108 may be referred to as a room manifold. Specifically, pipe 116, extending from pipe 118, can also be part of auxiliary cooling circuit 108 and may be referred to as a row manifold. Pipe 114, as part of auxiliary cooling circuit 108, enters the rack and may be referred to as a rack cooling manifold. Further, row manifold 116 extends along one row in data center 100 to all racks. The pipes in auxiliary cooling circuit 108, including manifolds 118, 116, and 114, can be improved by at least one embodiment of this disclosure. Cooler 120 may be provided in the main cooling circuit of data center 102 to support cooling before the cooling tower. In the presence of an additional circuit in the main control circuit, those skilled in the art will recognize and understand that this disclosure provides cooling outside the rack and outside the auxiliary cooling circuit; and can be used in conjunction with the main cooling circuit of this disclosure.

[0052] In at least one embodiment, during operation, heat generated within the server tray in rack 110 can be transferred via the flexible conduit of the row manifold 114 in the second cooling circuit 108 to the coolant discharged from rack 110. Specifically, a second coolant from CDU 112 for cooling rack 110 (in auxiliary cooling circuit 108) moves toward rack 110. The second coolant from CDU 112 passes through one side of the room manifold having pipe 118, via row manifold 116, to one side of rack 110, and via pipe 114 through one side of the server tray. The used second coolant (or the second coolant carrying heat from the computing components) is discharged from the other side of the server tray (e.g., entering from the left side of the rack and, after circulating through the server tray or components on the server tray, exiting from the right side of the rack of the server tray). The used second coolant discharged from the server tray or from rack 110 flows out from different sides (e.g., the outlet side) of pipe 114 and moves to the parallel outlet side of row manifold 116. The used second coolant moves in the parallel portion of room manifold 118 in the opposite direction to the incoming second coolant (which may also be a newer second coolant) and moves from row manifold 116 toward CDU 112.

[0053] In at least one embodiment, the used second coolant exchanges heat with the main coolant in the main cooling circuit 106 via CDU 112. The used second coolant is refreshed (e.g., relatively cooled compared to the temperature of the stage where the second coolant was consumed) and prepared to be circulated back to the computing unit via the second cooling circuit 108. Various flow and temperature control features in CDU 112 are capable of controlling the heat exchanged from the used second coolant or the second coolant flowing into and out of CDU 112. CDU 112 is also capable of controlling the flow of the main coolant in the main cooling circuit 106.

[0054] Figure 2This is a block diagram illustrating a server-level feature 200 for testing the thermal load of a data center liquid cooling system according to at least one embodiment. The server-level feature 200 includes a server, server tray, or server enclosure 202 having one or more cold plates 210A-210D; one or more cooling loops 212A, 212B; at least one server manifold 204 having an integrated or separate cooling distribution unit (CDU); and inlet pipes 206 and outlet pipes 208 to the server manifold 204, respectively. In at least one embodiment, the server manifold 204 is designed to distribute coolant to one or more cooling loops 212A, 212B and to distribute coolant from one or more cooling loops 212A, 212B, and is therefore integrated therein with the CDU. Furthermore, adjacent to one or more cold plates 210A-210D, thermal features 220A-220D are capable of testing the data center liquid cooling system by asserting cooling stress on the system. One or more cooling circuits, such as cooling circuit 212B, may have similar features to those described with respect to cooling circuit 212A. Furthermore, in at least one embodiment, aspects of cooling circuit 212B can be controlled independently compared to aspects of cooling circuit 212A.

[0055] In at least one embodiment, server-level feature 200 represents or includes at least a portion of a heat load system for testing a data center liquid cooling system. The server, server tray, or server enclosure 202 has at least one thermal feature 220A, 220B, 220C, 220D associated with at least one cooling feature 220A, 220B, 220C, 220D. Furthermore, the server, server tray, or server enclosure 202 has at least one flow controller 222A, 222B. In at least one embodiment, the at least one flow controller 222A, 222B may be a micropump or a valve that can be remotely controlled to provide control or regulation of coolant flow, as described in this disclosure. In at least one embodiment, the at least one flow controller 222A, 222B may be located at the inlet or outlet of pipes 212A, 212B. In at least one embodiment, when located at the outlet of pipes 212A, 212B, the at least one flow controller 222A, 222B allows the action of asserting the intake or expulsion of coolant by a micropump, or the action of opening the coolant flow using a valve. In at least one embodiment, when located at the inlet of pipes 212A, 212B, at least one flow controller 222A, 222B allows the action of pumping coolant asserted by a micropump, or the action of opening a valve to allow coolant to flow in.

[0056] In at least one embodiment, at least one flow controller 222A, 222B may be located at (or associated with) server manifold 204, or at (or associated with) one or more cold plates 210A-210B. In at least one embodiment, a single flow controller associated with a cold plate connected in series with another cold plate, via intermediate conduit 216, may facilitate or control the flow of coolant through the two cold plates. Coolant may flow from server manifold 204, through inlet conduit 214, through cold plate 210A, through intermediate conduit 216, through cold plate 210B, through outlet conduit 218, and back to server manifold 204. In at least one embodiment, coolant flows from the CDU of server manifold 204. In at least one embodiment, at least one thermal feature 220A, 220B, 220C, 220D, and at least one flow controller 222A, 222B are adjustable to generate cooling stress on the data center liquid cooling system. In at least one embodiment, the at least one thermal feature 220A, 220B, 220C, 220D is located below the cold plates 210A-210D in a manner similar to how data center equipment is coupled to the cold plates, so as to enable heat dissipation through the cold plates and associated coolant.

[0057] In at least one embodiment, the adjustable action of at least one thermal feature 220A, 220B, 220C, 220D and at least one flow controller 222A, 222B is to change the heat generated by at least one thermal feature 220A, 220B, 220C, 220D, or to change the flow of coolant through at least one flow controller 222A, 222B. Figure 2 Only some of at least one flow controller 222A, 222B are shown, but there may be flow controllers for each channel (whether inlet or outlet channel). In this way, in at least one embodiment associated with at least one flow controller 222A, 222B, the flow of coolant can be initiated, disabled, or controlled to a defined flow rate (or flow rate) for one or more cold plates 210A-D. In at least one embodiment, the heat generated by at least one thermal feature 220A, 220B, 220C, 220D can be stopped, initiated, or controlled at a defined heating rate (or temperature). In at least one embodiment, the control or regulation of the liquid flow rate (flow velocity or flow rate) replicates practical use cases in a plug-and-play (hot-swappable) manner.

[0058] In at least one embodiment, the control or regulation of flow rate or volume, and the control or regulation of the generated heat, replicate the hot-plugging, insertion, or removal of components (e.g., GPUs / CPUs / switches associated with racks having hoses and quick-disconnect mechanisms), thereby altering or influencing the flow of cooling medium into the rack server. Therefore, similar pressure, temperature, or flow rate variations of the coolant can be achieved using server chassis 202, as this is similar to the limitations of actual servers performing data center operations and being subject to the actions of connecting and disconnecting hoses (e.g., the inlets and outlets of pipes 212A, 212B) that may be required for new installations, removals, or hot-plugging. In at least one embodiment, cooling stress can be applied to the relevant data center liquid system for testing purposes simply by controlling or regulating the same or similar coolant flowing through the LTLB represented by server chassis 202 having server feature 200.

[0059] Therefore, in at least one embodiment, the adjustable action of at least one thermal feature 220A, 220B, 220C, 220D, and at least one flow controller 222A, 222B, enables the cooling stress on the cooling system to replicate many scenarios that may occur in a data center, such as during routine operation or during the installation, removal, or hot-swapping of data center equipment. Testing can accurately predict problems and their solutions when commissioning the cooling system in an online data center. In at least one embodiment, the LTLB, represented by at least server-level feature 200, is designed to provide the design thermal load and to provide design flow resistance that produces equivalent temperature rise and pressure drop as in a real electronic server.

[0060] In at least one embodiment, the thermal load system including server-level feature 200 may include a heating coil or element having the functionality of at least one thermal feature 220A, 220B, 220C. The heating element may be adjustable to different heating levels, including being completely off or on to the maximum heating level. In at least one embodiment, such as Figure 2 As shown, heating coils or elements forming at least one thermal feature 220A, 220B, 220C, 220D are positioned adjacent to cold plates 210A, 210B, 210C, 210D, and the cold plates 210A, 210B, 210C, 210D have inlet channels 224A and outlet channels 224B to support the inflow and outflow of coolant using one or more inlet pipes or conduits 214, intermediate pipes or conduits 216, and outlet pipes or conduits 218. Each of the channels 224A, 224B supports the circulation of cooling medium from the data center liquid cooling system and can be associated with a respective flow controller 226. In at least one embodiment, as Figure 2As shown, the flow controller 226 can control one or more of the channels 224A and 224B, or control the flow of coolant through the channels 224A and 224B.

[0061] In at least one embodiment, the flow controller 226 includes a valve disposed at least within or associated with a channel, and the valve can be adjusted to control the coolant flow rate. In at least one embodiment, the flow controller 226 includes a micropump to change the flow rate of the coolant fluid. In at least one embodiment, the flow controller is selected to match the characteristics of the channel. In at least one embodiment, the flow controller 226 may be the same as or different from flow controllers 222A and 222B. Similarities or differences may relate to the type of channel, the size of the channel, the type of coolant, or the type of control provided in each of the various channels in the liquid cooling system.

[0062] In at least one embodiment, the heat load system, such as server chassis 202, further includes at least one processor or is associated with at least one processor to control heating coils and at least one flow controller. In at least one embodiment, the at least one processor may be... Figure 14 The processor 1400 can use neurons 1502 and their components implemented with circuits or logic, including such... Figure 15 The document describes one or more arithmetic logic units (ALUs). In at least one embodiment, the heating coil has a temperature range that replicates that of the data center equipment, and at least one flow controller can be used to control the flow rate of the data center liquid cooling system connected to the data center equipment via a circuit breaker or liquid pipeline.

[0063] In at least one embodiment, a heat load system, such as server chassis 202, is associated with a second server chassis having a second thermal characteristic, a second cooling characteristic, and a second flow controller, such that the cooling stress becomes rack-level cooling stress on a data center liquid cooling system. In at least one embodiment, a plurality of server chassis, each similar to server chassis 202, are stacked on a rack (e.g., Figure 1 The available slots in the racks (and together they replicate the rack-level cooling stress on the data center liquid cooling system).

[0064] In at least one embodiment, the heat load system, such as including server chassis 202, further includes at least one first cold plate forming or being associated with at least one cooling feature or being associated with at least one first cold plate forming or being associated with at least one cooling feature or being associated with at least one thermal feature. As with respect to the cold plates 210A-210D discussed, these cold plates may be at least one cooling feature, but heat sinks capable of dissipating at least some of the generated heat in conjunction with the liquid cooling system may also be used as separate cooling features. In at least one embodiment, when this disclosure is used in a hot-swappable environment, at least one second cold plate may be present associated with data center equipment performing data center operations. Unlike at least one first cold plate which is used solely for testing cooling stress on a data center liquid cooling system, at least one second cold plate provides actual cooling for the data center equipment during active operation. This feature demonstrates the ability to test component, server, and rack-level cooling stress before associating new components, new servers, or new racks with an existing data center with a commissioning liquid cooling system.

[0065] In at least one embodiment, the heat load system, such as server chassis 202, further includes at least one cold plate forming or associated with at least one cooling feature or a portion thereof, the at least one cooling feature or a portion thereof being associated with at least one thermal feature. A heating element forms at least one thermal feature, wherein the heating element is associated with a socket on a board within the server chassis and contributes to cooling stress on the data center liquid cooling system. In at least one embodiment, this allows server chassis 202 to utilize existing circuitry on the operating system board to provide at least the power to regulate the heat generated from the heating element. Therefore, server chassis 202 can utilize existing data center technologies or features instead of a dedicated, feature-rich server chassis 202. This also enables the true replication of an operating server chassis or operating components.

[0066] Figure 3This is a block diagram illustrating component-level features 300 of a thermal load system for testing a data center liquid cooling system according to at least one embodiment. In at least one embodiment, the thermal load system, for example, includes a server chassis 202, and further includes or is associated with an operable or non-operable data center device 302 within a receptacle. In at least one embodiment, the operable data center device 302 is a GPU, CPU, or switch, unassigned but usable as a basis for asserting cooling stress on the data center liquid cooling system. In at least one embodiment, the data center device 302 is non-operable as a GPU, CPU, or switch because it is a virtual component with dimensions and connectors located within a receptacle of a board in the server chassis 202, but it can be used as a basis for asserting cooling stress on the data center liquid cooling system and can be assigned to make it operable for data center operations.

[0067] In at least one embodiment, a thermal insulation structure 304 is provided between the data center device 302 and the heating element or coil 306, wherein the heating element or coil 306 is a heating feature used to assert cooling stress on the liquid cooling system. The data center device 302 is protected by the thermal insulation structure 304 to avoid additional heat generated on the heating element 306. This allows the test feature to be removed upon completion of testing, and the data center device 302 to be used by switching the device to an operational mode based on a previous inoperable (or test) mode within the socket. In at least one embodiment, the inoperable mode is the unassigned mode of the data center device, while the operational mode is the assigned mode of the data center device. In at least one embodiment, the heating element 306 is thus powered directly from the input power supply 322 via circuit board line 326; or indirectly powered from the socket 322 via circuit board line 328, the socket 322 receiving power from the input power supply 322 via circuit board line 324.

[0068] In at least one embodiment, a thermal interface material 310 is disposed on a cover 308 on the heating element 306. The thermal interface material 310 may be a silver-based thermal paste interface between the heating element 306 and at least one cold plate 312. The at least one cold plate 312 includes, or is associated with, an inlet pipe or conduit 314, a through pipe or conduit 318, and an outlet pipe or conduit 316. In at least one embodiment, the through pipe or conduit 318 is not a straight channel, but rather forms a tortuous channel within the cold plate 312.

[0069] In at least one embodiment, the heat load system, including or associated with component-level feature 300 and server-level feature 200, further includes at least one cold plate 312 forming or forming part of at least one cooling feature, the at least one cooling feature being associated with at least one hot feature 306. In at least one embodiment, the heat load system, including or associated with component-level feature 300 and server-level feature 200, further includes conduits 314, 316 associated with at least one cold plate 312 to allow the inflow and outflow of coolant from the data center liquid cooling system. In at least one embodiment, the thermal insulation structure 304 is composed of a plastic material, such as a plastic insulator; the heating element 306 is composed of a ceramic material, such as a ceramic heating plate; the covering or cover material 308 is composed of copper, such as a copper block; and the thermal interface material 310 is composed of silver-based paste.

[0070] Figure 4 This is a block diagram illustrating a rack-level feature 400 of a heat load system for testing a data center liquid cooling system according to at least one embodiment. In at least one embodiment, the heat load system, including or associated with rack-level feature 400, further includes at least one power distribution unit (PDU) 410 associated with at least one thermal feature to modify at least one thermal characteristic of the at least one thermal feature to induce cooling stress on the data center liquid cooling system. In at least one embodiment, at least one thermal feature is located within one or more operable or inoperable server trays 408. In at least one embodiment, power and communication lines 412 from PDU 412 to each server tray 408 enable the initiation of controls or regulation associated with flow controllers and thermal features within one or more operable or inoperable server trays 408.

[0071] In at least one embodiment, PDU 410 is associated with a first bracket 404 of rack 402, while rack manifold 414 is associated with a second bracket 406 of rack 402. In at least one embodiment, rack manifold 414 may include or be integrated with a rack-level CDU. In at least one embodiment, rack manifold 414 is associated with an inlet conduit or pipe 418 and an outlet conduit or pipe 420 to allow coolant from the auxiliary cooling loop to reach server tray 408, and additionally to one or more cooling features of server tray 408. In at least one embodiment, one or more flow controllers 422A, 422B may be provided to control the coolant inlet conduit 418 and outlet conduit 420.

[0072] In at least one embodiment, the PDU 410 or Data Center Management System (DMS) includes a learning subsystem having at least one processor to evaluate cooling stress in a test or commissioning environment. The test environment is used to track the learning subsystem, while the commissioning environment can be used to find new features to add to data center operations and solutions for implementing cooling systems. In at least one embodiment, cooling stress can be represented at least by the temperature applied to at least one thermal feature and the flow rate or volume associated with at least one flow controller. In the commissioning or test environment, the learning subsystem can provide an output associated with at least one of the flow rate or volume representing expected cooling for at least one input temperature. In at least one embodiment, the at least one input temperature can be the rated temperature of data center equipment to be incorporated into the data center.

[0073] In at least one embodiment, the PDU 410 or DMS, including a learning subsystem, is adapted to execute a machine learning model. The machine learning model uses multiple neuron levels in the machine learning model, having temperature and a previously associated flow rate or flow rate, to process the temperature of the learning subsystem. The machine learning model is adapted to provide an output associated with at least one of the flow rate or flow rate to at least one controller. In at least one embodiment, the output is provided after an evaluation of the temperature having a previously associated flow rate or flow rate. Thus, in at least one embodiment, the historical operating condition of the changed or altered temperature, or the temperature that caused the start-up cooling, can be the temperature used in the evaluation, while the associated flow rate or previously associated flow rate can be the coolant volume or flow rate provided for initiating cooling or causing the temperature to change or be altered.

[0074] In at least one embodiment, the volume or flow rate of coolant provided to initiate cooling or cause a temperature transition or change can be calculated with reference to a desired cooling at at least one input temperature. In at least one embodiment, the learning subsystem can be configured using a deep learning application processor (e.g., Figure 14 The processor 1400 in the middle can be used to implement it, and it can also be implemented using neurons 1502 and their components implemented with circuits or logic, including such as Figure 15 One or more arithmetic logic units (ALUs) as described in the document.

[0075] In at least one embodiment, at least one processor can be used to test a cooling system by asserting cooling stresses on the cooling system. In at least one embodiment, at least one processor can use a deep learning application processor (e.g., Figure 14 The processor 1400 in the processor implements and can use the neuron 1502 and its components implemented with circuits or logic, the circuits or logic including such Figure 15The description refers to one or more arithmetic logic units (ALUs). In at least one embodiment, at least one processor therefore has at least one logic unit to control movement associated with at least one flow controller to cool the server chassis. The server chassis has at least one thermal feature associated with at least one cooling feature and at least one flow controller. The at least one thermal feature and at least one flow controller are adjustable to generate cooling stress on a data center liquid cooling system.

[0076] In at least one embodiment, at least one processor includes the aforementioned learning subsystem for evaluating cooling stress on a cooling system. The cooling stress is represented at least in part by the temperature applied to at least one thermal feature and a flow rate or volume associated with at least one flow controller. The at least one processor can provide an output associated with at least one of the flow rate or volume representing expected cooling for at least one input temperature. The output is provided in a training or debugging environment, as discussed elsewhere in this disclosure.

[0077] In at least one embodiment, at least one processor executes a machine learning model for a learning subsystem. The at least one processor processes the temperature using multiple neuron levels in the machine learning model, which have a temperature and a previously associated flow rate or flow rate. The at least one processor provides an output associated with at least one of the flow rate or flow rate to at least one controller. The output is provided after the temperature has been evaluated using the previously associated flow rate or flow rate of the cooling system.

[0078] In at least one embodiment, at least one processor includes an instruction output for transmitting an output associated with at least one of flow rate or flow rate to the at least one controller. Furthermore, the processor's instruction output, such as pins on a connector bus or balls in a ball grid array, enables the output to communicate with one or more flow controllers to modify the flow rate of coolant associated with a cooling loop and, in response to the output, generate a determined flow rate or flow rate of coolant. Additionally, during training, the output may also be associated with a thermal feature to generate cooling stress, in part, by increasing or decreasing the heat generated by the thermal feature. The generated heat may be a determined amount of heat generated to replicate one or more data center devices at different performance levels. In the training environment, the determined flow rate or flow rate of coolant may shut down or require the thermal feature of coolant to identify or replicate problems that may arise or exist in the operation of a data center with data center devices.

[0079] In at least one embodiment, at least one processor includes at least one logic unit adapted to receive temperature values ​​from a temperature sensor associated with data center equipment in a debugging or training (also known as a test) environment. The at least one processor is adapted to facilitate the movement of coolant at a threshold including at least one flow rate or volume. The threshold may include a range of values ​​representing a minimum and a maximum value of coolant operation through at least one flow controller.

[0080] In one embodiment, at least one processor for a cooling system is disclosed. In at least one embodiment, the at least one processor may be part of a test system for the cooling system. The at least one processor is used to train a system, such as a learning subsystem suitable for testing and debugging a cooling system. In at least one embodiment, the at least one processor, including at least one logic unit, is used to train one or more neural networks with hidden layers of neurons to evaluate cooling stress on the cooling system. Cooling stress is represented at least in part by the temperature applied to at least one thermal feature and the flow rate or flow rate associated with at least one flow controller. The at least one processor, through at least one logic unit, provides an output associated with at least one of the flow rate or flow rate representing expected cooling for at least one input temperature, such as the rated temperature of a data center device under operating conditions.

[0081] Figure 5 It is available for use or manufacture according to at least one embodiment. Figures 2-4 and Figures 6A-17D The process flow of the cooling system method 500 includes the following steps: Step 502 provides a server chassis having at least one thermal feature associated with at least one cooling feature and at least one flow controller. Step 504 allows adjustment of the at least one thermal feature and at least one flow controller. In at least one embodiment, adjustment is performed by providing at least one processor to provide an output that performs one or more actions, including changing the temperature associated with the at least one thermal feature and changing the flow rate or volume associated with the at least one flow controller. In at least one embodiment, as per [reference to...] Figures 2-4 The processor discussed includes a learning subsystem for executing a machine learning model, such as a neural network, for performing training within the machine learning model, and for providing output to at least one thermal feature and at least one flow controller. Therefore, changes in temperature or flow rate can generate cooling stress in a data center liquid cooling system. Thus, at least one thermal feature and at least one flow controller are adjustable to generate cooling stress in the data center liquid cooling system.

[0082] In at least one embodiment, method 500 includes a step 506 for determining a cooling requirement associated with at least one thermal feature and at least one flow controller. In at least one embodiment, via step 504, the temperature is a first asserted temperature, and the flow rate or volume is the first asserted flow rate or volume of the at least one flow controller. In step 506, the cooling requirement may be determined. The determination in step 506 may be the difference between the asserted first temperature and the sensed temperature, reflecting the action of the first asserted coolant. The determination of the cooling requirement in step 506 is further evaluated in step 508 to verify whether the asserted flow rate or volume is insufficient to handle the first asserted temperature (the cooling requirement is not met). In at least one embodiment, step 508 may be a timing step occurring over multiple time periods, with step 506 occurring at least once in each period. Steps 506 and 508 may be cycled until the multiple time periods are completed and the verification is complete while the cooling requirement is not met.

[0083] In at least one embodiment, step 510 adjusts at least one thermal feature and at least one flow controller to generate cooling stress on the data center liquid cooling system until cooling requirements are met. In at least one embodiment, the cooling requirements still exceed the capabilities of the data center liquid cooling system for the data center equipment to be deployed in the data center. In at least one embodiment, method 500 can be repeated by adjusting mechanical and physical features in the data center liquid cooling system. In at least one embodiment, the goal of method 500 is to assert cooling stress on the data center liquid cooling system to identify available cooling boundaries. In at least one embodiment, cooling stress means that there is at least one assertion of a temperature to the thermal feature that the cooling system cannot handle.

[0084] In at least one embodiment, the learning subsystem may be used in conjunction with steps 504-510 to continuously adjust the asserted temperature and / or flow rate (or volume) and track the performance of the cooling system. In at least one embodiment, along with at least a thermal characteristic, the learning subsystem is capable of acting as a data center device and is capable of associating different temperature ranges with the rated temperature of actual data center devices (e.g., GPUs, CPUs, and switches referenced elsewhere in this disclosure). The learning subsystem learns the operation of the data center devices and is capable of replicating their thermal responses such that at least one thermal characteristic is presented as a data center device for that thermal characteristic. In at least one embodiment, the learning subsystem is also capable of replicating the flow characteristics of a cooling system with interruptions. In at least one embodiment, the learning subsystem is capable of completely terminating fluid or adjusting the cooling flow rate or volume of the cooling system via the output of one or more flow controllers. The termination of fluid or the adjustment of the flow rate or volume can indicate previous flow problems within the cooling system and can be used to assert cooling stress on the cooling system in a test or training environment.

[0085] In at least one embodiment, the learning subsystem is capable of responding to stimuli from the cooling system. In at least one embodiment, the learning subsystem is capable of learning from a test or training environment the ability of an existing cooling system to respond to asserted thermal characteristics from heating coils, and is capable of using the asserted thermal characteristics and the capabilities of the existing cooling system to provide coolant, specifically addressing temperature inputs from data center equipment in a commissioning environment. The commissioning environment relates to handling new cooling requirements for data center equipment used in an existing cooling system. In at least one embodiment of the response characteristics, the learning subsystem first asserts different thermal characteristics of the heating coils, such as maximum heat (or temperature), minimum heat (or temperature), and intermediate heat (or intermediate temperature); and secondly, learns the cooling system's response using the available existing flow rates and volumes of the coolant in the existing cooling system. In at least one embodiment, the learning subsystem asserts existing flow rates and volumes to see if they handle the different thermal characteristics.

[0086] In at least one embodiment, the learning subsystem using the response characteristics is able to learn a temperature range threshold (reflecting the rated maximum temperature of the nearby data center equipment) reflected by a value below the maximum heat asserted to the heating coil, to allocate the maximum flow rate and volume of coolant to the cold plate associated with the heating coil, which may exceed the effective capacity of the cooling system. In at least one embodiment, the temperature range threshold may also reflect a value above the minimum heat asserted, and / or may reflect an intermediate heat value asserted to the heating characteristics to ensure the capability of existing cooling systems.

[0087] In at least one embodiment, the temperature range threshold may also reflect one or more temperature change rates, increasing or decreasing, to identify the response speed of the cooling system by asserting the coolant flow rate or flow rate. In at least one embodiment, the learning subsystem identifies the maximum available cooling from the cooling system (asserted maximum flow rate or flow rate and asserted maximum temperature), which may be an indication of the optimal response available to the existing cooling system. In at least one embodiment, the responsiveness of the learning subsystem enables the learning of information reflecting the cooling stresses on the cooling system caused by at least asserted temperature control of the heating coils and asserted available coolant flow rate and flow rate of at least one flow controller. The asserted coolant flow rate or flow rate represents step 506 for determining cooling demand associated with at least one thermal characteristic and at least one flow controller.

[0088] In at least one embodiment, the learning subsystem is trained to understand temperature range thresholds that can be used to operate a new data center device within the existing cooling system. The learning subsystem is capable of indicating cooling stresses on the cooling system used for the new data center device, including one or more maximum flow rates, maximum flow rates, and temperature range thresholds that the cooling system can handle. In at least one embodiment, this information can be used to select a data center device different from the new data center device, whose temperature values ​​are used to test or train the learning subsystem.

[0089] In at least one embodiment, the operational steps may include determining that data center equipment needs to be installed to handle the operational requirements of the data center. The temperature rating of the data center equipment can be obtained from the manufacturer's specifications, and a learning subsystem can be trained to assert temperatures exceeding a threshold percentage of the temperature rating (e.g., 5% at the high and low ends of the data center equipment). The learning subsystem can then output temperatures to a thermal characteristic, which is installed in the data center and supported by at least one cold plate of the data center. The learning subsystem can then provide an output range to one or more fluid controllers to handle the temperature through the cold plate. The learning subsystem may obtain a sensed ambient temperature (at least from the thermal characteristic of an adjacent data center) or may rely on the asserted temperature itself. The learning subsystem is capable of establishing a correlation between a temperature range threshold from the sensed temperature (or from the asserted temperature, if equal to the sensed temperature) and an output range provided to the cooling system (corresponding to the flow rate and / or flow rate of the coolant).

[0090] In at least one embodiment, the correlation represents an assessment of the cooling stress on the cooling system. The cooling stress is represented by at least one temperature applied to (or sensed on) at least one thermal characteristic and a flow rate or volume associated with at least one flow controller (to handle these temperatures). This correlation can serve as the basis for a learning subsystem to provide an output associated with at least one flow rate or volume representing the expected cooling of at least one input temperature of the new data center equipment (at the end of testing or for commissioning the cooling system). In at least one embodiment, commissioning the cooling system involves using an existing cooling system to handle the cooling needs of the new data center equipment.

[0091] In at least one embodiment, the rated temperature of the new data center equipment is used as the input temperature of the learning subsystem, and the output may be the flow rate or volume required to handle the input temperature. In at least one embodiment, when the temperature used exceeds the threshold percentage of the temperature rating, and the sensed temperature shows no relief at the maximum available flow rate or volume of the coolant. In at least one embodiment, information about the asserted or sensed temperature not being relieved is used in the learning subsystem to determine a different data center equipment than the new data center equipment used in the data center. This different data center equipment has a wider operating range to withstand the limits within the cooling system. Alternatively, in at least one embodiment, the cooling system may be adapted to locally increase the flow rate or volume within server manifolds, rack manifolds, or piping to cold plates. Adaptation may be performed by changing the diameter of certain pipes and by retesting the cooling system's ability to handle the temperature of the new data center equipment, rather than using a different data center equipment.

[0092] In at least one embodiment, the learning subsystem can be implemented using a deep learning application processor, for example... Figure 14 The processor 1400 can utilize neurons 1502 and their components implemented using circuits or logic, including such... Figure 15 The description includes one or more arithmetic logic units (ALUs). Therefore, the learning subsystem includes at least one processor for evaluating the temperature within one or more racks of a server, with assertions regarding the flow rate and volume of coolant. In at least one embodiment, the processing aspect of the deep learning subsystem can utilize collected data based on... Figure 14 , Figure 15 The information discussed pertains to the feature processing. In at least one embodiment, temperature processing utilizes multiple neuron levels of a machine learning model loaded with one or more of the collected temperature features described above and the corresponding flow rate or volume of the coolant in the cooling system. In at least one embodiment, testing or training can be performed using different coolants. The execution of the learning subsystem can be represented as training an evaluation of temperature changes associated with previous flow rates or volumes based on adjustments made to one or more flow controllers. The neuron levels can store values ​​related to the evaluation process and can represent the association or correlation between temperature changes and flow rates or volumes.

[0093] In at least one embodiment, the processor and the flow controller can operate synchronously. The processor (also referred to as a centralized or distributed control system) is at least one processor having at least one logical unit for controlling the flow controller associated with one or more cooling loops. In at least one embodiment, the at least one processor is located within a data center, for example... Figure 7AThe processor 702 in the controller facilitates the movement of respective coolants and promotes the cooling of associated thermal characteristics in the data center in response to a sensed or asserted temperature in that area. In at least one embodiment, at least one processor is a processor core of a multi-core processor, such as... Figure 9A The multi-core processors 905 and 906 are included. In at least one embodiment, at least one logic unit may be adapted to receive temperature values ​​from a temperature sensor associated with a server or one or more racks, and adapted to facilitate the movement of coolant. In at least one embodiment, the controller has a microprocessor to perform its communication and control tasks using its respective motion characteristics.

[0094] In at least one embodiment, a processor, for example Figure 9A The processor cores of the multi-core processors 905 and 906 may include a learning subsystem for evaluating temperatures from sensors at different locations within the data center (e.g., different locations associated with the surface area of ​​at least one heatsink, or associated with a server or rack, or even a component within the server). The learning subsystem provides outputs of instructions, such as those associated with at least temperature or flow rate, to facilitate coolant movement.

[0095] Data Center

[0096] Figure 6A It shows that it can be used from Figures 2-5 An example data center 600 is described in at least one embodiment. In at least one embodiment, data center 600 includes a data center infrastructure layer 610, a framework layer 620, a software layer 630, and an application layer 640. In at least one embodiment, for example, regarding... Figures 2-5 The characteristics of the liquid cooling system for a test data center in components of a heat load system, as described herein, can be performed within or in conjunction with example data center 600. In at least one embodiment, the infrastructure layer 610, framework layer 620, software layer 630, and application layer 640 can be provided, in part or in whole, via computing components on server trays located in rack 210 of data center 200. This enables the cooling system of this disclosure to directly cool certain computing components in an efficient and effective manner. Aspects of the data center including the data center infrastructure layer 610, framework layer 620, software layer 630, and application layer 640 can be used to support at least the aspects discussed herein. Figures 2-5 Intelligent control of the controller in the heat load system. Thus, regarding... Figures 6A-17D The discussion can be understood to apply to requirements that enable or support testing, for example... Figures 2-5 The hardware and software characteristics of the heat load system of the liquid cooling system in the data center.

[0097] In at least one embodiment, such as Figure 6A As shown, the data center infrastructure layer 610 may include a resource coordinator 612, packet computing resources 614, and node computing resources (“nodes CR”) 616(1)-616(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 616(1)-616(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more nodes CR 616(1)-616(N) may be servers having one or more of the aforementioned computing resources.

[0098] In at least one embodiment, the grouped computing resources 614 may include individual groups (not shown) of node CRs housed within one or more racks, or numerous racks (also not shown) housed within data centers in various geographical locations. The individual groups of node CRs within the grouped computing resources 614 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, the one or more racks may also include any number of power modules, cooling modules, and network switches, in any combination.

[0099] In at least one embodiment, resource coordinator 612 may configure or otherwise control one or more nodes CR616(1)-616(N) and / or grouped computing resources 614. In at least one embodiment, resource coordinator 612 may include a software design infrastructure (“SDI”) management entity for data center 600. In at least one embodiment, resource coordinator may include hardware, software, or some combination thereof.

[0100] In at least one embodiment, such as Figure 6AAs shown, framework layer 620 includes job scheduler 622, configuration manager 624, resource manager 626, and distributed file system 628. In at least one embodiment, framework layer 620 may include a framework of software 632 supporting software layer 630 and / or one or more applications 642 of application layer 640. In at least one embodiment, software 632 or one or more applications 642 may each include web-based service software or applications, such as service software or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, framework layer 620 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark™ (hereinafter referred to as "Spark") which can leverage distributed file system 628 for large-scale data processing (e.g., "big data"). In at least one embodiment, job scheduler 622 may include Spark drivers to facilitate the scheduling of workloads supported by the various layers of data center 600. In at least one embodiment, configuration manager 624 may be able to configure different layers, such as software layer 630 and framework layer 620 including Spark and distributed file system 628 for supporting large-scale data processing. In at least one embodiment, resource manager 626 is capable of managing cluster or group computing resources mapped to or allocated to support distributed file system 628 and job scheduler 622. In at least one embodiment, cluster or group computing resources may include group computing resources 614 on data center infrastructure layer 610. In at least one embodiment, resource manager 626 may coordinate with resource coordinator 612 to manage these mapped or allocated computing resources.

[0101] In at least one embodiment, the software 632 included in the software layer 630 may include software used by at least a portion of the nodes CR616(1)-616(N), the grouped computing resources 614, and / or the distributed file system 628 of the framework layer 620. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0102] In at least one embodiment, one or more applications 642 included in application layer 640 may include one or more types of applications used by at least a portion of nodes CR616(1)-616(N), grouped computing resources 614, and / or the distributed file system 628 of framework layer 620. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0103] In at least one embodiment, any of the configuration manager 624, resource manager 626, and resource coordinator 612 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. In at least one embodiment, self-modification actions can mitigate potentially poor configuration decisions by data center operators of data center 600 and can prevent underutilization and / or poor performance of the data center.

[0104] In at least one embodiment, data center 600 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information according to one or more embodiments herein. In at least one embodiment, a machine learning model can be trained by calculating weight parameters according to a neural network architecture using the software and computing resources described above with respect to data center 600. In at least one embodiment, information can be inferred or predicted using trained machine learning models corresponding to one or more neural networks using the resources described above with respect to data center 600 by using weight parameters calculated through one or more training techniques herein. As previously discussed, deep learning techniques can be used to support intelligent control of controllers in a heat load system used to test the liquid cooling system of the data center described herein by monitoring the regional temperature of the data center. Deep learning can be advanced using any suitable learning network and the computing capabilities of data center 600. Thus, hardware in the data center can be used to support deep neural networks (DNNs), recurrent neural networks (RNNs), or convolutional neural networks (CNNs) simultaneously or concurrently. For example, once a network has been trained and successfully evaluated to identify data in a subset or slice, the trained network can provide similar representative data for use with the collected data.

[0105] In at least one embodiment, the data center 600 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, or other hardware to utilize the aforementioned resources to perform training and / or inference. Furthermore, one or more of the aforementioned software and / or hardware resources may be configured as a service to allow a user to train or perform information inference, such as pressure, flow rate, temperature, and location information, or other artificial intelligence services.

[0106] Reasoning and training logic

[0107] Inference and / or training logic 615 can be used to perform inference and / or training operations associated with one or more embodiments. In at least one embodiment, inference and / or training logic 615 can be in the system Figure 6A This is used to infer or predict operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architecture, or the neural network use cases described herein. In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, hardware logic, wherein computing resources are dedicated or otherwise uniquely used in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within the neural network. In at least one embodiment, the inference and / or training logic 615 may be used in conjunction with an application-specific integrated circuit (ASIC), such as those from Google. Processing unit, from Graphcore TM Inference processing units (IPUs) or from Intel Corp. (e.g., "LakeCrest") processor.

[0108] In at least one embodiment, the inference and / or training logic 615 may be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware (e.g., field-programmable gate array (FPGA)). In at least one embodiment, the inference and / or training logic 615 includes, but is not limited to, code and / or data storage models, which may be used to store code (e.g., graphical code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. In at least one embodiment, each code and / or data storage module is associated with a dedicated computing resource. In at least one embodiment, the dedicated computing resource includes computing hardware, which further includes one or more ALUs that perform mathematical functions (e.g., linear algebraic functions) only on the information stored in the code and / or data storage modules, and stores the results stored therefrom in the active storage module of the inference and / or training logic 615.

[0109] Figure 6B , Figure 6CThe inference and / or training logic according to at least one embodiment is shown, such as in Figure 6A The inference and / or training logic used in at least one embodiment of this disclosure. The inference and / or training logic 615 is used to perform inference and / or training operations associated with at least one embodiment. The following is in conjunction with... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. This is distinguished from the computing hardware 602 and 606 by using an arithmetic logic unit (ALU) 610. Figure 6B and Figure 6C The inference and / or training logic 615. In at least one embodiment, each of computing hardware 602 and computing hardware 606 includes one or more ALUs, which perform mathematical functions (e.g., linear algebraic functions) only on information stored in code and / or data storage 601 and code and / or data storage 605, respectively, with the results stored in activation memory 620. Thus, unless otherwise stated, Figure 6B and Figure 6C They can be substitutes and used interchangeably.

[0110] In at least one embodiment, inference and / or training logic 615 may include, but is not limited to, code and / or data storage 601 for storing forward and / or output weights and / or input / output data and / or other parameters of neurons or layers of a neural network trained and / or used for inference in at least one embodiment. In at least one embodiment, training logic 615 may include, or be coupled to, code and / or data storage 601 for storing graph code or other software to control timing and / or sequence, wherein weight and / or other parameter information is loaded to configure logic, including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, code (such as graph code) loads weight or other parameter information into the processor ALU based on the architecture of the neural network to which the code corresponds. In at least one embodiment, code and / or data storage 601 stores weight parameters and / or input / output data of each layer of a neural network trained or used in conjunction with at least one embodiment during forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of at least one embodiment. In at least one embodiment, any portion of the code and / or data storage 601 may be included within other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0111] In at least one embodiment, any portion of the code and / or data storage 601 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 601 may be cache memory, dynamic random-addressable memory (“DRAM”), static random-addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice of whether the code and / or data storage 601 is internal or external to the processor, for example, or including DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip or off-chip storage space, the latency requirements of the training and / or inference functions being performed, the data batch size used in the inference and / or training of the neural network, or some combination of these factors.

[0112] In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, code and / or data storage 605 for storing backpropagation and / or output weights and / or input / output data neural networks corresponding to neurons or layers of a neural network trained and / or used for inference in at least one embodiment. In at least one embodiment, during training and / or inference using at least one embodiment, the code and / or data storage 605 stores weight parameters and / or input / output data for each layer of the neural network trained or used in conjunction with the input / output data and / or weight parameters during backpropagation of the neural network in at least one embodiment. In at least one embodiment, the training logic 615 may include or be coupled to code and / or data storage 605 for storing graph code or other software to control timing and / or sequence, wherein weight and / or other parameter information is loaded to configure logic including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)).

[0113] In at least one embodiment, code (such as graph code) loads weights or other parameter information into the processor ALU based on the architecture of the neural network to which the code corresponds. In at least one embodiment, any portion of the code and / or data storage 605 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of the code and / or data storage 605 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 605 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice between the code and / or data storage 605 being internal or external to the processor, for example, including DRAM, SRAM, flash memory, or some other type of storage, depends on whether the available storage is on-chip or off-chip, the latency requirements of the training and / or inference functions being performed, the data batch size used in the inference and / or training of the neural network, or some combination of these factors.

[0114] In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 may be separate storage structures. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 may be the same storage structure. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 may be partially the same storage structure and partially separate storage structures. In at least one embodiment, any portion of code and / or data storage 601 and code and / or data storage 605 may be included with other on-chip or off-chip data storage, including processor L1, L2, or L3 caches or system memory.

[0115] In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, one or more arithmetic logic units (“ALUs”) 610, each ALU including integer and / or floating-point units, for performing logical and / or mathematical operations at least in part based on or instructed by training and / or inference code (e.g., graphical code), the results of which may produce (e.g., output values ​​from layers or neurons within a neural network) activations stored in activation storage 620, which are functions of input / output and / or weight parameter data stored in code and / or data storage 601 and / or code and / or data storage 605. In at least one embodiment, activation is activated in response to execution instructions or other code, linear algebraic and / or matrix-based mathematical generation performed by ALU 610, and the activation is stored in activation storage 620, wherein weight values ​​stored in code and / or data storage 605 and / or code and / or data storage 601 are used as operands with other values, such as bias values, gradient information, momentum values, or other parameters or hyperparameters, and any or all of these can be stored in code and / or data storage 605 and / or code and / or data storage 601 or other on-chip or off-chip storage.

[0116] In at least one embodiment, one or more ALUs 610 are included in one or more processors or other hardware logic devices or circuits, while in another embodiment, one or more ALUs 610 may be located outside the processor or other hardware logic device or the circuits using them (e.g., coprocessors). In at least one embodiment, one or more ALUs 610 may be included within an execution unit of a processor, or otherwise included in a group of ALUs accessible by the execution unit of the processor, which may be within the same processor or distributed among different processors of different types (e.g., central processing unit, graphics processing unit, fixed-function unit, etc.). In at least one embodiment, code and / or data storage 601, code and / or data storage 605, and activation storage 620 may be on the same processor or other hardware logic device or circuit, while in another embodiment, they may be on different processors or other hardware logic devices or circuits, or in some combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of activation storage 620 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Furthermore, inference and / or training code may be stored together with other code accessible to the processor or other hardware logic or circuitry, and may be retrieved and / or processed using the processor’s fetch, decode, schedule, execute, exit, and / or other logic circuitry.

[0117] In at least one embodiment, the active memory 620 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the active memory 620 may be wholly or partially located inside or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the active memory 620 is internal to or external to the processor may depend on the available on-chip or off-chip storage, the latency requirements for training and / or inference functions, the batch size of data used in inference and / or training the neural network, or some combination of these factors. For example, it may include DRAM, SRAM, flash memory, or other storage types. In at least one embodiment, Figure 6B The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as those from Google. Processing unit, from Graphcore TM Inference processing units (IPUs) or from Intel Corp. (e.g., "Lake Crest") processor. In at least one embodiment, Figure 6B The inference and / or training logic 615 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware, or other hardware such as field programmable gate array (“FPGA”)

[0118] In at least one embodiment, Figure 6C An inference and / or training logic 615 according to at least one of the various embodiments is shown, which may include, but is not limited to, hardware logic, wherein computational resources are dedicated or otherwise uniquely used in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, Figure 6C The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (ASIC), such as those from Google. Processing unit, from Graphcore TM Inference processing unit (IPU) or from Intel Corp. (e.g., "Lake Crest") processor. In at least one embodiment, Figure 6CThe inference and / or training logic 615 shown can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware (e.g., field-programmable gate array (FPGA)). In at least one embodiment, the inference and / or training logic 615 includes, but is not limited to, code and / or data storage 601 and code and / or data storage 605, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. Figure 6C In at least one embodiment shown, each of code and / or data storage 601 and code and / or data storage 605 is associated with dedicated computing resources (e.g., computing hardware 602 and computing hardware 606).

[0119] In at least one embodiment, each of the code and / or data storage 601 and 605 and the corresponding computing hardware 602 and 606 corresponds to a different layer of the neural network, such that activation obtained from one “store / computation pair 601 / 602” of the code and / or data storage 601 and computing hardware 602 provides input as input to the next “store / computation pair 605 / 606” of the code and / or data storage 605 and computing hardware 606, in order to reflect the conceptual organization of the neural network. In at least one embodiment, each store / computation pair 601 / 602 and 605 / 606 may correspond to more than one neural network layer. In at least one embodiment, additional store / computation pairs (not shown) may be included in the inference and / or training logic 615 after or in parallel with the store / computation pairs 601 / 602 and 605 / 606.

[0120] Computer System

[0121] Figure 7A A block diagram of an exemplary computer system 700A according to at least one embodiment is shown. The exemplary computer system may be a system having interconnect devices and components, a system-on-a-chip (SOC), or some combination thereof with a processor. The processor may include an execution unit to execute instructions to support and / or enable intelligent control of a heat load system for testing a data center liquid cooling system as described herein. In at least one embodiment, the computer system 700A according to this disclosure (such as embodiments herein) may include, but is not limited to, components (such as processor 702) to use an execution unit including logic to execute algorithms for process data. In at least one embodiment, the computer system 700A may include a processor, such as one available from Intel Corporation, Santa Clara, California. Processor family, Xeon™ XScale™ and / or StrongARM™ CoreTM or Nervana TM The system can use a microprocessor, but other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors can also be used. In at least one embodiment, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces can also be used, the computer system 700B can execute a version of the Windows operating system available from Microsoft Corporation in Redmond, Washington.

[0122] In at least one embodiment, the exemplary computer system 700A may incorporate components 110-116 (from... Figure 1 One or more of these can be used to support intelligent control of the heat load system for testing data center liquid cooling systems. At least for this reason, in one embodiment, Figure 7A A system including interconnected hardware devices or "chips" is shown, while in other embodiments, Figure 7A An exemplary system-on-a-chip (SoC) can be illustrated. In at least one embodiment, Figure 7A The devices shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of the computer system 700B are interconnected using a Compute Fast Link (CXL) interconnect. Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments, such as those previously described. Figure 6A The discussion below is based on -C. Figure 6A -C provides details about the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be in the system Figure 7A The operation is used to infer or predict the operation based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0123] The embodiments can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip (SoC), a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system that can execute one or more instructions according to at least one embodiment.

[0124] In at least one embodiment, the computer system 700A may include, but is not limited to, a processor 702, which may include, but is not limited to, one or more execution units 708, to perform machine learning model training and / or inference according to the techniques described herein. In at least one embodiment, the computer system 700A is a single-processor desktop or server system, but in another embodiment, the computer system 700A may be a multiprocessor system. In at least one embodiment, the processor 702 may include, but is not limited to, a Complex Instruction Set Computer (“CISC”) microprocessor, a Reduced Instruction Set Computing (“RISC”) microprocessor, a Very Long Instruction Word (“VLIW”) microprocessor, a processor implementing instruction set combinations, or any other processor device, such as a digital signal processor. In at least one embodiment, the processor 702 may be coupled to a processor bus 710, which may transmit data signals between the processor 702 and other components in the computer system 700A.

[0125] In at least one embodiment, processor 702 may include, but is not limited to, a Level 1 (“L1”) internal cache (“cache”) 704. In at least one embodiment, processor 702 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory may reside external to processor 702. Depending on specific implementation and requirements, other embodiments may also include a combination of internal and external caches. In at least one embodiment, register file 706 may store different types of data in various registers, including but not limited to integer registers, floating-point registers, status registers, and instruction pointer registers.

[0126] In at least one embodiment, an execution unit 708, including but not limited to logic for performing integer and floating-point operations, is also located within the processor 702. In at least one embodiment, the processor 702 may further include a microcode (“ucode”) read-only memory (“ROM”) for storing microcode of certain macro instructions. In at least one embodiment, the execution unit 708 may include logic for processing a packaged instruction set 709. In at least one embodiment, by including the packaged instruction set 709 in the instruction set of a general-purpose processor, along with the associated circuitry for executing the instructions, packaged data in the general-purpose processor 702 can be used to perform operations used by numerous multimedia applications. In one or more embodiments, many multimedia applications can be executed more quickly and efficiently by using the full width of the processor’s data bus to perform operations on the packaged data, which may eliminate the need to transfer smaller data units on the processor’s data bus to perform one or more operations on one data element at a time.

[0127] In at least one embodiment, execution unit 708 may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuitry. In at least one embodiment, computer system 700A may include, but is not limited to, memory 720. In at least one embodiment, memory 720 may be implemented as a dynamic random access memory (“DRAM”) device, a static random access memory (“SRAM”) device, a flash memory device, or other storage device. In at least one embodiment, memory 720 may store instructions 719 and / or data 721 represented by data signals that can be executed by processor 702.

[0128] In at least one embodiment, the system logic chip may be coupled to a processor bus 710 and a memory 720. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub (“MCH”) 716, and the processor 702 may communicate with the MCH 716 via the processor bus 710. In at least one embodiment, the MCH 716 may provide a high-bandwidth memory path 718 to the memory 720 for instruction and data storage, as well as for storage of graphics commands, data, and textures. In at least one embodiment, the MCH 716 may initiate data signals between the processor 702, the memory 720, and other components in the computer system 700A, and bridge data signals between the processor bus 710, the memory 720, and the system I / O 722. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, MCH 716 can be coupled to memory 720 via high-bandwidth memory path 718, and graphics / video card 712 can be coupled to MCH 716 via Accelerated Graphics Port (“AGP”) interconnect 714.

[0129] In at least one embodiment, the computer system 700A may use system I / O 722 as a proprietary hub interface bus to couple MCH 716 to I / O controller hub (“ICH”) 730. In at least one embodiment, ICH 730 may provide direct connectivity to certain I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripheral devices to memory 720, chipset, and processor 702. Examples may include, but are not limited to, an audio controller 729, a firmware hub (“Flash BIOS”) 728, a wireless transceiver 726, a data storage 724, a conventional I / O controller 723 including a user input and keyboard interface, a serial expansion port 727 (e.g., a Universal Serial Bus (USB) port), and a network controller 734. Data storage 724 may include a hard disk drive, floppy disk drive, CD-ROM device, flash memory device, or other mass storage device.

[0130] Figure 7B This is a block diagram illustrating an electronic device 700B utilizing processor 710 according to at least one embodiment to support and / or enable intelligent control of a heat load system for testing a data center liquid cooling system as described herein. In at least one embodiment, electronic device 700B may be, for example, but not limited to, a laptop computer, tower server, rack server, blade server, desktop computer, tablet computer, mobile device, telephone, embedded computer, or any other suitable electronic device. In at least one embodiment, exemplary electronic device 700B may incorporate components 110-116 (from... Figure 1 One or more of these can be used to support the handling aspects of the heat load system used for testing data center liquid cooling systems.

[0131] In at least one embodiment, system 700B may, but is not limited to, processor 710 communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, processor 710 uses a bus or interface coupling, such as an I²C bus, system management bus (“SMBus”), low pin count (LPC) bus, serial peripheral interface (“SPI”), high-definition audio (“HDA”) bus, serial advanced technology accessory (“SATA”) bus, universal serial bus (“USB”) (versions 1, 2, and 3), or universal asynchronous receiver / transmitter (“UART”) bus. In at least one embodiment, Figure 7B The system shown includes interconnected hardware devices or "chips," while in other embodiments, Figure 7B An exemplary system-on-a-chip (“SoC”) may be illustrated. In at least one embodiment, Figure 7BThe device shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, Figure 7B One or more components are interconnected using Computational Fast Link (CXL) interconnects.

[0132] In at least one embodiment, Figure 7B This may include a display 724, a touchscreen 725, a touchpad 730, a near-field communication unit (“NFC”) 745, a sensor hub 740, a thermal sensor 746, a fast chipset (“EC”) 735, a trusted platform module (“TPM”) 738, a BIOS / firmware / flash memory (“BIOS, FW Flash”) 722, a DSP 760, a drive 720 (e.g., a solid-state drive (“SSD”) or a hard disk drive (“HDD”)), a wireless local area network unit (“WLAN”) 750, a Bluetooth unit 752, a wireless wide area network unit (“WWAN”) 756, a global positioning system (GPS) unit 755, a camera (“USB 3.0 camera”) 754 (e.g., a USB 3.0 camera), and / or a low-power double data rate (“LPDDR”) memory unit (“LPDDR3”) 715 implemented in, for example, the LPDDR3 standard. These components may each be implemented in any suitable manner.

[0133] In at least one embodiment, other components may be communicatively coupled to processor 710 via the following components: In at least one embodiment, accelerometer 741, ambient light sensor (“ALS”) 742, compass 743, and gyroscope 744 may be communicatively coupled to sensor hub 740. In at least one embodiment, thermal sensor 739, fan 737, keyboard 746, and touchpad 730 may be communicatively coupled to EC 735. In at least one embodiment, speaker 763, earphone 764, and microphone (“mic”) 765 may be communicatively coupled to audio unit (“audio codec and Class D amplifier”) 762, which in turn may be communicatively coupled to DSP 760. In at least one embodiment, audio unit 764 may include, for example, but not limited to, audio encoder / decoder (“codec”) and Class D amplifier. In at least one embodiment, SIM card (“SIM”) 757 may be communicatively coupled to WWAN unit 756. In at least one embodiment, components such as WLAN unit 750, Bluetooth unit 752, and WWAN unit 756 can be implemented as next-generation form factor (NGFF).

[0134] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6CDetails are provided regarding the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be implemented in the system. Figure 7B It is used in this context to infer or predict operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0135] Figure 7C A computer system 700C according to at least one embodiment is illustrated, which is used to support and / or enable intelligent control of the heat load system for testing data center liquid cooling systems as described herein. In at least one embodiment, the computer system 700C includes, but is not limited to, a computer 771 and a USB stick 770. In at least one embodiment, the computer 771 may include, but is not limited to, any number and type of processors (not shown) and memory (not shown). In at least one embodiment, the computer 771 includes, but is not limited to, servers, cloud instances, laptops, and desktop computers.

[0136] In at least one embodiment, the USB stick 770 includes, but is not limited to, a processing unit 772, a USB interface 774, and USB interface logic 773. In at least one embodiment, the processing unit 772 can be any instruction execution system, device, or device capable of executing instructions. In at least one embodiment, the processing unit 772 can include, but is not limited to, any number and type of processing cores (not shown). In at least one embodiment, the processing unit or core 772 includes an application-specific integrated circuit (“ASIC”) optimized to perform any number and type of operations associated with machine learning. For example, in at least one embodiment, the processing core 772 is a tensor processing unit (“TPC”) optimized to perform machine learning inference operations. In at least one embodiment, the processing core 772 is a vision processing unit (“VPU”) optimized to perform machine vision and machine learning inference operations.

[0137] In at least one embodiment, the USB interface 774 can be any type of USB connector or USB receptacle. For example, in at least one embodiment, the USB interface 774 is a USB 3.0 Type-C receptacle for data and power. In at least one embodiment, the USB interface 774 is a USB 3.0 Type-A connector. In at least one embodiment, the USB interface logic 773 may include any amount and type of logic enabling the processing unit 772 to connect to a device (e.g., computer 771) via the USB connector 774.

[0138] Reasoning and / or training logic 615 (such as regarding Figure 6B and Figure 6CThe described method is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 can be used for Figure 7C In the system, the operation is inferred or predicted based at least in part on the weight parameters calculated using the neural network training operation, neural network function and / or architecture, or neural network use case described herein.

[0139] Figure 8 Further exemplary computer system 800, according to at least one embodiment, is illustrated for implementing various processes and methods described throughout this disclosure for testing the heat load system of a data center liquid cooling system. In at least one embodiment, computer system 800 includes, but is not limited to, at least one central processing unit (“CPU”) 802 connected to a communication bus 810 implemented using any suitable protocol, such as PCI (“Peripheral Device Interconnect”), Peripheral Component Interconnect Express (“PCI-Express”), AGP (“Accelerated Graphics Port”), HyperTransport, or any other bus or point-to-point communication protocol. In at least one embodiment, computer system 800 includes, but is not limited to, main memory 804 and control logic (e.g., implemented in hardware, software, or a combination thereof), and data may be stored in main memory 804 in the form of random access memory (“RAM”). In at least one embodiment, a network interface subsystem (“network interface”) 822 provides an interface to other computing devices and networks for receiving data from computer system 800 and transferring data to other systems.

[0140] In at least one embodiment, the computer system 800 includes, but is not limited to, an input device 808, a parallel processing system 812, and a display device 806, which may be implemented using a cathode ray tube (“CRT”), a liquid crystal display (“LCD”), a light-emitting diode (“LED”), a plasma display, or other suitable display technologies. In at least one embodiment, user input is received from the input device 808 (such as a keyboard, mouse, touchpad, microphone, and more). In at least one embodiment, each of the above-described modules may reside on a single semiconductor platform to form the processing system.

[0141] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments, such as those previously mentioned. Figure 6A The discussion below is based on -C. Figure 6A-C provides details about the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be in the system Figure 8 It is used to perform inference or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architecture, or the neural network use cases described herein. In at least one embodiment, the inference and / or training logic 615 can be used in the system. Figure 8 It is used to perform inference or prediction operations based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architecture or neural network use cases described herein.

[0142] Figure 9A An exemplary architecture is illustrated, in which multiple GPUs 910-913 are communicatively coupled to multiple multi-core processors 940-943 via high-speed links 905-906 (e.g., bus / point-to-point interconnect, etc.). In one embodiment, the high-speed links 940-943 support communication throughput of 4GB / s, 30GB / s, 80GB / s, or higher. Various interconnect protocols can be used, including but not limited to PCIe 4.0 or 5.0 and NVLink 2.0.

[0143] Furthermore, in one embodiment, two or more GPUs 910-913 are interconnected via high-speed links 929-930, which may use the same or different protocols / links as those used for high-speed links 940-943. Similarly, two or more multi-core processors 905-906 may be connected via high-speed link 928, which may be a symmetric multiprocessor (SMP) bus operating at speeds of 20GB / s, 30GB / s, 120GB / s, or higher. Alternatively, the same protocol / link (e.g., via a common interconnect structure) may be used. Figure 9A This shows all communication between the various system components.

[0144] In one embodiment, each multi-core processor 905-906 is communicatively coupled to processor memories 901-902 via memory interconnects 926-927, and each GPU 910-913 is communicatively coupled to GPU memories 920-923 via GPU memory interconnects 950-953. Memory interconnects 926-927 and 950-953 may utilize the same or different memory access technologies. By way of example and not limitation, processor memories 901-902 and GPU memories 920-923 may be volatile memories, such as dynamic random access memory (DRAM) (including stacked DRAM), graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6), or high-bandwidth memory (HBM), and / or may be non-volatile memories, such as 3D XPoint or Nano-RAM. In one embodiment, certain portions of processor memories 901-902 may be volatile memories, while other portions may be non-volatile memories (e.g., using a two-level memory (2LM) hierarchy).

[0145] As shown below, although the various processors 905-906 and GPUs 910-913 can be physically coupled to specific memories 901-902 and 920-923 respectively, a unified memory architecture can be implemented, in which the virtual system address space (also known as the "effective address" space) is distributed among the various physical memories. In at least one embodiment, processor memories 901-902 can each include 64GB of system memory address space, and GPU memories 920-923 can each include 32GB of system memory address space (resulting in a total addressable memory size of 256GB in this example).

[0146] As discussed elsewhere in this disclosure, at least the flow rate and associated temperature are established for the first stage of an intelligent learning system (e.g., a neural network system). Since the first stage represents prior data, it also represents a smaller subset of data from which the system can be improved through retraining. Testing and training can be performed in parallel using multiple processor units, making the intelligent learning system robust. For example, [the following can be used]. Figure 9A The architecture within the system. When achieving convergence of the intelligent learning system, the amount of data points used to induce convergence, along with the data within those points, is labeled. The data and data points can be fully utilized to control the heat load system used for testing data center liquid cooling systems, for example, regarding... Figures 2-5 The subject of discussion.

[0147] Figure 9BAdditional details are shown for the interconnection between a multi-core processor 907 and a graphics acceleration module 946 according to an exemplary embodiment. The graphics acceleration module 946 may include one or more GPU chips integrated on a line card coupled to the processor 907 via a high-speed link 940. Optionally, the graphics acceleration module 946 may be integrated with the processor 907 in the same package or on a single chip.

[0148] In at least one embodiment, the processor 907 shown includes a plurality of cores 960A-960D, each core having a translation back buffer 961A-961D and one or more caches 962A-962D. In at least one embodiment, the cores 960A-960D may include various other components (not shown) for executing instructions and processing data. Caches 962A-962D may include level 1 (L1) and level 2 (L2) caches. Furthermore, one or more shared caches 956 may be included in the caches 962A-962D and shared by the respective groups of cores 960A-960D. In at least one embodiment, one embodiment of the processor 907 includes 24 cores, each core having its own L1 cache, twelve shared L2 caches, and twelve shared L3 caches. In this embodiment, two adjacent cores share one or more L2 and L3 caches. The processor 907 and graphics acceleration module 946 are connected to system memory 914, which may include... Figure 9A The processor memory 901-902 in the memory.

[0149] The consistency bus 964 maintains consistency for data and instructions stored in the various caches 962A-962D, 956 and system memory 914 via inter-core communication. In at least one embodiment, each cache may have associated cache consistency logic / circuitology to communicate via the consistency bus 964 in response to the detection of a read or write to a particular cache line. In one implementation, a cache snooping protocol is implemented via the consistency bus 964 to snoop on cache accesses.

[0150] In at least one embodiment, the proxy circuit 925 communicatively couples the graphics acceleration module 946 to the coherence bus 964, thereby allowing the graphics acceleration module 946 to participate in the cache coherence protocol as a peer of the cores 960A-960D. Specifically, in at least one embodiment, interface 935 provides connectivity to the proxy circuit 925 via a high-speed link 940 (e.g., PCIe bus, NVLink, etc.), and interface 937 connects the graphics acceleration module 946 to link 940.

[0151] In one implementation, the accelerator integrated circuit 936 represents multiple graphics processing engines 931, 932, N of the graphics acceleration module, providing cache management, memory access, context management, and interrupt management services. The graphics processing engines 931, 932, N may each include a separate graphics processing unit (GPU). In at least one embodiment, the graphics processing engines 931, 932, N may optionally include different types of graphics processing engines within the GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, the graphics acceleration module 946 may be a GPU with multiple graphics processing engines 931-932, N, or the graphics processing engines 931-932, N may be individual GPUs integrated on a general-purpose package, line card, or chip. Depending on the application, it may be... Figure 9B The above determination of reconstruction parameters and reconstruction algorithm is performed in the GPU 931-N.

[0152] In one embodiment, the accelerator integrated circuit 936 includes a memory management unit (MMU) 939 for performing various memory management functions, such as virtual-to-physical memory translation (also known as effective-to-real memory translation), and a memory access protocol for accessing system memory 914. The MMU 939 may also include a translation back buffer (“TLB”) (not shown) for caching virtual / effective-to-physical / real address translations. In one implementation, cache 938 may store commands and data for efficient access by graphics processing engines 931-932,N. In at least one embodiment, data stored in cache 938 and graphics memories 933-934,M may be kept consistent with core caches 962A-962D, 956 and system memory 914. As before, this task can be accomplished via proxy circuitry 925 representing cache 938 and graphics memory 933-934, M (e.g., sending updates related to the modification / access of cache lines on processor caches 962A-962D, 956 to cache 938 and receiving updates from cache 938).

[0153] A set of registers 945 stores context data for threads executed by graphics processing engines 931-932, N, and context management circuitry 948 manages the thread context. In at least one embodiment, context management circuitry 948 can perform save and restore operations to save and restore the context of individual threads during context switching (e.g., saving the first thread and storing the second thread so that the second thread can be executed by the graphics processing engine). In at least one embodiment, context management circuitry 948 can store the current register value to a designated area in memory (e.g., identified by a context pointer) during context switching. The register value can then be restored when returning to the context. In one embodiment, interrupt management circuitry 947 receives and processes interrupts received from system devices.

[0154] In one implementation, MMU 939 translates virtual / effective addresses from graphics processing engine 931 into real / physical addresses in system memory 914. One embodiment of accelerator integrated circuit 936 supports multiple (e.g., 4, 8, 16) graphics accelerator modules 946 and / or other accelerator devices. Graphics accelerator module 946 may be dedicated to a single application executing on processor 907, or may be shared among multiple applications. In one embodiment, a virtualized graphics execution environment is presented, wherein the resources of graphics processing engines 931-932, N are shared with multiple applications or virtual machines (VMs). In at least one embodiment, resources may be subdivided into “slices” based on processing requirements and priorities associated with VMs and / or applications, which are allocated to different VMs and / or applications.

[0155] In at least one embodiment, the accelerator integrated circuit 936 acts as a bridge to the system of the graphics acceleration module 946, providing address translation and system memory caching services. Additionally, the accelerator integrated circuit 936 can provide virtualization facilities for the host processor to manage the virtualization, interrupt, and memory management of the graphics processing engines 931-932,N.

[0156] Because the hardware resources of graphics processing engines 931-932, N are explicitly mapped to the real address space seen by the host processor 907, any host processor can directly address these resources using valid address values. In at least one embodiment, a function of the accelerator integrated circuit 936 is to physically separate the graphics processing engines 931-932, N, so that they appear as independent units to the system.

[0157] In at least one embodiment, one or more graphics memories 933-934, M are coupled to each graphics processing engine 931-932, N, respectively. The graphics memories 933-934, M store instructions and data, which are processed by each graphics processing engine 931-932, N. The graphics memories 933-934, M may be volatile memories, such as DRAM (including stacked DRAM), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and / or may be non-volatile memories, such as 3DXPoint or Nano-RAM.

[0158] In one embodiment, to reduce data traffic on link 940, a biasing technique is used to ensure that the data stored in graphics memories 933-934, M is the data most frequently used by graphics processing engines 931-932, N, and that cores 960A-960D may not use (or at least not frequently use). Similarly, the biasing mechanism attempts to keep data needed by the cores (and may not be graphics processing engines 931-932, N) in caches 962A-962D, core 956, and system memory 914.

[0159] Figure 9C Another exemplary embodiment is shown, wherein the accelerator integrated circuit 936, according to at least one embodiment disclosed herein, integrates intelligent control within the processor 907 for enabling and / or supporting the thermal load system for testing a data center liquid cooling system. In at least this embodiment, graphics processing engines 931-932, N communicate directly with the accelerator integrated circuit 936 via interface 937 and interface 935 (again, any form of bus or interface protocol may be utilized) through a high-speed link 940. The accelerator integrated circuit 936 can perform operations related to... Figure 9B The described operation is similar. However, due to its close proximity to the coherence bus 964 and caches 962A-962D, 956, it may have higher throughput. At least one embodiment supports different programming models, including a dedicated process programming model (without graphics acceleration module virtualization) and a shared programming model (with virtualization). The programming model may include a programming model controlled by the accelerator integrated circuit 936 and a programming model controlled by the graphics acceleration module 946.

[0160] In at least one embodiment, graphics processing engines 931-932,N are dedicated to a single application or process within a single operating system. In at least one embodiment, a single application can funnel requests from other applications to graphics processing engines 931-932,N, thereby providing virtualization within a VM / partition.

[0161] In at least one embodiment, graphics processing engines 931-932,N can be shared by multiple VM / application partitions. In at least one embodiment, the shared model can use a hypervisor to virtualize graphics processing engines 931-932,N to allow each operating system to access them. For a single-partition system without a hypervisor, the operating system owns the graphics processing engines 931-932,N. In at least one embodiment, the operating system can virtualize graphics processing engines 931-932,N to provide access to each process or application.

[0162] In at least one embodiment, the graphics acceleration module 946 or the individual graphics processing engines 931-932,N uses a process handle to select a process element. In at least one embodiment, the process element is stored in system memory 914 and can be addressed using the effective address to physical address translation techniques described herein. In at least one embodiment, the process handle may be an implementation-specific value provided to the host process when registering its context with the graphics processing engines 931-932,N (i.e., invoking system software to add the process element to the process element linked list). In at least one embodiment, the lower 16 bits of the process handle may be the offset of the process element in the process element linked list.

[0163] Figure 9D This document illustrates intelligent control for implementing and / or supporting a heat load system for testing a data center liquid cooling system, according to at least one embodiment disclosed herein. As used herein, a "slice" includes a designated portion of the processing resources of an accelerator integrated circuit 936. An application is an effective address space 982 in system memory 914, which stores a process element 983. In at least one embodiment, the process element 983 is stored in response to a GPU call 981 from an application 980 executing on processor 907. The process element 983 contains the process state of the corresponding application 980. A job descriptor (WD) 984 contained in the process element 983 may be a single job requested by the application, or it may contain a pointer to a job queue. In at least one embodiment, WD 984 is a pointer to a job request queue in the application's address space 982.

[0164] The graphics acceleration module 946 and / or the various graphics processing engines 931-932, N, can be shared by all processes or a subset of processes in the system. In at least one embodiment, infrastructure may be included for setting process states and sending WD 984 to the graphics acceleration module 946 to begin operations in a virtualized environment.

[0165] In at least one embodiment, the dedicated process programming model is implementation-specific. In this model, a single process owns either the graphics acceleration module 946 or an individual graphics processing engine 931. When the graphics acceleration module 946 is owned by a single process, the hypervisor initializes the accelerator integrated circuit for the owned partition, and when the graphics acceleration module 946 is assigned, the operating system initializes the accelerator integrated circuit 936 for the owned process.

[0166] In operation, the WD acquisition unit 991 in the accelerator integrated slice 990 acquires the next WD 984, which includes instructions for the work to be performed by one or more graphics processing engines of the graphics acceleration module 946. Data from the WD 984 can be stored in register 945 and used by the MMU 939, interrupt management circuitry 947, and / or context management circuitry 948, as shown. In at least one embodiment, one embodiment of the MMU 939 includes segment / page roaming circuitry for accessing segment / page tables 986 within the OS virtual address space 985. The interrupt management circuitry 947 can handle interrupt events 992 received from the graphics acceleration module 946. In at least one embodiment, when performing graphics operations, a valid address 993 generated by graphics processing engines 931-932,N is translated into a real address by the MMU 939.

[0167] In one embodiment, the same register set 945 is copied for each graphics processing engine 931-932, N and / or graphics acceleration module 946, and the same register set 945 can be initialized by a hypervisor or operating system. Each of these copied registers can be included in the accelerator integration slice 990. Exemplary registers that can be initialized by the hypervisor are shown in Table 1.

[0168] Table 1 – Supervisor Initialization Registers

[0169] 1 Slice Control Register 2 Real address (RA) pointer to the region of the scheduling process 3 Permission mask overwrite register 4 Interruption vector table entry offset 5 Interrupt vector table entry limit 6 Status Register 7 Logical partition identifier 8 Real Address (RA) management accelerator utilizes record pointers 9 Storage description register

[0170] Table 2 shows exemplary registers that can be initialized by the operating system.

[0171] Table 2 – Operating System Initialization Registers

[0172] 1 Process and thread identification 2 Valid Address (EA) Context Save / Restore Pointer 3 Virtual address (VA) accelerators utilize record pointers 4 Virtual address (VA) memory segment table pointer 5 Access Control 6 Job descriptor

[0173] In at least one embodiment, each WD 984 is specific to a particular graphics acceleration module 946 and / or graphics processing engine 931-932,N. It contains all the information required for the graphics processing engine 931-932,N to complete its work, or it may be a pointer to a memory location where the application has set up a command queue for the work to be completed.

[0174] Figure 9E Additional details of an exemplary embodiment of the shared model are shown. This embodiment includes a hypervisor real address space 998, in which a list of process elements 999 is stored. The hypervisor real address space 998 can be accessed via a hypervisor 996, which virtualizes the graphics acceleration module engine for the operating system 995.

[0175] In at least one embodiment, the shared programming model allows all processes or subsets of processes from all partitions or subsets of partitions in the system to use the graphics acceleration module 946. Two programming models exist where the graphics acceleration module 946 is shared by multiple processes and partitions: time-slice sharing and graphics-oriented sharing.

[0176] In this model, the hypervisor 996 owns the graphics acceleration module 946 and makes its functionality available to all operating systems 995. For the graphics acceleration module 946 to support virtualization through the hypervisor 996, the graphics acceleration module 946 may comply with the following requirements: (1) the job requests of the application must be autonomous (i.e., no state needs to be maintained between jobs), or the graphics acceleration module 946 must provide a context saving and restoration mechanism; (2) the graphics acceleration module 946 guarantees that the job requests of the application are completed within a specified amount of time, including any conversion errors, or the graphics acceleration module 946 provides the ability to preempt job processing; and (3) when operating in a directed shared programming model, fairness among the processes of the graphics acceleration module 946 must be ensured.

[0177] In one embodiment, application 980 needs to make a system call to operating system 995 using the graphics acceleration module type, working descriptor (WD), authority mask register (AMR) value, and context save / restore region pointer (CSRP). The graphics acceleration module type describes the target acceleration function for the system call. In at least one embodiment, the graphics acceleration module type can be a system-specific value. In at least one embodiment, the WD is specifically formatted for graphics acceleration module 946 and can take the form of graphics acceleration module 946 commands, valid address pointers to user-defined structures, valid address pointers to command queues, or any other data structure describing the work to be performed by graphics acceleration module 946. In at least one embodiment, the AMR value is the AMR state for the current process. In at least one embodiment, the value passed to the operating system is similar to that of the application that sets the AMR. If the implementation of accelerator integrated circuit 936 and graphics acceleration module 946 does not support the User Authority Mask Overwrite Register (UAMOR), the operating system can apply the current UAMOR value to the AMR value before passing the AMR in the hypervisor call. In at least one embodiment, the hypervisor 996 may apply the Current Privilege Mask Overwrite Register (AMOR) value before placing the AMR into process element 983. In at least one embodiment, the CSRP is one of registers 945 containing the effective address of a region in the application's effective address space 982 for the graphics acceleration module 946 to save and restore context state. This pointer is used in at least one embodiment, but is optional if saving state between jobs is not required or when a job is preempted. In at least one embodiment, the context save / restore region may be fixed system memory.

[0178] Upon receiving a system call, the operating system 995 can verify that the application 980 has been registered and granted permission to use the graphics acceleration module 946. Then, the operating system 995 uses the information shown in Table 3 to invoke the hypervisor 996.

[0179] Table 3 – Call parameters from OS to hypervisor

[0180] 1 Working Descriptor (WD) 2 Access Control Register (AMR) value (potentially masked) 3 Valid Address (EA) Context Save / Restore Region Pointer (CSRP) 4 Process ID (PID) and optional thread ID (TID) 5 Virtual address (VA) accelerators use record pointers (AURP). 6 Virtual address of the segment table pointer (SSTP) 7 Logical Interruption Service Number (LISN)

[0181] Upon receiving a hypervisor call, hypervisor 996 verifies that operating system 995 has been registered and granted permission to use graphics acceleration module 946. Then, hypervisor 996 adds process element 983 to the linked list of process elements of the corresponding graphics acceleration module 946 type. The process element may include the information shown in Table 4.

[0182] Table 4 – Process Element Information

[0183] 1 Working Descriptor (WD) 2 The Authority Mask Register (AMR) value is potentially masked. 3 Valid Address (EA) Context Save / Restore Region Pointer (CSRP) 4 Process ID (PID) and optional thread ID (TID) 5 Virtual address (VA) accelerators use record pointers (AURP). 6 Virtual address of the segment table pointer (SSTP) 7 Logical Interruption Service Number (LISN) 8 Interrupt vector table, exported from hypervisor call parameters 9 Status Register (SR) Value 10 Logical Partition ID (LPID) 11 Real Address (RA) administrator accelerator uses record pointers 12 Storage Descriptor Register (SDR)

[0184] In at least one embodiment, the management program initializes multiple accelerator integration slice 990 registers 945.

[0185] like Figure 9F As shown, in at least one embodiment, a unified memory is used, which can be addressed via a common virtual memory address space for accessing physical processor memories 901-902 and GPU memories 920-923. In this implementation, operations performed on GPUs 910-913 utilize the same virtual / effective memory address space to access processor memories 901-902, and vice versa, thereby simplifying programmability. In one embodiment, a first portion of the virtual / effective address space is allocated to processor memory 901, a second portion to second processor memory 902, a third portion to GPU memory 920, and so on. In at least one embodiment, the entire virtual / effective memory space (sometimes referred to as the effective address space) is thus distributed across each of processor memories 901-902 and GPU memories 920-923, thereby allowing any processor or GPU to access that memory using a virtual address mapped to any physical memory.

[0186] In one embodiment, the bias / coherence management circuitry 994A-994E within one or more MMUs 939A-939E ensures cache coherence between the caches of one or more host processors (e.g., 905) and the GPUs 910-913, and implements biasing techniques that indicate the physical memory in which certain types of data should be stored. While in Figure 9F Several instances of bias / coherence management circuitry 994A-994E are shown, but bias / coherence circuitry can be implemented within the MMU of one or more host processors 905 and / or within accelerator integrated circuits 936.

[0187] One embodiment allows GPU-attached memory 920-923 to be mapped as part of system memory and accessed using shared virtual memory (SVM) technology without suffering the performance drawbacks associated with full system cache coherence. In at least one embodiment, the ability to access GPU-attached memory 920-923 as system memory without the heavy overhead of cache coherence provides a favorable operating environment for GPU offloading. This arrangement allows host processor 905 software to set operands and access computation results without the overhead of conventional I / O DMA data copying. Such conventional copying includes driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, all of which are less efficient than simple memory accesses. In at least one embodiment, the ability to access GPU-attached memory 920-923 without cache coherence overhead can be critical to the execution time of offloaded computations. For example, in cases with high streaming write traffic to memory, cache coherence overhead can significantly reduce the effective write bandwidth seen by GPUs 910-913. In at least one embodiment, the efficiency of operand setting, the efficiency of result access, and the efficiency of GPU computation can play a role in determining the effectiveness of GPU offloading.

[0188] In at least one embodiment, the selection of GPU bias and host processor bias is driven by a bias tracker data structure. For example, a bias table can be used, which may be a page-granular structure (e.g., controlled at the memory page level) comprising 1 or 2 bits of memory pages attached to each GPU. In at least one embodiment, with or without a bias cache (e.g., for caching frequently / recently used entries in the bias table) in GPUs 910-913, the bias table can be implemented across one or more stolen memory ranges of GPU-attached memory 920-923. Alternatively, the entire bias table can be maintained within the GPU.

[0189] In at least one embodiment, prior to actual access to GPU memory, the bias table entry associated with each access to GPU-attached memory 920-923 is accessed, resulting in the following operations: Local requests from GPUs 910-913 to locate their pages in the GPU bias are directly forwarded to the corresponding GPU memory 920-923. Local requests from GPUs to locate their pages in the host bias are forwarded to processor 905 (e.g., via the high-speed link described above). In one embodiment, a request from processor 905 to locate the requested page in the host processor bias completes a request similar to a normal memory read. Alternatively, requests to GPU bias pages can be forwarded to GPUs 910-913. In at least one embodiment, if the GPU is not currently using the page, the GPU may subsequently migrate the page to the host processor bias. In at least one embodiment, the page bias state can be changed through software-based mechanisms, hardware-assisted software mechanisms, or, in limited cases, purely hardware-based mechanisms.

[0190] One mechanism for changing the bias state employs an API call (e.g., OpenCL), which subsequently invokes the GPU's device driver. The device driver then sends a message (or enqueues a command descriptor) to the GPU, instructing the GPU to change the bias state and, in some migrations, performs a cache refresh operation on the host. In at least one embodiment, the cache refresh operation is used for migrations from the host processor 905 bias to the GPU bias, but not for the reverse migration.

[0191] In one embodiment, cache coherence is maintained by temporarily rendering GPU bias pages that the host processor 905 cannot cache. To access these pages, the processor 905 may request access from the GPU 910, which may or may not grant access immediately. Therefore, to reduce communication between the processor 905 and the GPU 910, it is beneficial to ensure that the GPU bias pages are those required by the GPU, not those required by the host processor 905, and vice versa.

[0192] Inference and / or training logic 615 is used to execute one or more implementations. This can be combined with... Figure 6B and / or Figure 6C Provide details about reasoning and / or training logic 615.

[0193] Figure 10AExemplary integrated circuits and associated graphics processors according to various embodiments described herein are illustrated, which may be manufactured using one or more IP cores to support and / or enable thermal load systems for testing data center liquid cooling systems. In addition to the illustrations, other logic and circuitry, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores, may be included in at least one embodiment.

[0194] Figure 10A This is a block diagram illustrating an exemplary system on a chip integrated circuit 1000A manufactured using one or more IP cores according to at least one embodiment. In at least one embodiment, the integrated circuit 1000A includes one or more application processors 1005 (e.g., CPUs), at least one graphics processor 1010, and may additionally include an image processor 1015 and / or a video processor 1020, any of which may be a modular IP core. In at least one embodiment, the integrated circuit 1000A includes peripheral or bus logic including a USB controller 1025, a UART controller 1030, an SPI / SDIO controller 1035, and an I2S / I2C controller 1040. In at least one embodiment, the integrated circuit 1000A may include a display device 1045 coupled to one or more of a High Definition Multimedia Interface (HDMI) controller 1050 and a Mobile Industrial Processor Interface (MIPI) display interface 1055. In at least one embodiment, storage may be provided by a flash memory subsystem 1060, including flash memory and a flash memory controller. In at least one embodiment, a memory interface may be provided via a memory controller 1065 for accessing an SDRAM or SRAM memory device. In at least one embodiment, some integrated circuits also include an embedded security engine 1070.

[0195] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 may be used in the integrated circuit 1000A to infer or predict operations based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0196] Figure 10B-10CExemplary integrated circuits and associated graphics processors according to various embodiments herein are illustrated, which may be manufactured using one or more IP cores to support and / or implement the heat load systems described herein for testing data center liquid cooling systems. In addition to the illustrations, other logic and circuitry, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores, may be included in at least one embodiment.

[0197] Figure 10B-10C This is a block diagram illustrating an exemplary graphics processor used within a SoC according to embodiments described herein to support and / or implement the thermal load system described herein for testing data center liquid cooling systems. In one example, the graphics processor can be used for intelligent control of the thermal load system for testing data center liquid cooling systems because existing mathematical engines are capable of processing multi-level neural networks quickly. Figure 10B An exemplary graphics processor 1010 of a system-on-a-chip according to at least one embodiment is shown, which can be manufactured using one or more IP cores. Figure 10C Further exemplary graphics processor 1040 of a system-on-a-chip according to at least one embodiment is shown, which can be manufactured using one or more IP cores. In at least one embodiment, Figure 10B The graphics processor 1010 is a low-power graphics processor core. In at least one embodiment, Figure 10C The graphics processor 1040 is a higher-performance graphics processor core. In at least one embodiment, each graphics processor 1010, 1040 may be... Figure 10A A variant of the 1010 graphics processor.

[0198] In at least one embodiment, the graphics processor 1010 includes a vertex processor 1005 and one or more fragment processors 1015A-1015N (e.g., 1015A, 1015B, 1015C, 1015D to 1015N-1 and 1015N). In at least one embodiment, the graphics processor 1010 can execute different shader programs via separate logic, such that the vertex processor 1005 is optimized to perform operations for the vertex shader program, while one or more fragment processors 1015A-1015N perform fragment (e.g., pixel) shading operations for fragments or pixels or shader programs. In at least one embodiment, the vertex processor 1005 performs the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. In at least one embodiment, one or more fragment processors 1015A-1015N use the primitive and vertex data generated by the vertex processor 1005 to generate framebuffers for display on a display device. In at least one embodiment, one or more fragment processors 1015A-1015N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform operations similar to those of pixel shader programs provided in the Direct 3D API.

[0199] In at least one embodiment, the graphics processor 1010 additionally includes one or more memory management units (MMUs) 1020A-1020B, one or more caches 1025A-1025B, and one or more circuit interconnects 1030A-1030B. In at least one embodiment, one or more MMUs 1020A-1020B provide a virtual-to-physical address mapping for the graphics processor 1010, including for the vertex processor 1005 and / or fragment processors 1015A-1015N, which can reference vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in one or more caches 1025A-1025B. In at least one embodiment, one or more MMUs 1020A-1020B can be synchronized with other MMUs within the system, including with... Figure 10A One or more application processors 1005, image processors 1015, and / or video processors 1020 are associated with one or more MMUs, enabling each processor 1005-1020 to participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 1030A-1030B enable the graphics processor 1010 to connect to other IP cores within the SoC via the SoC's internal bus or via a direct connection.

[0200] In at least one embodiment, the graphics processor 1040 includes Figure 10AThe graphics processor 1010 includes one or more MMUs 1020A-1020B, caches 1025A-1025B, and circuit interconnects 1030A-1030B. In at least one embodiment, the graphics processor 1040 includes one or more shader cores 1055A-1055N (e.g., 1055A, 1055B, 1055C, 1055D, 1055E, 1055F to 1055N-1 and 1055N), such as Figure 10B As shown, it provides a unified shader core architecture, where a single core or type or core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores can vary. In at least one embodiment, the graphics processor 1040 includes an inter-core task manager 1045, which acts as a thread dispatcher to assign execution threads to one or more shader cores 1055A-1055N and a tile unit 1058 to accelerate tile-based rendering operations, where scene rendering operations are subdivided in image space, for example, to take advantage of local spatial consistency within the scene or optimize the use of internal caches.

[0201] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the inference and / or training logic 615 may be integrated into an integrated circuit. Figure 10A and / or Figure 10B The above is used for inference or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions or architectures, or the neural network use cases described herein.

[0202] Figure 10D-10E Additional exemplary graphics processor logic according to embodiments described herein is illustrated to support and / or implement the heat load system described herein for testing data center liquid cooling systems. In at least one embodiment, Figure 10D It shows that it can be included in Figure 10A The graphics core 1000D within the graphics processor 1010, and in at least one embodiment, may be as follows: Figure 10C The unified shader cores shown are 1055A-1055N. Figure 10B A highly parallel general-purpose graphics processing unit (“GPGPU”) 1030 suitable for deployment on a multi-chip module is shown in at least one embodiment.

[0203] In at least one embodiment, the graphics core 1000D may include multiple slices 1001A-1001N or partitions of each core, and the graphics processor may include multiple instances of the graphics core 1000D. In at least one embodiment, slices 1001A-1001N may include supporting logic, including local instruction caches 1004A-1004N, thread schedulers 1006A-1006N, thread dispatchers 1008A-1008N, and a set of registers 1010A-1010N. In at least one embodiment, slices 1001A-1001N may include a set of additional functional units (AFU 1012A-1012N), floating-point units (FPU 1014A-1014N), integer arithmetic logic units (ALU 109A-109N), address calculation units (ACU 1013A-1013N), double-precision floating-point units (DPFPU 1015A-1015N), and matrix processing units (MPU 1017A-1017N).

[0204] In at least one embodiment, the FPU 1014A-1014N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while the DPFPU 1015A-1015N performs double-precision (64-bit) floating-point operations. In at least one embodiment, the ALU 1016A-1016N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision, and can be configured for mixed-precision operations. In at least one embodiment, the MPU 1017A-1017N can also be configured for mixed-precision matrix operations, including half-precision floating-point operations and 8-bit integer operations. In at least one embodiment, the MPU 1017A-1010N can perform various matrix operations to accelerate machine learning application frameworks, including enabling support for accelerated generalized matrix-to-matrix multiplication (GEMM). In at least one embodiment, the AFU 1012A-1012N can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0205] As discussed elsewhere in this disclosure, reasoning and / or training logic 615 (at least in Figure 6B , Figure 6C (Referencing) is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided. In at least one embodiment, inference and / or training logic 615 may be used in the graphics core 1000D to infer or predict operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases herein.

[0206] Figure 11A A block diagram of a computer system 1100A according to at least one embodiment is shown. In at least one embodiment, the computer system 1100A includes a processing subsystem 1101 having one or more processors 1102 and a system memory 1104, the system memory 1104 communicating via an interconnect path that may include a memory hub 1105. In at least one embodiment, the memory hub 1105 may be a separate component within a chipset component or may be integrated within one or more processors 1102. In at least one embodiment, the memory hub 1105 is coupled to an I / O subsystem 1111 via a communication link 1106. In one embodiment, the I / O subsystem 1111 includes an I / O hub 1107, which enables the computer system 1100A to receive input from one or more input devices 1108. In at least one embodiment, the I / O hub 1107 enables a display controller to provide output to one or more display devices 1110A, the display controller being included in one or more processors 1102. In at least one embodiment, one or more display devices 1110A coupled to the I / O hub 1107 may include local, internal, or embedded display devices.

[0207] In at least one embodiment, the processing subsystem 1101 includes one or more parallel processors 1112 coupled to the memory hub 1105 via a bus or other communication link 1113. In at least one embodiment, the communication link 1113 may use any of many standards-based communication link technologies or protocols, such as, but not limited to, PCI Express, or may be a vendor-specific communication interface or communication architecture. In at least one embodiment, the one or more parallel processors 1112 form a compute-intensive parallel or vector processing system, which may include a large number of processing cores and / or processing clusters, such as multi-core integrated (MIC) processors. In at least one embodiment, the one or more parallel processors 1112 form a graphics processing subsystem that can output pixels to one or more display devices 1110A coupled via an I / O hub 1107. In at least one embodiment, the one or more parallel processors 1112 may also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 1110B.

[0208] In at least one embodiment, system storage unit 1114 may be connected to I / O hub 1107 to provide a storage mechanism for computer system 1100A. In at least one embodiment, I / O switch 1116 may be used to provide an interface mechanism to enable connectivity between I / O hub 1107 and other components, such as network adapter 1118 and / or wireless network adapter 1119 which may be integrated into one or more platforms, and various other devices that can be added via one or more add-on devices 1120. In at least one embodiment, network adapter 1118 may be an Ethernet adapter or another wired network adapter. In at least one embodiment, wireless network adapter 1119 may include one or more of Wi-Fi, Bluetooth, Near Field Communication (NFC), or other network devices including one or more wireless devices.

[0209] In at least one embodiment, the computer system 1100A may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., and these other components may also be connected to the I / O hub 1107. In at least one embodiment, the interconnection can be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI-Express) or other bus or point-to-point communication interface and / or protocol. Figure 11A The communication paths of the various components, such as NV-Link high-speed interconnect or interconnect protocols.

[0210] In at least one embodiment, one or more parallel processors 1112 include circuitry optimized for graphics and video processing, including, for example, video output circuitry, and constituting a graphics processing unit (GPU). In at least one embodiment, one or more parallel processors 1112 include circuitry optimized for general-purpose processing. In at least one embodiment, components of the computer system 1100A may be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, one or more parallel processors 1112, a memory hub 1105, one or more processors 1102, and an I / O hub 1107 may be integrated into a system-on-a-chip (SoC) integrated circuit. In at least one embodiment, components of the computer system 1100A may be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least a portion of the components of the computer system 1100A may be integrated into a multi-chip module (MCM), which may interconnect with other MCMs to a modular computer system.

[0211] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 can be... Figure 11A Used in systems for reasoning or predicting operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0212] processor

[0213] Figure 11B A parallel processor 1100B according to at least one embodiment is illustrated. In at least one embodiment, various components of the parallel processor 1100B may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). In at least one embodiment, the illustrated parallel processor 1100B is according to an exemplary embodiment. Figure 11B The variant of the one or more parallel processors 1112 shown.

[0214] In at least one embodiment, the parallel processor 1100B includes a parallel processing unit 1102. In at least one embodiment, the parallel processing unit 1102 includes an I / O unit 1104 that enables communication with other devices, including other instances of the parallel processing unit 1102. In at least one embodiment, the I / O unit 1104 can be directly connected to other devices. In at least one embodiment, the I / O unit 1104 is connected to other devices using a hub or switch interface (e.g., a memory hub 1105). In at least one embodiment, the connection between the memory hub 1105 and the I / O unit 1104 forms a communication link 1113. In at least one embodiment, the I / O unit 1104 is connected to a host interface 1106 and a memory crossbar switch 1116, wherein the host interface 1106 receives commands for performing processing operations, and the memory crossbar switch 1116 receives commands for performing memory operations.

[0215] In at least one embodiment, when host interface 1106 receives a command buffer via I / O unit 1104, host interface 1106 can direct work operations to execute those commands to front end 1108. In at least one embodiment, front end 1108 is coupled to scheduler 1110, which is configured to assign commands or other work items to processing cluster array 1112. In at least one embodiment, scheduler 1110 ensures that processing cluster array 1112 is correctly configured and in an active state before assigning tasks to processing cluster array 1112. In at least one embodiment, scheduler 1110 is implemented via firmware logic executed on a microcontroller. In at least one embodiment, the microcontroller-implemented scheduler 1110 can be configured to perform complex scheduling and work assignment operations at both coarse and fine granular levels, enabling fast preemption and context switching of threads executing on processing array 1112. In at least one embodiment, host software can demonstrate workloads scheduled on processing array 1112 via one of the multiple graphics processing doorbells. In at least one embodiment, the workload can then be automatically distributed on the processing array 1112 by the scheduler 1110 logic within the microcontroller, which includes the scheduler 1110.

[0216] In at least one embodiment, the processing cluster array 1112 may include up to "N" processing clusters (e.g., clusters 1114A, 1114B to 1114N). In at least one embodiment, each cluster 1114A-1114N of the processing cluster array 1112 may execute a large number of concurrent threads. In at least one embodiment, the scheduler 1110 may use various scheduling and / or work allocation algorithms to allocate work to the clusters 1114A-1114N of the processing cluster array 1112, which may vary depending on the workload generated by each type of program or computation. In at least one embodiment, scheduling may be handled dynamically by the scheduler 1110, or may be partially assisted by compiler logic during the compilation of program logic configured to be executed by the processing cluster array 1112. In at least one embodiment, different clusters 1114A-1114N of the processing cluster array 1112 may be assigned to process different types of programs or to perform different types of computations.

[0217] In at least one embodiment, the processing cluster array 1112 can be configured to perform various types of parallel processing operations. In at least one embodiment, the processing cluster array 1112 is configured to perform general-purpose parallel computing operations. In at least one embodiment, the processing cluster array 1112 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations, including physical operations, and performing data transformations.

[0218] In at least one embodiment, the processing cluster array 1112 is configured to perform parallel graphics processing operations. In at least one embodiment, the processing cluster array 1112 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, the processing cluster array 1112 may be configured to execute shader programs related to graphics processing, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. In at least one embodiment, the parallel processing unit 1102 may transfer data from system memory via I / O unit 1104 for processing. In at least one embodiment, during processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 1122) and then written back to system memory.

[0219] In at least one embodiment, when the parallel processing unit 1102 is used to perform graphics processing, the scheduler 1110 may be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations among the multiple clusters 1114A-1114N of the processing cluster array 1112. In at least one embodiment, portions of the processing cluster array 1112 may be configured to perform different types of processing. In at least one embodiment, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen-space operations to generate rendered images for display, if valve control simulation for a heat load system used to test a data center liquid cooling system is required. In at least one embodiment, intermediate data generated by one or more of the clusters 1114A-1114N may be stored in a buffer to allow intermediate data to be transferred between the clusters 1114A-1114N for further processing.

[0220] In at least one embodiment, the processing cluster array 1112 may receive processing tasks to be executed via a scheduler 1110, which receives commands defining the processing tasks from a front end 1108. In at least one embodiment, the processing task may include an index of data to be processed, such as surface (patch) data, raw data, vertex data, and / or pixel data, as well as state parameters and commands defining how the data is processed (e.g., what program to execute). In at least one embodiment, the scheduler 1110 may be configured to acquire an index corresponding to a task, or may receive an index from the front end 1108. In at least one embodiment, the front end 1108 may be configured to ensure that the processing cluster array 1112 is configured to be active before initiating a workload specified by an incoming command buffer (e.g., a batch buffer, push buffer, etc.).

[0221] In at least one embodiment, each of one or more instances of the parallel processing unit 1102 may be coupled to the parallel processor memory 1122. In at least one embodiment, the parallel processor memory 1122 may be accessed via a memory crossbar switch 1116, which may receive memory requests from the processing cluster array 1112 and the I / O unit 1104. In at least one embodiment, the memory crossbar switch 1116 may be accessed via a memory interface 1118. In at least one embodiment, the memory interface 1118 may include a plurality of partition units (e.g., partition units 1120A, 1120B to 1120N), each of which may be coupled to a portion (e.g., a memory cell) of the parallel processor memory 1122. In at least one embodiment, the plurality of partition units 1120A-1120N are configured to be equal to the number of memory units, such that the first partition unit 1120A has a corresponding first memory unit 1124A, the second partition unit 1120B has a corresponding memory unit 1124B, and the Nth partition unit 1120N has a corresponding Nth memory unit 1124N. In at least one embodiment, the number of partition units 1120A-1120N may not be equal to the number of memory devices.

[0222] In at least one embodiment, memory cells 1124A-1124N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory cells 1124A-1124N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). In at least one embodiment, rendering targets such as frame buffers or texture maps may be stored across memory cells 1124A-1124N, allowing partitioning cells 1120A-1120N to write portions of each rendering target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 1122. In at least one embodiment, local instances of the parallel processor memory 1122 may be excluded to facilitate a unified memory design that combines system memory with local cache memory.

[0223] In at least one embodiment, any of the clusters 1114A-1114N of the processing cluster array 1112 can process data to be written to any memory cell 1124A-1124N within the parallel processor memory 1122. In at least one embodiment, the memory crossbar switch 1116 can be configured to transfer the output of each cluster 1114A-1114N to any partition cell 1120A-1120N or another cluster 1114A-1114N, and the clusters 1114A-1114N can perform further processing operations on the output. In at least one embodiment, each cluster 1114A-1114N can communicate with the memory interface 1118 via the memory crossbar switch 1116 to read from or write to various external storage devices. In at least one embodiment, the memory crossbar switch 1116 has a connection to a memory interface 1118 for communication with I / O unit 1104, and a connection to a local instance of parallel processor memory 1102, thereby enabling processing units within different processing clusters 1114A-1114N to communicate with system memory or other memory not local to parallel processing unit 1102. In at least one embodiment, the memory crossbar switch 1116 may use virtual channels to separate traffic flows between clusters 1114A-1114N and partition units 1120A-1120N.

[0224] In at least one embodiment, multiple instances of the parallel processing unit 1102 may be provided on a single insert card, or multiple insert cards may be interconnected. In at least one embodiment, different instances of the parallel processing unit 1102 may be configured to interoperate, even if the different instances have different numbers of processing cores, different numbers of local parallel processor memories, and / or other configuration differences. In at least one embodiment, some instances of the parallel processing unit 1102 may include higher-precision floating-point units relative to other instances. In at least one embodiment, a system combining one or more instances of the parallel processing unit 1102 or the parallel processor 1100B can be implemented in various configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0225] Figure 11C This is a block diagram of a partitioning unit 1120 according to at least one embodiment. In at least one embodiment, the partitioning unit 1120 is... Figure 11B This is an example of one of the partitioning units 1120A-1120N. In at least one embodiment, partitioning unit 1120 includes an L2 cache 1121, a frame buffer interface 1125, and a ROP 1126 (raster operation unit). The L2 cache 1121 is a read / write cache configured to perform load and store operations received from memory crossbar switch 1116 and ROP 1126. In at least one embodiment, L2 cache 1121 outputs read miss and urgent write-back requests to frame buffer interface 1125 for processing. In at least one embodiment, updates can also be sent to the frame buffer for processing via frame buffer interface 1125. In at least one embodiment, frame buffer interface 1125 communicates with memory cells in parallel processor memory (such as...). Figure 11B The memory cells 1124A-1124N (e.g., within the parallel processor memory 1122) interact with each other.

[0226] In at least one embodiment, ROP 1126 is a processing unit that performs raster operations such as stenciling, z-testing, blending, and so on. In at least one embodiment, ROP 1126 then outputs processed graphics data stored in graphics memory. In at least one embodiment, ROP 1126 includes compression logic to compress depth or color data written to memory and decompress depth or color data read from memory. In at least one embodiment, the compression logic may be lossless compression logic utilizing one or more of a variety of compression algorithms. The compression logic performed by ROP 1126 may vary based on the statistical characteristics of the data to be compressed. In at least one embodiment, incremental color compression is performed based on depth and color data on a per-tile basis.

[0227] In at least one embodiment, ROP 1126 is included within each processing cluster (e.g., Figure 11B Clusters 1114A-1114N are used instead of partition units 1120. In at least one embodiment, read and write requests for pixel data are made via memory crossbar switch 1116 instead of pixel fragment data transfer. In at least one embodiment, the processed graphics data can be displayed on a display device (such as...). Figure 11A Displayed by one or more display devices 1110, routed by processor 1102 for further processing, or by... Figure 11B One of the processing entities within the parallel processor 1100B is routed for further processing.

[0228] Figure 11D This is a block diagram of a processing cluster 1114 within a parallel processing unit according to at least one embodiment. In at least one embodiment, the processing cluster is... Figure 11B An instance of one of the processing clusters 1114A-1114N. In at least one embodiment, one or more processing clusters 1114 can be configured to execute a number of threads in parallel, where a "thread" refers to an instance of a specific program executing on a particular set of input data. In at least one embodiment, Single Instruction Multiple Data (SIMD) instruction issuing technology is used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, Single Instruction Multiple Threading (SIMT) technology is used to support the parallel execution of a large number of synchronized threads, which uses a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster.

[0229] In at least one embodiment, the operation of the processing cluster 1114 can be controlled by a pipeline manager 1132 that assigns processing tasks to the SIMT parallel processors. In at least one embodiment, the pipeline manager 1132... Figure 11BThe scheduler 1110 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1134 and / or texture unit 1136. In at least one embodiment, the graphics multiprocessor 1134 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, the processing cluster 1114 may include various types of SIMT parallel processors with different architectures. In at least one embodiment, the processing cluster 1114 may include one or more instances of the graphics multiprocessor 1134. In at least one embodiment, the graphics multiprocessor 1134 can process data, and the data crossover switch 1140 can be used to distribute the processed data to one of a number of possible destinations, including other shader units. In at least one embodiment, the pipeline manager 1132 can facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data crossover switch 1140.

[0230] In at least one embodiment, each graphics multiprocessor 1134 within the processing cluster 1114 may include the same set of functional execution logic (e.g., arithmetic logic units, load-memory units, etc.). In at least one embodiment, the functional execution logic may be configured in a pipelined manner, wherein new instructions may be issued before previous instructions complete. In at least one embodiment, the functional execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, shift operations, and computation of various algebraic functions. In at least one embodiment, the same functional unit hardware may be used to perform different operations, and any combination of functional units may exist.

[0231] In at least one embodiment, instructions sent to the processing cluster 1114 constitute threads. In at least one embodiment, a group of threads executed across a set of parallel processing engines is a thread group. In at least one embodiment, the thread group executes programs on different input data. In at least one embodiment, each thread within the thread group may be assigned to a different processing engine within the graphics multiprocessor 1134. In at least one embodiment, the thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 1134. In at least one embodiment, when the number of threads included in the thread group is less than the number of processing engines, one or more processing engines may be idle during a loop that is processing the thread group. In at least one embodiment, the thread group may also include more threads than the number of processing engines within the graphics multiprocessor 1134. In at least one embodiment, when the thread group includes more threads than the number of processing engines within the graphics multiprocessor 1134, processing can be performed in consecutive clock cycles. In at least one embodiment, multiple thread groups can be executed simultaneously on the graphics multiprocessor 1134.

[0232] In at least one embodiment, the graphics multiprocessor 1134 includes an internal cache memory for performing load and store operations. In at least one embodiment, the graphics multiprocessor 1134 may forgo the internal cache and use a cache memory within the processing cluster 1114 (e.g., L1 cache 1148). In at least one embodiment, each graphics multiprocessor 1134 may also access partition units (e.g., Figure 11B The L2 cache is located within partition units 1120A-1120N, which are shared across all processing clusters 1114 and can be used to transfer data between threads. In at least one embodiment, the graphics multiprocessor 1134 can also access off-chip global memory, which may include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory outside of the parallel processing unit 1102 can be used as global memory. In at least one embodiment, the processing cluster 1114 includes multiple instances of the graphics multiprocessor 1134, which can share common instructions and data that can be stored in the L1 cache 1148.

[0233] In at least one embodiment, each processing cluster 1114 may include a memory management unit (“MMU”) 1145 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of the MMU 1145 may reside in Figure 11B The memory interface 1118 is located within the MMU 1145. In at least one embodiment, the MMU 1145 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles and, in at least one embodiment, to cache line indices. In at least one embodiment, the MMU 1145 may include an address translation back buffer (TLB) or a cache that may reside within the graphics multiprocessor 1134, the L1 cache 1148, or the processing cluster 1114. In at least one embodiment, physical addresses are processed to allocate surface data access locality for efficient request interleaving between partition units. In at least one embodiment, cache line indices may be used to determine whether a request for a cache line is a hit or a miss.

[0234] In at least one embodiment, the processing cluster 1114 can be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 to perform texture mapping operations, which determine texture sample locations, read texture data, and filter texture data. In at least one embodiment, texture data is read as needed from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 1134, and texture data is also retrieved from an L2 cache, local parallel processor memory, or system memory. In at least one embodiment, each graphics multiprocessor 1134 outputs one or more processed tasks to a data crossbar switch 1140 to provide the processed tasks to another processing cluster 1114 for further processing or to store one or more processed tasks in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 1116. In at least one embodiment, a preROP 1142 (pre-raster operation unit) is configured to receive data from the graphics multiprocessor 1134 and direct the data to a ROP unit, which may be associated with partitioning units (e.g., ...) Figure 11B The PreROP 1142 unit is positioned together with the partition units 1120A-1120N. In at least one embodiment, the PreROP 1142 unit can perform optimizations for color blending, organize pixel color data, and perform address translation.

[0235] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. (See below for details.) Figure 6B and / or Figure 6C Details regarding inference and / or training logic 615 are provided. In at least one embodiment, inference and / or training logic 615 may be used in a graphics processing cluster 1114 to perform inference or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0236] Figure 11EA graphics multiprocessor 1134 according to at least one embodiment is illustrated. In at least one embodiment, the graphics multiprocessor 1134 is coupled to a pipeline manager 1132 of a processing cluster 1114. In at least one embodiment, the graphics multiprocessor 1134 has an execution pipeline including, but not limited to, an instruction cache 1152, an instruction unit 1154, an address mapping unit 1156, a register file 1158, one or more general-purpose graphics processing unit (GPGPU) cores 1162, and one or more load / store units 1166. The one or more GPGPU cores 1162 and one or more load / store units 1166 are coupled to a cache memory 1172 and a shared memory 1170 via a memory and cache interconnect 1168.

[0237] In at least one embodiment, instruction cache 1152 receives a stream of instructions to be executed from pipeline manager 1132. In at least one embodiment, instructions are cached in instruction cache 1152 and dispatched to instruction unit 1154 for execution. In one embodiment, instruction unit 1154 may dispatch instructions as thread groups (e.g., thread bundles), assigning each thread group to a different execution unit within one or more GPGPU cores 1162. In at least one embodiment, instructions can access any local, shared, or global address space by specifying an address within a unified address space. In at least one embodiment, address mapping unit 1156 may be used to translate addresses in the unified address space into different memory addresses that can be accessed by one or more load / store units 1166.

[0238] In at least one embodiment, register file 1158 provides a set of registers for functional units of graphics multiprocessor 1134. In at least one embodiment, register file 1158 provides temporary storage for operands of data paths connected to functional units of graphics multiprocessor 1134 (e.g., GPGPU core 1162, load / store unit 1166). In at least one embodiment, register file 1158 is partitioned among each functional unit, such that a dedicated portion of register file 1158 is allocated to each functional unit. In at least one embodiment, register file 1158 is partitioned among different thread bundles being executed by graphics multiprocessor 1134.

[0239] In at least one embodiment, each of the GPGPU cores 1162 may include a floating-point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1134. The GPGPU cores 1162 may be architecturally similar or may differ in architecture. In at least one embodiment, a first portion of the GPGPU core 1162 includes a single-precision FPU and an integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. In at least one embodiment, the FPU may implement the IEEE 754-2008 standard for floating-point algorithms or enable variable-precision floating-point algorithms. In at least one embodiment, the graphics multiprocessor 1134 may additionally include one or more fixed-function or special-function units to perform specific functions, such as copying rectangles or pixel blending operations. In at least one embodiment, one or more of the GPGPU cores may also include fixed-function or special-function logic.

[0240] In at least one embodiment, the GPGPU core 1162 includes SIMD logic capable of executing a single instruction on multiple sets of data. In one embodiment, the GPGPU core 1162 can physically execute SIMD4, SIMD8, and SIMD16 instructions and logically execute SIMD1, SIMD2, and SIMD32 instructions. In at least one embodiment, the SIMD instructions for the GPGPU core can be generated by a shader compiler at compile time or automatically generated when executing a program written and compiled for a Single Program Multiple Data (SPMD) or SIMT architecture. In at least one embodiment, multiple threads of a program configured for a SIMT execution model can be executed using a single SIMD instruction. For example, in at least one embodiment, eight SIMD threads performing the same or similar operations can be executed in parallel using a single SIMD8 logic unit.

[0241] In at least one embodiment, the memory and cache interconnect 1168 is an interconnect network connecting each functional unit of the graphics multiprocessor 1134 to the register file 1158 and the shared memory 1170. In at least one embodiment, the memory and cache interconnect 1168 is a cross-switch interconnect that allows the load / store unit 1166 to perform load and store operations between the shared memory 1170 and the register file 1158. In at least one embodiment, the register file 1158 can operate at the same frequency as the GPGPU core 1162, resulting in very low latency for data transfer between the GPGPU core 1162 and the register file 1158. In at least one embodiment, the shared memory 1170 can be used to enable communication between threads executing on functional units within the graphics multiprocessor 1134. In at least one embodiment, the cache memory 1172 can be used, for example, as a data cache to cache texture data communicated between functional units and texture units 1136. In at least one embodiment, the shared memory 1170 can also be used as a program-managed cache. In at least one embodiment, in addition to the automatically cached data stored in cache memory 1172, the thread executing on GPGPU core 1162 can also programmatically store data in shared memory.

[0242] In at least one embodiment, a parallel processor or GPGPU, as described herein, is communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. In at least one embodiment, the GPU may be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., high-speed interconnects such as PCIe or NVLink). In at least one embodiment, the GPU may be integrated with the core on the same package or chip and communicatively coupled to the core via an internal processor bus / interconnect (within the package or chip in at least one embodiment). In at least one embodiment, regardless of how the GPU is connected, the processor core may assign work to the GPU in the form of a sequence of commands / instructions contained in a job descriptor. In at least one embodiment, the GPU then uses dedicated circuitry / logic to efficiently process these commands / instructions.

[0243] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6CDetails are provided regarding the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 may be used in the graphics multiprocessor 1134 to perform inference or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0244] Figure 12A A multi-GPU computing system 1200A according to at least one embodiment is illustrated. In at least one embodiment, the multi-GPU computing system 1200A may include a processor 1202 coupled to a plurality of general-purpose graphics processing units (GPGPUs) 1206A-D via a host interface switch 1204. In at least one embodiment, the host interface switch 1204 is a PCI Express switch device that couples the processor 1202 to a PCI Express bus, through which the processor 1202 communicates with the GPGPUs 1206A-D. The GPGPUs 1206A-D may be interconnected via a set of high-speed P2P GPU-to-GPU links 1216. In at least one embodiment, the GPU-to-GPU links 1216 are connected to each of the GPGPUs 1206A-D via dedicated GPU links. In at least one embodiment, the P2P GPU links 1216 enable direct communication between each of the GPGPUs 1206A-D without communication via the host interface bus 1204 to which the processor 1202 is connected. In at least one embodiment, when GPU-to-GPU traffic is directed to the P2P GPU link 1216, the host interface bus 1204 remains available for system memory access or, for example, communication with other instances of the multi-GPU computing system 1200A via one or more network devices. While in at least one embodiment, the GPGPU 1206A-D is connected to the processor 1202 via the host interface switch 1204, in at least one embodiment, the processor 1202 includes direct support for the P2P GPU link 1216 and can be directly connected to the GPGPU 1206A-D.

[0245] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details are provided regarding inference and / or training logic 615. In at least one embodiment, inference and / or training logic 615 may be used in a multi-GPU computing system 1200A for performing inference or prediction operations based at least in part on weight parameters computed using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0246] Figure 12B This is a block diagram of a graphics processor 1200B according to at least one embodiment. In at least one embodiment, the graphics processor 1200B includes a ring interconnect 1202, a pipeline front end 1204, a media engine 1237, and graphics cores 1280A-1280N. In at least one embodiment, the ring interconnect 1202 couples the graphics processor 1200B to other processing units, which include other graphics processors or one or more general-purpose processor cores. In at least one embodiment, the graphics processor 1200B is one of many processors integrated within a multi-core processing system.

[0247] In at least one embodiment, the graphics processor 1200B receives multiple batches of commands via a ring interconnect 1202. In at least one embodiment, the input commands are interpreted by a command streamer 1203 in a pipeline front-end 1204. In at least one embodiment, the graphics processor 1200B includes scalable execution logic for performing 3D geometry processing and media processing via graphics cores 1280A-1280N. In at least one embodiment, for 3D geometry processing commands, the command streamer 1203 provides the commands to the geometry pipeline 1236. In at least one embodiment, for at least some media processing commands, the command streamer 1203 provides the commands to a video front-end 1234, which is coupled to a media engine 1237. In at least one embodiment, the media engine 1237 includes a video quality engine (VQE) 1230 for video and image post-processing, and a multi-format encoding / decoding (MFX) engine 1233 for providing hardware-accelerated media data encoding and decoding. In at least one embodiment, the geometry pipeline 1236 and the media engine 1237 each generate an execution thread for the thread execution resources provided by at least one graphics core 1280A.

[0248] In at least one embodiment, the graphics processor 1200B includes scalable thread execution resources featuring modular cores 1280A-1280N (sometimes referred to as core slices), each graphics core having multiple sub-cores 1250A-1250N, 1260A-1260N (sometimes referred to as core sub-slices). In at least one embodiment, the graphics processor 1200B may have any number of graphics cores 1280A. In at least one embodiment, the graphics processor 1200B includes graphics cores 1280A having at least a first sub-core 1250A and a second sub-core 1260A. In at least one embodiment, the graphics processor 1200B is a low-power processor with a single sub-core (e.g., 1250A). In at least one embodiment, the graphics processor 1200B includes multiple graphics cores 1280A-1280N, each graphics core including a set of first sub-cores 1250A-1250N and a set of second sub-cores 1260A-1260N. In at least one embodiment, each of the first sub-cores 1250A-1250N includes at least a first set of execution units 1252A-1252N and media / texture samplers 1254A-1254N. In at least one embodiment, each of the second sub-cores 1260A-1260N includes at least a second set of execution units 1262A-1262N and samplers 1264A-1264N. In at least one embodiment, each of the sub-cores 1250A-1250N and 1260A-1260N shares a set of shared resources 1270A-1270N. In at least one embodiment, the shared resources include shared cache memory and pixel operation logic.

[0249] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, the inference and / or training logic 615 may be used in the graphics processor 1200B to perform inference or prediction operations based at least in part on weight parameters calculated using neural network training operations, neural network functions and / or architectures or neural network use cases described herein.

[0250] Figure 13This is a block diagram illustrating the microarchitecture of a processor 1300 according to at least one embodiment, which may include logic circuitry for executing instructions. In at least one embodiment, the processor 1300 may execute instructions, including x86 instructions, ARM instructions, and special-purpose instructions for application-specific integrated circuits (ASICs). In at least one embodiment, the processor 1300 may include registers for storing packaged data, such as the 64-bit wide MMX™ registers in an Intel microprocessor enabled by MMX technology in Santa Clara, California. In at least one embodiment, the MMX registers available in integer and floating-point forms may operate with packaged data elements accompanied by Single Instruction Multiple Data (“SIMD”) and Streaming SIMD Extensions (“SSE”) instructions. In at least one embodiment, a 128-bit wide XMM register associated with SSE2, SSE3, SSE4, AVX, or later (generally referred to as “SSEx”) technologies may hold such packaged data operands. In at least one embodiment, the processor 1300 may execute instructions to accelerate machine learning or deep learning algorithms, training, or inference.

[0251] In at least one embodiment, processor 1300 includes an ordered front end (“front end”) 1301 to fetch instructions to be executed and prepare instructions for later use in the processor pipeline. In at least one embodiment, front end 1301 may include several units. In at least one embodiment, instruction prefetcher 1326 fetches instructions from memory and provides the instructions to instruction decoder 1328, which in turn decodes or interprets the instructions. For example, in at least one embodiment, instruction decoder 1328 decodes the received instructions into one or more machine-executable so-called “micro-instructions” or “micro-operations” (also referred to as “micro-operations” or “micro-instructions”). In at least one embodiment, instruction decoder 1328 parses the instructions into opcodes and corresponding data and control fields, which can be used by the microarchitecture to perform operations according to at least one embodiment. In at least one embodiment, trace cache 1330 may assemble the decoded micro-instructions into a program-ordered sequence or trace in micro-instruction queue 1334 for execution. In at least one embodiment, when the trace cache 1330 encounters complex instructions, the microcode ROM 1332 provides the microinstructions required to complete the operation.

[0252] In at least one embodiment, some instructions may be converted into a single micro-operation, while others require several micro-operations to complete the entire operation. In at least one embodiment, if more than four micro-instructions are required to complete an instruction, the instruction decoder 1328 may access the microcode ROM 1332 to execute the instruction. In at least one embodiment, an instruction may be decoded into a small number of micro-instructions for processing at the instruction decoder 1328. In at least one embodiment, if multiple micro-instructions are required to complete an operation, the instruction may be stored in the microcode ROM 1332. In at least one embodiment, the trace cache 1330 references an entry point programmable logic array (“PLA”) to determine the correct micro-instruction pointer for reading a microcode sequence from the microcode ROM 1332 to complete one or more instructions, according to at least one embodiment. In at least one embodiment, after the microcode ROM 1332 has completed the micro-operation ordering of the instructions, the machine front end 1301 may resume fetching micro-operations from the trace cache 1330.

[0253] In at least one embodiment, the out-of-order execution engine (“out-of-order engine”) 1303 can prepare instructions for execution. In at least one embodiment, the out-of-order execution logic has multiple buffers to smooth and reorder the instruction flow to optimize performance as instructions descend the pipeline and are scheduled for execution. In at least one embodiment, the out-of-order execution engine 1303 includes, but is not limited to, an allocator / register renamer 1340, a memory microinstruction queue 1342, an integer / floating-point microinstruction queue 1344, a memory scheduler 1346, a fast scheduler 1302, a slow / general-purpose floating-point scheduler (“slow / general-purpose FP scheduler”) 1304, and a simple floating-point scheduler (“simple FP scheduler”) 1306. In at least one embodiment, the fast scheduler 1302, the slow / general-purpose floating-point scheduler 1304, and the simple floating-point scheduler 1306 are also collectively referred to as “microinstruction schedulers 1302, 1304, 1306”. In at least one embodiment, the allocator / register renamer 1340 allocates the machine buffers and resources required for the sequential execution of each microinstruction. In at least one embodiment, the allocator / register renamer 1340 renames logical registers to entries in a register file. In at least one embodiment, the allocator / register renamer 1340 also allocates entries for each microinstruction in one of two microinstruction queues, a memory microinstruction queue 1342 for memory operations and an integer / floating-point microinstruction queue 1344 for non-memory operations, preceding the memory scheduler 1346 and microinstruction schedulers 1302, 1304, and 1306. In at least one embodiment, the microinstruction schedulers 1302, 1304, and 1306 determine when they are ready to execute a microinstruction based on the readiness of their dependent input register operand sources and the availability of the execution resource microinstructions that need to be completed. In at least one embodiment, the fast scheduler 1302 of at least one embodiment can schedule on each half of the master clock cycle, while the slow / general-purpose floating-point scheduler 1304 and the simple floating-point scheduler 1306 can schedule once per master processor clock cycle. In at least one embodiment, microinstruction schedulers 1302, 1304, and 1306 arbitrate the scheduling ports to schedule microinstructions for execution.

[0254] In at least one embodiment, execution block 1311 includes, but is not limited to, integer register file / bypass network 1308, floating-point register file / bypass network (“FP register file / bypass network”) 1310, address generation units (“AGU”) 1312 and 1314, fast arithmetic logic units (“fast ALU”) 1316 and 1318, slow arithmetic logic unit (“slow ALU”) 1320, floating-point ALU (“FP”) 1322, and floating-point move unit (“FP move”) 1324. In at least one embodiment, integer register file / bypass network 1308 and floating-point register file / bypass network 1310 are also referred to herein as “register files 1308, 1310”. In at least one embodiment, AGUs 1312 and 1314, fast ALUs 1316 and 1318, slow ALU 1320, floating-point ALU 1322, and floating-point movement unit 1324 are also referred to herein as "execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324". In at least one embodiment, execution block 1311 may include, but is not limited to, any number (including zeros) and type of register files, bypass networks, address generation units, and execution units (in any combination).

[0255] In at least one embodiment, register networks 1308, 1310 may be arranged between microinstruction schedulers 1302, 1304, 1306 and execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324. In at least one embodiment, integer register file / bypass network 1308 performs integer operations. In at least one embodiment, floating-point register file / bypass network 1310 performs floating-point operations. In at least one embodiment, each of register networks 1308, 1310 may include, but is not limited to, a branch network that can bypass or forward recently completed results not yet written to a register file to a new dependent object. In at least one embodiment, register networks 1308, 1310 may communicate data with each other. In at least one embodiment, integer register file / bypass network 1308 may include, but is not limited to, two separate register files, one register file for low-order 32-bit data and a second register file for high-order 32-bit data. In at least one embodiment, the floating-point register file / bypass network 1310 may include, but is not limited to, 128-bit wide entries, since floating-point instructions typically have operands with a width of 64 to 128 bits.

[0256] In at least one embodiment, execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324 can execute instructions. In at least one embodiment, register files 1308 and 1310 store integer and floating-point data operation values ​​that the microinstructions need to execute. In at least one embodiment, processor 1300 can be, but is not limited to, any number of execution units 1312, 1314, 1316, 1318, 1320, 1322, and 1324, and combinations thereof. In at least one embodiment, floating-point ALU 1322 and floating-point move unit 1324 can perform floating-point, MMX, SIMD, AVX, and SSE or other operations, including specialized machine learning instructions. In at least one embodiment, floating-point ALU 1322 can be, but is not limited to, a 64-bit multiplication-64-bit floating-point divider to perform division, square root, and remainder micro-operations. In at least one embodiment, floating-point hardware can be used to process instructions involving floating-point values. In at least one embodiment, ALU operations can be passed to fast ALUs 1316 and 1318. In at least one embodiment, fast ALUs 1316 and 1318 can perform fast operations with an effective delay of half a clock cycle. In at least one embodiment, most complex integer operations are routed to slow ALU 1320, because slow ALU 1320 can include, but is not limited to, integer execution hardware for long-latency type operations, such as multipliers, shifters, flag logic, and branching. In at least one embodiment, memory load / store operations can be performed by AGUs 1312 and 1314. In at least one embodiment, fast ALU 1316, fast ALU 1318, and slow ALU 1320 can perform integer operations on 64-bit data operands. In at least one embodiment, fast ALU 1316, fast ALU 1318, and slow ALU 1320 can be implemented to support various data bit sizes, including sixteen, thirty-two, 128, 256, etc. In at least one embodiment, the floating-point ALU 1322 and the floating-point movement unit 1324 can be implemented to support a range of operands with various bit widths. In at least one embodiment, the floating-point ALU 1322 and the floating-point movement unit 1324 can operate on 128-bit wide packaged data operands in conjunction with SIMD and multimedia instructions.

[0257] In at least one embodiment, microinstruction schedulers 1302, 1304, and 1306 schedule dependent operations before the parent load completes execution. In at least one embodiment, since microinstructions can be speculatively scheduled and executed within processor 1300, processor 1300 may also include logic for handling memory misses. In at least one embodiment, if a data load miss occurs in the data cache, there may be a dependent operation running in the pipeline that temporarily deprives the scheduler of the correct data. In at least one embodiment, a replay mechanism tracks and re-executes instructions that use incorrect data. In at least one embodiment, it may be necessary to replay dependent operations and may allow independent operations to be completed. In at least one embodiment, the scheduler and replay mechanism of at least one embodiment of the processor may also be designed to capture instruction sequences used for text string comparison operations.

[0258] In at least one embodiment, "register" can refer to an onboard processor storage location that can be used as part of an instruction that identifies an operand. In at least one embodiment, a register can be one that can be used externally to the processor (from a programmer's perspective). In at least one embodiment, a register may not be limited to a particular type of circuit. Rather, in at least one embodiment, a register can store data, provide data, and perform the functions described herein. In at least one embodiment, the registers described herein can be implemented using a variety of different techniques via circuitry within the processor, such as dedicated physical registers, dynamically allocated physical registers renamed using register renaming, a combination of dedicated and dynamically allocated physical registers, etc. In at least one embodiment, an integer register stores 32-bit integer data. The register file of at least one embodiment also includes eight multimedia SIMD registers for encapsulating data.

[0259] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, some or all of the inference and / or training logic 615 may be incorporated into execution block 1311 and other memories or registers shown or not shown. For example, in at least one embodiment, the training and / or inference techniques described herein may use one or more ALUs shown in execution block 1311. Furthermore, weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown) that configure the ALUs of execution block 1311 to execute one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0260] Figure 14A deep learning application processor 1400 according to at least one embodiment is illustrated. In at least one embodiment, the deep learning application processor 1400 uses instructions, which, if executed by the deep learning application processor 1400, cause the deep learning application processor 1400 to perform some or all of the processes and techniques described herein. In at least one embodiment, the deep learning application processor 1400 is an application-specific integrated circuit (ASIC). In at least one embodiment, the application processor 1400 performs matrix multiplication operations or is "hardwired" into hardware as a result of executing one or more instructions or both. In at least one embodiment, the deep learning application processor 1400 includes, but is not limited to, processing clusters 1410(1)-1410(12), inter-chip links (“ICL”) 1420(1)-1420(12), inter-chip controllers (“ICC”) 1430(1)-1430(2), memory controllers (“Mem Ctrlr”) 1442(1)-1442(4), high-bandwidth memory physical layers (“HBM PHY”) 1444(1)-1444(4), management controller central processing unit (“management controller CPU”) 1450, serial peripheral interfaces, internal integrated circuits and general-purpose input / output blocks (“SPI, I2C, GPIO”), peripheral component interconnect fast controllers and direct memory access blocks (“PCIe controllers and DMA”) 1470, and sixteen-channel peripheral component interconnect fast ports (“PCI Express x 16”) 1480.

[0261] In at least one embodiment, processing cluster 1410 can perform deep learning operations, including inference or prediction operations based on weight parameters computed using one or more training techniques, including those described herein. In at least one embodiment, each processing cluster 1410 can include, but is not limited to, any number and type of processors. In at least one embodiment, deep learning application processor 1400 can include any number and type of processing cluster 1400. In at least one embodiment, inter-chip link 1420 is bidirectional. In at least one embodiment, inter-chip link 1420 and inter-chip controller 1430 enable multiple deep learning application processors 1400 to exchange information, including activation information generated from executing one or more machine learning algorithms embodied in one or more neural networks. In at least one embodiment, deep learning application processor 1400 can include any number (including zero) and type of ICL 1420 and ICC 1430.

[0262] In at least one embodiment, the HBM2 1440 provides a total of 32GB of memory. The HBM2 1440(i) is associated with both the memory controller 1442(i) and the HBM PHY 1444(i). In at least one embodiment, any number of HBM2 1440s can provide any type and total amount of high-bandwidth memory and can be associated with any number (including zero) and type of memory controller 1442 and HBM PHY 1444. In at least one embodiment, any number and type of blocks can replace SPI, I2C, GPIO 3360, PCIe controller 1460, and DMA 1470 and / or PCIe 1480 to implement any number and type of communication standards in any technically feasible manner.

[0263] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the deep learning application processor is used to train a machine learning model (e.g., a neural network) to predict or infer information provided to the deep learning application processor 1400. In at least one embodiment, the deep learning application processor 1400 is used to infer or predict information based on a trained machine learning model (e.g., a neural network) that has been trained by another processor or system or by the deep learning application processor 1400. In at least one embodiment, the processor 1400 may be used to execute one or more neural network use cases as described herein.

[0264] Figure 15 This is a block diagram of a neuromorphic processor 1500 according to at least one embodiment. In at least one embodiment, the neuromorphic processor 1500 may receive one or more inputs from a source external to the neuromorphic processor 1500. In at least one embodiment, these inputs may be transmitted to one or more neurons 1502 within the neuromorphic processor 1500. In at least one embodiment, the neurons 1502 and their components may be implemented using circuitry or logic including one or more arithmetic logic units (ALUs). In at least one embodiment, the neuromorphic processor 1500 may include, but is not limited to, thousands upon thousands of instances of neurons 1502, but any suitable number of neurons 1502 may be used. In at least one embodiment, each instance of neuron 1502 may include a neuron input 1504 and a neuron output 1506. In at least one embodiment, neuron 1502 may generate an output that can be transmitted to the inputs of other instances of neuron 1502. In at least one embodiment, neuron input 1504 and neuron output 1506 may be interconnected via synapses 1508.

[0265] In at least one embodiment, neuron 1502 and synapse 1508 may be interconnected, causing neuromorphic processor 1500 to operate to process or analyze information received by neuromorphic processor 1500. In at least one embodiment, neuron 1502 may send an output pulse (or “trigger” or “peak”) when the input received through neuron input 1504 exceeds a threshold. In at least one embodiment, neuron 1502 may sum or integrate the signal received at neuron input 1504. For example, in at least one embodiment, neuron 1502 may be implemented as a leaky integral-triggered neuron, wherein if the summation (referred to as a “membrane potential”) exceeds a threshold, neuron 1502 may use a transfer function such as a sigmoid or threshold function to generate an output (or “trigger”). In at least one embodiment, the leaky integral-triggered neuron may sum the signal received at neuron input 1504 to a membrane potential and may apply an attenuation factor (or leak) to reduce the membrane potential. In at least one embodiment, a leaking integral-triggered neuron may trigger if multiple input signals are received at neuron input 1504 quickly enough to exceed a threshold (in at least one embodiment, before the membrane potential decays too low to trigger). In at least one embodiment, neuron 1502 may be implemented using circuitry or logic that receives input, integrates the input to the membrane potential, and decays the membrane potential. In at least one embodiment, the input may be averaged, or any other suitable transfer function may be used. Furthermore, in at least one embodiment, neuron 1502 may include, but is not limited to, comparator circuitry or logic that generates an output spike at neuron output 1506 when the result of applying the transfer function to neuron input 1504 exceeds a threshold. In at least one embodiment, once neuron 1502 is triggered, it can ignore previously received input information by, for example, resetting the membrane potential to 0 or another suitable default value. In at least one embodiment, once the membrane potential is reset to 0, neuron 1502 may resume normal operation after a suitable period of time (or recovery period).

[0266] In at least one embodiment, neurons 1502 can be interconnected via synapses 1508. In at least one embodiment, synapses 1508 are operable to transmit signals from the output of a first neuron 1502 to the input of a second neuron 1502. In at least one embodiment, neurons 1502 can transmit information on more than one instance of synapses 1508. In at least one embodiment, one or more instances of neuron outputs 1506 can be connected via instances of synapses 1508 to instances of neuron inputs 1504 within the same neuron 1502. In at least one embodiment, an instance of neuron 1502 that produces an output to be transmitted on the instance of synapse 1508 may be referred to as a "presynaptic neuron". In at least one embodiment, an instance of neuron 1502 that receives input transmitted via an instance of synapse 1508 may be referred to as a "postsynaptic neuron". In at least one embodiment, regarding various instances of synapse 1508, since instances of neuron 1502 can receive input from one or more instances of synapse 1508 and can also transmit output through one or more instances of synapse 1508, a single instance of neuron 1502 can be both a "presynaptic neuron" and a "postsynaptic neuron".

[0267] In at least one embodiment, neurons 1502 may be organized into one or more layers. Each instance of neuron 1502 may have a neuron output 1506, which can fan out to one or more neuron inputs 1504 via one or more synapses 1508. In at least one embodiment, the neuron output 1506 of neuron 1502 in the first layer 1510 may be connected to the neuron input 1504 of neuron 1502 in the second layer 1512. In at least one embodiment, layer 1510 may be referred to as a “feedforward layer.” In at least one embodiment, each instance of neuron 1502 in an instance of the first layer 1510 may fan out to each instance of neuron 1502 in the second layer 1512. In at least one embodiment, the first layer 1510 may be referred to as a “fully connected feedforward layer.” In at least one embodiment, each instance of neuron 1502 in an instance of the second layer 1512 fan out to fewer than all instances of neuron 1502 in the third layer 1514. In at least one embodiment, the second layer 1512 may be referred to as a “sparsely connected feedforward layer.” In at least one embodiment, neurons 1502 in the (identical) second layer 1512 may fan out to neurons 1502 in multiple other layers, including neurons 1502 fan out to the second layer 1512. In at least one embodiment, the second layer 1512 may be referred to as a "recurrent layer". In at least one embodiment, the neuromorphic processor 1500 may be any suitable combination of recurrent layers and feedforward layers, including but not limited to sparsely connected feedforward layers and fully connected feedforward layers.

[0268] In at least one embodiment, the neuromorphic processor 1500 may include, but is not limited to, a reconfigurable interconnect architecture or dedicated hardwired interconnects to connect synapses 1508 to neurons 1502. In at least one embodiment, the neuromorphic processor 1500 may include, but is not limited to, circuitry or logic that allows synapses to be assigned to different neurons 1502 as needed, depending on the neural network topology and neuron fan-in / fan-out. For example, in at least one embodiment, synapses 1508 may be connected to neurons 1502 using interconnect structures (such as on-chip networks) or via dedicated connections. In at least one embodiment, synaptic interconnects and their components may be implemented using circuitry or logic.

[0269] Figure 16AA processing system according to at least one embodiment is illustrated. In at least one embodiment, system 1600A includes one or more processors 1602 and one or more graphics processors 1608, and may be a single-processor desktop system, a multi-processor workstation system, or a server system having a large number of processors 1602 or processor cores 1607. In at least one embodiment, system 1600A is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices.

[0270] In at least one embodiment, system 1600A may include or be integrated into a server-based gaming platform, including a game console, mobile game console, handheld game console, or online game console, encompassing game and media consoles. In at least one embodiment, system 1600A is a mobile phone, smartphone, tablet computing device, or mobile internet device. In at least one embodiment, processing system 1600A may also include components coupled to or integrated into a wearable device, such as a smartwatch, smart glasses, augmented reality, or virtual reality device. In at least one embodiment, processing system 1600A is a television or set-top box device having one or more processors 1602 and a graphical interface generated by one or more graphics processors 1608.

[0271] In at least one embodiment, each of the one or more processors 1602 includes one or more processor cores 1607 for processing instructions that, when executed, perform operations against the system and user software. In at least one embodiment, each of the one or more processor cores 1607 is configured to process a particular instruction set 1609. In at least one embodiment, the instruction set 1609 may facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computation via Very Long Instruction Word (VLIW). In at least one embodiment, each processor core 1607 may process a different instruction set 1609, which may include instructions that facilitate the emulation of other instruction sets. In at least one embodiment, the processor core 1607 may also include other processing devices, such as a digital signal processor (DSP).

[0272] In at least one embodiment, processor 1602 includes cache memory 1604. In at least one embodiment, processor 1602 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory is shared among various components of processor 1602. In at least one embodiment, processor 1602 also uses an external cache (e.g., a Level 3 (L3) cache or a last-level cache (LLC)) (not shown), which can be shared among processor cores 1607 using known cache coherence techniques. In at least one embodiment, processor 1602 further includes a register file 1606, which may include different types of registers for storing different types of data (e.g., integer registers, floating-point registers, status registers, and instruction pointer registers). In at least one embodiment, register file 1606 may include general-purpose registers or other registers.

[0273] In at least one embodiment, one or more processors 1602 are coupled to one or more interface buses 1610 to transmit communication signals, such as address, data, or control signals, between the processor 1602 and other components in the system 1600A. In at least one embodiment, the interface bus 1610 may be a processor bus, such as a version of the Direct Media Interface (DMI) bus. In at least one embodiment, the interface bus 1610 is not limited to the DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, the processor 1602 includes an integrated memory controller 1616 and a platform controller hub 1630. In at least one embodiment, the memory controller 1616 facilitates communication between memory devices and other components of the processing system 1600A, while the platform controller hub (PCH) 1630 provides connectivity to input / output (I / O) devices via a local I / O bus.

[0274] In at least one embodiment, memory device 1620 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or a device with suitable performance for use as processor memory. In at least one embodiment, memory device 1620 may be used as system memory for processing system 1600A to store data 1622 and instructions 1621 for use when one or more processors 1602 execute an application or process. In at least one embodiment, memory controller 1616 is also coupled to an external graphics processor 1612 of at least one embodiment, which may communicate with one or more graphics processors 1608 of processor 1602 to perform graphics and media operations. In at least one embodiment, display device 1611 may be connected to processor 1602. In at least one embodiment, display device 1611 may include one or more internal display devices, such as in mobile electronic devices or laptop devices, or external display devices connected via a display interface (e.g., DisplayPort). In at least one embodiment, the display device 1611 may include a head-mounted display (HMD), such as a stereoscopic display device for virtual reality (VR) or augmented reality (AR) applications.

[0275] In at least one embodiment, the platform controller hub 1630 enables peripheral devices to connect to the storage device 1620 and the processor 1602 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include, but are not limited to, an audio controller 1646, a network controller 1634, a firmware interface 1628, a wireless transceiver 1626, a touch sensor 1625, and a data storage device 1624 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 1624 may be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCIe). In at least one embodiment, the touch sensor 1625 may include a touchscreen sensor, a pressure sensor, or a fingerprint sensor. In at least one embodiment, the wireless transceiver 1626 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, or LTE transceiver. In at least one embodiment, the firmware interface 1628 enables communication with the system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, network controller 1634 may enable network connectivity to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to interface bus 1610.

[0276] In at least one embodiment, the audio controller 1646 is a multi-channel high-definition audio controller. In at least one embodiment, the processing system 1600A includes a legacy I / O controller 1640 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system 1600A. In at least one embodiment, the platform controller hub 1630 may also be connected to one or more Universal Serial Bus (USB) controllers 1642 that connect input devices, such as a keyboard and mouse combination 1643, a camera 1644, or other USB input devices.

[0277] In at least one embodiment, instances of the memory controller 1616 and platform controller hub 1630 may be integrated into a discrete external graphics processor, such as external graphics processor 1612. In at least one embodiment, the platform controller hub 1630 and / or the memory controller 1616 may be external to one or more processors 1602. In at least one embodiment, system 1600A may include external memory controller 1616 and platform controller hub 1630, which may be configured as a memory controller hub and peripheral controller hub in a system chipset communicating with processor 1602.

[0278] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, some or all of the inference and / or training logic 615 may be incorporated into the graphics processor 1600A. In at least one embodiment, the training and / or inference techniques described herein may use one or more ALUs embodied in the graphics processor 1612. Furthermore, in at least one embodiment, the inference and / or training operations described herein may use, except for... Figure 6B and / or Figure 6C The logic is performed in a manner other than that shown. In at least one embodiment, the weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown), which configure the ALU of the graphics processor 1600A to execute one or more of the machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0279] Figure 16BThis is a block diagram of a processor 1600B having one or more processor cores 1602A-1602N, an integrated memory controller 1614, and an integrated graphics processor 1608 according to at least one embodiment. In at least one embodiment, the processor 1600B may include additional cores, up to and including additional cores 1602N, indicated by dashed boxes. In at least one embodiment, each processor core 1602A-1602N includes one or more internal cache units 1604A-1604N. In at least one embodiment, each processor core may also access one or more shared cache units 1606.

[0280] In at least one embodiment, internal cache units 1604A-1604N and shared cache unit 1606 represent a cache memory hierarchy within processor 1600B. In at least one embodiment, cache memory units 1604A-1604N may include at least one level of instruction and data cache within each processor core and one or more levels of cache in a shared intermediate cache, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, wherein the highest level of cache preceding external memory is classified as LLC. In at least one embodiment, cache coherence logic maintains coherence between the various cache units 1606 and 1604A-1604N.

[0281] In at least one embodiment, the processor 1600B may further include a set of one or more bus controller units 1616 and a system agent core 1610. In at least one embodiment, the one or more bus controller units 1616 manage a set of peripheral buses, such as one or more PCI or PCIe buses. In at least one embodiment, the system agent core 1610 provides management functions for various processor components. In at least one embodiment, the system agent core 1610 includes one or more integrated memory controllers 1614 to manage access to various external memory devices (not shown).

[0282] In at least one embodiment, one or more processor cores 1602A-1602N include support for multithreaded concurrent processing. In at least one embodiment, system agent core 1610 includes components for coordinating and operating cores 1602A-1602N during multithreaded processing. In at least one embodiment, system agent core 1610 may additionally include a power control unit (PCU) including logic and components for regulating one or more power states of processor cores 1602A-1602N and graphics processor 1608.

[0283] In at least one embodiment, processor 1600B further includes a graphics processor 1608 for performing image processing operations. In at least one embodiment, graphics processor 1608 is coupled to a shared cache unit 1606 and a system proxy core 1610 including one or more integrated memory controllers 1614. In at least one embodiment, system proxy core 1610 further includes a display controller 1611 for driving graphics processor outputs to one or more coupled displays. In at least one embodiment, display controller 1611 may also be a separate module coupled to graphics processor 1608 via at least one interconnect, or it may be integrated within graphics processor 1608.

[0284] In at least one embodiment, ring-based interconnect unit 1612 is used to couple internal components of processor 1600B. In at least one embodiment, alternative interconnect units, such as point-to-point interconnects, switched interconnects, or other technologies, may be used. In at least one embodiment, graphics processor 1608 is coupled to ring interconnect 1612 via I / O link 1613.

[0285] In at least one embodiment, I / O link 1613 represents at least one of a variety of I / O interconnects, including packaged I / O interconnects that facilitate communication between various processor components and high-performance embedded memory module 1618 (e.g., eDRAM module). In at least one embodiment, each of processor cores 1602A-1602N and graphics processor 1608 uses embedded memory module 1618 as a shared last-level cache.

[0286] In at least one embodiment, processor cores 1602A-1602N are homogeneous cores executing a common instruction set architecture. In at least one embodiment, processor cores 1602A-1602N are heterogeneous in terms of instruction set architecture (ISA), with one or more processor cores 1602A-1602N executing a common instruction set, while one or more other processor cores 1602A-1602N execute a subset of the common instruction set or a different instruction set. In at least one embodiment, processor cores 1602A-1602N are heterogeneous in terms of microarchitecture, with one or more cores having relatively high power consumption coupled to one or more power cores having lower power consumption. In at least one embodiment, processor 1600B can be implemented on one or more chips or implemented as a SoC integrated circuit.

[0287] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, some or all of the inference and / or training logic 615 may be incorporated into the processor 1600B. For example, in at least one embodiment, the training and / or inference techniques described herein may use one or more ALUs, the ALUs being embodied in... Figure 16A The graphics core 1612, one or more processor cores 1602A-1602N, or other components are used. Furthermore, in at least one embodiment, the inference and / or training operations described herein can use, except... Figure 6B and / or Figure 6C The logic other than that shown is used to perform this task. In at least one embodiment, the weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown), which configure the ALU of the graphics processor 1600B to execute one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0288] Figure 16C This is a block diagram of the hardware logic of a graphics processor core 1600C according to at least one embodiment of the present invention. In at least one embodiment, the graphics processor core 1600C is included in a graphics core array. In at least one embodiment, the graphics processor core 1600C (sometimes referred to as a core slice) may be one or more graphics cores within a modular graphics processor. In at least one embodiment, the graphics processor core 1600C is an example of a graphics core slice, and the graphics processor of the present invention may include multiple graphics core slices based on target power and performance envelopes. In at least one embodiment, each graphics core 1600C may include a fixed-function block 1630, also referred to as a sub-slice, coupled to a plurality of sub-cores 1601A-1601F, which includes modular blocks of general-purpose and fixed-function logic.

[0289] In at least one embodiment, the fixed-function block 1630 includes a geometry fixed-function pipeline 1636, which, for example, in a lower-performance and / or lower-power graphics processor implementation, may be shared by all sub-cores of the graphics processor 1600C. In at least one embodiment, the geometry fixed-function pipeline 1636 includes a 3D fixed-function pipeline, a video front-end unit, a thread generator and a thread dispatcher, and a unified return buffer manager that manages a unified return buffer.

[0290] In at least one fixed embodiment, the fixed functional block 1630 also includes a graphics SoC interface 1637, a graphics microcontroller 1638, and a media pipeline 1639. In at least one embodiment, the fixed graphics SoC interface 1637 provides an interface between the graphics core 1600C and other processor cores in the on-chip integrated circuit system. In at least one embodiment, the graphics microcontroller 1638 is a programmable subprocessor configurable to manage various functions of the graphics processor 1600C, including thread dispatch, scheduling, and preemption. In at least one embodiment, the media pipeline 1639 includes logic that facilitates the decoding, encoding, preprocessing, and / or post-processing of multimedia data, including image and video data. In at least one embodiment, the media pipeline 1639 implements media operations via requests for computation or sampling logic within subcores 1601-1601F.

[0291] In at least one embodiment, the SoC interface 1637 enables the graphics core 1600C to communicate with a general-purpose application processor core (e.g., a CPU) and / or other components within the SoC, including memory hierarchy elements such as shared last-level cache, system RAM, and / or embedded on-chip or packaged DRAM. In at least one embodiment, the SoC interface 1637 also enables communication with fixed-function devices within the SoC (e.g., a camera imaging pipeline) and enables the use and / or implementation of global memory atoms that can be shared between the graphics core 1600C and the CPU within the SoC. In at least one embodiment, the SoC interface 1637 also implements power management control for the graphics core 1600C and enables interfacing between the clock domain of the graphics core 1600C and other clock domains within the SoC. In at least one embodiment, the SoC interface 1637 enables the reception of command buffers from a command stream converter and a global thread dispatcher, configured to provide commands and instructions to each of one or more graphics cores within the graphics processor. In at least one embodiment, when a media operation is to be performed, commands and instructions can be dispatched to the media pipeline 1639, or when a graphics processing operation is to be performed, they can be assigned to the geometry and fixed-function pipelines (e.g., geometry and fixed-function pipeline 1636, geometry and fixed-function pipeline 1614).

[0292] In at least one embodiment, the graphics microcontroller 1638 can be configured to perform various scheduling and management tasks on the graphics core 1600C. In at least one embodiment, the graphics microcontroller 1638 can perform graphics and / or compute workload scheduling on various graphics parallel engines within the execution unit (EU) arrays 1602A-1602F, 1604A-1604F in subcores 1601A-1601F. In at least one embodiment, host software executing on the CPU core of the SoC including the graphics core 1600C can submit a workload of one of a plurality of graphics processor doorbells, which invokes scheduling operations on the appropriate graphics engine. In at least one embodiment, the scheduling operation includes determining which workload should be run next, submitting the workload to a command stream converter, preempting existing workloads running on the engine, monitoring the progress of the workload, and notifying the host software when the workload is completed. In at least one embodiment, the graphics microcontroller 1638 may also facilitate a low-power or idle state of the graphics core 1600C, thereby providing the graphics core 1600C with the ability to save and restore registers across low-power state transitions within the graphics core 1600C, independent of the operating system and / or the graphics driver software on the system.

[0293] In at least one embodiment, the graphics core 1600C may have up to N more or fewer modular subcores than the illustrated subcores 1601A-1601F. For each group of N subcores, in at least one embodiment, the graphics core 1600C may further include shared functional logic 1610, shared and / or cache memory 1612, geometry / fixed-function pipeline 1614, and additional fixed-function logic 1616 to accelerate various graphics and computational processing operations. In at least one embodiment, the shared functional logic 1610 may include logic units (e.g., samplers, mathematical and / or inter-thread communication logic) that can be shared by each of the N subcores within the graphics core 1600C. In at least one embodiment, the fixed, shared, and / or cache memory 1612 may be the last-level cache of the N subcores 1601A-1601F within the graphics core 1600C, and may also be used as shared memory accessible by multiple subcores. In at least one embodiment, a geometry / fixed function pipeline 1614 may be included to replace the geometry / fixed function pipeline 1636 within the fixed function block 1630, and similar logic units may be included.

[0294] In at least one embodiment, the graphics core 1600C includes additional fixed-function logic 1616, which may include various fixed-function acceleration logics for use by the graphics core 1600C. In at least one embodiment, the additional fixed-function logic 1616 includes additional geometry pipelines for use in position-only shading. In position-only shading, there are at least two geometry pipelines, and in the full geometry pipeline and culling pipeline within the geometry and fixed-function pipelines 1614, 1636, it is an additional geometry pipeline that can be included in the additional fixed-function logic 1616. In at least one embodiment, the culling pipeline is a trimmed version of the full geometry pipeline. In at least one embodiment, the full pipeline and the culling pipeline can execute different instances of the application, each with a separate environment. In at least one embodiment, position-only shading can hide long culling runs of discarded triangles, thereby allowing shading to be completed earlier in some cases. In at least one embodiment, the culling pipeline logic in the additional fixed-function logic 1616 can execute the position shader in parallel with the main application and generate critical results faster than the full pipeline because the culling pipeline acquires and occludes the positional attributes of vertices without performing rasterization and rendering pixels to the framebuffer. In at least one embodiment, the culling pipeline can use the generated critical results to compute visibility information for all triangles, regardless of whether those triangles were culled. In at least one embodiment, the full pipeline (which may be referred to as the replay pipeline in this case) can consume visibility information to skip culled triangles and only occlude the visible triangles that are ultimately passed to the rasterization stage.

[0295] In at least one embodiment, the additional fixed-function logic 1616 may also include machine learning acceleration logic, such as fixed-function matrix multiplication logic, for implementing optimizations for machine learning training or inference.

[0296] In at least one embodiment, each graphics subcore 1601A-1601F includes a set of execution resources that can be used to perform graphics, media, and compute operations in response to requests from the graphics pipeline, media pipeline, or shader program. In at least one embodiment, the graphics subcore 1601A-1601F includes multiple EU arrays 1602A-1602F, 1604A-1604F, thread dispatch and inter-thread communication (TD / IC) logic 1603A-1603F, 3D (e.g., texture) samplers 1605A-1605F, media samplers 1606A-1606F, shader processors 1607A-1607F, and shared local memory (SLM) 1608A-1608F. Each of the EU arrays 1602A-1602F and 1604A-1604F contains multiple execution units, which are general-purpose graphics processing units capable of servicing graphics, media, or computational operations, performing floating-point and integer / fixed-point logic operations, including graphics, media, or computational shader programs. In at least one embodiment, the TD / IC logic 1603A-1603F performs local thread dispatch and thread control operations for the execution units within the subcore and facilitates communication between threads executing on the execution units of the subcore. In at least one embodiment, the 3D samplers 1605A-1605F can read data associated with textures or other 3D graphics into memory. In at least one embodiment, the 3D samplers can read texture data differently based on the sampling state and texture format configured and associated with a given texture. In at least one embodiment, the media samplers 1606A-1606F can perform similar read operations based on the type and format associated with the media data. In at least one embodiment, each graphics subcore 1601A-1601F may alternatively include a unified 3D and media sampler. In at least one embodiment, threads executing on execution units within each subcore 1601A-1601F may utilize shared local memory 1608A-1608F within each subcore, enabling threads executing within a thread group to utilize a common pool of on-chip memory for execution.

[0297] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, some or all of the inference and / or training logic 615 may be incorporated into the graphics processor 1610. In at least one embodiment, the training and / or inference techniques described herein may be used in… Figure 16BThe graphics processor 1612, graphics microcontroller 1638, geometry and fixed-function pipelines 1614 and 1636, or other logic embodied in one or more ALUs. Furthermore, in at least one embodiment, the inference and / or training operations described herein can use, except... Figure 6B and / or Figure 6C The logic other than that shown is used to perform the task. In at least one embodiment, the weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown), which configure the ALU of the graphics processor 1600C to execute one or more of the machine learning algorithms, neural network architectures, use cases or training techniques described herein.

[0298] Figure 16D-16E The diagram illustrates thread execution logic 1600D of an array of processing elements including a graphics processor core, according to at least one embodiment. Figure 16D At least one embodiment is shown in which thread execution logic 1600D is used. Figure 16E Exemplary internal details of an execution unit according to at least one embodiment are shown.

[0299] like Figure 16D As shown, in at least one embodiment, the thread execution logic 1600D includes a shader processor 1602, a thread dispatcher 1604, an instruction cache 1606, a scalable execution unit array including multiple execution units 1608A-1608N, one or more samplers 1610, a data cache 1612, and a data port 1614. In at least one embodiment, the scalable execution unit array can be dynamically scaled, for example, based on the computational requirements of the workload, by enabling or disabling one or more execution units (e.g., execution units 1608A, 1608B, 1608C, 1608D, and any one of 1608N-1 and 1608N). In at least one embodiment, the scalable execution units are interconnected via an interconnect structure linking to each execution unit. In at least one embodiment, the thread execution logic 1600D includes one or more connections to memory (such as system memory or cache memory) via one or more of the instruction cache 1606, data port 1614, sampler 1610, and execution units 1608A-1608N. In at least one embodiment, each execution unit (e.g., 1608A) is an independent programmable general-purpose computing unit capable of executing multiple concurrent hardware threads, processing multiple data elements in parallel for each thread. In at least one embodiment, the array of execution units 1608A-1608N is scalable to include any number of individual execution units.

[0300] In at least one embodiment, execution units 1608A-1608N are primarily used to execute shader programs. In at least one embodiment, shader processor 1602 can process various shader programs and dispatch execution threads associated with the shader programs via thread dispatcher 1604. In at least one embodiment, thread dispatcher 1604 includes logic for arbitrating thread initialization celebrations from the graphics and media pipeline and for instantiating requested threads on one or more execution units 1608A-1608N. In at least one embodiment, the geometry pipeline can dispatch vertex, tessellation, or geometry shaders to thread execution logic for processing. In at least one embodiment, thread dispatcher 1604 can also handle runtime thread generation requests from executing shader programs.

[0301] In at least one embodiment, the execution units 1608A-1608N support an instruction set that includes native support for many standard 3D graphics shader instructions, enabling shader programs in graphics libraries (e.g., Direct3D and OpenGL) to execute with minimal conversion. In at least one embodiment, the execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., computation and media shaders). In at least one embodiment, each execution unit 1608A-1608N includes one or more arithmetic logic units (ALUs) capable of performing multiple-issue single-instruction multiple-data (SIMD) operations, and multithreaded operation enables an efficient execution environment despite higher latency memory access. In at least one embodiment, each hardware thread within each execution unit has a dedicated high-bandwidth register file and associated independent thread states. In at least one embodiment, execution is multiple issues per clock cycle to a pipeline capable of integer, single-precision, and double-precision floating-point operations, SIMD branching functions, logical operations, a priori operations, and other operations. In at least one embodiment, while waiting for data from one of the memory or shared functions, dependency logic within execution units 1608A-1608N causes the waiting thread to sleep until the requested data is returned. In at least one embodiment, while the waiting thread is sleeping, hardware resources can be dedicated to processing other threads. In at least one embodiment, during the latency associated with vertex shader operations, the execution unit can perform operations on the pixel shader, fragment shader, or another type of shader program (including different vertex shaders).

[0302] In at least one embodiment, each execution unit in the execution units 1608A-1608N operates on an array of data elements. In at least one embodiment, the plurality of data elements is an "execution size" or the number of instruction channels. In at least one embodiment, an execution channel is a logical unit for execution of data element access, masking, and flow control within an instruction. In at least one embodiment, the plurality of channels may be independent of the plurality of physical arithmetic logic units (ALUs) or floating-point units (FPUs) for a particular graphics processor. In at least one embodiment, the execution units 1608A-1608N support integer and floating-point data types.

[0303] In at least one embodiment, the execution unit instruction set includes SIMD instructions. In at least one embodiment, various data elements may be stored in registers as encapsulated data types, and the execution unit will process the various elements based on the data size of those elements. In at least one embodiment, when operating on a 256-bit wide vector, 256 bits of the vector are stored in registers, and the execution unit operates on the vector as four separate 64-bit packed data elements (quad-word (QW) size data elements), eight separate 32-bit packed data elements (double-word (DW) size data elements), sixteen separate 16-bit packed data elements (word (W) size data elements), or thirty-two separate 8-bit data elements (byte (B) size data elements). However, in at least one embodiment, different vector widths and register sizes are possible.

[0304] In at least one embodiment, one or more execution units can be combined into a fused execution unit 1609A-1609N having thread control logic (1607A-1607N) for executing fused EUs. In at least one embodiment, multiple EUs can be merged into an EU group. In at least one embodiment, each EU in the fused EU group can be configured to execute a separate SIMD hardware thread. The number of EUs in the fused EU group can vary depending on the embodiments. In at least one embodiment, each EU can execute various SIMD widths, including but not limited to SIMD8, SIMD16, and SIMD32. In at least one embodiment, each fused graphics execution unit 1609A-1609N includes at least two execution units. In at least one embodiment, the fused execution unit 1609A includes a first EU 1608A, a second EU 1608B, and thread control logic 1607A shared by the first EU 1608A and the second EU 1608B. In at least one embodiment, thread control logic 1607A controls the threads executing on the fused graphics execution unit 1609A, thereby allowing each EU within the fused execution units 1609A-1609N to execute using a common instruction pointer register.

[0305] In at least one embodiment, one or more internal instruction caches (e.g., 1606) are included in the thread execution logic 1600D to cache thread instructions for the execution unit. In at least one embodiment, one or more data caches (e.g., 1612) are included to cache thread data during thread execution. In at least one embodiment, a sampler 1610 is included to provide texture sampling for 3D operations and media sampling for media operations. In at least one embodiment, the sampler 1610 includes dedicated texture or media sampling functions to process texture or media data during the sampling process before providing sampled data to the execution unit.

[0306] During execution, in at least one embodiment, the graphics and media pipeline sends thread initiation requests to thread execution logic 1600D via thread creation and dispatch logic. In at least one embodiment, once a set of geometric objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within shader processor 1602 is invoked to further compute output information and cause the results to be written to output surfaces (e.g., color buffer, depth buffer, stencil buffer, etc.). In at least one embodiment, the pixel shader or fragment shader computes values ​​of various vertex attributes to be interpolated on the rasterized objects. In at least one embodiment, the pixel processor logic within shader processor 1602 then executes a pixel or fragment shader program provided by an application programming interface (API). In at least one embodiment, to execute the shader program, shader processor 1602 dispatches threads to execution units (e.g., 1608A) via thread dispatcher 1604. In at least one embodiment, shader processor 1602 uses texture sampling logic in sampler 1610 to access texture data in a texture map stored in memory. In at least one embodiment, arithmetic operations on the texture data and the input geometry data are performed to calculate pixel color data for each geometric segment, or one or more pixels are discarded for further processing.

[0307] In at least one embodiment, data port 1614 provides a memory access mechanism for thread execution logic 1600D to output processed data to memory for further processing on the graphics processor output pipeline. In at least one embodiment, data port 1614 includes or is coupled to one or more cache memories (e.g., data cache 1612) to cache data for memory access via the data port.

[0308] like Figure 16EAs shown, in at least one embodiment, the graphics execution unit 1608 may include an instruction fetch unit 1637, a general-purpose register file array (GRF) 1624, an architecture register file array (ARF) 1626, a thread arbiter 1622, a send unit 1630, a branch unit 1632, a set of SIMD floating-point units (FPUs) 1634, and in at least one embodiment, a set of dedicated integer SIMD ALUs 1635. In at least one embodiment, the GRF 1624 and ARF 1626 include a set of general-purpose register files and architecture register files associated with each concurrent hardware thread that can be active in the graphics execution unit 1608. In at least one embodiment, the architecture state of each thread is maintained in the ARF 1626, while data used during thread execution is stored in the GRF 1624. In at least one embodiment, the execution state of each thread, including the instruction pointer of each thread, may be stored in thread-specific registers in the ARF 1626.

[0309] In at least one embodiment, the graphics execution unit 1608 has an architecture that is a combination of simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT). In at least one embodiment, the architecture has a modular configuration that can be fine-tuned at design time based on a target number of simultaneous threads and the number of registers per execution unit, wherein execution unit resources are logically allocated for executing multiple simultaneous threads.

[0310] In at least one embodiment, the graphics execution unit 1608 can jointly issue multiple instructions, each of which can be a different instruction. In at least one embodiment, the thread arbiter 1622 of the graphics execution unit thread 1608 can dispatch instructions to one of the sending unit 1630, the branching unit 1632, or the SIMD FPU 1632 for execution. In at least one embodiment, each execution thread can access 128 general-purpose registers in the GRF 1624, where each register can store 32 bytes and can be accessed as a SIMD 8-element vector of 32-bit data elements. In at least one embodiment, each execution unit thread can access 4KB of the GRF 1624, although the embodiments are not limited thereto, and more or fewer register resources may be provided in other embodiments. In at least one embodiment, although the number of threads per execution unit may also vary depending on the embodiment, a maximum of seven threads can be executed simultaneously. In at least one embodiment where seven threads can access 4KB, the GRF 1624 can store a total of 28KB. In at least one embodiment, the flexible addressing mode can allow registers to be addressed together to efficiently build wider registers or rectangular block data structures representing strides.

[0311] In at least one embodiment, memory operations, sampler operations, and other longer-latency system communications are scheduled via a "send" instruction executed by message sending unit 1630. In at least one embodiment, branch instructions are dispatched to a dedicated branching unit 1632 to facilitate SIMD divergence and eventual convergence.

[0312] In at least one embodiment, the graphics execution unit 1608 includes one or more SIMD floating-point units (FPUs) 1634 to perform floating-point operations. In at least one embodiment, one or more FPUs 1634 also support integer computation. In at least one embodiment, one or more FPUs 1634 can perform up to M 32-bit floating-point (or integer) operations in SIMD, or up to 2M 16-bit integer or 16-bit floating-point operations in SIMD. In at least one embodiment, at least one of the one or more FPUs provides extended mathematical capabilities to support high-throughput a priori mathematical functions and double-precision 64-bit floating-point operations. In at least one embodiment, a set of 8-bit integer SIMD ALUs 1635 is also present and can be specifically optimized to perform operations related to machine learning computations.

[0313] In at least one embodiment, an array of multiple instances of the graphics execution unit 1608 may be instantiated in a graphics sub-core group (e.g., a sub-slice). In at least one embodiment, the execution unit 1608 may execute instructions across multiple execution channels. In at least one embodiment, each thread executing on the graphics execution unit 1608 executes on a different channel.

[0314] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 6B and / or Figure 6C Details are provided regarding the inference and / or training logic 615. In at least one embodiment, some or all of the inference and / or training logic 615 may be incorporated into the execution logic 1600D. Furthermore, in at least one embodiment, additional... Figure 6B and / or Figure 6C The logic other than that shown is used to perform the inference and / or training operations described herein. In at least one embodiment, weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown), which configure the ALU of execution logic 1600D to execute one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0315] Figure 17AA parallel processing unit (“PPU”) 1700A according to at least one embodiment is illustrated. In at least one embodiment, the PPU 1700A is configured with machine-readable code that, if executed by the PPU 1700A, causes the PPU 1700A to perform some or all of the processes and techniques described herein. In at least one embodiment, the PPU 1700A is a multi-threaded processor implemented on one or more integrated circuit devices and utilizes multi-threading as a delay-hiding technique designed to process computer-readable instructions (also referred to as machine-readable instructions or simple instructions) executed in parallel on multiple threads. In at least one embodiment, a thread refers to an execution thread and is an instance of a set of instructions configured to be executed by the PPU 1700A. In at least one embodiment, the PPU 1700A is a graphics processing unit (“GPU”) configured to implement a graphics rendering pipeline for processing three-dimensional (“3D”) graphics data to generate two-dimensional (“2D”) image data for display on a display device, such as a liquid crystal display (“LCD”) device. In at least one embodiment, the PPU 1700A is used to perform computations, such as linear algebra operations and machine learning operations. Figure 17A An example parallel processor is shown for illustrative purposes only and should be interpreted as a non-limiting example of a processor architecture contemplated within the scope of this disclosure, which may be supplemented and / or replaced by any suitable processor.

[0316] In at least one embodiment, one or more PPU 1700A processors are configured to accelerate high-performance computing (“HPC”), data center, and machine learning applications. In at least one embodiment, the PPU 1700A is configured to accelerate deep learning systems and applications, including, but not limited to, the following non-limiting examples: autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analytics, molecular simulation, drug discovery, disease diagnosis, weather forecasting, big data analytics, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations.

[0317] In at least one embodiment, the PPU 1700A includes, but is not limited to, an input / output (“I / O”) unit 1706, a front-end unit 1710, a scheduler unit 1712, a job allocation unit 1714, a hub 1716, a crossbar (“Xbar”) 1720, one or more general-purpose processing clusters (“GPCs”) 1718, and one or more partitioning units (“memory partitioning units”) 1722. In at least one embodiment, the PPU 1700A is connected to a host processor or other PPU 1700A via one or more high-speed GPU interconnects (“GPU interconnects”) 1708. In at least one embodiment, the PPU 1700A is connected to a host processor or other peripheral device via interconnect 1702. In one embodiment, the PPU 1700A is connected to local memory including one or more memory devices (“memory”) 1704. In at least one embodiment, the memory device 1704 includes, but is not limited to, one or more dynamic random access memory (“DRAM”) devices. In at least one embodiment, one or more DRAM devices are configured and / or configurable as a high-bandwidth memory (“HBM”) subsystem, and multiple DRAM dies are stacked within each device.

[0318] In at least one embodiment, the high-speed GPU interconnect 1708 may refer to a wire-based multi-channel communication link used by the system for scaling, and includes one or more PPU1700As (“CPUs”) combined with one or more central processing units, supporting cache coherence between the PPU 1700As and the CPU, as well as CPU master control. In at least one embodiment, the high-speed GPU interconnect 1708 transmits data and / or commands to other units of the PPU 1700A via a hub 1716, such as one or more copy engines, video encoders, video decoders, power management units, and / or other components. Figure 17A Other components that may not be explicitly shown.

[0319] In at least one embodiment, I / O unit 1706 is configured to connect to the host processor via interconnect 1702. Figure 17A(Not shown) Sending and receiving communications (e.g., commands, data). In at least one embodiment, I / O unit 1706 communicates directly with the host processor via interconnect 1702 or via one or more intermediate devices (e.g., memory bridges). In at least one embodiment, I / O unit 1706 may communicate with one or more other processors (e.g., one or more PPUs 1700A) via interconnect 1702. In at least one embodiment, I / O unit 1706 implements a Peripheral Component Interconnect Express (“PCIe”) interface for communication via the PCIe bus. In at least one embodiment, I / O unit 1706 implements an interface for communicating with external devices.

[0320] In at least one embodiment, I / O unit 1706 decodes packets received via interconnect 1702. In at least one embodiment, at least some packets represent commands configured to cause PPU 1700A to perform various operations. In at least one embodiment, I / O unit 1706 sends the decoded commands to various other units of PPU 1700A as specified by the commands. In at least one embodiment, the commands are sent to front-end unit 1710 and / or to hub 1716 or other units of PPU 1700A, such as one or more copy engines, video encoders, video decoders, power management units, etc. Figure 17A (Not explicitly shown in the text). In at least one embodiment, I / O unit 1706 is configured to route communication between various logic units of PPU 1700A.

[0321] In at least one embodiment, a program executed by the host processor encodes a command stream in a buffer that provides a workload to the PPU 1700A for processing. In at least one embodiment, the workload includes instructions and data to be processed by those instructions. In at least one embodiment, the buffer is an area in memory accessible (e.g., read / write) by both the host processor and the PPU 1700A—the host interface unit can be configured to access a buffer in system memory connected to the interconnect 1702 via memory requests transmitted through the interconnect 1702 via the I / O unit 1706. In at least one embodiment, the host processor writes a command stream to the buffer and then sends a pointer indicating the start of the command stream to the PPU 1700A, such that the front-end unit 1710 receives pointers to one or more command streams and manages one or more command streams, reads commands from the command streams, and forwards the commands to the respective units of the PPU 1700A.

[0322] In at least one embodiment, front-end unit 1710 is coupled to scheduler unit 1712, which configures various GPCs 1718 to process tasks defined by one or more command streams. In at least one embodiment, scheduler unit 1712 is configured to track status information related to the various tasks managed by scheduler unit 1712, wherein the status information may indicate which GPC 1718 a task is assigned to, whether the task is active or inactive, the priority associated with the task, etc. In at least one embodiment, scheduler unit 1712 manages multiple tasks executed on one or more GPCs 1718.

[0323] In at least one embodiment, scheduler unit 1712 is coupled to job allocation unit 1714, which is configured to dispatch tasks for execution on GPC 1718. In at least one embodiment, job allocation unit 1714 tracks multiple scheduled tasks received from scheduler unit 1712 and manages a pool of pending tasks and an active task pool for each GPC 1718. In at least one embodiment, the pool of pending tasks includes multiple time slots (e.g., 16 time slots) containing tasks assigned to a particular GPC 1718; the active task pool may include multiple time slots (e.g., 4 time slots) for tasks actively processed by GPC 1718, such that as one of the GPCs 1718 completes its execution, that task is evicted from the active task pool of the GPC 1718, and one of other tasks is selected from the pool of pending tasks and scheduled for execution on the GPC 1718. In at least one embodiment, if the active task is idle on GPC 1718, for example while waiting for data dependency resolution, the active task is evicted from GPC 1718 and returned to the task pool, while another task in the task pool is selected and scheduled to be executed on GPC 1718.

[0324] In at least one embodiment, the work allocation unit 1714 communicates with one or more GPCs 1718 via XBar 1720. In at least one embodiment, XBar 1720 is an interconnect network that couples a plurality of units of PPU 1700A to other units of PPU 1700A, and can be configured to couple the work allocation unit 1714 to a specific GPC 1718. In at least one embodiment, one or more other units of PPU 1700A can also be connected to XBar 1716 via hub 1716.

[0325] In at least one embodiment, tasks are managed by scheduler unit 1712 and assigned to one of GPCs 1718 by job allocation unit 1714. GPC 1718 is configured to process tasks and produce results. In at least one embodiment, results may be consumed by other tasks in GPC 1718, routed to different GPCs 1718 via XBar 1716, or stored in memory 1704. In at least one embodiment, results may be written to memory 1704 via partitioning unit 1722, which implements a memory interface for writing data to or reading data from memory 1704. In at least one embodiment, results may be transferred to another PPU 1704 or CPU via high-speed GPU interconnect 1708. In at least one embodiment, PPU 1700A includes, but is not limited to, U partitioning units 1722, where the number of partitioning units 1722 equals the number of separate and different memory devices 1704 coupled to PPU 1700A. In at least one embodiment, the following will be combined with… Figure 17C The dividing unit 1722 is described in more detail.

[0326] In at least one embodiment, the host processor executes a driver core that implements an application programming interface (API) that enables one or more applications executing on the host processor to schedule operations for execution on the PPU 1700A. In one embodiment, multiple computing applications are executed concurrently by the PPU 1700A, and the PPU 1700A provides isolation, Quality of Service (“QoS”), and independent address spaces for the multiple computing applications. In at least one embodiment, an application generates instructions (e.g., in the form of API calls) that cause the driver core to generate one or more tasks for execution by the PPU 1700A, and the driver core outputs the tasks to one or more streams processed by the PPU 1700A. In at least one embodiment, each task includes one or more associated thread groups, which may be referred to as a warp. In at least one embodiment, a warp includes multiple associated threads (e.g., 32 threads) that can be executed in parallel. In at least one embodiment, a cooperating thread may refer to multiple threads, including instructions for performing tasks and exchanging data via shared memory, in combination with... Figure 17C Threads and cooperative threads are described in more detail according to at least one embodiment.

[0327] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. The following is combined with... Figure 6B and / or Figure 6CDetails regarding the inference and / or training logic 615 are provided. In at least one embodiment, the deep learning application processor is used to train a machine learning model (such as a neural network) to predict or infer information provided to the PPU 1700A. In at least one embodiment, the PPU 1700A is used to infer or predict information based on a trained machine learning model (e.g., a neural network) that has been trained by another processor or system or the PPU 1700A. In at least one embodiment, the PPU 1700A can be used to perform one or more neural network use cases as described herein.

[0328] Figure 17B A general-purpose processing cluster (“GPC”) 1700B according to at least one embodiment is illustrated. In at least one embodiment, the GPC 1700B is Figure 17A The GPC 1718. In at least one embodiment, each GPC 1700B includes, but is not limited to, multiple hardware units for processing tasks, and each GPC 1700B includes, but is not limited to, a pipeline manager 1702, a pre-raster operation unit (“PROP”) 1704, a raster engine 1708, a work assignment crossbar switch (“WDX”) 1716, a memory management unit (“MMU”) 1718, one or more data processing clusters (“DPC”) 1706, and any suitable combination of components.

[0329] In at least one embodiment, the operation of the GPC 1700B is controlled by the pipeline manager 1702. In at least one embodiment, the pipeline manager 1702 manages the configuration of one or more DPCs 1706 to handle tasks assigned to the GPC 1700B. In at least one embodiment, the pipeline manager 1702 configures at least one of the one or more DPCs 1706 to implement at least a portion of the graphics rendering pipeline. In at least one embodiment, the DPC 1706 is configured to execute vertex shader programs on a programmable streaming multiprocessor (“SM”) 1714. In at least one embodiment, the pipeline manager 1702 is configured to route packets received from the work allocation unit to appropriate logic units within the GPC 1700B, and in at least one embodiment, some packets may be routed to fixed-function hardware units in the PROP 1704 and / or the raster engine 1708, while other packets may be routed to the DPC 1706 for processing by the raw engine 1712 or the SM 1714. In at least one embodiment, pipeline manager 1702 configures at least one of DPCs 1706 to implement a neural network model and / or computation pipeline.

[0330] In at least one embodiment, the PROP unit 1704 is configured to route data generated by the raster engine 1708 and DPC 1706 to the raster operation (“ROP”) unit in the partition unit 1722, in conjunction with the above. Figure 17A More detailed description. In at least one embodiment, the PROP unit 1704 is configured to perform optimizations for color blending, organize pixel data, perform address translation, etc. In at least one embodiment, the raster engine 1708 includes, but is not limited to, multiple fixed-function hardware units configured to perform various raster operations, and in at least one embodiment, the raster engine 1708 includes, but is not limited to, a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, a tile aggregation engine, and any suitable combination thereof. In at least one embodiment, the setup engine receives transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices; the plane equations are passed to the coarse raster engine to generate coverage information of basic primitives (e.g., x, y coverage masks of tiles); the output of the coarse raster engine is passed to the culling engine, in which fragments associated with primitives that fail the z-test are culled, and passed to the clipping engine, in which fragments located outside the view frustum are clipped. In at least one embodiment, the clipped and culled fragments are passed to the fine raster engine to generate properties of pixel fragments based on the plane equations generated by the setup engine. In at least one embodiment, the output of the raster engine 1708 includes fragments that will be processed by any appropriate entity (e.g., by the fragment shader implemented within the DPC 1706).

[0331] In at least one embodiment, each DPC 1706 included in the GPC 1700B includes, but is not limited to, an M-pipeline controller (“MPC”) 1710; a primitive engine 1712; one or more SMs 1714; and any suitable combination thereof. In at least one embodiment, the MPC 1710 controls the operation of the DPC 1706, routing packets received from the pipeline manager 1702 to the appropriate units within the DPC 1706. In at least one embodiment, packets associated with vertices are routed to the primitive engine 1712, which is configured to retrieve vertex attributes associated with vertices from memory; conversely, packets associated with shader programs may be sent to the SMs 1714.

[0332] In at least one embodiment, the SM 1714 includes, but is not limited to, a programmable streaming processor configured to process tasks represented by multiple threads. In at least one embodiment, the SM 1714 is multithreaded and configured to execute multiple threads (e.g., 32 threads) from a specific thread group concurrently, and implements a Single Instruction, Multiple Data (“SIMD”) architecture, wherein each thread in a group of threads (e.g., a thread bundle) is configured to process a different dataset based on the same instruction set. In at least one embodiment, all threads in the thread group execute the same instructions. In at least one embodiment, the SM 1714 implements a Single Instruction, Multiple Thread (“SIMT”) architecture, wherein each thread in a group of threads is configured to process a different dataset based on the same instruction set, but wherein individual threads in the thread group are allowed to diverge during execution. In at least one embodiment, a program counter, call stack, and execution state are maintained for each thread bundle, thereby achieving concurrency between the thread bundle and serial execution within the thread bundle when threads in the thread bundle diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, thereby ensuring equal concurrency among all threads within and between thread bundles. In at least one embodiment, an execution state is maintained for each individual thread, and threads that execute the same instructions can converge and run in parallel to improve efficiency. At least one embodiment of SM 1714 is described in more detail below.

[0333] In at least one embodiment, the MMU 1718 is integrated with the GPC 1700B and memory partitioning unit (e.g., Figure 17A The MMU 1718 provides an interface between partition units 1722 and provides virtual address to physical address translation, memory protection, and memory request arbitration. In at least one embodiment, the MMU 1718 provides one or more translation back buffers (“TLBs”) for performing virtual address to physical address translation in memory.

[0334] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. The following is combined with... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the deep learning application processor is used to train a machine learning model (such as a neural network) to predict or infer information provided to the GPC 1700B. In at least one embodiment, the GPC 1700B is used to infer or predict information based on a machine learning model (e.g., a neural network) that has been trained by another processor or system or the GPC 1700B. In at least one embodiment, the GPC 1700B can be used to perform one or more neural network use cases as described herein.

[0335] Figure 17C A memory partitioning unit 1700C of a parallel processing unit (“PPU”) according to at least one embodiment is illustrated. In at least one embodiment, the memory partitioning unit 1700C includes, but is not limited to, a raster operation (“ROP”) unit 1702; a secondary (“L2”) cache 1704; a memory interface 1706; and any suitable combination thereof. In at least one embodiment, the memory interface 1706 is coupled to memory. In at least one embodiment, the memory interface 1706 may implement a 32, 64, 128, or 1024-bit data bus, or a similar implementation for high-speed data transfer. In at least one embodiment, the PPU includes U memory interfaces 1706, one memory interface 1706 per pair of partitioning units 1700C, wherein each pair of partitioning units 1700C is connected to a corresponding memory device. In at least one embodiment, the PPU may be connected to up to Y memory devices, such as a high-bandwidth memory stack or Graphics Dual Data Rate Version 5 Synchronous Dynamic Random Access Memory (“GDDR5 SDRAM”).

[0336] In at least one embodiment, memory interface 1706 implements a high-bandwidth second-generation (“HBM2”) memory interface, and Y is equal to half of U. In at least one embodiment, the HBM2 memory stack resides on the same physical package as the PPU, providing significant power savings and area savings compared to a GDDR5 SDRAM system. In at least one embodiment, each HBM2 stack includes, but is not limited to, four memory dies, with Y = 4, and each HBM2 stack includes two 128-bit channels per die for a total of eight channels and a 1024-bit data bus width. In at least one embodiment, the memory supports Single Error Corrected Double Error Detection (“SECDED”) error correction code (“ECC”) to protect data. In at least one embodiment, ECC provides higher reliability for data corruption-sensitive computing applications.

[0337] In at least one embodiment, the PPU implements a multi-level memory hierarchy. In at least one embodiment, the memory partitioning unit 1700C supports unified memory to provide a single unified virtual address space for the central processing unit (“CPU”) and PPU memory, thereby enabling data sharing between virtual memory systems. In at least one embodiment, the frequency of PPU accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU that accesses pages more frequently. In at least one embodiment, the high-speed GPU interconnect 1708 supports address translation services, which allow the PPU to directly access the CPU's page tables and provide full access to the CPU's memory through the PPU.

[0338] In at least one embodiment, the replication engine transfers data between multiple PPUs or between a PPU and a CPU. In at least one embodiment, the replication engine can generate page faults for addresses not mapped to page tables, and the memory partitioning unit 1700C then servicees the page faults, mapping the addresses to page tables, after which the replication engine performs the transfer. In at least one embodiment, multiple replication engines operating on fixed (in at least one embodiment, non-pageable) memory across multiple processors substantially reduce available memory. In at least one embodiment, in the event of a hardware page fault, an address can be passed to the replication engine regardless of whether a memory page resides, and the replication process is transparent.

[0339] According to at least one embodiment, from Figure 17A Data from memory 1704 or other system memory is retrieved by memory partitioning unit 1700C and stored in L2 cache 1704, which is located on-chip and shared among various GPCs. In at least one embodiment, each memory partitioning unit 1700C includes, but is not limited to, at least a portion of the L2 cache associated with the corresponding memory device. In at least one embodiment, lower-level caches are implemented in various units within a GPC. In at least one embodiment, each SM 1714 may implement a Level 1 (“L1”) cache, wherein the L1 cache is a private memory dedicated to a specific SM 1714, and data is retrieved from L2 cache 1704 and stored in each L1 cache for processing within the functional units of the SM 1714. In at least one embodiment, L2 cache 1704 is coupled to memory interface 1706 and XBar 1720.

[0340] In at least one embodiment, ROP unit 1702 performs graphic raster operations related to pixel color, such as color compression, pixel blending, etc. In at least one embodiment, ROP unit 1702 performs depth testing in conjunction with raster engine 1708, receiving depth from the culling engine of raster engine 1708 for sample locations associated with pixel fragments. In at least one embodiment, depth is tested for the corresponding depth in the depth buffer at the sample location associated with the fragment. In at least one embodiment, if the fragment passes the depth test for the sample location, ROP unit 1702 updates the depth buffer and sends the depth test result to raster engine 1708. It will be appreciated that the number of partition units 1700C may differ from the number of GPCs; therefore, each ROP unit 1702 may be coupled to each GPC in at least one embodiment. In at least one embodiment, ROP unit 1702 tracks packets received from different GPCs and determines which result the result generated by ROP unit 1702 is routed to via XBar 1720.

[0341] Figure 17D A streaming multiprocessor (“SM”) 1700D according to at least one embodiment is shown. In at least one embodiment, the SM 1700D is Figure 17BThe SM 1700D includes, but is not limited to, an instruction cache 1702; one or more scheduler units 1704; a register file 1708; one or more processing cores (“cores”) 1710; one or more special function units (“SFUs”) 1712; one or more load / store units (“LSUs”) 1714; an interconnect network 1716; a shared memory / Level 1 (“L1”) cache 1718; and any suitable combination thereof. In at least one embodiment, a work allocation unit schedules tasks to execute on a general-purpose processing cluster (“GPC”) of parallel processing units (“PPUs”), and each task is assigned to a specific data processing cluster (“DPC”) within the GPC, and if the task is associated with a shader program, the task is assigned to one of the SM 1700Ds. In at least one embodiment, a scheduler unit 1704 receives tasks from the work allocation unit and manages the instruction scheduling of one or more thread blocks assigned to the SM 1700D. In at least one embodiment, scheduler unit 1704 schedules thread blocks to execute as thread bundles of parallel threads, wherein each thread block is assigned at least one thread bundle. In at least one embodiment, each thread bundle executes a thread. In at least one embodiment, scheduler unit 1704 manages multiple different thread blocks, assigns thread bundles to different thread blocks, and then dispatches instructions from multiple different cooperative groups to various functional units (e.g., processing core 1710, SFU 1712, and LSU 1714) in each clock cycle.

[0342] In at least one embodiment, a cooperative group can refer to a programming model for organizing groups of communicating threads, allowing developers to express the granularity at which threads are communicating, thereby enabling richer and more efficient parallel decompositions. In at least one embodiment, the cooperative startup API supports synchronization between blocks of threads to execute parallel algorithms. In at least one embodiment, the application of the programming model provides a single, simple construct for synchronizing cooperative threads: a barrier (e.g., the `syncthreads()` function) across all threads in a thread block. However, in at least one embodiment, programmers can define thread groups at a granularity smaller than that of thread blocks and synchronize within the defined groups to achieve higher performance, design flexibility, and software reuse in the form of a set of group-wide functional interfaces. In at least one embodiment, cooperative groups enable programmers to explicitly define thread groups at the sub-block (as small as a single thread in at least one embodiment) and multi-block granularity and perform set operations, such as synchronizing threads within the cooperative group. In at least one embodiment, the programming model supports clean composition across software boundaries, allowing library and utility functions to be safely synchronized in their native environment without having to make assumptions about convergence. In at least one embodiment, the cooperative group primitives enable new patterns of cooperative parallelism, including but not limited to producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire thread block mesh.

[0343] In at least one embodiment, scheduling unit 1706 is configured to send instructions to one or more functional units, and scheduler unit 1704 includes, but is not limited to, two scheduling units 1706 that enable two different instructions from the same thread bundle to be scheduled in each clock cycle. In at least one embodiment, each scheduler unit 1704 includes a single scheduling unit 1706 or additional scheduling units 1706.

[0344] In at least one embodiment, each SM 1700D includes, but is not limited to, a register file 1708 that provides a set of registers for functional units of the SM 1700D. In at least one embodiment, the register file 1708 is partitioned between each functional unit, thereby allocating a dedicated portion of the register file 1708 for each functional unit. In at least one embodiment, the register file 1708 is partitioned between different thread bundles executed by the SM 1700D, and the register file 1708 provides temporary storage for operands connected to data paths of functional units. In at least one embodiment, each SM 1700D includes, but is not limited to, a plurality of L processing cores 1710. In at least one embodiment, the SM 1700D includes, but is not limited to, a large number (e.g., 128 or more) of different processing cores 1710. In at least one embodiment, each processing core 1710 includes, but is not limited to, a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, which includes, but is not limited to, floating-point arithmetic logic units and integer arithmetic logic units. In at least one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point arithmetic. In at least one embodiment, the processing core 1710 includes, but is not limited to, 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0345] According to at least one embodiment, tensor cores are configured to perform matrix operations. In at least one embodiment, one or more tensor cores are included in processing core 1710. In at least one embodiment, tensor cores are configured to perform deep learning matrix arithmetic, such as convolution operations for neural network training and inference. In at least one embodiment, each tensor core operates on a 4×4 matrix and performs matrix multiplication and accumulation operations D = A×B + C, where A, B, C, and D are 4×4 matrices.

[0346] In at least one embodiment, matrix multiplication inputs A and B are 16-bit floating-point matrices, and accumulation matrices C and D are either 16-bit or 32-bit floating-point matrices. In at least one embodiment, the Tensor Core performs 32-bit floating-point accumulation on the 16-bit floating-point input data. In at least one embodiment, the 16-bit floating-point multiplication uses 64 operations to obtain a full-precision product, which is then accumulated with other intermediate multiplications using 32-bit floating-point addition to perform a 4x4x4 matrix multiplication. In at least one embodiment, the Tensor Core is used to perform matrix operations on larger two-dimensional or higher-dimensional matrices composed of these smaller components. In at least one embodiment, APIs (such as the CUDA 9 C++ API) expose specialized matrix loading, matrix multiplication and accumulation, and matrix storage operations to efficiently utilize the Tensor Core from CUDA-C++ programs. In at least one embodiment, at the CUDA level, the thread bundle level interface assumes a 16×16 matrix spanning all 32 thread bundle threads.

[0347] In at least one embodiment, each SM 1700D includes, but is not limited to, M SFU 1712s that perform special functions (e.g., attribute estimation, inverse square root, etc.). In at least one embodiment, the SFU 1712 includes, but is not limited to, tree traversal units configured to traverse hierarchical tree data structures. In at least one embodiment, the SFU 1712 includes, but is not limited to, texture units configured to perform texture map filtering operations. In at least one embodiment, the texture unit is configured to load texture maps (e.g., a 2D array of texture pixels) from memory and sample the texture maps to produce sampled texture values ​​for use by a shader program executed by the SM 1700D. In at least one embodiment, the texture maps are stored in shared memory / L1 cache 1718. In at least one embodiment, according to at least one embodiment, the texture unit uses mip maps (e.g., texture maps with different levels of detail) to implement texture operations (such as filtering operations). In at least one embodiment, each SM 1700D includes, but is not limited to, two texture units.

[0348] In at least one embodiment, each SM 1700D includes, but is not limited to, N LSUs 1714 that implement load and store operations between the shared memory / L1 cache 1718 and the register file 1708. In at least one embodiment, an interconnect network 1716 connects each functional unit to the register file 1708, and the LSUs 1714 are connected to both the register file 1708 and the shared memory / L1 cache 1718. In at least one embodiment, the interconnect network 1716 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 1708 and to connect the LSUs 1714 to memory locations in both the register file 1708 and the shared memory / L1 cache 1718.

[0349] In at least one embodiment, the shared memory / L1 cache 1718 is an array of on-chip memory that, in at least one embodiment, allows data storage and communication between the SM 1700D and the primitive engine, as well as between threads within the SM1700D. In at least one embodiment, the shared memory / L1 cache 1718 includes, but is not limited to, a storage capacity of 128KB and is located on the path from the SM1700D to the partition unit. In at least one embodiment, the shared memory / L1 cache 1718 is used for cache reads and writes. In at least one embodiment, one or more of the shared memory / L1 cache 1718, the L2 cache, and memory are backup storage.

[0350] In at least one embodiment, combining data caching and shared memory functionality into a single memory block provides improved performance for both types of memory access. In at least one embodiment, the capacity is used by programs that do not use shared memory or is used as a cache, for example, if shared memory is configured to use half its capacity, and texture and load / store operations can use the remaining capacity. According to at least one embodiment, integration within the shared memory / L1 cache 1718 enables the shared memory / L1 cache 1718 to be used as a high-throughput pipeline for streaming data, while providing high-bandwidth and low-latency access to frequently reused data. In at least one embodiment, a simpler configuration can be used compared to graphics processing when configured for general-purpose parallel computing. In at least one embodiment, a fixed-function graphics processing unit is bypassed, creating a simpler programming model. In at least one embodiment, in a general-purpose parallel computing configuration, the work allocation unit directly allocates and distributes blocks of threads to the DPC. In at least one embodiment, threads within a block execute a general-purpose program, using unique thread IDs in computation to ensure each thread produces unique results, using an SM 1700D to execute the program and perform computations, using shared memory / L1 cache 1718 for communication between threads, and using an LSU 1714 to read and write global memory via shared memory / L1 cache 1718 and memory partitioning units. In at least one embodiment, when configured for general-purpose parallel computing, the SM 1700D writes commands to the scheduler unit 1704 that can be used to start new jobs on the DPC.

[0351] In at least one embodiment, the PPU is included in or coupled to a desktop computer, laptop computer, tablet computer, server, supercomputer, smartphone (e.g., wireless, handheld device), personal digital assistant (“PDA”), digital camera, vehicle, head-mounted display, handheld electronic device, etc. In at least one embodiment, the PPU is implemented on a single semiconductor substrate. In at least one embodiment, the PPU is included in a system-on-a-chip (“SoC”) along with one or more other devices (e.g., additional PPUs, memory, reduced instruction set computer (“RISC”) CPU, one or more memory management units (“MMU”), digital-to-analog converters (“DAC”), etc.).

[0352] In at least one embodiment, the PPU may be included on a graphics card that includes one or more storage devices. The graphics card may be configured to connect to a PCIe slot on a desktop computer motherboard. In at least one embodiment, the PPU may be an integrated graphics processing unit (“iGPU”) included in a chipset of the motherboard.

[0353] Inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. This document combines... Figure 6B and / or Figure 6C Details regarding the inference and / or training logic 615 are provided. In at least one embodiment, the deep learning application processor is used to train a machine learning model (such as a neural network) to predict or infer information provided to the SM 1700D. In at least one embodiment, the SM 1700D is used to infer or predict information based on a machine learning model (e.g., a neural network) that has been trained by another processor or system or by the SM 1700D. In at least one embodiment, the SM 1700D can be used to perform one or more neural network use cases as described herein.

[0354] In at least one embodiment, a single semiconductor platform may refer to a unique, single semiconductor-based integrated circuit or chip. In at least one embodiment, a multi-chip module with increased connectivity can be used, simulating on-chip operations and representing a substantial improvement over implementations utilizing a central processing unit (“CPU”) and buses. In at least one embodiment, the various modules may also be placed separately or in various combinations of semiconductor platforms, depending on the user's needs.

[0355] In at least one embodiment, a computer program in the form of machine-readable executable code or computer control logic algorithms is stored in main memory 804 and / or secondary storage. According to at least one embodiment, if executed by one or more processors, the computer program enables system 800 to perform various functions. In at least one embodiment, memory 804, storage, and / or any other storage are possible examples of computer-readable media. In at least one embodiment, secondary storage can refer to any suitable storage device or system, such as hard disk drives and / or removable storage drives, representing floppy disk drives, magnetic tape drives, optical disk drives, digital versatile disc (“DVD”) drives, recording devices, universal serial bus (“USB”) flash memory, etc. In at least one embodiment, the architecture and / or functionality of the various preceding figures are implemented within the context of CPU 802; parallel processing system 812; integrated circuits capable of having at least a portion of the capabilities of two CPUs 802; parallel processing system 812; chipsets (e.g., a set of integrated circuits designed to operate as units performing related functions and sold); and any suitable combination of integrated circuits.

[0356] In at least one embodiment, the architecture and / or functionality of the various preceding figures are implemented in an environment of a general-purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, a special-purpose system, etc. In at least one embodiment, the computer system 800 may take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless, handheld device), a personal digital assistant (“PDA”), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, a mobile phone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic.

[0357] In at least one embodiment, the parallel processing system 812 includes, but is not limited to, a plurality of parallel processing units (“PPUs”) 814 and associated memory 816. In at least one embodiment, the PPUs 814 are connected to a host processor or other peripheral device via interconnects 818 and switches 820 or multiplexers. In at least one embodiment, the parallel processing system 812 distributes computational tasks across the parallelizable PPUs 814, for example, as part of a computational task distribution across multiple graphics processing units (“GPUs”) thread blocks. In at least one embodiment, memory is shared and accessed (e.g., for read and / or write access) among some or all of the PPUs 814, although such shared memory may incur a performance penalty relative to the use of local memory and registers residing on the PPUs 814. In at least one embodiment, the operation of the PPUs 814 is synchronized using commands such as __syncthreads(), wherein all threads in a block (e.g., executing across multiple PPUs 814) arrive at a certain point of code execution before proceeding.

[0358] Other variations are within the spirit of this disclosure. Therefore, although the disclosed technology is readily adaptable to various modifications and alternative constructions, certain embodiments thereof are illustrated in the accompanying drawings and have been described in detail above. However, it should be understood that the disclosure is not intended to be limited to one or more specific forms disclosed, but rather, it is intended to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of this disclosure as defined in the appended claims.

[0359] Unless otherwise stated or obviously contradicted by the context, the terms “a,” “an,” and “the,” and similar references, used in the context of describing the disclosed embodiments (particularly in the context of the appended claims), should be interpreted as encompassing both singular and plura...

Claims

1. A heat load system for testing a liquid cooling system in a data center, comprising: The server enclosure includes at least one thermal feature associated with at least one cold plate and at least one flow controller for the data center liquid cooling system. Wherein, the at least one thermal feature includes a heating element having an adjustable heat level and being positioned adjacent to the cold plate; The at least one cold plate includes at least one first cold plate and at least one second cold plate, the at least one first cold plate being used to induce cooling stress on the data center liquid cooling system, and the at least one second cold plate providing cooling for data center equipment during active operation, and the at least one thermal feature and the at least one flow controller being adjustable to induce the cooling stress on the data center liquid cooling system.

2. The heat load system according to claim 1, wherein, The cold plate has an inlet channel and an outlet channel for circulating the cooling medium from the liquid cooling system of the data center.

3. The heat load system according to claim 2, further comprising: At least one processor is configured to control the heating element and to control the at least one flow controller, the heating element having a temperature range that replicates that of the data center equipment, and the at least one flow controller being configured to control the flow rate to replicate the flow rate of the data center liquid cooling system in connection with the data center equipment via a circuit breaker or liquid line connection.

4. The heat load system according to claim 1, further comprising: One or more second server enclosures, including one or more second thermal features, one or more cold plates, and one or more second flow controllers, for applying the cooling stress to the rack-level cooling stress of the data center liquid cooling system.

5. The heat load system according to claim 1, wherein, The heating element is associated with a socket on a board in the server chassis and contributes to the cooling stress on the data center liquid cooling system.

6. The heat load system according to claim 5, further comprising: Data center equipment, which may or may not be operable within the socket; as well as A thermal insulation structure exists between the data center equipment and the heating element, allowing the data center equipment to be switched between operable and inoperable states within the socket.

7. The heat load system according to claim 5, further comprising: Covering material on the heating element; as well as A thermal interface material on the covering is used to form an interface with the at least one cold plate.

8. The heat load system according to claim 1, further comprising: A conduit associated with the at least one cold plate is used to allow the inflow and outflow of coolant in the data center liquid cooling system.

9. The heat load system according to claim 1, further comprising: At least one power distribution unit associated with the at least one thermal feature is used to alter at least one or more thermal properties of the at least one thermal feature to induce the cooling stress on the data center liquid cooling system.

10. The heat load system according to claim 1, further comprising: At least one processor is configured to evaluate the cooling stress, represented by at least one or more temperatures applied to the at least one thermal feature and one or more flow rates or flow rates associated with the at least one flow controller, and to provide an output associated with at least one of the flow rates or flow rates representing expected cooling for at least one input temperature.

11. The heat load system according to claim 10, wherein: The at least one processor also executes a machine learning model to: The temperature is processed using multiple neuron levels of the machine learning model that has one or more temperatures and one or more previously associated flow rates or one or more previously associated flow volumes; as well as The output is provided after evaluating the one or more temperatures using the previously associated one or more flow rates or the previously associated one or more flow rates.

12. At least one processor for testing a cooling system, comprising: One or more circuits for controlling actions associated with at least one flow controller to cool a server chassis used to test a heat load system of a data center liquid cooling system, the server chassis including at least one thermal feature associated with at least one cold plate and the at least one flow controller. Wherein, the at least one thermal feature includes a heating element having an adjustable heat level and being positioned adjacent to the cold plate; The at least one cold plate includes at least one first cold plate and at least one second cold plate, the at least one first cold plate being used to induce cooling stress on the data center liquid cooling system, and the at least one second cold plate providing cooling for data center equipment during active operation, and the at least one thermal feature and the at least one flow controller being adjusted to induce the cooling stress on the data center liquid cooling system.

13. The at least one processor according to claim 12, wherein: The one or more circuits are also used to evaluate the cooling stress on the data center liquid cooling system, the cooling stress being represented at least in part by one or more temperatures applied to the at least one thermal feature and a flow rate or flow rate associated with the at least one flow controller, and to provide an output associated with at least one of the flow rates or flow rates representing expected cooling for at least one input temperature.

14. The at least one processor according to claim 13, wherein: The at least one circuit also executes a machine learning model to: The temperature is processed using multiple neuron levels of the machine learning model that have the one or more temperatures and one or more previously associated flow rates or one or more previously associated flow volumes; as well as The output is provided after evaluating the one or more temperatures using the previously associated one or more flow rates or the previously associated one or more flow rates in the cooling system.

15. The at least one processor according to claim 13, wherein: The one or more circuits are also used to transmit the output to the at least one flow controller.

16. The at least one processor according to claim 12, wherein: The one or more circuits are also configured to receive temperature values ​​from temperature sensors associated with data center equipment and to facilitate the movement of coolant at a threshold based on the temperature values.

17. A cooling system, comprising: A heat load system for testing a data center liquid cooling system includes a server chassis, the server chassis including at least one thermal feature associated with at least one cold plate and at least one flow controller for the data center liquid cooling system, wherein the at least one thermal feature includes a heating element having an adjustable heat level and positioned adjacent to the cold plate, the at least one cold plate including at least one first cold plate and at least one second cold plate, the at least one first cold plate being used to induce cooling stress on the data center liquid cooling system, and the at least one second cold plate providing cooling for data center equipment during active operation; and At least one logic unit is configured to use one or more neural networks to estimate the cooling stress on the data center liquid cooling system caused by the at least one thermal feature and the at least one flow controller, the cooling stress being represented at least in part by one or more temperatures applied to the at least one thermal feature and one or more flow rates or flow rates associated with the at least one flow controller, and is further configured to provide an output associated with at least one of the flow rates or flow rates representing desired cooling for at least one input temperature.

18. The cooling system according to claim 17, wherein: The at least one logic unit is further configured to: The one or more temperatures are processed using multiple neuron levels of the one or more neural networks, the multiple neuron levels having the one or more temperatures and having one or more previously associated flow rates or one or more previously associated flow volumes; as well as The output is provided after the temperature has been evaluated using one or more previously associated flow rates or flow rates of the cooling system.

19. The cooling system according to claim 17, wherein: The at least one logic unit is also configured to output at least one instruction associated with at least temperature to facilitate movement of coolant at a threshold including at least one of the one or more flow rates or flow rates.

20. The cooling system according to claim 17, wherein: The at least one logic unit is also used to transmit the output to the at least one flow controller.

21. The cooling system according to claim 17, wherein: The at least one logic unit is also configured to receive a temperature value from a temperature sensor associated with the data center equipment, and to facilitate the movement of coolant at a threshold based on the temperature value.

22. A test system for a cooling system, comprising: A heat load system for testing a data center liquid cooling system includes a server chassis, the server chassis including at least one thermal feature associated with at least one cold plate and at least one flow controller for the data center liquid cooling system, wherein the at least one thermal feature includes a heating element having an adjustable heat level and positioned adjacent to the cold plate, the at least one cold plate including at least one first cold plate and at least one second cold plate, the at least one first cold plate being used to induce cooling stress on the data center liquid cooling system, and the at least one second cold plate providing cooling for data center equipment during active operation; and At least one logic unit is configured to train one or more neural networks to evaluate the cooling stress on the data center liquid cooling system caused by the at least one thermal feature and the at least one flow controller, the cooling stress being represented at least in part by one or more temperatures applied to the at least one thermal feature and one or more flow rates or flow rates associated with the at least one flow controller, and the at least one logic unit is further configured to provide an output associated with at least one of the flow rates or flow rates representing expected cooling for at least one input temperature.

23. The testing system according to claim 22, wherein: The at least one logic unit is further configured to: The one or more temperatures are processed using multiple neuron levels of the one or more neural networks, the multiple neuron levels having the one or more temperatures and having one or more previously associated flow rates or one or more previously associated flow volumes; as well as The output is provided after the temperature has been evaluated using one or more previously associated flow rates or one or more previously associated flow rates.

24. The testing system according to claim 22, wherein: The at least one logic unit is also used to transmit the output to the at least one flow controller.

25. The testing system according to claim 22, wherein: The at least one logic unit is also configured to output at least one instruction associated with at least temperature to facilitate movement of coolant at a threshold including at least one of the one or more flow rates or flow rates.

26. A method for cooling data center equipment, comprising: Control at least one thermal characteristic of the server enclosure used to test the heat load of a data center liquid cooling system; The at least one thermal feature is associated with at least one cold plate, and the at least one thermal feature includes a heating element having an adjustable heat level, being positioned adjacent to the cold plate, and being associated with at least one flow controller; The at least one cold plate includes at least one first cold plate and at least one second cold plate, the at least one first cold plate being used to induce cooling stress on the data center liquid cooling system, and the at least one second cold plate providing cooling for data center equipment during active operation, and the at least one thermal feature and the at least one flow controller being adjusted to induce the cooling stress on the data center liquid cooling system.

27. The method according to claim 26, wherein: The at least one cold plate includes an inlet channel and an outlet channel for circulating the cooling medium from the liquid cooling system of the data center.

28. The method according to claim 26, wherein: Each of one or more heating elements has a temperature range that replicates that of the data center equipment, and the at least one flow controller is capable of controlling the flow rate to replicate the liquid cooling system of the data center and the disconnection or liquid piping connection of the data center equipment.

29. The method of claim 26, further comprising: The cooling stress is evaluated using at least one logic unit, the cooling stress being represented by at least one or more temperatures applied to the at least one thermal feature and one or more flow rates or flow rates associated with the at least one flow controller; and Provides an output associated with at least one of the flow rate or flow rate representing the expected cooling for at least one input temperature.

Images