Intelligent and integrated liquid-cooled rack for datacenters

Abstract

A cooling system for a datacenter is disclosed. The datacenter cooling system includes a cooling manifold (304A) located within a rack (300A) having a first controllable fluid coupler (308) to receive

Description

feus For instance, the valves 326 may include a sensor to senw piexst^v nt mmHi-mmw of the coolant within the val ve: An i nsertion of the mating couplers: io a distribution coupler having a vah c > .'a may cause a variation in the                          within the valve by virtue of the WS server tray now associated with the mating couplers. The temperature change may: trigger the valve 326 to turn-on and may cause the coolant to flow into the sewer tray. With this process, the flow of the coolant may be automated when the server tray is press-feed into the rack (eg., the mating oopplm are press-fitted against the distribution couplers). [OOPS] In at least one embodiment, a further intelligence control offered using the intelligent, safe-c>mtu>l assemble fe(\ fed, rife tm rmewm e mfoim.mon '? created between ;l-v ones of the coolant imo certain areas or zones within the rack and a temperature in the areas or zones Such reference inf wruadon may also include pressure iu&rmation sensed in the areas or zones. The reference information may be stored and used terrain a neural network to recognize what antoimt Of upmiing is                   disc to achieve afiow: rate of coolant in a specific grim or zone to further achieve a. reduction in temperature in the specific area or feme. In at least fee embodiment, neurons having .input and outputs may be used in a single layer, a nmlti-Iayen a folly recurrent, a simply reeurrem, or competitive neural network. A first layer of neurons may be provided with:inputs that have adjusted pressure and flow rates, and different umraspondmg angles of opehihg valties of a valve (which may translate to linear opening, for instance, of a disc valve). The. adjustment iv that various values of pressure and: few rates may be represented in a fixed range scale:. A hidden or intermediate, layer of neurons: mav be provided with error values based in part on the result fem pdor data, which may be a temperature adjustment obtained for h, ‘ nts pi', dc< ofm fiirt jsC t m'm.ws I w         <> mor w ‘ e’o po^me o output: using the input values that is a combination of the pressure and the flow rale, for instance. The resulting temperature adjustmem may be applied when a sensor associated with an area ar zone indicates a damge m the temperature in the area or zone. [006:6] In at least one embodiment, the mtelligmice of the present system is able to then provide an input few au and associate pressure intended for a valve in the area or zone ta restrict or allow flow of the coolant into a rigid cooling manifold or sewer tray associated with the area or zone. In at least one embodiment, reference to the area or zone is made to illustrate 16 that th®                      (at many points within the rigid coding manifold), the servan trays (and each server tray or further regions within the server trays), and the couplers may be all maintained at differentteffipm raws a-msg :0-, ptewm smm in nt hast >hu vmbodimvm a portion of the server tray maj he heated more a different porU-m because of a CPI' or GPU at the portion Sensors, along with the nnellmem control of the present disci-wue is able to ensure that coolant of an appropriate temperature Bows at an appropriate flow rate to tire portion of the server tray that is hotter so that a different portion is not subject to unnecessary cooling, for instance. This may be achieved by increasing the How rate of the coolant so that the coolant teaches the portion that requires immediate cooling, in at least one embodiment. Further details of the 'mdhgent control offered to the cooling system of the present disclosure are provided with reference to at least FIGS. 14 and 15. [0G67] FIG. 4 is a feature diagram illustrating parts of internal rack components 400 including an exmnme se^c? uav oi ,ei e\.rnrle coobug pine mg 44 2 and an reampk xi-mtmimg component 412; 416 for cooling computing componems 412,416 using the example cooling system in accordance with at least one embodimen t In at least one embodiment, the computing cnmponeiits may be a circuit board or assembly 412 with cmmcetbrs 426 or a singular computing componedi 416, such as a GPU. The circuit board 412 mas include a- ta 424 ter an additional computing component, such as one or more GPUs oi a ('Pl's -GF Further. tits example server tray or the exam pie cooling plate Includes one or more ofa bottom pm non 402 and a top portion 432. When select circuit components are liquid cooled, inlet and outlet couplers: 422,424 may exist ed tile addit.^^                             The inlet and outlet couplers 422,424 enable distribution of coolant Sw the second codling loop via inlet and outlm mating couplers 408,406 of the server tray or cooling plate 402. The additional compmmg component may be plugged ip and secured to the circuit board or assemhiv i 12 m the area 42$ presided. The circuit board or assembly 412 may itself include an inlet coupler 420 and an outlet coupler 418 for receiving and returning coolant from theseeond cooling loop. As such, the additional computing coiupWvm 416 may couple to the mating couplers 468, 466 or may couple to the iniet or outlet coupler j 426 418 of the eireui t board or assembly 412.

[0668] In at least one embodiment, the servertray or cooling plate, either at the bottom portion 4< os the n?",> norm--' I mm, mUmio an widnred -Jnowel 4O-i a to re^-' ibu-mame chjm el 31.6 referenced i n FIG. 3D) that may loop through the tray or plate. The emAw.-d channel -Go or the top portion 432 couples to the rack cooling manifold via server tray coupkrs tor mating 1? copiers) 44¾ 434, Further, th© enclosed charred 436 way b© a flexible tube within the server tray coupling urehe wdv^             via,server couplers 438,440, to the compnting component couplers -F22. 424. Th© ©ndosed dlaned may be ody f?n the top portion 432 and may be supported by fans 430 On the top portion instead .of the bottom portion 402, but may exist ok both portions as well. The enclosed channel of either port? an of the server tray or cooling plat© may provide cooling without rec^                to the computing components ch the additional computiug components, via the couplers on the top portion coupling directly to ©•••uplers on dm ©(imputing eoniponem or the additional computing centponents ifoum ©?, Mh, the enclosed channel and the further coupling to the computing and additional computing components may exist concurrently. Fans 430 may exist when the tray 402 is a cooling piste The fans may provide additional forced air circulation for the coolant. The tray includes spacing 41 &to accommodate the connectors 426,

[0069] FIG. 5 is a process 500 flow of steps svaiUbieTbr a method of using or making the cooling sy stem of FIGS . 1-17D© accordingdo at least one embodiment, lb at I nasi cue embodiment of.the process 500, a sub-process 502 provides a rigid coding manifold within a rad; Sub-pr.-i.-ev-, 5o4 provides coupling for a first controllable fluid coupler of the right e-'dine manifold wi th a cooling loop external to thcrack to rcedv© a coolant. A determination may b© made: via sub-process SOh that a rack of server trays repair©: coding using the cooling system. In at l east odd embodiment, when it is determined that a computing component has been changed arid that cedant Is required for a new computing ©omponem or to rod heated areas, the present process may instead provide such / cooling, via sub-process 508. In sub-process 508, coupling is offered for one or more, second controllable fluid couplers of the rigid cooling manifold to the one or more rigid server cooling manifolds within the server trays of the rack. The coup sub-process: FGS enables distribution of the cod ant within the rack. The process 500 may be used wish any rack or server tray having propriemry computing components. via at least the derennination in sub-pt ©cess 506 that a change has occurred, or the existing cooling system of sub-process 504 may be maintained if no change has occurred in the components Alternatively, the process 500 may be applied to any datacenter to adapt an edsting -cooling system for i ntel hgem cooling. GAIAfTNTFR

[0070] FIG, 6A illustrates an example datacenter 600, in which at least one embodiment from. FIGS: 2-5 may be used. Tp at least one embodiment, datacenter 600 includes a datacenter inlfettuctuto layer $1¾ a framework layer o2v, a software kwcr 630, and an application layer 640. In at least one embodiment' smdrae descrfe           a? FIG 2 features in components 204-214 may be pestomicd i aside or In collaborafion with the example datacenter 600; In at least one embodiment. the int'rasimeture layer 610, the framework layer 620, the software layer 630, and the application layer 640 may be partly or fully provided via computing componenls on see, er trays located m mcks 2 lu of the datacenter 200. This enables cooling systems of the present dswfeurc to cool the computing components in an efficient and effective manner, reduces chances of leaks, and enables-.replacement of the computing components without downtime because of the imeHigem cooling features used in the cooling system Further, aspects of the datacenter, including the datacenter infrastructure layer 610, the femework layer 620, the software layer 630, and the application layer 64Q may be used to support the mtc-lfeem processing discussed with at least reference to FIG. 3D above V such the discussion in reference to FIGS 6A-17D may be understood to apply to the hardware and software features required to enable or support intelligent control of the cooling sy stem for ths datacenter 200, of f IG 2, for ifeiauce, [0071 ]  In at fe s t ous. embodiment, as in FIG. 6^          infrastructure layer 616 may include a resource orchestrator 612, grouped cmfefeng resources 614, and node computing resources f mode                             “N” represents a«y whole, positive integer, In at least otic embodiment,, node CR a 616( 1r6l6(N) may include, but arenot limited to, any num ber of central processing units ("GPUs”) os other processors (i nduding accelerators, field programmable gate way s          graphics processors, etc.}, memory devices (e,g,, dynamic read-only memory), storage devim               or disk drives), network mpufeutgut ("NW I / O’devices^ network switches, virtual machines (”VMs'), power modules, and cooling modules, etc. In at least one umbodnnetn, <w o, nn>ie node C R.s from among node C,Rw 616(1 )-616(N) may be a server having one or more of above-mentioned computing resources. ((>< >72) In al least one embpdiat^nl, grouped, computing resources: 614 may include separate ^'•'upiu^s of node C R s housed v.uhb; one oi nne racss (m-l <n mam racks fiorofe re datacenters' at various geographical locations (also not shown); Separate groupings of node C.R s within grouped computing resources 614 may include grouped compute, network, memory cr storage resources that may be configured or allocated to support one or more workloads. In at lea&i one embodiment several node f R.s including CPUs or processors may grouped within one or more racks: io provide compute resources to support one or more workloads. In at least one emhodimestt, One dr more racks m melude a w number of power modules, cooling, modules, and network switches, in any crnnbinatibn.

[0073] In at least one embodiment, resource orchestrator 612 may configure or otherwise control one or mom node CXs 616(1)-61 6(H) and / or grouped computing resources 614. In at least one embodiment, resource orchestrator 612 may include a software design infrastructure RSDF) management entity: for datacenter 600; In at least one embodiment, resource orchestrator may include hard wan- softwaie or some combination thereof. [IK174] in al least one embodimem, as shown in FIG. 6A, framework layer 620 includes a job scheduler 622. a coniiguration manager 624 a resource manager 626 and a distributed Ilie system. t>28. In at least one embodiment, framework layer 620 may include a framework to \uppm > software 632 oi snow me lavrr oSo and m one or more applications si o-l2 of apphuamm layer 6-10. In al least one embodiment, software 632 or appliesitems) 642 may respectively include web-based aery ice software or applications, such as: those provided by Amazon Web Services, Google Cloud mid Microsoft: Azure. In at least One embodiment, framework layer 620 may be, but is not limited to, a type of free add open-source; software web applied lion, framework such as Apache Spark(hereinafter ASpark") that may       distributed file sy stem 628 for large-scale data processing (¢^,.5¾ data '). In at least one embodiment,, job scheduler 622 may include a Spark driver to facilitate scheduling of workloads supported by various layers of datacenter 606.. Ingt least one embodiment, configuration manager 62-1 may be capable of configuring different layers such as software layer 630 and framework layer 620: mcludlng Spark and dwmbuted ole m stem ig: M-ppoimurimw scale date mocmwmg In at Rsm -.-nc em bodi ment, resource manager 626 may be capable of managing clustered or grouped computing resources mapped tow anoeuwe for mpport of distributed file sy stem 628 and job wmedufri 62? te at Rao mH' embodiment cUiMew-d or y-ouurd exmpmmg resources ma> include grouped computing resource 614 m datacenter mfmstmcrnre layer 610. In at least one embodiment, resource manager 626 may coordinate with resource orchestrator 612 to manage these mapped or allocated computing resources. 10075] h-s at least one embodiment, software .632 included in software layer 630 may include software used by at least portions of node C.R.s 616(1 )-616(N), grouped computing resources 614 arm dmuilmted fik srrtcm :'2S ofifoineuvik laws ufO. One or more types of software may include, feist are not- limited: to, Internet web page search software, e-mail vires scan, software, database software, and streaming video content software. [0076} In at least one embodiment, application's) 042 included In appUeato layer 640 may Include one or more types of applications used by at least portions of node C.R a 616( I )-6l6(Nf grouped computing resources 61.4, and / or distributed file system 628 of frame work layer 620. One or more typos of applications may include, but are not lumied to, any number of a genomics application, a cognitive compute, and a machine learning application, including iraimng or mierenciug srd^     machine I earning. framework software (e.g , Py Torch, Twwoi 1 low ¢. affty etc) or other ma chine learning applications used in conjunction with one or embodimems. (0077} In at least one embodiment, any of configuration manager 624, resow ce manager 626, and resource orchestrator 612 may implement any number and type of self-modi ft reg actions based on any amount and type of data acquired tn any technically feasible fashion In at least one embodiment, self-modifying actions may relieve a datacenter operator of datacenter 606 ftom making possibly bad eonftguration decisions and. possibly avoiding undmhilw and / or poor perform lag portions of a datacenter.

[6078] In at least one.embodiment, datacenter 600 may Include louis, sereices, software or other resources to tram one of more machine learning models or predict or infer information using one or more machine learning models according to one or more embodiments described herein. In at least one embodiment. In at least one embodiment, a machine learning model may be trained by calculming weight parameter according to a neural network archin Oha- fwiry software and computing resources described above with respect to datacenter 606 Pt at nreo ■me embodiment, trai nod maehi ne learnt ng model s corresponding to; o; re or mure nuu. al: w tw os k s may be used io infer or predict information using resources described above with respect to datacenter 600 by using weight parameters oaladated through one or more training techniques, described herein. As previously discussed, deep learning techniques may be used to support intcUigcm control of valves in the intelligent cooling system Deep learning may be ads aueed using any appropriate learning network and rhe computing capabilities of the datacenter w?0. As such, a deep neural network (DNN), a recurrent neural network (RMN) or a-convolutional neural network (CNN j may be supported either simultaneously or concurrently using the, hard ware in the daUireme- Ch me a network is muned and sucuessfuHy evaluated to recognize data within a subset or a slice, for instance, the named nee1,ere an provide sirmlar tepresentam e data .for using wnh the collected data.

[0070] In at least one embodiment, datacenter 600 may use UPUs,:apphcation~specific integrated orams (ASICs), GPUs, FPGAs, or either hardware to perform training and / or inforencmg using above-described resot trees. Moreover, one or more software and / or hardware resources described above may be configured as a. service to allow users to train or performing inferencing of information such as pressure, flow rates, temperature, and location information, or other artificial imdhgence services, l\l Fi<l \CE \XDTR UMAG UU fofidfo Inference and / or training logic 615 may be used to perform inferencing and / or training operations associated with one or store embodiments In at least one embodiment, inference and or training logic 615 may he used in system FIG bA for inferencing or predicting operations based, at least in part, m weight parameter calculated using neural network iraifoug operations, utuud umwed bunmonx mid m mdmemmr^ ot neural netwmk axe uws descubed Iwaem In at ieast one embodiment,.inference,and / or training bogie o I5 may include, without limitation, hardware ! ogle in which computational resources are dedicated or otherwise exefesivdy used in conjunction, with weight values or other informan on eonesponding to one or mote layeux neurons within a neural network. In at least one embodimeuti inference andfor training: logic 616 may be used in conjunction with an. appiicafiomppecific integrated circuit (ASICsuch as Tensorflow® Processing Unit from Google, an inference processing umt (1PU) 'from Graphcure™, or a Newana® (e g,, "'Lake Cresf?) processor from Intel Corp. [0081 ] In. at least one embodiment inference audfor training logic 61,5 may Im used s a conjunction with central processing: unit (CPU) hardware, graphics processing unit vUl’U): hardware or other hardware, such as field, progmmmahle gate: arrays (FPGAs). In at least one embodiment, inference and / or training logic 615 mdudes, without limitation, code and / or data storage modifies which may be used to Sion? code (e,g„ graph code.;-, weight values and / or other information, including bias values, gradientinfiirmation, momentum values, and / or other pmaineWf <r mpmpammern; wfounmum In ?; least one &mnodmwm, each of me code and m data storage modules is associated with a dedicated computational resource In at least one embodiment, the dedicated computational resource includes computational hardware that farther m^lude one isr more ALUs that perform nmfnmwmca! fuucuunx such us linear algebraic functions, dpte on Information stored in code and / or data storage modules. and results front which are stored in an activation storage module of the inference and / or tmining Iogic 615.

[0682] FIGS. ®, 6C illustrates inference;and / or training logic, .such as used in FIG. 6A and in at: least: one embodiment of die: present disci osure, according: to at 1 east one cmbodimcnt. The inference sed er training logic 615 are used to perform mferenemg and / or training operations associated wdh at least one embodiment. Detads regarding Inference and / or naming logic 615 are provided below in canjonction syith 6B and / or 6C. The inference and / nr training logic 615 of FIGS. 6B and 6C are distinguished by the use of the arithmetic logic units (ALL\) 610 \crsn\ tbe cmmmiatiomd hmdwme otC 666 In at least one onmediioent, each m'computational hardware 662 and computational hardware 606 includes one or mor© ALUs that perform mathematical tVechons, such as linear algebraic functions, only on information stored in code and or data storage 601 and code and / or data storage 605, respecti vely, result of which is stored in activation storage 620 As such, FIGS, 68 and 6C may be alternatives andmay be used interchange^ unless stated otherwise..

[0083] In at least one embodiment Inference and / or training logic 615 may include., without Imu&timg code and / or data storage 661 to store forward and / or output weight and / or mput / putput data, and / or other parameters to configure neurons pt Use? -> of a neural ‘vtunrR trained and / or used for referencing in at InaM niie embodiment I he Ux mmay be ire. d w u\ levels of compression as previously described elsewhere in this disclosure. In at least one embodiment, training logic 615 may include,, or be coupled to wdc and or data storage 60 I to store graph, code or other software to emrirorti mi                in which weigltemid / or other parameter mformation is to be loaded to configure, logic, including w    and / or Ifoaling: point units (collectively, arifomeftt logic units (ALUsf In at least ona embodiment code, such as graph code, loads xveigbt or other parameter infoouatipn intp processor ALUs based on an architecture:of a neural network to which the code corresponds. In at least one embodiment, code and / or data storage 601 stores wei gin parameters and / or mpat / cutput dam of each kwrr of a neural network trained or used it; conjunction with at least one embodiment during forward propagation of input / output data and / or weight parameters during training and / or infofencing using at least one embodimem in at least one embodiment, any portion of code and / or data storage 601 may be included with other on-chip or off-chip data storage, including a processor’s LL L2, or L3 cache or system memory

[0084] In at least                      po *iec oi u©de and / or data storage 601 may fee internal or external to oneor more processors or other Jiardwaw devices or circuits. In at least one embodiment, code ahd / dr code aad / of data / stonve " 6I « w beeache nwmory, dynamic randomly addressable memory (^DRAM”),. static randomly addressable memory (“SRAM”), mn-volaide memory (e.g;, Flash memory), or other storage. In at least one embodiment, choice of whether code and / or code and / or data storage 601 is internal or external io a processor, for example, or included of DRAM. SR\M, Flash or some other storage ty pe may depend on available storage: on-chip versus oil ch p, latency requirements of training and / or inforencing functions being performed, batch sizv of data 'used in foferencmg and / or training: of a neural network, or some combi nation of these factors

[0085] fo nt least one embodiment, inference and / or u aimfig louse 6 15 nw include, without limitation, a code and / or data storage 605 to store backward and / or output weight and / or inptrfomtput data emrespondmy io neurons or layers of a. neural network trained tmd / or used for infeamcing in at least one embvdiment. In at least one embodiment, code and / or data storage 605 stores weight parameter                data of each layer of a neural network hamed er ased in coujunctiou with at least one embpdime'U dmmg b^kw^ propagati on of input / outpui data and / or weight parameters dmlug trai ning and / or iuferencmg using at least one embodiment. In at least one embodiment, training logic 615 may indude. or be coupled t© code and / or data storage 665 to store graph code or other;software to control timing and / or pfoep in which weight and / or other parameter mfommmm m lobe loaded to conllgnm, lope, mc.foding integer and / or flofong pomt wms moUcetw eh, authnittm logic uwls t Al I As

[0086] In at least one embodiment, code, such as graph code,.loads weight or other parameter information into processor ALUs based on an architecture of a neural network ;o winch the c>Me corresponds. In at least one embodiment, any portion of code and / or data storage ©05 mar be included with                oil 'Chiu dale storage, inelt^ a processor s 1.1,12. o? 13 cache or system memory. In at least one emb- aliment any portion of code and / or data storage 6()5 may be unequal <n eOerual tn un om or more proewwois o; <nhm haidwaic Mme deuces vi cuemm in al least one embodimeni. code and / or dam storage 605 may be cache memory, DRAM, SRAM, uGuwulatiie memory (e.g., Flash memory), or other storage. In at least one embodiment, choice of whether code and / or data storage 605 is internal or external to a processor, for example, or included of DRAM, SRAM, Flash er some other storage type may depend on available storage oh-chip versus off-chip, latency requirements of framing and / or inferepeing fused ems being peribrm^ batch size of data wed m rales cueing and / or gaining of a neural network, or some combination of these motors. [00W] In at least one embodiment, code and / or data storage 66I and code and / or data storage 605 may be separate storage structures In ar least one embodiment code and / or data storage 601 and code and / or data storage 605 may be same storage structure. In al least one embodiment code and / or data storage 601 and code and / or data, storage 605 mav be partially-same storage -structure- and parti ally separate storage structures,- In at least one emhoA meng any portion of code and / or data storage 601 and code and / or dam storage 605 may be included with other on-chip or otf-emp data storage mei tiding a processors LI, L2, orL3 cache o> system memory. 10088] In at least one embodiment, inference and / or training logic 61 may include, without hmudlion, one or ww miiluww hym uun(\}{" M I s sU) o irahmmg ranger :md os P.sUUi^ point units, to perform logical mid or malhemmical operations based, at least in part on. or i nd! cated by,: training and / or inference code (e. g., graph code), a result of which may produce activations (e.g , Output values from layers or neurons within a neural network) stored in an act v adon storage 620 that are functions, of m                 weight parameter data stored in code and / or data storage 601 and / or code and / or data storage 605. In at least one embodimen t, activations stored in activation storage 620 are generated according to linear algebraic and or matrix-based mathematics performed by ALL sm b I!? in response to pqrfpnmng instniciions or other code, wherein -weight values stored in codeand / or data storage 605 audmr code anAor data storage 601 are used as operands along with other values, such as bias values, gradient nifonnatwn. mu-mmmm wdacs o: other pmtmenn $ oi hvwi parameters am, c -if. of which maybe stored in code and / or data storage..60S or code and / or data storage 601 or another storage on or oiLchip.

[0089] In at least one embodiment, ALUff) 610 are included within one or more processors or other hardware logic devices or circuits, whereas In another embodiment, ALIAs) 6lu* may be «' leu -.- n» a processor or other hardware logic desace or circuit that uses them (e g.. a coprocessor)- In at least one embodiment, ALOs 610 may be included within a processor’s execution units or otherwise within a bank of ALUs accessible by a processor s / execution units either wiihm same nrocess^i os disn-brnwl between different processors of different types (e g , eenvul processing unM gpaphi^                               units, etc ). In at least one embodiment, code.and / or data storage 601, code and / or data storage: 605, and aetn ation storage 620 mav be oft same processor or other hardware logic device or cr< m\ vnereas in another e>nbodimem. thev mav be in different processors or other hardware Ionic devices or circuits, or some combination of same and dtifeem processors or other hardware logic devices er circui ts. In at least one embodintem. any poitmn of aetu adon storage 620 may be included with other on-chip or uff<mp data storage, including a processor's LI. L2, or 1.3 cache or w stem memory. Furthermore, infereneing and / or training code may be slot ed with other code accessible to a processor or other hardware logic or circuit and feudud and / or processed using a processor s fetch decode, scheduling, execution. retirement and / or other logical circuits [0()9()] hi at least one embodhusm, activation storage 620 mav be cache nu-motx. DR AM, SRAM, hon-volatile memory (e.g., Flash memory), or other storage. In at least one embodiment, activation storage 620 may be completely or partialB wttbm ns eMmtial to one or more processors or other logical circuits. In at least one embodiwai, choice of whether activation sioragw620 is internal of external to a processor, for example, or included of DRAM; SRAM, Flash or some mlmr oor;uw h pemay depe^ available storage on-ehip versus off-ehip, latency regmremems of imiumn and / or inferenciug fun.ctiws being performed, batch size of data used in mferehciftg and or u.doing of a neural network, or some combination of these foctmw ini at leastone embowmem, imerence and / or training logic 615 illustrated in FIG. 6B may be used in cogi unction with an application-speeibc integrated circuit (“ASKAl such as TensorFow® Processing Unit from Google, ah inference processing unit (IBU)rfwh < n epheote™, or a Mewaua® fo.g.,;4Lake CreA’) processor from Intel Corp. In at least one embodmmm, inforence audfor t^ o;A illustrated In FIG. 6B may be used in eonjunction with central processing nnu f CG. ) hardware, graphics precessing unit f'GPU0) hardware or other hardware, sad5, w field programmable gate arrays t' FPG As”). [0W1] In at least one embodiment, as illustrated in FIG, 6C, inference and / or tt.ur mg logic M5 m.w mclmh. 'Adhm.f hmitae.-u fe .Cware mmc m which c<<>?puhmmml .Cw'umex am dedicated or otherwise exclusively used in conjunction with weight x aloes or other information corresponding to one or more lay ers of neurons within a neural network. In at least one embodiment, inference and / or training logic 615 illustrated in FIG. 6C mav be used in conjunction with an applicMion-speciftc integrated circuit (ASIC), such as Tensorfiow® Processing Uni? from Google, an inference processing unit (IPU) from Graphcore w. or a bwivamkh (e.g, "Lake Crest5'i processor from Intel Corp, In at least one embodiment, inference: and / or training logic <?• i 5 illustrated in FIG; 6C m ay be used m wry unction with central processing unit (CPU) hardware, graphics processing    (GPU) hardware or other hardwire, such as field pwvrammable gate arrays (FPGAs) In at least one embodiment, inference and / or training kxm e i s i resides, without Umiration, code and / or data storage 601 and code and / or data stocrw -"Of. which may be used to store code (e.g , graph code), weight values and / or other informatioR, including bias values, gradient infoanation, momentum values, and / or other or hyperparameter information. In at least one embodimen t Illustrated in FIG. 6C, e.n h -4 code and / or date storage 60 i and code and / or data storage oOS is associated with a dedicated computational resource,,such as computational hardware 602 and computational hardware o06, respecn s ely.

[0092] In at least one'embodiment, each of code and / or data siomge 601 and 605 and corresponding computational hardware: 602 and 606, respemi vd\, eoncspmid to dillerem layers of a neural network, such that resulting activation from ode" slot age comouHuonal pair 601 / 602'" of code and / or datastorage 60.1 and computational hardware o92 is pwv ided asan input to Gtomge / bomputMionaf 605 o0o" of code and / or data rrmago 60$ 1})U| utmpmatianal hardware 666, in DF&r wm              oiganimiGi Of a neurnl w bxoil In at least one em bodi ment, each of storage / computatioual pairs 601 / 6Q2 and 665 / 666 may correspond to mure than one neural network layer. In at least one embodiment, additional siorage / cumputatlon pairs (not shown) subsequent to or in parallel with storage cmnpm.mon pairs 601 / 602 and 695 / 666 gray be included in inference,and / or training logic 615. COMPUTER SYSTEMS. [009)] FIG, 7A is a block diagram illustrating:an exemplary computer system 700A, which may be a system with interconnected devices and components, a svstemwm--a-chip (SOC) or some combmaui'n thewof formed wm a pu vowur -hat mm include ewwiitma units as evwmte tin in stria: tit.m io support and / or to enable the inidligerst cooling system described herein according to at least one embodiment. In at least one embodiment,, computer system 700A may include, without limitation, a compo-vem, such as a processor / 02 to empuw eseemion units including L-gic to perform algorithms fur process Juia, in asvordance with present disclosure, such as in embodiment described herein In a? least one embodiment, computer system 7imA 27 may include processors. such as PENTIUM® Processor family. Xeon!M, Itanium®, XScalem and / or Strong ARM ’Intel® Core™^ or Intel® Nervaafo** microprocessors ..available from imd Corporation of Santa Clara. California, although other systems (inciudmg PCs basing other microprocessors, engineering workstations, set-fop boxes and like) may afro be used. In at least one embodiment, computer system 700B may execute a version of WINDOWS’ operating .■>s Mem as vlnb e Mw-'.W'ii * oipuidifo'i of Redmond. ^asb , ihhongb odmr operating.-systems (UNIX and Linux for example), embedded sol^arv. and / or graphical user interfaces, may also be used.

[0094] 5u ,u Umm <embodiment the exempla™ .•ompnu.r m stem 7w;A mny incorpmate one or more <'f eompoHem-' i M-i U-1 uom FIG D - o support ph\as$hig aspect.' for the intelligent cooling s\ swm For at least tins reason tn -me embodiment. FU s "A JiuMratvs a system, which includes mtercmuected hardware devices or “chips”, whereas in other embodiments, FIG. 7A may illustrate an exemplary System on a Chip f“$oC') In at least cue embodiment, devices may be interconnected with proprietor, t oerconnects, standardized interconnects (eg.. PCk) or some combination thereof. In at least one embodi ment, one or more components of computer system 7008 are mterconnected using compute express link (CXL > interconnects. Inference and / or training logic 61J are used to perform infereucmg and / or mairfoig o^^ with one or .mofo ernbedimemg / previously discussed with respect to FIGS. 6A~C, for instance. Details regarding Infemnce and / or framing logic 015 are provided below m ermjmmtitm with FIGS, 6 / VE. In at least one embodimem, inference and / or■ training lowe o 15 may he used in system FIG. 7A for miWimcing bFprediding operations based 3t least m pan on weight parameters calculated usmgmenral aetwprk forming operations, nemsi network functions and / or . amhito^     or neural . network use cases described herein. [WD51 Embodi-menU may be used in other devices such as handheld devices:and embedded applications Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants ('• PDAs" >, and handi-eld PCs. In at least one embodiment, embedded applications mav include a mierocimtroUsr. a digital signal processor t 'D.xP''}. system on a chip, netwojk compmers r NotPUs'T set-top box> ncowd hnK, wnlc area network (“'WAN") switches, or any other system that may perform one or m<Pe instructions in accordance with at least one embodiment

[0099] In at lesst ooe             dQ&pufef System                  without Bm nation, processor 702 tot may Include without limi tation, one Of mare: executi on units 7u& to perform mtone tornto mtow toitog:atotototonto                    toeribed herein. tn at leto mtotoaditonf computer system 700A is a single processor desktop or server system., bm in <mutof eimtoimem sompmm sv-aem ~ m?X max be a miikmu* ess-'t ax cvm In as to< one embodiment, processor 7v2 may include, without limito^ a complex, -instruction set computer s 'CSC ') microprocessor, a reduced instruction set computing (“RISC") microprocessor a very long mstruuifon wurd C VLIW") microprocessor, a processor nnptowmmg s wtomiamm o: urhm mm wo m mx ether ^loosd dexue, such a etond signal processor, for example. In at least one embodiment, processor 702 may be coupled to a processor bus. 710 that may transmit data signals betxveen processor 702 and other components in computer system 700A.

[0097] In at least <w embodiment, processor 702 may Include, wt thorn limitation, a Level 1 (“LI”) -internal cache memory         704. In at tost one embodiment, processor 702 may have a single inrerpal cache or multiple levels of internal cache. In at least one embodiment, each® memory may reside external to. processor 702. Other embodiments may also include a combinabon of both mte; to ami external caches spending           hopfewtotlon and needs, list tost one embodiment register file 706 may store different typesofdata in various registers Including, without I imi tad on, integer registers, floating point registers, status registers, and instruction pointer register. [O09S] In at least one embtomem, execmi on amt ?M> mtoding. without Smiiaboos logic to perform integer and floatmg point operations, also resides in processor 702, In at least one embodiment, processor 702 mas also include a microcode (“ucpde”) read only memory mmrucude for certain macro instructions. In al least one enibodiment execution unit 708 may include logic to handle a packed instruction set 709, In at least one embodiment, by including packed touuction set 709 ia an instruction set of a general -purpose ,>,vmAxut ~l>2 ahr g xvlb c^socaud hx i.' e-wenm ipq;uem j > efx?, u>« s uto. fo m.-m muliirnedm applications may be pexfooned using packed data in a general-purpose processor 702 In one or more embodiments, many multimedia applicatioxxs may be accelerated and executed mure efllcientl.y by using full width of a processorw data bus for performing uperatiuns on packed daia ui?vh mas eliminate need to uansfer smaller units of data across processor's data bps to perform one or more operapons one data elemeur at a time.

[0099] In at least rmeembodiment, execution unit 708 may also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits In at least one embodiment. computer system 7ou.A may iuch.dc. without Itmirrtnon., a memory 720 kt at least one embodiment, memory 720 may be implemented as a Dynamic Random Access Memory (“DRAM") device, a Static Random Access Memory f:SRAM"i device. flash memory device, or other memory device. In at least one mnbodiment, memory 720 may store iostruedonfs) 719 and / or data 721 represented by data signals that may be executed by processor 702. [GW] In at least one embodiment, system logic chip may be coupled to processor bus 7 W and memory 720, In at least one embodiment, system logic chip may include, without liinflation, a memory controller huh ■ 'MCH”i7h\ and processor 702 mav communicate with MCH 71 6 via processor bus 710 In at least one embodiment. MCH 716 may provide a high bandwidth memory path 7 is to memory 720 for instruction and data storage and .tor storage of graphics commands, data and textures. In at least one embodiment, MCH 716 may direct data signals between processor 702, memory 730, and other components in computer sy stem 700A and to bridge data signals between processor bus 710, memory 720, and a system I / O 722. & at least one embodiment, system Ingle chi p may provide a graphic s port for coupling tom graphics controller In at least one embodiment, MCH 716 may be coupled to memory 720 through a high bandwidth memory path 718 and graphics / video card 712 may be coupled to MCH 710 through an Accelerated graphics Port C‘AGP’’} intercom 7M (0101) In at least one embodiment, computer system.700A may use sy stem I / O 722 that is a proprietary hub imerfaee bus to couple MCH 716 to I / O controller hub f 3CH !) 730. In m mast one embodiment, ICH 730 may provide direct connections to some I / O devices via a local I / O bus. In at l east one embodiment, local 170 bus may include, without limitation, a high-speed 170 bus for connecting pes ipherals to memory 720, chipset, and processor 702. Examples may ndmk* vuhomb mm mi. an aadm crmmrle^ a f imw ,e hub t f mt Ilir fo" i 728 a vmdev* tmocrucT "26 a dm,i swage ~2a a egam I O ^rnrmvki '2' > cm rising use mpm and keyboard interfaces 725, a serial expansion port 727, such as Universal Serial Bus (MJSBn, and. a network controller 734. Data storage 724 niay mdude.a hard disk drwe, a floppy disk drive a CD-ROM device, a flash memory device, or other triads storage device. iGI02] FIG. ~B is 3 I* dmgrmn illustrating fin electronic device 700B for utilizmg a processor 71.(1 to support and / or to enable^          cooling system described herein, according to at least one embodiment: In at least one embodiment, ekctromc device 7008 may be, .for example and without limitation, a notebook, a tower served a rack server, a blade server, a laptop, a desktop, a tablet. a mobile device, a phone, an embedded computet, or any other suitable electronic device. In at least one embodiment, the exemplary electronic device 7008 may Incorporate one or more ofc opponents 328, 332 (from FIG. 3DMo support proc e^) ng aspects for theinteliigent cooling .system. iObU| him least one embodiment wstem 7vO8 max irelud<- wukm.; hum mon ptoce’^'t 7bJ commmncauvds coupled loans suitable number or kind of vise punem^ ootip-t* taw modules, or devices. In at least one embodiment, processor 710 coupled using a bus or interface, such as a rC bus, a Sx stem Management Bas (x<SMBus?’)s a Low Pin Count (IPG) bus. a Serial Peripheral Interface f'SPF) a High Definition Airdip ("HDA" t bus, a Serial Advance Technology Attachment (''SATA") bus, a Universal Serial Bus ('USB") (versions 1, 2, 3), or a Universal .Asynchronous Receiver / Transraitter (“UART") bus. In at least one embodiment, FIG. 7B illuslrmss a system, which includes imes-connected Iwdw^                     whereas in other embodiments. FIG.. 7B may illtetmte an exemplary System on a Chip C'ScC'). In at least one embodiment, devices illustrated in FIG. 7B may be interconnected with proprietary rmcmomieets, t^ndard^ed interconnects (wg:, FCIe) or some combination thereof In at least one embodiment, one or more components of FIG. 7B are Interconnected using compute express jink ¢( XL) mnxc<'nnrms [0 104] In at least one.embodiment, FIG 7B may mcludb a display 724, atOHch screen 725. a touch pad 730, a'Near Field Communications unit (^NFC) 745, a sensor hub 740. a thermal sensor 746, an Express Chipset (“BC”) 73 5, a Trusted Platform Module CTPM'T TM. BIQS / finnwai^flash memory fsB10S, FW Flash”) 722, a DSP 760, a drive "Md such as a Solid. Slate Disk ("SSD'l or a Hard Disk Drive < "HDD' k a uhdess h-cal area Hewed- unit f WLAV ) "fir j Bhteloodi dim 752. a '4 u<k-v? ide \iea Xkwm k ami f \VW.AX's ”56. a Gkvij Posilkmmg bvstem iGPSr 755. a emmna i '1 'SB , <j <.,uutuf' ■ "^4 vm 1' as a I 'xR ' 0 camem. amber a Low Power Double .Dam K.Uc ('1 IM >DR i mcmorx urm ( I PDUR I” i / w ■ mpk-mented in, for example, LPDDR3 standard. These components mav each he implemented in any suitable mamier.

[0103] In at least on<; embodiment. other components may be conununicanvely coupled to processor 710 through conipoT’<mts discussed above. In at least one embodiment an accelerometer 741. Ambient 1 n'c Sensor ALS”) 742, compos 74 \ mid a gyroscope 744 may be cotemubicaiivUy coupled to sensorIfeb 740. B at least one embodiment, die-nal sensor 739, a fan 737, a keyboard 746, and a touch pad 730 may be communicate ely coupled to EC 7 15 In at least, one embodiment, speaker 763, headphones764, and microphone (“mic”) 765 may be communicatively coupled to an audio unit (“audio codec and class d amp-’) 762, which mav in turn be communicatively coupled to DSP 760. In at least one embodiment, audio unit 764 may include, for example and without limitation. an audio coder / decodcr I 'codec") and \ ohms D ampEdes la ,h one embodlmcm, SIM emd mslM';- 7':~ mav be couminnltaewb. •,copied io WW AN unit 75b In ai least one embodiment, components such as WLAN uni- 750 and Bluetooth unit 752, as well as WW AN unn “5c mav be implemented ma NextGeneration Form Factor ("NGEF')

[0106] Inference and / or training logic 615 are used to perform inferencing and / or training, operations associated with one or more embodiments. Details regarding inference and / or training. logic 615 are provided bekw m conjaaenou -with F1GS. 6B and / or 6C, In at least one embodiment., inference and / or training          may be used in system FIG. 7B for inferenefeg: or predicting: operations based, at least in part, on weight parameters calculated using neural network training operations, neural network fences and / or architectures, or neural network use cases desen bed. herein.

[0107] RG. 7C illustrates a computer system "GOG according.to at least one embodiment, to support and / or to enable the intelligent cooling system described herein. In M-teaSt one embodiment, computer system 70QC mdudes. without limitation, a computer 771 and a USB stick: 770. In at least one embodiment, computer 771 may include, without limitation. any number and type of procevsorts) (not shown} and a .memory (not shown), lu at least one embodiment, computer 771 includes, without limitation, a Server, a cloud instance, a laptop, and a desktop compute?

[0108] In at least on« embodiment. USB stick 770 includes, without limitation, a processing unit 772, a USB interface 774., and USB interface logic 773. In at least one embodiment, processing unit 772 may be any instruction execution system, apparatus, or device capable of executing insintetions. In at least one embodiment, processing unit 772 may include, without limitation, any number and type of p’ <\ cwmg ceres (not shown t In at least one embodiment, processing unit or cere 772 comprises an-application specific integrated circuit ('‘ASK?') that is optimized io perform any ambtmt and type oimwratious associated with machine leunnne For instance,, lot least one embodiment, processing, core 772 iga tensor pwBsing gait f kTPC”) that is optimized to perfem machine learning inference operations. In at least one embodiment, processing core 772 is a vision processing unit (“ATIF) that Is optimized to perform machine vision and machine learning inference operations.

[0109] In at least one embodiment USB interface 774 may be any type of USB connector or USB socket. For instance, in al least one embodimern. USB interface 774 is a USB 3 0 Type-C socket for data and power. In nt least one embodiment USB interface 774 is a USB 3 B Type-A connector. In at least one embodiment, USB interface logic 773may include atty amount and type of logic that enables processing unit 772 to interface with or devices (e,g,, computer 771) si 11'SB wtwcM

[0110] Inference and / or training logic 615, as described with respect to FIGS oB and 6(7 are used to perform inferen ci ng and / or training operations associated with one or more enfoodi m ents. Detail s regarding inference and / or train! ng J ogi c 61 bare provt ded below in. emptfochon with FIGS, 6B andfor 60. In at least one embodimrM inference and / or training logic &I5 mav be used in system FIG. 7C for mferencing or predicting operations based, at least in pari, on v ew In parameters.calculated using neural network training opmntions, neural network functions arZ , i architectures, or neural network use cases described herein. [0 I N | FIG. 3 iilustraws a further esampto cm -pmc? o sw-w * fo at cojehu to at least ore embodiment, to 'implement forioils processes and methods for du- imelhgent cooling owem described throughout this disclosure. In at least one embodiment, computer system 300 includes, wiihom iimufoon, at least one eerstrai processing unu ( CPU"} F02 that is connected to a communication bus 810 implemented using any suitable protocol, such as PCI ^Peripheral Component Interccmro / f' k peripheral compum-m unereunnect express cTCfoExpressfo, AGP (''decelerated Graphics Port"), Ry per I ran spurs, or any other bus or point-tonoint communication protocol^?. In al iwor one embodiment, computer system SCO weh.w a, uuUmt limitation, a main memory 804 and control logic {Ag., implemented as hardware, softycare, or a ernubmation. thereoffond data are stored in main memory 804 which may take form of random access mtmires t          In ar toast out ^nbeehntetH a utwaotl Hilcfowe subsx Ucm ( netootk interface") 822 provides an interface to ether .cam puling devices and networks for receiving data from and transmitting data. to other system iw computer system 800. [011.2] In at least ope embodiment, computer system 800, ip at I east one embedi m en;. i ncludes, without limitation, input devices 801, parallel processing system 812, and display dm ices 800 which can be implemented using a cathode ray tube C'CRT"X lifted crystal display Ul.CD'fo 1 ighi emitting diode cl. EP'O plasma display, nr other suitable display -technologies. In at least one embodiment, user input is received from input devices 808 such as keyboard, mouse, touchpad, microphone, and more. In at least one embodiment, each of foregoing modules can be situated on a single semiconductor platform to form aprocessing system. 10 JI 3 | lafonmce and / or training logic bl 5 are used to perform htfereneing and / or training operations associated with one or msw embodiments, us pre*. ion sly discussed with respect io FIGS 6A-C for instance. Details regarding irdcrence and / or training logic 6 15 are provided below in conjunction with FIGS- tvX-C In at least one embodiment, inference mid-or training logic 615 may be used in system FIG 8 for imenmemg or predicting operations based., at least in part, on weight parameters calculated using neural network training operations, neural network ftmenons and / or architectures, or neural network use eases described herein. font least one embodiment, mforence and / or fuming logic 615 may he used in system FIG, 8 for infemuemg or predicting operations: based, at least in part, on weight parameters: calculated using neural iMteofk training operations, neural network functions and / or architectures, or neural network use cases described herein. 10114-1 FIG illustrates an exemplary architecture in which a plurality df GPUs 910-933 is < ommunwam civ coupled to a plurality of mnlti-eore processors 905^1)0 o' er high-speed links 940-943 teg . buses, poim-to-pOint interconnects. etc.). In one embodiment, high-speed links 940-943 support a communication throughput of 4GB / $,30GB / s, 80GB / s or higher. Vamms interconnect protocols may be used including, but not limited to, POe 4,0 or 5.0 and NVUnk 2.0. [01 '15] la addifi on, a nd i n one embodiment, two or more of GPUs 910-913 are mtetcotmected over high-speed links 929-930, which may be implemented using same or diffinern protocols / links than those used for high-speed links 940-943 . Simiharly, two m more of multi core processors 905-906 may be connected over high speed link 928 which may be symmv-ric midii-processor (SMP) buses operating at 2UGB-X 3«iB. s. 120GB<s or higher A.Iternativeto, all 34 communication between various sywm components «shown m HU a a max be accomplished using same protocols / ! inks (e g., over a common reteicommcmm 1 tbm) [01 lb] femeembodiment, each midfecpre,precessor9d5fe^            cs ively coupled to a propesw memory 901-902, viavnemofe interconnects 926-927, respectively, and each GOU 910-913 is communicatively coupled to GPU memory 920-923 over OPU memory interconnects 950-953, respectively, Memory, imerconnects 926-927 and 950-953 may utilize same or different memon access technologies. By way of example, and not limitation, processor memories 90 l -'02 and GPU memories 920-923 may be volatile memories such as dynamic random access memories tDRAMs) (inchiding stacked DRAMs), Graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6). or High Bandwidth Memory (HBM) and / or may be nonvolatile memories such as 3D XPoint or Nano-Ram. In one embsvlimcm, some portion of processor memories 901 -902 may be volatile memory and another portion may be non-volatile memory (e.g., usings two-level memory (2LM) himarehys.

[0117] .As described below, although -carious processors 905-90o and GPUs 910-01 > may be physically coupled to a particular me-ut     । .vol, 929-923, respectively, a umfied memory architecture may be implemented in -which a same virtual :system address space (also referred to as “effective address” space) is distributed am® re x arious physical memories. In at least one embodiment / processor memories 901-902may each include 64GB of system memory address sisace: and GPU memories :920-923 may each include 32GB of system memory address space (resulting i.n a total of256GB addressable memory in: this example). [OHS] As discussed elsewhere in this disclosure, at least flow curs and a pressure ubi« are established for a sum level of an intelligent learning system, such ax a cereal network ‘vst«a As die fast level represents the pri or associated data, it also represents^: smaller subset of the data that may be available io improve the system by retraining fee system. The testing and ■ ramreg may be perfbnned in parallel using the muirip processor units so that the Intelligent learning system is robust. An architecture, such as in FIG -A. may be used. When convergence is achieved for the intelligent learning system, an amount of data points and the data in the dam points used to cause Cm amv^rgeuce is noted Fbc data and data points -nay b< axed io coimcl fee intdhgem cooling system as fegcussed in reference, for in stance to FIG. 3D.

[0119] FIG, vB jHusiviitcs additional details for interconnection between a multi-core processor 907 and a graphics acceleration module 946 in accordance: w embodiment Graphics aeceleMfos mte bde < »4« < may include: one or mom GPU chips integrated cP a line card which is coupled to processor 907 via high-speed link 940, Alternatively, graphics acceleration module 94b may be integrated on a same package orchip as processor 907. p*l 201 hs at least one embodiment tHustramd processor 90? includes a plurality of cores ^u0 \-9wgy each with a translation lookaside butfe 961A-961D o e or moreroaches 962A-‘<0    ?♦ teas:. ne ctnbvd mete emos 9m' V'oUD p' a induce xa rom odu c>te'geuvms tot executing instructions and processing dam which are not illustrated (laches 962A-962D may include level 1. ¢1 / 1) and level 2 (L2) caches. In addition, one pr more shared caches 956 may be included an caches 962A-962D and shared by sets of cores 960A-960D; In at least one embodiment v sc tmibodtnwm vf proceswa 9o? mchidts 2-1 o-scs, vacn ^nh ns onu 1 I cache twelve shared L2 caches, and twelve shared L3 caches. In this embodiment, one or more L2 and L3 caches are shared by two adjacent cores. Processor 99? and graphics acceleration module °46 connect with system memory 914, which may include processor memories 961-902 of FIG. 9A.

[0121] Coherency is mmniafoed for date and: iostamtlrms stored In various caches 956 and system memory 914 viadnhr-core crmimmucMron over a coherence bus 964, in at least one embodlnient, each cache may have cache coherency Ipgic / mcuiiry associated therewith to cummwicwte over coherence bus 964 In response to detected tends or writes to particular cache lines, hi one implememmiou, a cache woopteg protocol is implemented: over coherence bus 964 to snoop cache accesses.

[6122] In one embodimem, a proxy circuit communicatively couples graphics: acceleration module 946 to coherence bus 964, allowing graphics acceleration module 946: to participate in a cache coherence protocol as a peer of cores 9<>0A..960D. In pateeular, an interface 935 provides connectivity to proxy circuit 925 over high-speed link 940 (e.g., a PCle bus, NVLink etc J and an interface 937 commits graphics acceleration module 94o uj link '!4<;

[0123] fe one implemeuteworn   accelerater integration circuit 936 provides cache management, memory access, context management, and interrupt management sen sees on behalf of a plurality of graphics processing engines 931, 932, A of grtpuro; acvdviatwm module ugo thro \s pk\k’sm_. crgoKV' t K}w A "\w roc't u\ ifoc a wpmme i.,apl'A ' pu-i e-rong 36 unit (GPU). Ahernameb. graphics processing engines 931, 932, M                   upes of graphics processing engines within a GPU- such asgraphics ew                processing engines (e g . vidWencoders.4l?codersl .fo           blit engines.: In atleast We embodiment, graphics aecel eratfon module 940 may be a GPU with a plumlity of graphics processing engines 9> I -932. U or graphics processing engines 931 -932, N may be individual GPUs integrated on a common package line card, or chip. As is the case, the. above determination for the reconstruvtson parameter and the reconstruction algorithm may be performed m GPUs 93 UN of HG f’B,

[0124] In one embodiment, accelerator integration circuit 936 iududes a memory twmageme';* unit (MMU) 930 for performing various memory management functions such as virtual-to-phvsical memory trandatfons (also referred to as eiTecto e-to-real memory translations) and memory access protocols for accessing system ^14 MMU 939 may also include a translation lookaside bittier (TLB) (not shown) for caching virtuaVefJe^ivu to physical / real address Umrslations. In orie implm a cache 933 stores commands ;md data for emclent access by graphics processing engines 931.-932, M In one embpdrerem, data stored in cache ”3S and graphicsmwmmUs 933-934, M Is kept coherent with core cat:ks '^.VkCD. v5#x and system, memoir e 14 As mentioned above, this may be: accompli shed via. proxy circuit 925 on. behalf of cache 93b and memories: 933 -934, M (e.g., sending updates to cm he 9 IS folatedto niodiS^           of cache lines on processor capites 9W U-U2D, 95u, and receiving updates from cache 938),

[0125] A set of registers 94S store contest data for threads.executed by graphics processing: engines 931 -93.2, N wd a context management circuit 948 manages thread contexts. In st. least one embodimeht, context management, etreud 94S may perform save and restore operations to save and restore, contexts of various threads doting contexts swi tches (og., where a ft rat thread is saved and a second thread is stored so that a second thread can be executed by a mapbics processing engine). In at least one embodiment, on a context switch, context management circuit 948 may store current register values to a designated region in. memory (e,g,, identified by a: contevi pointer} n may then restore register values when returning to a. context. In one embodiment, an imemipt management circuit 947 receives and precesses interrupts received from svstem desires.

[0120] In oneimplmnentati^ vinual / effecdve addreases from a graphics processing engine 921 are translated■toreai^^ addresses in system memory $14 bv MMl' $39. One OTbtxidn^ of accelerator Integra don circuit $36 supports multiple (e g . 4. 8. lb) graphics accelerator modules 946:yn^                        Graphics accelerator be dedicated to a single application executed on processor 907:or tnay be shared between multiple applications. in one embodiment, a virtualized graphics execution cnvironnwir is presented in which resources of graphics processing engines 93l -“32. N are shared with multiple applications or virtual machines (VMs). In at least one embodiment, resources may be: subdivided into “slices'' which are allocated io different VMis and / or applications based on processing requirements and priorities associated with VMs and or applications

[0127] In at least one embodim-mt, accelerator integration circuit 936 performs as a bridge to a sy stem fm guppies acueiuramm mc-dule $49 and pirn irks .iddsvas tmasianon and sx wm memory cache services. In addition, accelerator integration circuit $3o may provide virtualization fadbdes for a: host processor to manage virtualization of graphics processing engines 931 -932, N. interrupts, and memory management

[9128] Because hardware resources of graphi cs processing engines 931 -932, N are mapped explici tly to a real address space seen by host processor 907, any host processor can address these resources directly using an effective address: value. One function of accelerator, integration, circuit o>,: m one embodiment, is physical separation of graphics processing engines 931-932 , N so that they appear to a system as mdependmtt units. [01 29] In at least one embodiment, one or more graphics memmh-s ot rtvO; v 3Je coupled O' each of graphics processing engines 931-932, ^respectively. Graphics memories 933-934, M store instructions and data being processed by each of graphics processing engines 931-932, N , Graphics memories 933-934, M may be volatile memories such as: DRAMs (including stacked DRAMs), GDDR memory (e.g , GDDR5, GDDR6). or HBM, and / or may be non-voiatHe memories such as 3D APoint or Nano-Ram.

[0139] In one embodiment, to reduce data traffic over link 940, biasing techniques are used to epsumthst data stored in graphics memories 933-934, Mds data which will be used most frequently bv graphics processing engines 931-93.2, bi and mav not used by cures $60A-9<:>0D (at least not frequently)- Similarhn a biasing mechanism attempts to keep data needed by cores (and may not graphs > P'A'ee.'Mne vmntx 'W0 '2 S} w a ?u caches VkdX °50 of cores and system mommy 514, [013I] FIG, FC illustrates another exemplary embodiment in which accelerator integratiorr circuit 936 is integrated within processor 903: for enabling and / or supporting intelligent coolfog system, according to at least one embodiment of the disclosure herein In at least this embedimem, graphics processing engines 931 -932, N communicate directly over high-speed itch 940: to accelerator mtegranon circuit 936:via interface 937 and interface 935 (which, again,, may be utilize any form of bus or interface protocol). Accelerator integration circuit 936 may perform same operations as those described with respect to FIG 9B. but potentially at a higher throughput given its close proximity to coherence bus 964 and caches 962A-962D, 956; At least (sue embodiment supports .bt'Atcm programming models including a dedicated-process programminginodei (no graphics acceleration module virtualization) and shared programming, models (with virtual izm ion ), which may Include programming models which arc controlled by accelerator integration Jrcmt. 936 and programming ..models which are controlled by graphics acceleration module 946,

[9132] In at least one embodiment processing engines 931 -932, :N are dedicated to a single appHmpon. or process: udder a single operating system.. In at least ope embpdfo'O'c, a single application can found other application requests to graphics processing engines “ H -932, N, providing virtualization within a VM / parddOn, «9 toj to atltoto ewb, .brawn waprato pu>ues,'»geng'iics ;'G J>32 b. mas I c foased fo multiple VM-dpplicatlon p»i’um»iis, .In. at Ie wt one vmbodimcm shared models may use a system .hypervisor ton nns mre emphu pwefong euymex 9G .vhl N to allow access by each operating system; For single-partition systems without a hypervisor, graphics: processing engines 931 -932, N are owned by an operating System . In at least one embodiment, an operating system can virtualize graphics processing engines 931-932, N to provide access to each process or apph notion. (01341 In at least one embodiment, graphics acceleration module 946 or an individual graphics p-wvossmg engine 931-032 N selects a process demem using a process handle. In at least one embodiment. prute>$ elements are stored in s^ rtent memrav o j-i two are tukbf.'sablt usmg an eHcctoc add-ess to ted .ddiv-ss tunfoit on teebragu;; ocwnl'cd heswn hi at kast mw embodiment, a process handle may be an mrpktrmnmtfomspecdk value pourtdvd to t host process when registering its context with graphics processing engine 931 -932,N (that is, calling sy stem ,$ofiware:to add a process element to a process element linked list). In at least one embodiment, a lower I ri-bits of a process handle may be an offset of a process element within a. process element linked lisp (id 351 FIG. 9D illustrates an exemplary accelerator integration slice 990 for enabling and / or supporting the intelligent cooling system, according to at least one embodiment of the disclosure herein. As used herein, a 'Allee” comprises a specified portion of processing resources of accelerator Integration circuit 936, Application effective address space 982 within system memory 914 stores process elements 933. In one embodiment, process elements 98?» are stored In texpanse to GP1; uv ocamms US: bum applnamr,," '•’bn esw otwl on pow<"oot w>? X process element 983 remains process state for corresponding application 980, A work descriptor (WD)984 contained in process element ^85 can be a singleJob requested by an application or nitty contain a pointer to a queue of jobs. In at beast cue embodiment, WD 984 is ;t pointer to a job request queue in an application a address space 982; I91.KJ Graphics Kveicrahon module c,4o and re indn’dual graphics processing engines -31 -93¾ A can be .shared by all or a subset of processes In a. sy stem, In at least one embodiment, an infrastructure for settingup process state and seeding a WD 984 to a graphics acceleration: module 946 to start a. job in a virtualized envirtHwent may be included, (9137] In atleast one embodiment, a dedicated-process: programming model is implemematipti-specific. In this,model, a single process owns graphi cs acceleration module 946 or an individual graphic pioe<\->sing mug ne 9) ; Because graphics acceiemuon moth.de ‘ Ari is owned by a Angle process, a hypcrxixra initializes accelerator integration circuit 936 for an owning partition and an operating system initializes accelerator integration circuit 936 for an own? ng process When. graphics acceleration in Odule 946 i s assigned, (9138] In operation, a WD fetch twit 991 in accelerator IntegrafiGu slice 99Q fetches next WD 984 which includes an indication of work to be done by one or mure graphres yr> wessmg engmea of graphics acceleration mudt.de 946. Data front WD 984 may be stored in registers 945 and used by MMll 939, mterrupt management circuit 947, and / or context management circuit 948 as illustrated, in at least one embodiment, one embodiment of MMU 939 includes segmem / page wo'.k ci!cuUi\ nv accvssmg ^eraent page uulex uribm <>x ' ntual add'esx space ^S'-interrupt management circuit 947 may process interrupt evems 992 received from graphics accelmmfee module 94S. When petfon^                  -an effective address 0® generated by a graphics processing engine® 1-932, N is W to a real address by MMIJ 939

[0139] In one embodiment, a same set of registers 9® are duplicated Err each graphics processing engine 931-932, N and / or graphics acceleration module ®6 and may be initialized by a hypervisor or operating system, bach of these duplicated registers may be included in an accelerator -integration slice 999. Exemplary registers that may be'initialized by a hypervisor are shewn in Table 1. Table 2 --Operating System InmahzeJ Registers' I             | Frosess and Thread Identification 2            | Effective Address (EA) Context Save / Restore .Pointer .......................................................... 3             | Virtual Address (VA) Accelerator Utilization Record Pointer 14           | Vhtu.d Xdd-w.rek \} Sior^ Seymour Table Powe; i™-................................3.....,........................................................................................................................................................................................... | 5            | Authodt^ Mask | 6             | Work descriptor

[0141] In one embedtmenk each WD 984 is specific to a particular graphics acceleration modulo 946 and / or graphics processing engines 931-932^ N . ft contains all information required In a graphics processing engine 93 i -932. N m do noik or n can be- a pointer to a mmeon. location where an spplicatioa / has set up a command queue of work to be completed,

[0142] FIG fo Illustrates addiuonal deems for one exemplary embodiment of a shared model . This embodiment includes a hypervisor real address space 998 in which a precess element list 999 is stored, Ilyperv <sor real address '■.pace 998 is accessible via a hypervisor 996 which virtualizes graphics acceleration module engines for operating system 995, [01-fo| In al least one embodiment, shared programming models ullou for all or a subset of precesses from all or a subset of partitions in a system to use a gramnes acceleration module 946 There are two programming models where gum a. § accelsmimmoibre ou, vS sSued by multiple processes and partitions: time-sliced shared wdgmphics-dwm^

[9144] In this model, system hypervisor 996, owns graphics acceleration module 949 and makes its fonctirm available to all operating systems 995, For a eiaphres acceleration module 946 to support virtualization by system: hypervisor 996. graphics, acceleration module: 946 may adhere to tire following: j) An, application's job request must be autonomous (that fo state does not need to be maintained between jobs), or graphics acceleration module 946 must provide a context save and restore mechanism, 2) An application's job request is guaranteed by graphics acceleration: module 946 to compl ete in a specified amount of lime, including any translation fafots. or graphics acceleration module 946 provides an ability to preempt processing of a job. 3; Graphics acceleration module 946 must be guaranteed fairness between processes when operating in a directed shared programming model.

[0145] In al least one embodiment, application 980 is required io make an operating system 995 system call with a graphics aecvlerwrem w 946 type, a work descriptor (WD), an authority reemtef cWllO \^|ue, and a context save / resture area pointer (CSRP). la at least one embodimem graphics acceimaifou nt<'‘lme °4o b-pe Jevamex a nuge-ud .kccIci it ion funcuon for a system call. In m leas? one embodiment, graphs acvelcrahcn nnmute °4r< pc may be a system-spe^ In ai feast on® embodiment. WD is fonua-u^d snecu'icalh fo? graphics acceleration module: 946: and can be in a form: of a graphics accd^      modish' «4«» COTmaM effective address pointer to a user-defined structure, fei effective address pointer to a queue of commands. or easy othe? data structure to describe work to be dene by graphics acceleration module 946. In one embodiment. an .AMR value is an AMR state to use for a current process. In at least one embodiment, a value passed m an optraong system is similar to an application v an AMR. If acvel era tor integration ci a ui t 0 3b and gt aphi c s aa el eradou m.-Julo Mo mrcen.vnt.ihon', co mm xappi'tl a I w-s Autbo.m Mmk( hm.tde RemMet (I-AMOR), an operating system may apply a current IJAMOR. value to an AMR uhse bet ore passing an AMR in a by petvisor call. Hypervisor 996 may, in at least one embodiment. apply a current Authority Mask Override Register (AMOR) value before placing an AMR mt-.- process element OO In ar lm,A mu end idwer CSKP rs mse of tcutsR<s s-n? c<mk-:nn.u. an .mecme address of an area in an application A effects'. k / dress space 982 for graphics m\ eknau-.m module 9-16 to.save and restore context state : m> pointer is used in at least oneembodiment if no state is required to be saved between jobs or when a job is preempted. In at feast bite dm bodi mem; cpmevt save / rest»re area m ay M firmed system m W®y i [01:46] Upon receiving a system ealh d^btetidg: system 995 may verify that                has registered and been given authority to use graphics acceleration module 946. Operating: system 90S then calls hypervisor 996 with information shown in Table: X Table 3 -OS to Hypervisor Call Pammeters: 1 A work descriptor (WD) ■j .An Authority Mask RvgiAer (AMR.) value (potentially masked} 3 Ac effective Address (l;i A) Context Sas c Restore Area Pointer (CSRP} 4 A process ID (PIO? and optional thread ID (TlDt 5 A virtual address <VA) accelerator utilization record pointer (AURP) 6 V irtual address of storage segment table psjirster (SSI P) 7           LA logical intenupt wvm^                                         j

[0147] Upon receiving a hypervisor hypervisor 990 verifies that operating system 995 has: registered and been given authority to use graphics acceleration module 046 Hypervisor 996 then puts process dmmmi 9*3 imo a process element linked list for a corresponding graphics acceleration module '46 type. A. process element .may include intonmfim shown in Table 4. Table 4 -Pnaeess Element Information I             I .A work descriptor (WD) 2             | An Authority Mask Register (Ak-IRi value (potentially masked). 3            i Xu effvctixv address U ^ omext Vne Restore A«ea Pomt>.r (( \RPi 4        ....^.^^ 5             | A virtual address t VA) accelerator utilization record pointer (AURP) 6           i xirtua «<\Mwx m mmwe owmv. m emit-poimer s'SSTP} 7             1 A logical intenupt. service number 8             | Interrupt vector table, derived from hypervisor call parameters 9             i A state register (SR) value 10r A logical pammnH.l>a.HlM 11            I A.real address (RA), hypervisor accelemtor utilization record painter 12           | Storage Descriptor Register (SDR)

[0148] In at least one embodiment, hypervisor imtialkes a .plurality of accelerator integration slice 990 registers 945.

[0149] As illustrated in FIG. 91g id at least one embodiment a wified memory is used, addressable via a common vwmai memors address space used lo access physical processor memories 901-902 and GPU memories 920-923 In this implmnernmiom operations executed on GPUs 910-913 utilize a same vinual / efTeeuve memory address space to access processor 44 memories 901-902 and x ice' ihctebv Amplifying progratnmabiliw: In one embodiment a first portion of a                               allocated to processor memo-v 'ML a second portion to second processor memory 902, a third portion to GPU memory 920-add so w, hr at Ieastdne: embodiment, an entire virtualfoOctivv memory space (sometimes referred to as: an effective address space) is thereby distributed across each of processor memories 90L902and GPU memories 920-923, allowing any processor or GPU to access any physical memory with a virtual address mapped to that memory

[0150] In one embodiment, blas / coherenee management circuitry 994A-094E xvithln one or more of MMU$ 939A-939E ensures cache coherence between caches of one or more host processors (e g, 995) and GPUs 9JU-913 and bnpiements biasing techniques indicating physical memories in which certain types of data should be stored. Win »• multiple instances of h ,w coherence manage cent cwmmrv -fM VLUr an- dWtiame m FIG °l loss ci-hmcncv circuitry may be implemented whhin an MMU of one or more host, processors 905 and / or within accelerator integratiott circuit 936. 101511 One embodiment allows GFU-attached memory 929-923 to be mapped as part of system memory, and accessed using shared virtual memory (SUM) technology , bm wi thout suffering performance drawbacks associated vs G full ss stern cs^he coherence.. Ih at least one embodiment, an. ability' for GPU-a ttached menx>r\ • s) * m he accessed as system, memory wi thorn on grow cache coherence overhead pwvt des a bene ft cml operating wi rohm em for GPU offi md. Thi s: arrangement al I ows host processor 905 isoffware to setup operands, and access computaiirm results, wi t bout overhead of tradition 1 / 0 DM A data copies. Such traditional comes involve driver calls) Imefnrpts and memory: mapped I / O (MM 10) accesses that are Mt 9^^ relative to simple memom accesses.. In at least one embe^jr.eut a i ?h n> to access GPU attached memory 920-923 without cache coherence oseriwmff w,\ b^ to execution time of an offloaded computation. In eases with substantial streaming write menioiy traffic, for example, cache coherence overhead can si gel tie antis reduce an effective write bandwidth seen by a GPU 910-913. In m least one embodiment, efficiency of operand setup, efficiency of rewrite access., and efficiency of GPU computation may play ;■ role in detenuming effectiveness of a GPt offload. [0 i 52] In at least one embodiment, selection of GPU bias and host processor bias is driven by a bias tracker data, struenne A bias table may he used., for example, which may he a page- granular structure (in at least one embpdimem ihi's may be controlled at a granularity of a memory page) that inckides I or 2 buy                 memory page, In at teast one embodimmfo a bias table may be implemented m a stolen mem cry range of one or more GPU-attached memories 930-923, with br without a 'bias. Cache id GFU 910-913 (e.g, w cache freqaendy / recendy used entries of a bias table) Alternatively, an entire bias table may be maintained within a GPU. [0 i 5 3 J in at least one embodi mem, a bi as table entry associated with each access to GPU-attached memory 920-933 is accessed prior to actual access to a GPU memory, causing the following operations Fl? st, local requests from GPU °10-^13 that find their page in GPU bias ate forwarded directly to a corresponding GPU memory 920-923, Local requests from a GPU that find their page m host bias are forwarded to processor 905 (e.g,, over a high-speed link as discussed above).. In one embodiment, requests from processor 905 that find a requested page in host processor bias complete a request like a normal memory read. Alternatively, requests directed to a GPU-biased page may be forwarded to GPU 910-913, In at least one embodiment, a GPU may then transitiml a page to a host processor bias if ri is out cmremly using a page In at least, oneembodiment, bigs state of a page can be changed either by a     are-based mechanism, a hardware-assisted software-based mechanism, or, for a limited set of cases, ® purely hardware-based mechanism, (9154] One mechanism for changing bias state employs an AFI call (e.g., 0penf I ), which, in turn, calls a GPU’s device driver which, in turn, sends a message (or enqueues a command descriptor) to a GPU direcimgft to change a bias state and, for some iRmsitforw perform a cache flushing operation inahost in at least one embodiment, cache flushing operation is used for a transition from host processor 90? bias to GPU bias, but u not for an opposite traiwidon,

[0155] In one embodiment cacho coherency is maintained by temporarily rendering GPU-biased pages uncacheable by host processor 90S. To access these pastes, processor 905 may request access from GPU 910 which may or may not grant access right away, Thus, to reduce communication between processor 905 ami GPU 910 it is beneficial to ensure that GPU-bia^ed-pages me shoe which are required by a GPU hm not host processor 005 and vice versa

[0156] Inference and / or training logic 615 are used to perform one or more embod.imems. Details regarding the inference andUr naming logic 015 are pnwick-d below in conjunction with 1IGS 6B and / or 60

[0157] FIG. iOA ihutfratex c.xe^phrs i nictated circuits and associated graphics processors ihat may be fabricated using one nr more IP coses according to various embodiments described... terbin; to support and / or to en. b'c ne inrallk-em s oulmg system tn addition to what is other logic Md circuits may be iWluded in at least one embodiment indwdih^ additional graphics prucessoWcoms, peripheral interface controllers, or geumripurppse processor cores.

[0155] FIG. Iff A is a block diagram illustrating an exemplary system on a chip integrated circuit IWA that may be .-fabricated using one or more IP coress according to at least one embodiment hi at least one embodiment, imegrated circuit IGmA includes one or more application processors} 1005 (e g.. CPUs), al least one graphics processor 1010. and mas additionally include an image processor 1015 and / or a video processor 1020, any ot whk h may he a modular IP core. In at I cart one embodiment. integrated ciicuh 1000A indudes peripheral or bus tome incluuim? a USB controller 6)25, UARF controller 1050 an SPlraDK) control tor 1035, and an FS / FC controller 1040: In at least.one embodiment, integrated circuit I IWA can include a display device 1045 coupled to one or .more of a high-definition multimedia interface (HDMh controller 1050 and a mobile industry processor interface (M1P1) display interface 1055. In at least one embodimem, storage may be provided by a flash memory subsystem 1060 including flash memory and a flash memory controller Iri at least 6M esMddimeot, memory mterfp.ee may be provided via a memory conbpller 1065 tor access toWRAMor SRAM memory devices, to: at l east one embodiment, some integrated circuits additionally include an embedded security engine 1070: [0 ISO] Inference -arid / or trai to ng I oglu 61 / 1 used to perform toferencing aMtoo Operations associated with one or more embodiments. Details regarding inference aad'or training logic 615 are prowled be! w in gonjupction with FIGS. 6B and / or 6C. to at least one embodiments inference Mdtortte         to 5 may be used in integrated ci rotor 1000A tor Inferenc’my. or pradiCtma operaiimis Erased, m toast u- part, on ueight parametera ealeutoted using neural network training operations, neural network fimetious and / ^ architectures, or neural network use cases described hereto, [OtoQ] FIGS. lOB-IOCillustrate exemplary -integrated cireuits and associated emtom.s processors that may be fabricated using one or more IP cores, aceortong to:vanou> vrabodEmeuts <.t wubeo t er o jmppon endAjr to enable the iritelHgen! cooling system. In midribm to what is 47 illustrated, other.loginabdeh^           included in at least oneembodiment including additional graphics processors / cores, peripheral imes face conn oilers, or general-purpose processor cores. [0lol] FIGS, IbB-lbC are block diagrams illustrating egsnrphry graphics proc^ within an Soil, according to embodiments described herein, to support and / or to enable the intelligent cooling system. In an example, the graphic processors may be used to perform simulated testing of the intelligent cooling w yem. because ofexisting math engines capable of faster processing of meln ■ Ie\ ei iteural aetwor U. FIG. 1 QB illimitates an exemplary graphics processor 1010 of a system on a chip integrated ci reel! that may be fabricated using one or more IP cores, according to at least one embodiment .FIG, 19C illustrates an additional exemplary graphics processor 1040 of a system on a chip integrated .circuit that may be fabricated using one or mote IP cores, accordi ng Mat least one embodiment. In at least one embodiment, graphics processor 1010 of FIG 10 A is a low power graphics processor cose In at least one embodiment, graphics processor I04Q: of FIG: 10C Is a higher performance graphics processor core. In at least one embodiment, each of graphics processors 1010, b’M can be variants of graphics processor 1010 of FIG, MA. [GIP2] In at least one gmbodim^t, graphics processor 1010 mdmle'' a vertex processor I005 and one. m more,fragmmup          lOlSA-d01SN (e g.: 101 SA, 101 *B, 10 MG I oISD, through 1O15N-I,and W l SN), In at least one embodiment, graphics processor 1010 can execute different shader programs via separate logic; foch that \ ertex processor i005 is optimized to execute operations^ vertex shader programs, while mm pr more f^ processors) 1015 A~ .101SM execute fragment (e.g,,.pixel), shading operations for fragment or pixel shader programs; In at least one embodi them, vertex processor 1005 performs a vertex processing stage of a 3D graphics pipeline and generates primitives and vertex data. In at least one embodiment, fragment procexsous) 1015M1015N use primitive and vertex data generated by vertex processor 1005 to Produce a ifomebuffer that is displayed on a display device. In atleasf one embodiment, llagment processorfs)              are optimized to execute fragment shader programs as provided for m anOpenGL API. which may beused to perform similar operations as a pixel shader program as provided for in a Direct 3D AIM. [0 i63 ] In at 1 east one embodiment, graphics prodeSsor 1010 addmimaily includes one or more memory wan igt-mem amis (MMl's) iirfb ICCoB. cachef m 1025 A-102Gf and eu>.im intefeonne^              In at least ope embodiment, one or more MMLG) 1029A-IQ26B provide for formal Io physical address mapping for graphics processor KUO. including for vertex processor 1005 and / or fragment procwsor(s) W15A^ which may refejwnce vertex or image / textw data stored iri memory, in addition to vertex or image / texture data Stored in one or rune c foav ' iO2’X-HC^B In a ea< uim cmKxnnrem ore re mree MM1 sfe 102U Vbfofo may be synchronized with other MMUs within system including one or more MMUs associated with one or more application processors) 1005, image processors 1015, and-or \ idee processors 1020 of FIG. ISA, such that each processor 1005-1020 urn participate in a shared er unified virtual memory system. In at least one embodinrem, one or more circuit interconnect^) 10.30A-1030B enable graphics processor 10 Id to im.erfaee with other IP cores within SoG, either via an internal Pus of SoC or via s direct connection

[0164] In al least one embodiment, graphics processor 1040 includes one or more MM I >(s) IG2€iAG020Bs cachef.s) 1G25A-IG25B, and oircfot interconnect 10IfeX-lOfo of graphics processor 1G10 of FIG: WA, In at feast one emboditeem. graphics processor l-nfo* includes one: or more shader coretst IO55A- I0S5N (e.g,,.105^   1055^, 1055(2 lO^^IX 1055F, 1O55R through 1055661, and 1055N), which provides for a unified shader core architecture in which a single career typeor core can execute all types of prtsgrammable shader code, indudingchader progratn code to implement vertex shaders, fragment shaders, andfor com pufe shaders. In at least onaembodiment, a number ^                    In at feast one embodiment, graphics processor 1040 includes an inter-core task manager 1045, which acts as a thread dKXHCbes to dispatch exeeudbu threads to cate or more shader cores 1O55AG055N and a ulutg mm B>5b m accelerate tiling operations for tile-based rendering, hi which rendering poem-ionx fw a scene are subdivided in image space,.for example to exploit local spatial coherence within a scene or to optimize use of In temal caches.

[0165] Inference and / or training logic 61.5 are used to perform inforencing and / or training operations associated with one or more embotihnems. Details regarding intervnce and / or u.dmue mmc 015 are pso^ nkd berew re cmnam'mn wuh I IGS oB and os oC In <u Ie wt cue embodiment, inference andfor training logic 615 may be used in Integrated circuit IGA and / or 10B for infereucing or predicting operations based, at feast in port on weis.fht parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use eases described herein. [016b] FIGS. IGD-lOU II In Afaie additional exemplary graphics processor: logic according to. embodime^               to support and / w to enable the intelligent cooling system FIG. 10D illustrates a graphics Core 1000D that may beincluded within graphics processor 1010 of FIG. I0A, in at least one embodiment w              shader core Wo5A-1055WO in FIG 10C in at least one embodiment FIG I OB illustrates a highly-parallel general-pwpose graphics processing unit 1030 suitable for depict mem on a multi-chip module in at least one embeds a-ent. [Old?] In al leas! one embodiment graphics core I GOOD can include .multi pie slices W01A-1001N qj partition for each core, and a graphics processor can include muhipie instances of grap hics core 1GOOD; S11ees 1001 A-1001N can m»l ode support logic i nduding a local instruction cache 1004A-1OO4M,a thread scheduler 100dA-1006N, a: thread dispatcher IOA» WQSbg and a set of registers I010A-IQI0A. In at least one embodiment slices 1001 A-100 IN cm eic-mLa sm or addaa-md luucmvi u:ms t V'l A 00 7 Viol 7N j, r^amig-poup ixF {FPl 10l4A-d0]4Ny integer arithmetic logic units I ALUs 1016-1016N), address computational units (ACU 101MG013N), double-precision fl.^tihg-point units (DPFFU lOISA-HMcNh and matrix: processing units (MPU 101 FA-fofoN J. [0 IOS] In at least one embodiment FilUs I014A- 101.4N cam perform, single-precision (dOAlf) and half-precision (Id-bit) floating point operations, while DPFPUs 10I5A-1O1.5M double precision (64-6(1) floating point operations.. In at least ©he embodiment;. ALUs IO16A-101 ON can perform variable precision integer operations at 8-bit, 16-bii, and 32-bltprecision, and can be cpubgimjd lw mixed precision operations. In at least one embodiment MFUs 1617A-1M7N canalso be cortfigmed for mixed, precision matrix operations, including half-precision floating .point andS-bit Integer operations. In at least one embodiment, MFUs 10T7A-1017N can perform a yarktv of matrix operations,to: accelerate machine learning application flAmewofos, inchiding enab hag snpportdbr.accelerated general msfox to matrix multiplication (GFMMi In mi east one embvd-ment, AFUs 10L7A-I0MN can perform additional logic eperitium; not ^ipported by floahng-pmm oi h tugm units mchkhne tngor-emctnc epcranors (e g . Slug, Cosine, etc.). 10160] As discussed elsewhere in fhis.disdosme, inference and / or training logic 615 (referenced at. least in FIGS. 68, 6C) may be used to perform inferencmg ahd / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 015 are provided below in eonjunciion with FIGS. 6B and / or bC In al least one. embodiment, mferenee and / or training logic 615 mag be used in graphics core 1000D for interendrig or predicting operatiuiw based at least in part, on weight parameters calculated using neural network training opergpond neural Wwork fonctioBn                  brneural rerxsoj k use cases described herein. o 1" 'J FIG i 1 A is a block diagram illustrating a computing system 1I OGA according to at least one embodiment. In at least one embodiments compel ng xx stem IWOA includes a processing subsy stem 1101 having one ormoreprocessoi'CG . b 2 and a system memory 1104 communicating via an interconnection path that may Include a memory hub 1105 hi at least one embodiment, memory hub 1105 may be a separate component widiin a chipset eomponem or may be integrated within one or more prutessm'fs) 1 H'C In at least one embodiment, memory hub 1105 couples with an I / O subsystem 111I x ia a communication link 1106. In at least one embodimem, I / O subsystem 1111 includes an I / O huh 1107 that can eadbl.e;.compuurjg system 1100A to receive input from one .or more input dwice(s) I108. In at least one embodiment. HO hub I107 can enable a display cmitronem may be included in one or more processor^) 1102, to provide outputs to me nr mor© display          11 IiM. In at least one embodiment, one or more display deyiee(s) Fl IGA coupled with I / O bub 1107 can include a local, internal, or embedded display device. (0171] In at least one embodiment, processing subsystem 1101 includes one or more parallel processdr(s) 1112 coupled to memory hub 1105 via a bus or other conmiumcatiph link 1113. lit at least one embodiment, commimication link 1113 may be one of any number of standards based coniniunicatiori lirik technologies or protocols, such as. but not limited to: PCI Express, or may be a vendor specif               micdaee or communications fabric. In at least one embodiment, one or more parallel processor(s) 1112 form a compmationalh focused parallel or vector processing system that can mdude a large number of processing cows sml / or processing clusters, such as a many integrated core (MIC) processor. In at least one embodiment, one or more parallel pmcessousi 1112 hum a giaplw processing subsystem that can output pixels to one of one or more display dev icels} 11 b:A eoapled via j / O Hub 1107. In at least one embodiment, one or more parallel pmeessoris) 1112 can also include a display controller and display interface trad shown* to enable a direct connection to one or more di spins Jvxkxw l HOB |01 721 In at least ofo eufo, ,hmeat, a system storage unit 1114 can cohbect io I / O hub 1197 (0 pros ide a storage meenwsrp for computing system 1 IClOA. In. at least o                  1 / Q Switch ll fo can be used to pfovide           mechanism to enable connections between I / O hub II07 and other components, such as a network adapter 11IS and / or wireless Network adapter 11 19 that may be integrated imo a platformis), and various other devices that can be added via one or more add-in devices) 1 120. In at least one embodiment, network adapter 1118 can be an Ethernet adapter or another wired network adaptor. In at ienst one emhodhnem, wireless network adapter 1119 can include one or more of a Wi-Fi, Bluetooth, near field communication (^FCh or other network device that includes one or more wireless radios.

[0173] In at least one embodiment, computing system 11OOA can include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture dev ices, and so w, may also be connected to I / O hub i 107. In at least one embodiment, communication- paths interconnecting various componenis in FIG. 11A may ba im plemented uri-ig any suitable protocols, such as PO (Peripheral Component Interconnect} based protocols (e g„ P( M-spirss), or other bus or point-lopomt cummunication interfaces andfor pruocohs), such as W-Lmk high-speed interconnect, er intewomaset protocols.

[0174] 1g at least one embodWtM one ornmre parallel prouessoffs) II12 incorporate circuitry optimised for graphics and video processing, Incladirig, for example, video output circuitry, and: constitutes a graphics processing uni t (GPU). In at: least one embodiment, one or more parallel procesfof(s) 1112 incorporatu s 1 imu try optimised for general purpose .pfocessi ug. In at least one embodiment, components, of computing sy stem 1 lOOAmay he integrated wi th on e or more other system: elements oh a single Integrated circuit. In at least one embodiment, m at least one embodiment, one or more parallel processorls! UK. memon hub I IOS, processor(s) 1102. and I / O hub 1107 can be integrated into a,system on chip (SoCl integrated circun In at least one embodiment, components of computing system U(«» X can be integrated into a single package to form a system in package (SIP) configuration, In at least one embodimein, at least a portion of components of computing system I bUA can be integrated imp a multi-chip module (MCM). which can be iniercotmected with other nmlti-ehip modules into a moduia: compuuug >x sum

[0175] Inference and / or training logic: 615 are used to perform mferencing. and / or training operations: associated -with one or more embodiments. Details regarding inference aad / Or training logic b 15 are provided below in conjunction u >th FIGS. 6B and / or 6C. In at least one embodithen^ Inference and / or training logic 615 / may be used .in system FIG. I1A fbr inferebcing or predicting operations based, at least m pan. on weighl promoters calculated using neural network training operations, neural network junctions and or arChiteeUsres, or neural network, use cases described herein. PROCESSORS

[0176] FIG . 1 IB iHtotmes                   1lOOB according to: at least one embodiment. In at least one embodiment, various components of parallel process! a- HOOB may be implemented using one or more integrated circuit devices such as programmable processors, application specific integrated circuits (ASICs), or field programmable gate 'arrays (FPGA); In at least one embodiment. ilhisuaied parallel processor 1.1100B 100 is a variant of one or more parallel processed si 1112 shown m FIG, I IB according to an exemplary embodiment. 01 "roi hi m least one embodiment, paraHei processor 1 100B includes a parallel processing unit 00' In at le.ro one en.o-'.dm'enf. parallel piuiewm.' end ) 102 meludro an i O emu I Ite- dcu enables commummion with other devices, including other iustaoces of paraHei pmemtog unit I102, In at least one embodiment I / O unit 1104 may be directly connected io other devices. In at least one: embodiment I O unit 1104 connects with other devi ces via use of a hub or switch interface, such as mcmorv hub I IOS.. In at least one embodiment connections between memory hub 1105 and 1 / 0 W 1104 form a communication link 1113. hi at least one embodiment, I / O unit llWepnneds with a host interface 1106 and. a memory crossbar 11 in. where host imerfece II06 receives- com man& di reeled .to perform! ng processing operati ons and m emory crossbar 1 lie receives commands directed to pmWmingmemery operations [Rl A] in at leist due embodiment, when: host interface 1106 receives a command buffer via 1 / 0 unit I UM, host interface 1106 can direct work operations to perform thpse commands to a front end 1108. In at least one embodiment, front end 11 OS coupl ex -with a sehed pier 1110, which is configured to distribute commands or ether work items to a processing duster array 1112, hi at feast one embodiment schedule! 1:10 ensures that processing cluster array 1112 is properly configured and in a valid state before tasks are distributed to processing cluster array 1112, In at least one embodiment, scheduler 1.1.1.0 is implemented via firmware logic exeunimg on a microcontroller. Tn at least one embodiment, microcontroller implemented scheduler 1110 ■s configurable to perform complex scheduling and work distribution operations at coarse and fine granularity, emtbli ng rapid preemption and context switching of threads executing on 53 processing array 1112. In ai least one embodiment, host software can prove workloads for scheduling on process!ng array 11 12 via pw of multiple graphics processing doorbells. In at least rate embodiment, workloads can then be authtnaticahy di sirs bmed across proeessihg array 11.12 by scheduler 1110 logic wtihima mioroto               scheduler 1110. [017§| In at least one embodiment, processing cluster array 1112 can include up to "N” processing clusters (e g., cluster 11 14A, cluster I1143, through cluster 11 MM) hi at least one embodiment, each cluster 1114A-1114Nof processing duster array 1112 can execute a number of eoncurrem threads.. In at least one embodiment, scheduler 1110 can allocate 0 >« \ to clusters 1 i 14A-.1 i 14N of processing cluster array 1112 using various scheduling and / or work distribution algorithms, which may vary depending on workload arising for each type of program or computation. In at least one embodiment, scheduling can be handled dynamically by scheduler 1110, or van be assisted in part by compiler logic during compilation of program logic configured for execution by processing cluster array 1112. In at leasi one embodiment, different clusters 1114A-I114N of processing cluster array 11.12 can be al located for processing di ilerem types of programs or tor performing different typos of computations. [01 Th at least one embodiment, processing cluster array 1112 can be configured tb perform various types of parallel processing operations. In at leasi one embodiment, processing clyster array 1112 is configured to perform generabpurpose parallel compute operations, In at least one. embodiment, in at least one embodiment, processing cluster array 1112 can include logic fo execute processing tasks including filtering of video and / or audio data, performing modeling Operations,, including: physics .gperatious, and perffoming data transformntforra [31 b 11 fo at least one embodiment, processing cluster array 1112 is configured to perform parallel graphics processing operations.: In at l east one embodiment, processing cluster array 1112 can include additional logicfo support          of such graphics processing operator including, but not limited to wxture sampling logic to perform texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, cluster array 1112 can be configured to execute graphics processing related shader ptogs aws such as. but not limited to sertex shaders, ressedation shadera, geomeny shaders, and pi\d shaders. In at least one eminxliment, parallel processing unit II02 can transfer data from system memory via I / O trait 1104 tor processing. In at least one embodiment, during processing, Mnst'eueo date can be stored to dn-chip jnenu^-.Ce.g., parallel processor memory 1122} during o e^e^sl w the . written back to system memory.

[0182] fe m least ow embodiment, when, parallel processing: unit 1102 is used to perform graphics processing, scheduler 1110 can be configured to- divide a processing workload into: approximately equal sized tasks, to better enable distribution of graphics processing operahens to muluple clusters 1114A-11 MN oi processing cluster array 1J12 In at least one embodiment, portions of processing cluster array 1I12 can be configured to perform different types of processing. In at least one embodiment, in at least one embodimem, a first portion may be cenfimnee to penbr'n ' ertox fofenu and topokym g<Terom>r a strom mmfe nuw hr configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations, to produce a rendered image: for 2 ? >fe r a mnabit on of xahu cento'' f, i toe rtel reent coohoy wstem o se.ss.vd lc at le^t one embodiment, intermediate data produced by one or more of dusters 1114A-1114N may be stored in buffers to allow intermediate data to be transmitted between ciusters 11 14 A-1114N for fonder processing. [01 §3] In at least one embodiment, processing cluster array 1112 can receive processing tasks, to be executed via scheduler I1 Ifo which rece-ra orarmandsdefw^                fem fem end 11PK fe at teaM one embodiment prace^mg tasks can include indices of data to be processed, e / g., surface (patch) data, primitive data, vertex data, and / or pixel data, as well ax state parameters and .comm ands defmmg how data is to be processed (e.gg what program is tp be executed). feat fest one etoboditomfoM^^ II10 may be configured to letch indices corresponding to tasks or may receive indices fem front end HOfo fearleastone em.b^^ fem end 1108 can: be configured to ensure processing cluster arrax fe 2 is configured fo a x aim state before a workload specified by incoming command buffers feg.,. batfebuffes, push buffers, etc,) is initiated.;

[0184] In at least one embodiment, each of one or more instances of parallel processing unit 1102 can couple with parallel .processor memory 1122. In at least one embodiment, parallel processor mefumy 1122 can be accessed via memory crossbar 11 lb, which can receive memory requests few processing cluster array 1112 as well as. I / O uni t 1104. In at least one embodiment, memory crossbar 11 fo can access parallel processor memory 1122 via a memory interface 1118. In al least one embodimem, memory interface 1118 can include multiple n.nUhon mte we u paroum-. u:ut I I parlr-Vw amt 1 CuB, through pasut.on tarn I l^’V that can each couple           ieu . owners igs-O nf partt-lel processor memory 1122, lo st hast rme tenbodlment a number ol pmnheo unhs 1 i 20 A-1 2d\ s > configured to be equal to a comber of memory units, such rhat a first phrrito und 1120A has a corresponding first memory unit 112d A, a second pa ration unit i 120B has a corresponding memory unit 1124B, and a Nth partition unit 1I20N has a corresponding Nth memory unit 112 IN. In. at least one embodiment, a number of partition units I120 A-1120N may not be equal to a number of memory devices.

[0185] In al least one embodiment memory units 1124A-1124N can include various types of memory devices, including dynamic random access memory t DRAM* or graphics random access memory, such as synchronous graphics random access memory (SGRAMk including graphics double data rate (GDDR) memory. In at least one embodiment memorc units 1124 A-1124N may also include 3D stacked memory, including but nut limited to high bandwidth memory (HBM) In ai least one embodiment, render targets, such as frame buffers or texture maps may be stored across memory units 1124A-1124N, allowing partition units 1120A-1120N to write portions of each render target tn parallel to efficiently use available bandwidth of paraHei processor memory .1122. In at least one embodiment, a local instance of parallel processor memory 1.122 may be excl uded mfiwpr of aum^^ memory design that utilizes system memory ineeHunetion with local cache

[0186] In at least me embodiment, any one of blusters I1I4V11 UN of processing cluster mow 11'2 can pu-^ew erUa Hsm roll ba wntlen to mo oi memow amts 1124 VI 12-IN wuhtn parallel processor memory 1122. In at least one. embodiment, memory crossbar 1116 can be configured to transfer an output of uadi duster 1114 A-1114N to any panfikm unit I j 20 A-1120N or to another cluster 1U4A-1114N, which can perform additional processing operation^ on an output. In at least one enfoudmKmt. each cluster 1114A-1114K can eommumcate tv th memory interface HIS through memory crossbar 11 Id to read from or write W various external memory devices. In at least one -embodiment, memory crossbar 1116 has a connection to memory mleilhce I I 1$ c<«inimewu sviih i O urn I IU4 as ueH tsa cmmeCUeu --.- a local m-mmcs oi parallel processor .memory 1122, enabling processing umis widna different processing clusters 1114A-1114N to cmnmumcate with system memory or other memory that is not local to parallel processing unit 1102, in at least one embodtpteni, memory crossbar 1116 can use virtual channels to separate tmfilc streams between clusters; 1I MA-l IHN and partition units 1120A- I I2o\ [018?] In at least one embodiment, multiple instances of parallel processing: unit I102 can be. provided on a. single add-m card, or multiple add-in cards can be interconnected, In at least one embodiment. rftferem instances of paraHei processing unit 1102 car; be configured to interoperate even tf different instances have different numbers of processing cores, different amounts of local parallel processor nwn, and / or other conftguration differences. In at least one embodiment, in at least one embvu meat- some instances of parallel processing unit 1102 can include higher precision floating point units ■elative to other instances, in at least one embodiment, systems incorporating one or more instances of parallel processing unit 1102 or parallel processor 110OB can be implemented in a variety of configurations and form factors, including bul not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems. p o SS j FIG i 1C is a block diagram of a partition .unit 1120 according .toat least one embodiment , in at least one embodiment partiti on unit 1120 is an instance of on e rtf parti tion units .1120A-H20N of FIG. I IB, In at. least one embodiment, partition 'unit 1120 includes an L2 cache 112 L a frame boiler interfece                                         12b. h2 cache I121 is a read / Wte cache that is configured to perform load and store operations received ffom memory etossbar 11 IB and ROP 1126. In at least one embodiment, read mfeses and. urgent write-back requests are output by L2 cache 1121 to frame buffet interface 1125 for processing: In rft feast one embodiment, ;updates: can also be sent to a frame buffer via frame buffer imerfeee 11.‘. 5 lor processing. In m least one embodiment, frame buffer interface 1125 interfaces with one of memory units in parallel procesw momma, such as memois units 1 Q4A ifeiri ,-fFlG. 11B (Ag,, within parallel processor'memory 1122),

[0189] In at least one embodiment, ROP 1158 is a processing unit that performs raster operations such as stencil, z test, blending, and so forth, fa at feast,one embodiment, BOP 1126 then outputs processed graphics data that is stored in graphics memory In at least one embodiment, ROP 1126 includes compressfau logic to compress depth or color data that is written to memory and de> «'Huass depth or color data that is read from memory. In at least one embodimen t, compression Io < can be lossless com logic that makes use of one or more of multiple compression algor'll'.u\ Compression logic that is performed by ROP 1126 can vary based oh statistical characteristics of data to be compressed, In at least one emboduneht in at least o»e m               color compression is performed rm. depth and color data on a per-tde basis,

[0190] In at least one embodiment, ROP 1126 is included within each processing cluster (wg., cluster 1114 A-l 114N of FIG. I IB} instead of within partition enn 1120 In at least one embodiment, read and mite requests for pixel data are transmitted over memory crossbar 1116 instead of pixel fragment data. In at least oneembodiment, processed graphics data nvy be displayed on a display device, such as one of oneor more display: devicefs} 111G of IIG. II, routed for further processing by processors) 1102, or routed for further processing by one of processing entities within parallel processor MO0B of FIG. 11B.

[0191] FIG. I ID is a block diagram of a processing cluster II14 wlthm a parallel processing mat according to st least one embodiment In at least one embodiment a ptt^essmg duster is an Instance of one of processing clusters I114A-11I4N of FIG. I1B, In at least one embodiment, one of more of processing clusters) I114 can be contlgured to execute ptany threads iu parallel where Ghread’" to an instance of a particularprogram executing on a particular set of input data, Ih a t least one embodi men u smgl e~I extraction, multi pie-data t SIMD) instRictioh issue techniques are used to support parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, ^wk-instruction. multiple-thread iSiMT) techmques axe used to support parallel execution of a large number of synchronized threads, using a common msuuchon unit configured iq issue insuuctions to a set of processing epgines ‘within each one of processing clusters,

[0192] In at least one embodiment, operation of processing duster 1114 can be controlled vi a a. pipeline, manager 1132 that distributes processing tasks to SI MT parallel processors. In at least one embodiment, pipeline manager 1132 receives instructions from scheduler 1110 of FIG. HB and manages execution of those i n struc dons via a gmplucs multi processor I134 and / or a texture unit 1 tM In at least one embodiment, graphics muitiprucessor 1134 is an exemplary instance of a SIMT parallel processor. However, m at least one embodiment, various types of SI MT parallel processors of differing architectures may be included withm processing ci aster 1114. In at least one embodiment, one or more instances, of graphics multiprocessor 1134 can be Included within a processing cluster 1114., In at least one embodiment, graphics 'multiprocessor 1134 can. process data and a data crossbar 1140 can be used to distribute processed data to one of multiple possible 58 destinauons, induding other shader units. In at least one errrofemm, pipeline manager 1132 pap facilitate distribution of processed data by specs l\ mg de^marions tor processed data to be distributed vis data crossbar 114b.

[0103] In at least one embroilment each graphics multiprocessor 1134 within processing cluster 1114 can include an identical set of fund tonal execution, logic (eg., arithmetic logic units, load-store units. etc ). In at least one embodiment. functional execution logic can be eonflgured in a pipelined manner i n which new instructions can be issued before previous instructions are: complete. In at least one embodiment, functional execution logic supports a variety of opera lions including integer and floating point arithmetic, comparison opera lions, Boolean operations, bit-shifting, and compulation of various algebraic functions. In at least one embod imem, same functional -tmil hardware can be leveraged to perfbon different operations and any combination of fimctional units may be present. iofod1 in m least one cmhodinwm, mroucmws tumonutod to processing eh^fm i H 4 constitute a thread In at least one embodiment, a set of threads executing across a set of parallel processing engines is,a thread group. In at least one embodiment, thread group executes a program on different input data. In: at least one embodiment, each thread within a thread group can be assigned to a differom processing engine within a graphics nmltipw^ 1134. In at feast one embodi ment, a thread group may include fewer threads than a number of processing engines within graphics multi processor 1134. In at least one am bedim ent, when a thread group includes fewer threads than a dumber of processing engines, one or nwre processing engines: may bridle dm-ng t yeles in-which feat thread group is being processed. In at least one embodiment, a tbrojd gump may dso indade mprefe^ than a number of processing engines within graphics multiprocessor 1134. In at least one embodiment hen a thread group includes more threads than processing engines, within graphics multiprocessor 1131, processing can be performed over consecutive d©ct cycles; & at least one embodiment, mnkmfe t.ctae can He executed concurrently on a graphics multiprocessor I i 34 fofo^j la al Icasi one embodiment, g?aphics multiprocessor 1134 includes an miernal cache menrors to nmfutm load and store operations. In at feast one embodin'ient, graphics multi processor J134 can forego an internal cache and use a cache memory (e.g,, Li cache 1148) within processing cluster 1114, hi at least one embodiment, eadi graphics nmfo^ 1134 dLi'hasa.w.ssU I 3 cwlwx n m,w p romms rn'Os ■g paromm umls 11 Ifo VII dfo ef 1 hl .:1 ] B) that are shared among all processing el isvi s 11I4 and may be used m transfer data b^TOf threads.. In at bast one embodimmg graphics multiprocessor I134 may also access offi chip e obal memory, which can. mcWe one nr more pan? Ie’ processor memory and / or system memcdy, In at least one embodiment, any Memory exteruhrto parallel processing unit 1102 may be used as global memory. In at least one embodiment processing cluster 1114 includes multiple instances of graphics multiprocessor 1134 can share common insuuctions and data, which may bv stored m I. I cache I MS. [()196] In al least one embodiment, each processing cluster 1114 may include a memory managemem unit MM MU' ) i 145 that is configured to map virtual addresses into physical addresses. In at least one embodiment, one or more instances of MMU I145 may reside within memory interlace 1118 of FIG. i lB, In atleast one embodiment, MIU 1145. indodes a set Of page table entries (PTE< used to map a virtual address to a physical address of a tile and, id at least one embodiment, a cache line index tn at least, one embodiment, MMU 1145 may include address translation lookaside buffers {TLB i or caches that max' reside within graphics multiprocessor I134 or LI cache or processing duster 1114. In at least one embodiment, pb\ sied address is processed to          surface data access .locality torn! tew efficient .request interleaving.among partition units, In at least:me embodiment, cache line index may be used to determine whether a .request for a cache: line is 4 hit or miss.

[0197] In at least one embodiment, a processing cluster 1114 may be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1,5n tor performing texture mapping operafipcs,, eg,, determining texture sample: positions, reading texture: data, and fi I teri ug texture data, fit at least One embodiment, texture data is read from, an imemal texture 11 cache (not show ni or from an LI cache within graphics multiprocessor 1134 and is fetched from an L2 cache, local parallel processor memory, or system memory, as needed. In at least one embodiment, each graphics multiprocessor 1134 outputs processed tasks to data crossbar 1140 to provide processed taskt s) to another processing cluster 1114 for further processing or to store processed taskfiO in an L2 cache, local parallel processor memory, or system memory via memory crossbar 1116. In at least one embodiment preKOP 1142 (pre-raster operations unit) is configured to receive data from graphics multiprocessor 1134. direct data to HOP units, which max be Meat*.J with panm-m emts as Jescrewc rere.r! re g, past-ho' umis 11 / 6 \ 11 ;.<>% of FIG. J IB); In Mt teastms embodiment, PruROP 1142 unu peiioim crema* atonx lei ^olo'-blending,- orgauiae pixel color data, and perform address translations,

[0198] Inference and / or training logic: PIS are used to perform inferenci ng and / or training operations associated with me or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS oB and'or 6(1 In al least one embodiment inference and / or training logic 615 may be used in graphics processing cluster 1114 for inferencing or predicting operations based, at least in part, on weight parameters ealcwlated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein to FIG. 11E shows a graphics multiprocessor 1134 according to at least one embodiment. In at least one embodlment, graphics multiprocessor 1134 couples with pipeline manager 1132 of processing eluster 1114. In at least one emhudiment, graphics multiprocessor 1134 has an esecution pipeline including bus not limited to an instruction cache. 1152, an histmction unit 1154, an address mapping unit 1156, g register file 1158, one or more general purpose graphics processing mil (GPGPU) cores 1162, and one or more: load / sture units 1166. GPGPU core{s) 1162 and load / store unit(s) 1166 are coupled with cache: memory 1 172 and shared memory 1170 via a memory and cache Interconnect 1168.

[0200] In at least one embodiment, instruction cache: 1152 receives a stream: of instructions to execute from pipeline manager 1132. In at least one embodiment, instruciions are cached in instruction cache II32 and .-dispatched for execution by instruction unit 1134. In ar least one embodimeuL wstimbuu mm 115-1 can dispatch instructions as duead groups (eg . warps), with each thread gtoup■ assigned to a different execution unit within GPGPU corefe) 1162 hi at feast one embodiment an instruction can access any of a loeal, shared, or global address space by specifying an address within a unified address space. In at least one embodiment, address mapping unit I i56 can be used: to translate addresses in a unified address space into a distinct memory address that can beaccessed by Ipad / stpre umtra) I166. toloi j In 4i kwv ore eemofenwm, icgwipi Ole 1 fog pKwnks a set of -egrawra tfei fimi tornm units of graphics muitiprocessnr 1134. In at least one embodiment, register file 115S provides umpoiaw stoiCKC f u opwands ^om ected to data paths of functional units (e.g., GPGPU cores 11 cC, load store units 1 tout of graphic^ wulng ecerao. 1 la t (n at least one embodiment, register file 115d is chwded between each of -functional arms such that each functional unit is 61 ^Upcaied a dedicated portion of register tile 1158 - In a, lexn one embodimen t, register file 1158 Is divided between difWent                    bygrapbies multiprocessor 1134,

[0202] Is at least one embodiment, GPQPtd cores 1102 can each include floating point twte 0 ul x) and / or integer arithmetic logic units (AL Us) that are used to execute instructions of graphics multiprocessor 1134. GPGPU cores 1162 can be similar in archuecture or can differ in arch; memo. in ai least one enmwlm'eno a lire; portion ,<uPGIH ’ cores 1 b'U include a single precision FPU and an integer ALU while a. second portion of GPGPU lores include a doo de precision FPU. In at least one embodiment, FPU's can implement foEIi 754-2003 slandu'd uv floaimg point a?hmnetie or enable variable precision floating point arithmetic in at least one embodiment, gmphies multiprocessor 1134 can additionally indude one or mure fixed function or special function unus to perform specific functions such as copy rectangle nr pixel blending opeummw In at least one embodiment one or more of GPGPU cores can also include fixed or special function logic,

[0203] In at. least one embodiment GPGPU»urns 11(>2 include SIMD logic capable of performing: a single instruction on mul tipi e sets of data. In al least one embodiment GPGPU cotes I.102 can physically exfo^ S1MD8, and SIMUfo instructions and logically execute SIMDI, 8IMD2. atfo SMDSfoi^ la at least one embodiment, instructions for GPGPU cures can be generated at compile time by a shader compiler or automatical I y generated when executing programs written and compiled for single program multipl e data (SPMD) or SIMT architectures. In at least one embodiment, multiple threads of a program configured for an §IMT execufi on model can executed via a single SWD instruction, In at least we embodiment, in at least one embodiment, eight fliMT threads that; perform same or similar operations can be executed m parallel via a single SIM US logic unit. [ 0204] I n at least on e embodiment, m emory and cache intercuimect 1158 i s an i ntsrcormect network that connects each functional unit of graphics multiprocessor 1134 to register file 1158 and to shared memory 1170. In at least one embodiment, memory and cache imeremmect I IPS is a crossbar interconnect that allows foad / store unit 1160 to implernem load and store operations between shared memory 1170 and register file 1158 in at least one embodiment,, register file 11.58 can operate at a               as GPGPU cores 1162, thus data transfer between GPGPU cores 1162 and register file 1158 is very- low latency. In at least one embodimenL shared memory 1170 can be used to enable eum muni call an between threads that execute on functional within Graphics maitipfocesscr 1134, In at least one embodiment, cache ^sjnory 1172 can be used as ;a data cache for example, to whe w       communicafed between functional Uu lx and texture unit I136. In at feast oire embodiment, shared memoW 1170 can also be used as a program managed cache. In at least one embodiment, threads executing on GPGPU edres I ioO can programmaiiegdy store data within shared memory in addition to automatically cached data that is stored within cache memory 1172

[0205] in at least one embodiment, a parallel processor or GPGPU as described herein is communicatively coupled to hbst / processor cores to accelerate graphics operations, machine-learning operations, partem analysis operations and various general purpose GPU (GPGPU} functions. In at least one embodiment. GPU may be communicatively coupled to host pn lessor A orex over a bus or other interconnect te g., a high speed interconnect such ax PCIe or NVLink). In at least one embodiment, GPU may be .integrated bn same package or chip as cores and communicatively coupled to cores over an internal processor bus / mterconnect (in at least one embodiment, imerual to package or chip) In at least one embodiment. regardless of warmer in which GPU is connected, processor cores may allocate work to GPl nt form of sequences of sojnma.ndsdnstructio.us cpntamedln a work descriptor, hi at least o% e'nbvJn'ieu\ GP)' t uu uses dedicated clreuitsydogle for efficiently processing these sormnands / instwetions.

[0206] Inference and / or training fog; c n Is are used to perform inforenci ng and / or framing operations associated with one or mors embodiments. Details regarding inference and / or training logic 615 are provided below m conjunction with FIGS. SB and / or 6G In at least one. embodiment inference author training feme bla rney be used in graphics mfetipOT     1134 for inferencmg or predicting operations based, at least in pad, on weight parameters calculated using neural network training operations, neural netwosk functions^and / or architectures. or deurMI network use eases described herein. fofofo MG 2A dhsstraws a multi-GPl e-mmramg extern 1200 X ;Kcredim> pres was-one embodiment In ai least one embodiment, multi -GPI' computing system 1200A can include a ■veeex'wi 12u2 tuupled to nndmn? u&wia. pmp.wc rep huw 'Vfvv'-sn.g urnlx {GP< jH \u 1 fee \-D via a host imertace switch 1204. 1» m least vuo embodiment., host interface switch 120-1 is a PO express switch device that couples processor 1202 to a P('l express bus over which processor 1202 cab communicate with GPGPUs i 20oA -1) GPGPUs 1206A-D interconnect via a sei of high-speed point re pram GPU io GPU hmra 12 Io In at least one embodiment. GPU to GPU 1216 connect to each of GPGPUs 1206A-D via a dedicated GPU link. In at least osa embodhneM P2P Gto Into 1216 enable direct coimm^             each ofGPGPUs 1206A~D without requiring fommimicatirm over host interface bus 1264 to which processor 1202 is connected, bi at least due embadiwtu with GFUau-GFU traffic directed to P2P GPU Imks 12h\ hoM hues Gee bus I 2’G semams avaPable fo; ss^lem mweow <u'wvit ;o comnrmkvue noh uber instances of umlti-GPU computing system 1200 / 4, for example, via one of more ntwl dm ices While in at least one embodiment GPGPU's 12064-0 connect m processor 1202 via host interface switch. 120¾ m at least one embed!mem processor 1202 includes direct support for P2 P GPU links 1216 and can connect directly to GPGPUs 1206A-D, [0208| Inference and / or trammg logic 615 are used to perform htferencmg and / or training operations associated with one or more embodiments. Details regarding inference and / dr training logic 615 are provided: below in epnjunction with FIGS 6B and / or 6C. In at least one embodiment, inference and / or naming logic 615 may he used in muki-GPU computing system .1200A for isfe'endug or predicting           based, at least m part, on weight parameters Calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases: described herein, RG 12B is.a block diagram of a, graphics processor 1200B; according to at least ode embudimem, :In at least one embodiment, graphics processor 1200B includes a ring interconnect 1202, a pipeGhe to     1204, a media c >~me 125”, and graphics cores l2hOA-12WN. In at least one embodiment, ring iiWcomfoct 2c2 rouples graphics processor 1200B to other processing units, iuglu^mg other graphics processors or one or more genera I-purpose processor cores: In at least: one embodiment, graphics processor 12O0B is one of many processors mtcgmmd within a-multi-core processing system. [to :1| In m least dtmembodim^^          processor T200B receives batches of commands via ring mtereormect 1202, In at least one embodimem, iacoming commands are imerpreted by a command streamer 1203 in pipeline front-end 1204 hi at least one embodimem, graphics nn'cev'-.m I 2uuB unhides scalable execution logic to perform 31) geometry processing and unxliu moeewfog \m graphics coreis? I28OA-128ON in at least one embodimem. for 3D geometry processing commands, command streamer 1203 suppiies commands to geometry pipeline 1236, In at least one embodiment, for at least some media processing commands, <.umm m<i oi,-wnm 120' a-imhes K,-mei mda to <i Gito non ere 12^4, wKen cenplt * wuh a media 1237, In at least one             media engine 1237 includes a Video Quality Engine (VQE11230 for video and image post-processing arfo a mfom enoode / decode (MIX; 1233 engine to ptwide Iwdware-aco lowed media data encode and decode, 1'n at least one embodiment, geometry pipeline 1236 and media engine 1237 each generate execution threads for thread execution resources provided by at. least one graphics cure 128GA, i 021 1 ] In at least one embodiment., graphics processor 1200B includes scalable thread execution resources featuring modtte cores 128OA^^              referred to as core slices), each imdng mul^                        1260A-1260N (sometimes .referred tons core suh-Uav^i In al least one embodiment, graphics processor 1200B can have any number of graphics cores 1280A through J280N. In at least one embodiment, graphics processor 120QB includes a graphics core 128OA having at least a first sub-core 125OA and a second sub-core 1200 X In m leas? <>re embed mem, gmj Ines mecew! I FW u i te >o",m poKcsm wh ,( single sub-core (e.g, 1250A) in al mast we endmd'mem, gimdncs processor 1200B includes multiple graphics cores 1280A-128«IN, each md-ming a $<l of lust sub-cores 1250A-12SGN and a set of second sub-cures I260A-12G0N, In al least one embodiment, each sub-core in first sub-cores 1250A-I250N includes at least a first sei of execution units. I252A-1252N and nmdiaAexture w ,1254 A-1254N. lot least pn& enfoodimw^              second Mib- cores 1260A- I.260A includes at least a second set of execution units 1262A-12.62A and samplers 1263 A-te i\; Imat least mo                       I250A 1250% I260A-I260H shares a: set of shared resources 1SlbA-WOK In at least one embodiment slmred r^^^ shared: carte         arid pixel opetafiou fogi c.

[0212] Inference wd / or traifonglogimfo.fi used to perform iptewing and / or training operations associatedwith one or more embodimenis. Details regarding inference midfor tmihmg logic 615 are provided below in conjunction with FIGS, 6B and / or M', In at least one embodiments inference andfor training logic 615 may betted in graphics processor 12G0B for Inferenclny, or predtetmu openm-ns based, at test m part, on uenihi yaramwew calculated using neural network training operations, neural network ftmctiuns and / or architectm'es, or neural network use cases described herem,

[0213] FIG. 13: is a block diagram illustrating micro-architecture for a processor 1300 that may include logic circuits to perform instructions., according to at least one embodiment. In at least one errtedimem. processor 130\)m.ay perform insiructions, inchidirm instiucdons, ARM 65 hWractkw, specialised           for application-^pacific integrated urouu\ t \SKkk etc In at least one embodiment. processor 13<Xlmay include registers to suw packed data, sash as64~ bit xsidte registers in tnieropfocessdrs tabled wh MMX technology from Intel Corporation of Santa Clara. Calif In at leasf ohfo^           MMX registers, available in both integer and floating point forms, may operate with packed data elements that arc.-moany single 't^ua mm. mu tip c data f SlM5)"i and spa arena MM1) extensions f 'sSI ' » mtenenerw I" <0 .as; v ne embodiment 12Xbn wide XMM registers relating to SSE2. SSF3. $SE4. AYS’. or beyond (referred to generically as CSSEx" i technology may hold such packed data operands.- In at least'one embodiment, processor 1300 may perform iustmcifoas to accelerate maehfoe learning or deep learning algorithms, training..or iaBtaidng.

[0214] In at least one embodiment, processor 1300 includes an in-order front end flrbnt end") 1301 to fetch instructions to be executed and prepare instructions to be used later in processor pipeline. In at least one embodiment, from end 1301 may include several units. In at least one embodiment, an instruction prefMehvr I Co fetches instructsone from memmy and feeds instructions to an instruction decoder 1523 Much in turn decodes or interprets instructions. In at least, one embodiment, in at ken. xmbod-ment, msmmtion decoder 1323 decodes a received mswetiun Into one or mote operations.called “mfeiw-iustructiwy ’ or “micro-operations” (also edited “micro ops”er“uops'') that machine may exm me In at least one enfoodimtmt instruction, decoder 1328 parses Instruction into an opcode and .ameypopdfog data, and control fields that may be used by micro-architecture to perform operations in accordance -with at least one enibudhwii. In least onmembodinteht, a irate cache 1330 may assemble decoded imps Imo program ordered sequencesfo traces in a pop-^uwe 1334 for execution. In at least one embodiment, when trace cache 1330 encounters a complex instruction, a microcode ROM 1332 provides oops needed to complete operation.

[0215] In at least one embodiment, some instructions may be converted Imo a singl e mlcr0"0p, whereas others need several rnlerorops to complete full operation. In at least one embodimenl; if more than four micro-ops are needed to complete an msuuction, instruction decoder 1328 may access microcode ROM 1332 io perform instruction- la ,u least one embodiment, an instruction may be decoded into a small number of micro-bps for processing at instruction decoder I32R in at least one embodiment, an insmsetion may be stored within microcode R0M 1332 should a number of micro-ops be needed to accomplish operation. In at least one embodiment, trace cache 1330 refers io an entry point programmable lopic asray (“FLA?’) to determine a correct micro-instruction pointer for reading, microcode sequences to complete one or more: instructions from microcode ROM 1332 m accordance with at least one embodiment. In at least one embodiment, after microcode ROM 1332 finishes sequencing miefo-aps for all instruction, front end fofo cl machine may winy. fetching niko-;^ fiom to.ee cfow I foa pftfoj In st least one embodiment, omxfoorder execution engine ("out of order engine") 1 fol may prepare instructions for execution. In nt least one embodiment, ouLof-order execution logic has a number of buffers to smooth out and re-order flow of instructions to optimize performance as they go down pipeline and get scheduled for ex ecu non In at least one embodimem.. out-of-order execution engine 1303 includes, without limitation, an allocator register renamer 1340. a memory uop queue 1342, an inte          point won queue I 34«, a memory scheduler 1.346. a fest scheduler 1302, a siowftgmeral floating point scheduler i foofogenerm IIP scheduler”) 1304, and a simple floating point scheduler ("simple FP scheduler") i 306 in al feast one embodiment, last schedule 1302, slow,general floating point scheduler i 304, and simple floating point scheduler 1306 are also collectively referred to herein as "imp schedulers 1302. 1304, 1306.' in at feast one embodiment, allocator / register renamer 1340 allucaics machine butlers and resoumes that each wop needs i n order teexeeate; In at lea st one umbodi meat, tdfocatotfregister reeawr 1340 renames logic registers onto entries m a. register file. In Wlfost <me embodiment, allocator,also allocates an entry for wh-W in one oftwo uop queues,, memory uopqueue 1342 formemory operations and mtegsr / fleating point imp queue 1344 for nommempry operations, in front of memory schedule) 1346 and wop schedulers 1302, 1304, 1306, In at least one embodiment, wop schedulers 1302,130¾ 1306 determine when a uop is ready to execute based w reafe   of their dependent input register operand sources and.: availability of execmicm resources maps need to compfete their operation. In at feast one t mboemwsn m.st tobed.de> 13>?2 «4 ai Icust one onmcdm-wni may schedule on each half of main ch<\ c\de u -vie <ow gcnei J floamm yomi \cheem vf I hi l and simple floating point scheduler I3oo may schedule once per main processor dock cycle. In at least one embodiment, imp sche.hu,as I >02 I 30-* kfon mlhume fm dispaieh pm w to schedule neps Un eseciUbm

[0217] In al least one embodiment, execution block 13 H includes, withc^t l-mitatiou, an ■ nleger register file / by pass network J 308, a floating point register file / bypass itotoofl. ("f P register file-bypass ueiwom") * 3 10, address generation mills (‘ AGUs' ) 1312 and 13 Ifo fast Atuhmetfe Louie I mts (ALUs)Uihst AM < ) 15loand 13 IS, a slow Arithmetic Logic Unit (eWw ALIF): 1320, a floating piaim ALU UFF)) 1323. and a floating point move unit UFP move’’) 1324,: In at ito ©nd embodiment, integer register fee / bypass network 1308 and floating point register Ole / bypWs h^twork 1310 are also referred to herein as ^register fees 1308, 1310,” In at least one embodiment, AGUs 1312 and 1314, fast ALUs 1310 and 1318, slow ALU 1320, uo.ni PI , LU are fexit-ng porn e " e r / ,r 11?a me <sfe'seft iii e L' hou,' a» “execution units 1312,. 1314, 1316, 1318, 1320,1322, and 1324.” In at least one embodiment, execution block b 11 may include, wt thou t li m had on, any n u m ber (i nd udmg zero) and type of register fees, bypass networks, -address generation wits and exeeution uni ts, in any: combi nation. p»'' s] bi at least one embodiment, register files 1308,13 IQ may be arranged between cop schedulers 1302,1304, 1306, and execution wits 1312, 1314, 1310,1318, 1320,1322, and 1324. In at I east we embedim ent,:integer register ill e / bypass network 1308 perl 1 u ms i meger operations. In at feast one embodiment, floating point register feefeypass network 1310 performs floating point ’'peram-ns in at kaM ends-Jmient. each of remoter fifes 13<fo 1 / 10 may include, without          a bypass network that may bypass or forward just completed results that have mm yet been, written into register .fife fette             tiOpL In. at least one embodiment register flics Wfe 1310 may cmtmnmicate data with each other. In at least one embpdiment, integer register feebypass network. 1308 may include, without limitation, two separate register fees, we register fee luw-ofeer thirty-two bits uffeata .a^ ;cgsM«r fee for high order thirty dwd bits of data. In. at least one embodiment, floating point ugiMet foe / bypass hetwork 131:0 wy inefe^ without limitation, 128^ wide entries because ibemug point instructions typically have operands from 04 to: 128: hi Is in width. (02W| In atleasfew                                 1314, 1316, 1318, 1320, 1322. 1324 may execute imttudtkm. id at feast erne embodi^ register files 1308, 1310 store integer add fusating point data operand valuts that snicro-instatetions nevd to execute. In at least one embodimern. processor 1300 may include, without liraltatlon, any number and combination of execution units 1312. 1314. 1310, 1318. i 320 1122. 1324 In ai Ie cu one embodimem, ffomm” polm. ALU i 323 and Ifoarmg point move unit 1324, may execute floating point, MMX, SIMD, A\ X and SSL, or other upwciirmo, Im / u-dutg specialized machine teaming insirectums. In at least tele embediment, floating point ALU 1321 may mdmte, without limitation, adh-bubv 64» hit floating pouu du kk-r to execute divide, square root and reminder micro ops in at least one embodiment, imamctiom hmtW           point value may be handled with Hoaung point hardware. In at least one embodiment, ALU operations may be passed to fast XI .1 s ; Ho 1318 In at least one Unbodiment, fast ALUS 13 IS, 1318 mav execute feat operations with, an effective latency of half a duck cycle In al least one embodiment, most complex integer operations go to slow ALU 1320 as slow ALU 1320 may include, without limitation, integer execution .hardware for long-latency u-pc 11 operations, such as a multiplier, shifts, tUg logic, and branch processing. In at leas! one embodiment, memory load / Aoreoperations may be executed bv AGUS 1312. 1314. .In at least one embodiment, fast ALU 1316, fest ALU 1318, and slow ALU 1326 may pedorm um-grt m.wrafoms on (U-bb data operands In .it Umst mm embodiment. fast Ai *' U to. last ALL 13 IS, and slow ALU 1320 may be implemented to support a variety of data bit sves Including sixteen, thirty-two, 128, 256. eu. in at least one embodiment, floating point ALU 1322 tud Homme p<rm nn-xr met I C4 max be u'sphmemed 6- support a muyt cu I'petamh Um mg hits oi various ninths In at lea-* one embodiment footwg point XI 1’ I >22 aid tfommg point move unit 1324 may operam              packed data operands in conjunction with SIMO and md amedi a instruct!on s.

[0220] & at least ous.embodiment, uop scheduler 1302,,1364. 1300, dispatch dependent opem.fidPa before parent load has finished executing; In at least one embodiment, as npps may be speculatively scheduled and executed in processor 1300, processor 1300 may also include logic to handle memory misses. In arleast one embodiment, If a data load misses m data cache, there may be dependent opet'arious In flight in pipeline that have left scheduler xwth temporarily ipeomect data. In at least one embodiment, a replay mechanism tracks and re-executes instructions that use jneoneet data. In at least turn embodiment, dependent operations might need to bcayplayedand independent ones may beollowed to complete; In at least one tnibo0inw-n{ schodnic-is and mpUx medku-lsm of m least um; embodiment of a processor may also be dsommed to euich mxroucrion sequence* ror text s-ring comparison aerations.

[0221] In at least one embodiment registers may refer to omboard processor storage locations that may be used as part of instructions to identity operands In at least one embodiment, registers may be those that may be- usable from omside of processor tfrum a programmers perspective) In at least one embodiment, registers might not be: limited to a particular .type of circuit. Rather, In at least one embodiment, a register may store data, provide data, and perform functions described herein. In at least one embodlinent, registers described herein m;w be implemented -by circuitry within a processor using any number of different techniques, such as ruxhwrted physical registers, dynamically allocated physical registers using register renaming, combinations of dedicated and dynamically allocated physical registers, etc. In at least one embodiment- integer registers store 32-bit integer data, A register file of at least one embodiment also contains eight multimedia SIA® registers for packed data.

[0222] inference and / or training logic 615 are med to perform tofereno■ -- mdfor training operations associated -with one or more embodiments. Details regarding tafot exe and / or training.logic 615 are pros idol below in conjunction with FIGS. 68 and / or 6C. In at least one embodiment portions or ail of mfemtme-and / or training logic 615 may be incorporated into execution block 1311 and other memory or registers shown or not shown. In at least one embodiment tn at least one embodiment, training and / or inference^ techniques described herein may use one or more of ALUs illustmted in execution block 1311 Moreover, weight parameters may be stored in on-chip or ofUchip memory and / or regise is <s mun <» not ,»nc‘Aa) ♦bar configure ALUs of execution block 1311 to perform one oi mu.e maehme Icanmg algoiahms,, neural network -uchitectums, use eases, or training techniques w>' refers 10223] FIG 14 illustrates a deep teaming applxatfon processor 1400, according to at least one embodiment In at least one embodiment, deep leer m ng application processor 14()() uses: instructions that, if executed by deep learning application processor MOO, cause deep learning application processor 1400 ip perform some or all of ptcce&ses and techniques described throughout this disclosure In at least one embodiment, deep learning application processor 1400 is an: application-specific integrated, circuit (ASIC), In at least one embodiment, application processor 1400 pes terms matrix muhiply operations either "'hard-wired" into hardware as a result of performing one or more instructions or both. In at least one embodiment, deep learning application processor 1400 includes, without limitation, processing dusters 14 J 0( 1 )4410(12), Inter- Ch ip I mis r IC'l s’ ) i-lfo 1)-1426(12), Inter-Chsp Controllers ('ICCs ) 1430(1)- LUOfe), menusn controller t’hk-m Ctrlrs”) 1442(1)-1442(4). high bandwidth memory physical layer t HUM PIIY"t Mac; J. 1444(4s a n ammernmn-omtiHile' ceunal rnmnwuru < ml (^managemcnKuhtrullerCPC") i -150. a Serial Peripheral Interface mtes -Integrated Cfomii., and Gener.fi Purpose Input Output block f'SPL 12C, GPIO"), a peripheral component interconnect express eowdW wl direct memory access block f*FOe Controller and DMA" ? 1470, and a si xteendane peripheral eomponentrntereM          port CPCI Express x 16'j 14M

[0224] In at least opeembodiwah processing 1.410 may perform deep foaming operations, including inference or prediction operations based on weight parameters calculated one or amre uni ring techniques, including those described herein In at least one embodiment, each processing duster 14ri1 rmg mcGde nidiom li-mutecn. mo numhe- and type of ? 00^015 vn m least one u">Xxhr:nL ccep ra mi appKanv .nocc'AOi > Co include any number and type of process! ng clusters 1400. In at least one embodiment, Inter-Chip Linhs 1420 are bi-dhectional In at least one embodiment. hirer-Chip Links 1420 and Inter-Chip 0 omrollers 1430 enable multiple deep learning application processors 1400 to exchange information, including activation information resulting from performing one or more machine learning algorithms: embodied In one or more neural networks. In at least one embodimerit, deep learning application processor 1400 may include any number (including zerol and typed ICLs 1420 and ICCs 1430-.

[0225] In at least oneembWimerfo HBM2s 1440 provide a total of 32 Gigabytes ^GB) <tf memory. HBM2 1440(i) i s associ ated with both memory con fool I er 1442(1) an d I IB M I ’ll Y 1444(11 hr at least one embodiment,    number of HBM2s 1440 may provide any type and total amount of high bandwidth memory and may be associated with any number (including zero) and ty pe of memory controllers 1442 andHBM FHYs 1444, In at least one embodiment.: SPL 12C. GFiG 1460, PCIe Controller and DMA 147¾ arfofor PCB1480 may be replaced with any number and type of bfocks that          number and type of cemnmmcatfou^^^ any technically feasible fashion.

[0220] Inference and / or training logic o 15 are used to perform inferenelng and / or training operations associated with one or more embodiments:. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 6B and / or 6G. In. at least one embodiment, deep learning application processor 1400 is used to train a machine learning model, such as a neural network, to predict or infer information provided to deep learning application processor .I4«J0. in at feast one embodiment, deep learning application processor 1400 is used to infer or predict information based on a trained machi ne learning mood te.g., neural neMo.rk) that has been trained by another processor or sy stem or by deep les rm ng appli cation processor .:1400. In at least one embodiment, processor 1400 may be used to perform one or more neural network use eases described here! rr. u 2'] FIG. 1 S is a block diagram of a neuromurphic proceasgr 1500. according to at least one enbuo inwit In at least one'emta newomorphic^              may receive: one or more inputs from sources external to neuromorphic processor i 500. In at least one embodiment, these inputs may be transmitted to one or more Fearons 1502 within neuromorphic processor 1500. In at least one embudimem, neurons 1502 and components thereof may In- nuaomwied using circuitry or logic, 'including one or snore arithmetic logic units (.ALL"G. In at lew one embodiment, neuromorphic processor 1500 .may include, without limitation, thrmsods.ar millions of instances of neurons 1502. but any sui tabi e number of neurons 1502 may be used. In at least one embodiment, each instance of neuron i 502 may include a neuron input 1504 and a neuron output 1506, In at least one embodiment, neurons 1502 may generate outputs that may be transmitted Io inputs of other instances of neurons 1502. In at least one embodimem. in at least one embodiment, neuron Inputs 1504 and neuron outputs: 1506 may be interconnected via. synapses i508, to228| In at least one embodiment neurons 1502 and synapses 1308 may be imercunnected such that neuromarphic processor 1500 operates to process or analyze information received by neuromorphk          I su0 I mat I east ano            neurons 1502 may transmit an. output pulse for “fife’ > * \txkc'} K-n inputs received through neuron input 1504 exceed a threshold, in at lea? one end o<cmem. mmimn ,5-'2 mm s.nn <r .morale wm; s uteacd m ueeum np t m 1504, In at least one embodiment in at kas:t aim embodiment, neurons 1502 may be implemented as leaky integratemed-fire: neurons, wherein if a sum (referred to as a “membrane potemlar s exceeds a threshold value, neuron i 502 may generate art output for “fre:') using a transfer function such as a sigmoid or threshold function. In at least one embodiment, a,leaky in.tegraferand4ire neuroft may sum signals received m neuron inputs 1504 into ampmbme potential and may also apply a decay factor (or leak)to reduce a membrane potential. In at least one embodiment, a leaky mtegrateomd-fhe neuron may fire if multiple input signals are received at neuron inputs i 504 rapidly enough to exceed a threshold value (in at least one ernbudimem. this is before a membrane potential decays uro low io fire). In at least out embodiment, neurons 1502 may be implemented using circuits or logic that receive inputs, integrate inputs into a nron u ,nt pom.md and ,ke-o a mend u ro p<rem',\ to d boo .»j >.< <mbvdir ott ,nput\ ' nu be averaged, or wy other suitable transfe? function mm be used Furthermore, in at least one embodimem, nw may inelude, vxnh^t lundaunn, com paratar circuits or logic that generate an oinput : spike: at neuron output I Mu when result ol applying a W tteuroh input 1584 exceeds a threshold. In at least one embodiment, feme neuron 1:502 fires, it mav disregard pres leavy rvceix cd input tot m unition bs.. I os example reseilmg a memhs tu ie potential to 0 or another suitable default value. In at I east one embodiment, once membrane potential 1$ reset toO, neuron 1502 may resume normal operation after a suitable period of time (or refractory period). i! ‘22^ | hi >u k-ao embodiment neurons 1502 may be interconnected through synapses I Ms in at k\Kt one emu-Mimeno synapses ISOS may operate to transmit signals from an output of t fust nwiou iM2 toon mput of a second neuron 1502. In at least one eatbodirnent, neurons 1502 may transmitinformation evermore than one instance of synapse 1508. In at least one embodiment, >nv os snore instances of -neuron output 1506 may be connected, via an instance of synapse .1508, to m instance, of neuron input 1504 tn same neuron 1502. In at least one embodiment. an instance of hemt® 1502 generating an output lo be transmitted over ah instance ofsynapse 1505 mas be referred to as a “pye-^apti^neuixm’’-^^ to that instance of sy napse 150fe boat l east one »          as instance of neuron 1502 receiving §n input rrmsmittod o^          Of synapse IfOSmay he referred to as a 'feoMsynapbe neuron * with respect to that instance ofsynapse 15:08, Because an instance of neuron 1502 may receive inputs from one or more instances of synapse 1508,. and may also transmit outputs over one or more inslauees of sy nap se 15 08 , a singl e instance of neuron 1502 m ay therefore be both a "pre-sy naptic uemon” and: ’■'post-synaptic neuron,” with respect to various i nstances of synapses 1508. in at least one embodiment. [02310 In at least one embodiment,neurons 1502 may be organized into one or more layers. Each instance of'neuron 1502 may Iwm                1506 that may fan ou: t cough one or m.-iv w-rapses 1 M2 to one or nnw neunm rap Us 1 M I to at orau en- vmoodemu-.'., no or outputs 1506 of neurons 1502 m a first layer .1510 .may lmco.nnecLedt.o neuien in.puls 1504 of neurons 1502 hi a second layer 1512. In at least one embodiment., layer 1510 may be referred to as a. “feed-forward layer” In at least one embodiment, each instance of neuron 1502 in an instance of first laver 1510 mav tom out to each instance of neuron I M2 m sec-md fever IM 2 In at least one embodimenf first Igy< 1.510 may be referred to as a ■'lldly cotmected feed-lbmard layer"' fe at least one embody each instance of neuron 1502 ;n an m^'inca of second layer 1512 may fen out to, few than all instances of neuron 1502 in a third Lives 1514 In at least one embodiment, second Uyer 1*12 mas -h- ><»£ uedfeas a “sparseis omm ted reedooossol fe at least one embpdimem. neurons 1502: in second layer 1512 may fife out to neurons 1502 in multiple other fevers. inehidiug io neurons i ^02 in (same ; second layer 1512 In as least one embodiment, second lay ei 15 i 2 may be referred to as a “recurrent layer. 5' la at least one: embodiment, neuromofehic processor 1500 may include, without limitation. any suitable combinatipp of recurrent layers apd feed-ibi’ward layers, including, without limitation, both sparsely .connected feed-forward layers and felly connected feed-forward layers. [023 11 in at least, one embodiment, mmromorphic processor 1500 may include, without Inniunion a rewnfigumble interconnect architecture or dedicated hard wired interconnects to connect synapse i AA tv neurons 1502. In al kari one embodiment, neuromorphiv processor 1500 may include, wifhom lifetawu, terete try or logic that allows synapses to be allocated to diflereni neurons 1502 as needed based on neural network topology and neuron fen-in / om. fe at leastone efebodimefein atleast one embodiment synapses 1508 may be connected to neurons. I using an interconnect fabric, such as netv^           or with dedicated connections. In at least mm embodimefe synapse irriemonnectfons and           thereof m ay be implemented usihg effeuihy or logic,

[0232] FIG. MA is a block diagram of a processing system, according to at least one? embodiment, In at least one embodiment, system 1600A includes one or more processors 1602 and oueor more graphics processors:. 1608, and may be :a single processor desktop system, a umhiprecessor workstation system, or a server system having a large number of processors 1602 or processor cores 1^7, lb at least one enibMmlefe system 1600.3 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuii for use in mobile, handheld, or embedded devices.

[0233] In at least on^embsdiment, system loOOA can include, or be incorporated within a server-based gaming platform, a game console, including a game and media console, a mobile gaming console, a handheld game console, or an online game console in at least one embodiment, system 1600A is a. mobi le phone, smart phone, Mbiet e-minuting device or mobile Internet device, la at least one embodiment, process ng w mem i oGe can also include, couple veh, > <v be unegusted wuhri a vre.uabk dewee such a-, a t waich ^tasa'de dm ice smml eyewear device, augmented rcalirs device, m 'm md u-ahw dvo ice In ;n leas? one embodiment, processing system 1600A is a television or am bvx device baong one «p mow processors. 1602 and mgrapbicM mtmfe          by one or m ore graphics processors 1608.

[0234] In at least one embodiment, one or more processors 1602 each include one or more processor cores 1607 to process instructions which, when executed, perform operations for system and user software. In at least one embodiment. each of we or more processor cores 1697 is configured to process a specific instruction set 1609. In at least one embodiment, instruction set 1may facilitate Complex Instruction Set Computing (CISC),.Reduced hlsiractian Set Computing (RISC), or computing via a Very long instruction Word (VLIW}. In at least one embodiment, processor cores 1 oO? may each process adHYercoi instruction set 1609, which may include hsfiruetions to facilitate emulation of eider m.^trucnon sets. 10 at least one embodiment, moow.wi cvw |m’7 am also i -elum1 ethti pmeewme dm mvs, such a IhgPai S.enaj Pu^ewm (DSP) pC'-^ hi m h-,wi owembodwK'ut pu^essoi Ieo2 nu bide^ s ,icm nwmow lm>4 in m Da'O one embodiment, processor 0602 can have a single infernal cache or multiple levels of infernal cache, In af least one embodiment, cache memory is shared among various components of processor I60S. In ai least one embodiment, processor 1602 also uses an cache (eg,, a Level-3 (L3) cache or Last. Level Cache (LLC)) (not shovm), which may be shared.among processof cores 1607 using hnown cache coherency techniques. In at least one embodiment, register file 1606 is addition              processor 1602: which may Include different types of registers storing 61¾ of data (e,g:.s integer regi sters, boating poi n t registers, status rigiMem, and an instruction pointer register), In at least one en^                  1606 may mclnde general -purpose registers or other registers.. p,£3n| In at least One embodiment, rmeor mmg:process     1602 are coupled with one or mure imermee bus(vs) lo 10 to imnsnut >.<>mmwmamm dumais such address, datm oreomroi signals-between processor 1602 and other components in system 1600 A. In at least one embodirnem, interface bus I61u, in one embodiment, can be a processor bus such as a version of a Direct Media Interface (DMI s bus In at least one embodiment imerfacc 1010 is not Ihmu J s' a DMI bus, and may include one- or more Penphemi Component Interconnect buses (e.g, PCI, PCI Express), memory busses, or other types of interface busses. In at least one embodiment processons; Iou2 mch.ide an miegrawd mcmorv cimtrUkn Solo and t pi tlfeim wmmoiki hub 75 1636 inm kad eurin'Jutxul .aetm-tr oemJle- lob? Lictbimcx co'nuie.ncauoa beiwcxn a memory device and other components ofsystem I6Q0A,. while platform controller hub fPCH) I630 provides eowectious to:.VO devices via a focal W bos. [023 7] In at least one embodiment, memory desdes 1020 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device. Hash memory device, phase-change memory device, nr some other memory device having suitable performance to serve as process memory; la at least me embodiment memory device 1626 can operate as system memory for system 1600Agio store data 1622 and instructions 1621 for use when one or more processors 1602 executes an application nr process. In at least one embodiment, memory controller 1616 also couples with an. in at least one embodiment, external graphics processor 1612, which may communicate with one or more graphics processors 1668 m processors 1602 to perform graphics and media operations. In at least one embodiment, a display device 1611 can connect to processor* st 1602 In at least one embodimeni display des ice 1611 can include one or m ore of an internal display device, as in a mobile electronic device ora laptop device or an external display device attached via a display unerfoce (e g., Display Port, etc 1 In al least one embodiment, display device Ibl I <ao foehto^             display <RMD) such asa stereoscopic display device for            reality (VR) appli^ or augmented reality (AR) applications.

[0238] In at least one embodiments platform controller hub 1630 enables peripherals to connect to memory device 1620 and pfocessor 1602 via a high-speed I / O bus. In at least pne embrfoiments l^                   but are: not limited to, an audio controller 1646, a neiworir controller 1634, a fimwato interface 1628, a wireless transceiver 1626, touch sensors 1625. a. data storage device 1624 (e.g„ hard disk drive, Hash memory, etc.). In at least o"e embodiment, data storage: device 1624 can oonnect via a storage interface (e.g„ SATA) or via a peripheral bus, such as a Peripheral Component Intereomiect bus (e.g., PCI, PCI Express). In at least one embodiment, touch sensor 1623 can mdudv touch <%■■'. sensors, pressure sentow or llmieytom w'nxvrs In at teast one t-mhodmiem '■wide-w uanweri b<2u c-m be a WoFi transceiver. a Bluetooth transceiver; or a mobile now oik trio wet ver such as a3G, 4G, or Long term Ewlution (LTE)transceiver. In at least oneemhodunent, finnware interface 1628 enables conununication with system firmware, and can he, fot exampte, a unified extensible firmware i nterlace (I®    In at least one embodiment, nciwoil c>mtu<Her 1634 can enable a network connection to a wired network. In at least mte embodiment, a high-performance network controller (not shown) couples with interfacebus 1616. In at least one embodiment,:audio cwtroHer 1646 is a mnlti-charmel high definition: audio controller. In at least one embodiment, system IdOOA includes a legacy I / O controller fodfefor coupling legacy (PS / 2)) decker so system In at least one embodiment platform controller hub 1630 can also connect to one or more Universal Serial Bus (USB) controllers 1642 connect input devices, such <re board and mouse lo43 combinations, a camera kWh or other USB input devices.

[6239] In al least one embodiment an instance of memory controller 1516 and platform coime'Jm hub tnafotm fo'mummed mm a dwueet txtetnx emu m"- ptoewwt 'i,cb external graphics processor .1612. In at least one embodiment, platform controller hub 1630 and / or memory control ter Info may be external io one or more pmcessor(s) 1602. In at least one embodiment m at least one embodiment system food A can include an external memory controller 1616 and platform controller ho 1630, which may be configured as a memory controller hub and peri pbeml controller hub within a system chipset that is in communication with processor-a) Ie02.

[6246] Inference and / or trai mog: I ogle 613 are used to perform. referencing; and / or training, operations. associated with one or more embodiments:. Details regatdmg inference and / or training logic 615 are provided; below in comunution wife. FIGS: 6B and / or 6C, In at least one embodiment portions or all of inference and / or traini ng logic 613 may be incorporated: info graphics processor 1600A. In at least one embodiment, in at least one embpdimeut, training and / or infereuemg iechm^ described herein may use one er more of ALUs embodied In graphics processor 1612. Moreover, in at least one embodiment, inference ng and / or training operations described herein may be done using logic other tban logic illuwrated in FIGS, 6B or 6C: In at least one embodiment, weight parameters may be stored, in on~chip or ut¥-chip memory and / or registers (shown or not: shown) that configure ALUs of graphics processor WA'W acifom one erm maenme trarmr.n algor th w n-mol network me In lecture-;, use caws, or training techniques described herein. 1’0241 ] FIG 16B is a block diagram of a processor 10G0B having one or more processor cores 1602A»1602N, an integratfo memory controller 1614, and an integrened graphics processor 1668, according to at least one embodiment In at least one embodiment, processor 1600B can include additional cores tip to and including additional core I602N represented by dashed Imed: 77 boxes. In at least ope etifoudi mend each Of processor cores 1602-VI602N includes one er more Eternal cache units 16Q4A:-I604fo In at least one embodiment, each processor core also has access to One or more shared cabbed u: k ' 61M

[0242] In at least one embodiment, internal cache units 1604A- I604N and shared cache units I606 represents cache memory hierarchy withm processor ibrioB In at least one embodiment, cache memory units 16G4A-1604H may i nclude at least one level of instruction and data cache within each processor core and: one or more levels ofshared mid-level cache, such as a Level 2 s 1,.2), Level 5 (131 Level 4 (L4X or other levels ofcache, where a highest level of Cache before external memory is classified as an 1..1..(). in at least one embodinrent cache coherency logic maimains coherency between various cache units IbOb and .lb04A~1604N.

[0243] la at least one u" moment, processor I60QB may also include a set of one or more bus control lor u ni ts fo l d at d a wstem agent core 1610:. In at least one embodiment, one or more bus controller units lulu manage a sei of peripheral bases, such as one or more PCI or PCI express busses. In at least one embodiment, system agent core 1610 provides management functionality forwfokg!S processor w          la at least one embodiment, system agent core 1616 includes on©:or more integrated memory controllers 1614 to manage access to various external memory devices (not shown).

[0244] In at least one Wbfolimeht, one or snore of processor cores M02 A-1602N include-support for simultaneous muMAhreadh^. Id ar least one embodiment system agent core 16 IQ i ncludes eomponents for coordinating and opemiregccws 1602A-160261 during multi-threaded processing In at least one emlxn?ntent w stem agent core 1610 fogy additionally include a power control unit (PCU), which includes logic and components to regulate one or more power states of processor cores 1602A-I6G2N and graphics processor 1608.

[0245] In at least one embodiment, processor 16O0B additionally includes graphics processor 1608 to execute graphics processing operations. In at least one -embodiment, graphics processor 1608 couples with shared cache unitv IbOb. and s\ sum agent core 1610, including one or more-integrated memory controllers 1014. In m least one embodiment, system agent core 1610 also includes a display controller 1611 to drive graphics processor output to one or more coupled displays. In at least one embodiment / display controller 1611 may also be a separate module coupled with graphics processor 1608 Maat least one Interconnect, or may be integrated within graphics processor .1608. [024$] In at least one             a rmg based rnlarcphitoct unit I612 is used to couple internal components: of processor 1600B. to at least eneem bodiment, an alteumiive / intete^ unit may be used, such as a pmnt-tteputut inttewnnect, a switched interconneet- : or other teclwqites, to at least tote embodiment.. graphics processor 1608 couples with rise interconitect 1612 via an I / O Itoh 1613.

[0247] In at least one embodiment, I / O hnk lol 3 represents at loess, one of multiple varieties oi I / O interconnects, i ncl uding an on package I / O interconnect which facilitates communication between various processor components and a high-performance embedded memory module 161 $, such as an eDRAM morh.de In a? least one embodiment. each of processor cores lo02A-1602N and graphics processor 1608 use embedded memory modules '1618 as a shared Last Level Cache.

[0248] hr m least one embodiment, processor cores 1602A4602N are homogenous cores executing a common instruction set architecture, to at least one embodiment., processor cores 1O02A-I602N are heterogeneous in terms ofinstiuctiort set architecture (1$A), where one or more of processor cores 16Q2A-1602M execute a. common instruction set, while one or more other cores of processor cores 16S2A-16fo2N executes a subset of a eomntou instruction set or a diftorentjnstmetipn set. In ar least one embodiment, processor cores 16Q2A-I602N are heterogcneouym terms of mkroarchiteciute, where: one or more cores having a relatively higher power eonsumpiioh couple with one or morepower cores having a lower power consumption. In at least one smbodlnmnt, processor IWOB can be implemented on one or more chips or as an SoC Integrated circuit. [$249] Inference andfor Whihgdp^s 615 are used to perform mferoudng and / or training operations associated with one or more embodiments. Details regard!ng:inference and / or training l ogic 615 ore ptwlded below in cotpanction with FIGS. 6B and / or 6G In at least one embodiment pop ions or all of inference and / br training logic 615 may be incorporated, into processor 1600B. In at least one embadiment, in at least one embodiment, training and / or inferendhg techniques described herein may use one or more of ALUs embodied In graphics processor 1612. graphics core!st J602AG602N. or other components in FIG 16. Moreover, in at least ©ne embodiment, infemicing and o? training operafions described herein may be done using logic other than logic illustrated in b IGS 6B or 6C. In at least one embodiment, weight parameteis may be stored in on-chip or oil-chip memory and or register? (.diown or not showr© 79 that configure ALUs pf graphics processor 1600B to perform one or more machine learning algorithms, .neural netwwk architectures, use cases, or training wh^                heretn.

[0250] FIG, I OC ia a block di agram of hardware logic of a graphics p rocessor core 1WOC, according to at least one embodiment described herein In at loast one embodiment, graphics processor cere loOOC is mduded within a graphics core array. Ln at least one embodiment, graphics processor core 1(>O<X.\ sometimes referred to as a core slice, can be one or multiple graphic cores within a modular graphics processor. In at least w embodiment, graphics processor eox^ 160OC is exemplary- of one graphics core slice, and a graphics processor as described herein may include multiple graphics core slices based on target power and performance envelopes. In at least me embodiment, each-graphics core 1600C can include a fixed Amotion block 1630 coupled wh multiple sob~c^^            IF, also referred to as sub- slices, that include modular blocks of general-purpose and fixed function logic! (625 i | In at least one embodiment, fixed function block 1630 includes a geosnetry / fixed function pipeline I63o that can be shared by ail sub-cores in graphics processor 1666(1, for example m lower performance mt d / or lower pmyer graph ics processor w pl ^         In at least we fembodimem, geumetry / fixed fimufo pipeline 1636 includes a 3D fixed fimotlon pipeline, a video fimnt-exid trait, a thread spawuer and thread dispatcher, and a unified return, buffer manager,, which, manages unified return buffers, (0252] In afleast one embodiment, fixed, fimctlcm block 1630 also: includes a graphics SoC i nterface 1637, a graphi cs microcontroller’ 1638, and a media pipeli ne 1639. In at least one embodiment fiard, graphics SoC interface 1637 provides an iutermce between graphics core 1600C and other processor cores within a stent on a chip integrat’d circuit In at least one embodiment, graphics mierwooW 63.8 is a programmable sub-processor that is configumbie to maoage various fimetions of graphics processor 160OG, Including thread dispatch, scheduling, and pre-emptiom in at least one embodiment, media pipeline 1639 includes logic to facilitate,decoding, encoding, pie-precessing, and or pew-processing of multimedia dram, including image and video data. In at least one embodiment media pipeline 1630 implements media operations? via requests to compute or sampling logic within sab-cores 16ODI601F. (0253} In at least ow t-einodn-w-m, SeC mtenacu Iu3 7 emraGs graphics core 1600? to cwuminkatv " Im gcevral mu spew ram urahon pu ^es^'s uotes tv g 4 'Olfisi andOr other 60 components whhin an So<\ including memory hierarchy dements such as a shared last level cache memory, system RAM, and / or embedded on-chip or w-package DRAM. In at least one embodlmeut, SoCmmrface 1637 can also enable reran mmication with fixed function devices within an SoC, such as. Camera Imaging pipelines, and enables use of Sd / or implements global memory atomics that may be shared between graphics core 1600C and CPUs within an SoC In at least one embodiment, SoCInterlace lo37 can also implement power managemem controls for graphics core 1 600C und enable an interlace between a clock domain of graphic core l 000C and other clock,domains within an SoC. hr at;cast one embodiment SoC interface o >7 enables receipt of command buffers from a command streamer and global thread dispatcher that are configured to provide commands and instructions to each of one or more graphics cores within a graphics processor hi at least one embodiment, commands and instructions can be dispatched to media pipeline 163^, when media operations are to he performed, or a geometry and fixed iuuctusi p'rdou u                                          ^M-menx and lb.ed finui.>m pipeline 1614) when graphics processing operations are to be performed

[0254] In at least one embodiment graphics rnicrocontroller 163$ can be c. ufim cd to perform various scheduling and management tasks for graphics core 1600C. in at least one embodiment, graphies mieioconrioner 163:8span perform graphics and / or compute workload waedn'mv nr various graphics parallel engines within execution amt (EU) arrays :1602.6-1602F, OwUV A' IF within sub-ceres 1601A460IE tarn least one embodiment, bust software executing on a CPU cote of an SoC including graphics core 16QQC can submit workloads: one of multiple gtaphid processor doorbells, which involos a scheduling operation on an appropriate graphics engine. In at least one embodiment, sc hrduling operations include determining which workload M ran next, submitting a workload to a command streamer, pre-empting existing workloads framing on an engine, monitoring progress of a workload, and notifying host software when a workload is complete. In at least one embodiment, graphics microcontroller 1638 can also facilitate low-power or idle states for graphics core I600C, providing graphics core 1600C oath an ability to save and restore registers within graphics core I600C across low-power state namui’Fis uidependcutlv hum an operauny nud w grauhKs m.w raftvmw >m a p.sum

[0255] In al least one embodiment, graphics core 160()(3 may have greater than or fewer than illustrated sub-cures 1601 A-160IF, up N modular sub-cores. For each set ofN sub-cores, in at least one'-embodim.en.t, graphics core 1600C can also Include shared fenedoa logic-1610. shared and / orcache memory 1612, a gemnetryW                  1614, as well as additional fixed function logie 1616 to accelerate various:         and compute processing i v i ^,1 it! \h> boo >   -. <H ee f i k’ or s o m ide cm mmie ingth, anfobr W communication logic) that can be shared by each Mfob-fow within graphics core 160OC. la at least one embodiment fixed, shared and / or cache memory b-<2. ^164 4^^1-1^.^1-.^1¾ tor \-to-w-- ,S\ V cd !F v-ikm gr ger. ec-i . Obi <mJ eon also serve as shared memoiy that is accessible by multiple sub-cores. In at least one embodiment -fixed function pipeline. 1614 can be included instead of gemuetry / fixed function pipeline 1636 within fixed nmcuou block 1630 and can include same or similarlogic uhito

[0256] In at least one embodiment graphics core 16(\K nulodes additional fixed function logic 1616 that can include various fixed function acceleration logic for use by graphics core lemur In ai least one embodimem, additional fixed fimctkm logic 1616 includes an additional geometry pipeline for use in position only shading In position-only shading m bniM two -womens w.pdmex owt wlrnmax in a foil geowj<. pipjmc withm gc-.-metn ilxeu :m er. n pipell n e 1616, I636, and a cuII pi pell ne, whi ch i s an: addi tiomd geometry pipeline wm en m ay be mdudM withlfr^                        1616. In at least one embodiment cull pipeline:is a trimmed down version of a full geometry pipeline. In at least one embodiment, a full pipeline and a null plpeime can execute di Went instances of an application, each instance having: a separate context. In at least one entbodimenfi position only shading can hide long cull runs of e xend'-d empales, enabling shading to be compiled earlier in some instances. In at least one embomm- m, n ’ te.wmme emU-dm-em tali e pehre log -\uhm wMmor / fixed fnm nor logic 1616 can execute position shaders in -parallel with a. main application and generates critical results faster than a full pipeline, as cull pipeline fetches and shades position attribute of vertices, without performing msterixmion and rendering of pixels to a frame boiler Ih at. least one embodiment, cull pipeline can use generated cnlleal results to compute 'visibility information for ail triangles whhom regard io whether those triangles are culled. In at least one embodiment, full pipdims (which in this instance may be referred to as a replay pipeline) can consume visibility information to skip culled triangles to:shade only visible triangles that arc finally passed to a rasterization phase 102« -1 In at least one embodiment, additional fixed tune lion logic 1616 can also include mai,b.ne-leaming acceleration logic, such as fixed function matrix muhiplicatkm logic, for implementations including opffmm^               learning trainm^ [025S] In at least one embodiment, within each graphics snb-ctee 16Q1A-1O1F inchdes a set of execution resources that mav be used to perform graphics, media, and compute operations In response to requests by graphics pipeline, media pipeline, or shader programs in at least, one embodiment, graphics sub-cores 1601 A-160 i F include multiple EU arrays i 602 A-1 mUl I604A-16O4F, thread dispatch and imer-thread c-Onim                 logic I6O3A- rev?F, a 3D (eg , texture} sampler 1605A-J605F, a media sampler 1606A-1606F, a shader processor foO7A- i?"i'                                 (ASF fl .ma's huWii'iCK I604A-16O4F each include multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-poi nt logic operations in service of a graphics, media, or ampule operation. including graphics. media, or compute shader programs. In at least one embodiment, 1 D-l(? logic I6O3zV16O3F performs local thread dispatch and thread control operations for execution units within a sub-core and facilitate communication between threads executing on ^waiinm units cf a sub-cure. In at least one embodiment. 3D sampler .160S A-1605F ean read tex ture or other 3D graphics related data into memory. In at least we embtfoh 3D sampler aim read texture data differently based on a configured sample: stats and texture format associated with a given texture, In at least one embodiment, media sampler 1606A-', un petuxm similar read operations based on a type anti format asMiCLHed whkmMi-a data In m le -.st oae arnbodim^ eachgraphics sab-core 1601 A-l 66^ can alternately include a unified 3D and media, sampler. In at least one embudimeuL threads execuli ng on executi on units withi n each: of sub-cores 1601 A-1601F can m ake use of shared focal memory I60M- I6O0F within each sub-core, tc enable threads executing within a thread group to execute using a common pool of on-chip memory.

[0250] Inference and / or training logic 615 are cred to perform mforencmg and / or training operations associated with one or more embodiments. Details regarding inference and or training:logic 615 are provided belowIn conjunction with FIGS. 6B and / or 6C. In at least one embr>dimeat. portion*; or all of inference and / or training logic 615 -nay be incorporated fore graphics processor 1610. in at least one embodhneni, in at least one embodiment, Irai nine umf'er inferencing techniques described-hereinmay use one or more of ALUs embodied in graphics processor 1612, graphics miorocpptr^lpr 1638, gewnem & ’ixed function pipeline 1614 and 1636, or other logic in FIG. lOB. Moreover, .sc al Last cue embodiment, inferenciug and / or training operations described teelnw using other than logic inustratedlnilijS §B nr 6C. In at least bod embodiment, typist parameters may be stored in on~ehip or off-dap memory and / or registers (shown or not shown) that configure ALUs of graphics processor 1600C to perform one or more machine learning algorithms, neural network architectures. use cases, or training techniques described herein.

[0266] FIGS. 16DG6E.iIiiMrate thread execution logic 1600D including an array of rxcsMug elements of a graphics processor core according to at least raw embodiment, FIG loD jlkhmaK-s at lea-^t one emUxhment. m winch thread exreum'U MgU IramD used vh.s IM. iilusrratcs exemplary internal details of an execution unit, according^ at least one embodiment. [<626I1 As illustrated In FIG I6D. in at least one embodiment, thread execution logic I GOOD includes a shader processor 1602, a thread dispatcher 1604, i nstruction cache 1606, a scalable execution unit array i ncl udrag a plurali ty: of execution uni is I608 A.-16WN, sampler^) 1610, a data cache 1612, and a data port 1614, In at least one embodiment a scalable execution umt a wui'tA i)H’>ek M sit n !"i'l nk>'HlNXH evtr ' ore esei mm >, n h u\ ars >4 execution umt M® A, I &Wk. 1608(  o' bp, through M®M-1 and 1608.K) bused on computati onal requirements of a workload, for example. In at least one embodiment, sealable exception imityare ImefootinW      an intercmmeei Pbrie that links ip each of execution unit In at:least one embodiment,. thread execution logic 16G0D includes one or more connections to memory , such as system memory or cache memory. through, one or more of instruction cache 1606, data port 1614, sampler 1610, and execution units tWAxISQB'N; In at least one embodinwh each exwrtian unit (mg., I6.08A)is a stand-alone programmable general-purpose conipatatipnsl unit that is capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In at least one embodiment, array of execution units ItiuSA-1 hOSN is scalable to include any number individual execution units. 10262] In at Last one embodiment execution units 1608A-1608N are primarily used to execute shader programs. In al least one embodiment, shader processor 1602 can precess various shader programs and dispatch execution threads associated with, shader programs via a thread dispatcher 1604, In at least one embodiment, thread di spateher 1604 includes logic to arbitrate thread Initiation requests from graphicsand media pipelines and instantiate requested threads m one or wore execution units mexecution units IO08A-160BN In at least one embodiment, fo m leaM o                a geometry pipeline can dispatch ' ertex, tessellation, orgeometrY shaders: to thread execution logic for processing- In at least one embodiment, thread dispatcher 1604 can also process runtime thread spawning requests from executing shader programs.

[0263] in at least one embodiment, execution units i 60S A-1 t'OS N support an instruction set that includes native support for many standard,® gmph ics shader inxtructions, such that shader programs from graphics libraries (e.g , Direct 3D and OpenGL) are executed with a minimal imnslatfou. hr at least one embodiment, execution units support vertex and geometry processing ie g.. vertex programs, geometry programs, vertex shaders}, pixel processing u g, pixel Sabers, fragment shaders) and general -purpose processing (e.g., compute and media shaders). In at least one embodiment, each of execution units 1608A-160$ N, which include one or more arithmetic logic units (ALUs). is capable of multi-issue single instruction mu tmlo «Uta (SIMDs execution-and mult;-threaded upemmm enables an efficism execution environment despite higher latency mwy accesses, In at mast one embodiment, each hardware thread wrthm each execution unit has 8 dedicated highfoimdwidfo ffle and associated mdepsndent thread-state, In at least one em bedim ent, exeemi on is m rd 6 -i sgue per cloofc to pi pel i nes yapab lb of i nt ggei; si dgl e: an d double precision boating puipt operations, SMD branch capability, fogies! operations,; transcendental: operations, and other mi seel laneous operations. In at least one embodiment, while waiting for data from memory or one of shared foneriphs, dependency logic wi ihi u execution. units I6QSA-1 SOW causes a waiting thread to sleep until requested da ta has been returned. In at least one embodiment, while a. waiting: thread,is-sleeping, hardwareresources may be devoted to processing other threads. In at least one embodiment, in at least one t-mbcKhnw-nl during a ddav assocrucd w Uli a xerwx \lmder operation, an execution unit can perform operations for a pixel shader, fragmvm shaevn another type of shader program, ineludfog a different x ertex foader.

[0264] lu at least one embodiment, each execution unit in execution units IO08A-I60BN operates on arrays of data demerits. In at least one embodiment, a number of data elements is "execution size,” or number of channels for an instruction. In at least one embodiment, an execution channel is a logical unit of execution for data element access, maski ng. and fiovi control within instructions. In nt least one embodiment. a number of channels may be independent of a number of physical Arithmetic Logic Units (ALUs) or Floating Point Units (FPUs) for a panh-ular graphics processor. hr least one embodiment, execution units 1608A-166§N support integer and flbarihg-point data types.

[0265] In al least one embodiment, an execution unit insane-ion set includes SIMD instructions In at least one embodiment. various data elements can be stored as a packed data type- in a register and an execution unit will process various elements based on data size of elements. In at least one embodiment mat least one embodiment 'when operating on a 256-bit wide vector 256 bits of a sector arc stored in a register and an execution unit operates on a vector as four separate 64-bil packed data elements i Quad AV ord (QW} size data elements), eight separate 22-blt packed Uma elements (Double Amd (DW) size data elements), si riven separate 16-bit packed data elements (Word (W) size data elements^ orthirty -two separate 8-blt data elements (byte (B) size data elements). However, in at least, one embodiment. dHTereat vector widths and register sizes are possible;

[0266] In al least one embodiment, one or mora executton uufe               mtora / feed execution unit IPOWV lbb^                  logic ( W07A-1667N) that Is common to fused BUS. to at least ode embodiment, multiple EUs can.be feed into an EU group. In Mleast one embodmient, each EU in feed El wuupean be configured to execute a separate SWD ted ware thread. Number of EUs m atused EU group can vary according to various embodiments. In at least one embodiment, various SIMD widths can be performed per-EU, including but norlimited to SIMD8, SIMDfe^ SlMD:32f In at least one- embodiment, each fused graphics execution, unit IbfeAHPOfe mdud.esat least two execctotm fe   tout leastone: embodiment. In at least one embodiment, fused e mamon unit 16’NA includes a first Bl; 16G8A, second EU I608B, and thread control logic IbO? V tout i$ common to first EU I608A and second EU 160® . In at least one embodiment, thread umbo: topic 1607A controls threads executed on fused graphics execution unit 1609a, allowingeach El usthin fused execution units I609A-feife to execute using a common iastwetion poimci segister. pfef] lu at least one embodiment, one or more Imemai instruction caches (e g„ 1606; are included in thread execution logic 1506D to cache thread inrtructions fbr execution units. In at least one embodi ment, one or more data caches (e,g, 1612) a re Included to cache thread data during thread exfetom. In attest one embodmmiu, a sampler 1610 is included to provide texture ^            operaiiuns and media sampling for media operations.; In at least one embodiment sampler fofo includes specialized texture or media sampling functionality to protest texturedata during a. sampling process before providing sampled data to an executimi unit.

[0268] During execution. in al least one embodiment, graphics and media pipelines send thread initiation requests to thread execution logic 16000 via thread spawning and dispatch logic. In at least one-emboditnent, once a group of geometric-objec^has been processed and rasterized into pixel data, ptxel processor logic Ie g.. pixel shader logic, fragment shader fogic. etc,) within shader processor 1602 is invoked to further compute output infonnation and cause results to be written to output surfaces (e g : color buffers, depth buffers, stencil buffers, etc} In at least one embodiment, u pixel shader or Augment shader calculates values ol'various vertex attributes that are to be interpolated across a t astenzed object In at least one embodiment, pixel processor logic within shader processor Io02 then executes an application programming interface (API)-snpphcd pixel or fragment shader program. In at least one-embodiment, to execute a shader reogum, shade m octroO! 1 etrexitchw- t rrace to an t-v venture mut t.e g . H\ > x?a traced Cisoateher ‘ e? - In at least one embodiment, shader processor 1602 uses texture sampling logic •n s tnj pier 161 (I to access texture data in texture maps stmxl in mein cry, In at least one embutontenf authmem' >'p' nurere o: w\xne cat., and umw yeomen'', data compere n;\d coku data for each geometric fragnre   or discards one or more pixels from further processing.

[0269] In at least one entbudiment data port fol 4 provides a memory 'access mechanism for thread execution logic foOQIMo output: processed data to memory for further precessing on a graphics processor output pipeline.. In at. least one embodiment, data port 1614 includes or couples to one or more cache memories (wg,, date cache 1612) to cache data for memory access vta a data port. 16270] As illustrated in FIG loF. in at least one embodiment, a graphics execuf ret unit 1608 can include an instruction fetch unit 1637, u general register rile array (GRr) fo24. an architectural register Hie array f ARF> 1o2o. s thread arbiter 1622. a send unit 1636, a branch unit 1632, a set of SIMD Heating point umfs $FPl.’s) 1^34, and in al least one emhodsmem a set <>f dedicated integer S1MD ALUs 1635. In at least care emhodimem, GBF ib.Li and AR.F Io2<> includes a set of general register foes and architecture register foes associated with each Mumlumtuii? hardware thread dart may be active in graphics execution urm 1605. hr al least cue embodm-xul {mead ascfeuxh-mi Maw i, m mnauwe m ARJ' if<<\ uhfic w cd d mug thread execfeon is stored in GB.F 1624. 1« at least me embodiment, execution state of each thread, felifengmshW                                 held in thread-specific registers in. ARF 1626. 6427 i j In al fecal cue emmxlfecm, gmpfecx excx mvt lot'S h.sa an whshWe den is i cornbhumon of Simultaneous Mu’m-l'hreadinglSM I t and fine-grained interleaved Muhi-Threading iWT). In at least one embodiment, architecture has a modular confab be fine-tuned at design time based on a target number of simultaneous threads and number of registers per execution unit, where execution unit resources arc divided across logic used to execute multiple si m>fememis threads

[0272] In atleast one embodiment, graphics execution unit 1608 camco-issue multiple-instructions, which may each be different instructions.. In at least cate embodiment, thread arbiter I 022 of graphics execution unit thread 16x8 can dispatch instructions to one of send unit 163«;. branch unit 1642, or SIMD FPlJ(s) 16^ for execution In at least one embodiment. each exeeotiun thread can access 128 general-purpose registersm uhi n GRF1624, where each register can. store 32 byfes,,aecessibie as a SWD S-element vector ol 32-fe data elements, hi at least one embodiment each execution unit thread has access to 4 Kbytes within GRr 1624: although embodiments are not so limned, and greater or fever register resources may be provided m other embodiments; In at least one embodiment, up to seven thraads ean exec^ al though a number of threads per execution unit can also vary according to embodiments. In at festtme embodiment, in which seven threads may access 4 Kbytes, GRF 1624 can store a total of 2^'Kbytes; In at. fest one embodiment flexible: addressing modes can permit registers to be addressed together to build effectively wider registers or to represent strided rectangular block data structures.

[0273] In at least one embodiment, memory operaifew sampler operations, and other longer-latency system communications are dispatched via -'send' instructions that are executed by message passing send unit 1030. In at least one embodiment, branch instructions ate dispatched to a dedicated branch unit Io32 to facilitate SIMD divergence and evemnai convergence.

[0274] In at least one cm,»-xliment graphics execution unit 1608 Includes one or more SIMD floating point units IFPLiso 1634 to perform floating-point operations. In at least one embodimem, FPl.hs» 1634 also support integer compufeon. In at least one embodfeem FPUls) 88 1634 can SIMD execute up to M number of 32-bh floating-point (or integer) operations, or SIMP execute up to 2M 1 O-bit integer or 16-bit floating-point operations In m least one embodiment, at least one of FPU{s) provides extended math capability to support high-throughppt tjwscendwtdl rnath fonstfefoamd double: pfeeisiou. 64-biX Boating-point In at least one embodiment, a set of 8~bit integer SIMD ALLIs 1635 are also present, and may be specifically opdnhzed to perform operations associated with machine learning computations

[0275] in at least one embodiment, .arrays of multiple instances of graphics execution, unit I6P8 can be instantiated in a graphies sub-core grouping: (e.g,, a sub -slwe). 1« at least one embodiment, execution unit 1608 can execute instructions across a plurality of execution channels. In at least otto embodiment, each thread executed on graphics execution unit 1608 is executed on a different channel.

[0276] Inference and / or framing logic.615 are used to perform inforencibg andfor training operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS. 68 and / or 6G In st least one smbodimenf portions or all of mfetwee and / or training]ogic -615 may be incorporated into execution logic I609D. Moreover in at least one embodimentrnnferencmg anfeor training operations. described herein may be done using logic other than, logic illusuated in FIGS, 6B m bC. In at least one embodiment,, wmglu parameters may be stored in on-chip <x ofGcbip memory and / or registers (show or not. shown) that configure ALUs of execution logic 1606D to perform one or more machine leaming aigpnthmx dwfal network architectures, use cases, or training fechmques described herem.

[0277] FIGI . "A uhisiiaits a parallel process n^, unit (pPPL'') 17iMA5 according to at Jeasterns embodiment. In at least, one em bodiment, 1700A1 s conf gured with machineoeadabfe code that. If executed by PPU 1700A, causes PPU 1709A to perform some or all of processes and techniques described throughout this diseboare. hi at least one embodiment, PPU 17606 is a trmlii-threaded processor that is impleow ■ cd on one or more integrated circuit devices and that mihzes multithreading as a latency-hiding technique designed to process computer-readable instructions (also referred to as machine-readable instructions or simply instructions) on multiple threads in parallel. In at least one embodiment, a thread refers to a thread of execution and is an instantiation of a set of instructions configured to be executed by PFU 170OA< In. at least one embodiment PP-J 1700A is a graphics processing unit ( ‘GPU'') configured to implement a graphics reiuk-img pipeline fur process!ng three-dimensional rSD':) graphics data ta order to genes ale tsyo-dimeusional ('2D'} linage data tor di^plas on a ds spl ay device such as a Urpnid crystal display ;'1 CD”) devise. Irrat least one embodi'ne'U, PPI: 1700 A A mi fixed to perforin computations sasehfelmw^                   mgchine-Ieanung operations, FIG 17A foe Suites ar m ample 'wfelrl pi -»      io: de sinm^e puipo^ ,'uk <md shvitld A co-wmied as a non-Iimbmg example of processor architectures contemplated within scope of this disclosure and that anv suitable processor may be employed to supplement and / or substitute for same.

[0278] In at least one embodiment one or more PPOs 1700A are configured, to accelerate High Performance Computing ("HPC*’h datacenter, and machine learning applications. In tn least enr embodiment., PPU I700A is configured to accelerate deep leanimg systems and applications mcludmy folknsmg non bmitiry examples lutunomuiis xchtc:e platforms, deep lemmm., Inch-,nc-i;ac\ speech, image text iceogmuon sxMe ns, imefogem \ fern ,4ralvties.: 'ofeulm Shwiatfons, drug discovery, disease diagnosis, weather forecasti ng, big data analytics, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation online, search optimizations, and personalized user recommendfe ons, an d more,

[0279] 1g at least one embodimght, PPI J 1700 A. includes, without limitation, an InpaWfopm (“I / O”) unit 1706, a fronted unit 1710, a scheduler unit 1712, a work distribution unit 1714, a hub 1716, a crossbar foXbar) 1720, one or more general processing clusters f'GPGs”)1718, and one or more partition units (''memory partition units”} 1722 In at least one embodiment. WtLl 7O0A. is cpimecW                or other WUs 1700A vis one or mmv highspeed GPU interconnects (^GPU mtemormects”! 17®; In at least one embodiment, PPU I fotA is cormected to a host processor or other psriphei'al devices via an iufe connect 1702. In at least one embodiment, PPG l70QA is connected to a local menHsrv compiismg one or more memory dm wtS (fownoiy ’) 1704. In at least one embodiment, memoi v dm ices 1704 include, without limitation, one or more dy namic random access memory (“DRAM”) devices. In at least one embodiment, one or more DRAM devices are configured and / or configurable as highAwfewidth memory (“HBM” i subsy stems, with multiple DRAM dies A-ickfe v Idan e.k h Jfe me [02S0] In al least one embodiment, high-speed GPU mtemomiect 170S may refer to a wire-based multi-lane communications link that, is used by systems to scale and include one or more FPUs' 17O0A. combined: with one or more central processing units foCPUs'1), supports cache coherence between FPUs 1700A and CPUs, and UPl m«,>teu\g In at least one embodiment, data wWr commands are transmitted by high-speed GPU mteremmect 1708 through hub 1715 to / ff'Om other uretx or'PPU I700A such as one or snore dopy engines, video encoders, video decoders, power 'm^gefaent units, and other components which may not be explicitly ill astral in FIG, 17 A. i<*?s11 In at least one emboduuent I / O unit 1706 is configured to mmamh and receive communications (e.g., commands, data) from a host pn^essor (mg illustrated in FIG, 17A) over system bus 1702. In at least .Une embodiment, I / O unit 1705 communicates with host processor direefo via system bus 1702 or through one or more intermediate devices such as a memory bridge. In at least one embodiment, I / O unit 1706 may communicate with one or more other processors, such as one or more of FPUs 1700.A via system bus 1702. In al least one emboennent 1 Oumt IGF? impk-incn^ a Peuphcrol fon-ponent tekm.Ofmect Express UPCk' 0 interface for commute cad ons; over a PCIe bus. fa at least one embodiment, I / O unit 1705 implements interfaces for commumcating with external devices.

[0282] fa at least one embodiment, I / O unit 1705 decodes packets received via system bus 1702. In at least one embodiment, at least some: packets represent commands configured to cause PPG 1700 to perform, various operation In at least one embodihsem, I / O unit 1706 transmits decoded commands to various: other units ofPPU 170QA as specified by commends:. Ih -at least fate embodiment, commandsare transmitted to front-end unit 1710 and / or transmitted tu hub 1710 dr other units of PPG I700A such as one or m ore copy engines, a video encoder, a video decoder, a power management unit, esc. (nut explicitly ifamWted in FIG. 17A), In M least one embodiment, It© unit 1756 is configured to route communications between and among various logical umis <.sf PPU 1700A.

[0283] In at least One embodiment, a program executed by host processor encodes a command stream in a buffer: that provi des work i oads to FPU: 1700A for processing. In at 1 east one embodiment, a workload comp: sws rwe-uct-oas and data to- be processed by those instructions, In at I ease one unbodimvnt, hue! ix $ .egum m a memory that is accessible «e g . read u nte) by both host processor and PPG 1700A — a host interface unit may be configured to access: buffer m a s' stem mervsy commoted to system bus 1702 via memory requests transmitted over system bus 17>C by I G emt 1706, In at least one embodiment, host processor writes command stream te ■nufe? and -lav nwromta a mmitrn to start of e^ruumnd stream io PPI' I ~ mA ^cu fam front- end unit 17 W receives pointers to one or more         streams and manages one or more command streams, reading commands from command streams and forwarding commands to various units of FPU 1700A,

[0284] In at hast one endxrdi^             1710 is ewpled.ta*&h^uler^ configures various GPCs 1718 to process tasks defined by one or more command streams, in at least one embodiment, scheduler unit 1712 is configured to track state information related to various tasks managed by scheduler unit 1712 where state information may indicate which of GPCs 1718 a task is assigned to, whether task Is active or inactive, a priority level associated with task, and so forth. In at least one embodiment, scheduler unit 1712 manages execution of a pluraluy of tasks on wax more of GPCs 1718

[0285] In at least one embodiment, scheduler unit 1712 is coupled to work distribution unit 1714 that Is configured to dispatch tasks for execution on GPCs 17 1 8, In at least one embodiment, work distributem mfo 1 714 tracks a number of scheduled tasks received from oc-tolu er umt foil an 1 wmk Jivuihu-on ano 7 ■ } >,\u'nge* a nt mhre 'ask pool and an amne task pool for each of GPCs i 718: at least.one embodiment pending task pool comprises a number of slots 32. slots) that con tain tasks assigned to be processed by a particular GPG 1718t active task pool may comprise a number of slots (e.g , 4 slots} for tasks that are actively being processed by GPCs. 1718 such thatas one of GPCs 1718 completes execution, of a task, that task is evicted from active task pool for GPC 1718 and one of other tasks from pending task pool is selected and scheduled for execution pn GPC 1718. Id at least one embodiment, if an active: task         GPC 1718, such as while waiting for a data dependency to be resolved, then active task is evicted from GPC 1718 and returned to pending task ptml while another task in pending task pool is selected and scheduled for execution on GPC 1718,

[0286] In at least one embodiment, work distribution: unit 1714 communicates with one Or more GPCs 1718 via XBar 1720. In at least one embodiment XBar 1720 is an interconnect ncOMvk that couptvs many of units of PFU I7C0A to«sther units of PPG I700A and can be configured to uouple work distribution unit 1714 to a particular GPC 1718 bi at least one i mboCmwm, one m more other units of PPU 1700A may also be connected to XBar 172u via hub 17IG [0287 j In ut least one embodiment tu^ks are managed by scheduler unit 1712 and dispatched to one of GPCs 1718 by work riismbtUnm mm I7fo GPC 1718 m coullgmed to |«<kus? task 92 and generate results, In at least one .embodiment, results. may be consumed by other tasks with in GPC I7I SvTouied to a              1.718 vutXBar 1720. or stored in memory 1704. In ai least oneembodiment results can be written to memory 1 704 via partition units 1722, which implement a memory Interface for reading. and writing data to / from memory J 704. In at least one embodiment, mstdtis.wi be transmitted to another PPG i 7iM or CPU via high-speed GPU ■Mwi'Tic! ri': 8 Ir a; least ore em jo limern PPI I "no \ ujehnk-s, w ulmm bcutaram a number U of partition units 1722 that is equal to number of separate and distinct memory devices 1704 coupled to PPU I70QA, In at least one embodiment partition unit 1722 will be described in more detail below in corijunction with FIG. I76.

[0288] hr at least one embodiment, a host p-wsw executes a driver kernel that implements an application programming interface (“AFI" t that enables one or more applications executing on host processor to schedule operations fra execution on PPU 1700A. In at least one embodiment, multiple compute applications are sisnuhaneousiy executed by PPU 1700A and PPU 17:00A provides isolation, quality of service;(“QoS”), and mdepcndom address spaces fur multiple compute applications; In at least one embodiment, an application generates instructions (e.g., in form of API cMI-S) that cause driver kernel to generate, one or more tasks for execution by FW 1700A and dri ver kernel outputs tasks to: one or more streams being processed by PPU 170QA. Iri at least orie embodiment, each task comprises one rir more groups of related threads, which nw be referred te as a warp. In at teastone embodiment, a warp compri ses a plurality of related threads (e.g., 32 threads? that can be executed in parallel. In at least one: embodiment, cooperating threads can refer to a plurality of threads including instructions io perform task and that exchange data through shared memory In at least one embodiment, threads and cooperating threads are described.hi more detail , In acepfoance with at least one embodiment, in cofeanetion with FIG I7C

[0289] foference and / or training logic 615 are used to perform inferencing and / or training operations associated with one or more embodiments, Details regarding mterwwe and / or uJmng muic 615 me provided beiuw in tamitmenvh oith I IGS oB and os oU In <a Ie ra one embodiment, deep learning application processor is used to train a m-mhine learning model, such as a. neural network, to predict or inter information provided to PPU i 70bA. In at least one embodiment, PPU 1700A is used to infer or predict information based on a trained machine teaming..model (e g , neural network} that has been, trained by another processor or system or by FPU 1700A. hi a? least one embt>dii^.en^ PFU 1.700A may be used to perform one or more neural network use cases described herein.

[0290] FIG, 17® illustrates a general prceessim? duster OXjPC’G 17Q0B,. aucorfc one embodiment In. at least me embodiment, GPC I ?“‘>H is GPC 1718 esf FIG; 17A. In at least one embodlmem, each GPC 1?00B includes, without limitation,. a number of hardware units for process! ng tasks arid each GPG 1700B includes, without limitation, a pipeline manager I702, a pi e raster operations unit rPROP”)-1704, a raster engine 1708, a work .distribution crossbar i AWX 7 1716, a memory management unit CMMU'1} 1718, one or mure Data Processing Cmaters i''DPCs"} 1706, and any suitable combination of parts. I 070; ] In at least one embodiment, operation of GFC 1700® is controlled by pipeline manager I ~ C tn at least one embodiment, pipeline munaect 1 ” C maaag^sx-mfiguration of one or more DPC> 1706 for processing tasks allocated to GPV I 6'. m least oneembodiment, pipeline manager 1702 configures at least one of one or more DPCs 1706 to implement at least a portion of a graphics rendering pipeline. In at least one embodiment. DFC 1 700 is conf gored to execute a vertex shader program on a pregTammabie:Strea.mmg^              [” SM’) 1714, In at least oneembodimeht, pipeline, manager 1701$ ccmOgured^^                    from a work distribution unit to appropriate logical units within GPC 170OB, in-at least one embodiment, and some packets may be routed to fixed function hardware units in PROP 1704 and / or raster engine 1708 while other packets may be routed id WCs 1706 bar .processing by a ptmutw c engine 1712 1714. hi at least one embodiment, pipeline .manager 1702 configures si leas? one uf DPCs 1706 to implement a neural network model and / or a computing pipeline;

[0292] In at least one embodiment, PROP unit 1704 is cmifigured, in at Iea^t one i-mhod-mem, to ronie: data generated by raster engine 1708 ami DPCs 1706 to a Raster Opcunotw t ROP 0 mut in partition unit 1722, described in more.detai I above hi conjunction with FIG. 17A. In.at least one embodiment, PROP unit 1704 is configured to perform optimizations for color hkmdmg, organize pixel dam, perform address translations, and more in at least one embodimern, raster engine I7v8 includes, without bmimtion, a number of fixed function hardware units configured io perform various raster operations. In at least one embodiment, and raster engine 1708 includes, without limitation, it setup.engine, a coarse raster emm-e, a culling engine, a clipping engine, a fine raster engine, a tile, coalescing engine, and am smtabic combination thereof. In at least one embodiment, serup engine receives transformed vertices and 94 generates plane equations associated with geometric primitive defined by vertices; plane equations are transmitted to coarse raster engine to generate coverage information (e.g. an x, y coverage mask fora tile) for pummvg; output of coarse raster engine is transmitted to culling engine yvhere fipgments associated with primitive tha? fail a z-tcst are culled, and tranomttcd to a clipping engine where fragments lying outside a viewing fiusrum are clipped In as krat one' embodiment, fragments that survive dipping and culling are passed to fine ra^ei engine to generate attributes tor pixel fragments based on plane equations generated by setup erre.no In at least one embodiment, output of raster engine 1708 comprises fragments to be processed by any ranelnoenrw me'. as w a tommenrahaees uvr'e uentec wrhm DK Hito [07 X*1 in at least one embodiment, each DPC 1706 included in GPC 17008 comprise, without imutation an M-Ptpe Controller ("MFC") 1710; primitive engine 1712; one or more SMs 1714, and saw joitabfe combination thereof, hi at least one embudinrent MFC 1710 controls operation el'DPC 17(10, routingpackets received from pipeline manager I 7G2 to appropriate traits m DPC 1706. In at least one embodiment, packets associated with a vertex are routed to prim re x o engine 1712, which is configured to fetch venex attributes ass«?ciated with vertex from memory; in contrast, packets associated with a shafe program may be transmitted io SM 1714

[0294] In at least one embodiment, SM .1714 comprises, wiihoiu limitation, a prpgrammabfe streaming processor that is con 6go; ed to process tasks represented by a number of threads. In- at least orie embodiment, SM 1714 i ■; muhidhteaded and configured; to execute a plurality of threads (e,g,, 32 threads) from i particular group of threads concurrently and implements a Singlednstrucfipa, &4ttlfipI;edData.F^                   where each thread in a woup of threads (a g a warp) is configured: to process a different set of data based on same set oi infractions. In at least rate embodiment, hh threads ill group of threads execute same instructions. In at least one wnbe^imen^ SM 1714 implements a Smgle-Instmetion, Multiple Thread ("SIMT") mehpecturewher^ each thread in a group of threads is configured to process a ditferem set of data based on same set of instructions, but where individual threads in group of threads are allowed to diverge during execution. In at feast one embodiment, a program counter; call stack, and execution state is maintained for each warp, euabhng concurrency between warps and serial execution within warps when threads within warp diveige In another embodiment, a program counter, call stack, and execution state is maintained for each individual thread, enabling equal ienc\ bviwetif ad threads vothm and         wraps In leav vut- unbediniuil cxeemitstate is smimaiued fo? each iodn idea- Oueed and threads executing ssmte utstructions be converged and executed in parallel for beitei efoetency. At least one embodimentof SM I714 are described id more detail below.

[0295] In at least one embodiment, MMU 1718 provides an imerffoe; between WC 17O0B and rteirw ptefomu emt <e a , gv-mon unit I ~22 nfFKI l~ M and MMl J 718 ;m-whs translation of virtual addresses into physical addresses, memory protection, and arbitration of memory requests. In at least one embodiment, MMU 1 7 IS provides one or moretranslation, lookaside butlers (“TLBs”) for performing translation of virtual addresses into physical addresses in memory 10296] buerence and / or training logic 61S are used to perform mferenemg and / or tmimng operations associated with one or mm c embodiment; Details regarding inference and / or training logic 615 are prtodded n- eeniunedon with FIGS 6B and / or 6C In at least one embodiment, deep learning application processor is used to train a machine learning model, such as a neural network, to predict or infer information provided to GPL J 7DC?B In at least one embodiment, GPC ITOOB Is used, to Infor or predict inform afiou based on a trained machine learning model fogg neural network) that has been Pained by another processor or system or by GPC 170011 fo at least one embodiment, GPC 1,70GB may be used to perform cw or more neural .network use cases described herein. (0297] FIG, I7C illuslrates a memory partition unit .17000 of a parallel processing unit fTPlF), in accordance with at leastone embodiment. In at least one embodiment memory partition milt 1700C includes, without lbmummr Raster F)pwame« r'RQPfoumt 1702; a level two (“LT) cache 1704; a memory interffoe 17nm md any srnwbk- eumbmadph thereof In at least one embodiment, .memory interface I Top coupted to mcmon In at least one embodiment, memory interface 1706 may implement ?C. 64, 128, 1024-bit data buses, or similar ■ mpi'.-ment<W'-to, tor lm>btepeC0 foia wansfc1 In ,tt lerna one embodiment PPI memp-'mies I memory interfaces 17(16, one memory interface 1706 per paLtrof partition units 1700C, where each pair of partition units 1700(7 is connected to a corresponding memory device. In at least out ernbodirnem. in at least one embodiment, PPI; may be connected to up to ¥ memory devices, such as high bandwidth memory stacks or grtmmes double-datartW, version 5, svnvhummw Jimmue random al ’ess mc-mor^ :"GDDR5 SDRAM '}

[0298] In at least prre ertfeodi^                          implements a cub bandwidth memory see-end generation CHBM2”) memory interface and Y equals half U. In at least one embodiment, HBM2 memory stacks ate located on sam e phy si cal package as FPU, providing substantial power and area savings compared with GDDR5 SDRAM systems. In at least one embodiment, each HBM2 stack includes, without limitation. four memory dies and V equals 4,. with each HB-M2 stack including two 128-bit channels per Jie for a total of 8 channels and a data bus width of 1024 bits. In at least one embodiment, memory supports Single-boor C-awcmw Double-Error Detecting ('‘SECDED" j Error Couvclion Code {"ECC" ? io protect data. In at least o <. end or nne r mJ. prvito ' ^Ikwehh k\ m tonpuxovv          .tx'nsb. .- data corruption

[0299] In at least one embodinwm, FPl m-pkmems. amumde<el memory hierarchy. In at least one embodiment memory partition unit 1700C suppuns a unshed memory to provide a single unified virtual address space fix central. processing. mn 11' CPL> and FPU memory., enabling data sharing between virtual memory systems. In at least one embodiment, frequency-of accesses hv a PFU to memory located on other processors is traced to ensure that memory pages are moved to physical memory of FPU that is accessing pages more F-equeutly. In at least one emhodrmeng high-speed GPU mtereonnect 17W supports address translation services allowifig FPU to directly access a CPU’s page tables: and providmg full access to CPU memory by FPU..

[0300] In at least pne embodiment copy engines transfer data between multiple FPUs or between PPUs and: CPUs. In at least one embodiment copy engines can generate page faults for addresses, that are not mapped into page tables and memory partition unit 17G0C then services page faults:, mapping addresses into page table, after which copy engine performs transfer. In at least one- embodiment, memory is pinned (in at least one embodiment, non-pageable) for multiple copy engine operations between, multiple processors, substantially reducing available memory. In at least one embodiment, with hardware page fauhmg. a-idresses can be passed to eepv eumces wrlfout receY as to whedre. vm'.w>n p, ves are wamlcm w.d isyw puKm-u transparent. [030 i ] Data from memory 1704 of FIG. 17A or other system -memory is fetched by memory partition unit I?noc and stored in 1-2 cache 1704, which is,located on-chip and is shared between various GPCs. in accordance with at least one embodiment. Each memory partition unit 97 17W, pi at I^t owe             i«dudesy without Jimkaiion, at least aportion of L2 cache assaulted w uh a corresponding memo         In at least one embodiment, lower level caches me implemented. in various units within GPCs, In nt least one embodiment, each of SMs I 7 M may implement a level one                     LI cache is pnyate:            is dedicated to a particular SM 1714 and datafrom L2: cache 1704 is fetched and stored in each of LI caches for processing in functional units of SMs 1714. In ai least one embodiment, L2 cache 1704 is coupled to memory interfile e 1706 asci XBar OC0.

[0302] R0P unit 1702 performs graphics raster operations related to pixel cofog such as color compje^mu pixel blending, and more., in at least one embodiment. R,0P unit 1702, in at least one embodiment, implements depth testing in conjunction with;raster engine '1708, receiving a depth fur a sample location associated with a pixel fragmentfrom culling engine of raster engine 170$ In at least one embodiment, depth Is tested against a corresponding depth in a depth buffer for a sample location, associated with fragment, In at least one embodiment, if fragment passes depth test for sample location, then HOP unit 1702 updates depth buffer and transmits a result of depth test to raster engine 1708.. ft will be appreciated that number of partition units I700C may be different than number of GFCs and, therefore, each ROP mut fou2 can, in at Iwfo -me em bodimen t, be coopted fo^          In. ad east one                    l~C tracks packets received from d liferent GFCs and determines which that s result generated by ROP unit 1702 is routed to. through XBar 1720.

[0303] FIG. I7D Illustrates a&freaming hmltl-processof feSM’fr I 70QO, according to at feast one embodiment. In at least one;embodiment, SM 1700D is RM 1714 of FIG. 17B. In at least one embodiment, SM 17000 includes, without limitation, o msWction:cache 1702; one or more scheduler units 1704; a register fife 1708; one or more processing cores (feares”). 1710; on e or more fi \<j muim^i Ml •> i fo,2 sw t more load ’W mm > < , s', ' ) foil an fotereonnect network 1716; a shared memory / lwel me (“I G) cache 17 i 8: and any suitable combination thereof. In at teao one embodiment, a work distribution Unit dispatches tasks for execution on gmeial processing dustoss ("GWVh of pai;Mel puicessing units s' PPUs' ) ;mJ each utsk is al located to a pawiealar Data Processing (Muster ("DPCM within a (jPC' and, if task is associated with a shader program, task is allocated to one of SMs 1 7000 In at least one embodiment, scheduler unit 1704 rect-wes tasks from work disoibufom urm and manages instruction scheduling for one or more thread blocks assignee to SM 1700D In ar least one embodiment scheduler unit 1704 schedules thread blocks for execution §s warps of parallel threads, wherein, each thread block is allocated at least one warp.. In at least one embodiment, each: warp executes threads, In at least one embodiment scheduler unit 1704 manages a plurality of different thread Hocks, ullocadn< warps to different thread blocks and then disoaWnfe irwunmiotw iron; plurality of different uxmeumse groups to vuriotw funehonai orbs tr processing cores 1710, SFUs 1712, and LSUs 1714) during each clock cycle.

[0304] in at least one embodiment, Cooperati ve Groups may refer to a programming model for organising groups of communicating threads, that allows developers to express granularity at which thread? me communicating, enabling expression of richer, more efficient parallel decompositions. In at least one embedimetn. cooperative launch APIs support synchronization amongst thread blocks for execution of parallel algorithms. In at least one embodiment, applications of programming models provide a single, simple construct for s\ nehromzitig cooperating threads: a barrier across all threads of a thread block                   } function). However, in at least one embodrmem,. programmers may doline groups of threads at smaller than thread block granularities and ss nchronize witbui defined groups to enable greater performance, design ffexlbilii^ software reuse in form, of collective group-wide function interfaces. In at least one. embodimen t. Cooperati ve Groups en ables programmers to define groups of thread s explicitly at sub-block (in at least one embothmeht as small as a single thread) and multi-block granularities, and to perform collective operations: such as synchronization on threads in. a cooperative group. In at least one embodiment, programming, model, supports clean composition across software boundaries, so dial libraries and utility functions can synebumiae xal'dy within their local context without having tri make assumptions about convergence. In at least one embodiment. Cooperative Groups pr m .senable new patterns of cooperative parallelism, including, without limitation, prodacer-cohsumer parallelism, opportunistic parallelism, and global $vnchmniMtfoh across an entire mid of thread blocks.

[0305] In at least one embodiment, a dispatch unit 1706 is configured to transmit instructions to one <x more of functional units and scheduler imii 1704 includes, without limitation, two dispatch units 170o that enable two different instructions from stmts wmp to be dispatched during each dock cycle. In st least one embod ment, each scheduler unit I 704 includes a single dispatch unit 17(m or additional dispatch units 1706. [030b] In at least cw erobadiment^ aaeh 'W in at least one embodiment, include^ wi thout limitation, register file i 71$ ihaf prevides a set of registers for fimcttonai units of SM 17000, In at least One embodime              I OS divided. between each of functional, wilts such that eabh functional uni t is allocated a dedicated parboil of register file 1708. In at least one embodiment register file 1708 is divided between different warps being executed by SM 1 700D ami register file I 708 provides temporary storage tor operands connected to data paths of tunctsonsi units In at least one embodiment, each SM 1700D comprises, without bn oats on a / megu ef l precess ntrews s’ d In ,tt lew? ,>ne cn b-dn tent 1^001) includes, without limualion, a large number \c.g , 128 or more} of distinct processing cores 1710. In at least one embodiment, each processing core 1710. m at least one embodiment, includes, without limitation, a fully-pipelined, single-precision, double-prevision, and / or mixed prevision processing unit that induces, without limitation, a floating point unthmetiv logic unit and an integer arithmetic logic unit. In at least one embodiment, floating point arithmetic logic units implement 11:.111:- 754-2(858 standard fix                           In at ore embodiment, ptpe^og cores 1710 include, without limitation, .M'^ngb-prepisicsd (32’bitlfloating point coms, fid integer cores, 33 double-precision (dfobifi fioati^            and 8 tensor cores.

[0307] Tensor corns are configured to perform matrix operations in accordance: withat least one embodiment In at least one enfoodiment ate or more tensor cotes are iucfodeil in processing cores 1710, In at least one embodiment, tensor cores ate configured tp perform deep, learning matrix arithmeiie, such as eenyplution operations for neural nefwofo training and infisrewng. In at least one embodiment each tensor core operates on a 4x4 matrix and performs a matrix multiply and accumulate operation D ~ A A B C, where A, B, C, and D are 4x4 i$| In at least one embodiment,matrix:multiply inputs A and B are 16-bii floatingpoint nwwes and accumulation matrices C and D arefo-bit fioafing point or 32»bit floating point matrices. In at least one embodiment, tensor cores operate on 16-hit floating point input data, nub ?2-M 110300« poiat a^umnlauo-, to a „\w- oue uebodimtut lo-bd tloXine pmut multiply uses 04 opemuioug and results in a full precision product that is then accumulated using 32~b:it floating point addition with other intermediate products for a 4x4x4 matrix multiply. Tensor cores me used to perform much larger two-dimensional or higher dimensional matrix operations, built up from tltese smaller element^ ia at least: one entbodimeni:. In atieast one embodipwn^ an AFI such as CODA 9 C-H- API exones specialized matrix lead, matrix multiply and accumulate, and matrix stere qperauous to cllkienuy use tensor cores from a CUDA-C- program. In at least one.embodiment, at CUDA Im J, 'a .up-k - c nmrface assumes IAx W size matrices spanning all 32 threads of warp. [O3O9] In at least one embodiment, each SM 1700D comprises, without limitation.. M SFUs 1712 dm oefonn special functions (e g.. attribute evaluation, reciprocal square root, etc.} in at least one embodiment, SFUs 1712 Include* without limitation:, a tree traversal unit configured to traverse a. hierarchical tree data structure. In at least one embodiment, SFUs 1712 include, without limitation, a texture unit configured to perform texture map filtering operations. In -at least one embodiment, texture units are-configured to load texture maps (e.g . a 2D array of tends) from memory and sample texture maps to produce sampled texture values for use in shader programs executed by SM 1700b,. In at least one embodiment, texture maps are stored in shared memory!. I cache 1718 In at least one embodiment, texture mots implement texture operations such as filtering operations using mip-maps (e g., texture maps of varying levels of dmai I). in accordance with at least one embodiment In at least pne embodiment, each SM I “MOD .meiudes, without limitation, two texture units.

[0310] Each SM .l 7O0Dcmnprises, withwidw              1714 that implement load and store operations between shared memory / Ll cache 1718 and register file 370S, m at least one embodiment Each SM 1700D includes, without limitation, Interconnect network 1710 that cannscts each of functional units to register file 1708 and LSD 1734 to register file 1708 and shared memory / LI cache 1718 in at least one embodiment. In at least one embodiment, interconnect network 17161$ acrossbar that: can be configured to connect any of fimctional. omiS. to any of registers in register file 1708 and connect LSUs 1714 to segister file 1708 and memory locations in shared memonM-d cache. 1718.. [0311 ] In at least one embodiment, shared / memory / L I cache 1718 is an array of on-chip memory that allows for dam storage and communication between SM 1700b and primitive; engine ahd between threads in SM 1700b, in m least one embodiment. In at least one embodiment, shared memory / 1.,I cache 1718 comprises, without Iimim■ ion, 128KB of ston^e capacity and is in path from SM 1700D to partition unit. In at least one embodiment, shared memoiMLl cache 17.18, in at least one embodiment is used to cache reads and writes. In at least fee embodiment, one ormore of sIwM          cache 1718, L2 cache, and memory bavktug stoics. [031.2] 'Cq^Wb data cache and shared memory fimfepnaliiy Inta a single memory block provides improved perfennawe for both types                     at least one embodiment. In at least one embodiment, capacity is used oris usable as a cache by programs that do not use shared memory, such as if shared memory is cohtagored to use half of capacity, texture and loadfoore.operations can use remaining capacity, Integration within shared memw / Ll cache 1718 enables shared memoryfol cache 1718 tofune^en as a high-throughput conduit for streaming data while simuhaneously providing high-bandwidth and iow-huency access to llequemly reused data, in accordance with al least one embodiment In at least one-embodiment, xJren configured tor mineral purpose parallel comomatinn, a Simpler configuration can be used compared with graphics processing, feat least one embodiment, feed fimetion graphics processing units are bypassed, creating a much simpler' programmingmodel.. In genwl pu'.xwe parallel compmatiotr corfo work distribution unit assigns and distributes blocks of threads direcdv to DPCs, in at leas? one embodiment in at least one embodiment threads m a block execute same program, wag a unique thread ID in calculation to ensure each thread generates umqtm.results, using SM 170QD toevecute program and: perform, calculations, shared mcmoryfol cache 1718 to femmumcate between threads, and LSU 1714 to mad and write global memory through shared memory / L I. cache 1718 and memory partition: unit. In at least one embodiment, when configured for general purpose parallel computation, SM HOOD writes commands that scheduler unit 1704 can use tri launch new work on DPCs. [03-13] fo nt least one embodiment, PPU is inefoded in or coupled io a desktop: computer, a laptop computer, a tablet computer, servers, supercomputers, a smart-phone u g, a wireless, hand-heki devkel personal digital assistant C-PDA"). a digital camera, a vela de, a head mouftted display, a hand-held electronic device, and more., hi a- least one embudmwm. PPU is embodied un a single semiconductor substrate. In at least one embodiment, PPU is included in a systmmon-awhip ("SoCfo along with.one or more other devices such as additional PPUs, memory, a reduced msmsfoon set computer foRISC”) CPU, a. memory management unit fMMUh, a digs tai-to-analog convener I l> Aa h, and like

[0314] 'fa at least one embodiment, PPU may he included on a graphics card that includes one <r mmememw mo X ^-aphuw <ato m<o                                     i ntottaboard of a desktop computer. In at least one embodiment, PPU may be an integrated graphics pmmshtg ^                   chipset of motherboard.: [0315 ] Inference and / or training logic 61S are used to perform inferencing and / or training Operations associated with one or more embodiments. Details regarding inference and / or training logic 615 are provided below in conjunction with FIGS OB and / or 6C. In at least one embodiment, deep learning application processor is used to train a machine learning model, such as a neural network, to predict or infer information provided to SM 170SD: In at least one embodiment, SM 170OD h used to infer or predict information based on a trained machine Ieanung model (e g . neural network) that has been trained by another processor or system or by SM I "oap |5J at least one embodiment. SM I ?00D may be used to perform one or more neural Mtnofe use cases described herein

[0316] In at least oneiembodiment,- a single semiconductor platform may refer to a sole unitary semiconductor-based integrated circuit or chip tn at teas! one embodiment, muki-chtp modules urns for nwc with iwuvoed comnwto it\ which smu-l.ne omchtp upmum! and make sttbst.muni jmproyomws oyer utilizing a central -prGpmhtg unit (" GPU'') and bus implemeniation. In at least one embodiment, various modules may also be situated separately or in various coipbinations of semiconductor platforms, per desires: of user,

[0317] In at least one embodiment, computer programs i n. form of taachine-mddable executable code .or computer control logic algorithms are stored. In main memory 4ee04 and / or secondary storage, Computer programs, if executed by oneor mom processors, enablesystem 4ee00 to perform varionx hmeuonx hi to osdamw w-th at least one emhodimem In at least one embodiment, memory 4tvN storage, and or any elhei dotage ate possible examples of computer-wdable media. In at least one embodiment, secondary storage may refer to any suiiable storage device or system such as a hard disk dri ve and / or a removable storage drive, representing a floppy disk drive, a magnetic tape drive, a compact disk drive, digital versatile disk t 'DVD”) drive, recesdiira device, universal serial bus f USB") Hash memory, etc. In at least one embodiment, architecture andfor functionality of various previous figures are unpiememfo m ewnK-w m't Pl 4ee?2 awfee pfocesxmg stem lev :2 ?n >mem aec newt capable of at least a portion of capabilities o^               parallel processing s\ stem tee 12; a chipset (e.g„ a group of integrated circuits designed to work and sold as a unit for performing related functions, etc }. and any suitable combination of integrated cirauitfsk

[0318] In at least Oto emb.-Jmumt aiCThxk re and / or functionality of various previous figures ar©'            in context of a general computer system, a circuit board system, a game console System dedicated for men.fomn'm purposes, aaappl scat to        System, and. more. In at Ieastone:embodinteul,: computer system 4eeO0 may take form of a desktop computer, a. laptops computer, a tablet computer, servers, sapereoinputers, a wart-phone tog., a wireless, hand-held device}, personal digito assistant ("PDA”), a digital camera, a vehicle, a hetomomPed display, a hand-held electronic detoe. a mobile phone device, a television, workstation. game consoles, endteddtd x\atem, and\i: am. oihw ope os logx [93I9] in a) least one embodiment parallel processing system dec 12 includes, whom limitation, a plurality of parallel processing units C'PPUsfo 4eel4 ami associated memories 4ee16. in at least one embodiment, FPUs 4eel 4 are connected to a host processor or other peripheral devices via an interconnect 4^8 ard a switch 4ee20 or mdtiptoer in at least one embodiment, parallel processing system 4eel 2 distributes computational tasks across PPUs 4eel4 which can be parallelizable......for example, as part rOistrrbutow^              tasks across multiple graphics processing tot ("GPU") thread btoU. In at least one embodiment, mWW is shared and accessible tog?, for read andto write access) acres* some or all of PPUs 4to.4,nltough such shared memory may iucnr pei'formancc penalties relative to u se of focal memory and registers resident to a PPU dee 14. In at least one embodiment, operation of PPto 4eel:4 is synchronized through use of a command such a& :syncthreads(), wherein all ^threads in a, block (e.g<, executed across. multi pi e PPUs 4ee 14) to reach a: certai n point of executi on: to code: before proceeding

[0320] Other variations are within spirit of present disclosure. Thus, while disclosed techniques are susceptible to various modificatiohs and alternate v c>mstruw. raes, certtun Illustrated, embodiments thereto are shown In drawings and have been described above in detail. It should (>«.• uademood, however, that there is no mreutto to limit disclosure to specific form w form's disclosed, but on contrary.»mention is to cover all modifications, alternative constructions, and ecuudtecw fafoug ouhn. -4-0 J and s^-pc vl dm> forore a* <kfired m ..-ppended chum*

[0321] Use of terms “a” and tof’ and "the" and similar referents in context of describing disclosed embodiments (especially in context of following claims) are to be construed to cover both singular and plural, unless otherwise indicated herein or clearly contradicted by context, and n<v >.s a dtomi'i'ii ot a forms \nJuding ' "l.m mg '' irwhtong." and ’omtemimg' ;.w m 104 be construed as open-ended terms (meaning' including, but not limited to,"} unless otherwise noted. Teon. “connected / ' when unmodified and referring to physical connecUons, is to be construed as partly or wholly Contained withm, attached to. or joined together, even if there is something, iate^     Recitation of ranges of whies herein ate merely imended to serve as a dertband method of referring individually to each separate value tailing within range, tiniest otherw'-k- indo atod herein and each separate value is incorporated into specification as if it were sn-hx iduaih waited herein, Use of a set (e.g, a set of items) or subset, unless otherwise: noted or conn<kbeied hv context, is to beconstrued as a nonempty collection including one or more members. Further, unless otherwise noted or contradicted by context, a subset of a corresponding set does not necessarily denote g proper subset of conespombug set, but subset and corresponding sei may be equal. [0? 22] Conjunctive language, such   phrases of form “at least one of A, B, and: C,” or “at least one of A, B and C." unless specifically stated otherwise or otherwise clearly contradicted by 5 ome\t othe-nwse uixlmrtood u mt ow> t a-- iwed m general to pmsed that an nem. term, etc., may be either A or B or C, Or any nonempty subset of set of A and 8 andC. For instance,., in il lus trative example of a s.eidmw                 conjunctive phrases “at least one of A. B, and C” and “at least one of A, B and C” refer ts? any cl following sets: fAj, (B|, {0), ffoBf {A, C}, {B, C},. {A, B, Cj, Thus, such qqnhmetive language may not tie intended to imply that certain embodiments require at least one of A, at least onemf 8, and at 1 east one of C each to be present. In addition, unless oiherwise noted or contradicted by context,: a plurality indicates a state of being: plural folk a plurality of items indicates multiple items). A plurality is al least two items, but can be more when so indicated either explicitly or by context. Further, unless stated otherwise or otherwise clear from context, based on means basedat least in part on and mot. based solely on. JU■’ > j Opcmwiw of mocevwx raweibul hmm t-w be m-ror rax’'' mx \m foe uokx mlr-o otherwise indvated herein or other,vise ck-arh contradicted by tontext In at least one embodiment a process such a\ those processes described herein (or variations and-or eombinatioas thereof) is performed under conuol of one or more computer systems configured with executable instructions ami is implemented as code te g.. executable instructions, one or more computer programs or one or more applications) executing collectively on one or more processors, by hardware or combinations thereof bi at least one embodiment, code is stored on a computer-readable storage medi um, for exampl e, In form of a torn pa ter program mcl odiag a plurality of mairadw^ executable by one or more,processors. In at least one embodiment, a compoier-raa&W              is a nmMransitUry computer-readabw storage medium: that excludes transitory signals (6g„ a propagating transient electric or electromagnet!e transmission) but includes non-transitory data storage drcai-ty (e.g , buffers, cache, and queues) within transceivers of transitory signals. Jn at least one embodiment, code (e.g,, executable code or source code) is stored on a set of one or mare non-tumsnory cumputerreadahk storage media having stored thereon executable instructions (or other memora to store executable owl ructions) that, when executed (mat least one embodiment, as a result of being executed} by one or more processors of a computer system, cause computer sy stem to perform operations described herein A set of uon-transitury computer-readable storage media. in at least one embodiment, includes multiple nnratrass^ computer-readable storage media and one or more of individual non-transitory storage media rtf multiple non-traosiloty computer-readable storage media lack all of code while-multiple swmtransitory compato^ storage media collectively store ail of code. In at least one embodiment, executable instructions are executed such that different instruettons are executed by different processors. for example, aman-transitory computer-leadable storage medium store ihstructioris and a main central processing end r CPI   ext cutes sexne (rfmsW            a graphics process!ng unit (“CPU'') executes other instructions. In at least mie embodiment, different components of a computer system have separate processors and dhKaem processors execute different .subam^ oi msnuffens

[0324] Accordingly, in at least one embodiment, computer systems me configured to implement one or more services that singly or collectivel y perform opera daps of processes described herein and such computer systems are configmed with applicable hardware: and / or software that enable performance of operations. Further, a, computer system that implements at least one embodiment of present disclosure is a single device and. in another embodiment, isg distributed computer system ineluding multiple devices that operate differently such that distributed computer system, performs operations described herein and such that a single device docs not perform all operations.

[0325] Use of any and all examples, or exemplary language (e.g , “such, as”) provided herein, is imcnacu merely m better ilkmimim embodiments of disclosure and does not pose a limitation on scope of disclosure unless oihemtse claimed, Ntrlanguage specification should be cunsiruedas indicating any iion-claimed cleinern as essential to practice of disclosure

[0326] All references, including publications patent applications, and patents, cited herein are hereby incorporated by reference to same extent as if each reference were individually and specifically indicated to be incorporated by reference and wtee set forth. in its entirety herein.

[0327] In -desenpiiou and claims, terms “coupled’’ and “connected,** along with their derivatives, may be used. It should be understood that these terms may be not intended as sv «<»nx ms foi tach oihci Rather, in particular examples, "connected” or "coupled" may be used: to indicate that two ei more elements are in direct or indirect phvswal or-electrical contact with each other. ‘Too pled” may also mean that two or more-elements are not in direct contact wife each other, but yet still co-operate or interact with each other [(1328] Unless specihcaily stated otherwise,, it 'may be appreciated that throughout xpecifmahea reference.' io piocesjiug. compel ng. cdcitlaiiue. Jetcimmreg, the hkta relm io action and / or processes of a computer or computi ng system; or si mil ar electronic cc» t ‘ I m g device, that manipulate and / or transform data represented as physical, such as electronic, gaabtities Within computing systenO registers and / or memories, into other data similarly represented as physical quantities within computing system’ s memories, registers or other such i nformation storage, transm issi on , of di spl ay devices; [03 20] fo-a similar mw^, a processor may refer to any device or portion, of a device that processes electronic data from registers a:nd / of memory and transform that electronic data into other electronic data Him may be stored in registers and / or memory. As non-limiting examples, “proeasw” may be a CPU or a GPU, A “computing platform" may include one or more processors. As used herein, "software’ ' processes may include, for exam pie,: software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Also, each process may refer te multipte prucesv.w, fo$ cam uw cut womenmis in seque^ce or in parallel, cominuouslv or Intermittently.. Terms' svstem" and Aueihod" are used herein interchimgeabh owvim as sxsteui may embody »<ne oi mo.e inedb'ds .a u m-ihods may be considered a system

[0330] In present document, .references may be made to obtaining, acquiring, receiving or inputting analog or digital data into a subsystems computer system, or computerfemplemem^^^ machine Obtaining, acquiring, receh urn, or inputting analog and digital data can be accomplished irt a "Variety of ways such as by receiving data as a parameter of a function call or a cd to an application grogranw^            la some irn piemen tanoag process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transferring data via a serial or parallel interface. In another i mpl ementatiom procesy of obtaining, acquiring, .com mg, of mnunmu analog <p d.gsnd data ca*> be 'u.empKmeJ In nawb.^na 3.n t s. i \ ^mputernetwoti; from providing entity to'acquiring enti ty. References may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, process of providing, outputring, trama*riling, sending, or presenting analog or digital data can be accomplished by transferring data as an Input or output parameter of a function call, a parameter of an application programming. interlace or intemmeess eommmm mion nwchamsrn,

[0331] Although discussion above sets forth example implementations of described techniques, other architectures mav be used to implement described Rmctionaht), and are intended to be within scope of this disclosure, Furthermore, although specific distributions of respcmxibihties are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances.

[0332] Furthermore, although subject matter has been described in language specific to structami features wd / or methodological acts, it i s to be understood that subject matter claimed in appended claims is not necessarily limited to specific features or acts described, Rather, specific features and acts are disclosed as exethplary: forms of implementing the claims.

Claims

1. A datacenter cooling system, comprising:a cooling manifold located within a rack and comprising a first controllable fluid coupler to receive a coolant from a cooling loop external to the rack and one or more second controllable fluid couplers to distribute the coolant to one or more server cooling manifolds within server trays of the rack;one or more sensors including a temperature sensor to provide an input to a learning system; andthe learning system comprising at least a processor for executing a machine learning model to process the input using multiple neuron levels of the machine learning model having prior temperature values and having prior associated flow values to provide an output, and for communicating the output as control signals to the first controllable fluid coupler or the one or more second controllable fluid couplers to adjust a flow rate of the coolant flowing within the server trays of the rack based on sensor data from the one or more sensors.

2. The datacenter cooling system of claim 1, further comprising:the first controllable fluid coupler modifying a restriction to a flow of the coolant from the cooling loop or within the cooling manifold in response to the output.

3. The datacenter cooling system of claim 1, further comprising:a temperature sensor within the rack to provide an input to the learning system of the datacenter cooling system; andthe learning system executing a machine learning model to:process the input using multiple neuron levels of the machine learning model having prior temperature values and having prior associated flow values; andprovide an output to at least one of the one or more second controllable fluid couplers.

4. The datacenter cooling system of claim 3, whereinat least one of the one or more second controllable fluid couplers modifies at least one restriction to a flow of the coolant in the one or more server cooling manifolds within the server trays of the rack.

5. The datacenter cooling system of claim 1, further comprising:one or more temperature sensors associated with the rack or one or more of the server trays to provide at least one first input to the learning system of the datacenter cooling system;one or more flow sensors associated with the cooling loop, within the cooling manifold, or within the server trays to provide at least one second input to the learning system; andthe learning system executing a machine learning model to:process the first input and the second input using multiple levels of the machine learning model having prior temperature values and having associated prior flow values; andprovide an output to one or more of the first controllable fluid coupler or the one or more second controllable fluid couplers.

6. The datacenter cooling system of claim 1, wherein the one or more second controllable fluid couplers are adapted to receive mating couplers on at least one of the server trays by a press-fit connection.

7. The datacenter cooling system of claim 1, wherein the cooling manifold with the rack has at least one side coupler for coupling a neighboring rack in front-facing, sidefacing, or rear-facing direction.

8. The datacenter cooling system of claim 1, wherein the one or more server cooling manifolds extends within the server trays and comprise mating couplers to couple with device couplers from one or more devices in the server trays.

9. A method of cooling a datacenter, the method comprising:providing a cooling manifold within a rack;coupling a first controllable fluid coupler of the cooling manifold with a cooling loop external to the rack to receive a coolant; andcoupling one or more second controllable fluid couplers of the cooling manifold to one or more server cooling manifolds within server trays of the rack to distribute the coolant within the rack;providing an input to a learning system of a datacenter cooling system from one or more sensors including a temperature sensor within the rack;executing a machine learning model using the learning system to:process the input using multiple neuron levels of the machine learning model having prior temperature values and having prior associated flow values, andprovide an output; and;communicating, using the learning system comprising at least a processor, the output as control signals to one or more of the first controllable fluid couplers or the one or more second controllable fluid couplers to adjust a flow rate of the coolant flowing in the rack or the server trays of the rack based on sensor data from the one or more sensors.

10. The method of claim 9, further comprising:modifying a restriction to a flow of the coolant from the cooling loop or within the cooling manifold in response to the output.

11. The method of claim 9, further comprising:providing, from a temperature sensor within the rack, an input to the learning system of a datacenter cooling system; andexecuting a machine learning model using the learning system to:process the input using multiple neuron levels of the machine learning model having prior temperature values and having prior associated flow values; andprovide an output to at least one of the one or more second controllable fluid couplers.

12. The method of claim 11, further comprising:modifying at least one restriction to a flow of the coolant in at least one of the one or more server cooling manifolds within the server trays of the rack.

13. The method of claim 9, further comprising:providing at least one first input, from one or more temperature sensors associated with the rack or one or more of the server trays, to the learning system of a datacenter cooling system;29 07 241                  providing at least one second input, from one or more flow sensors associated2 with the cooling loop, within the cooling manifold, or within the server trays, to the learning3 system; and4                 executing a machine learning model using the learning system to:5                         process the first input and the second input using multiple levels of the6          machine learning model having prior temperature values and having associated prior flow7          values; and8                         provide an output to one or more of the first controllable fluid coupler or9          the one or more second controllable fluid couplers.10                 14. The method of claim 9, wherein the one or more second controllable fluid11 couplers are adapted to receive mating couplers on at least one of the server trays by a press-12 coupling action.13                  15. The method of claim 9, further comprising:14                  providing at least one side coupler for the cooling manifold within the rack to15 enable a coupling of the rack to a neighboring rack in front-facing, side-facing, or rear-facing16 direction.17                 16. The method of claim 9, wherein the one or more server cooling manifolds18 extends within the server trays and comprise mating couplers to couple with device couplers19 from one or more devices in the server trays.

Images