Chiplet-integrated machine learning accelerator

Abstract

Techniques are provided for performing machine learning operations, including configuring a first portion of a first chiplet as a cache, performing cache operations via the first portion, configuring at least a first sub-portion of the first portion of the chiplet as directly accessible memory, and performing machine learning operations on the first sub-portion with a machine learning accelerator in the first chiplet.

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 62 / 877,241, entitled "CHIPLET APPROACH FOR COUPLING GPU WITH MACHINE LEARNING ACCELERATION AT HIGH POWER EFFICIENCY," filed July 22, 2019, which is incorporated by reference in its entirety. This application claims priority to U.S. Provisional Patent Application No. 62 / 877,249, entitled "HIGH BW INTER-CONNECTED CHIPLETS AND GPU FOR HIGH PERFORMANCE GAMING AND MACHINE LEARNING WORKLOADS," filed July 22, 2019, which is incorporated by reference in its entirety. This application claims priority to U.S. Patent Application No. 16 / 933,863, entitled “CHIPLET-INTEGRATED MACHINE LEARNING ACCELERATORS,” filed July 20, 2020, the entire contents of which are incorporated by reference herein. [Background technology]

[0002] Machine learning is a rapidly evolving field, with constant improvements in hardware for machine learning operations such as training and inference.

[0003] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0004] [Figure 1] 1 is a block diagram of an example device capable of implementing one or more features of the present disclosure. [Diagram 2] FIG. 2 illustrates details of the device of FIG. 1, according to an example. [Diagram 3]FIG. 3 is a block diagram showing further details of the graphics processing pipeline shown in FIG. [Figure 4] FIG. 2 illustrates a block diagram of an APD, showing details of a cache / machine learning accelerator chiplet, according to one example. [Diagram 5] A diagram illustrating details of a cache / machine learning accelerator chiplet, according to an example. [Figure 6] FIG. 1 is a flow diagram of a method for performing machine learning operations using chiplets, according to an example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Techniques are provided for performing machine learning operations that include configuring a first portion of a first chiplet as a cache, performing cache operations via the first portion, configuring at least a first sub-portion of the first portion of the chiplet as directly accessible memory, and performing machine learning operations on the first sub-portion with a machine learning accelerator in the first chiplet.

[0006] 1 is a block diagram of an example device 100 capable of implementing one or more features of the present disclosure. The device 100 may be, for example, but not limited to, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a tablet computer, or any other computing device. The device 100 includes a processor 102, a memory 104, a storage 106, one or more input devices 108, and one or more output devices 110. The device 100 also includes one or more input drivers 112 and one or more output drivers 114. Any input driver 112 may be embodied as hardware, a combination of hardware and software, or software, and serves the purpose of controlling the input device 108 (e.g., controlling operation, receiving input from the input driver 112, and providing data to the input driver 112). Similarly, any output driver 114 may be embodied as hardware, a combination of hardware and software, or software to serve the purpose of controlling an output device (e.g., controlling operation, receiving input from, and providing data to, output driver 114). It should be understood that device 100 may include additional components not shown in FIG.

[0007] In various alternatives, the processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, the memory 104 is located on the same die as the processor 102 or is located separately from the processor 102. The memory 104 includes volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0008] Storage 106 includes fixed or removable storage (e.g., but not limited to, a hard disk drive, a solid state drive, an optical disk, a flash drive). Input devices 108 include, but are not limited to, a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals). Output devices 110 include, but are not limited to, a display, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals).

[0009] The input driver 112 and the output driver 114 include one or more hardware, software, and / or firmware components configured to interface with and drive the input device 108 and the output device 110, respectively. The input driver 112 communicates with the processor 102 and the input device 108, allowing the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, allowing the processor 102 to send output to the output device 110. The output driver 114 includes an Accelerated Processing Device (APD) 116 coupled to a display device 118, which in some examples is coupled to the display device 118 (a physical display device or a simulated device that uses a remote display protocol to display output). The APD 116 is configured to accept computational and graphics rendering commands from the processor 102, process the computational and graphics rendering commands, and provide pixel output to the display device 118 for display. As described in further detail below, APD 116 includes one or more parallel processing units configured to perform computations according to the Single-Instruction Multiple-Data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in combination with APD 116, in various alternatives, the functions described as being performed by APD 116 are not driven by a host processor (e.g., processor 102), but are additionally or alternatively performed by other computing devices having similar functionality that are configured to provide graphical output to display device 118. For example, it is contemplated that any processing system that performs processing tasks according to the SIMD paradigm may be configured to perform the functions described herein.Alternatively, it is contemplated that computing systems that do not perform processing tasks according to the SIMD paradigm may perform the functions described herein.

[0010] FIG. 2 illustrates details of device 100 and APD 116, according to one example. Processor 102 (FIG. 1) executes operating system 120, drivers 122, and applications 126, and may alternatively or additionally execute other software. Operating system 120 controls various aspects of device 100, such as managing hardware resources, handling service requests, scheduling and controlling process execution, and performing other operations. APD driver 122 controls the operation of APD 116, sending tasks, such as graphics rendering tasks or other work, to APD 116 for processing. APD driver 122 also includes a just-in-time compiler that compiles programs to be executed by processing components of APD 116 (such as SIMD unit 138, described in more detail below).

[0011] The APD 116 executes commands and programs for selected functions, such as graphics and non-graphics operations that may be suitable for parallel processing. The APD 116 may be used to perform graphics pipeline operations, such as pixel operations, geometric calculations, and rendering of images to the display device 118, based on commands received from the processor 102. The APD 116 also performs computational operations not directly related to graphics operations, such as operations related to video, physics simulation, computational fluid dynamics, or other tasks, based on commands received from the processor 102. In some examples, these computational operations are performed by executing computational shaders on the SIMD units 138.

[0012] The APD 116 includes computational units 132, which include one or more SIMD units 138 configured to execute operations in parallel according to the SIMD paradigm at the request of the processor 102 (or another unit). The SIMD paradigm is a paradigm in which multiple processing elements share a single program control flow unit and program counter, thereby executing the same program, but with different data. In one example, each SIMD unit 138 includes 16 lanes, each lane executing the same instruction simultaneously with other lanes in the SIMD unit 138, but capable of executing the instruction with different data. When not all lanes need to execute a given instruction, a lane can be turned off by prediction. Prediction can also be used to execute programs with branching control flows. More specifically, for programs with instructions such as conditional branches where the control flow is based on calculations performed in the individual lanes, the lane corresponding to the currently not executed control flow path can be predicted, and different control flow paths can be executed consecutively, allowing arbitrary control flow.

[0013] The basic unit of execution in the compute units 132 is the work item. Each work item represents a single instantiation of a program that executes in parallel on a particular lane. Work items may execute simultaneously (or partially simultaneously, partially sequentially) on a single SIMD processing unit 138 as a "wavefront." One or more wavefronts are included in a "workgroup," which includes a collection of work items designated to execute the same program. A workgroup may be executed by executing each of the wavefronts that make up the workgroup. In another way, a wavefront may execute on a single SIMD unit 138, or on different SIMD units 138. A wavefront may be thought of as the largest collection of work items that can execute simultaneously (or pseudo-simultaneously) on a single SIMD unit 138. "Pseudo-simultaneous" execution occurs in the case of wavefronts that are larger than the number of lanes of the SIMD unit 138. In such a situation, a wavefront executes in multiple cycles, with different collections of work items executing in different cycles. The APD scheduler 136 is configured to perform operations related to scheduling various workgroups and wavefronts on the compute units 132 and the SIMD units 138 .

[0014] The parallelism provided by the compute units 132 is well suited for graphics-related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus, in some cases, the graphics pipeline 134, which accepts graphics processing commands from the processor 102, provides computational tasks to the compute units 132 for parallel execution.

[0015] Computation units 132 are also used to perform computational tasks that are not related to graphics or that are not performed as part of the "normal" operation of graphics pipeline 134 (e.g., custom operations performed to complement the processing performed for the operation of graphics pipeline 134). Applications 126 or other software executing on processor 102 send programs defining such computational tasks to APD 116 for execution.

[0016] Figure 3 is a block diagram illustrating additional details of the graphics processing pipeline 134 shown in Figure 2. The graphics processing pipeline 134 includes stages that each perform a particular function of the graphics processing pipeline 134. Each stage may be implemented partially or fully as a shader program executing on the programmable compute unit 132, or may be implemented partially or fully as fixed-function, non-programmable hardware external to the compute unit 132.

[0017] The input assembler stage 302 reads primitive data from a user-filled buffer (e.g., a buffer filled at the request of software executed by the processor 102, such as application 126) and assembles the data into primitives for use by the rest of the pipeline. The input assembler stage 302 can generate different types of primitives based on the primitive data contained in the user-filled buffer. The input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

[0018] The vertex shader stage 304 processes the vertices of the primitives assembled by the input assembler stage 302. The vertex shader stage 304 performs various per-vertex operations such as transformation, skinning, morphing, per-vertex lighting, etc. Transformation operations include various operations for transforming the coordinates of the vertices. These operations include one or more of the following: modeling transformations, view transformations, projection transformations, perspective division, and viewport transformations that modify vertex coordinates, as well as other operations that modify non-coordinate attributes.

[0019] The vertex shader stage 304 is partially or fully implemented as a vertex shader program that executes on one or more compute units 132. The vertex shader program is provided by the processor 102 and is based on a program pre-generated by a computer programmer. The driver 122 compiles such a computer program to generate a vertex shader program having a format suitable for execution on the compute units 132.

[0020] The hull shader stage 306, tessellator stage 308, and domain shader stage 310 work together to perform tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader stage 306 generates patches for tessellation based on input primitives. The tessellator stage 308 generates a set of samples for the patch. The domain shader stage 310 calculates vertex positions for vertices that correspond to the samples of the patch. The hull shader stage 306 and domain shader stage 310, like the vertex shader stage 304, can be implemented as shader programs compiled by the driver 122 and executed on the compute unit 132.

[0021] The geometry shader stage 312 performs vertex operations on a per-primitive basis. The geometry shader stage 312 can perform a variety of different types of operations, including operations such as point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, per-primitive material settings, etc. In some cases, geometry shader programs compiled by the driver 122 and executed on the compute unit 132 perform the operations of the geometry shader stage 312.

[0022] The rasterizer stage 314 accepts and rasterizes simple primitives (triangles) that have been generated upstream of the rasterizer stage 314. Rasterization involves determining which screen pixels (or sub-pixels) are covered by a particular primitive. Rasterization is performed by fixed function hardware.

[0023] The pixel shader stage 316 calculates output values ​​for screen pixels based on the upstream generated primitives and the results of rasterization. The pixel shader stage 316 can apply textures from texture memory. The operations of the pixel shader stage 316 are performed by pixel shader programs compiled by the driver 122 and executed on the compute unit 132.

[0024] The output merger stage 318 accepts the outputs from the pixel shader stage 316, merges these outputs into the frame buffer, and performs operations such as z-testing and alpha blending to determine the final color of the screen pixels.

[0025] Implementations of APD 116 are disclosed that include a graphics processing pipeline 134 and can perform graphics rendering. However, the teachings of this disclosure extend to implementations of APD 116 that do not include a graphics processing pipeline 134 or that do not utilize such a pipeline to perform graphics rendering.

[0026] 4 depicts a block diagram of the APD 116, showing details of a cache / machine learning accelerator chiplet 404, according to an example. The APD 116 includes the APD scheduler 136 and the compute unit 132 described with respect to FIG. 2. The APD 116 also includes one or more cache and machine learning accelerator chiplets 404 that are coupled to an APD core 402 via an APD-to-cache interface 406 and coupled to other memory (e.g., the system memory 104 or memory of the APD 116) via an external interface 410. In some embodiments, one or more chiplets 404 are connected to one or more other chiplets 404 via one or more intra-chiplet interfaces 408.

[0027] The cache / machine learning accelerator chiplet 404 includes a memory module configured to store data and a machine learning accelerator. In some embodiments, the machine learning accelerator includes a matrix multiplication circuit configured to perform matrix multiplication on an input matrix to provide an output result.

[0028] In some embodiments, the cache / machine learning accelerator chiplet 404 is a separate physical die from the APD core 402. In some embodiments, the cache / machine learning accelerator chiplet 404 is manufactured in a larger scale manufacturing process than the manufacturing process used for the APD core 402. The manufacturing process refers to the scale at which device features are manufactured. The manufacturing process may be referred to in the art as a "process node." Some exemplary manufacturing processes include 10 nanometer (nm) processes and 7 nm processes. By using a larger manufacturing process scale for the chiplet 404 compared to the APD core 402, the chiplet 404 can be manufactured at a lower cost and with a higher yield compared to the APD core 402 while still providing the high performance of the APD core 402.

[0029] The memory modules of the cache / machine learning accelerator chiplet 404 are switchable between being used as cache memory for the operation of the APD core 402 and being used as memory to store input operands and output results for the operation of the machine learning accelerator. More specifically, the cache / machine learning accelerator chiplet 404 is configurable between operating as a cache memory for the APD core 402 and operating as a directly accessible memory accessible, for example, by the machine learning accelerator 404 of the cache / machine learning accelerator chiplet. In some embodiments, either or both of the APD scheduler 136 and the compute unit 132 can direct any portion of the cache / machine learning accelerator chiplet 404 to operate as a cache or as a directly accessible memory.

[0030] In some embodiments, the APD core 402 includes one or more cache memories that form at least a portion of a cache hierarchy. The memory hierarchy also includes a cache memory of the cache / machine learning accelerator chiplet 404. In some examples, the cache memory of the cache / machine learning accelerator chiplet 404 serves as a level 3 cache for a portion of the cache hierarchy of the APD core 402.

[0031] In some embodiments, the cache / machine learning accelerator chiplet 404 also serves as a physical interface between the APD core 402 and memory that is higher in the memory hierarchy than the cache hierarchy (memory dedicated to the APD 116 or system memory 104). In other words, the cache / machine learning accelerator chiplet 404 serves as one level of the cache hierarchy and includes memory that physically interfaces with other levels of the hierarchy (including lower levels of the APD core 402 and higher levels, such as memory in the APD 116 or system memory 104). Note that FIG. 4 shows that the external interface 410 is connected "to the memory." In various examples, the "memory" referred to is general-purpose (e.g., non-cached) memory of the APD 116 or system memory 104. Thus, the cache / machine learning accelerator chiplet 404 serves as a physical interface between the memory and portions of the cache hierarchy in the APD core 402.

[0032] 5 illustrates details of a cache / machine learning accelerator chiplet 404, according to an example. As shown, the cache / machine learning accelerator chiplet 404 includes multiple machine learning accelerators 502 and a chiplet memory 504. In some embodiments, the machine learning accelerators 502 are hardware circuits configured to perform matrix multiplication operations.

[0033] Matrix multiplication operations are commonly used in machine learning operations, such as performing operations to generate layer outputs from layer inputs for fully connected layers and convolutional layers. In various examples, either the APD scheduler 136 or the compute unit 132, or both, can send commands to either the cache / machine learning accelerator chiplet 404 to fetch data into the chiplet memory 504 and perform the matrix multiplication operation on the fetched data via the machine learning accelerator 502 to output the result. In various examples, the cache / machine learning accelerator chiplet 404 stores the result of the matrix multiplication in the chiplet memory 504. In various examples, the cache / machine learning accelerator chiplet 404 transmits the result to an external entity, such as the APD core 402, the memory of the APD 116, or the memory 104.

[0034] In some examples, the neural network is implemented as a series of interconnected layers. Each layer receives one or more inputs from different layers or inputs to the neural network. The computations of the different layers can be performed by different entities of the device 100. In one example, the cache / machine learning accelerator chiplet 404 performs matrix multiplication or convolution operations, and the APD core 402 (e.g., the compute unit 132) performs other computations to implement the neural network, such as activation, batch normalization, or other operations. In some examples, a coordinator, such as the APD scheduler 136 or the processor 102, instructs these different entities to perform various operations to perform training or inference on the neural network. For example, the coordinator instructs the cache / machine learning accelerator chiplet 404 to perform a matrix multiplication operation on input data for a layer that requires a matrix multiplication, and instructs the compute unit 132 to perform other operations on the neural network for a layer that utilizes such other operations.

[0035] The APD scheduler 136 can schedule many different tasks for execution simultaneously on different compute units 132 and cache / machine learning accelerator chiplets 404. In one example, the APD scheduler 136 can schedule shader programs for execution on the compute units 132 while scheduling operations for execution on the cache / machine learning accelerator chiplets 404. As shown in FIG. 5, the chiplet memory 504 can be configured between memory configured as a cache 506 and directly accessible memory 508. More specifically, an entity such as the processor 102, the APD scheduler 136, or the compute units 132 may request that a particular portion of the chiplet memory 504 of a particular cache / machine learning accelerator chiplet 404 be configured as a cache 506 or as directly accessible memory 508. In response, the cache / machine learning accelerator chiplet 404 configures the requested portion as cache 506 or directly accessible memory 508 and configures the remaining portion as the other of cache 506 or directly accessible memory 508.

[0036] The memory configured as the cache 506 functions as a general cache memory. Specifically, the cache 506 functions as a higher level in the cache hierarchy than the cache of the APD core 402. In one example, the memory configured as the cache 506 functions as a level 3 cache memory, where the APD core 402 includes one or more level 0 caches, one or more level 1 caches, and one or more level 2 caches. In such an example, the level 3 cache memory processes misses from the level 2 caches, receives and stores cache lines evicted from one or more level 2 caches, and evicts cache lines to a backing memory, such as a memory in the APD 116 or the system memory 104. In some examples, the cache memory functions as a cache for shader programs executing in the compute units 132 of the APD 116. It should be noted that the memory configured as the cache 506 is not "directly accessible" in the sense that an execution unit, such as the machine learning accelerator 502 or the compute units 132, cannot specifically request data to be placed in such a cache 506. For example, in a conventional memory, an execution unit may request that data be placed at an address in that conventional memory, but in a cache, data is placed in the cache by a cache controller in response to an action such as a miss in the cache, and the execution unit only has indirect control over the data stored in the cache.

[0037] In contrast, directly accessible memory 508 is directly accessible by an execution unit. The term “directly accessible” means that an execution unit, such as APD scheduler 136, compute unit 132, or machine learning accelerator 502, can explicitly request data to be stored in or loaded from directly accessible memory 508. In some embodiments, these requests specify a particular cache / machine learning accelerator chiplet 404 to store the data or read the data from, as well as an address within that cache / machine learning accelerator chiplet 404. As described elsewhere, machine learning accelerator 502 can, and sometimes does, perform machine learning operations, such as matrix multiplication, that consume data in directly accessible memory 508 of the same chiplet 404 and output the results of the operation to directly accessible memory 508 of the same chiplet 404.

[0038] In some embodiments, the chiplets 404 include inter-chiplet connections 408. As described elsewhere herein, the chiplets 404 obtain data from other sources and write data to other locations. In one example, the chiplets 404 perform operations to generate output that is consumed by different chiplets 404. In embodiments that include inter-chiplet connections 408, the chiplets 404 can send or receive such data directly to or from other chiplets 404.

[0039] As described elsewhere herein, operations of the chiplets 404 and APD cores 402 are performed for training or inference of the machine learning network. In some examples, a graph compiler (not shown) compiles a graph description of the machine learning network that indicates the layers of the network, the operations of each layer, the inputs of each layer, and the outputs of each layer. The inputs of any layer may be the outputs of a different layer or inputs to the network, and the outputs of any layer may be the inputs of a different layer or output of the network. The graph compiler generates a set of operations to be executed by the machine learning accelerator 502 of the chiplet 404, and in some embodiments, by the APD scheduler 136, and in some embodiments, by the compute units 132. In some embodiments, one or more shader programs include instructions for performing the operations of one or more layers. In some embodiments, some such shader programs include instructions that request the machine learning accelerator 502 to perform matrix multiplication operations required for such layers, and optionally include instructions to send data to the chiplet memory 504 configured as directly accessible memory 508 for input to the layers. In some embodiments, some such shader programs include instructions to move data from directly accessible memory 508 to a different memory, such as a directly accessible memory 508 of a different chiplet 404 or memory within the APD core 402. In some embodiments, the APD scheduler 136 performs operations to request a chiplet 404 to perform a machine learning accelerator operation and / or to read data from or write data to the chiplet 404 instead of or in addition to the compute units 132.

[0040] In some embodiments, chiplet 404 includes a direct memory access engine configured to read data into directly accessible memory 508 and / or store data from directly accessible memory 508 to another memory. In various alternative embodiments, compute unit 132 or APD scheduler 136 instructs the direct memory access engine to read data and / or write data.

[0041] As discussed above, the chiplet memory 504 can be configured between a cache 506 and a directly accessible memory 508. It should be understood that the chiplet memory 504 can be configured such that a first portion is a cache memory 506, and then at least a portion of the first portion is configured to be a directly accessible memory 508. In other words, a chiplet memory 404 once used as a cache memory 506 can be reused as a directly accessible memory 508. Similarly, a chiplet memory 404 once used as a directly accessible memory 508 can be reused as a cache memory 506. It should also be understood that different portions of the same chiplet 404 configured as a cache and different portions of the same chiplet 404 configured as directly accessible can be used simultaneously. For example, it is possible to perform machine learning operations, such as matrix multiplication, on one chiplet 404 while simultaneously performing cache operations for the APD 116.

[0042] 6 is a flow diagram of a method 600 for performing machine learning operations using chiplets 404, according to an example. Although described with respect to the systems of FIGS. 1-5, one skilled in the art will understand that any system configured to perform the steps of method 600 in any technically feasible order is within the scope of the present disclosure.

[0043] The method 600 begins at step 602, where a chiplet 404 configures a first portion of a chiplet memory 504 as a cache 506. In various examples, this configuration occurs at the request of the APD scheduler 136 or the compute unit 132.

[0044] In step 604, the APD 116 performs cache operations using the first portion configured as the cache 506. The cache operations include storing cache lines evicted from the cache in the APD core 402 and providing the cache lines in response to requests to the APD core 402.

[0045] In step 606, the chiplet 404 configures at least a first sub-portion of the first portion of the chiplet 404 as directly accessible memory 508. In various examples, this configuration occurs at the request of the APD scheduler 136 or the compute unit 132. In step 608, the chiplet 404 performs a machine learning operation using the first sub-portion of the first portion of the chiplet 404 configured as directly accessible. In various examples, performing the machine learning operation includes performing a matrix multiplication on a layer of the machine learning network to obtain a result of the layer. In various examples, the operation includes storing data in the first sub-portion and transmitting data from the first sub-portion to an entity external to the chiplet 404, such as another chiplet 404 and the APD core 402.

[0046] Each unit shown in the figure represents hardware circuitry configured to perform the operations described herein, and certain units of graphics processing pipeline 300 are programmable and therefore capable of executing programs.

[0047] It should be understood that many variations are possible based on the disclosure herein, and although features and elements have been described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0048] The provided methods may be implemented in a general purpose computer, processor, or processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine. Such processors may be manufactured by configuring a manufacturing process using results of processing hardware description language (HDL) instructions and other intermediate data (instructions storable on a computer readable medium) including a netlist. Such processing results may be a mask work, which is used in a semiconductor manufacturing process to manufacture a processor implementing features of the present disclosure.

[0049] The methods or flow charts provided herein may be implemented in a computer program, software, firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. configuring a first portion of a first chiplet as a cache; performing a cache operation over the first portion; configuring at least a first sub-portion of the first portion of the first chiplet as directly accessible memory; and performing, by a machine learning accelerator in the first chiplet, a machine learning operation on the first subportion. method.

2. performing the cache operation includes performing the cache operation for a processing core on a die other than the first chiplet.

2. The method of claim 1.

3. performing a cache operation for the processing core includes one or more of: storing an evicted cache line from a cache of the processing core; and providing a cache line to the processing core in response to a miss in the cache of the processing core. The method of claim 2.

4. configuring the first portion as a cache or configuring the first sub-portion as a directly accessible memory is performed in response to a request from a scheduler of a processing core or a compute unit on a die other than the first chiplet.

2. The method of claim 1.

5. and storing data in the first subportion configured as a directly accessible memory in response to a request of a processor core other than the first chiplet.

2. The method of claim 1.

6. performing the machine learning operations includes performing machine learning operations that consume the data as an input. The method of claim 5.

7. The machine learning operations include matrix multiplication operations.

2. The method of claim 1.

8. the first portion includes a first amount of internal memory of the first chiplet; The method further includes performing machine learning operations on a second portion of the internal memory configured as directly accessible memory while performing the cache operations via the first portion.

2. The method of claim 1.

9. and further comprising: sending data to or receiving data from a second chiplet that is physically separate from a processing core requesting the first chiplet to perform the machine learning operation, the data being sent or received via a direct connection between the first chiplet and the second chiplet that does not flow through the processing core.

2. The method of claim 1.

10. one or more machine learning accelerators; a chiplet memory; The one or more machine learning accelerators and the chiplet memory include: configuring a first portion of the chiplet memory as a cache; performing a cache operation over the first portion; configuring at least a first sub-portion of the first portion of the chiplet memory as directly accessible memory; executing a machine learning operation on the first subportion by a machine learning accelerator of the one or more machine learning accelerators; [0023] device.

11. performing the cache operation includes performing the cache operation for a processing core on a die separate from the chiplet memory. The device of claim 10.

12. performing a cache operation for the processing core includes one or more of: storing an evicted cache line from a cache of the processing core; and providing a cache line to the processing core in response to a miss in the cache of the processing core.

12. The method of claim 11.

13. configuring the first portion as a cache or configuring the first sub-portion as a directly accessible memory is performed in response to a request from a scheduler of a processing core or a compute unit on a die separate from the chiplet memory. The device of claim 10.

14. the chiplet memory is further configured to store data in the first sub-portion configured as a directly accessible memory upon request of a processor core separate from the chiplet memory. The device of claim 10.

15. performing the machine learning operations includes performing machine learning operations that consume the data as an input.

15. The method of claim 14.

16. The machine learning operations include matrix multiplication operations. The device of claim 10.

17. the first portion includes a first amount of the chiplet memory of a first chiplet; the one or more machine learning accelerators are configured to perform machine learning operations on a second portion of the chiplet memory configured as a directly accessible memory while cache operations are performed via the first portion. The device of claim 10.

18. The one or more machine learning accelerators and the chiplet memory are further configured to send data to or receive data from a second chiplet that is physically separate from a processing core requesting the first chiplet to perform a machine learning operation, wherein the data is sent or received via a direct connection between the first chiplet and the second chiplet that does not flow through the processing core. The device of claim 10.

19. a first chiplet including a first chiplet memory and a first set of one or more machine learning accelerators; a second chiplet; and a processing core; The first chiplet comprises: configuring a first portion of the first chiplet memory as a cache; performing a cache operation over the first portion; configuring at least a first sub-portion of the first portion of the first chiplet memory as directly accessible memory; executing a machine learning operation on the first subportion by a machine learning accelerator of the one or more machine learning accelerators; [0023] device.

20. performing the cache operation includes performing the cache operation on the processing core.

20. The device of claim 19.

Images