Compression for deep learning in case of sparse value mapping to non-zero values

Abstract

Embodiments described herein provide a processing device comprising compute logic to generate neural network data for a convolutional neural network (CNN) and to write the neural network data to a memory buffer. The compute logic additionally comprises a direct memory access (DMA) controller comprising a hardware codec having an encoding unit and a decoding unit, the DMA controller to: read the neural network data from the memory buffer; encode the neural network data via the encoding unit; write encoded neural network data to a memory device coupled with the processing device; write metadata for the encoded neural network data to the memory device coupled with the processing device; and decode encoded neural network data via the decoding unit in response to a request from the compute logic.

Description

TECHNICAL FIELD

[0001] Embodiments relate generally to logic for performing processing operations using a general purpose graphics processing unit. More specifically, embodiments relate to encoding and decoding of feature maps and weights of a convolutional neural network. BACKGROUND

[0002] A neural network can be modeled as a collection of neurons connected in a non- cyclic graph fashion. A neural network can receive an input (a single vector) and transform it through a series of hidden layers. Each hidden layer consists of a set of neurons, where each neuron is fully connected to all neurons in the previous layer, and where neurons in a single layer act completely independently and do not share any connections. The last fully connected layer is called the "output layer" and in a classification setting it represents the classification scores. A convolutional neural network (CNN) is similar to a standard neural network. Each neuron receives some inputs, performs a dot product and optionally follows by a non-linearity. However, a CNN is explicitly tailored to handle input image data. Based on the raw image data input, the network outputs classification scores for the input data.

[0003] A CNN topology handles large amounts of data. Ideally, such data is processed in on-chip memory. Weight matrix kernel data can typically consume memory in the order of several hundred megabytes. Additionally, each layer in a CNN can produce large amounts of data in the form of output feature maps. During operation of a computing system implementing a CNN, kernel data can be read from system main memory. Output feature maps (OFMs) of a CNN layer can also be stored in main memory, where they are read as input feature maps (IFMs) of the next layer. Since large amounts of on-chip data are processed by a computing system executing a CNN, significant power can be spent on reading and writing CNN data. BRIEF DESCRIPTION OF DRAWINGS

[0004] In order to enable a detailed understanding of the above-mentioned features of the embodiments of the present application, a more particular description of the application briefly described above will be rendered by reference to embodiments illustrated in the drawings. It is to be noted that the appended drawings are not intended to be exhaustive or limiting of the application, but rather to be illustrative only.

[0005] FIG. 1 is a block diagram of a processing system according to an embodiment;

[0006] FIG. 2 is a block diagram of a processor according to an embodiment;

[0007] FIG. 3 is a block diagram of a graphics processor according to an embodiment;

[0008] FIG. 4is a block diagram of a graphics processing engine of a graphics processor according to some embodiments;

[0009] FIG. 5 is a block diagram of hardware logic of a graphics processor core according to some embodiments described herein.

[0010] FIGS. 6A-6B thread execution logic including an array of processing elements employed in a graphics processor core is shown according to embodiments described herein.

[0011] FIG. 7 is a block diagram showing a graphics processor instruction format according to some embodiments;

[0012] FIG. 8 is a block diagram of a graphics processor according to another embodiment.

[0013] FIGS. 9A-9B graphics processor command format and command sequence are shown according to some embodiments;

[0014] FIG. 10 an exemplary graphics software architecture of a data processing system is shown according to some embodiments;

[0015] FIG. 11A is a block diagram showing an IP core development system for use in designing and developing IP cores according to embodiments;

[0016] FIG. 11B is a cross-sectional side view of an integrated circuit package assembly according to some embodiments described herein;

[0017] FIG. 12 is a block diagram showing an exemplary system on a chip integrated circuit according to embodiments;

[0018] FIGS. 13A-13B is a block diagram showing an exemplary graphics processor for use within a SoC according to embodiments described herein;

[0019] FIGS. 14A-14B additional exemplary graphics processor logic is shown according to embodiments described herein;

[0020] FIG. 15 is an illustration of a primitive of a convolutional neural network;

[0021] FIGS. 16A-16B is an illustration of a conventional implementation of a convolutional neural network;

[0022] FIGS. 17A-17B a memory layout for feature map and kernel data is shown according to embodiments;

[0023] FIG. 18An example of a proposed compressed memory layout relative to a memory layout of raw data having according to embodiments is shown;

[0024] FIG. 19 A computing architecture configured to enable compressed transmission of CNN data according to embodiments is shown;

[0025] FIG. 20 A bitstream header according to embodiments is shown;

[0026] FIG. 21 A table encoded bitstream layout according to embodiments is shown;

[0027] FIG. 22 Various layouts of bitstream encoding according to embodiments are shown;

[0028] FIGS. 23-26 Decoding of an example bitstream is shown;

[0029] FIG. 27 is a flow diagram showing a method of performing a CNN operation according to embodiments;

[0030] FIG. 28 is a flow diagram of a method of encoding CNN data according to embodiments;

[0031] FIG. 29 Compression using a validity map compression mode according to additional embodiments is shown;

[0032] FIG. 30 Compression using a unique value coordinate compression mode according to additional embodiments is shown;

[0033] FIG. 31 Encoding and decoding operations with separate metadata according to embodiments are shown;

[0034] FIG. 32 An example codec system according to embodiments is shown;

[0035] FIG. 33 Compression of machine learning data having a single high frequency value according to embodiments described herein is shown;

[0036] FIG. 34 is a flow diagram showing compression for deep learning where sparse values map to non-zero values according to embodiments; and

[0037] FIG. 35 is a block diagram of a computing device according to embodiments. DETAILED DESCRIPTION

[0038] The embodiments described herein provide a CNN processing system including automatic and lossless data compression on feature map memory operations. A CNN-optimized system-on-chip integrated circuit is provided in which feature map writes to memory are automatically and losslessly compressed while feature map reads are automatically decompressed. This automatic compression and decompression preserves memory bus bandwidth and reduces system memory access power requirements when performing CNN operations. Additional embodiments described herein extend neural network compression techniques to support an additional compressor mechanism for mapping high frequency values to zero. This additional compression mechanism enables hardware support for low-precision matrix multiplication libraries that map zero values to non-zero values. These techniques can be implemented within a hardware module that implements automatic, run-time compression of machine learning data.

[0039] For explanatory purposes, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments described below. However, it will be apparent to one skilled in the art that embodiments can be practiced without some or all of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the basic underlying principles of the embodiments, and to provide a more thorough understanding of the embodiments. Although some of the following embodiments are described with reference to graphics processors, the techniques and teachings described herein can be applied to various types of circuitry or semiconductor devices, including general purpose processing devices or graphics processing devices. Reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0040] In the following description and claims, the terms "coupled" and "connected," along with their derivatives, can be used. It should be understood that these terms are not intended as synonyms for each other. "Coupled" can be used to indicate that two or more elements, which can or can not be in direct physical or electrical contact with each other, co-operate or interact with each other. "Connected" can be used to indicate the establishment of communication between two or more elements that are coupled with each other.

[0041] In the following description, FIGS. 1-1 4 An overview of exemplary data processing systems and graphics processor logic in conjunction with which various embodiments can be implemented is provided. FIGS. 15-35Specific details of various embodiments are provided. Some aspects of the following embodiments are described with reference to graphics processors, while other aspects are described in terms of general purpose processors, such as central processing units (CPUs). Similar techniques and teachings can be applied to other types of circuitry or semiconductor devices, including without limitation one or more instances of an integrated many-core processor, a GPU cluster, or a field programmable gate array (FPGA). In general, the teachings are applicable to any processor or machine that manipulates or processes images (e.g., samples, pixels), vertex data, or geometry data.

[0042] System Overview

[0043] FIG. 1 is a block diagram of a processing system 100 in accordance with an embodiment. In embodiments, system 100 includes one or more processors 102 and one or more graphics processors 108, and can be a single processor desktop system, a multiprocessor workstation system, or a server system that includes a large number of processors 102 or processor cores 107. In one embodiment, system 100 is a processing platform incorporated within a chip multiprocessor (CMP) integrated circuit that is used in a mobile device, handheld device, or embedded device.

[0044] In one embodiment, system 100 can include or be incorporated within a server-based gaming platform, a game console, including a games and media console, a mobile gaming console, a handheld game console, or an online game console. In some embodiments, system 100 is a mobile phone, a smart phone, a tablet device, or a web appliance. Processing system 100 can also include a wearable device, such as a smart watch wearable device, smart glass device, augmented reality device, or virtual reality device, coupled with, or integrated within, the wearable device. In some embodiments, processing system 100 is a television or set-top box device having one or more processors 102 and a graphical interface generated by one or more graphics processors 108.

[0045] In some embodiments, the one or more processors 102 each include one or more processor cores 107 for processing instructions that, when executed, perform the operations of system and user software. In some embodiments, each of the one or more processor cores 107 is configured to process a specific instruction set 109. In some embodiments, instruction set 109 can facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via a very long instruction word (VLIW). Multiple processor cores 107 can each process a different instruction set 109, which can include instructions to facilitate the emulation of other instruction sets. Processor cores 107 can also include other processing devices, such as digital signal processors (DSPs).

[0046] In some embodiments, the processor 102 includes cache memory 104. Depending upon the architecture, the processor 102 can have a single internal cache or multiple levels of internal caches. In some embodiments, the cache memory is shared among the components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a level 3 (L3) cache or last level cache (LLC)) (not shown), which can be shared among the processor cores 107 using known cache coherency techniques. Additionally, a register file 106 is included in the processor 102, which can include different types of registers for storing different types of data (e.g., integer registers, floating point registers, status registers, and instruction pointer registers). Some registers can be general registers, while others can be specific to the design of the processor 102.

[0047] In some embodiments, the one or more processors 102 are coupled with one or more interface buses 110 for transmitting communication signals between the processor 102 and other components in the system 100, such as address, data, or control signals. In one embodiment, the interface bus 110 can be a processor bus, such as a version of the direct media interface (DMI) bus. However, the processor bus is not limited to the DMI bus and can include one or more peripheral component interconnect buses (e.g., a PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor(s) 102 include an integrated memory controller 116 and platform controller hub 130. The memory controller 116 facilitates communication between the memory devices and other components of the system 100, while the platform controller hub (PCH) 130 provides connections to I / O devices via a local I / O bus.

[0048] The memory device 120 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase- change memory device, or some other memory device having suitable performance as a processing memory. In one embodiment, the memory device 120 can operate as system memory for the system 100, for storing data 122 and instructions 121 for use when the one or more processors 102 execute an application or process. The memory controller 116 is also coupled to an optional external graphics processor 112, which can communicate with the one or more graphics processors 108 in the processors 102 to perform graphics and media operations. In some embodiments, a display device 111 can be coupled to the processor(s) 102. The display device 111 might be an internal device, such as in a mobile electronic device or a laptop device, or an external device, such as a monitor or a television. In one embodiment, the display device 111 can be a head-mounted display (HMD) for a virtual reality (VR) application or an augmented reality (AR) application.

[0049] In some embodiments, the platform controller hub 130 enables peripherals to connect to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, a data storage device 124 (e.g., a hard disk drive, a flash memory, etc.). The data storage device 124 can connect via a storage interface (e.g., a SATA) or an external bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensor 125 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, or Long Term Evolution (LTE) transceiver. The firmware interface 128 enables communication with the system firmware, and can be, for example, a unified extensible firmware interface (UEFI). The network controller 134 can enable network connectivity to a wired network. In some embodiments, a high-performance network controller (not shown) couples with the interface bus 110. The audio controller 146 is, in one embodiment, a multi-channel high definition audio controller. In one embodiment, the system 100 includes an optional legacy I / O controller 140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to the system. The platform controller hub 130 can also connect to one or more Universal Serial Bus (USB) controllers 142 connected input devices, such as keyboard and mouse 143 combinations, a camera 144, or other USB input devices.

[0050] It will be recognized that the illustrated system 100 is exemplary and not limiting as other types of data processing systems configured in different ways can also be used. For example, instances of the memory controller 116 and platform controller hub 130 can be integrated into a discrete external graphics processor, such as external graphics processor 112. In one embodiment, the platform controller hub 130 and / or memory controller 160 can be external to the one or more processor(s) 102. For example, the system 100 can include an external memory controller 116 and platform controller hub 130 that can be configured as a memory controller hub and peripheral controller hub within a system chipset that is in communication with the processor(s) 102.

[0051] FIG. 2 is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A to 202N, an integrated memory controller 214, and an integrated graphics processor 208. FIG. 2 Those elements of having the same reference number (or name) in the figures herein as elements in any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such. The processor 200 can include additional cores up to and including the additional core 202N represented by the dashed box. Each of the processor cores 202A to 202N includes one or more internal cache units 204A to 204N. In some embodiments, each processor core can also have access to one or more shared cache units 206.

[0052] The internal cache units 204A to 204N and shared cache units 206 represent a cache memory hierarchy internal to the processor 200. The cache memory hierarchy can include at least one level of instruction and data caches within each processor core and one or more levels of shared mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of caches, with the highest level cache being classified as an LLC prior to external memory. In some embodiments, cache coherency logic maintains coherency among the cache units 206 and 204A to 204N.

[0053] In some embodiments, the processor 200 also includes a set of one or more bus controller units 216 and a system agent unit 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI or PCI express busses. The system agent unit 210 provides management functionality for the various processor components. In some embodiments, the system agent unit 210 includes one or more integrated memory controllers 214 to manage access to various external memory devices (not shown).

[0054] In some embodiments, one or more of the processor cores 202A-202N include support for simultaneous multi-threading. In such embodiments, the system agent unit 210 includes components for coordinating and operating the cores 202A-202N during multi-threaded processing. Additionally, the system agent unit 210 can also include a power control unit (PCU), including logic and components to regulate the power state of the processor cores 202A-202N, as well as the graphics processor 208.

[0055] In some embodiments, the processor 200 additionally includes a graphics processor 208 to execute graphics processing operations. In some embodiments, the graphics processor 208 couples with the shared cache unit 206 set, and the system agent unit 210, which includes one or more integrated memory controllers 214. In some embodiments, the system agent unit 210 also includes a display controller 211 to drive graphics processor output to one or more coupled displays. The display controller 211 may

[0056] In some embodiments, a ring-based interconnect unit 212 is used to couple the internal components of the processor 200. However, an alternative interconnect unit can be used, such as a point-to-point interconnect, a switched interconnect, or other technology, including technologies well known in the art. In some embodiments, the graphics processor 208 couples with the ring interconnect 212 via an I / O link 213.

[0057] The exemplary I / O link 213 represents at least one of a variety of I / O interconnects, including a package I / O interconnect to facilitate communication between the various processor components and a high performance embedded memory module 218, such as an eDRAM module. In some embodiments, each of the processor cores 202A-202N and the graphics processor 208 use the embedded memory module 218 as a shared last level cache.

[0058] In some embodiments, the processor cores 202A-202N are homogeneous cores executing the same instruction set architecture. In another embodiment, the processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), where one or more of the processor cores 202A-202N execute a first instruction set, while at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 202A-202N are homogeneous in terms of microarchitecture, where one or more of the cores have a relatively high power consumption and one or more power cores have a lower power consumption. Additionally, the processor 200 can be implemented on one or more chips or as a SoC integrated circuit having, among other components, the illustrated components.

[0059] FIG. 3 is a block diagram of a graphics processor 300, which can be a discrete graphics processing unit, or can be a graphics processor integrated with a number of processing cores. In some embodiments, the graphics processor communicates with memory via a mapped I / O interface to registers on the graphics processor and utilizes commands placed into processor memory. In some embodiments, the graphics processor 300 includes a memory interface 314 for accessing memory. The memory interface 314 can be an interface to local memory, one or more internal caches, one or more shared external caches, and / or to system memory.

[0060] In some embodiments, the graphics processor 300 also includes a display controller 302 for driving display output data to a display device 320. The display controller 302 includes hardware for one or more overlay planes for the display and composition of multiple layers of video or user interface elements. The display device 320 can be an internal or external display device. In one embodiment, the display device 320 is a head mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding, decoding, or trans-coding media into or from one or more media encoding formats, including, but not limited to: Motion Picture Expert Group (MPEG) formats, such as MPEG-2, Advanced Video Coding (AVC) formats, such as H.264 / MPEG-4 AVC, as well as the Society of Motion Picture & Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats, such as JPEG, and Motion JPEG (MJPEG) formats.

[0061] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 to perform two-dimensional (2D) rasterizer operations including, for example, bit- block transfer. However, in one embodiment, 2D graphics operations are performed using one or more components of graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0062] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional graphics shapes representing 3D objects and scenes along with the associated data. 3D pipeline 312 includes programmable and fixed function elements that perform various tasks to generate output to threadsawing threadsawing to 3D / media subsystem 315 and / or threadsawing generated for execution by 3D pipeline 312. While 3D pipeline 312 can be used to perform media operations, an embodiment of GPE 310 also includes a media pipeline 316 that is specifically used to perform media operations, such as video post-processing and image enhancements.

[0063] In some embodiments, media pipeline 316 includes fixed function or programmable logic unit to accelerate one or more specialized media operations, such as video

[0064] In some embodiments, 3D / media subsystem 315 includes logic to execute threads generated by 3D pipeline 312 and media pipeline 316. In one embodiment, the pipelines send thread execution requests to 3D / media subsystem 315, which includes thread dispatch logic to arbitrate the requests and dispatch the thread requests to available thread execution resources. The execution resources include an array of graphics execution units to process the 3D and media threads. In some embodiments, 3D / media subsystem 315 includes one or more internal caches to cache it instructions and data. In some embodiments, the subsystem also includes shared memory (including registers and addressable memory) to share data between threads and to store output data.

[0065] Graphics Processing Engine

[0066] FIG. 4is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the GPE 310 shown and described herein. FIG. 3 FIG. 4 Those elements of the GPE 310 having the same reference number (or name) as that of any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such. For example, the 3D pipeline 312 and media pipeline 316 are shown. FIG. 3

[0067] In some embodiments, the GPE 410 is coupled with or includes a command stream translator 403 that provides a command stream to the 3D pipeline 312 and / or media pipeline 316. In some embodiments, the command stream translator 403 is a separate component from the GPE 410, while in other embodiments, the command stream translator 403 is integrated within the GPE 410. In some embodiments, the command stream translator 403 is coupled with memory, which can be system memory, or one or more of internal cache memory and shared cache memory. In some embodiments, the command stream translator 403 receives commands from the memory and sends the commands to the 3D pipeline 312 and / or media pipeline 316. The commands are instructions for the 3D pipeline 312 and media pipeline 316 on operations such as processing vertex data, performing primitive operations, building texture maps for 2D surface rendering, etc. In one embodiment, the ring buffer includes batch command buffers to batch multiple commands within a batch entry.

[0068] ​​In various embodiments, the 3D pipeline 312 includes fixed function logic and programmable logic to process one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shader programs. The graphics processor array 414 provides unified execution resources to

[0069] In some embodiments, the graphics processor array 414 also includes execution logic to perform media functions, such as video and / or image processing. In one embodiment, the execution units include programmable tissue to perform parallel general-purpose computing operations, in addition to graphics processing operations. FIG. 1 The execution units, either alone or in combination, can perform the processing operations required by the FIG. 2 processor core(s) 107 or the general-purpose logic within the core(s) 202A-202N in the system 100.

[0070] Output data generated by threads executing on graphics processor array 414 can be stored in a unified return buffer (URB) 418 in memory. The URB 418 can store data for multiple threads. In some embodiments, the URB 418 can be used to transmit data between different threads executing on the graphics processor array 414. In some embodiments, the URB 418 can additionally be used for synchronization purposes between threads on the graphics processor array and fixed function logic within shared function logic 420.

[0071] In some embodiments, the graphics processor array 414 is scalable, such that the array includes varying numbers of graphics cores each having varying numbers of execution units based on the target power and performance level of the GPE 410. In one embodiment, the execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0072] The graphics processor array 414 is coupled with shared function logic 420 that includes resources shared among the graphics cores within the graphics processor array. Shared functions within the shared function logic 420 are hardware logic units that provide specialized supplemental functionality to the graphics processor array 414. In various embodiments, the shared function logic 420 includes, but is not limited to, samplers 421, math 422, and inter-thread communication (ITC) 423 logic. Further, some embodiments implement one or more caches 425 within the shared function logic 420.

[0073] Shared functions are implemented in cases where the demand for a given special-purpose function is insufficient to be included in the graphics core array 414. Instead, a single instance of the special-purpose function is implemented as a separate entity in shared function logic 420 and shared among the execution resources within the graphics core array 414. The exact set of functions that are shared among the graphics core arrays 414 and included within the graphics core array 414 varies between embodiments. In some embodiments, particular shared functions that are heavily used by the graphics core array 414 within the shared function logic 420 can be included within the shared function logic 416 within the graphics core array 414. In various embodiments, the shared function logic 416 within the graphics core array 414 can include some or all of the logic within the shared function logic 420. In one embodiment, all of the logic elements within the shared function logic 420 can be duplicated within the shared function logic 416 of the graphics core array 414. In one embodiment, the shared function logic 420 is executed in order to support the shared function logic 416 within the graphics core array 414.

[0074] FIG. 5 is a block diagram of hardware logic of a graphics processor core 500 in accordance with some embodiments described herein. FIG. 5 Those elements of having the same reference number (or name) as elements in any other figure herein can operate or function in any manner similar to the manner described elsewhere herein, but are not limited to such. In some embodiments, the illustrated graphics processor core 500 includes a number of sub-cores 501A-501F within the graphics core array 414 of FIG. 4 The graphics processor core 500— sometimes referred to as a core slice— can be one or more graphics cores within a modular graphics processor. An example of the graphics processor core 500 is one graphics core slice, and a graphics processor as described herein can include multiple graphics core slices based on target power and performance envelopes. Each graphics core 500 can include fixed function blocks 530 coupled with a number of sub-cores 501A-501F (also referred to as sub-slices) that include both modular general-purpose logic blocks and fixed function logic blocks.

[0075] In some embodiments, the fixed function blocks 530 include a geometry / fixed function pipeline 536 that can be shared by all of the sub-cores in the graphics processor 500, for example, in low performance and / or low power graphics processor implementations. In various embodiments, the geometry / fixed function pipeline 536 includes a 3D fixed function pipeline (e.g., as in 3D pipeline 312 in FIG. 3 and FIG. 4 a video front-end unit, a thread spawn / broadcast unit, and a thread dispatcher, as well as a unified return buffer manager that manages a unified return buffer 418 like FIG. 15 ​

[0076] In one embodiment, the fixed function block 530 also includes a graphics SoC interface 537, a graphics microcontroller 538, and a media pipeline 539. The graphics SoC interface 537 provides an interface between the graphics core 500 and other processor cores within the system-on-chip integrated circuit. The graphics microcontroller 538 is a programmable subprocessor that can be configured to manage various functions of the graphics processor 500, including thread dispatching, scheduling, and pre-emption. The media pipeline 539 (e.g., FIG. 3 and FIG. 4 The media pipeline 316 includes logic for facilitating decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image data and video data. The media pipeline 539 implements media operations via requests to computational or sampling logic within the sub-cores 501 to 501F.

[0077] In one embodiment, the SoC interface 537 enables the graphics core 500 to communicate with a general-purpose application processor core (e.g., a CPU) and / or other components within the SoC, including memory hierarchy elements such as shared last-level cache memory, system RAM, and / or embedded on-chip or on-package DRAM. The SoC interface 537 may also enable communication with fixed-function devices within the SoC, such as a camera imaging pipeline, and enable the use and / or implementation of global memory atomics that can be shared between the graphics core 500 and the CPU within the SoC. The SoC interface 537 may also implement power management controls for the graphics core 500 and enable interfaces between the clock domain of the graphics core 500 and other clock domains within the SoC. In one embodiment, the SoC interface 537 enables receiving command buffers from a command stream converter and a global thread dispatcher configured to provide commands and instructions to each of one or more graphics cores within the graphics processor. These commands and instructions may be dispatched to the media pipeline 539 when media operations are to be performed, or to the geometry and fixed-function pipelines (e.g., geometry and fixed-function pipeline 536, geometry and fixed-function pipeline 514) when graphics processing operations are to be performed.

[0078] The graphics microcontroller 538 can be configured to perform various scheduling and management tasks for the graphics core 500. In one embodiment, the graphics microcontroller 538 can perform graphics and / or compute workload scheduling for individual graphics processing engines within execution unit (EU) arrays 502A-502F, 504A-504F within the corelets 501A-501F. In this scheduling model, host software executing on a CPU core of a SoC including the graphics core 500 can submit a workload via one of a number of graphics processor doorbells, which invokes a scheduling operation on the appropriate graphics engine. The scheduling operation includes determining which workload to run next, submitting the workload to a command streamer, pre-empting existing workloads running on the engine, monitoring progress of the workload, and notifying host software when the workload completes. In one embodiment, the graphics microcontroller 538 can also facilitate low power or idle states for the graphics core 500, providing the graphics core 500 with the ability to save and restore registers across low power state transitions independently of operating systems and / or graphics driver software on the system.

[0079] The graphics core 500 can have more or fewer than the illustrated number of sub-cores 501A-501F, up to N modular sub-cores. For each set of N sub-cores, the graphics core 500 can also include shared function logic 510, shared memory and / or cache memory 512, geometry / fixed function pipeline 514, and additional fixed function logic 516 for performing various graphics and compute operations. The shared function logic 510 can include logic units associated with the shared function logic 420 that can be shared by every N sub-core within the graphics core 500. The shared memory and / or cache memory 512 can be a last level cache for the set of N sub-cores 501A-501F within the graphics core 500 and can also act as shared memory accessible by multiple sub-cores. The geometry / fixed function pipeline 514 can be included within the fixed function block 530 instead of the geometry / fixed function pipeline 536 and can include the same or similar logic units. FIG. 4 The shared function logic 420 associated logic units (e.g., sampler logic, math logic, and / or inter-thread communication logic). The shared memory and / or cache memory 512 can be a last level cache for the set of N sub-cores 501A-501F within the graphics core 500 and can also act as shared memory accessible by multiple sub-cores. The geometry / fixed function pipeline 514 can be included within the fixed function block 530 instead of the geometry / fixed function pipeline 536 and can include the same or similar logic units.

[0080] In one embodiment, graphics core 500 includes additional fixed function logic 516 which can include various fixed function acceleration logic to be utilized by graphics core 500. In one embodiment, additional fixed function logic 516 includes an additional geometry pipeline for use in position only shading. In position only shading, there are two geometry pipelines: a full geometry pipeline within geometry / fixed function pipeline 516, 536; and a cull pipeline, which is an additional geometry pipeline that can be included within additional fixed function logic 516. In one embodiment, the cull pipeline is a slimmed down version of the full geometry pipeline. The full pipeline and the cull pipeline can execute different instances of the same application, each with a separate context. Position only shading can hide the long cull run of discarded triangles, enabling completion of shading earlier in some instances. For example, and in one embodiment, cull pipeline logic within additional fixed function logic 516 can execute position shaders in parallel with the main application, and often generate critical results faster than the full pipeline because the full pipeline only fetches and shades position attributes of vertices, without performing rasterization and rendering of pixels to a frame buffer. The cull pipeline can use the generated critical results to compute visibility information for all triangles, without having to consider whether those triangles are culled. The full pipeline, which in this instance can be referred to as a replay pipeline, can consume the visibility information in order to skip culled triangles to only shade visible triangles that are ultimately passed to a rasterization stage.

[0081] In one embodiment, additional fixed function logic 516 can also include machine learning acceleration logic, such as fixed function matrix multiplication logic, for implementations including machine learning training or inferencing.

[0082] Within each graphics sub-core 501A-501F includes a set of execution resources that can be leveraged to perform graphics operations, media operations, and compute operations in response to requests by graphics pipeline, media pipeline, or shader programs. The graphics sub-cores 501A-501F include: multiple EU arrays 502A-502F, 504A-504F; thread dispatch and inter-thread communication (TD / IC) logic 503A-503F; 3D (e.g., texture) samplers 505A-505F; media samplers 506A-506F; shader processors 507A-507F; and shared local memory (SLM) 508A-508F. The EU arrays 502A-502F, 504A-504F each include multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations in service of a graphics operation, media operation, or compute operation for graphics programs, media programs, or compute shader programs. The TD / IC logic 503A-503F performs local thread dispatch and thread control operations for execution units within a sub-core and facilitate communication between threads executing on execution units of the sub-core. The 3D samplers 505A-505F can read texture or other 3D graphics related data into memory. The 3D samplers can read texture data differently based on a configured sample state and a texture format associated with a given texture. The media samplers 506A-506F can perform similar read operations based on a type and format associated with media data. In one embodiment, each graphics sub-core 501A-501F can alternately include unified 3D and media samplers. Threads executing on the execution units within each of the sub-cores 501A-501F can make use of shared local memory 508A-508F within each sub-core, to enable threads executing within a thread group to execute using a common pool of on-chip memory.

[0083] Execution Unit

[0084] FIGS. 6A-6B Thread execution logic 600 including a processing element array employed in a graphics processor core is shown in accordance with an embodiment described herein. FIGS. 6A-6B Those elements of Figure 1 having the same reference number (or name) as elements in any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such. FIG. 6A An overview of thread execution logic 600 is shown, which can include a set of shared function units (SFUs) 610A-610B and shared function control logic 620A-620B, which can be shared under the control of a shared function unit controller 630A-630B. The shared function units 610A-610B can be configured as warp schedulers, code fetch / execution units, register file units, or shared memory units. FIG. 5 Variations of the hardware logic of each sub-core 501A-501F. FIG. 6B Exemplary internal details of an execution unit are shown.

[0085] like FIG. 6A As shown in , in some embodiments, thread execution logic 600 includes a shader processor 602, a thread dispatcher 604, an instruction cache 606, a scalable execution unit array including a plurality of execution units 608A to 608N, a sampler 610, a data cache 612, and a data port 614. In one embodiment, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., execution units 608A, 608B, 608C, 608D, up to any one of 608N-1 and 608N) based on the computational needs of the workload. In one embodiment, the included components are interconnected via an interconnect structure that links to each of the components. In some embodiments, thread execution logic 600 includes one or more connections to a memory (such as system memory or cache memory) through the instruction cache 606, the data port 614, the sampler 610, and one or more of the execution unit arrays 608A to 608N. In some embodiments, each execution unit (e.g., 608A) is an independently programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various embodiments, the array of execution units 608A to 608N is scalable to include any number of individual execution units.

[0086] In some embodiments, execution units 608A to 608N are primarily used to execute shader programs. Shader processor 602 can process various shader programs and dispatch execution threads associated with the shader programs via thread dispatcher 604. In one embodiment, the thread dispatcher includes logic for arbitrating thread initiation requests from graphics and media pipelines and instantiating the requested threads on one or more execution units 608A to 608N. For example, the geometry pipeline can dispatch vertex processing, tessellation, or geometry processing threads to thread execution logic for processing. In some embodiments, thread dispatcher 604 can also handle runtime thread generation requests from executing shader programs.

[0087] In some embodiments, execution units 608A-608N support single program multiple instruction instructions 608A-608N (SPMD) with each execution unit 608A-608N being configured to provide one or more SMIDs for a thread. SPMD is a technique by which a single instruction is modified to include an operand that identifies data to be operated on by that instruction. The data is divided into a plurality of subsets, with each subset corresponding to a different thread. The execution units 608A-608N operate on the different subsets of data in parallel. The execution units 608A-608N can be configured for use in 32 single instruction multiple data (SIMD) or vector processing operations, 64 SIMIDs, or other targeted SIMD or vector length.

[0088] Each of the execution units 608A-608N operates on arrays of data elements. The number of data elements is the "execution size," or the number of channels that the instruction operates on. An execution channel is a logical unit of execution for data element access, masking, and flow control. The number of channels may

[0089] The execution unit instruction set includes SIMD and VLIW instructions. Various data elements can be stored in registers and the execution units will process data in single or double precision, depending on the mode setting. The execution units support a variety of data processing operations, including integer and floating-point dot product, multiplication and accumulation, and dot product instructions. For integer data, logic, and bit-bashing operations, the execution units can process 8, 16, 32, or 64 bits of integer data per cycle. Two 32-bit floats, four 16-bit floats, eight 8-bit floats, or sixteen 4-bit floats can be processed with each cycle. Multiple floats can be processed with each cycle.

[0090] In one embodiment, one or more execution units can be combined into a fused execution unit 609A-609N that has thread control logic (607A-607N) common to the fused EU. Multiple EU's can be fused into an EU group. Each EU in the fused EU group can be configured to execute a separate SIMD hardware thread. The number of EU's in a fused EU group can vary from one embodiment to another. In addition, different SIMD widths can be executed per EU including, but not limited to, SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 609A-609N comprises at least two execution units. For example, fused execution unit 609A comprises a first EU 608A, a second EU 608B, and thread control logic 607A common to first EU 608A and second EU 608B. Thread control logic 607A controls threads executing on fused graphics execution unit 609A, allowing each EU within fused execution units 609A-609N to use a common instruction pointer register to be executed.

[0091] One or more internal instruction caches (e.g., 606) are included in the thread execution logic 600 to cache thread instructions for the execution units. In some embodiments, one or more data caches (e.g., 612) are included to cache thread data during thread execution. In some embodiments, a sampler 610 is included to provide texture sampling for 3D operations and to provide media sampling for media operations. In some embodiments, the sampler 610 includes specialized texture or media sampling functionality to handle processing of texture or media data during sampling before it is provided to the execution units.

[0092] During execution, the graphics and media pipeline sends thread initiation requests to the thread execution logic 600 via thread generation and dispatch logic. Once a set of geometry objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 602 is invoked to further calculate output information and cause results to be written to an output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some embodiments, the pixel shader or fragment shader calculates values for various vertex attributes that are interpolated across the rasterized object. In some embodiments, the pixel processor logic within the shader processor 602 then executes an application programming interface (API) supplied pixel or fragment shader program. To execute the shader program, the shader processor 602 dispatches threads to execution units (e.g., 608A) via the thread dispatcher 604. In some embodiments, the shader processor 602 uses texture sampling logic in the sampler 610 to access texture data stored in a texture map stored in memory. Arithmetic operations on the texture data and input geometry data calculate pixel color data for each geometric fragment, or discard one or more pixels without further processing.

[0093] In some embodiments, the data port 614 provides a memory access mechanism for the thread execution logic 600 to output processed data to memory for further processing on a graphics processor output pipeline. In some embodiments, the data port 614 includes or is coupled to one or more cache memories (e.g., data cache 612) to cache data for memory access via the data port.

[0094] As shown in FIG. 6B The graphics execution unit 608 can include, in one embodiment, an instruction fetch unit 637, a general register file array (GRF) 624, an architecture register file array (ARF) 626, a thread arbiter 622, an issue unit 630, a branch unit 632, a set of SIMD floating point units (FPUs) 634, and in one embodiment, a set of dedicated integer SIMD ALUs 635. The GRF 624 and ARF 626 include the set of general and architecture register files associated with each synchronized hardware thread that can be active in the graphics execution unit 608. In one embodiment, per-thread architecture state is maintained in the ARF 626, while data used during thread execution is stored in the GRF 624. The execution state of each thread, including the instruction pointer of each thread, can be held in thread-specific registers in the ARF 626.

[0095] In one embodiment, graphics processing unit 608 has an architecture that is a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT). The architecture has a modular configuration that can be tuned at design time based on a target number of simultaneous threads and a target number of registers per execution unit, in which execution unit resources are partitioned across logic used to execute multiple simultaneous threads.

[0096] In one embodiment, graphics processing unit 608 can issue multiple instructions together that can each be different instructions. A thread arbiter 622 of graphics processing unit thread 608 can dispatch instructions to one of a send unit 630, a branch unit 642, or a SIMD FPU(s) 634 for execution. Each execution thread has access to 128 general purpose registers within a GRF 624, where each register can store 32 bytes that can be accessed as a SIMD 8-element vector with 32-bit data elements. In one embodiment, each execution unit thread has access to 4 kilobytes within the GRF 624, although embodiments are not so limited, and more or less register resources can be provided in other embodiments. In one embodiment, up to seven threads can execute synchronously, although the number of threads per execution unit can vary according to embodiments. In an embodiment in which seven threads have access to 4 kilobytes, the GRF 624 can store a total of 28 kilobytes. Flexible addressing modes can permit multiple registers to be addressed together, efficiently establishing wider registers or representing stride rectangular block data structures.

[0097] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via “send” instructions executed by the message passing send unit 630. In one embodiment, branch instructions are dispatched to a dedicated branch unit 632 in order to facilitate SIMD divergence and eventual convergence.

[0098] In one embodiment, graphics processing unit 608 includes one or more SIMD floating point units (FPUs) 634 to perform floating point operations. In one embodiment, the FPU(s) 634 also support integer computation. In one embodiment, the FPU(s) 634 can SIMD execute up to a number M of 32-bit floating point (or integer) operations, or up to 2M of 16-bit integer or 16-bit floating point operations. In one embodiment, at least one of the FPU(s) provides an extended math capability that supports high throughput transcendental math functions and double precision 64-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 635 also represent and can be specifically optimized to perform operations associated with machine learning computations.

[0099] In one embodiment, an array of instances of the graphics execution unit 608 can be instantiated in a graphics sub-core (e.g., a sub-slice) for graphics processing. In one embodiment, each graphics sub-core shares one instance of the graphics execution unit 608. In another embodiment, multiple instances of the graphics execution unit 608 are provided for graphics processing. For scalable performance, product architectures can include different amounts of execution units 608 for each sub-core. In one embodiment, the product architecture can include a number of execution units 608 for each sub-core, where the number of execution units 608 is based on the performance and power considerations.

[0100] FIG. 7 FIG. 7 is a block diagram illustrating a graphics processor instruction format 700 according to some embodiments. In one or more embodiments, a graphics processor execution unit supports an instruction set that includes a number of different types of instructions for performing various operations, including memory load / store operations, arithmetic operations, and so on. The real boxes in the illustration represent in general the components that are typically included in a graphics processor execution unit instruction, while the dashed lined boxes represent optional components that can be included or excluded from the instruction format as desired. In some embodiments the instruction format 700 described and illustrated is a macro-instruction, in that they are the instructions that are supplied to the execution unit, as opposed to micro-ops that are generated from the instructions once they are processed.

[0101] In some embodiments, the graphics processor execution unit natively supports instructions in a 128-bit instruction format 710. A 64-bit compact instruction format 730 can be used for some instructions based on a number of instruction options and operand sizes. The native 128-bit instruction format 710 provides access to all instruction options available while some options and operations are restricted to the 64-bit format 730. The native 128-bit instruction format 710 is used by some embodiments of the graphics processor execution units. In some embodiments, a set of index values in the instruction header are used to select one of a number of possible 128-bit instruction formats used for the instruction.

[0102] For each format, the instruction opcode 712 defines the operation that the execution unit is to perform. The execution units execute each instruction in parallel across the data elements of each operand. For example, in response to an add instruction, the execution units perform a simultaneous add operation across each color channel for the elements at positions in the source registers where the two operands overlap. In one embodiment, the default behavior is to perform an operation for all data channel elements. In some embodiments, instruction control field 714 enables control over a number of execution options, including channels selection (e.g., predication) and data channel order (e.g., swizzle). In some embodiments, an exec-size field 716 limits the number of data channels that will be executed in parallel. In some embodiments, exec-size field 716 is not available for all 128-bit instructions.

[0103] Some execution unit instructions have up to three operands, including two source operands (src0 720, src1 722) and one destination 718. In some embodiments, the execution unit supports dual destination instructions in which one of the destinations is implicit. Data operation instructions can have a third source operand (e.g., SRC2 724), where the instruction opcode 712 determines the number of source operands. The last source operand of an instruction can be an immediate (e.g., hard coded) value passed with the instruction.

[0104] In some embodiments, the 128-bit instruction format 710 includes a access / address mode field 726 that specifies, for example, whether to use a direct register addressing mode or an indirect register addressing mode. When using direct register addressing mode, the register address for one or more operands is provided directly by bits in the instruction.

[0105] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies the address mode and / or access mode of the instruction. In one embodiment, the access mode is used to qualify data access alignment for the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction can use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction can use 16-byte aligned addressing for all source and destination operands.

[0106] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction uses direct addressing or indirect addressing. When using direct register addressing mode, the register address for one or more operands is provided directly by bits in the instruction. When using indirect register addressing mode, the register address for one or more operands can be calculated based on an address register value and an address immediate field in the instruction.

[0107] In some embodiments, instructions are grouped based on the opcode 712 bit field to simplify opcode decoding 740. For 8-bit opcodes, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The precise opcode grouping shown is exemplary only. In some embodiments, the move and logic opcode group 742 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares the five most significant bits (MSBs), wherein the move (mov) instruction adopts the form of 0000xxxxb, while the logic instruction adopts the form of 0001xxxxb. The flow control instruction group 744 (e.g., call (call), jump (jmp)) includes instructions using the form of 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 746 includes a mixture of instructions, including synchronization instructions (e.g., wait (wait), send (send)) using the form of 0011xxxxb (e.g., 0x30). The parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, mul) in the form of 0100xxxxb (e.g., 0x40). The parallel math group 748 performs arithmetic operations in parallel across data lanes. The vector math group 750 includes arithmetic instructions (e.g., dp4) in the form of 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic operations on vector operands, such as dot products.

[0108] Graphics Pipeline

[0109] FIG. 8 is a block diagram of another embodiment of a graphics processor 800 . FIG. 8 Those elements having the same reference numbers (or names) as elements in any other figures herein may operate or function in any manner similar to, but not limited to, those described elsewhere herein.

[0110] In some embodiments, graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by commands issued to graphics processor 800 via ring interconnect 802. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 802 are interpreted by command stream converter 803, which supplies instructions to individual components of geometry pipeline 820 or media pipeline 830.

[0111] In some embodiments, the command streamer 803 directs the operation of the vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by the command streamer 803. In some embodiments, the vertex fetcher 805 provides vertex data to the vertex shader 807, which performs coordinate space transformation and lighting operations on each vertex. In some embodiments, the vertex fetcher 805 and the vertex shader 807 execute vertex processing instructions by dispatching execution threads to the execution units 852A-B via the thread dispatcher 831.

[0112] In some embodiments, the execution units 852A-B are vector processors having instruction sets specifically configured to perform graphics and media operations. In some embodiments, the execution units 852A-B have attached Ll caches 851, which are dedicated to each array or shared between arrays. The caches can be configured as data caches, instruction caches, or single caches that are partitioned into different regions for data and instructions.

[0113] In some embodiments, the geometry pipeline 820 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 811 configures tessellation operations. A programmable domain shader 817 provides post-processing of tessellation output. A tessellator 813 operates in the direction of the hull shader 811 and contains specialized logic to generate a detailed set of geometric objects based on a coarse geometric model provided as input to the geometry pipeline 820. In some embodiments, the tessellation component (e.g., hull shader 811, tessellator 813, domain shader 817) can be bypassed if tessellation is not used.

[0114] In some embodiments, complete geometric objects can be processed by a geometry shader 819 via one or more threads dispatched to the execution units 852A-B, or can pass directly to the clipper 829. In some embodiments, the geometry shader operates on entire geometric objects (rather than vertices or vertex patches as in previous stages of the graphics pipeline). If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 can be programmed by a geometry shader program to perform geometric tessellation when the tessellation unit is disabled.

[0115] The clipper 829 can process vertex data prior to rasterization. The clipper 829 can be a fixed function clipper or a programmable clipper with clip and geometry shader functionality. In some embodiments, the rasterizer and depth test components 873 in the render output pipeline 870 dispatch pixel shaders to convert a geometric object into a per-pixel representation. In some embodiments, the pixel shader logic is included in the thread execution logic 850. In some embodiments, the application can bypass the rasterizer and depth test components 873 and access un-rasterized vertex data via the outflow unit 823.

[0116] The graphics processor 800 has an interconnect bus, interconnect fabric, or some other interconnect mechanism to allow data and messages to be passed between components of the graphics processor, including the main unit, memory unit, and the graphics processing array. In some embodiments, the execution units 852A-852B and the associated logic units (e.g., LI cache 851, sampler 854, texture cache 858, etc.) are interconnected in a high-speed bus fabric, to perform memory accesses and to communicate with each other. In some embodiments, the sampler 854, caches 851, 858, and execution units 852A-852B each have separate memory access ports to the interconnect bus. In one embodiment, the texture cache 858 can also be configured as a sampler cache.

[0117] In some embodiments, the render output pipeline 870 includes a rasterizer and depth test components 873 that convert vertex-based objects into associated pixel-based representations. In some embodiments, rasterizer logic includes a windower / masker unit for fixed function triangle and line rasterization. An associated render cache 878 and depth cache 879 are also available in some embodiments. Pixel operation components 877 perform pixel-based operations, although in some instances pixel operations associated with 2D operations (e.g., bit block image transfers using blitting with alpha blending) are performed by the 2D engine 841 or are replaced by the display controller 843 using an overlay display plane at display time. In some embodiments, a shared L3 cache 875 is available for all graphics components, allowing for shared use of data without the need to have a copy of data in the main system memory.

[0118] In some embodiments, graphics processor media pipeline 830 includes a media engine 837 and a video front-end 834. In some embodiments, video front-end 834 receives pipeline commands from the command streamer 803. In some embodiments, media pipeline 830 includes a separate command streamer. In some embodiments, video front-end 834 processes media instructions before sending the media instructions to media engine 837. In some embodiments, media engine 837 includes thread spawning functionality to process media instructions.

[0119] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples with the graphics processor via the ring interconnect 802, or some other interconnect bus or fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 contains special purpose logic that is configured to operate independently of the 3D pipeline. In some embodiments, display controller 843 couples with a display device (not illustrated) that can be a system integrated display device, as in a laptop computer, or an external display device via an display device connector.

[0120] In some embodiments, geometry pipeline 820 and media pipeline 830 can be configured to perform operations based on a number of graphics and media programming interfaces. In some embodiments, driver software for the graphics processor translates API calls received by the graphics processor into commands

[0121] Graphics Pipeline Programming

[0122] FIG. 9A is a block diagram illustrating a graphics processor command format 900 according to some embodiments. FIG. 9B is a block diagram illustrating a graphics processor command sequence 910 according to an embodiment. FIG. 9AThe solid lined boxes illustrate the components typically that are included in a graphics command while the dashed lined boxes illustrate optional components that can or can not be included in the graphics command. FIG. 9A The exemplary graphics processor command format 900 includes data fields to identify a client 902, a command operation code (opcode) 904, and data 906 for the command. Some commands include an optional sub-opcode 905 and a command size 908.

[0123] In some embodiments, the client 902 specifies a client unit of the graphics device that handles the command data. In some embodiments, a graphics processor command parser examines the client field of each command to adjust further processing of the command and route the command data to the appropriate client unit. In some embodiments, the graphics processor client units include a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a respective processing pipeline to process the commands. Once a command is received by a client unit, the client unit reads the opcode 904 and the sub-opcode 905 (if present) to determine the operation to perform. The client unit uses the information in the data field 906 to perform the command. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, the commands are aligned via multiples of a doubleword.

[0124] FIG. 9B The flow diagram in FIG. 9 illustrates an exemplary graphics processor command sequence 910. In some embodiments, a software or firmware of a data processing system featuring an embodiment of a graphics processor uses a version of the command sequence shown to initiate, perform, and terminate a set of graphics operations. A sample command sequence is shown and described for illustrative purposes only, as embodiments are not limited to these specific commands or to this command sequence. Moreover, the commands can be issued as a batch of commands in a command sequence, such that multiple commands are issued in at least partially simultaneous fashion. The sample command sequence is not representative of all of the commands that can be issued in a real-world scenario.

[0125] In some embodiments, the graphics processor command sequence 910 can begin with a pipeline flush command 912 to ensure that any active pipeline does not complete pending commands. In some embodiments, the 3D pipeline 922 and media pipeline 924 are not operating at the same time. A pipeline flush can be performed to ensure that the active pipeline completes any uncompleted commands before the other pipeline is allowed to operate. The pipeline flush can be followed by a pipeline context save command 913 to save the pipeline application context so that these contexts can be restored for the next pipeline operation. In some embodiments, the graphics processor core 101 is a dual pipeline core, and the pipeline context save and restore commands are used to context switch between the two pipelines. In some embodiments, following the pipeline flush, the graphics processor core 101 is placed into a low power or standby mode.

[0126] In some embodiments, when the command sequence requires the graphics processor to explicitly switch between processing tasks on the pipeline, a pipeline select command 913 is used. In some embodiments, the pipeline select command 913 is only needed once for each pipeline context switch, and each pipeline context switch can include a full pipeline flush and context save and restore, so that the general purpose processor does not need to include a complete pipeline flush and context save each time a pipeline context switch occurs.

[0127] In some embodiments, pipeline control commands 914 configure a graphics pipeline for operation and set pipeline specific parameter values. In some embodiments, the pipeline control commands 914 configure the operation of the active pipeline for processing commands in the command sequence. In some embodiments, pipeline control commands 914 include pipeline context commands used for setting the processing environment or context for each of the pipeline stages as the pipeline state is restored.

[0128] In some embodiments, return buffer state commands 916 are used to configure a set of return buffers for data return from the pipeline. Some pipeline operations require one or more return buffers for data return, depending on the data size and type returned from the pipeline operation. In some embodiments, the graphics processor core 101 uses return buffers for output data storage and cross thread communication. In some embodiments, return buffer state commands 916 include selecting the size and number of return buffers for a pipeline operation.

[0129] The remaining commands in the command sequence vary depending on the active pipeline for processing. Based on the pipeline determination 920, the command sequence is customized for the 3D pipeline 922 beginning at 3D pipeline state 930, or for the media pipeline 924 beginning at media pipeline state 940.

[0130] Commands used to configure 3D pipeline state 930 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables to be configured prior to processing 3D primitive commands. The values for these commands are determined based at least in part on the particular 3D API in use. In some embodiments, 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements will not be used.

[0131] In some embodiments, 3D primitive 932 commands are used to submit 3D primitives to be processed by the 3D pipeline. The commands and associated parameters passed to the graphics processor via 3D primitive 932 commands are forwarded to a vertex fetch function in the graphics pipeline. The vertex fetch function uses the 3D primitive 932 command data to generate a plurality of vertex data structures. The vertex data structures are stored in one or more return buffers. In some embodiments, 3D primitive 932 commands are used to perform vertex operations on 3D primitives via a vertex shader. To process the vertex shader, 3D pipeline 922 dispatches shader execution threads to graphics processor execution units.

[0132] In some embodiments, 3D pipeline 922 is triggered via an execute 934 command or event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a 'go' or 'kick' command in the command sequence. In one embodiment, a pipeline synchronization command is used to trigger command execution in order to flush the command sequence with no-ops through the graphics pipeline. The 3D pipeline will perform geometry processing for 3D primitives. Once the operations are complete, the resulting geometry is rasterized and pixels are shaded by a pixel engine. Additional commands to control pixel shading and pixel back end operations can also be included for these operations.

[0133] In some embodiments, graphics processor command sequence 910 follows the media pipeline 924 path when performing media operations. Generally, the specific use and manner of programming for the media pipeline 924 depends on the media or compute operations to be performed. In media decode processes, specific media decode operations can be offloaded to the media pipeline. In some embodiments, the media pipeline can also be bypassed and media decode can be performed entirely or partially using resources provided by one or more general purpose processing cores. In one embodiment, the media pipeline also includes elements for general purpose graphics processor unit (GPGPU) operations, where the graphics processor is used to perform SIMD vector operations using a compute shader program that is not explicitly related to the rendering of a graphics primitive.

[0134] In some embodiments, the media pipeline 924 is configured in a similar manner as the 3D pipeline 922. A set of commands to configure the media pipeline state 940 is dispatched or placed into the command queue, prior to the media object command 942. In some embodiments, the commands 940 for the media pipeline state include data to configure the media pipeline elements that will be used to process the media object. This includes data to configure the video decode and video encode logic within the media pipeline, such as the encoding or decoding format. In some embodiments, the commands 940 for the media pipeline state also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0135] In some embodiments, the media object command 942 supplies a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer that contains video data to be processed. In some embodiments, all of the media pipeline state must be valid prior to issuing the media object command 942. Once the pipeline state is configured and the media object command 942 is queued, the media pipeline 924 is triggered via an execute 944 command or equivalent execution event (e.g., register write). The output from the media pipeline 924 can then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a similar manner as media operations.

[0136] Graphics Software Architecture

[0137] FIG. 10 An exemplary graphics software architecture for a data processing system 1000 is shown in accordance with some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 each execute in a system memory 1050 of the data processing system.

[0138] In some embodiments, the 3D graphics application 1010 contains one or more shader programs including shader instructions 1012. The shader language instructions can be in a high-level shader language, such as the High-Level Shader Language (HLSL) or the OpenGL Shader Language (GLSL). The application also includes executable instructions 1014 in a machine language suitable for execution by the general-purpose processor cores 1034. The application also includes graphics objects defined by vertex data 1016.

[0139] In some embodiments, operating system 1020 is WINDOWS® from Microsoft Corporation Operating system, a special UNIX®-based operating system, or an open source UNIX®-based operating system using a Linux kernel variant. Operating system 1020 can support a graphics API 1022, such as a Direct3D API, an OpenGL API, or a Vulkan API. When the Direct3D API is in use, operating system 1020 uses a front-end shader compiler 1024 to compile any shader instructions 1012 written in HLSL into a lower-level shader language. The compilation can be a just-in-time (JIT) compilation or the application can perform shader pre- compilation. In some embodiments, high-level shaders are compiled into low-level shaders in the course of compiling 3D graphics application 1010. In some embodiments, shader instructions 1012 are provided in an intermediate form, such as a version of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API.

[0140] In some embodiments, user mode graphics driver 1026 contains a back-end shader compiler 1027 to translate shader instructions 1012 into representations for a hardware- specific use. When the OpenGL API is in use, shader instructions 1012 in GLSL high-level language are passed to user mode graphics driver 1026 for compilation. In some embodiments, user mode graphics driver 1026 uses operating system kernel mode functions 1028 to communicate with kernel mode graphics driver 1029. In some embodiments, kernel mode graphics driver 1029 communicates with graphics processor 1032 in order to dispatch

[0141] IP Core Implementation

[0142] One or more aspects of at least one embodiment can be implemented by representative code stored on a machine-readable medium which represents and / or defines logic utilizing a plurality of lines of code. For example, the machine- readable medium can include a plurality of lines of code that represent various logical operations employed by a processor to implement techniques described herein. Such representations, known as "IP cores" can be stored on a tangible, machine-readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that make the integrated circuit.

[0143] FIG. 11A is a block diagram illustrating an IP core development system 1100 that can be used to fabricate integrated circuits to perform operations according to embodiments. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into larger designs or used to construct an entire integrated circuit (e.g., an SOC integrated circuit). A design facility 1130 can employ a high-level programming language (e.g., C / C++) to generate a software simulation 1110 of the IP core design. The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 can include functional, behavioral, and / or timing simulations. The register transfer level (RTL) design 1115 can then be created or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the behavior of the integrated circuit (including associated logic executed by the modeled digital signals) that models the flow of digital signals between hardware registers, including the associated logic performed thereon. In addition to an RTL design 1115, a lower-level design, such as a logic level or transistor level design, can also be created, designed, or synthesized.

[0144] The RTL design 1115, or equivalent, can be further synthesized, formatted, or prepared to produce a hardware model 1120, which can be in a hardware description language (HDL) or some other representation of the design. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored in non-volatile memory 1140 (e.g., hard disk, flash memory, or any non-volatile storage medium) for delivery to a third party fabrication facility 1165. Alternatively, the IP core design can be transmitted (e.g., via the Internet) over a wired 1150 or wireless 1160 connection. The fabrication facility 1165 can then fabricate an integrated circuit based at least in part on the IP core design. The fabricated integrated circuit can be configured to perform operations in accordance with at least one embodiment described herein.

[0145] FIG. 11BA cross-sectional side view of an integrated circuit package assembly 1170 is shown in accordance with some embodiments described herein. The integrated circuit package assembly 1170 shows an implementation of one or more processor or accelerator devices as described herein. The package assembly 1170 includes multiple hardware logic units 1172, 1174 connected to a substrate 1180. The logics 1172, 1174 can be implemented at least partially in configurable logic or fixed function logic hardware and can include one or more portions of processor core(s), graphics processor(s), or any of the other accelerator devices described herein. Each logic unit 1172, 1174 can be implemented within a semiconductor die and coupled with the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 can be configured to route electrical signals between the logics 1172, 1174 and the substrate 1180 and can include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 can be configured to route electrical signals such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of the logics 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the package substrate 1180 can include other suitable types of substrates. The package assembly 1170 can be connected to other electrical devices via a package interconnect 1183. The package interconnect 1183 can be coupled to a surface of the substrate 1180 in order to route electrical signals to other electrical devices such as a motherboard, other chipsets, or multi-chip modules.

[0146] In some embodiments, the logic units 1172, 1174 are electrically coupled with a bridge 1182 that is configured to route electrical signals between the logics 1172, 1174. The bridge 1182 can be a dense interconnect structure that provides routing for electrical signals. The bridge 1182 can include a bridge substrate composed of glass or a suitable semiconductor material. Circuitry features can be formed on the bridge substrate in order to provide chip-to-chip connections between the logics 1172, 1174.

[0147] Although two logic units 1172, 1174 and a bridge 1182 are shown, embodiments described herein can include more or fewer logic units on one or more dies. The one or more dies can be connected by zero or more bridges, as the bridge 1182 can be excluded when including logics on a single die. Alternatively, multiple dies or logic units can be connected by one or more bridges. Additionally, multiple logic units, dies, and bridges can be connected together in other possible configurations including three-dimensional configurations.

[0148] Exemplary System on a Chip Integrated Circuit

[0149] FIGS. 12-1 4 illustrates exemplary integrated circuits and related graphics processors that can be fabricated using one or more IP cores in accordance with various embodiments described herein. In addition to the graphics processors illustrated, other logic and circuits can also be included, including additional graphics processors / cores, peripheral interface controllers or general purpose processor cores.

[0150] FIG. 12 is a block diagram illustrating an exemplary system on a chip integrated circuit 1200 that can be fabricated using one or more IP cores in accordance with embodiments. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and can additionally include an image processor 1215 and / or a video processor 1220, any of which can be a modular IP core from the same or multiple different design facilities. The integrated circuit 1200 includes peripheral or bus logic including USB controllers 1225, UART controllers 1230, SPI / SDIO controllers 1235, and I2S / I2C controllers 1240. In addition, the integrated circuit can also include a display device 1245 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1250 and a mobile industry processor interface (MIPI) display interface 1255. Storage can be provided by a flash memory subsystem 1260, including a flash memory and a flash memory controller. Memory interfaces can be provided via a memory controller 1265 to access SDRAM or SRAM memory devices. In addition, some integrated circuits include an embedded security engine 1270. 2 S / I 2 C controller 1240. In addition, the integrated circuit can also include a display device 1245 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1250 and a mobile industry processor interface (MIPI) display interface 1255. Storage can be provided by a flash memory subsystem 1260, including a flash memory and a flash memory controller. Memory interfaces can be provided via a memory controller 1265 to access SDRAM or SRAM memory devices. In addition, some integrated circuits include an embedded security engine 1270.

[0151] FIGS. 13A-13B is a block diagram illustrating an exemplary graphics processor for use within a SoC in accordance with embodiments described herein. FIG. 13A illustrates an exemplary graphics processor 1310 of a system on a chip integrated circuit that can be fabricated using one or more IP cores in accordance with embodiments. FIG. 13B illustrates an additional exemplary graphics processor 1340 of a system on a chip integrated circuit that can be fabricated using one or more IP cores in accordance with embodiments. FIG. 13A The graphics processor 1310 of is an example of a low power graphics processor core. FIG. 13B The graphics processor 1340 of is an example of a higher performance graphics processor core. Each of the graphics processors 1310, 1340 can be a modular IP core that can be FIG. 12 a variant of the graphics processor 1210.

[0152] As FIG. 13AAs shown in FIG, the graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A to 1315N (e.g., 1315A, 1315B, 1315C, 1315D, up to 1315N-1 and 1315N). The graphics processor 1310 can execute different shader programs via separate logic, such that the vertex processor 1305 is optimized to perform the operations of the vertex shader program, while the one or more fragment processors 1315A to 1315N perform fragment (e.g., pixel) shading operations for the fragment or pixel shader program. The vertex processor 1305 executes the vertex processing stage of the 3D graphics pipeline and generates primitives and vertex data. The fragment processor(s) 1315A to 1315N use the primitives and vertex data generated by the vertex processor 1305 to generate a frame buffer for display on the display device. In one embodiment, the fragment processor(s) 1315A through 1315N are optimized to execute fragment shader programs provided in the OpenGL API, which can be used to perform similar operations as pixel shader programs provided in the Direct 3D API.

[0153] In addition, the graphics processor 1310 also includes one or more memory management units (MMUs) 1320A-1320B, one or more caches 1325A-1325B, and one or more circuit interconnects 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual-to-physical address mappings for the graphics processor 1310, including for the vertex processor 1305 and / or (multiple) fragment processors 1315A-1315N, and the virtual-to-physical address mappings can also reference vertex or image / texture data stored in memory in addition to the vertex or image / texture data stored in the one or more caches 1325A-1325B. In one embodiment, the one or more MMUs 1320A-1320B can communicate with the system, including those within the system. FIG. 12 The graphics processor 1310 may be synchronized with other MMUs, including one or more MMUs associated with the one or more application processors 1205, the image processor 1215, and / or the video processor 1220, so that each processor 1205 to 1220 may participate in a shared or unified virtual memory system. In accordance with an embodiment, the one or more circuit interconnects 1330A to 1330B enable the graphics processor 1310 to interact with other IP cores within the SoC via an internal bus of the SoC or via a direct connection.

[0154] like FIG. 13B As shown in FIG, the graphics processor 1340 includes FIG. 13Athe one or more MMUs 1320A-B, caches 1325A-B, and circuit interconnect 1330A-B of the graphics processor 1310. The graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1455A, 1355B, 1355C, 1355D, 1355E, 1355F, up to 1355N-1 and 1355N) that provide a unified shader core architecture in which a single core or type or core can execute all types of programmable shader code, including shader program code to implement vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores in the graphics processor 1340 can vary in embodiments and implementations. In addition, the graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher and task manager to accelerate tasks with thread dispatch granularity that is coarser than an individual task. In addition, the graphics processor 1340 includes a tiling unit 1358 to accelerate tiled rendering operations in which a rendering operation is subdivided into tiled regions that are rendered independently, for example to take advantage of local spatial coherence or to optimize use of internal caches.

[0155] FIGS. 14A-14B Additional exemplary graphics processor logic in accordance with embodiments described herein is shown. FIG. 14A A graphics core 1400 is shown that can be included within a graphics processor 1210 as in FIG. 12 and can be as in FIG. 13B the unified shader cores 1355A-1355N in FIG. 14B A highly parallel general purpose graphics processing unit 1430 suitable for deployment on a multi-chip module is shown.

[0156] As in FIG. 14AAs shown, the graphics core 1400 includes a shared instruction cache 1402, a texture unit 1418, and a cache memory / shared memory 1420 common to the execution resources within the graphics core 1400. The graphics core 1400 can include multiple slices 1401A-1401N or partitions for each core, and the graphics processor can include multiple instances of the graphics core 1400. The slices 1401A-1401N can include support logic including a local instruction cache 1404A-1404N, a thread scheduler 1406A-1406N, a thread dispatcher 1408A-1408N, and a set of registers 1410A. To perform

[0157] Some of these compute units operate at a specific precision. For example, the FPUs 1414A-1414N can perform single precision (32-bit) and half precision (16-bit) floating point operations, while the DPFPUs 1415A-1415N perform double precision (64-bit) floating point operations. The ALUs 1416A-1416N can perform variable precision integer operations at 8-bit precision, 16-bit precision, and 32-bit precision, and can be configured for mixed precision operations. The MPUs 1417A-1417N can also be configured for mixed precision matrix operations, including half precision floating point operations and 8-bit integer operations. The MPUs 1417A-1417N can perform a wide variety of matrix operations to accelerate machine learning application frameworks, including enabling support for accelerated general matrix to matrix multiplication (GEMM). The AFUs 1412A-1412N can perform additional logical operations not supported by the floating point units or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0158] As FIG. 14BAs shown in , a general purpose processing unit (GPGPU) 1430 can be configured to enable highly parallel computing operations to be performed by an array of graphics processing units. In addition, GPGPU 1430 can be directly linked to other instances of GPGPU to create a multi-GPU cluster, thereby increasing the training speed of deep neural networks in particular. GPGPU 1430 includes a host interface 1432 for implementing a connection with a host processor. In one embodiment, host interface 1432 is a PCI Express interface. However, the host interface can also be a supplier-specific communication interface or communication structure. GPGPU 1430 receives commands from the host processor and uses a global scheduler 1434 to distribute the execution threads associated with those commands to a group of computing clusters 1436A to 1436H. Computing clusters 1436A to 1436H share cache memory 1438. Cache memory 1438 can act as a higher-level cache of the cache memory within computing clusters 1436A to 1436H.

[0159] GPGPU 1430 includes memory 1434A-1434B coupled to compute clusters 1436A-1436H via a set of memory controllers 1442A-1442B. In various embodiments, memory 1434A-1434B may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory.

[0160] In one embodiment, computing clusters 1436A through 1436H each include a set of graphics cores, such as FIG. 14A The graphics core 1400 may include multiple types of integer logic units and floating-point logic units that can perform computational operations within a range of precision, including those suitable for machine learning computations. For example, and in one embodiment, at least a subset of the floating-point units in each of the compute clusters 1436A through 1436H may be configured to perform 16-bit or 32-bit floating-point operations, while a different subset of the floating-point units may be configured to perform 64-bit floating-point operations.

[0161] Multiple instances of GPGPU 1430 can be configured to operate as a compute cluster. The compute mechanism used by a compute cluster to synchronize and exchange data varies across embodiments. In one embodiment, multiple instances of GPGPU 1430 communicate over the host interface 1432. In one embodiment, GPGPU 1430 includes an I / O hub 1439 that couples the GPGPU 1430 with a GPU link 1440 that enables a direct connection to other instances of the GPGPU. In one embodiment, GPU link 1440 is coupled to a specialized GPU-to-GPU bridge that enables communication and synchronization between multiple instances of GPGPU 1430. In one embodiment, GPU link 1440 is coupled with a high-speed interconnect to transmit and receive data to other GPGPUs or parallel processors. In one embodiment, multiple instances of GPGPU 1430 are located in separate data processing systems and communicate over a network device that is accessible via host interface 1432. In one embodiment, GPU link 1440 can be configured to enable a connection to a host processor in addition to or as an alternative to host interface 1432.

[0162] While the illustrated configuration of GPGPU 1430 can be configured to train a neural network, one embodiment provides an alternative configuration of GPGPU 1430 that can be configured to be deployed within a high-performance or low-power inference platform. In an inference configuration, GPGPU 1430 includes fewer compute clusters of compute clusters 1436A-1436H than in the training configuration. Additionally, the memory technology associated with memory 1434A-1434B can differ between the inference configuration and the training configuration, with higher bandwidth memory technology being dedicated to the training configuration. In one embodiment, the inference configuration of GPGPU 1430 can support inference-specific instructions. For example, the inference configuration can provide support for one or more 8-bit integer dot product instructions that are commonly used during inference operations of a deployed neural network.

[0163] Encoding and decoding feature maps and weights of convolutional neural networks

[0164] Embodiments described herein provide hardware logic for compressing CNN kernel data in main memory via integration of low-complexity and low-latency decoder logic within compute blocks of graphics processing logic. A real-time encoder is configured to compress output feature maps and write corresponding compressed data to system memory. Upon read, a real-time decoder is configured to decompress data before it is consumed by a compute unit. The techniques described herein can both reduce memory footprint of CNN feature maps in memory, reducing required transfer bandwidth, and reduce power required to read and write CNN feature maps to and from memory.

[0165] FIG. 15 is a diagram of the primitives of a convolutional neural network. A convolutional neural network (CNN) comprises a plurality of convolutional layers and subsampling layers, optionally followed by one or more fully connected layers. The convolutional layer is the core building block of a CNN and is where most of the computational operations of a CNN are performed. Within a convolutional layer, an original image 1502 having some data to be analyzed is processed by a set of convolutional kernels, each of which applies a different filter 1504A, 1504B to the original image 1502. The filters 1504A, 1504B are learnable and are typically much smaller than the original image to which the filter is applied. The convolutional kernel outputs a set of feature maps 1506A, 1506B containing the features searched by the convolutional kernel. The feature map data output from one of these convolutional layers can be used as input to a filtering operation in a subsequent convolutional layer. The results of the filtering operations can be added together to provide an output from the convolutional layer to a subsequent layer such as a pooling layer or a fully connected neural network.

[0166] FIGS. 16A-16B is a diagram of a conventional implementation of a convolutional neural network 1600. FIG. 16A The convolutional neural network 1600 shown in FIG. 1 analyzes the red, green, and blue (RGB) components 1602 of an image. In general, a CNN can be used to analyze an m×m×r image, where m is the height and width of the image and r is the number of channels. For example, an RGB image has r=3 channels. The first layer in a CNN is a convolutional layer, where the convolutional neural network 1600 is shown as having multiple convolutional layers 1604A, 1604B. The convolutional layers 1604A, 1604B can have k filters (or kernels) of size n×n×q, where n is less than the size of the image and q can be the same as or less than the number of channels r and can be different for each kernel. The size of the filters creates a locally connected structure, and each is convolved with the image to produce k feature maps of size m-n+1. Some embodiments use convolution kernels with pixel sizes that are each much smaller than the input image size or feature map size (e.g., 11*11, 9*9, 7*7, 5*5, 3*3, and / or 1*1), but the specific feature map size can be customized according to the specific requirements of the embodiment.

[0167] Convolutional layers 1604A, 1604B can feed into one or more fully connected layers 1606. High-level reasoning in a neural network can be performed in fully connected layers 1606 because neurons in a fully connected layer have full connections to all activations in the previous layer. The output of fully connected layer 1606 is typically a classification result based on high-level features of the input image.

[0168] FIG. 16BA regular CNN implementation is shown, where the size of the feature map 1612 can be limited to (m*m) and the size of the convolution kernel 1614 is limited to (n*n) resulting in a convolution result 1616 with a size of (m-n+1)*(m-n+1) without padding zeros to truncate the input, where m is significantly larger than n (e.g., (m»n)). In some implementations, the convolution result can be further reduced using a down-sampling layer or a pooling layer, which can reduce the original input image to a plurality of feature maps with a size of less than 7*7 pixels, as in some instances, highly compact feature maps can be more information dense.

[0169] FIGS. 17A-17B A memory layout of feature maps and kernel data according to an embodiment is shown. FIG. 17A An exemplary memory layout of a set of feature block units including exemplary feature block units 1702A-1702N is shown. The feature block units (e.g., feature block units 1702A-1702N) are each independently decodable units of a feature map such as feature maps 1506A, 1506B, etc. in FIG. 15 The feature block units operate on an input feature map 1711 to generate an output feature map 1721. The output feature map 1721 is then used as an input feature map for a successive layer. The feature block units can be generated by a convolution kernel in a first convolution layer and used as input to a convolution kernel in a second convolution layer, as shown by FIG. 16A The convolution layers 1604A, 1604B shown. The feature block units can be individual feature maps, portions of feature maps, or combinations of multiple feature maps. Each feature block unit can be compressed by the compression techniques described herein.

[0170] FIG. 17B An exemplary memory layout of kernel block units 1712A-1712N is shown. The kernel block units (e.g., kernel block units 1712A-1712N) are each independently decodable units of convolution kernels, weights, and / or parameters used in the computation of individual layers of a CNN. The compression techniques described herein can be applied at the granularity of individual kernel block units. Each kernel block unit can contain 16 kernels, 32 kernels, or any power of two kernels.

[0171] FIG. 18An example of a proposed compressed memory layout relative to the original data layout of the memory according to embodiments is shown. The feature block units and kernel block units are losslessly compressed. Since lossless compression does not guarantee compression ratio, the memory can be allocated to the size of uncompressed kernel blocks, and metadata can be used to indicate which portions in the compressed kernel blocks contain valid data or invalid data. For example, a group of feature block units or kernel block units (e.g., KBUs / FBUs 1802A, 1802B, 1802C through 1802N) have the original data layout shown in the memory when in uncompressed state. Then, the compressed data layout of each KBU / FBU 1802A through 1802N can have a compressed data portion (e.g., compressed data 1812A, 1812B, 1812C through 1812N) and an invalid data portion (e.g., invalid data 1814A, 1814B, 1814C through 1814N). The size of each portion can vary for each compressed kernel block unit or feature block unit. The metadata of each compressed data block can be used to indicate which portion of a given block contains valid data.

[0172] In some embodiments, a memory controller including a direct memory access (DMA) controller can be configured to determine the size of the FBUs or KBUs according to the metadata associated with each KBU / FBU. In one embodiment, the metadata is placed at the beginning of the FBU / KBU. Alternatively, the metadata can be transmitted in a separate data stream that can be cached and prefetched by the DMA controller. For each FBU or KBU, the DMA controller can be configured to transmit only the valid compressed portion of the FBU or KBU and bypass the transmission of invalid data.

[0173] FIG. 18 An example memory layout 1820 of a fully connected layer of a CNN is also shown, in which a plurality of NxN kernel block units (e.g., KBUs 1822) are stored in the memory in the form of alternating valid compressed data bands and invalid data bands.

[0174] FIG. 19A compute architecture 1900 configured to enable compressed transmission of CNN data is shown in accordance with an embodiment. The compute architecture 1900 can be an integrated graphics processor execution logic as described herein, such as execution logic 600 of FIG. 6, for example. In one embodiment, the compute architecture 1900 includes a compute block 1902 and a hardware staging buffer 1904 coupled to a memory 1908 via a DMA controller 1906. The memory 1908 can be a main memory or system memory of a data processing system. The compute block 1902 can be a single execution unit or a cluster of execution units (e.g., EUs 608A-608N of FIG. 6, fragment processors 1315A-1315N, shader cores 1415A-1415N), as well as a streaming multiprocessor, a cluster of graphics processors, or any collection of computing resources suitable to perform processing operations required to implement a deep neural network such as a convolutional neural network described herein. The staging buffer 1904 can be a high-speed on-chip memory, such as on-chip static random access memory (SRAM). In one embodiment, the staging buffer 1904 is optimized to store units of feature blocks or units of kernel blocks for CNN operations performed by the compute block 1902.

[0175] In one embodiment, the decoder 1912 can be hardware decoder logic integrated into the compute block 1902 to enable compressed transmission of CNN data across the compute architecture. The compute block 1902 can generate output feature map (OFM) data in uncompressed format in the staging buffer 1904. The encoder 1916 is integrated into the DMA controller 1906 to enable writing of the output feature map data to the memory 1908 in compressed format. As OFMs of one layer become IFMs of the next layer, these IFMs are read from the memory 1906 as compressed data 1914 and stored in the staging buffer 1904. The decoder 1912 can enable the compute block 1902 to read in the compressed data 1914 without needing to decode the data. Alternatively, a codec unit having both encoding and decoding logic can be integrated into the DMA controller 1906 to enable the compressed data to be transmitted and read by the DMA controller 1906. The feature map data can then be decompressed by the DMA controller 1906 and written to the staging buffer 1904 in uncompressed format for reading by the compute block 1902.

[0176] In the embodiments described herein, the particular encoding format used for kernel and feature data can be changed based on the statistics of the data to be encoded. Analysis of CNN feature map data shows that many feature maps are highly sparse. Analysis of CNN kernel data shows that while kernel data is not as sparse as feature map data, many of the values in kernel data are repeated. The dynamic range of kernel data is relatively low, which indicates that the raw data is allocated more bits than are needed to store the coefficients. Using various encoding techniques, a selection of various encoding techniques can be used to compress feature map and kernel data in a lossless manner up to 80%.

[0177] In one embodiment, the encoding techniques include unique absolute value (UAV) table encoding, significance map (SM) encoding, table encoding (TE), unique value coordinate (UVC) encoding, and mean encoding (ME). The metadata for the encoded data indicates the type of encoding format used for the data. In one embodiment, a particular encoding format can be selected for a particular type of data, such as kernel data or feature data. In one embodiment, a statistical analysis is performed on the data prior to encoding to enable selection of the appropriate encoder for each block of data.

[0178] For UAV table encoding, the number of unique absolute values of an encoded kernel or feature data block can be encoded into the header. The specific unique absolute values can then be encoded, followed by encoding of an index map that enables derivation of each value of the bitstream from the unique absolute values. In the SM encoding mode, only the non-zero values in the block are encoded. The number of non-zero values in the sample block is indicated in the header, followed by a significance map indicating the mapping of non-zero values within the block. The non-zero values of the sample are then encoded in the order of their occurrence within the stream. When encoding a unique absolute value table, a table encoding (TE) mode can be enabled when the unique absolute value table forms an arithmetic progression. The start of the progression is encoded along with the offset (e.g., delta) value of the progression, followed by encoding of the remaining unique absolute values of the sample block. In the UVC encoding mode, the unique non-zero values of the block are encoded, followed by encoding of the coordinates of these unique values within the stream. The UVC encoding mode can be enabled when the number of unique values within a sample block is small and can be stored in a limited number of bits. As an example, but not a limitation, UVC encoding can be used to encode a bitstream sample that has only four unique and non-zero values. In the ME encoding mode, the mean of the sample block is encoded, followed by encoding of the difference of each value from the mean. The ME encoding mode can be enabled when the values to be encoded have a limited dynamic range and are generally clustered around a mean.

[0179] Using the encoding techniques described above, a low complexity and low latency codec can be implemented with very low area requirements, allowing multiple instances of the decoding logic to be used at various points within the processing logic, making it possible to encode and decode many systems in which each logic component is configured to read the encoded data once. FIGS. 20-22 Exemplary bit layouts for these various encoding modes are shown. However, the explicit implementation of each mode varies from embodiment to embodiment.

[0180] FIG. 20 A bitstream header 2000 is shown according to an embodiment. The bitstream header 2000 includes a first byte 2002 and a second byte 2004. In one embodiment, the first bit [0] of the first byte 2002 indicates whether the following data stream is compressed, and the remaining bits [7:1] of the first byte 2002 indicate the number of unique values ​​within the block. The second byte 2004 indicates whether certain encoding modes are enabled for compressed data. For example, and in one embodiment, the first bit [0] can indicate whether SM mode encoding is enabled, and the second bit [1] indicates whether TE mode encoding is enabled. A group of bits [5:2] following the first two bits can indicate a value N, where N=log2M, and M is the number of kernel blocks or feature blocks encoded in the sample block after the header. Bits [7:6] of the second byte 2004 indicate the size of the kernel associated with the bitstream.

[0181] FIG. 21 A table-encoded bitstream layout 2100 is shown according to an embodiment. The table-encoded bitstream layout 2100 can be used in the table-encoded (TE) mode described herein, in which a series of unique absolute values ​​is encoded with a reduced number of bits. TE mode can also be referred to as arithmetic progression encoding because TE mode can be specifically enabled when a set of unique absolute values ​​includes an arithmetic progression. In one embodiment, table-encoded data is presented starting with the third byte after the two-byte header, where the presence of table-encoded data is indicated by FIG. 20 The second bit (eg, [1]) of the second byte 2004 of the header shown in indicates However, other embodiments are not limited to this particular configuration. FIG. 21 Table encoding bit stream layout 2100 includes a first byte 2102, a second byte 2104, and a variable number of consecutive bytes 2106. The first byte 2102 contains the start of the sequence value of the encoded arithmetic sequence. The second byte 2104 defines the offset of the arithmetic sequence, which defines the difference between each value in the sequence. The length of the consecutive bytes 2106 is variable and can be used to encode the remaining unique absolute values. For example, to encode a bit stream with the values ​​(0, 3, 6, 9, 12, 27, 32, 63), these values ​​can be encoded as shown in Table 1 below.

[0182] Table 1 - Example table encoding

[0183] Sequence Start Offset Remaining UAV 0x0 0x3 0x1B; 0x20; 0x40

[0184] As shown in Table 1, the values (0, 3, 6, 9, 12) are encoded using a sequence value of 0x0 start, 0x3 offset. The metadata (e.g., bits [7:1] of the first byte 2002 in FIG. 20 indicates the total number of unique values within the encoded block, and the remaining UAV field indicates the three total values, from which the decoding logic can determine the first five values of the bitstream to be (0, 3, 6, 9, 12) using the sequence start and offset values.

[0185] FIG. 22 Various layouts for bitstream encoding are shown in accordance with embodiments. The bitstream encoding layouts are examples of one embodiment, as other embodiments can be configured to use modified versions of the shown layouts. A first layout 2210 can be used for UAV table encoding, in which the number of unique absolute values of an encoded kernel or feature data block can be encoded into a header 2212. The specific unique absolute values 2214 are then encoded, followed by an index map 2216 that enables each value of the bitstream to be derived from the unique absolute values. In FIG. 23 Decoding of an exemplary bitstream having the first layout 2210 is shown in

[0186] A second layout 2220 can be used for SM encoding mode, in which the number of non-zero values in a sample block is indicated in a header 2222, followed by a validity map 2224 that indicates the mapping of non-zero values within the block. The non-zero values 2226 of the sample are then encoded in the order of their occurrence within the stream. In FIG. 24 Decoding of an exemplary bitstream having the second layout 2220 is shown in

[0187] A third layout 2230 can be used for UVC encoding, in which the unique values in the unique value bitstream samples of the block are encoded, followed by the coordinates of these unique values within the stream. A header 2232 can indicate the number of non-zero values being encoded, coordinates 2234 indicate the location (e.g., offset) of the non-zero values within the bitstream, and a set of non-zero values 2236 in the bitstream are encoded in the order of their occurrence within the bitstream. In FIG. 25 Decoding of an exemplary bitstream having the third layout 2230 is shown in

[0188] A fourth layout 2240 can be used for mean encoding (ME) mode, in which the number of values to be encoded is indicated in a header 2242, the mean of the segment is indicated 2244, and the remaining values are encoded as residual values 2246. The residual values can be encoded with a reduced number of bits, as only the difference between the decoded value and the mean is encoded.FIG. 26 Decoding of an example bitstream 2310 from a bitstream having a first layout 2210 of

[0189] FIG. 23 Decoding of an example bitstream 2310 from a bitstream having a first layout 2210 of FIG. 22 In one embodiment, the illustrated layout of FIG. 23 The illustrated layout of

[0190] In one embodiment, the set of unique absolute values 2214 are stored in a sequential order, with the zeroth entry reserved for the value zero. In this configuration, the output buffer of the bitstream can be initialized to zero, and during decoding, the least significant bit of each index 2320 can be checked to determine if a non-zero value was referenced before performing a full decoding operation on the index. If the least significant bit of the index is zero, indicating that the output value will be zero, then the decoding of the index value can be bypassed.

[0191] FIG. 24 Decoding of an example bitstream 2410 from a bitstream having a second layout 2220 of FIG. 22 In one embodiment, the illustrated layout of FIG. 24 The illustrated layout of FIG. 20The first bit (e.g., [0]) of the second byte 2004 of the header shown in is indicated. The number of non-zero values ​​in the sample block is indicated in header 2222, along with metadata indicating that SM encoding is enabled. Next, a validity map 2224 is encoded that indicates a mapping of non-zero values ​​within the block, with one bit entry associated with each value. The non-zero values ​​2226 of the samples are then encoded in the order of their appearance within the stream. To decode the exemplary bitstream 2410, the decoder logic may initialize the output data buffer to zero. The decoder logic may then reference validity map 2224 to determine which values ​​in the bitstream are non-zero and unpack these values ​​sequentially at the output locations indicated by the validity map. For example, a value of zero (0b0) in validity map 2224 indicates that the corresponding decoded value is zero. A value of one (0b1) in validity map 2224 indicates that the corresponding decoded value corresponds to the next consecutive entry in non-zero values ​​2226 in the encoded bitstream.

[0192] FIG. 25 Demonstrated the FIG. 22 In one embodiment, FIG. 25 The illustrated layout is for UVC encoding mode. Header 2232 indicates the number of non-zero values ​​in the sample block, along with metadata indicating that UVC encoding mode is enabled. Next, the value coordinates 2234 are encoded, indicating the location or offset of the non-zero values ​​within the block. The non-zero values ​​2236 of the samples are then encoded in the order of their appearance within the stream. To decode the example bitstream 2410, the decoder logic can initialize the output data buffer to zero. Since only a small number of non-zero values ​​are found within the bitstream, the decoder logic can reference the coordinates 2234 of these values ​​to specifically determine where to place the non-zero values ​​within the decoded stream.

[0193] FIG. 26 Demonstrated the FIG. 22 In one embodiment, the exemplary bitstream 2610 of the fourth layout 2240 of the bitstream is decoded. FIG. 26The illustrated layout of is for mean encoding mode. The number of encoded values can be indicated in header 2242, along with metadata indicating that mean encoding is enabled. The mean 2244, which is the arithmetic mean of the values as a bitstream, is stored after the header, followed by residual values 2246, which are the differences between the original values and the mean 2244. In one embodiment, each residual value 2620 can be stored in a reduced bit representation that enables a first set of bits 2622 (e.g., [2:0]) to store the absolute value of the difference between the original value and the mean, with an additional bit 2624 (e.g., [3]) indicating the sign of the difference. For example, for the example bitstream 2610 with a mean 2244 of sixteen (e.g., 0x10), the values (16, 16, 14, 16, 18) can be represented as residual values (0, 0, -2, 0, +2).

[0194] FIG. 27 is a flowchart illustrating a method of performing a CNN operation according to an embodiment. In one embodiment, FIG. 27 the method is performed via FIG. 19 the computing architecture 1900 of, although different computing architectures can be configured to perform the illustrated method. The method includes decoding encoded kernel data while reading the encoded kernel data from memory, as shown at 2702. The decoding can be performed by decoding logic within the compute logic, within a DMA controller associated with the compute block, or in a separate codec. In one embodiment, kernel data can be stored persistently in non-volatile memory in encoded format, loaded into memory in encoded format, and held in encoded format until the kernel is consumed by the compute block.

[0195] Next, the compute logic (e.g., compute block, GPGPU logic, etc.) can be configured to generate feature map data for a CNN layer based on the kernel data, as shown at 2704. The feature map data for the CNN layer is then encoded during a write to memory, as shown at 2706. The compute logic can then read the encoded feature map data from memory, and decode the encoded feature map data during the read, as shown at 2708. The compute logic can then process the feature map data as input feature map data for a next CNN layer, as shown at 2710.

[0196] FIG. 28 is a flowchart of a method of encoding CNN data according to an embodiment. In one embodiment, FIG. 28 the method is performed via an encoder within a DMA controller, such as FIG. 19 the encoder 1916 and DMA controller 1906 of. In one embodiment, the method can be performed via a separate codec that includes an encoder, or can be performed by encoding logic within a compute block.

[0197] As different encoding methods have different efficiencies depending on the type of data to be encoded, the encoding logic can analyze the kernel or feature map data to be encoded, as shown at 2802. The encoding logic can then determine an encoding mode based on the data characteristics of the kernel or feature map data, as shown at 2804.

[0198] The encoding technique used for a kernel or feature map data segment can be determined based on data characteristics such as sparsity or dynamic range of the data. Mean encoding can be used to encode data with very low dynamic range. Unique value coordinate encoding can be used to encode data with very small number of unique values. In one embodiment, UAV table encoding can be used as the default encoding method, where significance map (SM) encoding or table encoding (TE) is enabled for certain types of data to further improve the efficiency of the UAV table encoding method.

[0199] The encoder can then encode the data based on the determined encoding mode at 2806, and write the metadata for the encoded data segment to enable decoding of the encoded data segment, as shown at 2808. To decode the encoded data segment, the decoder can read the metadata to determine the amount of data that was encoded and the type of encoding used on the data. The decoder can then decode the indicated amount of data using the decoding method associated with the indicated encoding mode.

[0200] Compression formats with separate metadata storage

[0201] In some embodiments, the compression formats described herein can be modified to store certain elements of the compression metadata separately from the compressed data. For example, and in one embodiment, the metadata can be stored within a separate control face of the compressed data block, where the metadata can indicate various compression details, including the compression type used for the compressed data block, and other descriptive information about the compressed data.

[0202] FIG. 29 Compression using a significance map compression mode is shown in accordance with additional embodiments. In one embodiment, the uncompressed data block 2920 can be compressed using a significance map (SM) compression mode, where a significance map 2224 is stored for the compressed data to indicate which portions of the data contain zero values and which portions of the data contain non-zero values. The significance map 2224 can indicate non-zero values with a bit value of one (Ob 1) and zero values with a bit value of zero (Ob 0). The non-zero values 2226 of the samples are then encoded in the order of their occurrence within the stream.

[0203] In one embodiment, a separate metadata block 2930 is stored for a compressed data region to indicate the compression format of the portion, as well as other descriptive information related to the compressed data portion.

[0204] FIG. 30 Compression using a unique value coordinate compression mode is shown according to additional embodiments. In one embodiment, a unique value coordinate (UVC) compression mode can be used to compress an uncompressed data block 3020, where only the unique values within a given block are stored. A set of unique non-zero values 2236 in a bitstream can be encoded in the order of their occurrence within the bitstream. In the shown variant of the UVC mode, the coordinates of each of the unique non-zero values 2236 within the bitstream can be stored within metadata 3030, rather than in the coordinates stored adjacent to the unique non-zero values within the compressed bitstream (e.g., as in the coordinates 2234 in FIG. 25

[0205] FIG. 31 Encoding and decoding operations utilizing separate metadata are shown according to embodiments. In one embodiment, original data 3102 can be encoded via an encoder 3104 and decoded utilizing a decoder 3124. In one embodiment, the encoder 3104 and decoder 3124 can each be included in a DMA controller 3103. Encoded data 3112 can occupy a reduced memory footprint in memory relative to unencoded data, freeing up memory for storing additional data. In one embodiment, encoded data 3112 and metadata 3114 can be stored separately in memory 3110, allowing metadata 3114 to be easily transmitted in a separate data stream from encoded data 3112. Transmitting metadata 3114 as a separate data stream enables metadata 3114 to be cached and / or prefetched by DMA controller 3103, which can preconfigure decoder 3124 based on the prefetched metadata 3114. The configured decoder 3124 can then output decoded data 3122 for processing by a computing element as described herein.

[0206] FIG. 32 An exemplary codec system 3200 according to embodiments is shown. Codec system 3200 can be included in a general purpose graphics processor, general purpose processor, or integrated many core processor as described herein to compress data associated with a neural network prior to storing such data to memory, thereby reducing the memory footprint required to perform training operations and / or inference operations using a neural network.

[0207] ​In one embodiment, codec system 3200 includes a DMA controller 3210 that is communicatively coupled with a host 3230. Host 3230 can be connected to DMA controller 3210 via a fabric interconnect, a memory interconnect, or a peripheral interconnect. DMA controller 3210 includes a controller 3215, an encoder 3219, and a decoder 3221. Host 3230 can communicate with configuration registers 3211 via an advanced peripheral bus (ABP) 3212. Configuration registers 3211 can be used to configure the operation of controller 3215, encoder 3219, and decoder 3221. When controller 3215 is configured to perform DMA operations to and from memory, controller 3215 can be configured to encode data written to memory via encoder 3219 and to decode data read from memory via decoder 3221.

[0208] Encoding / decoding unit 3210 includes an interface master 3226 and an interface slave 3224 that enable encoding / decoding unit 3210 to communicate with an interconnect fabric or interconnect bus. In embodiments, various interconnects can be coupled to interface master 3226 and interface slave 3224. In one embodiment, interface master 3226 and interface slave 3226 can be connected to an advanced extensible interface used in high clock frequency systems. Interface master 3226 and interface slave 3224 each include connection lanes, including but not limited to read / write lanes 3234A-3234B and address read / address write lanes 3236A-3236B. A metadata buffer 3214 is coupled between interface master 3226 and controller 3215 to buffer a stream of metadata read from memory of host 3230. In one embodiment, a bus access management channel (bchan) 3213 couples interface master 3226 with controller 3215.

[0209] In one embodiment, address read / address write lanes 3216 are provided by controller 3215 for output via interface slave 3224. Data to be written to memory is provided via write lanes 3217, which can be temporarily stored in one or more buffers 3218 before being encoded by encoder 3219 and written to memory of host 3230. Data to be read can be stored in a reordering buffer 3220 before being decoded by decoder 3221 and written out via read lanes 3222. Reordering buffer 3221 enables DMA controller 3210 to complete out-of-order read operations or receives.

[0210] Compression for deep learning with sparse value mapping to non-zero values

[0211] The encoding and decoding techniques and logic described herein enable data associated with neural networks (e.g., weight data, activation data, other parameter data, etc.) to be efficiently compressed and decompressed during writes and reads to memory, thereby reducing the memory footprint required for computational operations using neural networks. One embodiment leverages the above-described logic and techniques with an additional encoding mode enhancement to provide more efficient encoding for techniques that attempt to compress data streams by mapping values with high frequency of occurrence to zero values. In such an embodiment, the highest frequency value in a bitstream is identified, and an effectiveness map is generated in which the location of the highest frequency value is mapped to a zero (0b0) in the effectiveness map. Other values are indicated by a one (0b1) in the effectiveness map. The bitstream of remaining values can be sent in the encoded bitstream. A separate metadata stream is also transmitted. In this encoding format, the first byte of the encoded bitstream contains the high frequency value. Instead of replacing zeros as in the effectiveness map mode, the high frequency value is replaced.

[0212] FIG. 33 Compression for machine learning data with a single high frequency value according to embodiments described herein is illustrated. In one embodiment, an uncompressed data block 3310 can be processed by the encoding logic described herein. The processing can include an operation 3315 to generate a histogram of occurrences for each element in the uncompressed data block 3310. A histogram of occurrences 3320 for each element in the uncompressed data block 3310 can be generated to determine the element with the highest frequency 3321. The value 3224 of the highest frequency element can be stored in the encoded stream, followed by a storage data block effectiveness map 2224 that indicates the original location of each element within the uncompressed data block 3310. The non-zero values 2226 of the samples are then encoded in the order of occurrence within the stream, and metadata 3330 is generated for the compressed data block to describe the type of compression used for the data block. The encoded data can then be written to memory along with the metadata 3330 describing the compression of the encoded data.

[0213] In one embodiment, the illustrated effectiveness map encoding variant in which zeros are mapped to the high frequency value can be designated as a different encoding type relative to effectiveness map encoding, and can have a different encoding identifier within the metadata of the encoded block. For example, and in one embodiment, this separate encoding technique can be associated with a particular encoding technique used by one or more neural network frameworks or machine learning frameworks, such as but not limited to the low precision general matrix multiplication (GEMMlowp) library, or an equivalent library in which sparse values are mapped to the high frequency value.

[0214] FIG. 34is a flow chart illustrating a process 3400 for implementing compression for deep learning where sparse values ​​are mapped to non-zero values, according to an embodiment. FIG. 32 The process is implemented using a codec system 3200. Such a codec system can be integrated into a processing device (e.g., a CPU, an MIC processor, a GPGPU, an FPGA, etc.) as described herein. In one embodiment, the codec system is integrated into a DMA controller.

[0215] As shown at block 3402, logic associated with or integrated within the codec system may analyze the kernel data to be encoded. Such logic may determine that high frequency values ​​of the kernel data are mapped to zero values, as shown at block 3404. The codec may then map the high frequency values ​​to zero and store a single instance of the high frequency value in the encoded data, as shown at block 3406. As shown at block 3408, the encoder may then encode the kernel data into encoded kernel data, the encoded kernel data including a mapping of the high frequency values. In one embodiment, the codec may encode the kernel data using an encoding mapping where zeros are mapped to high frequency values, as shown at block 3409. FIG. 33 At block 3410, the codec may then write the encoded kernel data and associated metadata to memory. The metadata may be used to reverse the encoding when reading the kernel data from memory and includes an identifier of the encoding technique used to encode a given block of data.

[0216] FIG. 35 is a block diagram of a computing device 3500 including a graphics processor 3504 according to an embodiment. The computing device 3500 may be FIG. 1 The computing device 3500 may also be or be included in a communication device, such as a set-top box (e.g., an Internet-based cable TV set-top box, etc.), a device based on a global positioning system (GPS), etc. The computing device 3500 may also be or be included in a mobile computing device, such as a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, an e-reader, a smart TV, a TV platform, a wearable device (e.g., glasses, watches, bracelets, smart cards, jewelry, clothing, etc.), a media player, etc. For example, in one embodiment, the computing device 3500 includes a mobile computing device that uses an integrated circuit (“IC”), such as a system on a chip (“SoC” or “SOC”) that integrates various hardware components and / or software components of the computing device 3500 on a single chip.

[0217] The computing device 3500 includes a graphics processor 3504. The graphics processor 3504 represents any graphics processor described herein. The graphics processor includes one or more graphics engines, graphics processor cores, and other graphics execution resources as described herein. Such graphics execution resources can take a variety of forms, including but not limited to execution units, shader engines, fragment processors, vertex processors, stream processors, graphics processor clusters or any collection of computing resources adapted to process graphics resources or image resources, or to perform general purpose computing operations.

[0218] In one embodiment, the graphics processor 3504 includes a cache 3514, which can be a single cache or divided into multiple cache segments including but not limited to Ll, L2, L3, or L4 caches, a render cache, a depth cache, a sampler cache, and / or a shader unit cache. In some embodiments, the graphics processor 3504 includes a CNN logic unit 3524, a GPGPU engine 3544, and a DMA unit 3554. In one embodiment, the CNN logic unit 3524 is a fixed function logic unit that can be configured to generate feature map data using the GPGPU engine 3544. In one embodiment, the DMA unit 3554 includes a codec 3534 containing encoding and decoding logic for encoding neural network data prior to transmission of the data to memory via the DMA unit 3554. In one embodiment, the codec 3534 can be configured to losslessly encode and decode feature map data and kernel data for a neural network. In alternative embodiments, the encoding logic for the feature map codec is located in the DMA unit 3554, and the decoding logic is located in the CNN logic unit 3524 and / or the GPGPU engine 3544.

[0219] As shown, in one embodiment, and in addition to the graphics processor 3504, the computing device 3500 can further include any number and type of hardware components and / or software components, including but not limited to an application processor 3506, a memory 3508, and an input / output (I / O) source 3510. The application processor 3506 can interact with the hardware graphics pipeline as shown to share graphics pipeline functionality. Processed data is stored in buffers of the hardware graphics pipeline, and state information is stored in the memory 3508. The resulting data can be communicated to a display controller to be displayed via a display such as a liquid crystal display (LCD) or organic light emitting diode (OLED) display. FIG. 3 As shown, the application processor 3506 can interact with the hardware graphics pipeline to share graphics pipeline functionality. Processed data is stored in buffers of the hardware graphics pipeline, and state information is stored in the memory 3508. The resulting data can be communicated to a display controller to be displayed via a display such as a liquid crystal display (LCD) or organic light emitting diode (OLED) display. FIG. 3display device 320, among other display devices, to output. The display device can belong to various types, such as a cathode ray tube (CRT), a thin-film transistor (TFT), a liquid crystal display (LCD), an organic light-emitting diode (OLED) array, and the like, and can be configured to display information to a user via a graphical user interface.

[0220] The application processor 3506 can include one or more processors, such as FIG. 1 the processor(s) 102 of the computing device 3500, and can be a central processing unit (CPU) that is at least partially used to execute an operating system (OS) 3502 of the computing device 3500. The OS 3502 can serve as an interface between hardware resources and / or physical resources of the computing device 3500 and one or more users. The OS 3502 can include driver logic 3522 for various hardware devices in the computing device 3500. The driver logic 3522 can include graphics driver logic 3523, such as the user-mode graphics driver 1026 and / or the kernel-mode graphics driver 1029 of the computing device 1000. FIG. 10

[0221] It is contemplated that, in some embodiments, the graphics processor 3504 can exist as part of the application processor 3506 (such as part of a physical CPU package), in which case at least a portion of the memory 3508 can be shared by the application processor 3506 and the graphics processor 3504, although at least a portion of the memory 3508 can be dedicated to the graphics processor 3504, or the graphics processor 3504 can have a separate memory storage device. The memory 3508 can include pre-allocated regions of buffers (e.g., frame buffers); however, one of ordinary skill in the art will appreciate that embodiments are not so limited and any memory accessible to a low-level graphics pipeline can be used. The memory 3508 can include various forms of random access memory (RAM) (e.g., SDRAM, SRAM, etc.), including for applications that utilize the graphics processor 3504 to render desktop or 3D graphics scenes. Such as the graphics processor 3504 can be configured to access the memory 3508 via a memory controller 3524. FIG. 1 ​The memory controller hub 116, among other potential memory controllers, can access the data in the memory 3508 and forward the data to the graphics processor 3504 for processing in the graphics pipeline. The memory 3508 can be made available to other components of the computing device 3500. For instance, any data received from various I / O sources 3510 of the computing device 3500 (e.g., incoming graphics data) can be temporarily queued in the memory 3508 prior to being operated on by one or more processors (e.g., the application processor 3506) in implementing a software program or application. Similarly, data determined by the software program to be sent from the computing device 3500 through one of the computing system interfaces to an external entity or stored to an internal storage element is often temporarily queued in the memory 3508 prior to being transmitted or stored.

[0222] The I / O sources can include devices such as touchscreens, touch panels, touch pads, virtual or regular keyboards, virtual or regular mice, ports, connectors, network devices, and the like, and can be attached via a platform controller hub 130 as cited in FIG. 1 Additionally, the I / O sources 3510 can include one or more I / O devices implemented to transfer data to or from the computing device 3500 (e.g., a network adapter) or for large scale non-volatile storage within the computing device 3500 (e.g., a hard disk drive). User input devices that include alphanumeric and other keys can be used to communicate information and command selections to the graphics processor 3504. Another type of user input device is a cursor control device, such as a mouse, a trackball, a touch screen, a touchpad, or a cursor direction key, for communicating direction information and command selections to the GPU and for controlling cursor movement on a display device. Cameras and microphone arrays of the computing device 3500 can be employed to observe gestures, record audio and video, and receive and emit visual and audio commands.

[0223] The I / O sources 3510 configured as network interfaces can provide access to networks such as LANs, wide area networks (WANs), metropolitan area networks (MANs), personal area networks (PANs), Bluetooth, cloud networks, cellular or mobile networks (e.g., 3rd Generation (3G), 4th Generation (4G), etc.), intranets, the Internet, and the like. The network interface(s) can include, for example, a wireless network interface having one or more antennas. The network interface(s) can also include, for example, a wired network interface to communicate with remote devices via network cables, which can be, for example, Ethernet cable, coaxial cable, optical cable, serial cable, or parallel cable.

[0224] The network interface(s) can provide access to a LAN, for example, by conforming to IEEE 802.11 standards, and / or the wireless network interface can provide access to a personal area network, for example, by conforming to Bluetooth standards. Other wireless network interfaces and / or protocols can also be supported, including previous and subsequent versions of the standards mentioned. In addition to or instead of communicating via a wireless LAN standard, the network interface(s) can provide wireless communications using, for example, Time Division Multiple Access (TDMA) protocols, Global System for Mobile Communications (GSM) protocols, Code Division Multiple Access (CDMA) protocols, and / or any other type of wireless communications protocol.

[0225] It should be appreciated that for some embodiments, systems less than or more than the examples described in the foregoing can be preferred. Therefore, depending upon numerous factors, such as price constraints, performance requirements, technological improvements, or other considerations, the configuration of the computing device 3500 can be changed as embodiments are implemented. Examples include, but are not limited to: a mobile device, a personal digital assistant, a mobile computing device, a smartphone, a cellular telephone, a handset, a one-way pager, a two-way pager, a messaging device, a computer, a personal computer (PC), a desktop computer, a laptop computer, a notebook computer, a handheld computer, a tablet computer, a server, a server array or server farm, a web server, a network server, an Internet server, a work station, a minicomputer, a mainframe computer, a supercomputer, a network appliance, a web appliance, a distributed computing system, multiprocessor systems, processor-based systems, consumer electronics devices, programmable consumer electronics devices, television, digital television, set-top boxes, wireless access points, base stations, subscriber stations, mobile subscriber centers, radio network controllers, routers, hubs, gateways, bridges, switches, machines, or a combination of any of the above.

[0226] Embodiments can be implemented as any or a combination of: one or more microchips or integrated circuits interconnected using a motherboard, a hardwired logic circuit, software stored by a memory device and executed by a microprocessor, firmware, an application specific integrated circuit (ASIC), and / or a field programmable gate array (FPGA). By way of example, the term "logic" can include software or hardware and / or a combination of software and hardware.

[0227] Embodiments can be provided, for example, as a computer program product which can include one or more machine-readable media having stored thereon machine-executable instructions that, when executed by one or more machines such as a computer, network of computers, or other electronic devices, can cause the one or more machines to perform the operations described herein in accordance with embodiments described herein. The machine-readable media can include, but is not limited to, floppy diskettes, optical disks, CD-ROMs (Compact Disc-Read Only Memories), and magneto-optical disks, ROMs (Read Only Memories), RAMs (Random Access Memories), EPROMs (Erasable Programmable Read Only Memories), EEPROMs (Electrically Erasable Programmable Read Only Memories), magnet or optical cards, flash memory, or other type of media suitable for storing machine-executable instructions.

[0228] Moreover, embodiments can be downloaded as a computer program product, wherein the program can be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of one or more data signals embodied in and / or modulated by a carrier wave or other propagation medium via a communication link (e.g., a modem and / or a network connection).

[0229] The following clauses and / or examples pertain to specific embodiments or examples. The details of the embodiments or examples can be modified in any manner that is apparent to one of skill in the art, while remaining within the scope of the embodiments or examples. Various features and characteristics of the different embodiments or examples can be combined or separated to form various other embodiments or examples. The examples can include subject matter such as a method, means for performing the method, at least one machine-readable medium including instructions that, when performed by a machine cause the machine to perform the actions of the method, or of an apparatus or system according to embodiments and examples described herein. The various components can be implemented as devices for performing the operations or functions described.

[0230] Embodiments described herein provide a processing device comprising compute logic to generate neural network data for a convolutional neural network (CNN) and to write the neural network data to a memory buffer. The compute logic additionally comprises a direct memory access (DMA) controller comprising a hardware codec having an encoding unit and a decoding unit, the DMA controller to: read the neural network data from the memory buffer; encode the neural network data via the encoding unit; write encoded neural network data to a memory device coupled with the processing device; write metadata for the encoded neural network data to the memory device coupled with the processing device; and decode encoded neural network data via the decoding unit in response to a request from the compute logic. In one embodiment, the compute logic can request the DMA controller to read the encoded feature map data from the memory device. In response to the request from the compute logic, the DMA controller can prefetch the metadata for the encoded feature map data and decode the encoded neural network data based on the prefetched metadata. The neural network data can comprise feature map data and kernel data.

[0231] In one embodiment, the hardware codec can encode the feature map data using an encoding mode selected from a set of multiple encoding modes. The set of multiple encoding modes comprises an encoding mode to encode the neural network data in reduced bit representations via encoding two or more of unique absolute values, non-zero values, and residual values. The set of multiple encoding modes can additionally comprise an encoding mode to encode a sequence of equal difference values in reduced bit representations. In one embodiment, the set of multiple encoding modes additionally comprises an encoding mode to encode neural network data having high frequency values in reduced bit representations. The compute logic is comprised within a general purpose graphics processing unit.

[0232] One embodiment provides a method of performing processing operations to implement a convolutional neural network (CNN), the method comprising: decoding encoded kernel data of the CNN while reading the encoded kernel data from memory; generating feature map data of a certain layer of the CNN using decoded kernel data via compute logic within a general purpose graphics processing unit; encoding the feature map data of the layer of the CNN via hardware encoding logic within a direct memory access (DMA) controller during writing to memory; decoding encoded feature map data while reading the encoded feature map data from memory; and processing the feature map data as input feature map data for a next layer of the CNN, wherein decoding the encoded kernel data includes prefetching metadata associated with the encoded kernel data, the metadata associated with the encoded kernel data being stored separately from the encoded kernel data in a different set of memory addresses.

[0233] In one embodiment, decoding the encoded feature map data includes prefetching metadata associated with the encoded feature map data, the metadata associated with the encoded feature map data being stored separately from the encoded feature map data. The feature map data can be decoded via hardware decoding logic within the DMA controller. In one embodiment, encoding the feature map data via hardware encoding logic is performed using one or more encoding modes selected from a set of multiple encoding modes. The set of multiple encoding modes includes encoding modes for storing kernel data or feature map data in a reduced bit representation via encoding two or more of unique absolute values, non-zero values, and residual values, and wherein the set of multiple encoding modes additionally includes an encoding mode for encoding a sequence of equal difference values in a reduced bit representation.

[0234] In one embodiment, the method additionally includes encoding kernel data of the CNN via the hardware encoding logic within the DMA controller. Encoding the kernel data can include analyzing the kernel data to be encoded, determining that a high frequency value of the kernel data is mapped to a zero value, storing the high frequency value into an encoded kernel data block, encoding the kernel data into the encoded kernel data block, the encoded kernel data including the mapping of the high frequency value, and writing the encoded kernel data and the metadata associated with the encoded kernel data to memory.

[0235] One embodiment provides a data processing system configured to perform operations to implement a convolutional neural network (CNN). The data processing system includes a memory device to store feature map data of the CNN, a non-volatile storage device to persistently store kernel data of the CNN, and a processor including a general purpose graphics processor compute block and a DMA controller. The general purpose graphics processor compute block can generate output feature map data of the CNN using the kernel data and write the output feature map data to a memory buffer within the processor. The DMA controller includes a hardware codec including an encoding unit to automatically encode the output feature map data during writing of the output feature map data to the memory device. In one embodiment, the data processing system can include any aspect of the processing apparatus described above and can perform operations of a method as described herein.

[0236] Those skilled in the art will appreciate from the foregoing description that the broad techniques of the embodiments can be implemented in a variety of forms. Therefore, although the embodiments have been described in connection with certain examples, the actual scope of the embodiments should not be so limited.

Claims

1. A processing device comprising: computation logic to generate neural network data for a convolutional neural network (CNN) and to write the neural network data to a memory buffer, wherein the neural network data comprises feature map data and kernel data; and a direct memory access (DMA) controller comprising a hardware codec having an encoding unit and a decoding unit, the DMA controller to: read the neural network data from the memory buffer; encode the neural network data via the encoding unit; write encoded neural network data to a memory device coupled with the processing device; write metadata for the encoded neural network data to the memory device coupled with the processing device, the metadata being written to a different memory block in the memory device than the encoded neural network data; and decode encoded neural network data via the decoding unit in response to a request from the computation logic.

2. The processing device of claim 1, wherein: the computation logic is to request the DMA controller to read the encoded neural network data from the memory device; in response to the request from the computation logic, the DMA controller is to prefetch the metadata for the encoded neural network data; and the DMA controller is to decode the encoded neural network data based on the prefetched metadata.

3. The processing device of claim 1, wherein, To encode the kernel data, the DMA controller is to: analyze the kernel data to be encoded; determine that high frequency values of the kernel data are mapped to zero values; generate a mapping for each instance of the high frequency values in the kernel data; encode the kernel data into encoded kernel data, the encoded kernel data comprising the mapping for each instance of the high frequency values; and write the encoded kernel data and metadata associated with the encoded kernel data to the memory device.

4. The processing device of claim 1, wherein, the hardware codec is to encode the feature map data using an encoding mode selected from a set of multiple encoding modes.

5. The processing device of claim 4, wherein, the set of multiple encoding modes comprises an encoding mode to encode the neural network data via encoding two or more of unique absolute values, non-zero values, and residual values to reduce bit representation.

6. The processing device of claim 5, wherein, the set of multiple encoding modes additionally comprises an encoding mode to encode a sequence of difference values in reduced bit representation.

7. The processing device of claim 6, wherein, the set of multiple encoding modes additionally comprises an encoding mode to encode neural network data having high frequency values in reduced bit representation.

8. The processing device of claim 1, wherein, the computation logic to generate the neural network data is computation logic within a general purpose graphics processing unit.

9. A method of performing processing operations to implement a convolutional neural network (CNN), the method comprising: decoding encoded kernel data for the CNN while reading the encoded kernel data from memory, wherein encoding the kernel data for the CNN comprises: analyzing the kernel data to be encoded; determining that high frequency values of the kernel data are mapped to zero values; storing the high frequency values into an encoded kernel data block; encoding the kernel data into the encoded kernel data block, the encoded kernel data including the mapping of the high frequency values; and writing encoded kernel data and metadata associated with the encoded kernel data to memory; generating feature map data for a layer in the CNN using decoded kernel data via compute logic within a general purpose graphics processing unit.

10. The method of claim 9, further comprising: encoding the feature map data for the layer of the CNN via hardware encoding logic within a direct memory access (DMA) controller during a write to memory; decoding encoded feature map data while reading the encoded feature map data from memory; and processing the feature map data as input feature map data for a next layer in the CNN, wherein decoding the encoded kernel data includes prefetching metadata associated with the encoded kernel data, the metadata associated with the encoded kernel data stored separately from the encoded kernel data.

11. The method of claim 10, wherein, decoding the encoded feature map data includes prefetching metadata associated with the encoded feature map data, the metadata associated with the encoded feature map data stored separately from the encoded feature map data.

12. The method of claim 10, additionally comprising: decoding the feature map data via hardware decoding logic within the DMA controller.

13. The method of claim 10, additionally comprising: encoding the feature map data via hardware encoding logic using one or more encoding modes selected from a set of multiple encoding modes, wherein the set of multiple encoding modes includes encoding modes for storing kernel data or feature map data in a reduced bit representation via encoding of two or more of unique absolute values, non-zero values, and residual values, and wherein the set of multiple encoding modes additionally includes encoding modes for encoding a sequence of difference values in a reduced bit representation.

14. The method of claim 10, further comprising: encoding kernel data for the CNN via the hardware encoding logic within the DMA controller.

15. The method of claim 9, wherein, encoding the kernel data for the CNN includes writing the metadata to a different memory block than the encoded kernel data.

16. A machine readable medium having stored thereon data, which if used by at least one machine, causes the at least one machine to fabricate at least one integrated circuit for performing a method of performing processing operations to implement a convolutional neural network (CNN) as recited in any of claims 9-15.

17. A data processing system configured to perform operations to implement a convolutional neural network (CNN), the data processing system comprising: a memory device to store feature map data for the CNN; a non-volatile storage device to persistently store kernel data for the CNN; a processor, the processor including a general purpose graphics processor compute block and a DMA controller; wherein the DMA controller includes a hardware codec including an encoding unit to automatically encode kernel data during a write operation to the memory device, wherein to automatically encode the kernel data, the DMA controller is to: analyze the kernel data to be encoded; determine that a high frequency value of the kernel data is mapped to a zero value; generate a mapping of each instance of the high frequency value in the kernel data; encode the kernel data into encoded kernel data, the encoded kernel data including the mapping of each instance of the high frequency value; and write the encoded kernel data and metadata associated with the encoded kernel data to memory.

18. The data processing system of claim 17, wherein, The DMA controller is to write the feature map data to the memory device via the encoding unit of the hardware codec in an encoding format including one or more encoding modes selected from a set of multiple encoding modes, the set of multiple encoding modes including encoding modes based on two or more of unique absolute value encoding, arithmetic progression encoding, significance map encoding, unique value coordinate encoding, and mean value encoding.

19. The data processing system of claim 18, wherein, The DMA controller includes a hardware codec including a decoding unit to automatically decode output feature maps to be read from the memory device.

20. The data processing system of claim 19, the kernel data of the CNN stored on the non-volatile memory device as encoded kernel data, the encoded kernel data to be loaded to the memory device, and the DMA controller to automatically decode the encoded kernel data during a read of the encoded kernel data from the memory device.

21. The data processing system of claim 17, wherein, To automatically encode the kernel data, the DMA controller is to write the metadata to a different memory block than the encoded kernel data.

22. The data processing system of claim 17, wherein, The general purpose graphics processor compute block is to generate output feature map data of the CNN using the kernel data, and write the output feature map data to a memory buffer within the processor.

23. The data processing system of claim 22, wherein: The processor is to write the output feature map data to the memory device via the DMA controller in an encoding format including one or more encoding modes selected from a set of multiple encoding modes.

24. The data processing system of claim 23, wherein, The set of multiple encoding modes includes encoding modes based on two or more of unique absolute value encoding, arithmetic progression encoding, unique value coordinate encoding, significance map encoding, and high frequency value encoding.

25. The data processing system of claim 24, wherein, Each of the set of multiple encoding modes is to configure the encoding unit to encode a reduced bit representation of a bitstream, the reduced bit representation including a set of unique values of the bitstream and data indicating locations of the unique values within the bitstream.

26. The data processing system of claim 24, wherein, The set of multiple encoding modes additionally includes a mean value encoding mode to configure the encoding unit to encode a mean value and a set of residual values of a bitstream.

Images