Mapping storage addresses from a SIMT processor (single instruction multi-thread) to storage banks of bank storage during runtime in various ways

Abstract

A processor of one aspect comprises a bank memory. The bank memory has multiple memory banks. The processor also includes a single-instruction multi-threaded (SIMT) processor. The SIMT processor comprises several processor elements. The processor elements are used to perform memory access operations with memory addresses. The processor also includes a bank memory access circuitry that is coupled to the bank memory and to the SIMT processor. The bank memory access circuitry is designed to access data in the bank memory using each of the memory addresses. The bank memory access circuitry is reconfigurable at runtime to map the memory addresses to the memory banks in several different ways. Further processors, methods, systems, and non-volatile machine-readable storage media are also disclosed.

Description

BACKGROUND Background information

[0001] Graphics processing units (GPUs) and other SIMT processors (SIMT: Single Instruction, Multiple Thread) are commonly used for graphics processing as well as general-purpose computing. In GPUs and other SIMT processors, memory (e.g., shared memory) is often used by multiple processor elements to store data values. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The embodiments are best understood with reference to the following description and the accompanying drawings, which are used to illustrate them. The following applies to the drawings: Fig. Figure 1 is a block diagram of a processing system according to one embodiment. Fig. 2A is a block diagram of an embodiment of a processor comprising one or more processor cores, an integrated memory controller and an integrated graphics processor. Fig. Figure 2B is a block diagram of the hardware logic of a graphics processor core block according to some embodiments described herein. Fig. 2C shows a graphics processing unit (GPU) that contains dedicated sets of graphics processing resources arranged in multi-core groups. Fig. Figure 2D is a block diagram of a general-purpose graphics processing unit (GPGPU) that can be configured as a graphics processor and / or computational accelerator, according to the embodiments described herein. Fig. 3A is a block diagram of a graphics processing unit, which may be a discrete graphics processing unit or a graphics processing unit integrated into a variety of processing cores or other semiconductor devices, such as, but not limited to, storage devices or network interfaces. Fig. Figure 3B illustrates a graphics processor with a tiled architecture according to the embodiments described herein. Fig. Figure 3C shows a computing accelerator according to the embodiments described here. Fig. Figure 4 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments. Fig. Figure 5A illustrates a graphics core cluster according to one embodiment. Fig. Figure 5B illustrates a vector engine of a graphics kernel according to one embodiment. Fig. Figure 5C illustrates a matrix engine of a graphics kernel according to one embodiment. Fig. Figure 6 illustrates a tile of a multi-tile processor according to one embodiment. Fig. Figure 7 is a block diagram illustrating graphics processor instruction formats according to some embodiments. Fig. Figure 8 is a block diagram of another embodiment of a graphics processor. Fig. Figure 9A is a block diagram illustrating a graphics processor instruction format that can be used to program graphics processing pipelines according to some embodiments. Fig. Figure 9B is a block diagram illustrating a graphics processor instruction sequence according to one embodiment. Fig. Figure 10 illustrates an exemplary graphics software architecture for a data processing system according to some embodiments. Fig. Figure 11A is a block diagram illustrating an IP core development system that can be used to manufacture an integrated circuit for performing operations, according to one embodiment. Fig. Figure 11B illustrates a lateral cross-sectional view of an IC package arrangement according to some embodiments described herein. Fig. Figure 11C illustrates a package arrangement that includes multiple units of hardware logic chiplets connected to a substrate; Fig. Figure 11D illustrates a housing arrangement incorporating interchangeable chiplets, according to one embodiment. Fig. Figure 12 is a block diagram illustrating an exemplary integrated SoC (system on a chip) circuit that can be manufactured using one or more IP cores, according to one embodiment. Fig. Figure 13A illustrates an exemplary graphics processor of an integrated SoC circuit which can be manufactured using one or more IP cores, according to one embodiment. Fig. Figure 13B illustrates another exemplary graphics processor of an integrated SoC circuit which can be manufactured using one or more IP cores, according to one embodiment. Fig. Figure 14 is a block diagram of an embodiment of a processor. Fig. Figure 15 is a block diagram of a detailed exemplary embodiment of a bank memory access circuit arrangement that is reconfigurable during runtime to map memory addresses to memory banks in different ways. Fig. Figure 16 is a block flow diagram of an embodiment of a method for accessing data from a bank storage. Fig. Figure 17A is a block diagram illustrating three exemplary embodiments of address masks. Fig. Figure 17B is a block diagram illustrating an exemplary embodiment of how an address mask can be applied. Fig. Figure 18 is a block diagram of an embodiment of a processor that can be operated to perform an embodiment of a bank memory access reconfiguration instruction. Fig. Figure 19 is a block diagram of a first exemplary embodiment of a first bank memory access reconfiguration instruction and a second bank memory access reconfiguration instruction. Fig. Figure 20 is a block diagram of a second exemplary embodiment of a bank storage access reconfiguration instruction. Fig. Figure 21A is a block diagram of a third exemplary embodiment of a bank storage access reconfiguration instruction. Fig. Figure 21B is a block diagram of a fourth exemplary embodiment of a bank storage access reconfiguration instruction. Fig. Figure 22 represents a block diagram of a system that can implement arithmetic operations using a programmable logic circuit arrangement that may include digital signal processing or DSP blocks. Fig. Figure 23 presents an example of the integrated circuit device as a programmable logic device, such as a field-programmable gate array (FPGA). Fig. Figure 24 represents an exemplary embodiment of a soft GPU. Fig. Figure 25 represents an exemplary embodiment of a SP. Fig. Figure 26 presents an exemplary embodiment of a shared memory block for the SM and how it is connected with receiving inputs, outputs and signals. Fig. Figure 27 illustrates an embodiment of the PLC for the SM and how they are connected to receive inputs, outputs, and signals. Fig. Figure 28 illustrates an embodiment of the output part of the SM and how it is connected with receiving inputs, outputs and signals. Fig. Figure 29 illustrates another exemplary embodiment of a shared memory block and how it is associated with receiving inputs, outputs and signals. Fig. Figure 30 illustrates an exemplary embodiment of a sequencer and how it can be connected to receive inputs, outputs, and signals. DETAILED DESCRIPTION OF EXECUTION FORMS

[0003] This document reveals various methods for mapping memory addresses of graphics processing units (GPUs) or other SIMT processors to memory banks of shared memory during runtime. Numerous specific details are presented in the following description (e.g., specific GPU configurations, sequences of operations, instruction formats, etc.). However, implementations can be carried out without these specific details. In other cases, well-known circuits, structures, and techniques are not shown in detail to prevent obscuring the understanding of this description. System overview

[0004] Fig. Figure 1 is a block diagram of a processing system 100 according to one embodiment. The processing system 100 can be used in a single-processor desktop system, a multi-processor workstation system, or a server system with a large number of processors 102 or processor cores 107. In one embodiment, the processing system 100 is a processing platform integrated within an integrated system-on-chip (SoC) circuit for use in mobile, handheld, or embedded devices, such as within Internet of Things (IoT) devices with wired or wireless connectivity to a local or wide area network.

[0005] In one embodiment, the processing system 100 can comprise, be coupled to, or be integrated into a server-based gaming platform, a gaming console (including a gaming and media console), a mobile gaming console, a handheld gaming console, or an online gaming console. In some embodiments, the processing system 100 is part of a mobile phone, a smartphone, a tablet computing device, or a mobile, internet-connected device, such as a laptop with limited internal storage capacity.The Processing System 100 may also include, be coupled to, or be integrated with: a wearable device, such as a smartwatch wearable device; smart glasses or smart clothing enhanced with augmented reality (AR) or virtual reality (VR) features to provide visual, auditory, or tactile outputs to complement visual, auditory, or tactile real-world experiences, or otherwise to provide text, audio, graphics, video, holographic images, or video or tactile feedback; another augmented reality (AR) device; or another virtual reality (VR) device. In some embodiments, the Processing System 100 includes or is part of a television or set-top box device.In one embodiment, the processing system 100 can include, be coupled to, or be integrated into a self-driving vehicle such as a bus, a semi-trailer truck, a car, a motor vehicle, an electric bicycle, an aircraft, or a glider (or any combination thereof). The self-driving vehicle can use the processing system 100 to process the environment detected around the vehicle.

[0006] In some embodiments, the one or more processors 102 each contain one or more processor cores 107 for processing instructions that, when executed, perform operations for system or application software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a specific instruction set 109. In some embodiments, the instruction set 109 can support complex instruction set (CISC), reduced instruction set (RISC), or very long instruction word (VLIW) computations. One or more processor cores 107 can process a different instruction set 109, which may contain instructions to support the emulation of other instruction sets. The processor core 107 may also include other processing devices, such as a digital signal processor (DSP).

[0007] In some embodiments, the processor 102 includes a cache 104. Depending on the architecture, the processor 102 may have a single internal cache or multiple levels of internal caches. In some embodiments, the cache is shared by different components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a Level 3 (L3) cache or Last Level Cache (LLC)) (not shown), which can be shared by the processor cores 107 using known cache coherence techniques. A register file 106 may also be included in the processor 102 and may contain various types of registers for storing different data types (e.g., integer registers, floating-point registers, status registers, and an instruction pointer register).Some registers may be general-purpose registers, while other registers may be specific to the design of the processor 102.

[0008] In some embodiments, one or more processors 102 are coupled to one or more interface buses 110 to transmit communication signals such as address, data, or control signals between the processor 102 and other components in the processing system 100. In one embodiment, the interface bus 110 can be a processor bus, such as a version of the DMI (Direct Media Interface) bus. However, processor buses are not limited to the DMI bus but can also include one or more PCI (Peripheral Component Interconnect) buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the one or more processors 102 include a memory controller 116 and a platform control hub 130.The storage controller 116 enables communication between a storage device and other components of the processing system 100, while the platform controller hub (PCH: Platform Controller Hub) 130 provides connections to I / O devices via a local I / O bus.

[0009] The memory device 120 can be a dynamic random-access memory (DRAM) device, a static random-access memory (SRAM) device, a flash memory device, a phase-change memory device, or any other memory device that has suitable performance to serve as process memory. In one embodiment, the memory device 120 can operate as system memory for the processing system 100 to store data 122 and instructions 121 that are used when the one or more processors 102 execute an application or process. The memory controller 116 is also coupled with an optional external graphics processor 118, which can communicate with the one or more graphics processors 108 in the processors 102 to perform graphics and media operations.In some embodiments, graphics, media, or computational operations can be supported by an accelerator 112, which is a coprocessor that can be configured to perform a specialized set of graphics, media, or computational operations. In one embodiment, for example, the accelerator 112 is a matrix multiplication accelerator used to optimize machine learning or computational operations. In another embodiment, the accelerator 112 is a ray-tracing accelerator that can be used to perform ray-tracing operations in coordination with the graphics processor 108. In one embodiment, an external accelerator 119 can be used instead of, or in conjunction with, the accelerator 112.

[0010] In some embodiments, a display device 111 can be connected to the processor(s) 102. The display device 111 can be one or more internal displays, such as in a mobile electronic device or a laptop device, or external displays connected via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 111 can be a head-mounted display (HMD), such as a stereoscopic display for use in virtual reality (VR) or augmented reality (AR) applications.

[0011] In some embodiments, the platform control hub 130 enables peripheral devices to connect to the storage device 120 and processor 102 via a high-speed I / O bus. The I / O peripheral devices include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, touch sensors 125, and a data storage device 124 (e.g., non-volatile memory, volatile memory, hard disk memory, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCI Express). The touch sensors 125 can include touchscreen sensors, pressure sensors, or fingerprint sensors.The wireless transceiver 126 can be a WiFi transceiver, a Bluetooth transceiver, or a cellular network transceiver, such as a 3G, 4G, 5G, or LTE (Long-Term Evolution) transceiver. The firmware interface 128 enables communication with the system firmware and can, for example, be a Unified Expandable Firmware Interface (UEFI). The network controller 134 can enable a network connection to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to the interface bus 110. The audio controller 146 is, in one embodiment, a multi-channel high-definition audio controller. In one embodiment, the processing system 100 includes an optional legacy I / O controller 140 to couple legacy devices (e.g., Personal System 2 (PS / 2)) to the system.The platform control hub 130 can also be connected to one or more USB (Universal Serial Bus) controllers 142 to establish a connection with input devices such as keyboard and mouse combinations 143, a camera 144 or other USB input devices.

[0012] It is understood that the processing system 100 shown is exemplary and not limiting, since other types of data processing systems configured differently can also be used. For example, an instance of the memory controller 116 and the platform control hub 130 can be integrated into a discrete external graphics processor, such as the external graphics processor 118. In one embodiment, the platform control hub 130 and / or the memory controller 116 can be external to the one or more processors 102 and be located in a system chipset that communicates with the one or more processors 102.

[0013] For example, printed circuit boards (“sleds”) can be used to house components such as CPUs, memory, and other components, and these are designed for higher thermal performance. In some examples, processing components, such as the processors, are located on top of a sled, while nearby memory components, such as DIMMs, are located on the bottom. As a result of the improved airflow provided by this design, the components can operate at higher frequencies and power levels than in typical systems, thus increasing performance. Furthermore, the sleds are designed to be blindly connected to power and data communication cables in a rack, improving their ability to be quickly removed, upgraded, reinstalled, and / or replaced.Similarly, individual components located on the sleds, such as processors, accelerators, memory, and data storage drives, are configured for easy upgradeability due to their increased spacing. In the exemplary embodiment, the components also include hardware attestation features to verify their authenticity.

[0014] A data center can use a single network architecture (“fabric”) that supports several other network architectures, including Ethernet and Omni-Path. The sleds can be connected to switches via optical fibers, which provide higher bandwidth and lower latency than typical twisted-pair cabling (e.g., Category 5, Category 5e, Category 6, etc.). Because of these high-bandwidth, low-latency interconnects and the network architecture, the data center can pool resources such as storage, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and data storage drives that are physically separate, and make them available to compute resources (e.g., processors) as needed. This allows the compute resources to access the pooled resources as if they were local.

[0015] A power supply or power source can provide voltage and / or current to the processing system 100 or any component or system described herein. In one example, the power supply includes an AC-to-DC adapter for connection to an electrical outlet. This AC current can be derived from renewable energy sources (e.g., solar power). In another example, the power source includes a DC power source, such as an external AC-to-DC converter. In yet another example, the power source or power supply includes wireless charging hardware for charging by proximity to a charging pad. In yet another example, the power source can include an internal battery, an AC power supply, a motion-based power supply, a solar power supply, or a fuel cell power source.

[0016] Fig. 2A-2D represent computing systems and graphics processors provided by the embodiments described herein. The elements of Fig. 2A-2D, which have the same reference numbers (or names) as the elements of the other figures, may operate or function in a similar manner as described elsewhere, but are not limited to doing so.

[0017] Fig. Figure 2A is a block diagram of an embodiment of a processor 200 comprising one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The processor 200 may include additional cores up to and including the additional core 202N, represented by the dashed boxes. Each of the processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. The internal cache units 204A-204N and the shared cache units 206 constitute a cache memory hierarchy within the processor 200.The cache memory hierarchy can include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other cache levels, if the highest cache level prior to external memory is classified as LLC. In some embodiments, cache coherence logic maintains coherence between the different cache units 206 and 204A-204N.

[0018] In some embodiments, the processor 200 may also include a set of one or more bus control units 216 and a system agent core 210. The one or more bus control units 216 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. The system agent core 210 provides management functions for the various processor components. In some embodiments, the system agent core 210 includes one or more integrated memory devices 214 to manage access to various external memory devices (not shown).

[0019] In some embodiments, one or more of the processor cores 202A-202N support simultaneous multithreading. In such an embodiment, the system agent core 210 includes components for coordinating and operating the cores 202A-202N during multithreading. The system agent core 210 may also include a power control unit (PCU) containing logic and components for regulating the power state of the processor cores 202A-202N and the graphics processor 208.

[0020] In some embodiments, the processor 200 additionally includes a graphics processor 208 for performing graphics processing operations. In some embodiments, the graphics processor 208 is coupled to the set of shared cache units 206 and the system agent core 210, which includes one or more integrated memory controllers 214. In some embodiments, the system agent core 210 also includes a display controller 211 for controlling the output of the graphics processor to one or more coupled displays. In some embodiments, the display controller 211 may be a separate module coupled to the graphics processor via at least one intermediate connection, or it may be integrated into the graphics processor 208.

[0021] In some embodiments, a ring-based interconnect 212 is used to couple the internal components of the processor 200. However, another interconnect unit can also be used, such as a point-to-point interconnect, a switched interconnect, a mesh interconnect, or other techniques, including those known in the art. In some embodiments, the graphics processor 208 is coupled to the ring-based interconnect 212 via an I / O link 213.

[0022] The exemplary I / O connection 213 represents at least one of several types of I / O intermediate connections, including an in-chassis I / O intermediate connection, which facilitates communication between various processor components and an embedded high-performance memory module 218, such as an eDRAM module or an HBM (high-bandwidth memory) module. In some embodiments, each of the processor cores 202A-202N and the graphics processor 208 can use the embedded memory module 218 as a shared last-level cache.

[0023] In some embodiments, the 202A-202N processor cores are homogeneous cores executing the same instruction set architecture. In another embodiment, the 202A-202N processor cores are heterogeneous with respect to the instruction set architecture (ISA), wherein one or more of the 202A-202N processor cores execute a first instruction set, while at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the 202A-202N processor cores are heterogeneous with respect to the microarchitecture, wherein one or more high-power cores are coupled with one or more low-power cores. In one embodiment, the 202A-202N processor cores are heterogeneous with respect to computational capability.Furthermore, the Processor 200 can be implemented on one or more chips or as an integrated circuit (SoC) that includes the illustrated components in addition to other components.

[0024] Fig. 2B is a block diagram of hardware logic of a graphics processor core block 219 according to some embodiments described herein. In some embodiments, elements from Fig. 2B with the same reference numbers (or names) as the elements of any other figure presented here operate or function in a similar manner to those described elsewhere herein. The graphics processor core block 219 is an example of a graphics processor partition. The graphics processor core block 219 can be incorporated into the integrated graphics processor 208 of Fig. 2A or be integrated into a discrete graphics processor, parallel processor, and / or computational accelerator. A graphics processor as described herein may include multiple graphics core blocks based on target performance and capability envelopes. Each graphics processor core block 219 may include a function block 230 coupled to multiple graphics cores 221A to 221F, which include modular blocks consisting of fixed-function logic and general-purpose programmable logic. The graphics processor core block 219 also includes shared / cache memory 236, accessible to all graphics cores 221A-221F, rasterizer logic 237, and additional fixed-function logic 238.

[0025] In some embodiments, the functional block 230 includes a geometry / fixed-function pipeline 231 that can be shared by all graphics cores in the graphics processor core block 219. In various embodiments, the geometry / fixed-function pipeline 231 includes a 3D geometry pipeline, a video front-end unit, a thread generator and a global thread dispatcher, and a unified return buffer manager that manages unified return buffers. In one embodiment, the functional block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. The graphics SoC interface 232 provides an interface between the graphics processor core block 219 and other core blocks in a graphics processor or compute accelerator SoC.The graphics microcontroller 233 is a programmable subprocessor configurable to manage various functions of the graphics processor core block 219, including thread dispatch, scheduling, and preemption. The media pipeline 234 includes logic to facilitate the decoding, encoding, preprocessing, and / or post-processing of multimedia data, including image and video data. The media pipeline 234 implements media operations via requests to the computational or sampling logic in the graphics cores 221-221F. One or more pixel backends 235 can also be integrated into the function block 230. The pixel backends 235 include a cache for storing pixel color values ​​and can perform blending operations and lossless color compression of rendered pixel data.

[0026] In one embodiment, the graphics SoC interface 232 enables the graphics processing unit (GPU) core block 219 to communicate with general-purpose application processor cores (e.g., CPUs) and / or other components within a SoC or a system host CPU that is coupled to the SoC via a peripheral interface. The graphics SoC interface 232 also enables communication with off-chip memory hierarchy elements, such as shared last-level cache, system RAM, and / or embedded on-chip or on-package DRAM. The SoC interface 232 can also enable communication with fixed functional devices within the SoC, such as camera imaging pipelines, and allows the use of and / or implements global memory structures that can be shared between the GPU core block 219 and CPUs within the SoC.The graphics SoC interface 232 can also implement power management controls for the graphics processor core block 219 and provide an interface between a clock range of the graphics processor core block 219 and other clock ranges within the SoC. In one embodiment, the graphics SoC interface 232 allows the receipt of instruction buffers from an instruction streamer and a global thread dispatcher configured to deliver instructions and directions to each of one or more graphics cores within a graphics processor. The instructions and directions can be sent to the media pipeline 234 when media operations are to be performed, and to the geometry and fixed function pipeline 231 when graphics processing operations are to be performed.When computational operations need to be performed, the computational dispatch logic can allocate the instructions to the graphics cores 221A to 221F, bypassing the geometry and media pipelines.

[0027] The graphics microcontroller 233 can be configured to perform various scheduling and management tasks for the graphics processor core block 219. In one embodiment, the graphics microcontroller 233 can perform graphics and / or compute load scheduling on the various vector engines 222A-222F, 224A-224F and matrix engines 223A-223F, 225A-225F within the graphics cores 221A-221F. In this scheduling model, the host software, running on a CPU core of a SoC with the graphics processor core block 219, can submit workloads to one of several graphics processor doorbells, triggering a scheduling operation on the corresponding graphics engine.Scheduling operations include determining which workload to execute next, submitting a workload to an instruction streamer, anticipating existing workloads running on an engine, monitoring the progress of a workload, and notifying the host software when a workload is complete. In one embodiment, the graphics microcontroller 233 can also enable low-power or idle states for the graphics processor core block 219 and allow the graphics processor core block 219 to save and restore registers within the graphics processor core block 219 across low-power state transitions independently of the operating system and / or graphics driver software on the system.

[0028] The graphics processor core block 219 can have more or fewer than the graphics cores 221A-221F shown, up to N modular graphics cores. For each set of N graphics cores, the graphics processor core block 219 can also include a shared / cache memory 236, which can be configured as shared memory or cache memory, raster logic 237, and additional fixed-function logic 238 for accelerating various graphics and computational operations.

[0029] Within each 221A-221F graphics core is a set of execution resources that can be used to perform graphics, media, and computational operations in response to requests from the graphics pipeline, media pipeline, or shader programs. The 221A-221F graphics cores include several vector engines (222A-222F, 224A-224F), matrix acceleration units (223A-223F, 225A-225D), cache / shared local memory (SLM), a sampler (226A-226F), and a ray tracing unit (227A-227F).

[0030] The 222A-222F and 224A-224F vector engines are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations in service of graphics, media, or computational applications, including graphics, media, or computational / GPGPU programs. The 222A-222F and 224A-224F vector engines can operate with variable vector width in SIMD, SIMT, or SIMT+SIMD execution modes. The 223A-223F and 225A-225D matrix acceleration units incorporate matrix-to-matrix and matrix-to-vector acceleration logic that improves performance in matrix operations, particularly low-precision and mixed-precision matrix operations (e.g., INT8, FP16, BF16) used in machine learning.In one embodiment, each of the matrix acceleration units 223A-223F, 225A-225D includes one or more systolic arrays of processing elements that can simultaneously perform matrix multiplication or scalar product operations on matrix elements.

[0031] The 226A-226F sampler can read media or texture data into memory and can sample data differently based on a configured sampler state and the texture / media format being read. Threads running on the 222A-222F and 224A-224F vector engines or the 223A-223F and 225A-225D matrix acceleration units can utilize the 228A-228F cache / SLM within each execution core. The 228A-228F cache / SLM can be configured as cache memory or as a shared memory pool that is local to each of the respective 221A-221F graphics cores. The ray tracing units 227A-227F in the graphics cores 221A-221F contain ray traversal / intersection circuit arrangements using envelope hierarchies (BVHs) and the identification of intersections between rays and primitives contained within the BVH volumes.In one embodiment, the beam tracing units 227A-227F include circuits for performing depth checks and culling (e.g., using a depth buffer or similar arrangement). In another implementation, the beam tracing units 227A-227F perform traversal and overlap operations together with image denoising, at least some of which can be performed with an associated matrix acceleration unit 223A-223F, 225A-225D.

[0032] Fig. Figure 2C illustrates a graphics processing unit (GPU) 239 containing dedicated sets of graphics processing resources arranged in multi-core groups 240A-240N. Details of multi-core group 240A are shown. Multi-core groups 240B-240N may be equipped with the same or similar sets of graphics processing resources.

[0033] As illustrated, a multi-core group 240A can contain a set of graphics cores 243, a set of tensor cores 244, and a set of ray-tracing cores 245. A scheduler / dispatcher 241 schedules and dispatches the graphics threads for execution on the various cores 243, 244, and 245. In one embodiment, the tensor cores 244 are sparse tensor cores with hardware that allows multiplication operations to be bypassed with a zero input. The graphics cores 243 of the GPU 239 from Fig. The 2C differ in their hierarchical abstraction level relative to the 221A-221F graphics cores. Fig. 2B, which are analogous to the multi-core groups 240A-240N from Fig. 2C are. The graphics cores 243, tensor cores 244 and ray tracing cores 245 from Fig. 2C are analogous to the vector engines 222A-222F, 224A-224F, matrix engines 223A-223F, 225A-225F or the beam tracing units 227A-227F from Fig. 2B.

[0034] A set of register files 242 can store operand values ​​used by the cores 243, 244, and 245 when executing the graphics threads. These can include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as combined sets of vector registers.

[0035] One or more combined Level 1 (L1) caches and shared memory units 247 store graphics data, such as texture data, vertex data, pixel data, ray data, envelope data, etc., locally within each multicore group 240A. One or more texture units 247 can also be used to perform texturing operations such as texture mapping and sampling. A Level 2 (L2) cache 253, shared by all or a subset of the multicore groups 240A–240N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 253 can be shared across multiple multicore groups 240A–240N. One or more memory control units 248 couple the GPU 239 to a memory 249, which may be system memory (e.g. DRAM) and / or dedicated graphics memory (e.g. GDDR6 memory).

[0036] An input / output circuit arrangement 250 connects the GPU 239 to one or more I / O devices 252, such as digital signal processors (DSPs), network control units, or user input devices. An intermediate connection on the chip can be used to couple the I / O devices 252 to the GPU 239 and the memory 249. One or more I / O memory management units (IOMMUs) 251 of the I / O circuit arrangement 250 directly couple the I / O devices 252 to the memory 249. In one embodiment, the IOMMU 251 manages multiple sets of page tables to map virtual addresses to physical addresses in the memory 249. In this embodiment, the I / O devices 252, the CPU(s) 246, and the GPU 239 can share the same virtual address space.

[0037] In one implementation, the IOMMU 251 supports virtualization. In this case, it can manage a first set of page tables to map virtual guest / graphics addresses to physical guest / graphics addresses, and a second set of page tables to map the physical guest / graphics addresses to physical system / host addresses (e.g., in memory 249). The base addresses of both the first and second sets of page tables can be stored in control registers and swapped out on context switching (e.g., so that the new context has access to the relevant set of page tables). Even if this is in Fig. Not illustrated in 2C, each of the cores 243, 244, 245 and / or each of the multi-core groups 240A-240N can include translation lookaside buffers (TLBs) to cache guest virtual-to-guest physical translations, guest physical-to-host physical translations, and guest virtual-to-host physical translations.

[0038] In one embodiment, the CPUs 246, the GPU 239, and the I / O devices 252 are integrated on a single semiconductor chip and / or chip package. The memory 249 may be integrated on the same chip or coupled to the memory controllers 248 via an off-chip interface. In one implementation, the memory 249 comprises GDDR6 memory that shares the same virtual address space as other system-level physical memory, although the underlying principles of the embodiments described herein are not limited to this particular implementation.

[0039] In one embodiment, the TensorKerles 244 comprise several functional units specifically designed to perform matrix operations, which constitute the fundamental computational operation for deep learning operations. For example, concurrent matrix multiplication operations can be used for training and inferring neural networks. The TensorKerles 244 can perform matrix processing using a variety of operand precisions, including single-precision floating-point (e.g., 32 bits), half-precision floating-point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half-bytes (4 bits). In one embodiment, a neural network implementation extracts features from each rendered scene, potentially combining details from multiple frames (individual images) to create a high-quality final image.

[0040] In deep learning implementations, parallel matrix multiplication work can be scheduled for execution on the TensorKerles 244. In particular, training neural networks involves a significant number of matrix-scalar product operations. To process an inside-product formulation of an N × N × N matrix multiplication, the TensorKerles 244 can contain at least N scalar product processing elements. Before the matrix multiplication begins, an entire matrix is ​​loaded into tile registers, and at least one column of a second matrix is ​​loaded into each N cycle. In each cycle, N scalar products are processed.

[0041] Matrix elements can be stored with varying degrees of precision depending on the implementation, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4). Different precision modes can be specified for the Tensor Core 244 to ensure that the most efficient precision is used for different workloads (e.g., inference workloads that can tolerate quantization in bytes and half-bytes).

[0042] In one embodiment, the beam-tracking cores 245 accelerate beam-tracking operations for both real-time and non-real-time beam-tracking implementations. Specifically, the beam-tracking cores 245 include a beam traversal / intersection circuitry for performing beam traversal using bounding volume hierarchies (BVHs) and for identifying intersections between beams and primitives enclosed within the BVH volumes. The beam-tracking cores 245 may also include circuitry for performing depth checks and culling (e.g., using a Z-buffer or similar arrangement).In one implementation, the ray-tracking cores 245 perform traversal and intersection operations together with the image denoising techniques described herein, at least one part of which can be executed on the tensor cores 244. In one embodiment, for example, the tensor cores 244 implement a deep-learning neural network to perform denoising of frames generated by the ray-tracking cores 245. However, the one or more CPUs 246, graphics cores 243, and / or ray-tracking cores 245 can also implement all or one part of the denoising and / or deep-learning algorithms.

[0043] Furthermore, as described above, a distributed approach to denoising can be employed, in which the GPU 239 is located in a data processing device that is coupled to other data processing devices via a network or high-speed intermediary link. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0044] In one embodiment, the beam-tracking cores 245 process all BVH traversal and beam-primitive intersections, thus preventing the graphics cores 243 from being overloaded with thousands of instructions per beam. In another embodiment, each beam-tracking core 245 has a first set of specialized circuit arrangements for performing bounding frame tests (e.g., for traversal operations) and a second set of specialized circuit arrangements for performing beam-triangle intersection tests (e.g., for cutting beams that have been traversed). Thus, in one embodiment, the multi-core group 240A can simply send out a beam probe, and the beam-tracking cores 245 independently perform beam traversal and intersection, returning hit data (e.g., one hit, no hit, multiple hits, etc.) to the thread context.The other cores 243, 244 are freed to perform other graphics or computational work, while the ray-tracing cores 245 perform the traversal and intersection operations.

[0045] In one embodiment, each ray-tracing core 245 includes a traversal unit for performing BVH checks and an intersection unit that performs ray-primitive intersection checks. The intersection unit generates a response of "hit," "no hit," or "multiple hits," which it provides to the appropriate thread. During the traversal and intersection operations, the execution resources of the other cores (e.g., graphics cores 243 and tensor cores 244) are released to perform other forms of graphics work.

[0046] In a particular embodiment described below, a hybrid rasterization / ray tracing approach is used, in which work is distributed between the graphics cores 243 and the ray tracing cores 245.

[0047] In one embodiment, the ray-tracing cores 245 (and / or other cores 243, 244) include hardware support for a ray-tracing instruction set, such as Microsoft's DirectX Ray Tracing (DXR), which includes a DispatchRays instruction, as well as ray generation, nearest-hit, arbitrary-hit, and miss-hit shaders, which allow the assignment of unique sets of shaders and textures for each object. Another ray-tracing platform that can be supported by the ray-tracing cores 245, the graphics cores 243, and the tensor cores 244 is Vulkan 1.1.85. However, it should be noted that the principles underlying the embodiments described herein are not limited to any particular ray-tracing ISA.

[0048] In general, the various cores 245, 244, and 243 can support a ray-tracing instruction set that includes instructions / functions for ray generation, nearest hit, arbitrary hit, ray-primitive intersection, primitive-wise and hierarchical bounding frame construction, misses, visits, and exceptions. More specifically, one embodiment includes ray-tracing instructions to perform the following functions:

[0049] Beam generation - Beam generation instructions can be executed for each pixel, sample, or other user-defined task.

[0050] Closest hit - A closest hit instruction can be executed to find the closest intersection point of a ray with primitives within a scene.

[0051] Any Hit - An Any Hit instruction identifies multiple intersections between a ray and primitives within a scene to potentially identify a new nearest intersection point.

[0052] Intersection point - An intersection point instruction performs a beam-primitive intersection point test and outputs a result.

[0053] Primitive-wise envelope construction - This instruction builds an envelope around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0054] Miss - Indicates that a ray misses the entire geometry within a scene or a specific region of a scene.

[0055] Visit - Indicates the subordinate volumes that a beam traverses.

[0056] Exceptions - Includes various types of exception handling programs (e.g., for different error conditions).

[0057] In one embodiment, the ray-tracking kernels 245 can be designed to accelerate general-purpose computational operations, which can be accelerated using computational techniques analogous to beam intersection tests. A computational framework can be provided that allows shading programs to be compiled into low-level instructions and / or primitives that perform general-purpose computational operations on the ray-tracking kernels. Exemplary computational problems that can benefit from computational operations performed on the ray-tracking kernels 245 include calculations involving ray, wave, light-beam, or particle propagation within a coordinate space. Interactions associated with this propagation can be computed relative to a geometry or mesh within the coordinate space.For example, calculations related to the propagation of electromagnetic signals in an environment can be accelerated by using instructions or primitives executed via ray-tracing kernels. Diffraction and reflection of signals by objects in the environment can be calculated as direct ray-tracing analogies.

[0058] The ray-tracing kernels 245 can also be used for calculations that are not directly analogous to ray tracing. For example, mesh projection, mesh refinement, and volume sampling calculations can be accelerated using the ray-tracing kernels 245. Generic coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, the set of points near a given point can be discovered by defining a bounding box in the coordinate space around the point. BVH and ray probe logic within the ray-tracing kernels 245 can then be used to determine the set of point intersections within the bounding box. The intersections form the origin point and the nearest neighbors to that origin point.Computations performed using the ray-tracing kernels 245 can be performed in parallel with computations performed on the graphics kernels 243 and the tensor kernels 244. A shader compiler can be designed to compile a computational shader or other general-purpose graphics processing program into low-level primitives that can be parallelized across the graphics kernels 243, tensor kernels 244, and ray-tracing kernels 245.

[0059] Fig. Figure 2D is a block diagram of a general-purpose graphics processing unit (GPGPU) 270, which can be configured as a graphics processor and / or computational accelerator, according to the embodiments described herein. The GPGPU 270 can be connected to host processors (e.g., one or more CPUs 246) and memory 271, 272 via one or more system and / or memory buses. In one embodiment, memory 271 is the system memory, which can be shared with the one or more CPU(s) 246, while memory 272 is the device memory, which is dedicated to the GPGPU 270. In another embodiment, components within the GPGPU 270 and memory 272 can be mapped to memory addresses that are accessible to the one or more CPUs 246. Access to memory 271 and 272 can be enabled via a memory controller 268.In one embodiment, the memory controller 268 includes an internal direct memory access (DMA) controller 269, or may include logic for performing operations that would otherwise be performed by a DMA controller.

[0060] The GPGPU 270 includes several cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256, at least part of which can also be partitioned as cache memory. The GPGPU 270 also includes several processing units 260A-260N, which provide a hierarchical abstraction layer analogous to the graphics cores 221A-221F of Fig. 2B and the multi-core groups 240A-240N from Fig. 2C. Each compute unit 260A-260N includes a set of vector registers 261, scalar registers 262, vector logic units 263, and scalar logic units 264. The compute units 260A-260N may also include a local shared memory 265 and a program counter 266. The compute units 260A-260N may be coupled to a constant cache 267, which can be used to store constant data, i.e., data that does not change during the execution of the kernel or shader program on the GPGPU 270. In one embodiment, the constant cache 267 is a scalar data cache, and cached data can be directly accessed in the scalar registers 262.

[0061] During operation, one or more CPUs 246 can write instructions to registers or memory of the GPGPU 270, which is mapped to an accessible address space. The instruction processors 257 can read the instructions from registers or memory and determine how these instructions are processed within the GPGPU 270. A thread dispatcher 258 can then be used to distribute threads to the arithmetic units 260A-260N to execute these instructions. Each arithmetic unit 260A-260N can execute threads independently of the other arithmetic units. Additionally, each arithmetic unit 260A-260N can be independently configured for conditional computation and can conditionally output the results of the computation to memory. The instruction processors 257 can interrupt the one or more CPUs 246 when the transmitted instructions are complete.

[0062] Fig. Figures 3A-3C illustrate block diagrams of additional graphics processor and compute accelerator architectures provided by embodiments described herein. The elements of Fig. 3A-3C, which have the same reference numbers (or names) as the elements of the other figures, may operate or function in a similar manner as described elsewhere, but are not limited to doing so.

[0063] Fig. Figure 3A is a block diagram of a graphics processing unit (GPU) 300, which may be a discrete graphics processing unit or a GPU integrated into a variety of processing cores or other semiconductor devices, such as, but not limited to, storage devices or network interfaces. In some embodiments, the GPU communicates with registers on the GPU and with instructions stored in the processor's memory via a memory-mapped I / O interface. In some embodiments, the GPU 300 has a memory interface 314 for memory access. The memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0064] In some embodiments, the graphics processor 300 also includes a display controller 302 for driving display output data to a display device 318. The display controller 302 includes hardware for one or more overlay layers for displaying and assembling multiple layers of video or user interface elements. The display device 318 can be an internal or external display device. In one embodiment, the display device 318 is a head-mounted display device, for example, a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding, decoding, or transcoding media to, from, or between one or more media coding formats, including, but not limited to, Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9, ​​and Society of Motion Picture & Television Engineers (SMPTE) 421M / VC-1 and Joint Photographic Experts Group (JPEG) formats such as JPEG and Motion JPEG (MJPEG) formats.

[0065] In some embodiments, the graphics processor 300 includes a block image transfer (BLIT) engine to perform two-dimensional (2D) raster operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed by using one or more components of the graphics processing engine (GPE) 310. In some embodiments, the GPE 310 is a computation engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0066] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that act on 3D primitive shapes (e.g., rectangle, triangle, etc.). The 3D pipeline 312 comprises programmable and fixed-function elements that perform various tasks within the element and / or generate execution threads to a 3D / media subsystem 315. While the 3D pipeline 312 can be used to perform media operations, one embodiment of GPE 310 also includes a media pipeline 316, which is specifically used to perform media operations such as video post-processing and image enhancement.

[0067] In some embodiments, the media pipeline 316 includes fixed-function or programmable logic units to perform one or more specialized media operations, such as video decoder acceleration, video deinterlacing, and video encoder acceleration, instead of or in favor of the video codec engine 306. In some embodiments, the media pipeline 316 additionally includes a thread spawning unit to spawn threads for execution in the 3D / Media subsystem 315. The spawned threads perform computations for the media operations on one or more graphics cores contained in the 3D / Media subsystem 315.

[0068] In some embodiments, the 3D / Media Subsystem 315 includes logic for executing threads generated by the 3D Pipeline 312 and the Media Pipeline 316. In one embodiment, the pipelines send thread execution requests to the 3D / Media Subsystem 315, which includes thread dispatch logic for arbitrating and dispatching the various requests to available thread execution resources. The execution resources include an array of graphics cores for processing the 3D and media threads. In some embodiments, the 3D / Media Subsystem 315 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and storing output data.

[0069] Fig. Figure 3B illustrates a graphics processor 320 with a tiled architecture according to embodiments described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 with multiple instances of the graphics processing engine 310. Fig. 3A within a graphics engine tile 310A-310D. Each graphics engine tile 310A-310D can be connected via a set of tile interleaves 323A-323F. Each graphics engine tile 310A-310D can also be connected via memory interleaves 325A-325D to a memory module or memory device 326A-326D. The memory devices 326A-326D can use any graphics memory technology. For example, the memory devices 326A-326D can be double-speed graphics memory (GDDR). In one embodiment, the memory devices 326A-326D are HBM modules that can be located on a die with their respective graphics engine tile 310A-310D. In one embodiment, the storage devices 326A-326D are stacked storage devices that can be stacked on their respective graphics module tiles 310A-310D.In one embodiment, each graphics engine tile 310A-310D and the associated memory 326A-326D are located on separate chiplets bonded to a base die or base substrate, as detailed in . Fig. 11B-11D is described.

[0070] The 320 graphics processor can be configured with a non-uniform memory access (NUMA) system in which the memory devices 326A-326D are coupled to associated graphics engine tiles 310A-310D. A given memory device can be accessed by different graphics engine tiles than the tile to which it is directly connected. However, the access latency to the memory devices 326A-326D can be lowest when accessing a local tile. In one embodiment, a cache-coherent NUMA (ccNUMA) system is enabled, which uses the tile intermediaries 323A-323F to enable communication between cache controllers within the graphics engine tiles 310A-310D, so that a consistent memory image is maintained when more than one cache stores the same memory location.

[0071] The graphics processing engine cluster 322 can be connected to an on-chip or on-package fabric intermediary 324. In one embodiment, the fabric intermediary 324 includes a network processor, a network-on-chip (NoC), or other switching processor to enable the fabric intermediary 324 to act as a packet-switched fabric intermediary, mediating data packets between components of the graphics processor 320. The fabric intermediary 324 can enable communication between the graphics engine tiles 310A-310D and components such as the video codec engine 306 and one or more copy engines 304. The copy engines 304 can be used to move data to, from, and between the storage devices 326A-326D and to, from, and between memory located outside the graphics processor 320 (e.g., system memory).The fabric intermediate link 324 can also be coupled with one or more of the tile intermediate links 323A-323F to enable or improve the interconnection between the graphics engine tiles 310A-310D. The fabric intermediate link 324 can also be configured to connect multiple instances of the graphics processor 320 (e.g., via the host interface 328), thereby enabling tile-to-tile communication between graphics engine tiles 310A-310D of multiple GPUs. In one embodiment, the graphics engine tiles 310A-310D of multiple GPUs can be presented to a host system as a single logical device.

[0072] The graphics processor 320 can optionally include a display controller 302 to enable communication with the display device 318. The graphics processor can also be configured as a graphics or computational accelerator. In the accelerator configuration, the display controller 302 and the display device 318 can be omitted.

[0073] The GPU 320 can be connected to a host system via a host interface 328. The host interface 328 can enable communication between the GPU 320, system memory, and / or other system components. The host interface 328 can be, for example, a PCI Express bus or another type of host system interface. The host interface 328 can be, for example, an NVLink or NVSwitch interface. The host interface 328 and the fabric intermediary 324 can work together to allow multiple instances of the GPU 320 to function as a single logical device. The cooperation between the host interface 328 and the fabric intermediary 324 can also allow the individual graphics engine tiles 310A-310D to be presented to the host system as distinct logical graphics devices.

[0074] Fig. 3C discloses a computing accelerator 330 according to the embodiments described herein. The computing accelerator 330 may exhibit architectural similarities to the graphics processor 320 from Fig. 3B and is optimized for computational acceleration. A computation engine cluster 332 can comprise a set of computation engine tiles 340A-340D, which include execution logic optimized for parallel or vector-based general-purpose computing operations. In some embodiments, the computation engine tiles 340A-340D do not include fixed-function graphics processing logic, although in one embodiment, one or more of the computation engine tiles 340A-340D may include logic for performing media acceleration. The computation engine tiles 340A-340D can be connected to the memory 326A-326D via memory links 325A-325D. The memory 326A-326D and the memory links 325A-325D may be of a similar technology to that in the graphics processor 320, or they may be different.The compute engine tiles 340A-340D can also be interconnected via a series of tile intermediaries 323A-323F and can be connected to and / or through a fabric intermediary 324. Inter-tile communication can be facilitated via the fabric intermediary 324. The fabric intermediary 324 can also facilitate communication between the compute engine tiles 340A-340D of multiple instances of the compute accelerator 330 (e.g., via the host interface 328). In one embodiment, the compute accelerator 330 includes a large L3 cache 336, which can be configured as a device-spanning cache. The compute accelerator 330 can also be accessed via a host interface 328 in a manner similar to the graphics processor 320. Fig. 3B must be connected to a host processor and memory.

[0075] The computing accelerator 330 can also include an integrated network interface 342. In one embodiment, the network interface 342 includes a network processor and control logic that enables the computing engine cluster 332 to communicate via a physical layer intermediary 344 without requiring data to traverse a host system memory. In another embodiment, one of the computing engine tiles 340A-340D is replaced by network processor logic, and data to be transmitted or received via the physical layer intermediary 344 can be transferred directly to or from the memory 326A-326D. Multiple instances of the computing accelerator 330 can be connected via the physical layer intermediary 344 to form a single logical device.Alternatively, the different computing engine tiles 340A-340D can also be represented as different computing accelerator devices accessible via the network. Graphics processing engine

[0076] Fig. Figure 4 is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the one described in Fig. 3A shown GPE 310 and can also be a graphics engine tile 310A-310D from Fig. 3B represents elements from Fig. 4. Elements that have the same reference numbers (or designations) as the elements of any other figure herein may be operated or function in any similar manner as described elsewhere herein, but are not limited to doing so. For example, 3D Pipeline 312 and Media Pipeline 316 of Fig. Figure 3A illustrates this. The media pipeline 316 is optional in some embodiments of the GPE 410 and may not be explicitly included in the GPE 410. For example, in at least one embodiment, a separate media and / or image processor is coupled with the GPE 410.

[0077] In some embodiments, the GPE 410 is coupled to or includes an instruction streamer 403, which feeds an instruction stream to the 3D pipeline 312 and / or the media pipelines 316. Alternatively or additionally, the instruction streamer 403 can be directly coupled to a unified return buffer 418. The unified return buffer 418 can be communicatively coupled to a graphics core cluster 414. In some embodiments, the instruction streamer 403 is coupled to a memory, which can be system memory, internal cache memory, and / or shared cache memory. In some embodiments, the command streamer 403 receives commands from memory and sends the commands to the 3D pipeline 312 and / or the media pipeline 316. The commands are directives that are retrieved from a ring buffer in which commands for the 3D pipeline 312 and the media pipeline 316 are stored.In one embodiment, the ring buffer can additionally include batch instruction buffers that store batches of multiple instructions. The instructions for the 3D pipeline 312 can also contain references to data stored in memory, such as vertex and geometry data for the 3D pipeline 312 and / or image data and memory objects for the media pipeline 316. The 3D pipeline 312 and the media pipeline 316 process the instructions and data by performing operations via logic within their respective pipelines or by sending one or more execution threads to a graphics core cluster 414. In one embodiment, the graphics core cluster 414 comprises one or more blocks of graphics cores (e.g., graphics core block 415A, graphics core block 415B), each block containing one or more graphics cores.Each graphics kernel includes a set of graphics execution resources, including general-purpose and graphics-specific execution logic for performing graphics and computational operations, as well as fixed-function texture processing and / or machine learning and artificial intelligence acceleration logic, such as matrix or AI acceleration logic.

[0078] In various embodiments, the 3D pipeline 312 can include fixed-function and programmable logic for processing one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, computational shaders, or other shader programs, by processing the instructions and distributing execution threads to the graphics core cluster 414. The graphics core cluster 414 provides a unified block of execution resources for use in processing these shader programs. Universal execution logic within the graphics core blocks 415A-415B of the graphics core cluster 414 supports various 3D API shader languages ​​and can execute multiple concurrent execution threads associated with multiple shaders.

[0079] In some embodiments, the graphics core cluster 414 includes execution logic for performing media functions, such as video and / or image processing. In one embodiment, the graphics cores include general-purpose logic that is programmable to perform parallel general-purpose computational operations in addition to graphics processing operations. The general-purpose logic can perform processing operations in parallel or in conjunction with general-purpose logic within the processor core(s) 107. Fig. 1 or of core 202A-202N as in Fig. Perform 2A.

[0080] Output data generated by threads running on the graphics core cluster 414 can be output to memory in a Unified Return Buffer (URB) 418. The URB 418 can store data for multiple threads. In some embodiments, the URB 418 can be used to send data between different threads running on the graphics core cluster 414. In some embodiments, the URB 418 can also be used for synchronization between threads on the graphics core array and the fixed function logic within the shared function logic 420.

[0081] In some embodiments, the graphics core cluster 414 is scalable, such that the cluster comprises a variable number of graphics cores, each with a variable number of graphics cores based on the target performance and performance level of GPE 410. In one embodiment, the execution resources are dynamically scalable, such as by enabling or disabling them as needed.

[0082] The graphics core cluster 414 is coupled to a shared function logic 420, which includes several resources shared by the graphics cores in the graphics core array. The shared functions within the shared function logic 420 are hardware logic units that provide special additional functions for the graphics core cluster 414. In various embodiments, the shared function logic 420 can include, but is not limited to, a sampler logic 421, a math logic 422, and an inter-thread communication (ITC) logic 423. In some embodiments, one or more caches 425 are also implemented within the shared function logic 420. The shared function logic 420 can provide the same or similar functionality as the additional fixed-function logic 238 in Fig. Implement 2B.

[0083] A shared function is implemented at least when the demand for a particular specialized function is insufficient to warrant inclusion in the graphics core cluster 414. Instead, a single instantiation of this specialized function is implemented as a standalone entity in the shared function logic 420 and shared among the execution resources within the graphics core cluster 414. The exact set of functions shared by and contained within the graphics core cluster 414 varies between embodiments. In some embodiments, certain shared functions within the shared function logic 420 that are heavily utilized by the graphics core cluster 414 may also be contained within the shared function logic 416 within the graphics core cluster 414.In various embodiments, the shared function logic 416 within the graphics core cluster 414 can contain some or all of the logic elements within the shared function logic 420. In one embodiment, all logic elements in the shared function logic 420 can be duplicated in the shared function logic 416 of the graphics core cluster 414. In another embodiment, the shared function logic 420 is excluded in favor of the shared function logic 416 within the graphics core cluster 414. Graphics processing resources

[0084] Fig. Figures 5A-5C show an execution logic with an arrangement of processing elements used in a graphics processor according to the embodiments described herein. Fig. Figure 5A illustrates a graphics core cluster according to one embodiment. Fig. Figure 5B illustrates a vector engine of a graphics kernel according to one embodiment. Fig. Figure 5C illustrates a matrix engine of a graphics kernel according to one embodiment. Elements of Fig. 5A-5C, with the same reference symbols as the elements of any other figure herein, may operate or function in any manner similar to that described elsewhere herein, but are not limited to that. For example, the elements from Fig. 5A-5C in the context of graphics processor core block 219 from Fig. 2B and / or the graphics core blocks 415A-415B from Fig. 4 are considered. In one embodiment, the elements consist of Fig. 5A-5C offers similar functionality to corresponding components of the 208 graphics processor. Fig. 2A, the GPU 239 from Fig. 2C or the GPGPU 270 from Fig. 2D.

[0085] As in Fig. As shown in Figure 5A, the graphics core cluster 414 in one embodiment comprises a graphics core block 415, which is either the graphics core block 415A or the graphics core block 415B. Fig. 4 can act. The graphics core block 415 can contain any number of graphics cores (e.g., graphics core 515A, graphics core 515B to graphics core 515N). Multiple instances of the graphics core block 415 can be present. In one embodiment, the elements of graphics cores 515A-515N have similar or equivalent functionality to the elements of graphics cores 221A-221F in Fig. 2B. In such an embodiment, the graphics cores 515A-515N each include a circuit arrangement which, but is not limited to, includes vector engines 502A-502N, matrix engines 503A-503N, memory load / store units 504A-504N, instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, and samplers 510A-510N. The circuit arrangement of the graphics cores 515A-515N may additionally include fixed-function logic 512A-512N. The number of vector engines 502A-502N and matrix engines 503A-503N in the graphics cores 515A-515N of a design can vary depending on the workload, performance and power requirements of the design.

[0086] With respect to the 515A graphics core, the 502A vector engine and the 503A matrix engine are configurable to perform parallel computational operations on data in a variety of integer and floating-point data formats based on instructions associated with shader programs. Each 502A vector engine and 503A matrix engine can function as a programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. The 502A vector engine and the 503A matrix engine support the processing of variable-width vectors with various SIMD widths, including, but not limited to, SIMD8, SIMD16, and SIMD32. Input data elements can be stored as a packed data type in a register, and the 502A vector engine and the 503A matrix engine can process the different elements based on their data size.For example, when working with a 256-bit wide vector, the 256 bits of the vector are stored in a register, with the vector being processed as four separate packed 64-bit data elements (quad-word (QW)-sized data elements), eight separate packed 32-bit data elements (dual-word (DW)-sized data elements), sixteen separate packed 16-bit data elements (word (W)-sized data elements), or thirty-two separate packed 8-bit data elements (byte (B)-sized data elements). However, other vector widths and register sizes are also possible. In one embodiment, the 502A vector engine and the 503A matrix engine are also configurable for SIMT operations on warps or thread groups of varying sizes (e.g., 8, 16, or 32 threads).

[0087] The 515A graphics core also handles memory access requests for services of a 504A memory load / store unit, issued by the 502A vector engine, the 503A matrix engine, and / or other components of the 515A graphics core that have memory access. The memory access request can be processed by the 504A memory load / store unit to load or store the requested data into or out of the cache or memory in a register associated with the 502A vector engine and / or the 503A matrix engine. The 504A memory load / store unit can also perform prefetch operations.In one embodiment, the memory load / store unit 504A is configured to perform a SIMT scatter / collect prefetch or block prefetch for data stored in memory 610 from a memory local to other tiles, via the tile intermediary 608, or from system memory. The prefetch can be performed in a specific L1 cache (e.g., data cache / shared local memory 506A), the L2 cache 604, or the L3 cache 606. In one embodiment, a prefetch to the L3 cache 606 automatically results in the data being stored in the L2 cache 604.

[0088] The instruction cache 505A stores instructions to be executed by the graphics core 515A. In one embodiment, the graphics core 515A also includes an instruction fetch and prefetch circuitry that fetches or prefetches instructions into the instruction cache 505A. The graphics core 515A also includes instruction decoding logic for decoding instructions within the instruction cache 505A. The data cache / shared local memory 506A can be configured as a data cache managed by a cache controller implementing a cache replacement policy and / or as explicitly managed shared memory. The ray tracing unit 508A includes circuitry for accelerating ray tracing operations. The sampler 510A provides texture sampling for 3D operations and media sampling for media operations.The 512A fixed-function logic incorporates a fixed-function circuit arrangement shared by the various instances of the 502A vector engine and the 503A matrix engine. The 515B-515N graphics cores can operate in a similar manner to the 515A graphics core.

[0089] The functionality of the instruction caches 505A-505N, data caches / shared local memory 506A-506N, ray tracing units 508A-508N, samplers 510A-2710N, and fixed-function logic 512A-512N corresponds to equivalent functionality in the graphics processor architectures described herein. For example, the instruction caches 505A-505N can be accessed in a similar manner to the instruction cache 255 from Fig. 2D operation. The data caches / shared local memory 506A-506N, beam tracing units 508A-508N, and samplers 510A-2710N can be used in a similar way to the cache / SLM 228A-228F, beam tracing units 227A-227F, and samplers 226A-226F. Fig. 2B. The fixed-function logic 512A-512N can use elements of the geometry / fixed-function pipeline 231 and / or additional fixed-function logic 238 from Fig. 2B included. In one embodiment, the beam tracking units 508A-508N include a circuit arrangement for performing beam tracking acceleration operations, which are provided by the beam tracking cores 245. Fig. 2C will be carried out.

[0090] As in Fig. As shown in Figure 5B, the vector engine 502, in one embodiment, includes an instruction fetch unit 537, a general-purpose register file array (GRF) 524, an architecture register file array (ARF) 526, a thread arbitrator 522, a send unit 530, a branch unit 532, a set of SIMD floating-point units (FPUs) 534, and, in another embodiment, a set of integer SIMD ALUs 535. The GRF 524 and the ARF 526 comprise the set of general-purpose register files and architecture register files associated with each hardware thread that can be active in the vector engine 502. In one embodiment, an architecture state per thread is maintained in the ARF 526, while data used during thread execution is stored in the GRF 524. The execution state of each thread, including the instruction pointers for each thread, can be maintained in thread-specific registers in the ARF array 526.

[0091] In one embodiment, the Vector Engine 502 features an architecture that combines Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the desired number of concurrent threads and the number of registers per graphics core, allocating graphics core resources to the logic for executing multiple concurrent threads. The number of logical threads that the Vector Engine 502 can execute is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0092] In one embodiment, the vector engine 502 can output multiple instructions, each of which may be different instructions, simultaneously. The thread arbitrator 522 can send the instructions for execution to one of the send units 530, branch units 532, or SIMD FPU(s) 534. Each execution thread can access 128 general-purpose registers within the GRF 524, each register capable of storing 32 bytes accessible as a variable-width vector of 32-bit data elements. In one embodiment, each thread has access to 4 kBytes within the GRF 524, although embodiments are not limited to this, and more or fewer register resources may be provided in other embodiments. In one embodiment, the vector engine 502 is partitioned into seven hardware threads that can independently perform computational operations, although the number of threads per vector engine 502 may also vary according to embodiments.In one embodiment, for example, up to 16 hardware threads are supported. In an embodiment where seven threads can access 4 KB each, the GRF 524 can store a total of 28 KB. If 16 threads can access 4 KB each, the GRF 524 can store a total of 64 KB. Flexible addressing modes allow registers to be addressed together to effectively create wider registers or to represent evenly spaced rectangular block data structures.

[0093] In one embodiment, memory operations, sampler operations, and other system communications with longer latency are sent via "Send" instructions executed by the message forwarding sending unit 530. In another embodiment, branching instructions are sent to a dedicated branching unit 532 to facilitate SIMD divergence and ultimately convergence.

[0094] In one embodiment, the vector engine 502 includes one or more SIMD floating-point units (FPUs) 534 for performing floating-point operations. In another embodiment, the FPUs 534 also support integer arithmetic. In one embodiment, the FPUs 534 can perform up to M 32-bit floating-point (or integer) operations, or up to 2M 16-bit integer or 16-bit floating-point operations. In one embodiment, at least one of the FPUs provides enhanced mathematical capability to support high-throughput transcendental math functions and 64-bit double-precision floating-point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 is also present, which may be specifically optimized for performing operations related to machine learning computations.In one embodiment, the SIMD ALUs are replaced by an additional set of SIMD FPUs 534, which can be configured to perform integer and floating-point operations. In another embodiment, the SIMD FPUs 534 and the SIMD ALUs 535 are configurable to execute SIMT programs. In another embodiment, a combined SIMD+SIMT operation is supported.

[0095] In one embodiment, arrays of multiple instances of the Vector Engine 502 can be instantiated in a graphics core. For scalability, product architects can select the exact number of vector engines per graphics core grouping. In one embodiment, the Vector Engine 502 can execute instructions across multiple execution channels. In another embodiment, each thread executed by the Vector Engine 502 runs on a separate channel.

[0096] As in Fig. As shown in Figure 5C, the matrix engine 503, in one embodiment, comprises an array of processing elements configured to perform tensor operations, including vector / matrix and matrix / matrix operations, such as, but not limited to, matrix multiplication and / or dot product operations. The matrix engine 503 is configured with M rows and N columns of processing elements (552AA-552MN) that include multiplier and adder circuits organized in a pipeline. In one embodiment, the processing elements 552AA-552MN form the physical pipeline stages of an N-wide and M-deep systolic array that can be used to perform vector-matrix or matrix-matrix operations in a data-parallel manner, including matrix multiplication, fused multiplication-addition, dot product, or other general matrix-matrix multiplication (GEMM) operations.In one embodiment, the Matrix Engine 503 supports 16-bit floating-point operations as well as 8-bit, 4-bit, 2-bit, and binary integer operations. The Matrix Engine 503 can also be configured to accelerate certain machine learning operations. In such embodiments, the Matrix Engine 503 can be configured with support for the bfloat (Brain Floating Point) 16-bit floating-point format or a 32-bit tensor floating-point format (TF32), which have different numbers of mantissa and exponent bits relative to Institute of Electrical and Electronics Engineers (IEEE) 754 formats.

[0097] In one embodiment, each stage during each cycle can add the result of the operations performed in that stage to the output of the previous stage. In other embodiments, the pattern of data movement between the 552AA-552MN processing elements can vary after a set of computation cycles based on the instruction or macro operation performed. For example, in one embodiment, a subtotal loop is enabled, and the processing elements can instead add the output of a current cycle to an output generated in the previous cycle. In another embodiment, the last stage of the systolic array can be configured with a loop back to the initial stage of the systolic array. In such an embodiment, the number of physical pipeline stages can be decoupled from the number of logical pipeline stages supported by the 503 matrix engine.For example, if the processing elements 552AA-552MN are designed as a systolic arrangement of M physical stages, a return loop from stage M to the initial pipeline stage can enable the processing elements 552AA-552MN to operate as a systolic arrangement of, for example, 2M, 3M, 4M, etc. logical pipeline stages.

[0098] In one embodiment, the matrix engine 503 comprises the memories 541A-541N and 542A-542M for storing input data in the form of row and column data for input matrices. The memories 542A-542M are configurable to store row elements (A0-Am) of a first input matrix, and the memories 541A-541N are configurable to store column elements (B0-Bn) of a second input matrix. The row and column elements are supplied to the processing elements 552AA-552MN for processing. In one embodiment, row and column elements of the input matrices can be stored in a systolic register bank 540 within the matrix engine 503 before these elements are supplied to the memories 541A-541N and 542A-542M. In one embodiment, the systolic register file 540 is excluded, and the memory 541A-541N, 542A-542M is created from registers in an associated vector engine (e.g., GRF 524 of vector engine 502). Fig. 5B) or another memory location of the graphics core that includes the Matrix Engine 503 (e.g., data cache / shared local memory 506A for the Matrix Engine 503A of Fig. 5A). Results generated by the processing elements 552AA-552MN are then output to an output buffer and / or written to a register bank (e.g., systolic register bank 540, GRF 524, data cache / shared local memory 506A-506N) for further processing by other functional units of the graphics processor or for output to memory.

[0099] In some embodiments, the Matrix Engine 503 is configured with support for input sparsity, whereby multiplication operations for sparsely populated areas of input data can be avoided by skipping multiplication operations that have a zero operand. In one embodiment, the processing elements 552AA-552MN are configured to skip the execution of certain operations that have a zero input value. Loading zero-value operands into the processing elements can be avoided, and the processing elements 552AA-552MN can be configured to perform multiplications on the input elements with non-zero values.The Matrix Engine 503 can also be configured to support sparse output, thus avoiding operations with results predetermined as zero. For input sparsity and / or output sparsity, one embodiment provides metadata to the 552AA-552MN processing elements to specify which processing elements and / or data channels should be active during a processing cycle.

[0100] In one embodiment, the Matrix Engine 503 includes hardware to enable operations on sparse data using a compressed representation of a sparse matrix that stores non-zero values ​​and metadata that identifies the positions of the non-zero values ​​within the matrix. Examples of compressed representations include compressed tensor representations such as compressed sparse rows (CSR), compressed sparse columns (CSC), and compressed sparse fibers (CSF). Support for compressed representations allows operations to be performed on an input in a compressed tensor format without requiring decompression or decoding of the compressed representation. In such an embodiment, operations can be performed only on non-zero input values, and the resulting non-zero output values ​​can be mapped to an output matrix.In some embodiments, hardware support is also provided for machine-specific lossless data compression formats used when transferring data within the hardware or over system buses. Such data can be retained in a compressed format for sparse input data, and the Matrix Engine 503 can use the compression metadata for the compressed data to allow operations to be performed on only non-zero values ​​or to avoid blocks of zero data input for multiplication operations.

[0101] In various embodiments, input data can be provided by a programmer in a compressed tensor representation, or a codec can compress input data into the compressed tensor representation or another sparse data encoding. In addition to supporting compressed tensor representations, streaming compression of sparse input data can be performed before the data is delivered to the 552AA-552MN processing elements. In one embodiment, compression is performed on data written to a cache memory associated with the 414 graphics core cluster, using an encoding supported by the 503 matrix engine.In one embodiment, the Matrix Engine 503 includes support for structured sparsity input, where a predetermined level or pattern of sparsity is imposed on the input data. This data can be compressed to a known compression ratio, and the compressed data is processed by the processing elements 552AA to 552MN according to the metadata associated with the compressed data.

[0102] Fig. Figure 6 illustrates a tile 600 of a multi-tile processor according to one embodiment. In one embodiment, tile 600 is representative of one of the graphics engine tiles 310A-310D from Fig. 3B or the computing engine tiles 340A-340D from Fig. 3C. Tile 600 of the multi-tile graphics processor contains an array of graphics core clusters (e.g., graphics core cluster 414A, graphics core cluster 414B through graphics core cluster 414N), each graphics core cluster containing an array of graphics cores 515A-515N. Tile 600 also includes a global allocator 602, which allocates threads to the processing resources of tile 600.

[0103] The tile 600 can include or be coupled to an L3 cache 606 and a memory 610. In various embodiments, the L3 cache 606 can be omitted, or the tile 600 can contain additional cache levels, such as an L4 cache. In one embodiment, each instance of the tile 600 in the multi-tile graphics processor has an associated memory 610, as shown in Fig. 3B and Fig. 3C. In one embodiment, a multi-tile processor can be configured as a multi-chip module in which the L3 cache 606 and / or the memory 610 are located on different chiplets than the graphics core clusters 414A-414N. In this context, a chiplet is an at least partially packaged integrated circuit containing individual logic units that can be assembled with other chiplets to form a larger package. For example, the L3 cache 606 can be contained in a dedicated cache single chip or can be located on the same single chip as the graphics core clusters 414A-414N. In one embodiment, the L3 cache 606 can be contained in an active base die or an active interposer, as shown in Fig. 11C illustrates.

[0104] A memory interface 603 enables communication between the graphics core clusters 414A-414N, the L3 cache 606, and the memory 610. An L2 cache 604 is coupled to the memory fabric 603 and is configurable to cache transactions performed via the memory fabric 603. A tile interface 608 enables communication with other tiles on the graphics processors and can be accessed from tile interfaces 323A-323F. Fig. 3B and Fig. 3C. In embodiments where the L3 cache 606 is excluded from tile 600, the L2 cache 604 can be configured as a combined L2 / L3 cache. The memory fabric 603 can be configured to forward data to the L3 cache 606 or to memory controllers associated with memory 610, depending on whether the L3 cache 606 is present in a given implementation. The L3 cache 606 can be configured as a per-tile cache dedicated to processing resources of tile 600, or it can be a partition of a GPU-wide L3 cache.

[0105] Fig. Figure 7 is a block diagram illustrating a graphics processor instruction format according to some embodiments 700. In one or more embodiments, the graphics processor cores support an instruction set containing instructions in multiple formats. The boxes with solid lines illustrate the components generally included in a graphics core instruction, while the boxes with dashed lines contain components that are optional or that are included only in a subset of instructions. In some embodiments, the described and illustrated graphics processor instruction format 700 consists of macro instructions insofar as they are instructions fed to the graphics core, as opposed to microoperations that result from instruction decoding once the instruction is processed. Thus, a single instruction can cause hardware to perform multiple microoperations.

[0106] In some embodiments, the graphics processor natively supports instructions in a 128-bit instruction format 710. Based on the selected instruction, instruction options, and number of operands, a condensed 64-bit instruction format 730 is available for some instructions. The native 128-bit instruction format 710 provides access to all instruction options, while some options and operations are limited in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary depending on the embodiment. In some embodiments, the instruction is partially condensed using a set of index values ​​in an index field 713. The graphics core hardware references a set of condensation tables based on the index values ​​and uses the condensation table outputs to reconstruct a native instruction in the 128-bit instruction format 710.Other sizes and formats can also be used for instructions.

[0107] For each format, an instruction opcode 712 defines the operation to be performed by the graphics kernel. The graphics kernels execute each instruction in parallel across the multiple data elements of each operand. For example, in response to an add instruction, the graphics kernel performs a simultaneous add operation across each color channel representing a texture element or image element. By default, the graphics kernel executes each instruction across all data channels of the operands. In some embodiments, the instruction control field 714 allows control of certain execution options, such as the selection of channels (e.g., predicate) and the order of the data channels (e.g., swizzle). For instructions in the 128-bit instruction format 710, an exec-size field 716 limits the number of data channels that are executed in parallel. In some embodiments, the exec-size field 716 is not available for use in the compact 64-bit instruction format 730.

[0108] Some graphics kernel instructions have up to three operands, including two source operands, src0 720 and src1 722, and one destination operand 718. In some embodiments, the graphics kernels support dual destination instructions, where one of the destinations is implied. Data manipulation instructions can have a third source operand (e.g., SRC2 724), with the instruction opcode 712 determining the number of source operands. The last source operand of an instruction can be an immediate (e.g., hard-coded) value passed with the instruction.

[0109] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726, which specifies, for example, whether a direct register addressing mode or an indirect register addressing mode is used. In direct register addressing mode, the register address of one or more operands is specified directly by bits in the instruction.

[0110] In some embodiments, the 128-bit instruction format 710 has an access / address mode field 726 that specifies an address mode and / or an access mode for the instruction. In one embodiment, the access mode is used to define a data access alignment for the instruction. Some embodiments support access modes, including a 16-byte-aligned access mode and a 1-byte-aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and in a second mode, the instruction may use 16-byte-aligned addressing for all source and destination operands.

[0111] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction should use direct or indirect addressing. When direct register addressing mode is used, bits in the instruction directly provide the register address of one or more operands. When indirect register addressing mode is used, the register address of one or more operands can be calculated based on an address register value and an immediate address field in the instruction.

[0112] In some embodiments, the instructions are grouped based on the bit fields of opcode 712 to simplify the decoding of opcode 740. For an 8-bit opcode, bits 4, 5, and 6 allow the graphics kernel to determine the opcode type. The precise grouping of opcodes shown is merely an example. In some embodiments, a move and opcode logic group 742 includes data move and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 742 shares the five most significant bits (MSBs), with move instructions (mov) having the form 0000xxxxb and logic instructions having the form 0001xxxxb. A flow control instruction group 744 (e.g. call, jump (jmp)) contains instructions in the form 0010xxxxb (e.g. 0x20).A miscellaneous instruction group 746 contains a mix of instructions, including synchronization instructions (e.g., wait, send) in the form 0011xxxxb (e.g., 0x30). A parallel math instruction group 748 contains component-wise arithmetic instructions (e.g., add, multiply) in the form 0100xxxxb (e.g., 0x40). The parallel math instruction group 748 performs the arithmetic operations in parallel across data channels. The vector math group 750 contains arithmetic instructions (e.g., dp4) in the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic calculations such as the dot product on vector operands. The illustrated opcode decoding 740 can be used in one embodiment to determine which section of a graphics kernel is used to execute a decoded instruction.For example, some instructions can be designated as systolic instructions, which are executed through a systolic array. Other instructions, such as ray-tracing instructions (not shown), can be routed to a ray-tracing kernel or to ray-tracing logic within a slice or partition of execution logic. Graphics pipeline

[0113] Fig. Figure 8 is a block diagram of another embodiment of a graphics processor 800. Elements from Fig. 8, which have the same reference numbers (or designations) as the elements of any other figure herein, may be operated or function in any similar manner as described elsewhere herein, but are not limited to doing so.

[0114] In some embodiments, the graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, a thread execution logic 850, and a render output pipeline 870. In some embodiments, the graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by instructions issued to the graphics processor 800 via a ring interconnect 802. In some embodiments, the ring interconnect 802 couples the graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors.Commands from the ring interconnect 802 are interpreted by a command streamer 803, which delivers instructions to individual components of the geometry pipeline 820 or the media pipeline 830.

[0115] In some embodiments, the command streamer 803 manages the operation of a vertex retriever 805, which reads vertex data from memory and executes vertex processing instructions provided by the command streamer 803. In some embodiments, the vertex retriever 805 provides vertex data to a vertex shader 807, which performs coordinate space transformation and illumination operations for each vertex. In some embodiments, the vertex retriever 805 and the vertex shader 807 execute vertex processing instructions by distributing execution threads to the graphics cores 852A-852B via a thread dispatcher 831.

[0116] In some embodiments, the 852A-852B graphics cores are an array of vector processors with an instruction set for performing graphics and media operations. In some embodiments, the 852A-852B graphics cores have an attached L1 cache 851, which is specific to each array or shared by the arrays. The cache can be configured as a data cache, an instruction cache, or as a single cache partitioned to contain data and instructions in separate partitions.

[0117] In some embodiments, the geometry pipeline 820 includes tessellation components for performing hardware-accelerated tessellation of 3D objects. In some embodiments, a programmable envelope shader 811 executes the tessellation operations. A programmable domain shader 817 provides backend evaluation of the tessellation output. A tessellator 813 operates on the instructions of the envelope shader 811 and includes special logic for generating a set of detailed geometric objects based on a coarse geometric model provided as input to the geometry pipeline 820. In some embodiments, if tessellation is not used, tessellation components (e.g., envelope shader 811, tessellator 813, and domain shader 817) can be skipped. The tessellation components can operate based on data received from the vertex shader 807.

[0118] In some embodiments, complete geometric objects can be processed by a geometry shader 819 via one or more threads distributed to the graphics cores 852A-852B, or can be passed directly to the clipper 829. In some embodiments, the geometry shader operates on entire geometric objects, rather than on vertices or figures of vertices, as in previous stages of the graphics pipeline. When tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable by a geometry shader program to perform geometry tessellation when the tessellation units are disabled.

[0119] Prior to rasterization, a clipper 829 processes vertex data. The clipper 829 can be a fixed-function clipper or a programmable clipper with clipping and geometry shading capabilities. In some embodiments, a rasterizer and depth-testing component 873 distributes pixel shaders in the rendering output pipeline 870 to convert the geometric objects into pixel-wise representations. In some embodiments, pixel-shading logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth-testing component 873 and access unrasterized vertex data via an output streaming unit 823.

[0120] The 800 graphics processor has an interlink bus, interlink fabric, or some other interlinking mechanism that allows data and messages to be passed between the main components of the processor. In some embodiments, the graphics cores 852A-852B and the associated logic units (e.g., L1 cache 851, sampler 854, texture cache 858, etc.) are interconnected via a data port 856 to perform memory access and communicate with the processor's render-output pipeline components. In some embodiments, the sampler 854, caches 851 and 858, and graphics cores 852A-852B each have separate memory access paths. In one embodiment, the texture cache 858 can also be configured as a sampler cache.

[0121] In some embodiments, the render output pipeline 870 includes a rasterization and depth-testing component 873 that converts vertex-based objects into an associated pixel-based representation. In some embodiments, the rasterization logic includes a window / masking unit for performing fixed-function triangle and line rasterization. An associated render cache 878 and depth cache 879 are also available in some embodiments. A pixel operations component 877 performs pixel-based operations on the data, although in some cases pixel operations in conjunction with 2D operations (e.g., bitblock image transfers with blending) are performed by the 2D engine 841 or replaced at display time by the display controller 843 using overlay display layers.In some embodiments, a shared L3 cache 875 is available to all graphics components, so that data can be shared without using the system's main memory.

[0122] In some embodiments, the media pipeline 830 comprises a media engine 837 and a video front end 834. In some embodiments, the video front end 834 receives pipeline commands from the command streamer 803. In some embodiments, the media pipeline 830 includes a separate command streamer. In some embodiments, the video front end 834 processes media commands before sending the command to the media engine 837. In some embodiments, the media engine 837 includes thread creation functionality to generate threads for dispatch to the thread execution logic 850 via the thread dispatcher 831.

[0123] In some embodiments, the graphics processor 800 includes a display engine 840. In some embodiments, the display engine 840 is external to the processor 800 and couples to the graphics processor via the ring interconnect 802 or some other interconnect bus or interconnect fabric. In some embodiments, the display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, the display engine 840 includes special-purpose logic that can operate independently of the 3D pipeline. In some embodiments, the display controller 843 couples to a display device (not shown), which may be a system-integrated display device, such as in a laptop, or an external display device connected via a display device connector.

[0124] In some embodiments, the 820 geometry pipeline and the 830 media pipeline are configurable to perform operations based on multiple graphics and media programming interfaces and are not tied to a specific application programming interface (API). In some embodiments, the GPU driver software translates API calls specific to a particular graphics or media library into commands that the GPU can process. Some embodiments provide support for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL), and / or the Vulkan graphics and compute API, all from the Khronos group. Some embodiments may also provide support for the Microsoft Corporation's Direct3D library. Some embodiments may support a combination of these libraries.Support for the Open Source Computer Vision Library (OpenCV) can also be provided. A future API with a compatible 3D pipeline would also be supported if a mapping from the future API pipeline to the GPU pipeline can be created. Programming graphics pipelines

[0125] Fig. Figure 9A is a block diagram illustrating a Graphics Processing Unit (GPU) instruction format 900, which can be used to program graphics processing pipelines, according to some embodiments. Fig. Figure 9B is a block diagram illustrating a graphics processor instruction sequence 910, according to one embodiment. The boxes with solid lines in Fig. Figure 9A illustrates the components generally included in a graphics instruction, while the dashed lines indicate components that are optional or included only in a subset of graphics instructions. The example graphics processor instruction format 900 from Fig. 9A contains data fields to identify a client (902), a command operation code (opcode) (904), and a data field (906) for the command. Some commands also have a sub-opcode (905) and a command size (908).

[0126] In some embodiments, Client 902 specifies the graphics device client unit that processes the instruction data. In some embodiments, a graphics processor instruction parser examines the client field of each instruction to determine further instruction processing and route the instruction data to the appropriate client unit. In some embodiments, the graphics processor client units include a memory interface unit, a render unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that handles the instructions. Once the instruction is received from the client unit, the client unit reads opcode 904 and, if present, sub-opcode 905 to determine the operation to be performed. The client unit executes the instruction using the information in data field 906.For some instructions, an explicit instruction size of 908 is expected to specify the instruction size. In some implementations, the instruction parser automatically determines the size of at least some instructions based on the instruction opcode. In some implementations, instructions are standardized using multiples of a double word. Other instruction formats may also be used.

[0127] The flowchart in Fig. Figure 9B illustrates an exemplary graphics processor instruction sequence 910. In some embodiments, the software or firmware of a data processing system with a graphics processor uses a version of the instruction sequence shown to set, execute, and terminate a set of graphics operations. A sample sequence is shown and described only as an example, since the embodiments are not limited to these specific instructions or to this instruction sequence. Furthermore, the instructions can be issued as a batch of instructions in an instruction sequence, such that the graphics processor processes the instruction sequence at least partially concurrently.

[0128] In some embodiments, the GPU instruction sequence 910 can begin with a pipeline flush instruction 912 to cause an active graphics pipeline to complete its currently pending instructions. In some embodiments, the 3D pipeline 922 and the media pipeline 924 do not operate concurrently. The pipeline flush is performed to cause the active graphics pipeline to complete any pending instructions. In response to a pipeline flush, the GPU instruction parser suspends instruction processing until the active drawing engines have completed their pending operations and the corresponding read caches have been cleared. Optionally, any data in the rendering cache marked as "dirty" can be flushed into memory.In some embodiments, the Pipeline Flush instruction 912 can be used for pipeline synchronization or before putting the graphics processor into a low-power state.

[0129] In some embodiments, a pipeline select instruction 913 is used when an instruction sequence requires the graphics processor to explicitly switch between pipelines. In some embodiments, a pipeline select instruction 913 is required only once within an execution context before pipeline instructions are issued, unless the context specifies issuing instructions for both pipelines. In some embodiments, a pipeline flush instruction 912 is required immediately before switching pipelines via the pipeline select instruction 913.

[0130] In some embodiments, a pipeline control command 914 configures a graphics pipeline for operation and is used to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, the pipeline control command 914 configures the pipeline state for the active pipeline. In one embodiment, the pipeline control command 914 is used to synchronize the pipeline and clear data from one or more caches within the active pipeline before processing a batch of commands.

[0131] In some embodiments, instructions relating to the state of return buffer 916 are used to configure a set of return buffers for the respective pipelines used to write data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operations write intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and to perform cross-thread communication. In some embodiments, the return buffer state 916 involves selecting the size and number of return buffers to be used for a set of pipeline operations.

[0132] The remaining commands in the command sequence differ depending on the active pipeline for operations. Based on a pipeline determination 920, the command sequence is tailored to the 3D pipeline 922, starting with the 3D pipeline state 930, or the media pipeline 924, starting with the media pipeline state 940.

[0133] The commands for configuring 3D pipeline state 930 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that should be configured before 3D primitive commands are processed. The values ​​of these commands are determined, at least in part, based on the specific 3D API in use. In some implementations, 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements are not intended to be used.

[0134] In some embodiments, the 3D primitive instruction 932 is used to send 3D primitives to be processed by the 3D pipeline. Instructions and associated parameters passed to the graphics processor via the 3D primitive instruction 932 are forwarded to the vertex fetch function in the graphics pipeline. The vertex fetch function uses the data from the 3D primitive instruction 932 to create vertex data structures. The vertex data structures are stored in one or more return buffers. In some embodiments, the 3D primitive instruction 932 is used to perform vertex operations on 3D primitives via vertex shaders. To process vertex shaders, the 3D pipeline 922 sends shader programs to the graphics cores.

[0135] In some embodiments, the 3D pipeline 922 is triggered by an execution instruction 934 or an event. In some embodiments, a register write triggers instruction execution. In some embodiments, execution is triggered by a "Go" or "Kick" instruction in the instruction sequence. In one embodiment, instruction execution is triggered by a pipeline synchronization instruction to flush the instruction sequence through the graphics pipeline. The 3D pipeline performs geometry processing for the 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized, and the pixel engine colors the resulting pixels. Additional instructions for controlling pixel shading and pixel backend operations may also be included for these operations.

[0136] In some embodiments, the graphics processor instruction sequence 910 follows the path of the media pipeline 924 when performing operations. Generally, the specific use and programming of the media pipeline 924 depends on the media or computational operations to be performed. Specific media decoding operations can be offloaded to the media pipeline during decoding. In some embodiments, the media pipeline can also be bypassed, and media decoding can be performed wholly or partially using resources provided by one or more general-purpose processing cores.In one embodiment, the media pipeline also includes elements for general-purpose graphics processing unit (GPGPU) operations, where the graphics processor performs SIMD vector operations using computational shading programs that are not explicitly associated with rendering graphics primitives.

[0137] In some embodiments, the media pipeline 924 is configured similarly to the 3D pipeline 922. A set of commands for configuring the state of the media pipeline 940 is sent before the commands for the media object 942 or placed in a command queue. In some embodiments, the commands for media pipeline state 940 include data for configuring the media pipeline elements used to process the media objects. This includes data for configuring the video decoding and encoding logic within the media pipeline, such as the encoding or decoding format. In some embodiments, commands for media pipeline state 940 also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0138] In some embodiments, Media Object Instructions 942 provide pointers to media objects for processing by the media pipeline. The media objects have memory buffers containing video data to be processed. In some embodiments, all states of the media pipeline must be valid before an instruction is issued for Media Object 942. Once the pipeline state is configured and the instructions for Media Object 942 are queued, Media Pipeline 924 is triggered by an Execute Instruction 944 or an equivalent execution event (such as a register write). The output from Media Pipeline 924 can then be post-processed by operations provided by 3D Pipeline 922 or Media Pipeline 924. In some embodiments, GPGPU operations are configured and executed similarly to media operations. Graphics software architecture

[0139] Fig. Figure 10 illustrates an exemplary graphics software architecture for a data processing system 1000 according to some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 are each executed in the system memory 1050 of the data processing system.

[0140] In some embodiments, the 3D graphics application 1010 contains one or more shader programs, comprising shader instructions 1012. The shader language instructions may be in a higher-level shader language, such as HLSL (High-Level Shader Language) from Direct3D, GLSL (OpenGL Shader Language), and so on. The application also includes executable instructions 1014 in a machine language suitable for execution by the general-purpose processor core 1034. The application also includes graphics objects 1016 defined by vertex data.

[0141] In some embodiments, the 1020 operating system is a Microsoft® Windows® operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open-source UNIX-like operating system that uses a variant of the Linux kernel. The 1020 operating system can support a graphics API 1022, such as the Direct3D API, the OpenGL API, or the Vulkan API. When the Direct3D API is used, the 1020 operating system uses a front-end shader compiler 1024 to compile arbitrary shader instructions 1012 in HLSL into a lower-level shader language. The compilation can be just-in-time (JIT) compilation, or the application can perform shader pre-compilation. In some embodiments, when the 3D graphics application 1010 is compiled, higher-level shaders are compiled into lower-level shaders.In some embodiments, the shader instructions 1012 are provided in an intermediate form, such as a version of SPIR (Standard Portable Intermediate Representation) used by the Vulkan API.

[0142] In some embodiments, the user-mode graphics driver 1026 includes a backend shader compiler 1027 for converting the shader instructions 1012 into a hardware-specific representation. When the OpenGL API is used, shader instructions 1012 are passed in the higher-level programming language GLSL to a user-mode graphics driver 1026 for compilation. In some embodiments, the user-mode graphics driver 1026 communicates with a kernel-mode graphics driver 1029 using operating system kernel-mode functions 1028. In some embodiments, the kernel-mode graphics driver 1029 communicates with the graphics processor 1032 to send commands and instructions. IP Core Implementations

[0143] One or more aspects of at least one embodiment can be implemented by a representative code stored on a machine-readable medium that represents and / or defines the logic within an integrated circuit, such as a processor. For example, the machine-readable medium may contain instructions representing different logics within the processor. When read by a machine, the instructions can cause the machine to construct the logic to perform the techniques described herein. Such representations, known as "IP kernels," are reusable logic units for an integrated circuit, stored on a tangible machine-readable medium as a hardware model that describes the structure of the integrated circuit.The hardware model can be supplied to various customers or manufacturing facilities, which load the hardware model onto manufacturing machines that produce the integrated circuit. The integrated circuit can be manufactured such that the circuit performs operations corresponding to one of the embodiments described herein.

[0144] Fig. Figure 11A is a block diagram illustrating an IP core development system 1100, which can be used to fabricate an integrated circuit for performing operations according to an embodiment. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into a larger design or used to build an entire integrated circuit (e.g., a SoC circuit). A design facility 1130 can generate a software simulation 1110 of an IP core design in a higher-level programming language (e.g., C / C++). The software simulation 1110 can be used with a simulation model 1112 to design, test, and verify the behavior of the IP core. The simulation model 1112 can include functional, behavioral, and / or temporal simulations.A register transfer level (RTL) design 1115 can then be generated or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the integrated circuit's behavior, modeling the flow of digital signals between hardware registers, including the associated logic executed using the modeled digital signals. In addition to an RTL design 1115, lower-level designs at the logic or transistor level can also be generated, designed, or synthesized. Therefore, the specific details of the initial design and simulation can vary.

[0145] The RTL design 1115 or an equivalent can further be synthesized by the design facility into a hardware model 1120, which may be in a hardware description language (HDL) or another representation of physical design data. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored using non-volatile memory 1140 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to an external manufacturing facility 1165. Alternatively, the IP core design can be transmitted via a wired connection 1150 or a wireless connection 1160 (e.g., over the Internet). The manufacturing facility 1165 can then manufacture an integrated circuit based at least partially on the IP core design.The manufactured integrated circuit can be designed to perform operations according to at least one embodiment described herein.

[0146] Fig. Figure 11B shows a side cross-sectional view of an integrated circuit package 1170 according to some embodiments described herein. The integrated circuit package 1170 illustrates an implementation of one or more processor or accelerator devices as described herein. The package 1170 comprises multiple units of hardware logic 1172, 1174 connected to a substrate 1180. The logic 1172, 1174 may be implemented at least partially in configurable logic or fixed-function logic hardware and may include one or more sections of any one or more processor cores, one or more graphics processors, or other accelerator devices described herein. Each logic unit 1172, 1174 may be implemented in a semiconductor die and coupled to the substrate 1180 via an interconnect structure 1173.The interconnect structure 1173 can be configured to conduct electrical signals between the logic 1172, 1174 and the substrate 1180 and can include interconnects such as, but not limited to, contact bumps or pillars. In some embodiments, the interconnect structure 1173 can be configured to conduct electrical signals such as input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 can include other suitable types of substrates. The housing assembly 1170 can be connected to other electrical devices via a housing interconnect 1183.The housing interconnect 1183 can be coupled to a surface of the substrate 1180 to conduct electrical signals to other electrical devices, such as a mainboard, another chipset or a multi-chip module.

[0147] In some embodiments, the logic units 1172 and 1174 are electrically coupled to a bridge 1182 designed to route electrical signals between the logic units 1172 and 1174. The bridge 1182 can be a dense interconnect structure providing a route for electrical signals. The bridge 1182 can have a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to provide a chip-to-chip connection between the logic units 1172 and 1174.

[0148] Although two logic units 1172, 1174, and a bridge 1182 are illustrated, embodiments described herein may have more or fewer logic units on one or more dies. The one or more dies may be connected by zero or more bridges, since the bridge 1182 can be omitted if the logic is contained on a single die. Alternatively, multiple dies or logic units may be connected by one or more bridges. In addition, multiple logic units, dies, and bridges may be interconnected in other possible configurations, including three-dimensional configurations.

[0149] Fig. Figure 11C illustrates a package arrangement 1190 containing multiple units of hardware logic individual chips connected to a substrate 1180. A graphics processing unit, a parallel processor, and / or a compute accelerator as described herein can be composed of various silicon chiplets fabricated separately. A diverse set of chiplets with different core IP logic can be assembled into a single device. Additionally, the chiplets can be integrated into a base die or base chiplet using active interfacing technology. The concepts described herein enable interconnection and communication between the different forms of IP within the GPU.IP cores can be manufactured using different process technologies and assembled during production, thus avoiding the complexity of converging multiple IPs, especially in a large SoC with several IPs in different configurations, onto the same manufacturing process. The ability to use multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. Furthermore, disaggregated IPs are better suited for independent power management; components not in use at a given workload can be switched off, reducing overall power consumption.

[0150] In various embodiments, a package assembly 1190 can contain components and chiplets interconnected by a fabric 1185 and / or one or more bridges 1187. The chiplets within the package assembly 1190 can have a 2.5D arrangement using a chip-on-wafer-on-substrate stacking, in which multiple dies are stacked side-by-side on a silicon interposer 1189 that couples the chiplets to the substrate 1180. The substrate 1180 includes electrical connections to the package interconnect 1183. In one embodiment, the silicon interposer 1189 is a passive interposer that includes silicon vias (TSVs) to electrically couple chiplets within the package assembly 1190 to the substrate 1180. In another embodiment, the silicon interposer 1189 is an active interposer that includes embedded logic in addition to TSVs.In such an embodiment, the chiplets within the package assembly 1190 are arranged on the active interposer 1189 using a 3D face-to-face die stacking. The active interposer 1189 can include hardware logic for I / O 1191, a cache memory 1192, and other hardware logic 1193, in addition to the interposer fabric 1185 and a silicon bridge 1187. The fabric 1185 enables communication between the various logic chiplets 1172, 1174 and the logic 1191, 1193 within the active interposer 1189. The fabric 1185 can be a NoC interposer or another form of packet-switched fabric that mediates data packets between components of the package assembly. In complex arrangements, the structure 1185 can be a special chiplet that enables communication between the different hardware logics of the package arrangement 1190.

[0151] Bridge structures 1187 within the active interposer 1189 can be used to provide a point-to-point connection between, for example, logic or I / O chips 1174 and memory chips 1175. In some implementations, the bridge structures 1187 can also be embedded in the substrate 1180. The hardware logic chiplets can include special-purpose hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172 and the logic or I / O chiplets 1174 can be implemented, at least partially, in configurable logic or fixed-function logic hardware and can include one or more sections of any one or more processor cores, one or more graphics processors, parallel processors, or other accelerator devices described herein. The 1175 memory chiplets can be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.The cache memory 1192 within the active interposer 1189 (or the substrate 1180) can function as a global cache for the housing arrangement 1190, as part of a distributed global cache, or as a dedicated cache for the fabric 1185.

[0152] Each chiplet can be fabricated as a separate semiconductor chip and coupled to a base die that is embedded in or coupled to the substrate 1180. Coupling to the substrate 1180 can be achieved via an interconnection structure 1173. The interconnection structure 1173 can be configured to route electrical signals between the various chiplets and the logic within the substrate 1180. The interconnection structure 1173 can include, but is not limited to, contact ridges or pillars. In some embodiments, the interconnection structure 1173 can be configured to route electrical signals, such as input / output (I / O) signals and / or power or ground signals, associated with the operation of the logic, I / O, and memory chiplets.In one embodiment, an additional intermediate connection structure couples the active interposer 1189 to the substrate 1180.

[0153] In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 may comprise other suitable types of substrates. The housing assembly 1190 may be connected to other electrical devices via a housing interconnect 1183. The housing interconnect 1183 may be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multichip module.

[0154] In some embodiments, a logic or I / O chiplet 1174 and a memory chiplet 1175 can be electrically coupled via a bridge 1187 configured to route electrical signals between the logic or I / O chiplet 1174 and a memory chiplet 1175. The bridge 1187 can be a dense interconnect structure providing a route for electrical signals. The bridge 1187 can have a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to provide a chip-to-chip connection between the logic or I / O chiplet 1174 and a memory chiplet 1175. The bridge 1187 can also be referred to as a silicon bridge or interconnect bridge. For example, in some embodiments, the bridge 1187 is an embedded multi-die interconnect (EMIB) bridge.In some embodiments, the bridge 1187 can simply be a direct connection from one chiplet to another chiplet.

[0155] Fig. Figure 11D describes a package arrangement 1194 comprising interchangeable chiplets 1195 according to one embodiment. The interchangeable chiplets 1195 can be assembled into standardized slots on one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 can be coupled via a bridge interconnect 1197, which may be similar to the other bridge interconnects described herein, such as an EMIB. The memory chiplets can also be connected to logic or I / O chiplets via a bridge interconnect. The I / O and logic chiplets can communicate via an interconnect fabric. The base chiplets can each support one or more slots in a standardized format for logic, I / O, or memory / cache.

[0156] In one embodiment, SRAM and power supply circuits can be fabricated in one or more of the base chiplets 1196, 1198, which can be manufactured using a different process technology relative to the interchangeable chiplets 1195 stacked on the base chiplets. For example, the base chiplets 1196, 1198 can be manufactured using a more advanced process technology, while the interchangeable chiplets can be manufactured using a less advanced process technology. One or more of the interchangeable chiplets 1195 can be memory chiplets (e.g., DRAM). Different memory densities can be selected for the package assembly 1194, depending on the desired power consumption and / or performance for the product using the package assembly 1194.Furthermore, during assembly, logic chiplets with a different number or types of functional units can be selected, depending on the power consumption and / or performance targets for the product. Additionally, chiplets containing IP logic cores of different types can be inserted into the slots for interchangeable chiplets, enabling hybrid processor configurations that can mix and match IP blocks of different technologies. Example of an integrated circuit (system-on-chip)

[0157] Fig. Figures 12-13B illustrate exemplary integrated circuits and associated graphics processors that can be fabricated using one or more IP cores, according to various embodiments described herein. In addition to what is illustrated, other logic and circuitry may be included, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0158] Fig. Figure 12 is a block diagram illustrating an exemplary integrated system-on-chip circuit 1200, which can be manufactured according to one embodiment using one or more IP cores. The exemplary integrated circuit 1200 comprises one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may additionally include an image processor 1215 and / or a video processor 1220, each of which may be a modular IP core of the same or several different configurations. The integrated circuit 1200 includes peripheral or bus logic, including a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I 2 S / I 2C-controller 1240. Additionally, the integrated circuit may include a display device 1245, which is linked to one or more HDMI (High-Definition Multimedia Interface) control interfaces 1250 and MIPI (Mobile Industry Processor Interface) display interfaces 1255. Storage may be provided by a flash memory subsystem 1260, which includes flash memory and a flash memory controller. A memory interface may be provided via a memory controller 1265 for accessing SDRAM or SRAM memory devices. Some integrated circuits also include an embedded security engine 1270.

[0159] Fig. Figures 13A-13B are block diagrams illustrating exemplary graphics processors for use in a SoC, according to embodiments described herein. Fig. Figure 13A illustrates an exemplary graphics processor 1310 of an integrated system-on-chip circuit which can be manufactured using one or more IP cores, according to one embodiment. Fig. Figure 13B illustrates an additional exemplary graphics processor 1340 of an integrated system-on-chip circuit, which can be manufactured using one or more IP cores, according to one embodiment. The graphics processor 1310 from Fig. 13A is an example of a low-performance graphics processor core. The 1340 graphics processor from Fig. 13B is an example of a higher-performance graphics processor core. Both the 1310 and 1340 graphics processors can be variants of the 1210 graphics processor. Fig. 12 years old.

[0160] As in Fig. As shown in Figure 13A, the 1310 graphics processor comprises a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D through 1315N-1, and 1315N). The 1310 graphics processor can execute different shading programs via separate logic, so the vertex processor 1305 is optimized to perform operations for vertex shading programs, while the one or more fragment processors 1315A-1315N perform fragment (e.g., pixel) shading operations for fragment or pixel shading programs. The vertex processor 1305 performs the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. The one or more fragment processors 1315A-1315N use the primitive and vertex data generated by the vertex processor 1305 to create a single-frame buffer that is displayed on a display device.In one embodiment, the fragment processor(s) 1315A-1315N is optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform operations similar to a pixel shader program as provided in the Direct3D API.

[0161] The 1310 graphics processor additionally includes one or more memory management units (MMUs) 1320A-1320B, cache(s) 1325A-1325B, and interconnect(s) 1330A-1330B. The one or more MMUs 1320A-1320B provide a mapping between virtual and physical addresses for the 1310 graphics processor, including the vertex processor 1305 and / or the one or more fragment processors 1315A-1315N, which can reference vertex or image / texture data stored in the one or more caches 1325A-1325B, in addition to vertex or image / texture data stored in the memory. In one embodiment, the one or more MMUs 1320A-1320B can be synchronized with other MMUs in the system, including one or more MMUs connected to the one or more application processors 1205, the image processor 1215 and / or the video processor 1220. Fig. 12 are associated, so that each processor 1205-1220 can participate in a shared or unified virtual memory system. According to embodiments, the one or more interconnects 1330A-1330B allow the graphics processor 1310 to connect to other IP cores within the SoC, either via an internal bus of the SoC or via a direct connection.

[0162] As in Fig. As shown in Figure 13B, the 1340 graphics processor includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and interconnects 1330A-1330B of the 1310 graphics processor. Fig. 13A. The 1340 graphics processor includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F through 1355N-1 and 1355N), providing a unified shader core architecture in which a single core or type of core can execute all types of programmable shader core, including shader program code for implementing vertex shaders, fragment shaders, and / or computational shaders. The unified shader core architecture is also configurable to execute directly compiled, higher-level GPGPU programs (e.g., CUDA). The exact number of shader cores may vary among embodiments and implementations.In addition, the 1340 graphics processor includes a cross-core task manager 1345, which acts as a thread dispatcher to allocate execution threads to one or more shader cores 1355A-1355N, and a tiling unit 1358 to accelerate tiling operations for tile-based rendering, in which rendering operations for a scene are subdivided in image space to, for example, exploit local spatial coherence within a scene or optimize the use of internal caches.

[0163] Fig. Figure 14 is a block diagram of an embodiment of a processor 1402. In some embodiments, the processor may be a graphics processing unit (GPU), a single-instruction, multiple-thread (SIMT) processor, or the like. In some embodiments, the GPU or other processor may be a "hard" (e.g., hard-wired) processor, such as an application-specific integrated circuit (ASIC). In other embodiments, the GPU or other processor may be a "soft" processor implemented with a field-programmable gate array (FPGA) or another type of programmable logic device (PLD). In some embodiments, the hard or soft processor may optionally be a general-purpose GPU (GPGPU). For clarity, despite its name, the GPGPU does not necessarily have to process graphics data; it may, for example, process general-purpose data.In some embodiments, the processor may comprise at least one integrated circuit or semiconductor die (e.g., be arranged on it). In some embodiments, the processor may comprise at least some hardware (e.g., transistors, circuit arrangements, non-volatile memory for storing instructions / control signals at the circuit level).

[0164] The processor comprises a single-instruction multi-threaded (SIMT) processor 1402. The SIMT processor represents a data processing section of the processor. The SIMT processor can execute instructions in a SIMT manner. The SIMT processor includes N processor elements (PES) 1404-1 to 1404-N. A wide variety of different numbers of processor elements can optionally be used. In some cases, there can be between hundreds and many thousands of processing elements, although the scope of protection of the invention is not limited to any arbitrary number of processing elements. The processor elements can represent hardware elements, hardware units, or circuit arrangements.Examples of suitable processor elements include, but are not limited to, arithmetic and logical units (ALUs), floating-point ALUs, floating-point units, integer units, tensor units, ray-tracing cores, texture units, and various combinations thereof. Certain GPUs available from Nvidia Corporation of Santa Clara, California, USA, refer to the SIMT processor as a streaming multiprocessor (SM) and refer to the processor elements as either streaming processors (PEs) or cores (e.g., CUDA cores (Compute Unified Device Architecture)). Certain GPUs available from Advanced Micro Devices (AMD), Inc. of Santa Clara, California, USA, refer to the SIMT processor as an arithmetic unit.

[0165] For the SIMT processor (e.g., a streaming multiprocessor (SM), a compute unit, etc.), each PE can execute instructions from one to many threads of a parallel thread group. Workgroups can be divided into parallel thread groups or hardware-schedulable groups of threads for the PEs (e.g., stream processors (SPs), CUDA cores, etc.). These hardware-schedulable groups can also be called wavefronts, warps, or parallel thread groups (e.g., groups of threads that are intended to run or be executed in parallel). For example, a wavefront or warp might contain 8, 16, 32, 64, or any other number of PEs, each intended to be executed by one to many corresponding threads. The number of such threads represents the width of the wavefront or warp.Traditionally, these threads in the wavefront or warp can execute the same instruction simultaneously (e.g., during the same clock cycle). There can also be one or more wavefronts. For example, there can be 8, 16, 32, 64, or any other number of wavefronts. Traditionally, these wavefronts can execute instructions sequentially (e.g., in sequential clock cycles).

[0166] Referring again to Fig. The processor also includes a shared bank memory 1412. The shared bank memory can be read by the PEs 1404 (e.g., while they execute read / load instructions), and data can be written to by the PEs 1404 (e.g., when they execute write / store instructions). For example, the shared bank memory can be used as addressable local memory to store data for the PEs. The shared bank memory is typically on-chip and has higher bandwidth and lower latency than local or global memory. The amount of shared bank memory can vary significantly from one implementation to another.For example, many modern GPUs use approximately several kilobytes to many hundreds of kilobytes of shared memory per PE, or optionally more if desired, although the scope of protection of the invention is not limited to any amount of shared bank memory.

[0167] The shared bank memory is bank-based, or a bank-based structure, in that it is subdivided or partitioned into multiple (e.g., often equally sized) parts called memory banks. The illustrated shared bank memory comprises M memory banks 1414-1 to 1414-M, where the number M can be 8, 16, 32, 64, or any other number. Each of the memory banks can be coupled so that they can be accessed simultaneously and / or in parallel (e.g., they can have an interface, memory controller, or the like). Each of the banks can have the same width in bits, such as 16 bits, 32 bits, 64 bits, or any other number of bits. Although the shared memory is bank-based in the illustration, other embodiments may additionally or alternatively include other types of memory (e.g.,Storage at other levels of the storage hierarchy), such as a cache, global storage, or storage other than a bank storage, may be implemented using the approaches disclosed herein.

[0168] The processor also includes a bank memory access circuit assembly 1408. The bank memory access circuit assembly is coupled to the bank memory and to the SIMT processor. During operation, the processor elements 1404-1 to 1404-N can perform memory access operations 1406 using memory addresses. The memory access operations can include memory load and / or read operations (e.g., due to load, read, collect, or other such instructions) and / or store and / or write operations in memory (e.g., due to store, write, scatter, or other such instructions). The bank memory access circuit assembly is operable to access the data 1415-1 to 1415-M in the shared bank memory using any of the memory addresses.

[0169] One reason for implementing the shared bank memory 1412 as the multiple memory banks 1414-1 through 1414-M is to help increase memory access bandwidth. If there are no bank conflicts, each of the memory banks can be accessed concurrently and / or in parallel, and each can provide a memory access bandwidth of 16 bits, 32 bits, 64 bits, or any other number of bits per clock cycle. For example, memory load and / or store operations involving thirty-two addresses mapped to thirty-two different memory banks can all be served concurrently and / or simultaneously, providing a total memory access bandwidth thirty-two times that of a single memory bank.In some cases, however, two or more addresses for load and / or store operations may be mapped to the same memory bank. This constitutes a bank conflict. When bank conflicts occur, accesses to the same memory bank must be performed serially rather than concurrently, since each memory bank can only handle one request at a time. For example, with bank conflicts, the processor can divide the memory load and / or store operations into several smaller sets of load and / or store operations. Each of these smaller sets of memory load and / or store operations can be free of bank conflicts. The multiple smaller sets of memory load and / or store operations can be performed serially or sequentially. Such bank conflicts tend to reduce memory access bandwidth, and it is desirable to minimize their number.

[0170] The way in which memory addresses are mapped to the memory banks of the shared bank memory can significantly affect the number of bank conflicts encountered and / or the maximum achievable memory access bandwidth. As used herein, the term "bank mapping" refers to the mapping or other correspondence between memory addresses and / or consecutive / sequential data element positions in system memory and the memory banks of the bank memory.

[0171] Traditionally, processors have used only a single bank mapping and / or a single bank mapping has been defined for the processors. One disadvantage of using only a single bank mapping is that, for at least some data types, it can tend to increase the number of encountered bank conflicts and / or reduce the achievable memory access bandwidth. For example, a single bank mapping designed to achieve relatively few bank conflicts and relatively high memory access bandwidth when used to access a first data type (e.g., a one-dimensional data array) may tend to cause relatively more bank conflicts and relatively lower memory access bandwidth when used to access a second, different data type (e.g., a multi-dimensional data array, paired or otherwise tupled data, tiered data, etc.).The number of bank conflicts can also vary for different accesses to the same data. For example, different passes of a Fast Fourier Transform (FFT) will exhibit different access patterns. Furthermore, in a matrix transposition operation, the number of bank conflicts in one direction (e.g., reading) can differ significantly from the number of bank conflicts in the other direction (e.g., writing). This is because one direction (any direction, as the order is not important) can be by row and the other by column. Additionally, the same type of operation (e.g., matrix transposition operations) can exhibit different bank conflicts when the dimensions change, as the relationship between data storage and the number of banks can change.

[0172] In some embodiments, the bank memory access circuit arrangement 1408 can be reconfigured at runtime to map memory addresses of the memory access operations 1406 from the processor elements to the memory banks 1414-1 to 1414-M in several different ways. In some embodiments, the processor can be operated to support the use of at least two, three, four, or more different bank mappings and / or different ways of mapping the addresses to the memory banks. In some embodiments, the processor can be operated to dynamically change the bank mapping and / or mapping used to map the memory addresses to the memory banks at runtime and / or after the boot time and / or after the configuration time of the processor.In some embodiments, the processor can be operated to dynamically change the bank map and / or mapping used to map memory addresses to the memory banks in response to and / or based on switching from processing a first data type to processing a second data type. For example, a bank map can be selected that is better suited to the structural characteristics of the second data type. In some embodiments, the processor can be operated to dynamically select any one of several (e.g., at least two, three, four, or optionally more) possible and / or supported and / or predetermined bank maps and / or ways of mapping memory addresses to the memory banks during runtime and / or after boot time and / or after the processor's configuration time.

[0173] Fig. Figure 15 is a block diagram of a detailed exemplary embodiment of a bank memory access circuit arrangement 1508, which is reconfigurable at runtime to map memory addresses to memory banks in different ways. In some embodiments, the bank memory access circuit arrangement 1508 can optionally be configured as the bank memory access circuit arrangement 1408. Fig. 14 are used, although this is not necessary. The one shown here for the bank storage access circuit arrangement is from Fig. The 14 described components, features and specific optional details can optionally also be used for the bank storage access circuit arrangement. Fig. 15 apply.

[0174] During runtime, the bank memory access circuit arrangement 1520 can reconfigure, switch, or otherwise change between a first bank mapping and / or another first way 1522 for mapping memory addresses to memory banks and a second bank mapping and / or another second way 1524 for mapping memory addresses to memory banks. In some embodiments, this change can be based on a runtime instruction 1526 to reconfigure or otherwise change the way in which memory addresses are mapped to memory banks. As explained below, in some embodiments, this runtime instruction 1526 can be based on an instruction. In other embodiments, this runtime instruction 1526 can be based on a change of one or more bits in a control and / or configuration register.

[0175] The first bank diagram and / or the first mode 1522 shows the layout of forty data elements (e.g., data elements "0" through "39") in a portion of the bank memory with eight memory banks (e.g., banks "0" through "7"). The forty data elements originate from consecutive and / or sequential data element positions in system memory and / or from consecutive and / or sequential memory addresses in system memory. Each of the data elements can have the same width in bits as a memory bank, such as 16 bits, 32 bits, 64 bits, or any other number of bits. The leftmost column lists conceptual addresses "0" through "4" that are used to access the data elements of five corresponding multiple data element positions in system memory. These conceptual addresses are shown for simplicity.It is understood that each of the conceptual addresses "0" to "4" can be expressed in 32 bits, 64 bits, or similarly. Each of the eight rightmost columns represents a different one of the eight memory banks (e.g., banks "0" to "7"). According to the first method 1522, successive and / or subsequent data elements are mapped to successive and / or subsequent banks modulo the number of such banks (e.g., eight in this exemplary embodiment). For example, data element 0 is stored at address 0 in bank 0, data element 1 is stored at address 0 in bank 1, data element 2 is stored at address 0 in bank 2, and so on. Since there are eight memory banks in this example, the index used to access a data element from the eight-bank memory according to the first bank mapping is based on the three (e.g.,8 = 2^3) the least significant bits 0, 1, and 2 (where bit 0 is the least significant bit) of the memory addresses used to access the data elements. In an alternative embodiment, if there were sixteen memory banks, the index used to access a data element from the sixteen-bank memory according to the first bank mapping would be based on the four (e.g., 16 = 2^4) least significant bits 0, 1, 2, and 3 of the memory addresses used to access the data elements. Likewise, the five least significant bits for thirty-two memory banks would be used, and so on.

[0176] The second bank mapping and / or the second method 1524 shows the layout of the data elements (it should be noted that some of the forty are not shown and some others are) in the portion of the bank memory containing the eight memory banks (e.g., banks "0" through "7"). The data element numbers (e.g., 0, 1, 2, 3, etc.) represent consecutive and / or sequential data element positions in system memory and / or consecutive and / or sequential memory addresses in system memory. Each of the data elements can have the same width in bits as a memory bank, such as, for example, 16 bits, 32 bits, 64 bits, or any other number of bits. According to the second method 1524, consecutive or adjacent pairs of data elements at consecutive memory locations can be mapped to the same bank.For example, for data elements 0 and 1, data element 0 is stored at address 0 in bank 0, and data element 1 is stored at address 1 in bank 0. Similarly, for data elements 2 and 3, data element 2 is stored at address 0 in bank 1, and data element 3 is stored at address 1 in bank 1, and so on. Since there are eight memory banks in this example, the index used to access a data element from the eight-bank memory according to the second bank mapping is based on bits 3, 2, and 1 of the read or write addresses used to access the data elements. These bits 3, 2, and 1 are offset by one bit position from the least significant bit 0.In an alternative embodiment, if there were sixteen memory banks, the index used to access a data element from the sixteen-bank memory would be based on bits 4, 3, 2, and 1 of the read or write addresses used to access the data elements. Similarly, for thirty-two memory banks, bits 5, 4, 3, 2, and 1 would be used.

[0177] The second bank mapping and / or the second way 1524 can allow access to certain data types with fewer bank conflicts and / or higher memory access bandwidth than would be achieved with the first bank mapping and / or the first way 1522. This can be the case, for example, for complex numbers and certain other types of paired numbers. Complex numbers have the form a + bi, where a is a real component and bi is an imaginary component. The a and b of the complex number are often stored as a pair of consecutive or adjacent data elements. For example, the second way 1524 can allow the real and imaginary components of a complex number to be mapped to the same bank (e.g., the data elements 0 and 1 to bank 0, as in Fig. 15). Conversely, if the first method 1522 were used, the real and imaginary components of the complex number would be mapped to the same bank (e.g., data element 0 mapped to bank 0 and data element 1 mapped to bank 1, as in Fig. (15 shown). The second method, 1524, may tend to offer fewer bank conflicts and / or higher memory access bandwidth when accessing such complex numbers. This may also be the case for certain other data types, such as matrix data to be transposed, data for fast Fourier transforms, etc.

[0178] In some embodiments, several different sets of data (e.g., with different structural characteristics) can be stored simultaneously in the memory bank, and these multiple sets of data can be accessed according to different respective methods for mapping memory addresses to memory banks. It is usually appropriate to use the same method for mapping memory addresses to memory banks when reading data from them as is used when writing data to them. However, in some cases, it may also be possible to write data and read data using different methods, or to access the same data using different methods.For example, if only the real components of complex numbers are accessed for processing, a first method may be more efficient, whereas if both the real and imaginary components are accessed, a second, different method may be more efficient.

[0179] Fig. Figure 16 is a block flow diagram of an embodiment of a method 1630 for accessing data from a bank memory. In some embodiments, the method 1630 can be executed by the processor and / or within the processor 1400 by Fig. 14. The components, features, and specific optional details described here for Processor 1400 also apply optionally to Process 1630. Alternatively, Process 1630 may be performed by and / or within a similar or different processor or device. Furthermore, Processor 1400 may perform procedures that are the same, similar, or different from Process 1630.

[0180] Block 1631 accesses data in a first set of one or more memory banks, a plurality of memory banks of a bank memory, with multiple memory addresses for a first plurality of memory access operations performed by a plurality of processor elements of a SIMT (Single Instruction, Multiple Thread) processor. In some embodiments, the first set of one or more memory banks is accessed based on a first way of mapping the multiple memory banks to memory addresses and / or based on a first value (e.g., a first address mask). Address masks are discussed in more detail below.

[0181] In block 1632, the multiple memory addresses are remapped from the first set of one or more memory banks to a second set of one or more memory banks of the multiple memory banks. In some embodiments, this may involve changing the first way of mapping the multiple memory banks to memory addresses to a second, different way of mapping the multiple memory banks to memory addresses, and / or changing the first value (e.g., the first address mask) to a second, distinct value (e.g., a second address mask). For example, in some embodiments, such a remap may involve dynamically storing the second distinct value (e.g., the second address mask) in a memory location that overwrites the first value (e.g., the first address mask) previously stored in the memory location during runtime, with the second distinct value (e.g., the second address mask) being the first value stored in the memory location.The second address mask is then applied to the multitude of memory addresses to map these addresses to the second set of one or more memory banks. As another example, in some embodiments, such remapping can involve dynamically selecting the second value (e.g., the second address mask) from several different values ​​(e.g., several different address masks) in a memory location during runtime, with the selected second value (e.g., the different address mask) then being applied to the multitude of memory addresses to map these addresses to the second set of one or more memory banks.

[0182] In some embodiments, remapping at block 1632 can optionally be performed in response to receiving a specific instruction (e.g., with a specific opcode) and / or based on receiving that specific instruction. Examples of suitable instructions are discussed below. In other embodiments, remapping at block 1632 can optionally be performed in response to changes to one or more bits in a control and / or configuration register. As one example, a value (e.g., an address mask) can be stored in the one or more bits. As another example, a value (e.g., an index) can be stored in the one or more bits where the index selects from several predefined values ​​(e.g., address masks).

[0183] Block 1633 accesses data in the second set of one or more memory banks with the plurality of memory addresses for a second plurality of memory access operations performed by the plurality of processor elements. In some embodiments, the second set of one or more memory banks is accessed based on the second way of mapping the multiple memory banks to memory addresses and / or based on the second value (e.g., the second address mask).

[0184] In some embodiments, different values ​​can be used to map the memory addresses to the memory banks in different ways. In some embodiments, these values ​​can be applied to and / or combined with the memory addresses to map the memory address in different ways. For example, the values ​​can be combined with the memory addresses using one or more logical operations (e.g., a logical AND operation, a logical OR operation, etc.). These values ​​can be stored in the memory of a bank memory access circuit arrangement or at least be accessible to a bank memory access circuit arrangement. In some embodiments, there can be a memory to store a value (e.g., an address mask) that is rewritable and / or reconfigurable at runtime, so that different values ​​(e.g.,Different address masks can be stored in memory during runtime to map memory addresses to memory banks in different ways. In some embodiments, the value can be specified or indicated by an instruction (e.g., in a field or as a direct operand of the instruction). In other embodiments, there can be a memory to store multiple different values ​​(e.g., multiple different predefined address masks), and different values ​​can be selected for use during runtime to map memory addresses to memory banks in different ways. In some embodiments, the values ​​(e.g., address masks) can optionally be loaded at configuration time (e.g., at boot time). In some embodiments, the values ​​(e.g.,Address masks) can optionally be loaded into memory from an external source that is not on the same chip and / or located in system memory.

[0185] Fig. Figure 17A is a block diagram illustrating three exemplary embodiments of address masks 1710. In various embodiments, such address masks can be specified via a direct operand or other field of an instruction, or can be implicit in the opcode of an instruction, or can be stored in a register or other memory location (e.g., a control and / or configuration register, one or more dedicated address mask registers, etc.). In this example, the total address space is 8 bits, and the mapping is 3 bits, or for 8 (e.g., 2^3) memory banks. The address space for each memory bank is 5 bits, or 32 (e.g., 2^5) memory locations. The mapping bits are indicated by a '1', so the remaining bits are used for addressing each individual memory bank. The top-level address mask 1770-1 can be used for the first way 1522 to map memory addresses to memory banks. Fig. 15 can be used. The middle address mask 1770-2 can be used for the second way 1524 to map memory addresses to memory banks from Fig. 15 can be used. The lowest address mask 1770-3 can be used for a third, still different way of mapping memory addresses to memory banks and illustrates that the mask bits (e.g. the bits set to one) within the address mask do not have to be contiguous.

[0186] Fig. Figure 17B is a block diagram illustrating an exemplary embodiment of how an address mask 1770 can be applied. A logic function 1772 transforms the address mask bits into an address portion and a select portion. In this example, the address portion is five bits [4:0] and the select portion is eight bits [7:0], although this is only an example. The select portion can be used for write releases and / or to control select multiplexes. The address portion is common to all memory banks 1714. The select portion is uniquely decoded for each memory bank. In some cases, the logic function can be combinational and can map the global address bits to the bank address bits and select bits in a single cycle. In other cases, the logic function can be iterative or sequential and can perform the mapping over several cycles.

[0187] Fig. Figure 18 is a block diagram of an embodiment of a processor 1800 that can be operated to perform an embodiment of a bank memory access reconfiguration instruction 1850. The processor 1800 can be the same as, similar to, or different from the processor 1400. Fig. 14. In some embodiments, the 1800 processor can be a GPU (e.g., a GPGPU). The GPGPU, GPU, or other SIMT processor can be either "hard" or "soft," as described previously.

[0188] The processor includes a shared bank memory 1812. The shared bank memory contains M memory banks 1814-1 to 1814-M, where the number M can be 8, 16, 32, 64, or any other number. The processor also includes a SIMT processor 1802. The SIMT processor contains N processor elements (PEs) 1804-1 to 1804-N. The processor elements can perform memory access operations 1806 using memory addresses. The processor also includes a bank memory access circuit assembly 1808, which is coupled to the bank memory and to the SIMT processor. The bank memory access circuit assembly is designed to access data in the bank memory using each of the memory addresses. In some embodiments, the bank memory access circuit assembly is reconfigurable at runtime to map the memory addresses to the memory banks in a variety of different ways.Unless otherwise specified, the SIMT processor, processor elements, shared bank memory, memory banks, and bank memory access circuitry may optionally be the same as or similar to the correspondingly named components from . Fig. 14 (e.g., exhibit one or more characteristics that are the same or similar). To prevent obfuscation of the description, the different and / or additional characteristics of the embodiment are derived from Fig. 18 primarily described, without repeating all the optional characteristics which may be the same as or similar to those for the embodiment from Fig. 14 can be described.

[0189] In the embodiment of Fig. In the 1852 embodiment, the processor also includes an instruction unit 1852. The instruction unit is coupled to the bank memory access circuitry 1808 and to the SIMT processor 1802. The instruction unit is sometimes also referred to as the front-end unit. The instruction unit or front-end unit can be operated to receive and process instructions (e.g., data processing instructions to be performed by the SIMT processor, load / read instructions, store / write instructions, etc.). The instruction unit or front-end unit can act as a control plane for the processor. In some embodiments, the instruction unit may include: a thread generator unit (e.g., circuitry) for initiating threads, an instruction fetch unit (e.g., circuitry) for retrieving instructions, an instruction decoder unit (e.g.,A circuit assembly coupled with the instruction retrieval unit decodes the instructions (e.g., decoding their bits and / or fields). An instruction scheduling unit (e.g., a circuit assembly coupled with the instruction decoding unit) schedules the instructions on one or more threads. An instruction allocation unit (e.g., a circuit assembly coupled with the instruction scheduling unit) allocates the instructions for execution on the one or more threads. These units / circuit assemblies can also be optionally combined in various ways (e.g., the scheduling unit and the allocation unit can be combined to form a sequencing unit, and so on).

[0190] The 1852 instruction unit can also receive and process the 1850 bank memory access reconfiguration instruction. In some embodiments, the bank memory access reconfiguration instruction can be a low-level instruction or control signal (e.g., binary microcode, a machine-level instruction, a binary instruction, etc.) that the processor can execute natively. In other embodiments, there can optionally be a corresponding higher-level instruction with the same or similar attributes. The processor, or a system containing the processor, can include logic (e.g., a compiler, an instruction translator, or other instruction converter) for compiling, translating, or otherwise converting the higher-level instruction into the lower-level 1850 bank memory access reconfiguration instruction that the processor can execute natively.What is described for the lower-level bank storage access reconfiguration instruction may optionally also apply to the higher-level instruction, since it may have similar aspects only in a different encoding or format.

[0191] In some embodiments, the bank memory access reconfiguration instruction can specify, cause, control, and / or configure the processor 1800 and / or the bank memory access circuit assembly 1808 to be reconfigured, switched, replaced, or otherwise modified 1810 in how the bank memory access circuit assembly maps memory addresses to memory banks of shared memory. For example, the bank memory access reconfiguration instruction can be used to change the bank mapping. The bank memory access reconfiguration instruction can allow the mapping of memory addresses to memory banks to be dynamically changed by software and / or a programmer at runtime (e.g., potentially on an instruction-by-instruction basis rather than only at boot or configuration time).In some embodiments, the processor and / or the bank memory access circuitry can switch to one of the many different ways of mapping memory addresses to the memory banks based on the bank memory access reconfiguration instruction. In some embodiments, this new mapping can apply to all current threads, so that the bank memory access reconfiguration instruction does not need to execute as many cycles as a regular thread instruction, but can instead be issued in a single clock cycle. In various embodiments, the bank memory access reconfiguration instruction can be one of those described below in conjunction with... Fig. 19, Fig. 20, Fig. 21A-B will be described.

[0192] Fig. Figure 19 is a block diagram of a first exemplary embodiment of a first bank memory access reconfiguration instruction 1950-1 and a second bank memory access reconfiguration instruction 1950-2. The first bank memory access reconfiguration instruction includes a first opcode 1956. The second bank memory access reconfiguration instruction includes a second, different opcode 1958. In some embodiments, the first and second bank memory access reconfiguration instructions may optionally consist only of the first and second opcodes without any other fields or encoding. Alternatively, the first and second bank memory access reconfiguration instructions may optionally include other fields for various purposes. The opcodes may represent a plurality of bits or one or more fields that cause the instruction and / or the operation to be performed to be identified.In some embodiments, the first opcode can be operated to cause and / or control a processor to switch to a first (e.g., predetermined) way of mapping memory addresses to memory banks of a bank memory. In some embodiments, the second opcode can be operated to cause a processor to switch to a second, different (e.g., predetermined) way (e.g., different from the first) of mapping memory addresses to the memory banks of the bank memory. The first and second different ways of mapping the memory addresses to the memory banks can be implicit or fixed for a first and second opcode, respectively. For example, after identifying the first opcode, the processor can understand that the first way should be used, and after identifying the second opcode, it can understand that the second way should be used.As a specific example, the first opcode can cause and / or control the processor to execute 1522 in the first way. Fig. 15 changes, and the second opcode can cause and / or control the processor to use the first method, 1522 from Fig. 15 changes. Only two instructions are shown in the illustration, although three, four or more such instructions, each with a different opcode, can optionally be used to switch to three, four or more different (e.g., predetermined) ways of mapping memory addresses to the memory banks of the bank memory.

[0193] Fig. Figure 20 is a block diagram of a second exemplary embodiment of a bank memory access reconfiguration instruction 2050. The bank memory access reconfiguration instruction includes an opcode 2060. In some embodiments, the bank memory access reconfiguration instruction may optionally consist only of the opcode without any other fields or encoding. Alternatively, the bank memory access reconfiguration instruction may optionally include other fields for various purposes. The opcode may represent a plurality of bits or one or more fields that cause the instruction and / or the operation to be performed to be identified. In some embodiments, the opcode may be operable to cause and / or control a processor to switch or toggle between two alternative (e.g., predetermined) ways of mapping memory addresses to memory banks of a bank memory.For example, there can only be two such ways, and the opcode can cause the processor to switch from the current way to the other way, making it the new current way. As a specific example, the bank memory access reconfiguration instruction can cause and / or control the processor to switch between the first way 1522 and the second way 1524. Fig. 15 switches or changes.

[0194] Fig. Figure 21A is a block diagram of a third exemplary embodiment of a bank memory access reconfiguration instruction 2150A. The bank memory access reconfiguration instruction includes an opcode 2162 and one or more bits or fields 2164 for storing, specifying, or otherwise indicating a flexible and / or mutable value to indicate a flexible and / or mutable way of mapping memory addresses to memory banks of a bank memory. The opcode may represent a plurality of bits or one or more fields that cause the instruction and / or the operation to be performed to be identified. In some embodiments, the opcode may be operable to cause and / or control a processor change to the flexible and / or mutable way of mapping the memory addresses to the memory banks, as specified by the flexible and / or mutable value.As a specific example, the one or more bits or fields 2164 can be a single bit that can have a first value (e.g., be cleared to binary zero) to specify a first way of mapping the memory addresses to the memory banks (e.g., the first way 1522 of . Fig. 15), or which may have a second, different value (e.g., be set to binary one) to indicate a second, different way of mapping the memory addresses to the memory banks (e.g., the second way 1524 of Fig. 15), and the opcode can be operated to cause and / or control the processor to switch to the mode indicated by the one or more bits or fields 2164. As another specific example, the one or more bits or fields 2164 can be two bits that can have any one of four distinct values ​​to indicate any one of four different predetermined modes for mapping the memory addresses to the memory banks, and the opcode can be operated to cause and / or control the processor to switch to any one of the four modes indicated by the two bits. In other embodiments, the one or more bits or fields 2164 can include more than two bits to select between more than four distinct modes.Thus, in various embodiments, there can be two, three, four, five, six, seven, eight, or more than eight different predetermined ways of mapping memory addresses to memory banks. The one or more bits or fields can have a sufficient number of bits to specify any of the supported number of different predetermined ways, and the opcode can cause the processor to change in the specified way. In some embodiments, the one or more bits or fields can be a direct operand, although this is not required. In some embodiments, the bank memory access reconfiguration instruction may optionally include no other fields or encoding besides the opcode and the one or more bits. Alternatively, the bank memory access reconfiguration instruction may optionally include other fields for various purposes.

[0195] Fig. Figure 21B is a block diagram of a fourth exemplary embodiment of a bank memory access reconfiguration instruction 2150B. The bank memory access reconfiguration instruction includes an opcode 2166 and bits or one or more fields 2168 for storing or specifying a flexible and / or mutable value (e.g., a flexible and / or mutable address mask) to be combined with and / or applied to memory addresses (e.g., by one or more logical operations) to map the memory addresses to memory banks of a bank memory in several different flexible and / or mutable ways. The opcode can represent a plurality of bits or one or more fields that cause the instruction and / or the operation to be performed to be identified.In some embodiments, the opcode can specify and / or be operable to cause and / or control a processor to apply the value stored or specified in the bits or fields 2168 (e.g., the address mask) to memory addresses in order to map the memory addresses to the memory banks. As a specific example, in the case of a bank memory with eight memory banks, the bits or one or more fields 2168 can store or specify a first binary value, with all more significant bits up to the address size set to zero and the four least significant bits being "0111" [3:0] to represent the first way 1522 of . Fig. to specify 15, or to store a second, different binary value, where all the more significant bits up to the address size are set to zero and the four least significant bits [3:0] are “1110”, to represent the second way 1524 of Fig. 15. These values ​​can, if desired, also be specified in their corresponding decimal or hexadecimal formats. These values ​​can be applied to and / or combined with (e.g., via logical AND) memory addresses to map the memory addresses to memory banks. Likewise, other values ​​(e.g., address masks) can be stored or specified for still different ways of mapping memory addresses to memory banks. In some embodiments, the bits or one or more fields may contain a direct operand, although this is not required. In an alternative embodiment, the instruction may instead use the bits or fields to specify a register or other memory location (e.g., a dedicated special register, a control and / or configuration register, or some other location) to store the value (e.g., the address mask).In yet another alternative embodiment, such a register or other memory location can optionally be implicit in the instruction and / or opcode, so that the processor can understand how to locate the value (e.g., the address mask) in the register or other memory location based on the instruction and / or opcode, thus eliminating the need for the instruction to include bits or fields to specify the register or other memory location. In some embodiments, the bank memory access reconfiguration instruction may optionally include no other fields or encoding besides the opcode and the one or more bits. Alternatively, the bank memory access reconfiguration instruction may optionally include other fields for various purposes.

[0196] Fig. 19, Fig. 20 and Fig. Figures 21A-B show examples of the types of fields that may be included in an embodiment of a bank memory access reconfiguration instruction. Alternative embodiments may have a subset of the fields shown, and / or they may add additional fields. The arrangement of fields shown is not required; rather, the fields may be rearranged in various ways. Furthermore, each of the fields may consist of either a contiguous set of bits, or it may have non-contiguous or separate bits that logically represent the field. As used herein, a field comprises one or more bits.It is also possible for analogous data types provided by the instructions to be provided in one or more control and / or configuration registers. For example, one or more bits or fields of the control and / or configuration registers may contain a value, an address mask, or the like, and one or more other bits of the control and / or configuration registers may cause the value to be assigned in the various ways described above. Fig. 19, Fig. 20 and Fig. 21A-B is used instead of the opcodes.

[0197] Against this background, it illustrates Fig. Figure 22 shows a block diagram of a system 2236 that can implement arithmetic operations using a programmable logic circuit arrangement, which may include digital signal processing or DSP blocks. A designer may wish to implement functionality, such as, among other things, graphics processing or general-purpose computing on a GPU, on an integrated circuit device 2201 (such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). In some cases, the designer may specify a high-level program to be implemented, such as an OpenCL program, which may allow the designer to provide programming instructions more efficiently and easily to configure a set of programmable logic cells for the IC device 2201 without specific knowledge of low-level hardware description languages ​​(such as Verilog or VHDL).Because OpenCL is similar to other high-level programming languages, such as C++, designers of programmable logic who are familiar with such programming languages ​​may, for example, have a reduced learning curve than designers who might otherwise have to learn unfamiliar low-level hardware description languages ​​to implement new functionalities in the Integrated Circuit Device 2201.

[0198] Developers can implement their high-level designs using design software 2237, such as a version of Intel® Quartus® from Intel Corporation. The design software can use a compiler 2238 to translate the high-level program into a lower-level description. The compiler can provide machine-readable instructions representing the high-level program to a host 2239 and the integrated circuit device 2201. The host can receive a host program 2240, which can be implemented by kernel programs 2241. To implement the host program, the host can transmit instructions from the host program to the integrated circuit device via a communication link 2242, which can be, for example, DMA (Direct Memory Access) or PCIe (Peripheral Component Interconnect Express) communication.In some embodiments, the kernel programs and the host can configure one or more DSP blocks 2243 on the IC device 2201. The DSP block can include a circuit arrangement, for example, for implementing operations to perform matrix-matrix or matrix-vector multiplication for artificial intelligence (AI) or non-AI data processing. The integrated circuit device can contain many (e.g., hundreds to thousands) of DSP blocks. Additionally, the DSP blocks can be communicatively coupled to one another, so that data output by one DSP block can be made available to other DSP blocks.

[0199] Although the techniques described in the above discussion involve the application of a high-level program, in some embodiments the designer may use the design software to generate and / or specify a low-level program, such as the low-level hardware description languages ​​described above. In some embodiments, the system may also be implemented without a separate host program. Furthermore, in some embodiments the techniques described here may be implemented in a circuit as a non-programmable circuit design. Thus, the embodiments described herein are intended to be illustrative and not limiting.

[0200] As part of a more detailed discussion of the integrated circuit device 2201, it is now illustrated Fig. Figure 23 shows an example of the integrated circuit device 2301 as a programmable logic device, such as a field-programmable gate array (FPGA). Furthermore, the integrated circuit device 2301 can be any other suitable type of integrated circuit device (e.g., an application-specific integrated circuit and / or an application-specific standard product). As shown, the integrated circuit device can include an input / output circuit arrangement 2345 for routing signals away from the device and for receiving signals from other devices via input / output pins 2346. Interconnect resources 2347, such as global and local vertical and horizontal conductive lines and buses, can be used for routing signals on the integrated circuit device.Additionally, the interconnect resources can include fixed interconnects (traces) and programmable interconnects (e.g., programmable connections between specific fixed interconnects). The programmable logic 2348 can include combinational and sequential logic circuits. For example, the programmable logic can include lookup tables, registers, and multiplexers. In various embodiments, the programmable logic can be configured to perform a user-defined logic function. The programmable interconnects associated with the interconnect resources can be considered part of the programmable logic.

[0201] Programmable logic devices, such as integrated circuit devices, can contain programmable elements 2349 within the programmable logic 2348. For example, as discussed above, a developer (e.g., a customer) can program (e.g., configure) the programmable logic to perform one or more desired functions. For example, some programmable logic devices can be programmed by configuring their programmable elements using mask programming arrangements, which is done during semiconductor manufacturing. Other programmable logic devices are configured after semiconductor manufacturing operations are completed, such as by using electrical programming or laser programming to program their programmable elements.In general, programmable elements can be based on any suitable programmable technology, such as fuses, antifuses, electrically programmable read-only memory technology, random access memory cells, mask-programmed elements, and the like, as well as combinations thereof.

[0202] Many programmable logic devices are programmed electrically. In electrical programming arrangements, the programmable elements can be formed from one or more memory cells. During programming, configuration data, for example, is loaded into the memory cells using pins 2346 and the input / output circuit arrangement 2345. In one embodiment, the memory cells can be implemented as random-access memory (RAM) cells. The use of RAM-based memory cells described here is only an example. Because these RAM cells are loaded with configuration data during programming, they are also sometimes referred to as configuration RAM (CRAM) cells. Each of these memory cells can provide a corresponding static control output signal that controls the state of an associated logic component in the programmable logic 2348.In some embodiments, the output signals can be applied, for example, to the gates of metal-oxide-semiconductor (MOS) transistors within the programmable logic.

[0203] Taking the foregoing into account, the DSP block 2243, together with the programmable logic 2348, can be used to implement a soft logic GPU, also referred to herein as the soft GPU. The soft logic GPU or soft GPU can have any of the features described elsewhere herein (e.g., have a SIMT architecture, have multiple streaming processors (SPs) per streaming multiprocessor (SM), utilize the virtual multi-port aspects disclosed herein, and so on). The control plane (including instruction fetching, decoding and sequencing, and thread initialization circuitry) for the SMs can optionally be logically separated from the processing plane (including the SMs), so that it may not be necessary to return any data or signaling to the control plane. This can optionally allow the control signals and immediate data buses on their way to the SM to be pipelined.The SMs can contain the majority of the memory as well as the DSP blocks and can optionally be physically or logically placed in a sector for deterministic performance. The features of floating-point 32 (FP32) DSP blocks can be used to increase the efficiency of matrix operations. In contrast to the SM, the control plane can tend to have less logic and may tend to use more random logic (e.g., the SM can be designed as a highly structured design), which can help close timings at similar performance levels to the SM without significant compilation limitations.

[0204] In some embodiments, the soft GPU can optionally include a special function unit (SFU) to provide additional specialized and potentially complex functionality, such as elementary functions. The ability to incorporate such an SFU is an advantage of the FPGA or other PLA design. For example, the SFU can provide a specific function, such as an inverse square root function, several trigonometric operations, and so on.

[0205] Modern GPUs often run at a frequency of around 1 GHz, with an overclock of approximately 1.4 GHz. FPGA soft logic is often slower than ASIC logic, so the soft GPU may have a frequency of less than 1 GHz. In some implementations, the soft GPU can run at a high frequency for an FPGA (e.g., optionally up to approximately 1 GHz).

[0206] In some cases, the soft GPU may exhibit reduced memory capabilities compared to a hard GPU (e.g., particularly during the write-back phase to shared memory). In some instances, true dual-port memory (i.e., two read ports and two write ports) may be optionally supported by the soft GPU. In other cases, since multi-port memory tends to be expensive, it may be emulated (e.g., using an internal multicycle operation) instead of being supported by a dedicated hardware solution. Such emulation may reduce the maximum memory frequency in this mode. Different memory architectures (e.g., number of read and write ports and memory size) may be used according to the trade-offs deemed appropriate for the implementation.It is understood that, although the soft GPU may have one or more reduced performance attributes compared to a hard GPU in some implementations, the use of the soft GPU may be useful for other reasons (e.g. flexibility, adaptability, etc.).

[0207] In addition to the embodiments described above, other embodiments relate to an exemplary embodiment of a soft GPU, which is discussed below. The exemplary embodiment of the soft GPU discussed below can optionally use any of the embodiments discussed above (e.g., those that are suitable for Fig. 14-21). The embodiments discussed above (e.g., those which are for Fig. However, the embodiments described in sections 14-21 are certainly not limited to the exemplary embodiment of the soft GPU discussed below. Instead, the embodiments discussed above (e.g., those described for Fig. 14-21 are described) each being implemented in any of the other SIMT processors or GPUs discussed elsewhere herein, including hard GPUs.

[0208] Fig. Figure 24 represents an exemplary embodiment of a Soft GPU 2450. The illustrated Soft GPU includes eight SPs (SP1 to SP7) 2404, although in other embodiments it may include fewer or more (e.g., sixteen SPs). The Soft GPU also includes a shared memory 2405. A circuit arrangement for data flow to and from the SMs is also shown as various lines and multiplexers. In this example, the shared memory is configured with four read ports and one write port, implemented with four physical memories (shown as four rectangles within the shared memory) using a simple dual-port configuration (one read port and one write port). Such a configuration is directly supported by certain FPGAs. Other embodiments may use either fewer or more ports and less or more physical memory.Each SP has two outputs, one of which functions as an address port and the other as a 32-bit data port. Also shown are an address read multiplexer (mux) 2451, an address write multiplexer 2452, and a data write multiplexer 2453. The address ports are multiplexed by the address read multiplexer to provide four address read ports per clock cycle. For example, in a 16-SP soft GPU, a read instruction might take four cycles per wavefront. The four data ports from the shared memory can be distributed among the SPs. For a 16-SP soft GPU, this might involve four parallel 32-bit paths, each fanned out to the 16 SPs over four clock cycles. Since there is only one write port to the shared memory, both the address and data buses can be multiplexed 16 to 1 for a 16-SP soft GPU.

[0209] Fig. Figure 25 represents an exemplary embodiment of a SP 2504. The SP includes a dual-port thread register file 2554, implemented using two memories in simple dual-port mode. The SP includes a floating-point arithmetic logic unit (FP ALU) 2555 and an integer arithmetic logic unit (INT ALU) 2556. Two 32-bit data buses provide two operands (aa and bb) to the ALUs. Typically, depending on the application, most of the processing is performed by the FP ALU. The INT ALU can be used for address generation as well as data processing, since it has access to the entire register set for all threads. Control signals (immediate, shared, thread ID, fp_op, int_op) from the (not shown) instruction section of the soft GPU can be delayed so that they are aligned with the data information (read_aa, read_bb) to be written to the registers of the register file.For example, the delay could be a two-clock latency for direct operand data from the same instruction, a six-clock delay for a shared memory read operation, or a seven-clock delay when writing back from an ALU operation (e.g., fp_op, int_op). These are just examples of one implementation. In some embodiments, the FP ALU can be fully implemented in an FPGA digital signal processor (DSP) block. For example, in Intel Agilex DSP blocks, the configured mode can be set at compile time so that the two operations (FP multiplication and FP addition / subtraction) can be supported by the FP multiplication / addition configuration.In some embodiments, the FP addition / subtraction operation can optionally be implemented by multiplying an aa operand by an FP value of "1,0" (as shown, a multiplexer can select this "1,0" value) and then adding a bb operand. In some cases, the loop latency for an ALU operation can be about eight cycles. In this example, the thread registers read can be two cycles (one input and one output clock cycle). In the illustrated SP, there is a layer of registers 2557 (including the multiplexer for the FP selection "1,0" for the FP addition / subtraction operation described earlier) between the thread register stores and the ALUs, and a register after the selection multiplexer 2558 between the FP and the INT ALUs. In the illustrated SP, there is a different multiplexer and a register selection 2559 between the write-back path and the data bus from outside the SP.The write enable signals to the thread registers can be delayed in the same way, so that they are aligned with the write data.

[0210] Fig. Figures 26-28 together illustrate an embodiment of a SIMT processor (e.g., a streaming multiprocessor (SM)). In particular, Figure 26-28 illustrates Fig. 26 An exemplary embodiment of a shared memory block 2605 for the SM and how it is connected with receiving inputs, outputs, and signals. With reference to Fig. The four address read buses, which are multiplexed by the 16 SP cores, can be viewed as inputs to the shared memory block, and the single address write and data write bus as outputs of the shared memory block. Address read and data read each have a fanout of four, with four of the SP blocks being written per clock cycle. An example distribution pattern is shown in the illustration. Fig. Figure 27 illustrates an embodiment of the PLC for the SM and how they are connected to receive inputs, outputs, and signals. In this embodiment, the SM includes 16 PLCs, although other embodiments may optionally use fewer or more PLCs (e.g., 8, 32, 64, etc.). With reference to Fig. 27. The write input connections to the SP blocks mirror the output of the shared memory block. The outputs of the SP block feed the address read, address write, and data write multiplexers back into the shared memory block. An immediate offset value from the direct operand field in an instruction word can be added to all address values. As in Fig. Figure 26 shows write releases for the four data buses output from the shared memory, depicted alongside the shared memory, even though they are not part of the shared memory block, to illustrate the relationship between the data and write releases. The write releases are delayed within their respective SP targets to align with the arrival of the data. Fig. Figure 29 illustrates another exemplary embodiment of a shared memory block 2905 and how it is associated with receiving inputs, outputs, and signals. Four simple dual-port memories are arranged in parallel to create a memory with a single write port and four read ports. The shared memory can be accessed either by the SM (which comprises the sixteen individual SPs) or by an external agent. There is only a single read port from the outside of the soft GPU, although four could be provided with minimal additional resources if needed. The four output ports, which are routed to the rest of the SM, each have a fanout of four, as shown above in Figure 2905. Fig. 27 is shown. Fig. Figure 28 illustrates an embodiment of the output part of the SM and how it is connected with receiving inputs, outputs and signals.

[0211] The instruction unit of the soft GPU may include an instruction fetch unit for determining the next instruction memory address. In some cases, sophisticated logic may be used to perform such a determination, since many instructions run for many cycles, although some can be modified on an instruction-by-instruction basis to execute a different number of cycles or just a single cycle. Zero-overhead loops, subroutines, and simple branches can also affect address generation. A relatively wide (e.g., 40-bit) instruction word is defined. The program length is relatively short for this type of soft GPU and its intended uses, and 40 bits is a directly supported width in certain commercially available FPGAs, making this a reasonable implementation choice.The sequencer can track the number of cycles per operation and ensure that the correct wavefront is accessed. Each thread register space in the SPs can be initialized with a thread identifier (ID) that can be used to identify it (and, for example, multiple dimensions can optionally be supported). Individual thread IDs are typically used for address generation. In some embodiments, the ISA can include one or more instructions to load thread IDs generated by the thread generator and load them into the appropriate thread register space. In some embodiments, certain simplifications can optionally be made to the instruction unit to help increase speed. For example, simplifications can be made to branching support.For example, an assumed branch will potentially invalidate the following two statements. Thus, two NOPs can optionally be introduced after a branch statement, regardless of whether the branch statement is taken or not. This can include subroutine jumps and feedback loops, unconditional branches, and zero-overhead loops.

[0212] Fig. Figure 30 illustrates an exemplary embodiment of a sequencer and how it can be connected to receive inputs, outputs, and signals. In one embodiment, the sequencer can include multiple (e.g., four) free-running counters (e.g., labeled as circles with +1 inside) that can be synchronously reset to zero until their respective instruction or combination of instructions is issued. For example, a load counter (the counter on the far left) can be initialized by a load instruction (e.g., a multiple cycle reading from shared memory into the thread registers) and can be executed until all active threads have been loaded. In an embodiment with 16 SPs and shared memory reads ported quadruple, a four-phase control sequence can be issued to control the address read multiplexer, which is located in Fig.Figure 27 shows that a "save-instruction" memory sequence is more complex because it can address anywhere from a single thread to a subset of threads across a wavefront, a subset of threads across a subset of all threads, a subset of wavefronts, or all threads. Two counters (the middle two counters) are used to track the two-dimensional matrix of participating threads (e.g., based on the width of the wavefront and the number of wavefronts). In a case where the write access to shared memory is ported individually, a memory operation (e.g., a write to shared memory) can be performed in sixteen cycles per wavefront. The sequencer can also generate the thread read and write addresses, controlled by the operation counter (right).

[0213] The critical path of this architecture is in the instruction fetch section, with several paths returning approximately the same performance. The instruction section, which includes instruction fetch, instruction decoding, the wavefront sequencer, and the thread ID generator, is relatively small. The instruction memory also forms part of this section. An example instruction memory might be a 1K x 40-bit memory implemented in two M20Ks. The instruction memory can be reloaded with a new program from outside the soft GPU. There may be one or more relatively long combinational paths in this section, most of which are feedback to the instruction fetch section. For example, one might be the immediate branch value from the instruction memory to the program counter.This can be placed in a pipeline, although it would increase the branching penalty from two to three, making some programs less efficient. Another critical path can be the calculation of the signal indicating that the current instruction has completed and the program counter can be incremented. Such a calculation or signal can be based on various possible conditions, such as whether the instruction is a single cycle or multiple cycles, if the wavefront is complete (for example, multiple dynamic partial wavefront controls may be possible in some embodiments), or if the load or store operations are complete (for example, there may be multiple partial-run options in some embodiments).

[0214] In some cases, the control plane (including instruction retrieval, decoding and sequencing, and thread initialization circuitry) can be logically separated from the processing plane for the SIMTs. In some cases, no data or signaling can be passed from the processing plane to the control plane. In some cases, no data-dependent branches may be made, only loop-dependent decisions, all contained within the instruction section. There are no data-dependent operations in the SIMT processor that affect the instruction unit. There may be some data-dependent decisions in the SIMT processor, but no decision information is fed back to the instruction unit (e.g., instruction retrieval or sequencer).

[0215] This can enable various levels of pipelining between the instruction unit and the SIMT processor. For example, control signals and direct operand data buses can be pipelined on their way to the SM. This will ultimately allow the SM to be planned or placed relatively independently of the instruction part, making it easier to close the timing gap even in large, complex system designs. Furthermore, the development of its adaptation characteristics and placement work can be relatively independent. Because the instruction kernel is relatively small, it should exhibit similar placement and performance characteristics across a wide variety of environments.Because the structure of these two sections tends to differ—the instruction set section has relatively more random logic, and the SIMT processor has relatively more data paths—we can more easily close the timing gap on systems using the soft GPU, either through an automatically placed design or by chaining together two carefully planned components. The SM contains the majority of the memory and all DSP blocks and can be physically or logically placed in a sector for deterministic performance.

[0216] For the soft GPU, most of the functional logic is implemented in embedded FPGA features, such as the Intel M20K FPGAs. Some of the integer ALU may be constructed in soft logic, but much of the remaining logic in the SM can consist primarily of multiplexers and registers, which are typically directly and efficiently supported by FPGAs. The soft GPU can be compiled, for example, using Quartus 20.3 Prime in a Stratix 10-1SG280LN2F43E1VG device. In Stratix 10, this design can optionally have a clock frequency of approximately 500 MHz.

[0217] Components, features, and details described for any of the GPUs or other processors disclosed herein may optionally also be applied to any of the methods disclosed herein, which in embodiments may optionally be executed by and / or with such GPUs or processors. Any of the GPUs or other processors described herein in embodiments may optionally be included in any of the systems disclosed herein. Any of the instructions disclosed herein may optionally be executed by any of the GPUs or other processors disclosed herein.

[0218] References to "example," "an example," etc., indicate that the described example may include a particular feature, structure, or characteristic, but not every example necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example. Moreover, when a particular feature, structure, or property is described in connection with an example, it is implied that it is common knowledge among experts to obtain such a feature, structure, or property in connection with other examples, whether or not it is explicitly described.

[0219] GPUs and their components disclosed herein may be described as operable, serviceable, ready, capable, configured, adapted, or otherwise "for" performing one or more operations. As used herein, these terms refer to the characteristics, properties, or attributes of the GPU or its components when in a powered-off state and do not imply that the GPU or its components are currently operating or powered on. For clarity, it is understood that the GPUs and their components, as claimed herein, are not powered on or running.

[0220] In the description and claims, the terms "coupled" and "connected," along with their derivatives, may be used. These terms are not to be considered synonymous. Instead, "connected" may be used in embodiments to indicate that two or more elements are in direct physical and / or electrical contact with each other. "Coupled" may mean that two or more elements are in direct physical and / or electrical contact with each other. However, "coupled" may also mean that two or more elements are not in direct contact with each other but nevertheless work together or interact. Arrows are used in the figures to show connections and couplings.

[0221] Some embodiments include a manufactured article (e.g., a computer program product) that incorporates a machine-readable medium. The medium may include a mechanism that provides (e.g., stores) information in a form that is machine-readable. The machine-readable medium may provide (or have stored) an instruction or a sequence of instructions that, if and / or when executed by a machine, cause the machine to perform one or more operations, procedures, or techniques disclosed herein.

[0222] In some embodiments, the machine-readable medium may be a tangible and / or non-volatile machine-readable storage medium. For example, the non-volatile machine-readable storage medium may include a floppy disk, an optical storage medium, an optical disk, an optical data storage device, a CD-ROM, a magnetic disk, a magneto-optical disk, a read-only memory (ROM), a programmable ROM (PROM), an erasable and programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), flash memory, phase-change memory, phase-change data storage material, non-volatile memory, a non-volatile data storage device, or the like.The non-volatile, machine-readable storage medium does not consist of a transiently propagated signal. In some embodiments, the storage medium may comprise a tangible medium, including solid objects or materials such as, for example, a semiconductor material, a phase-change material, a magnetic solid, a solid data storage material, etc. Alternatively, a non-tangible, volatile, computer-readable transmission medium, such as, for example, electrical, optical, acoustic, or another form of propagated signal—such as carrier waves, infrared signals, and digital signals—may optionally be used.

[0223] Examples of suitable machines include, but are not limited to, GPUs, GPGPUs, FPGAs, digital logic circuits, integrated circuits, computer systems, and electronic devices. Examples of suitable computer systems and electronic devices include, but are not limited to, desktop computers, laptop computers, tablet computers, smartphones, servers, set-top boxes, video game controllers, and the like.

[0224] Furthermore, in the various examples described above, unless explicitly stated otherwise, disjunctive language such as the phrase "at least one of A, B or C" or "A, B and / or C" is to be understood as either A, B or C or any combination thereof (i.e. A and B, A and C, B and C as well as A, B and C).

[0225] When introducing elements of different embodiments of the present disclosure, the articles "a", "an", and "the" are to be understood as meaning that there is one or more of the elements. The terms "comprising", "including", and "having" are to be understood as inclusive and mean that there may be additional elements other than those listed. Furthermore, references to "an embodiment" of the present disclosure are not to be interpreted as excluding the existence of additional embodiments that also incorporate the aforementioned features.

[0226] While the embodiments set forth in this disclosure may take various modifications and alternative forms, certain embodiments are illustrated by way of example in the drawings and are described in detail herein. It should be understood, however, that the disclosure is not limited to the specific forms disclosed. The disclosure is intended to cover all modifications, equivalents, and alternatives that fall within the scope and protection of the disclosure, as defined by the following appended claims.

[0227] The methods presented and claimed herein are related to and applied to material objects and concrete examples of a practical nature that demonstrably improve the present technical field and are therefore not abstract, intangible, or purely theoretical. Furthermore, if claims appended to the end of this description contain one or more elements described as "means for [performing] [a function]..." or "steps for [performing] [a function]...", these elements shall be interpreted in accordance with 35 USC 112(f). However, for all claims containing elements described in any other way, these elements shall not be interpreted in accordance with 35 USC 112(f).

[0228] Specific details have been set forth in the above description to provide a comprehensive understanding of the embodiments. However, other embodiments can be implemented without some of these specific details. Various modifications and changes can be made to them without deviating from the broader nature and scope of protection of the disclosure as set forth in the claims. The description and drawings are accordingly to be regarded as illustrative and not limiting. The scope of protection of the invention is not to be determined by the specific examples provided above, but only by the claims set forth below. In other cases, circuits, structures, devices, and operations already known have been shown in block diagram form and / or without details to prevent obscuring the understanding of the description. EXAMPLE EXECUTION FORMS

[0229] The following examples refer to further embodiments. The features in the examples can be used in one or more embodiments.

[0230] Example 1 is a GPU, or other processor or device, that includes a bank memory. The bank memory has multiple memory banks. The processor also includes a single-instruction multi-threaded (SIMT) processor. The SIMT processor includes multiple processor elements. These multiple processor elements are used to perform memory access operations using memory addresses. The processor also includes a bank memory access circuitry that is coupled to the bank memory and to the SIMT processor. The bank memory access circuitry is intended to access data in the bank memory using the memory addresses. The bank memory access circuitry is reconfigurable at runtime to map the memory addresses to the multiple memory banks in a variety of different ways.

[0231] Example 2 comprises the process from Example 1, and further includes a memory for storing a value that is reconfigurable during runtime. The bank memory access circuitry is coupled to the memory. The bank memory access circuitry is intended to use the value to map the memory addresses to the multiple memory banks in one of several different ways.

[0232] Example 3 comprises the process from Example 1, and further includes a data storage for storing different values. The bank memory access circuitry is coupled to the storage. The bank memory access circuitry serves to select different values ​​during runtime and to use the selected values ​​to map the memory addresses to the multiple memory banks in several different ways.

[0233] Example 4 includes the processor from one of Examples 1 to 3, wherein the bank memory access circuit arrangement serves to apply different address masks to the memory addresses in order to map the memory addresses to the multiple memory banks in several different ways.

[0234] Example 5 comprises the processor according to one of Examples 1 to 4, and further comprises an instruction unit coupled to the bank memory access circuitry and coupled to the SIMT processor. The instruction unit serves to receive an instruction. Based on the instruction, the bank memory access circuitry is to switch to one of several different ways of mapping the memory addresses to the multiple memory banks.

[0235] Example 6 includes the processor from Example 5, where the instruction has an opcode to specify one of several different ways.

[0236] Example 7 includes the processor from Example 5, where the instruction has an opcode to indicate that it switches between the several different ways.

[0237] Example 8 includes the processor from Example 5, wherein the instruction has an opcode and one or more bits, wherein the one or more bits specify a value, the value being to indicate one of several different ways.

[0238] Example 9 includes the processor from Example 5, wherein the instruction has an opcode and several bits, where the several bits specify a value to be applied to the memory addresses in order to map the memory addresses to the multiple memory banks in one of several different ways.

[0239] Example 10 includes the processor from Example 9, where the value is an address mask.

[0240] Example 11 comprises the processor from one of Examples 1 through 4, and further comprises a control and / or configuration register coupled to the bank memory access circuitry. The control and / or configuration register has one or more bit positions to specify one of several different ways. Optionally, the bank memory access circuitry also serves to access the one or more bit positions to determine the way.

[0241] Example 12 includes the processor from one of Examples 1 to 11, optionally including shared memory in the bank memory and optionally including a general-purpose graphics processing unit (GPGPU).

[0242] Example 13 includes the processor from one of Examples 1 to 11, wherein the processor is optionally a field-programmable gate array (FPGA), and wherein the SIMT processor is optionally a soft SIMT processor.

[0243] Example 14 is a method that involves accessing data in a first set of memory banks of a multi-address bank memory by means of several memory access operations performed by multiple processor elements of a single-instruction multi-threaded (SIMT) processor. The method also involves remapping the multiple memory addresses from the first set of memory banks to a second set of memory banks. The method further involves accessing data in the second set of memory banks with the same multiple memory addresses by means of a second set of memory access operations performed by the SIMT processor.

[0244] Example 15 includes the procedure from Example 14, where a remapping is performed in response to an instruction.

[0245] Example 16 comprises the procedure from Example 14, wherein accessing the data in the first set of memory banks with the multiple memory addresses involves applying a first value to the memory addresses, and wherein accessing the data in the second set of memory banks with the multiple memory addresses involves applying a second value to the memory addresses.

[0246] Example 17 comprises the procedure from one of Examples 14 to 16, wherein accessing the data in the set of memory banks involves accessing a first type of component of tupled data. Optionally, accessing the data in the second set of memory banks also involves accessing both the first type of component and a second type of component of the tupled data.

[0247] Example 18 comprises the procedure from any one of Examples 14 to 16, wherein the data accessed by the first set of memory banks is a first record, and further comprises writing the first record to the first set of memory banks using a first way of mapping memory addresses to memory banks. Optionally, the data accessed by the second set of memory banks is also a second record, and optionally further comprises writing the second record to the second set of memory banks using a second, different way of mapping memory addresses to memory banks.

[0248] Example 19 is a machine-readable storage medium that stores instructions which, when executed, cause a machine to perform operations. These operations include accessing data in a first set of memory banks of a multi-memory bank memory by performing multiple memory access operations by multiple processor elements of a single-instruction multi-threaded (SIMT) processor. The operations also include remapping the multiple memory addresses from the first set of memory banks to a second set of memory banks. Finally, the operations include accessing data in the second set of memory banks by performing multiple memory access operations by the SIMT processor.

[0249] Example 20 comprises the machine-readable storage medium from Example 19, wherein the instructions for remapping the multiple memory addresses from the first set of memory banks to the second set of memory banks include instructions which, if executed, cause the machine to either select a different value from several values ​​stored in a memory location or to store a value of an instruction in a memory location.

[0250] Example 21 includes the machine-readable storage medium according to one of Examples 19 to 20, wherein accessing the data in the first set of memory banks with the multiple memory addresses involves applying a first value to the memory addresses. Optionally, accessing the data in the second set of memory banks with the multiple memory addresses also involves applying a second value to the memory addresses.

[0251] Example 22 is a GPU or other processor or other device operable to perform the method according to any of Examples 14 to 18.

[0252] Example 23 is a GPU or other processor or other device comprising means for carrying out the method according to any one of Examples 14 to 18.

[0253] Example 24 is a GPU or other processor or other device comprising any combination of modules and / or units and / or logic and / or circuitry and / or means capable of performing the method of any one of Examples 14 to 18.

Claims

[1] Processor, including: a bank storage, wherein the bank storage has multiple storage banks; a single-instruction multi-threaded or SIMT processor, wherein the SIMT processor comprises multiple processor elements, the multiple processor elements being intended to perform memory access operations with memory addresses; and a bank storage access circuit arrangement coupled to the bank storage and coupled to the SIMT processor, wherein the bank memory access circuit arrangement is intended to access data in the bank memory using the memory addresses, wherein the bank memory access circuit arrangement is reconfigurable during runtime to map the memory addresses to the multiple memory banks in several different ways. [2] Processor according to claim 1, further comprising a storage for storing a value that is reconfigurable during runtime, wherein the bank memory access circuit arrangement is coupled to the storage and wherein the bank memory access circuit arrangement is to use the value to map the memory addresses to the multiple memory banks in one of several different ways. [3] Processor according to one of claims 1 to 2, further comprising a storage for storing different values, wherein the bank memory access circuit arrangement is coupled to the storage, and wherein the bank memory access circuit arrangement serves to: Selecting different values ​​during runtime; and Using the selected different values ​​to map the memory addresses to the multiple memory banks in several different ways. [4] Processor according to one of claims 1 to 3, wherein the bank memory access circuit arrangement serves to apply different address masks to the memory addresses in order to map the memory addresses to the multiple memory banks in several different ways. [5] Processor according to one of claims 1 to 4, further comprising an instruction unit coupled to the bank memory access circuit arrangement and coupled to the SIMT processor, wherein the instruction unit is to receive an instruction, wherein the bank memory access circuit arrangement is to switch to one of the several different ways of mapping the memory addresses to the multiple memory banks based on the instruction. [6] Processor according to claim 5, wherein the instruction includes an opcode to specify one of several different ways. [7] Processor according to claim 5, wherein the instruction includes an opcode to specify to switch between the several different ways. [8] Processor according to claim 5 or claim 7, wherein the instruction comprises an opcode and one or more bits, wherein the one or more bits specify a value, the value indicating one of several different ways. [9] Processor according to claim 5 or claim 7, wherein the instruction comprises an opcode and several bits, the several bits specifying a value to be applied to the memory addresses in order to map the memory addresses to the multiple memory banks in one of several different ways. [10] Method according to claim 9, wherein the value is an address mask. [11] Processor according to any one of claims 1 to 4, further comprising a control and / or configuration register coupled to the bank memory access circuit arrangement, wherein the control and / or configuration register has one or more bit positions to specify one of several different ways, and wherein the bank memory access circuit arrangement is to access the one or more bit positions to determine the way. [12] Processor according to any one of claims 1 to 11, wherein the bank memory comprises shared memory and wherein the processor is a universal graphics processing unit (GPU). [13] Processor according to any one of claims 1 to 12, wherein the processor is a field-programmable gate array (FPGA) and wherein the SIMT processor is a soft SIMT processor. [14] Method comprising the following: Accessing data in a first set of memory banks of a bank memory with multiple memory addresses by first multiple memory access operations performed by multiple processor elements of a single-instruction multi-thread or SIMT processor; Remapping the multiple memory addresses from the first set of memory banks to a second set of memory banks; and Accessing data in the second set of memory banks with the multiple memory addresses of the second set of multiple memory access operations performed by the SIMT processor. [15] Method according to claim 14, wherein accessing the data in the first set of memory banks with the multiple memory addresses comprises applying a first value to the memory addresses and wherein accessing the data in the second set of memory banks with the multiple memory addresses comprises applying a second value to the memory addresses. [16] Method according to one of claims 14 to 15, wherein accessing the data in the first set of memory banks comprises accessing a first type of components of tupled data and wherein accessing the data in the second set of memory banks comprises accessing both the first type of components and a second type of components of the tupled data. [17] Method according to any one of claims 14 to 16, wherein the data accessed by the first set of memory banks is a first data set, and further comprising writing the first data set to the first set of memory banks using a first way of mapping memory addresses to memory banks, wherein the data accessed by the second set of memory banks is a second data set, and further comprising writing the second data set to the second set of memory banks using a second, different way of mapping memory addresses to memory banks. [18] Method according to any one of claims 14 to 17, wherein the re-imaging is carried out in response to an instruction. [19] Method according to claim 18, wherein the instruction comprises an opcode and one or more bits, wherein the one or more bits specify a value, wherein the remapping is based on the value. [20] Method according to claim 18, wherein the instruction comprises an opcode and multiple bits, the multiple bits specifying a value which, when applied to the multiple memory addresses, performs the remapping of the multiple memory addresses from the first set of memory banks to the second set of memory banks. [21] Method according to any one of claims 14 to 18, wherein the re-imaging comprises: Reconfiguring a value in memory during runtime; and Using the value to remap the multiple memory addresses from the first set of memory banks to the second set of memory banks. [22] Method according to any one of claims 14 to 18, wherein the re-imaging comprises: Selecting a different value from multiple values ​​in a memory location during runtime; and Using the selected distinct value to remap the multiple memory addresses from the first set of memory banks to the second set of memory banks. [23] Device comprising means for carrying out a method according to any one of claims 14 to 22. [24] Machine-readable memory comprising machine-readable instructions which, when executed, cause a computer to implement a method according to any one of claims 14 to 22. [25] Computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to execute the method according to any one of claims 14 to 22.

Images