A chip architecture with optional cacheless mode

Abstract

The application relates to the chip design technical field, in particular to a chip architecture with optional cache mode. The chip architecture comprises a cache mode selection switch, which is used for controlling the cache in the chip to execute a cache working mode or a non-cache working mode; when the cache working mode is executed, cache control logic is executed; when the non-cache working mode is executed, the cache does not execute the cache control logic, and uses the body memory in the cache as a memory with a unified coding address, comprising: determining a destination address in a received instruction as the unified coding address, judging whether the destination address is a coding address corresponding to the body memory in the cache, if yes, writing data in a storage position corresponding to the destination address or reading data from the storage position corresponding to the destination address according to the operation type of the instruction; otherwise, transmitting the received instruction to a lower level. The application can solve the problem of low ALU utilization rate of a cache-based processor when the data flow is complex.

Description

Technical Field

[0001] This invention relates to the field of chip design technology, and in particular to a chip architecture with an optional cacheless mode. Background Technology

[0002] Classic processor designs typically employ caching techniques to alleviate memory access bottlenecks, which has proven effective and efficient in the vast majority of programs. For example, in general-purpose graphics processing units (GPGPUs), programmers only need to focus on the algorithm implementation. When running on hardware, thanks to Single Instruction, Multiple Threads (SIMT) technology and L1 / L2 caching, the ALU utilization rate can almost always reach over 80%. This makes GPGPUs the preferred choice for parallel computing hardware. However, with the advent of Large Language Models (LLMs) and Transformer architectures, the data flow for matrix operations has become increasingly complex, leading to a significant decrease in cache hit rates. When a cache miss occurs, the ALU must wait for data to be loaded from lower-level memory (such as DRAM), causing computational pauses and reducing ALU utilization. Summary of the Invention

[0003] The purpose of this invention is to provide a chip architecture with an optional cacheless mode to solve the problem of low ALU utilization in cache-based processors under complex data flow conditions.

[0004] According to the present invention, a chip architecture with an optional cacheless mode is provided, the chip architecture comprising: The cache mode selection switch is used to control whether the cache in the chip operates in cache mode or non-cache mode.

[0005] When the cache is in cache working mode, the cache control logic is executed.

[0006] When the cache operates in non-cached mode, it does not execute the cache control logic within the cache and uses the cache's own memory as a memory with a unified coded address. Using the cache's own memory as a memory with a unified coded address includes: determining the destination address in the instruction received by the cache as a unified coded address, determining whether the destination address is the coded address corresponding to the cache's own memory, and if so, writing data to or reading data from the storage location corresponding to the destination address according to the operation type of the instruction; otherwise, transmitting the received instruction to the next level.

[0007] Compared with the prior art, the present invention has at least the following beneficial effects: In this invention, the chip architecture includes a cache mode selection switch, which controls whether the cache in the chip operates in cache mode or non-cache mode. When the cache operates in cache mode, it executes cache control logic. When the cache operates in non-cache mode, it does not execute cache control logic and uses the cache's physical memory as a memory with a unified coded address. Because the physical memory in the cache has a unified coded address, the uncertainty problem inherent in caching technology is eliminated. The physical memory in the cache is transformed into fast memory. Programmers can decide when to store what data in which location of the physical memory in the cache. For example, programmers can pre-store the required data in the physical memory (i.e., fast memory) in the cache to ensure that the ALU can quickly access the data during computation, eliminating the stuttering caused by cache misses and improving ALU utilization. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of the structure of a cache in the prior art provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of a cache with a cache working mode and a non-cache working mode provided in an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0012] According to this embodiment, a chip architecture with an optional no-cache mode is provided. The chip architecture includes a cache mode selection switch, which is used to control whether the cache in the chip performs a cache working mode or a non-cache working mode.

[0013] Those skilled in the art will understand that existing chip architectures include caches, such as the L1 cache and L2 cache in a GPU. The differences between the chip architecture in this embodiment and existing chip architectures include: the cache in existing chip architectures can only execute cache working mode, while the chip architecture in this embodiment also includes a cache mode selection switch. The cache has two working modes (only one of the two working modes can be executed at the same time). The mode selection switch can control which working mode the cache executes. When the cache executes in cache working mode, its working logic is the cache control logic executed in existing chip architectures. When the cache executes in non-cache working mode, the cache control logic in the cache is no longer executed, and the physical circuits in the cache used to implement the cache control logic are bypassed. The cache in non-cache working mode is just a uniformly coded and addressable memory.

[0014] In one specific implementation, the cache mode selection switch selects whether the first or second path is activated based on a control command or configuration signal; when the first path is activated, the cache operates in cache mode; when the second path is activated, the cache operates in non-cache mode; the first path includes physical circuitry for implementing cache control logic. As one specific implementation, a schematic diagram of the cache structure with cache mode and non-cache mode is shown below. Figure 2 As shown. Compared to Figure 1 The cache structure in the middle, Figure 2 The structure also includes a selection switch (i.e., a cache mode selection switch) and pathways for the physical circuitry used to bypass cache control logic. For example... Figure 2As shown, the dashed box represents SRAM (Static Random Access Memory), the thin black line outside the dashed box that is parallel to the thick black line is part of the first path, the thick black line outside the dashed box is part of the second path, and the area inside the dashed box is the common part of the first and second paths. The selector switch is used to enable the first path or the second path.

[0015] In this embodiment, the cache mode selection switch selecting the first or second path based on control instructions or configuration signals means that the cache mode selection switch can be controlled to select the first or second path either through instruction control or through configuration. Optionally, the programmer can modify the control instructions for the cache mode selection switch; for example, the `cache on` instruction indicates selecting cache mode, and the `cache off` instruction indicates selecting non-cache mode. Alternatively, the CPU can send control instructions to the GPU via the PCIe bus, and these instructions can include configuration signals for the cache mode selection switch. Alternatively, a mapping relationship between different programs and instructions controlling the cache mode selection switch can be pre-established, thereby automatically generating the correct control instructions for the cache mode selection switch before a specific program runs.

[0016] In one specific implementation, the cache mode selection switch is a multiplexer (mux).

[0017] When the cache operates in cache mode, it executes cache control logic. Those skilled in the art will understand that this cache control logic is prior art. For example, the cache control logic includes: determining the destination address in the received instruction as the main memory address; decomposing the main memory address to obtain a tag and an offset; if the tag matches the tag of a cache line, retrieving data from the matching cache line based on the offset and returning it; otherwise, initiating a missing data handling process. Those skilled in the art will understand the content of the cache control logic, and it will not be elaborated here.

[0018] When the cache operates in non-cached mode, it does not execute the cache control logic within the cache and uses the cache's own memory as a memory with a unified coded address. Using the cache's own memory as a memory with a unified coded address includes: determining the destination address in the instruction received by the cache as a unified coded address, determining whether the destination address is the coded address corresponding to the cache's own memory, and if so, writing data to or reading data from the storage location corresponding to the destination address according to the operation type of the instruction; otherwise, transmitting the received instruction to the next level.

[0019] In one specific implementation, the cached intrinsic memory is SRAM. When the cached intrinsic memory is used as a memory with a unified coded address, the programmer can read data from any storage location in the cached intrinsic memory using the load instruction, and write data to any storage location in the cached intrinsic memory using the store instruction. Those skilled in the art will understand how to access storage locations in SRAM, and will not elaborate further here.

[0020] In one specific implementation, the chip architecture includes a secondary cache and several primary caches. Each secondary cache and each primary cache corresponds to a different cache mode selection switch. Each cache mode selection switch controls whether the cache corresponding to that switch operates in cache mode or non-cache mode. Alternatively, the caches in the chip architecture share cache mode selection switches, and all caches in the chip uniformly operate in cache mode or non-cache mode; or some caches in the chip architecture share cache mode selection switches, and other caches share other cache mode selection switches. Thus, it is possible to select some caches in the chip to uniformly operate in cache mode or non-cache mode. It should be understood that this embodiment does not limit the number of caches and cache mode selection switches in the chip architecture.

[0021] In this embodiment, the unified encoding address of the cache's intrinsic memory is obtained by uniformly encoding the cache's intrinsic memory and other memories of the chip. For example, the chip includes a Level 1 cache and a Level 2 cache, both of which operate in non-cached mode. Uniform encoding of the cache's intrinsic memory and other memories of the chip refers to uniformly encoding the following memories: registers, shared memory (SM) in the chip, intrinsic memory in the Level 1 cache (L1 cache), intrinsic memory in the Level 2 cache (L2 cache), and main memory (DRAM). It should be understood that in the prior art, caches can only operate in cache mode, and programmers cannot perform deterministic access to storage locations in the intrinsic memory of the Level 1 and Level 2 caches; programmers can only perform deterministic access to storage locations in shared memory and main memory. However, in this embodiment, through unified encoding, programmers can perform deterministic access to any location in all of the above memories when the cache is in non-cached mode. Furthermore, since the cache is closer to the registers than the main memory, accessing the cache's intrinsic memory is more efficient than accessing the main memory.

[0022] In one specific implementation, the unified encoded address includes a high-order part and a low-order part. The high-order part is used to distinguish different memories within the chip, and the low-order part is used to distinguish different storage locations within a single memory. Therefore, by comparing the high-order part, it can be determined whether the destination address in the received instruction matches the address of its own included storage location. If it does, data is read from the destination address or stored at the destination address; otherwise, the instruction can be passed down to the next lower level.

[0023] As a specific implementation, using the cached ontology memory as a memory with a unified coded address further includes: dividing the coded address corresponding to the cached ontology memory into a first part and a second part, wherein the first part and the second part are alternately used for loading data or for computation by the processor in the chip. It should be understood that the alternation of the first part and the second part for loading data or for computation by the processor in the chip means that if the first part is used for data loading, then the second part is used for computation by the processor in the chip; if the first part of data loading is complete, the tasks of the two can be swapped, i.e., the first part is used for computation by the processor in the chip, and the second part is used for data loading. Therefore, data loading and computation can be performed simultaneously, reducing the impact of the data loading process on efficiency.

[0024] As a specific implementation, using the cached ontology memory as a memory with a unified coded address further includes: the compiler treating the cached ontology memory as an allocatable register class and allocating data based on the bandwidth and latency corresponding to the cached ontology memory. It should be understood that in the prior art, the compiler's perspective only includes registers and the main controller, and the caching behavior between them is transparent; however, in this embodiment, when the cache operates in non-cached mode, the cached ontology memory can be considered a type of register, and the bandwidth and latency corresponding to the cached ontology registers are deterministic. Once the registers obtain the bandwidth and latency corresponding to the cached ontology registers, they can make a more optimal choice in register allocation.

[0025] In this embodiment, the chip architecture includes a cache mode selection switch, which controls whether the cache in the chip operates in cache mode or non-cache mode. When the cache operates in cache mode, it executes cache control logic. When the cache operates in non-cache mode, it does not execute cache control logic and uses the cache's physical memory as a memory with a unified coded address. Since the physical memory in the cache has a unified coded address, the uncertainty problem inherent in caching technology is eliminated. The physical memory in the cache is transformed into fast memory. Programmers can decide when to store what data in which location of the physical memory in the cache. For example, programmers can pre-store the required data in the physical memory (i.e., fast memory) in the cache to ensure that the ALU can quickly access the data during computation, eliminating the stuttering caused by cache misses and improving ALU utilization.

[0026] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A chip architecture with an optional cacheless mode, characterized in that, The chip architecture includes: The cache mode selection switch is used to control whether the cache in the chip operates in cache mode or non-cache mode. When the cache is in cache working mode, the cache control logic is executed. When the cache operates in non-cached mode, it does not execute the cache control logic within the cache and uses the cache's own memory as a memory with a unified coded address. Using the cache's own memory as a memory with a unified coded address includes: determining the destination address in the instruction received by the cache as a unified coded address, determining whether the destination address is the coded address corresponding to the cache's own memory, and if so, writing data to or reading data from the storage location corresponding to the destination address according to the operation type of the instruction; otherwise, transmitting the received instruction to the next level.

2. The chip architecture with optional cacheless mode according to claim 1, characterized in that, The cache mode selection switch selects whether the first path or the second path is activated according to a control command or configuration signal; when the first path is activated, the cache operates in cache mode; when the second path is activated, the cache operates in non-cache mode; the first path includes physical circuitry for implementing cache control logic.

3. The chip architecture with optional cacheless mode according to claim 1, characterized in that, Using the cached ontology memory as a memory with a unified coded address also includes: dividing the coded address corresponding to the cached ontology memory into a first part and a second part, wherein the first part and the second part are used alternately for loading data or for the processor in the chip to perform calculations.

4. The chip architecture with optional cacheless mode according to claim 1, characterized in that, Using the cached ontology memory as memory with a uniformly encoded address also includes: the compiler treating the cached ontology memory as an allocatable register class and allocating data according to the bandwidth and latency corresponding to the cached ontology memory.

5. The chip architecture with optional cacheless mode according to claim 1, characterized in that, The chip architecture includes a level 2 cache and several level 1 caches. Each level 2 cache and each level 1 cache corresponds to a different cache mode selection switch. Each cache mode selection switch is used to control the cache corresponding to that cache mode to operate in cache mode or non-cache mode.

6. The chip architecture with optional cacheless mode according to claim 5, characterized in that, The unified encoding address is obtained by uniformly encoding the target memory in the chip. The target memory includes the following memory: registers, shared memory, the main memory in the first-level cache, the main memory in the second-level cache, and main memory.

7. The chip architecture with optional cacheless mode according to claim 1, characterized in that, The unified coded address includes a high-order part and a low-order part. The high-order part is used to distinguish different memories in the chip, and the low-order part is used to distinguish different storage locations in a memory.

8. The chip architecture with optional cacheless mode according to claim 1, characterized in that, The cache control logic includes: determining the destination address in the received instruction as the main memory address, decomposing the main memory address to obtain the tag and offset; if the tag matches the tag of a cache line, retrieving the data from the matching cache line according to the offset and returning it; otherwise, initiating the missing data handling process.

9. The chip architecture with optional cacheless mode according to claim 2, characterized in that, The cache mode selection switch is a multiplexer.

Images