A chip architecture with configurable shared and private storage

Abstract

The application relates to the chip design technical field, in particular to a chip architecture with shareable storage and private storage. The chip architecture comprises: a plurality of physical storage arrays, each of which comprises a plurality of storage units, each of which can be configured as a storage unit in a shared storage area or a storage unit in a private storage area; a shared storage address conversion module for realizing conversion of a shared storage address into a physical storage address; a register address conversion module for realizing conversion of a register address into a physical storage address; and a judgment module, first and second input ends of the judgment module being connected with the shared storage address conversion module and the register address conversion module respectively, the judgment module being used for judging whether a received physical storage address is sent to a physical storage array according to the configuration of a storage unit corresponding to the received physical storage address. The application can realize flexible configuration of shared storage and private storage in a chip.

Description

Technical Field

[0001] This invention relates to the field of chip design technology, and in particular to a chip architecture that can be configured with shared and private memory. Background Technology

[0002] A distinctive feature of GPUs is shared memory (SM). Threads in different warps can access the same shared memory address, enabling data sharing. However, in current technologies, data in shared memory typically needs to be loaded into registers before it can participate in computation, and the computation results also need to be written back to shared memory via registers, which impacts chip performance. Furthermore, in current technologies, the shared memory and registers in each stream processor of a GPU are pre-configured, but the needs for shared memory and registers vary depending on the scenario. For example, LLM-type tasks require large-scale matrix operations, resulting in a greater demand for shared memory; while convolution tasks require less. This pre-configured approach also fails to fully utilize the chip's storage space, affecting processing performance. Summary of the Invention

[0003] The purpose of this invention is to provide a chip architecture that allows for the configuration of shared and private storage, thereby enabling flexible configuration of shared and private storage within the chip and improving the chip's processing performance.

[0004] According to the present invention, a chip architecture configurable with shared and private storage is provided, the chip architecture including a plurality of stream processors, each stream processor including: Several physical storage arrays, each physical storage array including several storage units, each storage unit can be configured as a storage unit in a shared storage area or a storage unit in a private storage area; when a storage unit is configured as a storage unit in a shared storage area, the storage unit can be accessed by several threads; when a storage unit is configured as a storage unit in a private storage area, the storage unit can only be accessed by a single thread.

[0005] A shared storage address translation module is used to translate a shared storage address into a physical storage address, wherein the shared storage address is the address in the shared storage access instruction.

[0006] A register address translation module is used to translate register addresses into physical storage addresses, where the register address is the address in the register access instruction.

[0007] The judgment module has its first and second input terminals connected to the shared memory address translation module and the register address translation module, respectively. The judgment module is used to determine whether to send the received physical memory address to the physical memory array based on the configuration of the memory cell corresponding to the received physical memory address.

[0008] Compared with the prior art, the present invention has at least the following beneficial effects: In this invention, the chip architecture includes a chip unit in each stream processor and each physical memory array, which is configurable. The chip unit can be configured as a storage unit in a shared memory region, where the shared memory region can be used as shared memory in the stream processor; or it can be configured as a storage unit in a private memory region, where the private memory region can be used as a register in the stream processor. Furthermore, each stream processor in this invention's chip architecture also includes a judgment module. This module receives the physical memory address output by the shared memory address translation module and the register address translation module, and determines whether to send the received physical memory address to the physical memory array based on the configuration of the storage unit corresponding to the physical memory address (i.e., whether it is configured as a storage unit in a shared memory region or a storage unit in a private memory region). Therefore, this invention can realize a chip architecture with configurable shared and private memory, which can be applied to different application scenarios through different configurations, thereby improving the chip's processing performance. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of a stream processor in a chip architecture that can be configured with shared and private storage according to an embodiment of the present invention; Figure 2 A schematic diagram showing the locations of the logical arithmetic unit and the physical storage array provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating a merging method of logical arithmetic units provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating another method of merging logical arithmetic units provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0013] According to this embodiment, a chip architecture configurable with shared and private storage is provided. The chip architecture includes a plurality of stream processors, each stream processor including, for example... Figure 1 As shown: Several physical storage arrays, each comprising several storage units, each storage unit can be configured as a storage unit in a shared storage area or a storage unit in a private storage area; when a storage unit is configured as a storage unit in a shared storage area, that storage unit can be accessed by several threads; when a storage unit is configured as a storage unit in a private storage area, that storage unit can only be accessed by a single thread. It should be understood that... Figure 1 The SRAM array in the text is the physical storage array. SRAM is an abbreviation for Static Random Access Memory. The storage bank includes not only the SRAM array but also the SRAM index and the SRAM write / read circuit. The SRAM index is used to locate specific storage cells in the SRAM array. The SRAM index and the SRAM write / read circuit are existing structures and will not be described in detail here.

[0014] In one specific implementation, the ratio of the shared storage area to the private storage area is configurable and optional. This ratio is configured according to different task loads. For example, in Large Language Modeling (LLM) tasks, the proportion of the shared storage area is increased; in convolution computation tasks, the proportion of the shared storage area is decreased. The configuration of each storage unit—whether it is set as a storage unit in the shared storage area or a storage unit in the private storage area—is predetermined based on the task load. In this embodiment, for a storage unit, regardless of whether it is configured as a storage unit in the shared storage area or the private storage area, the connection relationship of the storage unit within the storage remains unchanged; the different configurations alter the judgment result of the subsequent judgment module.

[0015] It should be understood that all storage cells in a physical storage array can be configured as storage cells in a shared storage region, or all can be configured as storage cells in a private storage region; alternatively, a portion of the storage cells in a physical storage array can be configured as storage cells in a shared storage region, and another portion of the storage cells in a physical storage array can be configured as storage cells in a private storage region. For example, a physical storage array includes 128 storage cells. All 128 storage cells can be configured as storage cells in a shared storage region; all 128 storage cells can be configured as storage cells in a private storage region; or the 0th to xth storage cells can be configured as storage cells in a shared storage region, and the (x-127th)th storage cell can be configured as storage cells in a private storage region. <x<127。

[0016] A shared storage address translation module is provided, which is used to translate a shared storage address into a physical storage address, wherein the shared storage address is the address in the shared storage access instruction. Those skilled in the art will recognize that the process of translating a shared storage address into a physical storage address is prior art and will not be described further here.

[0017] A register address translation module is provided, which is used to translate register addresses into physical memory addresses, where the register address is the address in a register access instruction. Those skilled in the art will recognize that the process of translating register addresses into physical memory addresses is prior art and will not be elaborated upon here.

[0018] The judgment module has its first and second input terminals connected to the shared memory address translation module and the register address translation module, respectively. The judgment module is used to determine whether to send the received physical memory address to the physical memory array based on the configuration of the memory cell corresponding to the received physical memory address.

[0019] In this embodiment, the determination module can obtain configuration information indicating whether each storage unit is configured as a unit in a shared storage area or a unit in a private storage area. If the ratio of shared to private storage areas changes, this configuration information will also change. As a specific implementation, determining whether to send the received physical storage address to the physical storage array based on the configuration of the storage unit corresponding to the received physical storage address includes: If the physical storage address is received through the first input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in the shared storage area, then it is determined that the received physical storage address will be sent to the physical storage array.

[0020] If the physical storage address is received through the first input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in a private storage area, then it is determined that the received physical storage address will not be sent to the physical storage array.

[0021] If the physical storage address is received through the second input terminal, and the storage cell corresponding to the physical storage address is configured as a storage cell in the shared storage area, then it is determined that the received physical storage address will not be sent to the physical storage array.

[0022] If the physical storage address is received through the second input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in a private storage area, then it is determined that the received physical storage address will be sent to the physical storage array.

[0023] Based on the above judgment logic, when a shared memory access instruction is received and the memory unit corresponding to the address in the instruction is also configured as a memory unit in the shared memory area, access to the shared memory area can be realized. Compared with the prior art where shared memory must go through registers, the access to shared memory in this embodiment is more efficient. When a register access instruction is received and the memory unit corresponding to the address in the instruction is also configured as a memory unit in the private memory area, access to the private memory area can be realized, realizing the function of accessing registers in the prior art.

[0024] As a specific implementation, if each stream processor includes a physical memory array of at least two, the determining module is further configured to: parse the physical memory address to obtain the index bit of the physical memory array and the internal address of the physical memory array, and send the internal address to the physical memory array corresponding to the index bit. Figure 1 As shown, a dashed box represents a memory bank, and each memory bank includes an SRAM array. When the number of dashed boxes is greater than or equal to 2, it indicates that a stream processor includes more than or equal to 2 physical memory arrays. In this embodiment, different memory banks in a stream processor have the same structure. The judgment module parses the physical memory address to obtain the index bit and internal address of the physical memory array. The index bit of the physical memory array is used to determine which memory bank's physical memory array is being accessed, and the internal address of the physical memory array is used to determine which memory cells in the physical memory array are being accessed. Those skilled in the art will know that how to parse the physical memory address, how to send the internal address to the physical memory array corresponding to the index bit, and how to write or read data within the physical memory array are existing technologies, and will not be described in detail here.

[0025] In this invention, the memory cells included in each stream processor and each physical memory array of the chip architecture are configurable. They can be configured as memory cells in a shared memory region, which can be used as shared memory in the stream processor; or they can be configured as memory cells in a private memory region, which can be used as registers in the stream processor. Furthermore, each stream processor in the chip architecture of this invention also includes a judgment module. This module receives the physical memory address output by the shared memory address translation module and the register address translation module, and determines whether to send the received physical memory address to the physical memory array based on the configuration of the memory cell corresponding to the physical memory address (i.e., whether it is configured as a memory cell in a shared memory region or a memory cell in a private memory region). Therefore, this invention can realize a chip architecture with configurable shared and private memory, which can be applied to different application scenarios through different configurations, thereby improving the chip's processing performance.

[0026] As a specific implementation, each stream processor also includes several arithmetic logic units (ALUs). In this embodiment, the shared memory and registers of the prior art are combined into one, so that the physical memory is the same, only logically separated into shared memory areas and private memory areas. To adapt to changes in storage location, the position of the arithmetic logic units in the stream processor is optimized as follows: the several arithmetic logic units are arranged around the physical memory array, such as... Figure 2 The diagram shows (only the logical arithmetic units are shown positioned below the physical memory array). In one specific embodiment, the distance between the logical arithmetic units and the physical memory array is at the nanometer level; compared to the micrometer level distance between the logical arithmetic units and the physical memory array in the prior art, the chip architecture based on this embodiment can reduce the distance between the logical arithmetic units and the physical memory array to the nanometer level, achieving near-memory computing and improving performance.

[0027] In one specific implementation, a target number of logical arithmetic units constitute a channel, and the logical arithmetic units in a channel execute the same instruction; the target number is less than or equal to the number of logical arithmetic units included in a single stream processor. In another specific implementation, the target number is configured according to different task loads.

[0028] As a specific implementation, when the total number of logical arithmetic units in a stream processor is 16, it can achieve 2 channels, 4 channels, 8 channels, and 16 channels. For example... Figure 3 As shown, if there are 4 channels, then 16 logic arithmetic units can execute 4 different instructions. Figure 4As shown, if there are 16 channels, then 16 logic arithmetic units can execute one instruction; it should be understood that, through... Figure 4 The channel selection switch controls the number of logic arithmetic units (LAUs) in a single channel. If the left line is selected, the 16 LAUs execute the single instruction input from the leftmost instruction control module, while the other three instruction control modules remain inactive (shown as dashed lines). If the right line is selected, the 16 LAUs can execute four different instructions. It should be understood that with two channels, the 16 LAUs can execute eight different instructions, requiring eight instruction control modules. To switch between 2, 4, 8, and 16 channels, eight instruction control modules are needed, connected in the same way as... Figure 4 Similarly, this will not be elaborated upon here.

[0029] This embodiment allows for adjustment of the number of ALU channels controlled by each stream processor, adapting to different task loads. In practical applications, the number of channels can be selected based on experience to suit the task load; for example, a larger number of channels can be used when the data is highly regular, while a smaller number of channels can be used when the data changes frequently. Alternatively, the number of channels corresponding to the optimal performance can be selected through experimentation.

[0030] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A chip architecture configurable with shared and private storage, characterized in that, The chip architecture includes several stream processors, each stream processor comprising: Several physical storage arrays, each physical storage array including several storage units, each storage unit can be configured as a storage unit in a shared storage area or a storage unit in a private storage area; when a storage unit is configured as a storage unit in a shared storage area, the storage unit can be accessed by several threads; when a storage unit is configured as a storage unit in a private storage area, the storage unit can only be accessed by a single thread. A shared storage address translation module is used to translate a shared storage address into a physical storage address, wherein the shared storage address is the address in the shared storage access instruction. A register address translation module is used to translate register addresses into physical storage addresses, where the register address is the address in a register access instruction. The judgment module has its first and second input terminals connected to the shared memory address translation module and the register address translation module, respectively. The judgment module is used to determine whether to send the received physical memory address to the physical memory array based on the configuration of the memory cell corresponding to the received physical memory address.

2. The chip architecture configurable with shared and private storage according to claim 1, characterized in that, Determining whether to send the received physical storage address to the physical storage array based on the configuration of the storage unit corresponding to the received physical storage address includes: If the physical storage address is received through the first input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in the shared storage area, then it is determined that the received physical storage address will be sent to the physical storage array. If the physical storage address is received through the first input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in a private storage area, then it is determined that the received physical storage address will not be sent to the physical storage array. If the physical storage address is received through the second input terminal, and the storage cell corresponding to the physical storage address is configured as a storage cell in the shared storage area, then it is determined that the received physical storage address will not be sent to the physical storage array. If the physical storage address is received through the second input terminal, and the storage unit corresponding to the physical storage address is configured as a storage unit in a private storage area, then it is determined that the received physical storage address will be sent to the physical storage array.

3. The chip architecture configurable with shared and private storage according to claim 1, characterized in that, If each stream processor includes a number of physical storage arrays greater than or equal to 2, the determination module is further configured to: parse the physical storage address, obtain the index bit of the physical storage array and the internal address of the physical storage array, and send the internal address to the physical storage array corresponding to the index bit.

4. The chip architecture configurable with shared and private storage according to claim 1, characterized in that, The ratio between the shared storage area and the private storage area is configurable.

5. The chip architecture configurable with shared and private storage according to claim 4, characterized in that, The ratio of shared storage area to private storage area is configured according to different task loads.

6. The chip architecture configurable with shared and private storage according to claim 1, characterized in that, Each stream processor also includes several logical arithmetic units arranged around the physical memory array.

7. The chip architecture configurable with shared and private storage according to claim 6, characterized in that, The distance between the plurality of logical arithmetic units and the physical storage array is at the nanometer level.

8. The chip architecture configurable with shared and private storage according to claim 6, characterized in that, The plurality of logical arithmetic units constitute a channel consisting of a target number of logical arithmetic units, and the logical arithmetic units in a channel execute the same instruction; the target number is less than or equal to the number of logical arithmetic units included in a single stream processor.

9. The chip architecture configurable with shared and private storage according to claim 8, characterized in that, The target number is configured according to the different task loads.

Images