Memory access method and graphics processor

Abstract

The embodiment of the invention provides a memory access method and device, electronic equipment and a readable storage medium. The method comprises the following steps: receiving a cache line access request; extracting label information from a target memory address corresponding to the cache line access request, wherein the label information comprises a shared label, a private label, a group index, inter-block offset and intra-block offset; querying whether the cache line access request hits the L1 cache or not based on the label information; the group index is used for determining a target group, the shared tag and the private tag are used for determining a target cache line in the target group, and the inter-block offset is used for determining a target cache block in the target cache line; and if the cache line access request hits the L1 cache, determining a target data position in a hit target cache block based on the in-block offset, and executing a corresponding memory access request based on the target data position. According to the embodiment of the invention, on the basis of ensuring the data access efficiency, the storage space of the cache line is fully utilized, and the waste of the cache space is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a memory access method and a graphics processor. Background Art

[0002] The L1 cache (abbreviated as L1 cache) is the fastest and closest level of cache in a GPU (Graphics Processing Unit). Its primary function is to store data and instructions that the GPU is likely to access frequently in the near future, reducing the number of times the GPU accesses main memory, thereby increasing data access speed and improving overall GPU performance.

[0003] The computing mode of the GPU is usually SIMT (Single Instruction, Multiple Threads), that is, multiple threads execute the same instruction in parallel. In parallel computing, multiple threads may access adjacent data at the same time. The L1 cache usually uses larger cache lines (such as 64Byte or 128Byte). Larger cache lines can prefetch more adjacent data at a time. When a thread accesses certain data, the data adjacent to it can also be loaded into the cache line together. In this way, when other threads need to access these adjacent data, they can obtain them directly from the L1 cache without having to access the main memory again, which greatly improves data access efficiency and reduces memory access latency, thereby better supporting concurrent access to data by multiple threads in parallel computing.

[0004] However, in practical parallel computing, data is not always stored tightly and continuously. Sometimes data is sparse, meaning that there may be large gaps or invalid data between adjacent data. When using larger cache lines, valid data may only occupy a small portion of the cache line, resulting in insufficient cache line space utilization and wasted cache space. Summary of the Invention

[0005] In view of the above problems, an embodiment of the present invention is proposed to provide a memory access method that overcomes the above problems or at least partially solves the above problems, which can fully utilize the storage space of cache lines and reduce the waste of cache space while ensuring data access efficiency.

[0006] Correspondingly, an embodiment of the present invention further provides a graphics processor to ensure the implementation and application of the above method.

[0007] In a first aspect, an embodiment of the present invention discloses a memory access method applied to a GPU, wherein an L1 cache of the GPU is divided into a plurality of groups, each group comprising a first number of cache lines, and each cache line is divided into a second number of cache blocks, the method comprising:

[0008] receiving a cache line access request, where the cache line access request is obtained by aggregating memory access requests in a target warp based on a size of a cache line;

[0009] Extracting tag information from a target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset;

[0010] querying whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine the target group, the shared tag and the private tag are used to determine the target cache line in the target group, and the inter-block offset is used to determine the target cache block in the target cache line;

[0011] If the cache line access request hits the L1 cache, a target data location is determined in the hit target cache block based on the intra-block offset, and a corresponding memory access request is executed based on the target data location.

[0012] In a second aspect, an embodiment of the present invention discloses a graphics processing unit (GPU), the GPU comprising a stream multiprocessor and a memory system, the memory system comprising a private L1 cache subsystem of the stream multiprocessor and a memory aggregation unit; the L1 cache subsystem comprising an L1 cache and a first cache controller; the L1 cache being divided into a plurality of groups, each group comprising a first number of cache lines, each cache line being divided into a second number of cache blocks; wherein,

[0013] The memory aggregation unit is configured to aggregate memory access requests in the target warp based on a size of a cache line, generate a cache line access request, and send the cache line access request to the L1 cache subsystem;

[0014] The first cache controller is used to receive a cache line access request; extract tag information from a target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset; query whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine a target group, the shared tag and the private tag are used to determine a target cache line in the target group, and the inter-block offset is used to determine a target cache block in the target cache line; if the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed based on the target data location.

[0015] The embodiments of the present invention include the following advantages:

[0016] The memory access method proposed in the embodiment of the present invention further divides each cache line in the L1 cache of the GPU into cache blocks. The memory access request generated by the thread in the process of executing the computing task is still generated at the granularity of the cache line, and the cache line access request is generated. The L1 cache of the GPU responds to the cache line access request, extracts the tag information in the target memory address, including the shared tag, private tag, group index, inter-block offset and intra-block offset, and queries whether the cache line access request hits the L1 cache based on the tag information. If the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed on the target cache block at the target data location. By subdividing the cache line and tag information, the embodiment of the present invention realizes coarse-grained access to the L1 cache and fine-grained storage, which can fully utilize the storage space of the cache line and reduce the waste of cache space on the basis of ensuring data access efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of steps of an embodiment of a memory access method of the present invention;

[0018] Figure 2 Schematic diagram of the relationship between the target memory address and tag information in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the principle of a dynamic selection mechanism according to an embodiment of the present invention;

[0020] Figure 4 is a structural block diagram of a graphics processor according to an embodiment of the present invention;

[0021] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after the association are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0024] Streaming Multiprocessor (SM): A GPU has multiple Single Instruction, Multiple Data (SIMD) cores, called Streaming Multiprocessors (SMs). These cores are the GPU's core computing units and consist of multiple processing cores, shared memory, register files, and control logic. They are responsible for breaking down incoming GPU computing tasks into multiple threads and assigning them to the various processing cores for parallel execution. They also manage shared resources and coordinate collaboration and synchronization between threads to achieve efficient parallel computing.

[0025] A warp (thread bundle): The basic unit of GPU thread execution, typically consisting of 32 threads. These threads execute together as a bundle, executing the same instructions simultaneously but operating on different data. This data parallelism fully utilizes the GPU's parallel computing capabilities and improves computing efficiency.

[0026] The warp scheduler is a key component within the GPU's streaming multiprocessor (SM), responsible for managing and scheduling warp execution. It monitors the status of each warp, including thread readiness and resource usage. Based on a specific scheduling strategy, it selects a ready warp and launches it into the execution / memory pipeline stage, enabling threads within that warp to execute instructions simultaneously on the SIMD backend. This maximizes the GPU's parallel computing capabilities and improves overall computing efficiency.

[0027] The SIMD backend is the core component of the GPU that performs single-instruction, multiple-data (SIMD) parallel computing. It contains multiple processing units that can simultaneously process different data. These processing units are typically organized into an array to efficiently execute SIMD instructions.

[0028] Memory Access Unit (Load / Store Unit, LSU): It is the component in the GPU responsible for processing memory access requests. It is mainly responsible for merging, managing and executing memory requests from different threads.

[0029] The Memory Aggregation Unit (MAU) is a core component of the memory access unit in the GPU's streaming multiprocessor (SM). Its primary function is to collect memory access requests from different threads, analyze and process them, and merge adjacent, consecutively addressed requests into larger requests, aligning them to a specific access granularity (such as the size of a cache line). This reduces the number of memory accesses, improves memory access efficiency, and reduces memory bandwidth usage.

[0030] Reference Figure 1 , shows a flowchart of a memory access method embodiment of the present invention, the method is applied to a GPU, the L1 cache of the GPU is divided into a plurality of groups, each group contains a first number of cache lines, each cache line is divided into a second number of cache blocks, the method may include the following steps:

[0031] Step 101: Receive a cache line access request, where the cache line access request is obtained by aggregating memory access requests in a target warp based on a cache line size.

[0032] Step 102: extract tag information from the target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset;

[0033] Step 103: query whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine the target group, the shared tag and the private tag are used to determine the target cache line in the target group, and the inter-block offset is used to determine the target cache block in the target cache line;

[0034] Step 104 : If the cache line access request hits the L1 cache, determine the target data location in the hit target cache block based on the intra-block offset, and execute the corresponding memory access request based on the target data location.

[0035] The use of larger cache lines (such as 64Byte or 128Byte) for the GPU's L1 cache results in insufficient cache line space utilization, resulting in a problem of cache space waste. An embodiment of the present invention provides an L1 Cache storage subsystem for a GPU. The GPU's L1 cache usually adopts a multi-way group-connected structure, dividing the L1 cache into several groups (Sets), each group containing a fixed number (such as a first number) of cache lines (Cache Lines / Ways). An embodiment of the present invention further subdivides the cache lines in the L1 cache, dividing each cache line into a fixed number (such as a second number) of cache blocks. Based on the L1 Cache storage subsystem, an embodiment of the present invention provides a memory access method that can perform coarse-grained access and fine-grained storage on the GPU's L1 cache, fully utilizing the storage space of the cache lines and reducing cache space waste while ensuring data access efficiency.

[0036] The embodiment of the present invention does not limit the number of groups into which the L1 cache is divided, nor the first number and the second number. For example, the L1 cache can be divided into 64 groups, each group containing 4 cache lines (called a 4-way set associative structure), and each cache line is divided into 4 cache blocks.

[0037] In a specific implementation, the GPU's L1 cache may use a 128-byte cache line (one cache line is 128 bytes) or a 64-byte cache line (one cache line is 64 bytes). In the embodiment of the present invention, Byte is abbreviated as B.

[0038] Taking a 128B cache line as an example, assuming the second number is 4, the embodiment of the present invention divides the 128B storage space of the consecutive cache lines into 4 cache blocks, each of which has a size of 32B. The addresses of these 4 cache blocks may not be consecutive.

[0039] Taking a 64B cache line as an example, assuming the second number is 4, the embodiment of the present invention divides the 64B storage space of the consecutive cache lines into 4 cache blocks, each of which has a size of 16B. The addresses of these 4 cache blocks may not be consecutive.

[0040] For the L1 cache system with a 4-way group-associated structure, each group contains 4 cache lines, and each cache line is further divided into 4 cache blocks. Therefore, each group contains 16 cache blocks. In this 4-way group-associated structure, 16 different cache blocks are allowed to store data in the same group, which can reduce the probability of data conflicts. That is, when data with multiple different addresses are mapped to the same group, as long as there are free cache blocks in the group, the data can be stored in any free cache block in the group. For example, if all 16 cache blocks in the group are free, they can be stored in any cache block of these 16 cache blocks. In this way, fine-grained storage of the L1 cache can be achieved. Among them, fine-grained storage refers to storing data with cache blocks as the granularity.

[0041] The embodiments of the present invention are mainly described by taking a 128B cache line as an example. For an application scenario of a 64B cache line, the execution steps are similar and can be referred to each other.

[0042] To ensure data access efficiency, the embodiment of the present invention performs coarse-grained access to the L1 cache, wherein coarse-grained access refers to accessing data at the granularity of cache behavior.

[0043] In practice, applications submit computational tasks to the GPU through programming interfaces. These tasks contain code logic that must be executed in parallel on the GPU and memory access operations. The GPU scheduler assigns computational tasks to different streaming multiprocessors based on the nature of the computational tasks and resource requirements. Each streaming multiprocessor is responsible for executing a portion of the computational task, further decomposing the task into multiple threads. The streaming multiprocessor organizes the assigned threads into warps, typically containing 32 threads per warp. Under the control of the streaming multiprocessor, these threads execute the same instruction sequence in parallel, but operate on different data. During the execution of the computational task, threads within a warp need to read or write data from or to memory according to the instructions, generating memory access requests. Each streaming multiprocessor (SM) contains multiple warp schedulers, which are responsible for selecting a ready warp and issuing its instructions to the execution pipeline (including the memory pipeline). The memory access unit (LSU) is responsible for processing memory access requests for all threads within the warp. The memory aggregation unit in the memory access unit (LSU) collects memory access requests from each thread in the warp, analyzes and processes these memory access requests, and merges adjacent, continuously addressed requests into a larger request, aligning it to an access granularity of the size of a cache line (e.g., 128B). This converts multiple small-granularity memory access requests into a large-granularity memory access request (i.e., a cache line access request), reducing the number of memory accesses and improving memory access efficiency. The cache line access request and the data width requested for access generated by the memory aggregation unit are sent to the GPU's memory system.

[0044] The requested data width is the total size of the consecutive cache blocks to be accessed by the cache line access request. For example, if the cache line access request is to access one cache block, since the size of a cache block is 32B, the requested data width is 32B. For another example, if the cache line access request is to access two consecutive cache blocks, the requested data width is 64B.

[0045] It should be noted that only when the warp scheduler selects a ready warp (i.e., the target thread bundle) to be launched into the memory pipeline will the memory access requests in the warp be aggregated. In addition, the memory aggregation unit checks whether the request addresses of all threads in the warp are continuous or aligned. For example, if the addresses of the memory access requests of the 32 threads in the warp are continuous and fall within the same 128B interval, they can be merged into a cache line (e.g., 128B) request, and the data width to be accessed by the cache line request is 128B; otherwise, multiple discrete cache line requests may be generated, and the data width to be accessed by each of the generated discrete cache line requests is less than 128B.

[0046] The GPU's memory system receives the cache line access request and extracts tag information from the target memory address corresponding to the cache line access request. The tag information includes a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset. The tag information is used to complete the mapping from the virtual address to the physical location.

[0047] The target memory address corresponding to the cache line access request is a virtual address. When converting the virtual address to a physical address, the virtual address needs to be parsed to obtain tag information. The mapping from the virtual address to the cache physical address can be completed based on the tag information.

[0048] The embodiment of the present invention further subdivides cache lines into cache blocks, thereby re-dividing the tag information corresponding to the target memory address, which includes a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset.

[0049] Reference Figure 2 , which shows a schematic diagram of the relationship between the target memory address and tag information in an embodiment of the present invention. The set index (SetIndex) is used to determine the target group within a plurality of groups (e.g., 64 groups) of the L1 cache. The shared tag (Shared Tag) and the private tag (Private Tag) are used together to determine the target cache line within the target group. The chunk offset (Chunk Index) is used to determine the target cache block within the target cache line. The chunk offset (Chunk Offset) is used to determine the target data location within the target cache block.

[0050] If a 128B cache line is used, the size of a cache line is 128 bytes, the size of a cache block is 32 bytes, the target memory address is 32 bits, and the target memory address (from high to low) includes the following tag information in sequence: an 11-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 5-bit intra-block offset.

[0051] If a 64B cache line is used, the size of a cache line is 64 bytes, the size of a cache block is 16 bytes, the target memory address is 32 bits, and the target memory address (from high to low) includes the following tag information in sequence: a 12-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 4-bit intra-block offset.

[0052] In a specific implementation, a separate tag can be assigned to each cache block to distinguish each cache block. However, due to the large number of cache blocks, assigning a separate tag to each cache block will generate huge storage overhead and may increase cache access latency.

[0053] To solve this problem, the embodiment of the present invention divides the tag field (upper 19 bits) of the target memory address into two parts: a shared tag and a private tag. The upper 11 bits of these 19 bits are called shared tags, and the lower 8 bits are called private tags. All cache blocks in the same set have the same shared tag, and cache blocks in different cache lines in the same set have different private tags. That is, each cache block data is any 32B data aligned within every 2MB space, where 2 21 ×8bit=2MB.

[0054] In an embodiment of the present invention, for a 4-way group-associative structure, a group contains 4 cache lines, each cache line is divided into 4 cache blocks, and therefore, a group contains 16 cache blocks. In an embodiment of the present invention, cache blocks in the same group are referred to as a cache block group.

[0055] For 128B cache lines, private tags allow 32B cache blocks to be discontinuous, thereby refining the storage granularity of cache lines.

[0056] For different cache lines in the same set, each cache block has the same size (all 32B), has the same shared index and different private indexes, the data within the cache block is continuous, and the addresses between cache blocks are not continuous, but belong to the same 2MB memory space.

[0057] For the same cache line in the same set, each cache block has the same size (all 32B), has the same shared index and private index, the data within the cache block is continuous, and the addresses between cache blocks are not continuous, but belong to the same 2MB memory space.

[0058] When searching the L1 cache based on the target memory address, the tag information in the target memory address is extracted and the extracted tag information is compared. The comparison of shared tags and private tags can be performed in parallel, that is, after determining the target group, the shared tags and private tags of each cache line in the target group can be compared in parallel. The comparison of shared tags is performed relative to the cache block group and is used to locate a certain cache block group (target group); while the comparison of private tags is performed for each cache line and is used to locate a certain cache line (target cache line).

[0059] For the target memory address, if both the shared tag and the private tag are matched in the L1 cache, it is determined that the target cache line is found, and the target cache block can be further determined in the target cache line based on the inter-block offset. If found, it means that the cache line access request hits the L1 cache, then the target data location is determined in the hit target cache block based on the intra-block offset, and the cache line access request is executed based on the target data location.

[0060] In an example, assuming the target memory address is 0x1234ABCD, the corresponding binary value is: 00010010001101001010101111001101. Figure 2 The address structure shown can be used to extract the following tag information from high to low: Shared Tag is 00010010001 (high 11 bits), which converts to decimal as 145. Private Tag is 10100101 (8 bits), which converts to decimal as 165. Set Index is 010111 (6 bits), which converts to decimal as 23. Chunk Index is 10 (2 bits), which converts to decimal as 2. Chunk Offset is 01101 (low 5 bits), which converts to decimal as 13.

[0061] According to the group index 23, the 23rd group in the L1 cache can be found. Then, among the 4 cache lines in the 23rd group, if a cache line with both a shared tag and a private tag matching is found, it can be determined as the target cache line. Since a cache line consists of 4 cache blocks, 2 bits (2 2 =4) represents a cache line among the four cache lines. For example, an inter-block offset of 00 represents the first cache block in a cache line, an inter-block offset of 01 represents the second cache block in the cache line, an inter-block offset of 10 represents the third cache block in the cache line, and an inter-block offset of 11 represents the fourth cache block in the cache line.

[0062] Each cache block is 32 bytes (32 × 4 = 128). The inter-block offset can be used to locate a target cache block within the four cache blocks of the target cache line. In this example, the inter-block offset is 10, indicating the third cache block in the cache line. The intra-block offset can then be used to locate the target data location within the target cache block. In this example, the target data location is the 13th byte in the third cache block of the target cache line.

[0063] It should be noted that the cache line access request may be a read (Load) request, such as obtained by aggregating memory access requests (Load requests) in the target warp based on the size of a cache line. Alternatively, the cache line access request may be a write (Store) request, such as obtained by aggregating memory access requests (Store requests) in the target warp based on the size of a cache line.

[0064] For a read (Load) request, the target data location refers to a region of data to be read, and executing the cache line access request based on the target data location includes: copying the data in the region to a register of the requesting thread.

[0065] For a write (Store) request, the target data location refers to the area of ​​data to be updated, and executing the cache line access request based on the target data location includes: writing the register data of the requesting thread to the area (write allocation) or directly transmitting it to the lower-level storage (write eviction).

[0066] It should be noted that the number of bits of each tag information mentioned above is only for illustrative purposes, and the embodiment of the present invention does not limit the number of bits of each tag information in the target memory address. For example, in a 32-bit memory address configuration (i.e., 4GB of GPU memory), the total length of the shared tag and the private tag can be set to 19 bits. These 19 bits are divided into shared tags and private tags. If a larger private tag and a smaller shared tag are divided, more 128B cache lines can share the same cache block group, thereby improving the utilization of the cache space; if a smaller private tag and a larger shared tag are divided, less storage overhead can be brought about.

[0067] In a specific implementation, the ratio of shared tags to private tags can be set based on the trade-off between performance and overhead. Tests have shown that as the length of the private tag increases, the performance improvement rate becomes slower and slower. When the private tag is increased from 6 bits to 7 bits, the performance achieves a leap. Test results show that most programs tend to access memory areas of limited size in a short period of time. The 8-bit private tag allows a cache block group to store any cache block within a 1MB memory area. Therefore, the embodiment of the present invention preferably uses an 8-bit private tag, and the remaining 11 bits are used as a shared tag, so that the GPU's L1 cache system achieves a better trade-off between performance and overhead.

[0068] Furthermore, for a 128B cache line, an embodiment of the present invention divides it into four 32B cache blocks. It is understandable that the embodiment of the present invention does not impose any restrictions on the size of the divided cache blocks. Smaller cache blocks can bring higher cache space utilization, thereby improving the performance of some programs. On the other hand, for programs with good spatial locality, using smaller cache blocks will destroy the advantage of spatial location. In addition, every time the size of the cache block is reduced by half, the tag storage space will double. Taking into account the balance between performance and overhead, the embodiment of the present invention selects 32B as the cache block size. Furthermore, another reason for selecting a 32B cache block is that the minimum data transmission unit of the interconnection network between the L1 cache and the L2 cache (L2 Cache) is 32B, which can achieve the effect of reducing the intercommunication traffic between different caches.

[0069] The memory access method proposed in an embodiment of the present invention further divides each cache line in the GPU's L1 cache into cache blocks. Memory access requests generated by threads during the execution of computing tasks are still generated at the cache line granularity, generating cache line access requests. The GPU's L1 cache system responds to cache line access requests by extracting tag information from the target memory address, including shared tags, private tags, group indexes, inter-block offsets, and intra-block offsets. Based on the tag information, it queries whether the cache line access request hits the L1 cache. If the cache line access request hits the L1 cache, it determines the target data location in the hit target cache block based on the intra-block offset, and then executes the corresponding memory access request on the target cache block at the target data location. A cache line access request can include aggregated memory access requests for multiple target cache blocks. Therefore, from the perspective of the GPU's streaming multiprocessor, after sending a cache line access request, it may receive one or more return data. From the perspective of the streaming multiprocessor, the one or more return data still represents a continuous cache line of data, but from the perspective of the L1 cache, the continuous cache line is divided into multiple cache blocks that may or may not be continuous. This enables coarse-grained access to the L1 cache and fine-grained storage, fully utilizing the storage space of the cache line and reducing cache space waste while ensuring data access efficiency.

[0070] In an optional embodiment of the present invention, the method may further include:

[0071] A block access mask code is generated according to the inter-block offset, and each bit of the block access mask code is used to indicate a target cache block.

[0072] This embodiment of the present invention further divides each cache line in the GPU's L1 cache into cache blocks. For a 4-way set-associative L1 cache system, each set contains 16 cache blocks. These 16 cache blocks have the same shared tag. Cache blocks in different cache lines within the same set have different private tags. Cache blocks in the same cache line within the same set have the same private tag. This embodiment of the present invention uses block access masks to distinguish between different cache blocks within the same cache line.

[0073] The inter-block offset and the requested access data width may together indicate which cache block or blocks in a cache line need to be accessed. A block access mask may be generated based on the inter-block offset and the requested access data width, and bits corresponding to the target cache block in the block access mask may be set to a target value, such as 1. The bits in the block access mask that are the target value correspond to the target cache block.

[0074] In an embodiment of the present invention, each cache line is divided into four cache blocks, and a four-bit block access mask can be used. Each bit of the block access mask can be used to indicate a target cache block. For example, the four cache blocks in a cache line are divided into cache block 0, cache block 1, cache block 2, and cache block 3. A four-bit block access mask is used to represent these four cache blocks. For example, the four cache blocks are numbered 0 to 3, and bit 0 of the block access mask represents cache block 0, bit 1 of the block access mask represents cache block 1, bit 2 of the block access mask represents cache block 2, and bit 3 of the block access mask represents cache block 3.

[0075] In an example, assuming the target memory address is 0xABCD0000, the corresponding binary representation is: 101010111100110100000000000000000. The extracted tag information is as follows: shared tag = 10101011110, decimal 1374; private tag = 01101000, decimal 104; group index = 000000, decimal 0; inter-block offset = 00, decimal 0; intra-block offset = 00000, decimal 0.

[0076] The target memory address points to the target cache line in group 0, with a shared tag of 1374 and a private tag of 104. An inter-block offset of 0 indicates that the target cache line is cache block 0. If the requested data width is 32 bytes, this indicates a request for one cache block, namely, cache block 0. Therefore, the block access mask can be set to 0001. Setting bit 0 of the block access mask to 1 indicates that the target cache block to be accessed is cache block 0 in the target cache line.

[0077] In practice, when 32 threads in a warp execute the same Load / Store instruction simultaneously, the streaming multiprocessor aggregates all memory access requests in the warp through the memory aggregation unit, aligning them to the access granularity of a cache line (e.g., 128 bytes) as much as possible, generating a cache line access request. After receiving the cache line access request and the requested data width, the GPU's memory system extracts the tag information and generates a block access mask based on the extracted inter-block offset and the requested data width.

[0078] The memory system queries whether the cache line access request hits the L1 cache based on the tag information. For example, the target group is determined based on the group index, and the shared tags and private tags of each cache line in the target group are compared in parallel, and then the target cache line is determined in the target group. If the target cache line exists, the block access mask is further matched. If all the target cache blocks indicated by the block access mask exist in the L1 cache, it is determined that the cache line access request hits the L1 cache. The target data location is determined in the hit target cache block based on the offset within the block, and the corresponding memory access request is executed based on the target data location. For example, if a read operation is performed on a target cache block, the data read from the target cache block is returned to the stream multiprocessor. If a write operation is performed on a target cache block, it is determined whether to return the execution result based on the write policy.

[0079] Furthermore, querying whether the cache line access request hits the L1 cache based on the tag information may include:

[0080] Step S11: If it is found that the group index, shared tag, and private tag all hit the L1 cache, then a candidate cache block that hits is determined;

[0081] Step S12: matching the target cache block indicated by the block access mask code with the candidate cache blocks;

[0082] Step S13: If all target cache blocks indicated by the block access mask code exist in the candidate cache blocks, it is determined that the cache line access request hits the L1 cache;

[0083] Step S14: If part of the target cache blocks indicated by the block access mask code exists in the candidate cache blocks, determining that the cache line access request partially misses the L1 cache;

[0084] Step S15: If all target cache blocks indicated by the block access mask code do not exist in the candidate cache blocks, it is determined that the cache line access request misses the L1 cache.

[0085] In an embodiment of the present invention, each cache line in the L1 cache is further divided into cache blocks, and a cache line access request may include a memory access request for one or more target cache blocks. The cache line access request may fall into the following three situations: all target cache blocks to be accessed by the cache line access request are in the L1 cache, which is referred to as the cache line access request hitting the L1 cache; a portion of all target cache blocks to be accessed by the cache line access request are in the L1 cache, which is referred to as the cache line access request partially missing the L1 cache; and all target cache blocks to be accessed by the cache line access request are not in the L1 cache, which is referred to as the cache line access request missing the L1 cache.

[0086] Specifically, after the GPU's memory system receives a cache line access request and the requested data width, it extracts the tag information and generates a block access mask, and queries the L1 cache based on the tag information and the block access mask. If it is found that the group index, shared tag, and private tag all hit the L1 cache, the candidate cache block that hits is determined. The shared tag and the private tag can jointly determine the target cache line. Since the target cache line includes 4 cache blocks, the 4 cache blocks in the target cache line can be determined as candidate cache blocks that hit. Next, the target cache block indicated by the block access mask is matched with the candidate cache block. If all the target cache blocks indicated by the block access mask exist in the candidate cache block, it is determined that the cache line access request hits the L1 cache; if some of the target cache blocks indicated by the block access mask exist in the candidate cache block, it is determined that the cache line access request partially misses the L1 cache; if the target cache block indicated by the block access mask does not exist in the candidate cache block, it is determined that the cache line access request misses the L1 cache.

[0087] For partial misses and misses, it is necessary to further query the lower-level cache (such as L2 cache or global memory).

[0088] In an example, assume that the target memory address corresponding to the cache line access request is 0xABCD0000, and the data width requested for access is 64B, that is, the request is to access two consecutive cache blocks. Since the inter-block offset in the target memory address is 00, indicating cache block 0, the two consecutive cache blocks include cache block 0 and cache block 1. According to the inter-block offset and the data width requested for access, a block access mask code of 0011 can be generated, indicating that the target cache blocks to be accessed include cache block 0 and cache block 1 in the target cache line. If, after querying, there is only cache block 0 in the L1 cache and no cache block 1, it is determined that the cache line access request partially misses the L1 cache. Among them, cache block 0 is called the cache block that hits the L1 cache. For cache blocks that hit the L1 cache, they can be read directly from the L1 cache. Cache block 1 is called the cache block that misses the L1 cache. For cache blocks that miss the L1 cache, it is necessary to further read from the L2 cache or global memory.

[0089] Furthermore, the L1 cache can record a cache block mask for each cache line. This cache block mask is used to record the cache block information contained in the corresponding cache line. Initially, the cache block mask for each cache line can be 0000. When a cache block in a cache line (assuming it is cache block 0) is accessed for the first time and then filled into the L1 cache, the cache block mask for that cache line in the L1 cache is updated to 0001.

[0090] When querying whether a cache line access request hits the L1 cache, first locate the target group based on the group index, then locate the target cache line in the target group based on the shared tag and private tag. If the group index, shared tag, and private tag all hit, the target cache line can be determined. Next, compare the block access mask with the cache block mask of the target cache line in the L1 cache. If the bits with values ​​of 1 in the two are the same, the cache line access request hits the L1 cache; if the bits with values ​​of 1 in the two are partially the same, the cache line access request partially misses the L1 cache; if the bits with values ​​of 1 in the two are not the same, the cache line access request partially misses the L1 cache. For example, if the block access mask is 0011 and the cache block mask of the target cache line in the L1 cache is 0001, the cache line access request partially misses the L1 cache. Specifically, cache block 0 hits and cache block 1 misses.

[0091] In an optional embodiment of the present invention, the method may further include:

[0092] Step S21: If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, querying the L2 cache based on the target memory address;

[0093] Step S22: If it is found that the cache line access request hits the L2 cache, the cache line that hits is read from the L2 cache, and a first cache block is extracted from the cache line that hits, where the first cache block includes the cache block that is not hit by the L1 cache.

[0094] Step S23: Determine a second cache block in the L1 cache, and fill the first cache block into the second cache block in the L1 cache.

[0095] When a cache line access request misses the L1 cache, or when a cache line access request partially misses the L1 cache, the L2 cache can be queried for the cache blocks that missed the L1 cache. At this point, the GPU generates a miss signal, triggering a query operation on the L2 cache. The GPU searches the L2 cache based on the target memory address of the request. If the cache line access request hits the L2 cache, the data of the cache line that hits is returned to the L1 cache. After receiving the data of the cache line that hits from the L2 cache, the L1 cache extracts the cache blocks that missed the L1 cache and fills them into the appropriate locations. For example, if there are free cache blocks in the L1 cache, they can be directly filled into the free cache blocks. If there are no free cache blocks, or if there are not enough free cache blocks, invalid cache blocks can be replaced. If there are no invalid cache blocks, or if there are not enough invalid cache blocks, valid cache blocks need to be replaced.

[0096] A free cache block is a currently unused cache block that has not been allocated to store any valid data and is idle, waiting to be filled with data. An invalid cache block is a cache block whose data is invalid and does not contain any meaningful information. This may be because the cache block has just been initialized and has not yet been loaded with data, or because the data in it has been marked as invalid for some reason. A valid cache block is a cache block that stores valid data that has been correctly loaded from memory.

[0097] In a GPU, the L2 cache may correspond to multiple L1 caches. Therefore, the embodiment of the present invention does not further subdivide the L2 cache, and the access and storage to the L2 cache are still at the cache behavior granularity. For example, if the L1 cache misses, the same tag information as the L1 is used to search in the L2 cache. If the cache line access request hits the L2 cache, the cache line that hits is read, and the cache line that hits is the cache line where the cache block (such as cache block 1) that the L1 cache misses is located. A first cache block is extracted from the cache line that hits, and the first cache block includes the cache block that the L1 cache misses. The second cache block determined in the L1 cache may include an invalid cache block and / or a valid cache block.

[0098] The first cache block refers to a cache block found in the L2 cache that needs to be filled into the L1 cache, and the second cache block refers to a cache block in the L1 cache that needs to be filled (or replaced) with the first cache block.

[0099] For example, in the example above, the cache block that missed the L1 cache is cache block 1 in the target cache line. Therefore, if the query for the target cache line hits the L2 cache, cache block 1 can be extracted from the hit cache line as the first cache block. Next, a second cache block is determined in the L1 cache, and the first cache block is filled into the second cache block in the L1 cache, which is then returned to the streaming multiprocessor.

[0100] If the L2 cache also misses, it will search further in the global memory. After finding the target cache block, it will be filled into the L2 cache and the L1 cache.

[0101] In an optional embodiment of the present invention, determining the second cache block in the L1 cache may include:

[0102] Step S31: Searching the L1 cache for an invalid cache block that meets a first condition, wherein the first condition includes: all cache blocks in the same group as the invalid cache block are invalid, or a shared tag of the invalid cache block matches a shared tag of the first cache block;

[0103] Step S32: If there is an invalid cache block that meets the first condition, determine that the invalid cache block that meets the first condition is the second cache block.

[0104] When filling the L1 cache with the first cache block, in order to ensure the correctness of other cache blocks, the second cache block determined in the L1 cache in the embodiment of the present invention needs to meet certain conditions.

[0105] Specifically, when filling the L1 cache, a free cache block is preferentially searched in the L1 cache. If there is no free cache block in the L1 cache, the L1 cache is searched for an invalid cache block that meets a first condition. If there is an invalid cache block that meets the first condition, the invalid cache block that meets the first condition is determined to be the second cache block. The first condition includes: all cache blocks in the same group as the invalid cache block are invalid, or the shared tag of the invalid cache block matches the shared tag of the first cache block.

[0106] If a cache block group contains one or more valid cache blocks, and the shared tag of the cache block group is different from the shared tag of the first cache block, using a cache block in the cache block group as the second cache block will cause its shared tag to change, leading to an error. Therefore, a cache block in the cache block group that does not meet the first condition cannot be replaced as the second cache block.

[0107] In an optional embodiment of the present invention, the method may further include:

[0108] Step S41: If the number of invalid cache blocks that meet the first condition is less than the number of the first cache blocks, determine the remaining required number;

[0109] Step S42: Searching the L1 cache for a valid cache block that meets a second condition, wherein the second condition includes: the valid cache block has not been accessed recently;

[0110] Step S43: If there are valid cache blocks that meet the second condition in the L1 cache, and the number of valid cache blocks that meet the second condition is greater than or equal to the remaining required number, then determine the remaining required number of valid cache blocks from the valid cache blocks that meet the second condition as second cache blocks;

[0111] Step S44: If there is no valid cache block meeting the second condition in the L1 cache, or the number of valid cache blocks meeting the second condition is less than the remaining required number, a valid cache block is randomly selected from the L1 cache as the second cache block.

[0112] If there are no free cache blocks in the L1 cache, and the number of invalid cache blocks in the L1 cache that meet the first condition is less than the number of the first cache blocks, that is, the number of invalid cache blocks that can be filled (or replaced) in the L1 cache is insufficient, and the remaining required number of valid cache blocks needs to be replaced.

[0113] Furthermore, embodiments of the present invention can employ an NRU (Not Recently Used) replacement strategy for valid cache blocks. Specifically, when a cache block in the L1 cache needs to be replaced, invalid cache blocks that meet a first condition are prioritized. If the number of invalid cache blocks meeting the first condition is insufficient, valid cache blocks meeting a second condition are selected from the L1 cache to replace the remaining required number of invalid cache blocks. The second condition includes: the valid cache block has not been recently accessed.

[0114] In a specific implementation, each cache block in the L1 cache can be associated with a preset bit (such as the NRU bit), and the initial value of all NRU bits is 0. When a cache block in the L1 cache is accessed, its NRU bit is set to 1, indicating that the cache block has been accessed recently. When determining the second cache block to be replaced in the L1 cache, the cache block with the NRU bit of 0 is preferentially selected. If the number of cache blocks with the NRU bit of 0 is greater than the number of first cache blocks, the second cache block is randomly selected from the cache blocks with the NRU bit of 0. If the number of cache blocks with the NRU bit of 0 is less than the number of first cache blocks, the insufficient part can be randomly selected from the cache blocks with the NRU bit of 1.

[0115] In practice, the GPU's L1 cache primarily employs a write-eviction strategy for write operations. This strategy involves writing data directly to lower-level storage (such as the L2 cache or global memory) instead of loading it into the cache (e.g., the L1 cache) when a write request misses. Therefore, when determining the second cache block to replace in the L1 cache, if there are insufficient invalid cache blocks and a valid cache block needs to be replaced, after selecting a valid cache block, its valid bit can be reset to invalidate it, as the write-eviction strategy does not require data to be written back.

[0116] In an optional embodiment of the present invention, the method may further include:

[0117] Step S51: If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, checking whether a miss request entry corresponding to the cache line access request already exists in the miss request entries recorded in the miss status processing register;

[0118] Step S52: If a miss request entry corresponding to the cache line access request already exists, wait for the miss request corresponding to the cache line access request to be processed to be completed;

[0119] Step S53: If there is no miss request entry corresponding to the cache line access request, create a miss request entry corresponding to the cache line access request, and add a miss block mask code to the created miss request entry, where each bit of the miss block mask code is used to indicate a cache block that has an L1 cache miss;

[0120] Step S54: Initiate a miss request carrying the miss block mask code to the L2 cache to request the cache block indicated by the miss block mask code to be returned from the L2 cache.

[0121] MSHR (Miss Status Handling Register) is a key hardware structure of the GPU cache subsystem, used to efficiently manage cache miss requests.

[0122] If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, such as in the example above where cache block 0 hits but cache block 1 misses, the L1 cache system allocates a new miss request entry (MSHR entry) or uses an existing MSHR entry to record the miss. The MSHR sends the miss request to a lower-level cache (such as the L2 cache or global memory) and allocates space in the L1 cache for the cache block that missed (i.e., determines a second cache block).

[0123] To maintain compatibility with the segmented tag information of the present invention and reduce interconnect data transmission costs, embodiments of the present invention can add an issued block mask to each MSHR entry. The issued block mask is used to record which cache blocks in the cache line have issued miss requests to the lower-level cache. For cache blocks that have issued miss requests to the lower-level cache, the existing MSHR entry is used to avoid redundant requests.

[0124] When a new MSHR entry is assigned to a cache line request that misses, meaning the miss for that cache line request is the first miss in the L1 cache, a miss request containing a miss block mask is sent to the L2 cache, which returns only the cache blocks indicated in the miss block mask. For example, in the example above, if cache block 0 hits the L1 cache and cache block 1 misses the L1 cache, the miss block mask in the miss request can be 0010, indicating that cache block 1 is the only cache block to be returned, without returning cache block 0, thus avoiding redundant data transfers.

[0125] In an optional embodiment of the present invention, the current mode of the L1 cache of the GPU is a fine-grained mode. In the fine-grained mode, the L1 cache uses a cache block as a storage granularity. The method may further include:

[0126] Step S61: Determine a current target mode based on the sampled values, where the target mode includes a fine-grained mode or a coarse-grained mode. In the coarse-grained mode, the L1 cache stores granularity using cache behavior. The sampled values ​​include the number of misses and the transaction volume generated when the misses are counted within a preset period in the fine-grained mode and the coarse-grained mode, respectively.

[0127] Step S62: Switch the current mode to the target mode.

[0128] The memory access method provided in the embodiment of the present invention further divides the cache lines in the L1 cache into cache blocks, which can realize coarse-grained access to the GPU's L1 cache and fine-grained storage, which is called fine-grained mode.

[0129] In practice, fine-grained mode can improve cache hit rates for GPU applications that are sensitive to cache line length, such as those using cache line prefetching (CLP). However, for GPU applications with good spatial locality, adopting fine-grained mode—that is, storing only the currently requested cache block on a miss—can disrupt the application's otherwise good spatial locality, potentially leading to additional misses and negative performance.

[0130] To avoid the aforementioned issues, an embodiment of the present invention provides a variable-granularity memory access method that dynamically selects a more appropriate memory granularity for the L1 cache system, such as coarse-grained mode or fine-grained mode, based on the current actual situation. The coarse-grained mode uses a cache line for storage granularity, while the fine-grained mode uses a cache block for storage granularity. The access granularity in both coarse-grained and fine-grained modes is the cache line.

[0131] In coarse-grained mode, the generated block access mask is 1111, which means that all four cache blocks of the entire target cache line need to be retrieved. Therefore, all four bits of the block access mask are set to 1. Similarly, in the event of an L1 cache miss or partial hit, the miss block mask is also set to 1111, filling (or replacing) the entire cache line to maintain the application's original good spatial locality.

[0132] Furthermore, embodiments of the present invention provide a dynamic selection mechanism that can adaptively switch to an appropriate target mode during program execution. Specifically, the current target mode can be determined based on sampling values, including either a fine-grained mode or a coarse-grained mode. The sampling values ​​include the number of misses and the transaction volume generated during each miss, respectively, within a preset period in the fine-grained mode and the coarse-grained mode.

[0133] For example, in the fine-grained mode and the coarse-grained mode, a first counter is set to count the number of misses, and a second counter is set to count the transaction volume generated when a miss occurs. For example, the first counter is incremented by 1 each time a miss request is generated.

[0134] The transaction volume generated by a miss can be measured by the number of data packets generated by a miss request. The interconnection network between the L1 and L2 caches is packet-based. When the L1 cache misses or partially misses, the L1 cache sends a miss request packet to the L2 cache. Assume that the number of cache blocks with a miss in the L1 cache is n, where n is a positive integer. If the L2 cache hits, the L2 cache returns n data packets (i.e., n cache blocks) to the L1 cache. Therefore, a miss request can generate a transaction volume of n+1 data packets.

[0135] When a preset period arrives, the sampling value in fine-grained mode can be calculated based on the values ​​of the first and second counters in fine-grained mode; and the sampling value in coarse-grained mode can be calculated based on the values ​​of the first and second counters in coarse-grained mode. By comparing the sampling values ​​in the two modes, the target mode can be determined. For example, the sampling value can be the product of the first and second counters. A smaller sampling value indicates a lower performance overhead in the corresponding mode. Therefore, the mode with the smaller sampling value can be determined as the target mode.

[0136] In an optional embodiment of the present invention, the method may further include:

[0137] Step S71: Select any two groups in the L1 cache of the GPU, one of which is used as a first subset and the other as a second subset;

[0138] Step S72: within a preset period, counting the number of misses and the transaction volume generated when the misses occur for each cache line in the first subset and the second subset respectively;

[0139] Switching the current mode to the target mode may include:

[0140] Switching a current mode of a dependent subset in an L1 cache of the GPU to the target mode, the dependent subset including other groups in the L1 cache except the first subset and the second subset.

[0141] To collect real-time statistics of sample values ​​in both fine-grained and coarse-grained modes, the present invention randomly selects two groups in the GPU's L1 cache, uses one group as the first subset, and uses the other group as the second subset. The first subset is fixed in fine-grained mode, while the second subset is fixed in coarse-grained mode.

[0142] For example, the GPU's L1 cache is divided into 64 groups, each containing four cache lines. Assuming group 0 is selected as the first subset, the four cache lines in group 0 are fixed to use fine-grained mode; if group 1 is selected as the second subset, the four cache lines in group 1 are fixed to use coarse-grained mode. A first counter and a second counter are respectively set in the first subset to count the number of misses in fine-grained mode and the transaction volume generated by misses; a first counter and a second counter are respectively set in the second subset to count the number of misses in coarse-grained mode and the transaction volume generated by misses.

[0143] Reference Figure 3 , shows a schematic diagram of the principle of a dynamic selection mechanism according to an embodiment of the present invention. The multiplier is used to calculate the product of the first counter and the second counter. The subtractor is used to compare the sample values ​​in the two modes. When a preset period arrives, the mode with the smaller sample value can be selected as the target mode. In this embodiment of the present invention, the first subset (e.g., group 0) and the second subset (e.g., group 1) use fixed modes for real-time statistics of sample values ​​in fine-grained mode and coarse-grained mode, respectively. The dependent subsets in the L1 cache, i.e., groups other than the first subset (e.g., group 0) and the second subset (e.g., group 1), use a target mode determined by the dynamic mechanism. For example, when a preset period arrives, if the target mode is determined to be fine-grained mode, the current mode of the dependent subsets (groups 2 to 63) is switched to fine-grained mode. If the target mode is determined to be coarse-grained mode, the current mode of the dependent subsets (groups 2 to 63) is switched to coarse-grained mode. The first subset remains fixed in fine-grained mode, and the second subset remains fixed in coarse-grained mode, continuing statistics for the next period.

[0144] In a specific implementation, when certain applications display irregular memory access behavior, resulting in insufficient cache space and memory bandwidth utilization, the embodiments of the present invention can dynamically adjust the storage granularity of the L1 cache, and then can adapt to GPU applications that are sensitive to cache line lengths to improve cache hit rates and increase cache space and memory bandwidth utilization; or can adapt to GPU applications with good spatial locality to avoid performance degradation of GPU applications with good spatial locality while improving cache space and memory bandwidth utilization.

[0145] Furthermore, to adapt to changes in memory access behavior, embodiments of the present invention employ an aging mechanism for the first and second counters. The aging mechanism includes the following steps: if the value of the first counter in either the fine-grained mode or the coarse-grained mode exceeds a preset threshold, the values ​​of the first and second counters in the fine-grained mode, as well as the values ​​of the first and second counters in the coarse-grained mode, are shifted right by one bit. Shifting right by one bit is equivalent to dividing by two, returning the four counters to more reasonable values.

[0146] Experiments have found that when the value of the first counter exceeds a certain value (preset threshold), the accuracy of mode selection will be affected. The preset threshold is an empirical value, such as 1024. When the value of the first counter in a certain mode exceeds the preset threshold, the embodiment of the present invention shifts the values ​​of the four counters in both modes right by one bit instead of resetting them to zero. This allows the values ​​of the four counters to still reflect recent memory access behavior to a certain extent, ensuring the accuracy of mode selection.

[0147] In summary, the memory access method proposed in the embodiment of the present invention further divides each cache line in the L1 cache of the GPU into cache blocks. The memory access request generated by the thread in the process of executing the computing task is still generated at the granularity of the cache line, and the cache line access request is generated. The L1 cache of the GPU responds to the cache line access request, extracts the tag information in the target memory address, including the shared tag, private tag, group index, inter-block offset and intra-block offset, and queries whether the cache line access request hits the L1 cache based on the tag information. If the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed on the target cache block at the target data location. By subdividing the cache lines and the tag information, the embodiment of the present invention realizes coarse-grained access to the L1 cache and fine-grained storage, which can fully utilize the storage space of the cache lines and reduce the waste of cache space on the basis of ensuring data access efficiency.

[0148] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0149] Reference Figure 4, shows a structural block diagram of a graphics processor GPU400 according to an embodiment of the present invention, wherein the GPU400 includes a stream multiprocessor 401 and a memory system, wherein the memory system includes an L1 cache subsystem 402 private to the stream multiprocessor and a memory aggregation unit 403; the L1 cache subsystem 402 includes an L1 cache 4021 and a first cache controller 4022; the L1 cache is divided into a plurality of groups, each group includes a first number of cache lines, and each cache line is divided into a second number of cache blocks; wherein,

[0150] The memory aggregation unit 403 is configured to aggregate memory access requests in the target warp based on the size of a cache line, generate a cache line access request, and send the cache line access request to the L1 cache subsystem 402;

[0151] The first cache controller 4022 is used to receive a cache line access request; extract tag information from the target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset and an intra-block offset; query whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine the target group, the shared tag and the private tag are used to determine the target cache line in the target group, and the inter-block offset is used to determine the target cache block in the target cache line; if the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed based on the target data location.

[0152] The L1 cache is a private cache for the SM, which can be 128B in line size and can adopt a write eviction policy. The L2 cache is shared by all SMs and is used to handle L1 miss requests. The memory aggregation unit is integrated into the LSU and is used to merge the requests of the 32 threads in the target warp into 1-N cache line requests to ensure access alignment (such as 128B boundaries). It can be understood that the SM also includes key storage components such as register files and shared memory, which are not shown in the figure.

[0153] Optionally, in the L1 cache, the size of a cache line is 128 bytes, the size of a cache block is 32 bytes, the target memory address is 32 bits, and the target memory address includes an 11-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 5-bit intra-block offset; or,

[0154] In the L1 cache, the size of a cache line is 64 bytes, the size of a cache block is 16 bytes, the target memory address is 32 bits, and the target memory address includes a 12-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 4-bit intra-block offset.

[0155] Optionally, the first cache controller is further configured to:

[0156] generating a block access mask code according to the inter-block offset, each bit of the block access mask code being used to indicate a target cache block;

[0157] If it is found that the group index, shared tag, and private tag all hit the L1 cache, determining the candidate cache block that hits;

[0158] Matching the target cache block indicated by the block access mask code with the candidate cache block;

[0159] If all target cache blocks indicated by the block access mask code exist in the candidate cache blocks, determining that the cache line access request hits the L1 cache;

[0160] If part of the target cache blocks indicated by the block access mask code exists in the candidate cache blocks, determining that the cache line access request partially misses the L1 cache;

[0161] If all target cache blocks indicated by the block access mask code do not exist in the candidate cache blocks, it is determined that the cache line access request misses the L1 cache.

[0162] Furthermore, the memory system further includes a globally shared L2 cache subsystem, the L2 cache subsystem including an L2 cache and a second cache controller. Optionally, the first cache controller is further configured to:

[0163] If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, sending a miss request triggered by the cache line access request to the L2 cache subsystem;

[0164] The second cache controller is configured to query the L2 cache based on the target memory address; if it is found that the cache line access request hits the L2 cache, read the hit cache line from the L2 cache, extract a first cache block from the hit cache line, and return it to the L1 cache subsystem; the first cache block includes the cache block that was missed by the L1 cache;

[0165] The first cache controller is further configured to determine a second cache block in the L1 cache, and fill the first cache block into the second cache block in the L1 cache.

[0166] Optionally, the first cache controller is specifically configured to:

[0167] A search is performed in the L1 cache to determine whether there is an invalid cache block that meets a first condition, where the first condition includes: all cache blocks in the same group as the invalid cache block are invalid, or a shared tag of the invalid cache block matches a shared tag of the first cache block; if there is an invalid cache block that meets the first condition, determining that the invalid cache block that meets the first condition is the second cache block.

[0168] Optionally, the first cache controller is further configured to:

[0169] If the number of invalid cache blocks that meet the first condition is less than the number of the first cache blocks, determining the remaining required number;

[0170] Searching the L1 cache for a valid cache block that satisfies a second condition, wherein the second condition includes: the valid cache block has not been accessed recently;

[0171] If there are valid cache blocks that meet the second condition in the L1 cache, and the number of valid cache blocks that meet the second condition is greater than or equal to the remaining required number, determining the remaining required number of valid cache blocks from the valid cache blocks that meet the second condition as second cache blocks;

[0172] If there is no valid cache block meeting the second condition in the L1 cache, or the number of valid cache blocks meeting the second condition is less than the remaining required number, a valid cache block is randomly selected from the L1 cache as the second cache block.

[0173] Optionally, the first cache controller is further configured to:

[0174] If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, checking whether a miss request entry corresponding to the cache line access request already exists in the miss request entries recorded in the miss status processing register;

[0175] If a miss request entry corresponding to the cache line access request already exists, waiting for completion of processing of the miss request corresponding to the cache line access request;

[0176] If there is no miss request entry corresponding to the cache line access request, creating a miss request entry corresponding to the cache line access request, and adding a miss block mask code to the created miss request entry, where each bit of the miss block mask code is used to indicate a cache block of an L1 cache miss;

[0177] A miss request carrying the miss block mask code is initiated to the L2 cache to request that the cache block indicated by the miss block mask code be returned from the L2 cache.

[0178] Optionally, the current mode of the L1 cache of the GPU is a fine-grained mode. In the fine-grained mode, the L1 cache uses cache blocks as storage granularity. The L1 cache subsystem further includes a mode switching module, configured to:

[0179] Determine a current target mode based on the sampling value, where the target mode includes a fine-grained mode or a coarse-grained mode; in the coarse-grained mode, the L1 cache stores granularity in cache behavior; the sampling value includes the number of misses and the transaction volume generated when the misses are counted in the fine-grained mode and the coarse-grained mode, respectively, within a preset period; and switch the current mode to the target mode.

[0180] Optionally, the L1 cache includes a first subset, a second subset and a subordinate subset, the first subset and the second subset are any two groups pre-selected from the L1 cache, and the subordinate subset includes other groups in the L1 cache except the first subset and the second subset; the first subset fixedly uses the fine-grained mode, and the second subset fixedly uses the coarse-grained mode; the first subset includes a first counter and a second counter, which are respectively used to count the number of misses of each cache line in the first subset in the fine-grained mode and the transaction volume generated when the miss occurs within a preset period; the second subset includes a first counter and a second counter, which are respectively used to count the number of misses of each cache line in the second subset in the coarse-grained mode and the transaction volume generated when the miss occurs within a preset period; the current mode of the subordinate subset is switched to the target mode when the preset period arrives.

[0181] The graphics processor GPU proposed in the embodiment of the present invention further divides each cache line in the L1 cache of the GPU into cache blocks. The memory access request generated by the thread in the process of executing the computing task is still generated at the granularity of the cache line, and the cache line access request is generated. The L1 cache of the GPU responds to the cache line access request, extracts the tag information in the target memory address, including the shared tag, private tag, group index, inter-block offset and intra-block offset, and queries whether the cache line access request hits the L1 cache based on the tag information. If the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed on the target cache block at the target data location. By subdividing the cache lines and the tag information, the embodiment of the present invention realizes coarse-grained access to the L1 cache and fine-grained storage, which can fully utilize the storage space of the cache lines and reduce the waste of cache space on the basis of ensuring data access efficiency.

[0182] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] Reference Figure 5 , is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, the electronic device includes: a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the memory access method of the aforementioned embodiment.

[0184] An embodiment of the present invention provides a non-transitory computer-readable storage medium. When instructions in the storage medium are executed by a program or processor of a terminal, the terminal is enabled to perform the steps of the memory access method of the aforementioned embodiment.

[0185] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0186] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, graphics processors, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0188] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a predictable manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0190] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0191] Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A memory access method, characterized in that: Applied to a GPU, the L1 cache of the GPU is divided into a plurality of groups, each group contains a first number of cache lines, and each cache line is divided into a second number of cache blocks. The method includes: receiving a cache line access request, where the cache line access request is obtained by aggregating memory access requests in a target warp based on a size of a cache line; Extracting tag information from a target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset; querying whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine the target group, the shared tag and the private tag are used to determine the target cache line in the target group, and the inter-block offset is used to determine the target cache block in the target cache line; If the cache line access request hits the L1 cache, a target data location is determined in the hit target cache block based on the intra-block offset, and a corresponding memory access request is executed based on the target data location.

2. The method according to claim 1, characterized in that The size of a cache line is 128 bytes, the size of a cache block is 32 bytes, the target memory address is 32 bits, and the target memory address includes an 11-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 5-bit intra-block offset; or, The size of a cache line is 64 bytes, the size of a cache block is 16 bytes, the target memory address is 32 bits, and the target memory address includes a 12-bit shared tag, an 8-bit private tag, a 6-bit group index, a 2-bit inter-block offset, and a 4-bit intra-block offset.

3. The method according to claim 1, characterized in that The method further comprises: generating a block access mask code according to the inter-block offset, each bit of the block access mask code being used to indicate a target cache block; The querying, based on the tag information, whether the cache line access request hits the L1 cache includes: If it is found that the group index, shared tag, and private tag all hit the L1 cache, determining the candidate cache block that hits; Matching the target cache block indicated by the block access mask code with the candidate cache block; If all target cache blocks indicated by the block access mask code exist in the candidate cache blocks, determining that the cache line access request hits the L1 cache; If part of the target cache blocks indicated by the block access mask code exists in the candidate cache blocks, determining that the cache line access request partially misses the L1 cache; If all target cache blocks indicated by the block access mask code do not exist in the candidate cache blocks, it is determined that the cache line access request misses the L1 cache.

4. The method according to claim 1, wherein The method further comprises: If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, querying the L2 cache based on the target memory address; If it is found that the cache line access request hits the L2 cache, the cache line that hits is read from the L2 cache, and a first cache block is extracted from the cache line that hits, where the first cache block includes the cache block that is not hit by the L1 cache; A second cache block is determined in the L1 cache, and the first cache block is filled into the second cache block in the L1 cache.

5. The method according to claim 4, characterized in that Determining the second cache block in the L1 cache includes: Searching the L1 cache for an invalid cache block that satisfies a first condition, wherein the first condition includes: all cache blocks in the same group as the invalid cache block are invalid, or a shared tag of the invalid cache block matches a shared tag of the first cache block; If there is an invalid cache block that meets the first condition, the invalid cache block that meets the first condition is determined to be the second cache block.

6. The method according to claim 5, characterized in that The method further comprises: If the number of invalid cache blocks that meet the first condition is less than the number of the first cache blocks, determining the remaining required number; Searching the L1 cache for a valid cache block that satisfies a second condition, wherein the second condition includes: the valid cache block has not been accessed recently; If there are valid cache blocks that meet the second condition in the L1 cache, and the number of valid cache blocks that meet the second condition is greater than or equal to the remaining required number, determining the remaining required number of valid cache blocks from the valid cache blocks that meet the second condition as second cache blocks; If there is no valid cache block meeting the second condition in the L1 cache, or the number of valid cache blocks meeting the second condition is less than the remaining required number, a valid cache block is randomly selected from the L1 cache as the second cache block.

7. The method according to claim 1, characterized in that The method further comprises: If the cache line access request misses the L1 cache, or if the cache line access request partially misses the L1 cache, checking whether a miss request entry corresponding to the cache line access request already exists in the miss request entries recorded in the miss status processing register; If a miss request entry corresponding to the cache line access request already exists, waiting for completion of processing of the miss request corresponding to the cache line access request; If there is no miss request entry corresponding to the cache line access request, creating a miss request entry corresponding to the cache line access request, and adding a miss block mask code to the created miss request entry, where each bit of the miss block mask code is used to indicate a cache block of an L1 cache miss; A miss request carrying the miss block mask code is initiated to the L2 cache to request that the cache block indicated by the miss block mask code be returned from the L2 cache.

8. The method according to claim 1, characterized in that The current mode of the L1 cache of the GPU is a fine-grained mode. In the fine-grained mode, the L1 cache uses a cache block as a storage granularity. The method further includes: Determining a current target mode based on the sampled values, the target mode including a fine-grained mode or a coarse-grained mode; in the coarse-grained mode, the L1 cache stores granularity in cache behavior; the sampled values ​​including the number of misses and the transaction volume generated when the misses are counted within a preset period in the fine-grained mode and the coarse-grained mode, respectively; Switch the current mode to the target mode.

9. The method according to claim 8, characterized in that The method further comprises: Select any two groups in the L1 cache of the GPU, one of which is used as a first subset and the other as a second subset; Within a preset period, counting the number of misses and the transaction volume generated when the misses occur for each cache line in the first subset and the second subset respectively; The switching the current mode to the target mode includes: Switching a current mode of a dependent subset in an L1 cache of the GPU to the target mode, the dependent subset including other groups in the L1 cache except the first subset and the second subset.

10. A graphics processing unit (GPU), characterized in that: The GPU includes a stream multiprocessor and a memory system, wherein the memory system includes a private L1 cache subsystem of the stream multiprocessor and a memory aggregation unit; the L1 cache subsystem includes an L1 cache and a first cache controller; the L1 cache is divided into a plurality of groups, each group includes a first number of cache lines, and each cache line is divided into a second number of cache blocks; wherein, The memory aggregation unit is configured to aggregate memory access requests in the target warp based on a size of a cache line, generate a cache line access request, and send the cache line access request to the L1 cache subsystem; The first cache controller is used to receive a cache line access request; extract tag information from a target memory address corresponding to the cache line access request, the tag information including: a shared tag, a private tag, a group index, an inter-block offset, and an intra-block offset; query whether the cache line access request hits the L1 cache based on the tag information; wherein the group index is used to determine a target group, the shared tag and the private tag are used to determine a target cache line in the target group, and the inter-block offset is used to determine a target cache block in the target cache line; if the cache line access request hits the L1 cache, the target data location is determined in the hit target cache block based on the intra-block offset, and the corresponding memory access request is executed based on the target data location.

Images