Computing system and semiconductor integrated circuit module

Abstract

Provided is a semiconductor integrated circuit module including DRAM caches in a stacked structure, capable of preventing an increase in tag memory capacity on a processor side, reducing hit latency, and improving hit rate. A computing system (1000) is provided with a CPU (100), a main memory (300), and a cache DRAM circuit (400) stacked on the CPU (100). The cache DRAM circuit (400) operates as a group-connected cache memory. The address space of the main memory (300) is divided into a plurality of first sections, the address space of the cache DRAM circuit (400) is divided into a plurality of second sections in one-to-one correspondence with the first sections, and each second section comprises a plurality of paths. The cache controller (111) uniformly accesses rows in a plurality of paths corresponding to the address of the accessed main memory (300).

Description

Technical Field

[0001] This invention relates to computing systems and semiconductor integrated circuit modules, and more specifically, to the structure of a cache memory using stacked dynamic random access memory. Background Technology

[0002] (Stacked semiconductor integrated circuit module)

[0003] Previously, volatile memory (RAM) such as DRAM (Dynamic Random Access Memory) was known as a storage device. DRAM required larger capacities to withstand the increasing performance and data volume of computing devices (hereinafter referred to as logic chips). Therefore, there has been a continuous pursuit of increasing capacity through miniaturization of memory (memory cell arrays, memory chips) and planar expansion of cells. On the other hand, this capacity expansion has reached its limit due to factors such as increased vulnerability to noise and increased die area resulting from miniaturization.

[0004] Therefore, a technique for stacking multiple planar memories to achieve 3D rendering and thus increase capacity has recently been developed. Furthermore, with the increasing volume of data, there is a desire to achieve high-speed data communication between chips (logic chips and memory chips). Semiconductor modules, for example, in which logic chips and DRAM are overlapped are known (see, for example, Non-Patent Literature 1).

[0005] For example, in such 3D semiconductor integrated circuit modules, there are known 3D or 2.5D packaging technologies that mount multiple IC dies (IC chips) on a Si interposer using TSV (Through Silicon Via) technology. Based on such mounting technologies, high-density wiring can be formed on multiple IC dies (IC chips), enabling miniaturization, thin-film fabrication, and widebandwidth.

[0006] However, from the perspective of reducing manufacturing costs, Patent Document 1 discloses a 3D or 2.5D mounting technique that uses a redistribution layer (RDL) and small Si bridging dies instead of such a Si interposer, and mounts multiple IC dies (IC chips) on them. IC modules that use a redistribution layer to mount multiple IC dies (IC chips) in this way are mostly manufactured using FOWLP (Fan Out Wafer Level Package) technology. According to this packaging technology, high-density wiring can be formed on multiple IC dies (IC chips) relatively inexpensively, enabling miniaturization, thin-film construction, and wide bandwidth.

[0007] However, from the perspective of high speed and high performance, for example, in the aforementioned non-patent document 1, a structure is disclosed that uses so-called "hybrid bonding technology" to stack the dies of DRAM circuits and the dies of logic circuits.

[0008] In other words, by using a so-called flip-chip construction, interconnect electrodes are arranged in a 2D array in a direction perpendicular to the silicon surface (or substrate surface), stacking silicon wafers or dies manufactured using different process technologies. This allows multiple functions to be housed in a single package. Compared to integrating multiple functions with different processes on a single silicon die (a single die), this allows for the installation of a large number of functions at a low cost. The interconnect technology used when stacking silicon is crucial in 3D stacking. To improve data transmission per unit area, high-density interconnection of different silicon chips is required. Therefore, a "hybrid bonding technology" is used, where copper (Cu) electrodes formed on the surfaces of silicon wafers, dies, etc., are bonded together.

[0009] On the other hand, in arithmetic circuits (logic circuits), to avoid the time required for data interaction with main memory being limited by the access time of main memory (usually DRAM), so-called cache memory is used. Historically, SRAM (Static Random Access Memory), which is faster than DRAM, has been widely used in this context.

[0010] However, in recent years, for logic circuits, block-based cache memories using DRAM have begun to be used in a stacked configuration as described above. In block-based cache memories using DRAM, the capacity of the tag memory on the processor side is typically increased significantly. As a countermeasure, a method has been proposed (for example, see Patent Document 2) in which tags are also attached to the DRAM side, multiple tag-data pairs are configured in a DRAM row (ROW), and when an access request arrives from the processor, a row is selected from the entry address, the tag is read first and compared with the requested address, and if a match is found, the data that is paired with that tag is read.

[0011] While improving the accuracy of the mission by increasing the relevance through the above methods, the number of roads increases, the number of tags read and the number of address comparisons increase, and the latency increases.

[0012] Therefore, as a countermeasure, a method has been proposed to pre-cache the possible hit road IDs in the road predictor within the controller and compare them with the road predictor when an access request arrives (see, for example, non-patent literature 2).

[0013] Alternatively, as another countermeasure, a method has been proposed that instead reduce the correlation to 2, and then add a path location cache using SRAM in the controller to predict the hit path, thereby shortening the access latency (see, for example, non-patent literature 3).

[0014] Existing technical documents

[0015] Patent documents

[0016] Patent Document 1: Description of U.S. Patent Application Publication No. 2020 / 0185367;

[0017] Patent Document 2: Japanese Patent Application Publication No. 2022-151611.

[0018] Non-patent literature

[0019] Non-patent literature 1: Xiping Jiang, Student Member, IEEE, Fengguo Zuo, Song Wang, Xiaofeng Zhou, Yubing Wang, Qi Liu, Qiwei Ren, Senior Member, IEEE, and Ming Liu, "A1596-GB / s 48-Gb Stacked Embedded DRAM 384-Core SoC With Hybrid Bonding Integration", IEEE SOLID-STATE CIRCUITS LETTERS,VOL.5,2022,pp.110-113;

[0020] Non-patent document 2: Building a Low Latency, Highly Associative DRAM Cache with the Buffered Way Predictor International Symposium on Computer Architecture and High Performance Computing, (2016);

[0021] Non-patent document 3: Trade-off Between Hit Rate and Hit Latency for OptimizingDRAM Cache IEEE Transactions on Emerging Topics in Computing (Volume: 9, Issue: 1, Jan.-Mar. 2021). Summary of the Invention

[0022] The problem the invention aims to solve

[0023] However, in the technology disclosed in Non-Patent Document 2, the structure of the caching system becomes complex, and it is impossible to reduce the access latency when the route ID is mishit.

[0024] Furthermore, in the technology disclosed in Non-Patent Document 3, there is a problem that the hit rate is not as high as that of methods with a large number of paths.

[0025] The present invention was made to solve the aforementioned problems, and its purpose is to provide a semiconductor integrated circuit module that can prevent the increase of the tag memory capacity on the processor side, reduce hit latency, and improve hit rate in a block-based DRAM cache constructed on logic circuits such as processors.

[0026] Solution for solving the problem

[0027] According to one aspect of the present invention, a computing system is provided, comprising: a main memory for storing data required for computing processing; a DRAM circuit; and a processor circuit stacked in the same package as the DRAM circuit. The processor circuit includes one or more processor cores and a cache controller. The DRAM circuit is controlled by the cache controller and operates as a cache memory in a group-connected manner between the processor and the main memory. The address space of the main memory is divided into a plurality of first segments, and the address space of the cache memory is divided into a plurality of second segments corresponding one-to-one with the first segments. Each second segment includes a plurality of paths. In response to the processor's read access to data in the main memory, the cache controller performs a unified access to rows in the plurality of paths corresponding to the address of the accessed main memory. If there is data that has been hit, data is loaded from the hit row.

[0028] Preferably, the interface of the cache controller to the DRAM circuit side and the interface of the DRAM circuit die to the processor side correspond one-to-one for each path.

[0029] Preferably, the main memory address included in the access request from the processor circuitry includes a segment address for specifying a segment and an entry address located in the middle of the address as an index. The multiple paths each include multiple groups of data and tags from the main memory read units. The cache controller controls the process as follows: selects a second segment based on the segment address, selects the corresponding tag in each of the multiple paths based on the entry address, and reads the data grouped with the selected tag.

[0030] According to another aspect of the present invention, a semiconductor integrated circuit module is provided, which has multiple chips stacked on top of each other. The semiconductor integrated circuit module has: a first integrated circuit chip including a processor for performing arithmetic processing, the processor including a cache controller; and a second integrated circuit chip stacked on top of the first integrated circuit chip, and including at least one die of a DRAM circuit connected in a manner capable of data transmission and reception with the processor. The DRAM circuit is controlled by the cache controller and operates as a cache memory in a group-association manner between the processor and the main memory. The address space of the main memory is divided into multiple first segments, and the address space of the cache memory is divided into multiple second segments, each corresponding to one of the first segments. Each second segment includes multiple paths. In response to a read access to data in the main memory from the processor, the cache controller performs a unified access to rows in the multiple paths corresponding to the address of the accessed main memory, and loads data from the hit row if a hit exists.

[0031] Preferably, the interface of the cache controller to the DRAM circuit side and the interface of the DRAM circuit die to the processor side correspond one-to-one for each path.

[0032] Preferably, the main memory address included in the access request from the processor includes a segment address for specifying a segment and an entry address located in the middle of the address as an index. The multiple paths each include multiple groups of data and tags from the main memory read units. The cache controller controls the process as follows: selects a second segment based on the segment address, selects the corresponding tag in each of the multiple paths based on the entry address, and reads the data grouped with the selected tag.

[0033] Preferably, the DRAM circuit has multiple segment dies configured in the second integrated circuit chip corresponding to multiple segments, and the multiple segment dies are respectively divided into path dies corresponding to multiple paths.

[0034] Preferably, each of the die blocks includes: a memory array; electrodes corresponding to address pins for accessing the memory array; electrodes corresponding to command pins for determining the type of access to the memory array; and electrodes corresponding to data input / output pins for reading data from the memory array.

[0035] Preferably, the data input / output pins correspond to the number of bits of data and tags that can be read within a burst period.

[0036] Preferably, the first integrated circuit chip and the second integrated circuit chip are connected by hybrid bonding in a flip-chip configuration.

[0037] According to another aspect of the present invention, a computing system is provided, comprising: a main memory module; and a semiconductor integrated circuit module having multiple chips stacked thereon for performing arithmetic processing based on data and instructions stored in the main memory module. The semiconductor integrated circuit module comprises: a first integrated circuit chip including a processor and a cache controller for performing arithmetic processing; and a second integrated circuit chip stacked on the first integrated circuit chip and including at least one die of a DRAM circuit connected to the processor in a manner capable of data transmission and reception. The DRAM circuit is controlled by the cache controller and operates as a cache memory in a group-association manner between the processor and the main memory. The address space of the main memory is divided into multiple first segments, and the address space of the cache memory is divided into multiple second segments corresponding one-to-one with the first segments. Each second segment includes multiple paths. In response to a read access to data in the main memory from the processor, the cache controller performs a unified access to rows in the multiple paths corresponding to the address of the accessed main memory. If data is hit, data is loaded from the hit row. The main memory module includes an extended main memory module, which is controlled by an extension controller and connected to the semiconductor integrated circuit module via a bus interface.

[0038] Invention Effects

[0039] The semiconductor integrated circuit module according to the present invention can prevent the increase of the tag memory capacity on the processor side, reduce hit latency, and improve hit rate in a block-based DRAM cache constructed on a processor. Attached Figure Description

[0040] Figure 1 This is a functional block diagram illustrating the structure of an arithmetic system 1000, which includes a CPU, main memory, and cache memory.

[0041] Figure 2 This diagram illustrates the relationship between cache DRAM as a silicon wafer stacked on the die of CPU100, which serves as a processor circuit.

[0042] Figure 3 A cross-sectional view showing the structure of the CPU100 with stacked processor circuitry and cache DRAM.

[0043] Figure 4 This is a diagram illustrating the existing direct access method.

[0044] Figure 5 This is a schematic diagram illustrating the Set Associative method.

[0045] Figure 6 This is a schematic diagram illustrating the structure of a cache controller in a group-associative cache system.

[0046] Figure 7 This is a schematic diagram illustrating the segment structure and allocation example between the cache DRAM 400 and the main memory in this embodiment.

[0047] Figure 8 This is a schematic diagram illustrating an example of the segment structure within a cache memory.

[0048] Figure 9 This is a schematic diagram illustrating the layout of the cache DRAM400.

[0049] Figure 10 This is a schematic diagram illustrating an example of cache memory operation.

[0050] Figure 11 A timing diagram to represent the access patterns of the bare dies.

[0051] Figure 12 This diagram illustrates the system structure of Embodiment 2 of the present invention. Detailed Implementation

[0052] The structure of the semiconductor integrated circuit module according to embodiments of the present invention will be described below. It should be noted that in the following embodiments, structural elements and processing steps marked with the same reference numerals are the same or equivalent, and their description will not be repeated unless necessary.

[0053] The following is an example of a semiconductor integrated circuit module, in which a DRAM structure that functions as a cache memory is stacked on top of the CPU (Central Processing Unit), which is a logic circuit (more specifically, a processor circuit).

[0054] [Implementation Method 1]

[0055] (Functional structure of a semiconductor integrated circuit module)

[0056] Figure 1 This is a functional block diagram illustrating the structure of an arithmetic system 1000, which includes a CPU, main memory, and cache memory.

[0057] Reference Figure 1 The CPU100 has a control unit 110 and an arithmetic unit 150.

[0058] The program counter 114 in register 112 is a register used to store the address in memory where the next instruction to be executed is stored, indicating which address in memory the next instruction to be executed is stored. The base address register 116 stores the starting address when the program is loaded into memory.

[0059] Index register 118 in register 112 is used to fetch consecutive data and stores the relative position from the start. It can be used when repeatedly applying the same instruction to consecutive data such as arrays. Instruction register 120 in register 112 is used to temporarily store the fetched instructions.

[0060] The control unit 110 uses these registers to read instructions and data from the memory, or to perform data writing and other processes.

[0061] The arithmetic unit 150 performs the specified bitwise operations on the read data according to the command. The accumulator 154 in register 112 is a register used to temporarily store the operation results, and the general-purpose register 156 is a register without a particularly limited function and can be used for various purposes as needed.

[0062] In addition, although the arithmetic unit 152 is in Figure 1 It records one, but it can also have multiple cores.

[0063] CPU 100 also has multiple levels of cache memory 102, 104, and 106. For example, cache memory 102 is a level 1 (L1) cache memory, cache memory 104 is a level 2 (L2) cache memory, and cache memory 106 is a level 3 (L3) cache memory. Cache memories 102, 104, and 106 are installed as SRAM (Static Random Access Memory) on the same die as the arithmetic unit 152. Furthermore, the number of SRAM cache levels on the same die is not limited to this structure.

[0064] CPU100 also has an external interface 108 for sending and receiving data with the outside world.

[0065] CPU100 is connected to DRAM400, which is stacked in three dimensions on the CPU100 chip, via external interface 108. While DRAM400 is not specifically defined, it functions as a Level 4 (L4) cache memory. Therefore, DRAM400 is structured at the Last Level Cache (LLC) level. Hereinafter, DRAM400 will be referred to as cache DRAM400.

[0066] As will be described later, the cache configuration of the cache DRAM400 is equivalent to the set-associative configuration (hereinafter also referred to as "SetAssociative").

[0067] In addition, the CPU 100 transmits and receives data with the main memory 300 via the external interface 108. The memory cells MCI in the memory cell array MCA within the main memory 300 are accessed from the outside through predetermined addresses.

[0068] The control unit 110 has a cache controller 111. The cache controller 111 intercepts access to memory from the CPU 100 and performs control over access to cache memories 102, 104, 106, cache DRAM 400, and main memory 300.

[0069] Figure 2 This diagram illustrates the relationship between cache DRAM and the bare die when the CPU100 is stacked as a silicon wafer in the processor circuitry.

[0070] In this embodiment, although CPU100 is used as an example to describe the processor circuit, the structure is not limited to this as a logic circuit.

[0071] For example, in Type A, the CPU100, which is the processor circuit, has a die size of 1×1, based on the die size of the DRAM. In the cache DRAM wafer, the DRAM circuitry is also divided using scribe lines according to the die size of the processor circuitry. In Type A, the CPU100 wafer and the DRAM wafer are constructed using a flip-chip structure, stacked and electrically coupled using, for example, a hybrid bonding technique.

[0072] Subsequently, the CPU100 is cut into individual wafers according to its chip size, thereby enabling the generation of stacked chips with a bare die size of 1×1, based on the DRAM chip size. Furthermore, the number of stacked wafers is not limited to two; as shown in the figure, more can also be used.

[0073] Similarly, in Figure 2 In this context, the CPU100, which serves as the processor circuit, is of type B, with the die size set to 2×2 based on the die size of the DRAM.

[0074] In Type B, the CPU100 is also cut into individual chips based on its chip size, thereby enabling the generation of chips with a 2×2 die size stacked structure based on the DRAM chip size.

[0075] Through this Figure 2Such a structure, for example, even if Type A and Type B are designed and manufactured by different processor manufacturers and have different die sizes, can be addressed by manufacturing DRAM with the same design and the same process, as long as the length of each side is an integer multiple of 1, such as 1×1, 1×2, 2×2.

[0076] That is, when manufacturing stacked semiconductor integrated circuit modules, only one type of cache DRAM is needed.

[0077] Furthermore, in Type B, for example, an example was illustrated where the die size of the processor circuitry is equivalent to four 512MB cache DRAMs. In this case, even with four times the capacity of the cache DRAM, it can be a structure of four private caches corresponding to four CPU clusters. In the latter case, four CPUs correspond to four separate cache DRAMs.

[0078] Furthermore, as mentioned above, cache DRAM wafers are not limited to a single layer and can be further stacked.

[0079] Figure 3 A cross-sectional view showing the structure of a CPU100 with stacked processor circuitry and a cache DRAM400.

[0080] Figure 3 The structure implemented using the flip chip structure disclosed in Non-Patent Document 1 described above, with "hybrid bonding technology" as an example.

[0081] In other words, the cache DRAM and the processor circuitry are combined and housed in the same package through a flip-chip construction.

[0082] Here, hybrid bonding 401 refers to a technique that involves bonding copper (Cu) electrodes formed on the surface of silicon wafers, silicon dies, etc., together.

[0083] For example, in the interconnection of silicon wafers, electrodes can be connected with an extremely short spacing of 5μm to 1μm.

[0084] In hybrid bonding, solder bumps are not used; the surface of the semiconductor chip consists of Cu electrodes and an insulating film. The surface of the Cu electrodes is formed into a recessed shape, similar to a cutlery plate. The depth of the recess is, for example, less than 20 nm. Moreover, only a tiny amount, such as 0.5 nm, is removed from the surface of the insulating film.

[0085] The Cu electrode side surfaces of the silicon, after this pretreatment, are brought into contact with each other. This causes the insulating films to connect. Tiny gaps remain between the opposing Cu electrodes. Then, when heat treatment is performed while the silicon electrodes are pressed together, the opposing Cu electrodes expand and contact each other, forming a bond through mutual diffusion.

[0086] Building upon this, the logic circuitry is connected to the resin-based packaging substrate 700 on the back side using TSV (Through Silicon Via) technology, employing a solder ball connection method based on C4 (Controlled Collapse Chip Connection) bumps 600. For example, a multi-layer wiring circuit is formed on the surface (upper side) of the packaging substrate 700, and is connected to the multi-layer wiring circuit on the back side (lower side) through through-holes 702.

[0087] The back side of the package substrate 700 is connected to the motherboard (not shown) for example via a BGA (Ball Grid Array) 704.

[0088] In addition, although there are no particular limitations, in order to deal with the problem of thermal stress, a structure that improves thermal conductivity can also be used in the substrate design by using dummy bumps.

[0089] Furthermore, although the above description states that the CPU300 and the cache DRAM400 are formed by wafer-on-wafer stacking, the stacking method of semiconductor integrated circuits is not limited to such a structure. For example, it can also be a chip-on-wafer stack, a die-on-wafer stack, a chip-on-chip stack, or a die-on-die stack.

[0090] Furthermore, although it is stated that the electrical connection between the CPU300 and the cache DRAM400 is a hybrid bonding, bump bonding could also be used, for example.

[0091] Furthermore, the cache DRAM 400 can also be stacked in two or more layers. In this case, as disclosed in Patent Document 2, multiple layers that become DRAM cells are interconnected through holes penetrating these multiple layers.

[0092] Furthermore, the cache DRAM400 can also be equipped with a fault bit recovery mechanism and an ECC (Error Correcting Code) mechanism to improve the yield.

[0093] (Cache configuration methods for cache memory)

[0094] The following explanation, as a prerequisite for describing the cache configuration method of this embodiment, will cover the existing direct mapping method and the existing group association method.

[0095] Figure 4This is a diagram illustrating the existing direct access method.

[0096] The size of a unit of data stored in the cache is called a "cache line", and its size is called the "cache line size", or simply "line size".

[0097] Increasing line size reduces the number of data transfers to memory, making it efficient in this respect. However, conversely, it increases the amount of data transferred from main memory that is not used, thus reducing the efficiency of cache memory utilization.

[0098] On the other hand, when considering a fixed amount of cache memory, if the row size is increased, the number of rows that can be stored in the cache decreases inversely, and the cache hit rate decreases.

[0099] Therefore, for example, a row size of approximately 32 to 256 bytes is used.

[0100] exist Figure 4 The direct mapping method shown predetermines the location of cache rows based on the memory address. That is, the correspondence between main memory addresses and cache row locations is a fixed one-to-one relationship.

[0101] Specifically, a portion of the memory address called the "index" (the median address) is allocated to the address of a line in the cache memory.

[0102] For example, a structure that directly determines the row number using the memory block number (in the case of a 128-byte row, the high bits of the memory address after removing the low 7 bits).

[0103] exist Figure 4 In the main memory, the blocks shown as partitions are row-sized blocks, ordered as shown in the partition diagram. Since main memory is larger than the cache, a group of partitions in a horizontal column corresponds to a cache line.

[0104] The address of this cache line is called the index. As mentioned above, the middle part of the memory address after removing the least significant bit of the line size is the "index" part, which is used to directly select the cache line.

[0105] Direct mapping uses an "index" to read from the cache memory, thus offering advantages such as compact circuit size and high-speed operation.

[0106] On the other hand, while multiple rows of data with the same "index" can exist in memory, only one row can be stored in cache memory. Therefore, there is a drawback: in order to store data with the same "index" in a row of cache memory that is already filled with data, the previous data must be cleared and overwritten.

[0107] As a result, cache lines need to be replaced frequently. Depending on the situation, the performance may be even worse than without caching. This phenomenon is called thrashing.

[0108] Figure 5 This is a schematic diagram used to illustrate the way groups are connected.

[0109] The way to eliminate the drawbacks of the direct mapping method mentioned above is the group associativity method, which is currently the most common caching configuration method.

[0110] In group-associative mode, an "index" is also used to allocate a portion of the memory address (the median address) to the address of the cache line.

[0111] However, prepare multiple cached rows corresponding to the same "index". A group of cached rows corresponding to the same "index" is called a "way".

[0112] exist Figure 5 The diagram shows the construction of a 4-way set-associative cache.

[0113] and Figure 4 The direct mapping method is quite different in that the number of cache rows that can store memory blocks in a horizontal group increases to four.

[0114] Therefore, even if instructions exist in block 0 and data exists in block n within the same horizontal group, since there are multiple corresponding cache lines, they can be stored simultaneously without the aforementioned thrashing.

[0115] In group-association, to reduce the probability of thrashing, it is necessary to increase the number of cache lines that store the memory blocks of the main memory in horizontal groups. The number of these horizontal cache lines is the "path" mentioned above.

[0116] Although Figure 5 The standard is 4 channels, but sometimes 8 channels, 16 channels, or even 6 or 10 channels (not powers of 2) are used.

[0117] Figure 6 This is a schematic diagram illustrating the structure of a cache controller in a group-associative cache system.

[0118] exist Figure 6 The text indicates the group connection configuration of 4 channels.

[0119] caching is related to Figure 5 The structure is quite similar, with four groups for both the data array and the tag array.

[0120] When a request address is input from CPU100, the four tag arrays are accessed using an index based on the entry address used to select the cache. The tags of the four data arrays are read and compared with the tags of the request address using a comparator. The search is performed associatively within the four cached data sets; hence, this is called set-associative mode.

[0121] The comparator compares the data, and if any tag in the four groups matches, a hit is achieved. A logical AND operation is performed on the four comparison results to generate a hit signal. Furthermore, regarding the data array, word selectors and path selectors are used to read the data corresponding to the hit and output it to the CPU100.

[0122] Generally, from the perspective of power consumption and access time, in small caches such as L1 caches that require high-speed access, parallel access to the tag array and data array can be used. On the other hand, in large-capacity L2, L3, and L4 caches, power consumption becomes a greater issue than the increased access time. In such cases, a structure can be adopted that accesses only the data array of the hit path after confirming a tag hit.

[0123] Figure 7 This is a schematic diagram illustrating the segment structure and allocation example between the cache DRAM 400 and the main memory 300 in this embodiment.

[0124] exist Figure 7 As an example, let's assume that the main memory 300 has a storage capacity of 512GB and the cache DRAM 400 has a storage capacity of 512MB, and then explain.

[0125] like Figure 7 As shown, the address spaces of the main memory 300 and the cache DRAM 400 are divided into the same number of segments.

[0126] Furthermore, between the main memory 300 and the cache DRAM 400, by directly mapping the same segments to each other, a structure corresponding to the internal access requests of the cache memory segments is formed.

[0127] As a result, sections other than those corresponding to the access request do not work, which reduces power consumption.

[0128] By adopting such a structure, it is possible to form a structure that reads the tags and data of all paths in the cache memory corresponding to the access request from the CPU100, and processes them by the cache controller.

[0129] In other words, to summarize the above structure, in this embodiment, the main memory space and the cache memory space are divided into the same number of segments between the main memory 300 and the cache DRAM 400, and the segments are directly mapped.

[0130] Furthermore, the corresponding segments are mapped to each other by an N-way set-associative cache structure (N: natural number).

[0131] Here, in the segment cache, a large-capacity DRAM is used as the cache memory, forming a structure that increases N to 16 and accesses N paths simultaneously, as described later. Therefore, the hit rate is higher, and the path ID cache is not required as in the prior art (non-patent document 2).

[0132] In addition, in the cache controller, access requests for different segments can be issued simultaneously or interleaved.

[0133] In addition, it can also be configured as a cache DRAM path for additional devices. If there is a faulty path, it can be replaced with a normal path by switching the processor die.

[0134] Furthermore, the processor can also use the cache controller to access multiple cache segments in parallel and independently. For example, this parallel and independent access is possible when the main memory segment corresponds to a different main memory segment than the consecutively issued read requests, or when multiple processor cores are exclusively allocated to multiple main memory segments.

[0135] Figure 8 This is a schematic diagram illustrating an example of the segment structure within a cache memory.

[0136] exist Figure 8 In, with Figure 7 Correspondingly, the number of paths is 16, and the address space of the 512MB cache DRAM400 is divided into 32MB segments.

[0137] like Figure 8 As shown, in the address of the access request from CPU100, a group is selected from all 16 channels by the entry address of 14b, and 128B of data is read in each channel.

[0138] The cache controller 111 also reads the address of the 14b frame within the tag and compares it with the frame address of the access request.

[0139] In addition to the frame address of the corresponding data, the tag may also include a flag, ECC parity bit, etc.

[0140] The cache controller 111 sends data to the CPU 100 based on the data of the hit path, with a cache line size specified by the word address.

[0141] For example, with a cache line size of 64B, 64B of data in 128B is selected by using the high-order 1b of the word address.

[0142] In the event of a cache miss where no path is hit, the cache controller 111 accesses the main memory 300 and reads the data via the requested address.

[0143] That is, as Figure 5 The differences between the existing group connection method shown and the group connection method of this embodiment are as follows.

[0144] That is, in Figure 5 In the existing group-connected configuration shown, memory access and indexing are performed via memory access from CPU 100. Figure 8 All paths corresponding to the entry address in the main memory (equivalent to the address of the entry in the main memory), and in this embodiment, the access is formed through memory access from the CPU 100, accessing and indexing (in Figure 8 The structure represents all paths corresponding to the segment address plus the main memory entry address. As a result, access to the corresponding segment in cache DRAM400 can be selectively performed.

[0145] In addition, Figure 1 In this example, although it's a structure where a cache controller 111 controls all cache segments, it could also be a structure where controllers are set up for each cache segment. In the latter case, the controller that is in operation is selected based on the segment address.

[0146] Figure 9 This is a schematic diagram illustrating the layout of the cache DRAM400.

[0147] like Figure 9 As shown, with a cache DRAM400 storage capacity of 512MB, for example, it is divided into 16 segments, each segment having a capacity of 32MB.

[0148] In this case, the 16-segment die in the cache DRAM400 is configured as 4×4.

[0149] Furthermore, the 32MB segment of the bare die, for example, like... Figure 8 As shown, with 16 paths, it is divided into 16 bare lithography disks, each 2MB in size.

[0150] In this case, the 16 raw chips are configured in a 4×4 configuration. Alternatively, the configuration of the 16 segment raw chips and the 16 raw chips can be any configuration other than 4×4.

[0151] Each 2MB die is equipped with a memory array, a sense amplifier, and control circuitry. The pins for input commands and address signals are configured on one side, while the (128+8)b data input / output pins (DQ pins) are configured on the other side.

[0152] With this structure, a cache DRAM400 can be scalably configured relative to the CPU100.

[0153] That is, a path die is a structure that can work independently, and a segment die and cache DRAM are a collection of such path dies.

[0154] Therefore, the size of the segment die and the cache DRAM die can be arbitrarily constructed from the road dies, and the structure of the segment die and the structure of the cache memory die can be arbitrarily selected on a per-road die basis.

[0155] Each pin of the bare die is an interface to the circuit controller within the stacked CPU100. The interface connection is as described above, and hybrid bonding or Au bump bonding, etc., can be used.

[0156] Therefore, in the above structure, a unit DRAM circuit with multiple dies, designed to be accessed independently and with a wide bit width for each data input and output, is configured to be allocated to multiple paths.

[0157] The capacity of a single DRAM circuit is 2MB, and the structure of the DQ pin is 128B of data + 8B of tags (=136B (1088b=136DQ×8 burst)).

[0158] Moreover, with this structure, the cache DRAM operates in a random access manner with non-reused addresses.

[0159] That is, since the bit width of the address pin can also ensure the required number of bits, as described later, the address non-multiplexing method of using a one-time input address signal can complete the reading or writing of 136B of data in a single access, as well as pre-charging for the next access and random access operation.

[0160] Figure 10 This is a schematic diagram illustrating an example of cache memory operation.

[0161] Although with Figure 1 The structures shown are basically the same, but as the on-chip cache of CPU100, it is only configured to the L2 cache.

[0162] The control of the read operation of the cache controller 111 is described.

[0163] (1) When the corresponding data exists in the L1 or L2 cache memory

[0164] i) Hit the L1 or L2 cache memory.

[0165] ii) Load data from the L1 or L2 cache memory.

[0166] iii) In response to the access request from CPU100, return data.

[0167] (2) The corresponding data does not exist in the L1 or L2 cache memory, but exists in the cache DRAM.

[0168] i) Level 1 or Level 2 cache miss

[0169] ii) Cache DRAM hit

[0170] iii) Load data into cached DRAM

[0171] iv) In response to the access request from CPU100, return the data. Copy the data in the L1 cache and L2 cache.

[0172] v) When all rows in the L1 and L2 caches are filled, remove data from a row before copying to create a blank row.

[0173] Figure 11 A timing diagram to represent the access patterns of the bare dies.

[0174] Additionally, although there are no specific limitations, the clock frequency is, for example, 250MHz.

[0175] In addition, the burst length is set to 8 bits, and the access unit uses a (128+8) bit DQ pin as the data width of (data) + (tag), set to (128+8) bits.

[0176] Reference Figure 11 For data reading, at clock 0, a read command and address (= row address + column address) are provided. Starting from clock 6 after the delay period, data D0 to D7 are read sequentially according to the clock.

[0177] On the other hand, as part of the data writing process, at clock 0, a write command and address (= row address + column address) are provided. Starting from clock 3, data D0 to D7 are written sequentially according to the clock.

[0178] In addition, such as Figure 11 The recorded time sequence can also be used to perform refresh operations.

[0179] A more detailed explanation is provided below.

[0180] The commands are read, write, and refresh, and the index (address) must be entered at the same time.

[0181] Instead of performing the usual DRAM operations such as address non-multiplexing and page mode access, it adopts a structure dedicated to random access.

[0182] exist Figure 11 In the example, reading and writing are completed in 16 clock cycles (64 ns), and the next command is then processed.

[0183] The refresh is completed in 8 clock cycles (32ns) and then the next command is accepted.

[0184] A single read access is performed from the (128+8)-bit DQ pin in 8 bursts totaling (1024+64) bits (=(128+8) bytes).

[0185] A single write access is performed from the (128+8)-bit DQ pin in 8 bursts totaling (1024+64) bits (=(128+8) bytes).

[0186] The refresh is performed by accessing the entire index using a refresh command within a fixed time interval.

[0187] With the structure described above, access to the cache DRAM can simultaneously access all corresponding segments of the path. Since the tag and data are accessed simultaneously in each path, hit latency overhead can be eliminated.

[0188] Furthermore, in cache DRAM, such as Figure 9 As explained in the text, increasing the number of paths is also easy in terms of technology, therefore, the cache hit rate can be improved by increasing the number of paths.

[0189] Furthermore, in the structure described in Non-Patent Document 2, the following structure is required: the road ID is cached in the road predictor within the controller in advance, and it is compared with the road ID first when an access request arrives. In contrast, in this embodiment, the hit rate is improved by adopting a structure that can read multiple roads simultaneously, and therefore, the overhead time of road ID prediction is not required.

[0190] Furthermore, by segmenting the memory space and using a direct mapping structure corresponding to each segment, power consumption can be reduced without excessively increasing the associativity.

[0191] [Implementation Method 2]

[0192] Figure 12 This diagram illustrates the system structure of Embodiment 2 of the present invention.

[0193] Figure 12Based on the "CXL (registered trademark) interface standard", an "accelerator + expansion main module" structure is also set up externally for the CPU100 and cache DRAM400 of Implementation Method 1.

[0194] For example, in recent years, the development of technologies such as artificial intelligence, machine learning, and cloud computing has generated massive amounts of data. In data-driven technologies that utilize this massive amount of data, in order to manufacture powerful processors capable of processing data-intensive applications and meeting performance requirements, more and more cores are being integrated into single-processor chips, and the development of heterogeneous computing continues to advance.

[0195] The increase in the number of CPU cores necessitates further improvements in memory bandwidth and density. Under these circumstances, the development of technologies capable of expanding main memory continues.

[0196] Here, "CXL (registered trademark) interface standard" refers to a new bus interface standard developed as a technology for interconnecting high-speed, low-latency CPUs and devices built on the PCIe physical layer.

[0197] like Figure 12 As shown, according to this technology, an efficient connection is achieved between the host CPU 100 and connected devices such as the accelerator 800 and the extended main memory module 900.

[0198] Accelerator 800 is designed to address the bottlenecks and capacity limitations of processor memory bandwidth, making more efficient use of intensive memory resources and enabling scale-up access without compromising performance.

[0199] The structure of CPU 100 and cache DRAM 400 described in Embodiment 1 can also work effectively in a system that operates according to the standard of such extended main memory modules.

[0200] exist Figure 12 In the structure, the access between CPU100 and cache DRAM400 is controlled by cache controller 111, and the access between CPU100 and main module is controlled by main memory controller.

[0201] For example, the graphics processor 700 is controlled by a bus controller.

[0202] The extended main memory module 900 is controlled by the expansion controller.

[0203] As the cache controller 111 operates, with Figure 1 The operation is basically the same as described above, but in the event of a cache miss in the cache memory of the cache DRAM400, an access to the main memory module or the extended main memory module is performed according to the address.

[0204] This structure, which employs an extended main memory module, allows for the expansion of the main memory address space from the perspective of the CPU100. This expansion can be achieved by increasing the capacity of the cache DRAM400 and increasing the number of stacks, thereby increasing the number of segments, the number of paths in each segment, and the number of groups of paths.

[0205] Therefore, as in Embodiment 2, even with an expanded address space for the main memory, the memory bandwidth and density can be increased based on the structure of the CPU 100 and the cache DRAM 400.

[0206] The embodiments disclosed herein are examples of structures for specific implementation of the present invention and do not limit the technical scope of the present invention. The technical scope of the present invention is not indicated by the description of the embodiments, but by the claims, and should include the scope of the language used in the claims as well as modifications within the scope of equivalent meanings.

[0207] Explanation of reference numerals in the attached figures

[0208] 100: CPU; 102: L1 cache; 104: L2 cache; 106: L3 cache; 108: External interface; 110: Control unit; 111: Cache controller; 112: Register; 114: Program counter; 116: Base address register; 118: Index register; 120: Instruction register; 150: Arithmetic unit; 152: Arithmetic logic unit (ALU); 154: Accumulator; 156: General purpose register; 300: Main memory; 400: DRAM cache; 750: Graphics processor; 800: Accelerator; 900: Extended main memory module.

Claims

1. A computing system having: Main memory, which stores the data required for computation and processing; DRAM (Dynamic Random Access Memory) circuitry; and The processor circuitry is stacked in the same package as the DRAM circuitry. The processor circuitry includes one or more processor cores and a cache controller. The DRAM circuit is controlled by the cache controller and functions as a group-association cache memory between the processor and the main memory. The address space of the main memory is divided into multiple first segments. The address space of the cache memory is divided into multiple second segments, each corresponding one-to-one with the first segment. Each of the second sections includes multiple roads. In response to the processor's read access to data in the main memory, the cache controller performs a unified access to the rows in the plurality of paths corresponding to the address of the accessed main memory, and loads data from the hit row if there is hit data.

2. The computing system according to claim 1, wherein, The interface of the cache controller to the DRAM circuit side corresponds one-to-one with the interface of the DRAM circuit die to the processor side for each of the paths.

3. The computing system according to claim 2 or 3, wherein, The access request from the processor circuitry includes a main memory address that specifies the segment and an entry address located in the middle of the address as an index. The plurality of paths each include a plurality of groups of data and tags from the read units of the main memory. The cache controller is controlled in the following manner: selecting the second segment according to the segment address, selecting the corresponding tag in the plurality of paths according to the entry address, and reading the data grouped with the selected tag.

4. A semiconductor integrated circuit module having multiple chips stacked on top of each other, the semiconductor integrated circuit module having: A first integrated circuit chip includes a processor for performing computational processing, the processor including a cache controller; and A second integrated circuit chip, stacked on top of the first integrated circuit chip, includes at least one die of DRAM circuitry connected in a manner capable of data transmission and reception with the processor. The DRAM circuit is controlled by the cache controller and functions as a group-association cache memory between the processor and the main memory. The address space of the main memory is divided into multiple first segments. The address space of the cache memory is divided into multiple second segments, each corresponding one-to-one with the first segment. Each of the second sections includes multiple roads. In response to a read access to data in the main memory from the processor, the cache controller performs a unified access to the rows in the plurality of paths corresponding to the address of the accessed main memory, and loads data from the hit row if there is hit data.

5. The semiconductor integrated circuit module according to claim 4, wherein, The interface of the cache controller to the DRAM circuit side corresponds one-to-one with the interface of the DRAM circuit die to the processor side for each of the paths.

6. The semiconductor integrated circuit module according to claim 4 or 5, wherein, The access request from the processor includes a main memory address that specifies the segment and an entry address located in the middle of the address as an index. The plurality of paths each include a plurality of groups of data and tags from the read units of the main memory. The cache controller is controlled in the following manner: selecting the second segment according to the segment address, selecting the corresponding tag in the plurality of paths according to the entry address, and reading the data grouped with the selected tag.

7. The semiconductor integrated circuit module according to claim 6, wherein, The DRAM circuit has multiple segment dies configured within the second integrated circuit chip corresponding to the multiple segments. The multiple segment bare dies are respectively divided into road bare dies corresponding to the multiple roads.

8. The semiconductor integrated circuit module according to claim 7, wherein, The bare chips each contain: Memory array; Electrodes corresponding to address pins, which are used to access the memory array; An electrode corresponding to a command pin, the command pin being used to determine the type of access to the memory array; as well as Electrodes corresponding to data input / output pins, which are used to read data from the memory array.

9. The semiconductor integrated circuit module according to claim 8, wherein, The data input / output pins correspond to the data of the read unit and the number of bits of the tag that can be read within a burst period.

10. The semiconductor integrated circuit module according to claim 4, wherein, The first integrated circuit chip and the second integrated circuit chip are connected by hybrid bonding in a flip-chip configuration.

11. A computing system having: Main memory module; and A semiconductor integrated circuit module, which has multiple chips stacked on top of each other, is used to perform computational processing based on data and instructions stored in the main memory module. The semiconductor integrated circuit module has: A first integrated circuit chip, comprising a processor and a cache controller for performing computational processing; and A second integrated circuit chip, stacked on top of the first integrated circuit chip, includes at least one die of DRAM circuitry connected in a manner capable of data transmission and reception with the processor. The DRAM circuit is controlled by the cache controller and functions as a group-association cache memory between the processor and the main memory. The address space of the main memory is divided into multiple first segments. The address space of the cache memory is divided into multiple second segments, each corresponding one-to-one with the first segment. Each of the second sections includes multiple roads. In response to a read access to data in the main memory from the processor, the cache controller performs a unified access to the rows in the plurality of paths corresponding to the accessed address of the main memory. If a data match is found, the data is loaded from the matched row. The main memory module includes an extended main memory module, which is controlled by an extension controller and connected to the semiconductor integrated circuit module via a bus interface.

Images