Processing core including high capacity low latency storage memory

Abstract

A non-volatile memory stack that provides high bandwidth support for a dedicated processor, such as an AI processor. A high bandwidth flash memory (HBF) stack may be unitary, including all non-volatile memories with a memory controller, or it may be hybrid, including a hybrid of non-volatile and volatile memories with a controller. The processor may be mounted on the interposer, and one or more of the HBF stack and / or the hybrid HBF stack may then be mounted on the interposer side by side with the processor.

Description

[0001] Priority Statement

[0002] This application claims priority to U.S. Patent Application No. 18 / 933,962, filed October 31, 2024, entitled “Proceeding Core Incloding High Capacity Low Latency Storage Memory,” which claims priority to U.S. Provisional Patent Application No. 63 / 551,026, filed February 7, 2024, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Processing cores perform calculations, execute instructions, and manage components and peripherals to drive the operation of computers and other electronic devices. Typical processing cores include processors such as central processing units that operate using non-volatile and / or volatile memory. Non-volatile memory can include, for example, stacks of NAND semiconductor dies mounted on a substrate adjacent to or potentially distant from the processor. These dies offer large memory capacities, but in part, due to their separation from the processor on the circuit board, they provide relatively low bandwidth rates, high power requirements, and undesirable parasitic effects. Volatile memory can include, for example, stacks of DRAM semiconductor dies specifically designed to provide higher bandwidth and lower power requirements, but at the cost of lower memory capacity compared to NAND dies. Traditional processing cores optimize the balance between speed and memory capacity. Typically, DRAM acts as the primary working memory, providing fast access to frequently used data. NAND memory is used for secondary storage, providing ample capacity for long-term data storage, but with slower access speeds.

[0004] Recently, sophisticated dedicated processing cores have been developed, including high-speed artificial intelligence (AI) processing devices and graphics processing units (GPUs). AI processors are optimized for executing artificial neural networks using parallel processing, which allows them to process large amounts of data simultaneously. GPUs are dedicated processors designed to accelerate the rendering and manipulation of images, videos, and complex graphics computations by partially utilizing multiple processors operating in parallel. This allows GPUs to process large amounts of data simultaneously.

[0005] Specialized processing cores such as GPUs and AI processors have large memory capacity requirements that cannot be adequately served by conventional volatile memory. However, these devices also have high bandwidth and low power requirements that cannot be adequately served by conventional non-volatile memory.

[0006] Furthermore, the AI ​​processor is implemented and used in two distinct phases: a training phase, during which it is trained for its intended purpose, and an inference phase, during which it is deployed for query responses. During the training phase, the AI ​​processor performs a large number of read / write operations on memory. Such a high volume of write operations can degrade traditional non-volatile memory. Attached Figure Description

[0007] Figure 1 This is a flowchart for forming the processing core according to the implementation scheme of this technology.

[0008] Figure 2 This is a top view of a first semiconductor wafer and its first semiconductor tile according to an embodiment of the present technology.

[0009] Figure 3 This is a top view of a second semiconductor wafer and its second semiconductor piece according to an embodiment of the present technology.

[0010] Figure 4 This is a cross-sectional edge view of a first semiconductor wafer according to an embodiment of the present technology.

[0011] Figure 5 This is a cross-sectional edge view of a second semiconductor wafer according to an embodiment of the present technology.

[0012] Figure 6 This is a cross-sectional edge view of a CBA memory chip according to an embodiment of the present technology, the CBA memory chip including a first semiconductor chip bonded to a second semiconductor chip.

[0013] Figures 7 to 12 This is a cross-sectional edge view of a second semiconductor die according to an alternative embodiment of the present technology.

[0014] Figures 13 to 18 This is a view of a first semiconductor die according to an alternative embodiment of the present technology.

[0015] Figures 19 to 21 This is a perspective view showing various bump pad patterns on one of a first semiconductor wafer and a second semiconductor wafer according to an embodiment of the present technology.

[0016] Figure 22 and Figure 23 This is an edge view and perspective view showing an embodiment of the CBA memory chip according to the present technology.

[0017] Figure 24 This is a perspective view of a CBA memory chip mounted on an intermediary layer according to an embodiment of the present technology.

[0018] Figure 25 Installation according to the implementation scheme of this technology Figure 24 An exploded perspective view of one or more volatile memories on a CBA memory chip.

[0019] Figure 26 This is a cross-sectional edge view of a volatile memory chip according to an embodiment of the present technology.

[0020] Figure 27 This is a perspective view of an integrated processing core according to an embodiment of the present technology, including a processor mounted on an intermediary layer, a CBA memory chip, and one or more volatile memories.

[0021] Figure 28 This is a perspective view of a completed processing core according to an embodiment of the present technology, including a processor mounted on an intermediary layer, a CBA memory chip, one or more volatile memories, and an HBM stack.

[0022] Figure 29 An embodiment of this technology includes a processor mounted on an interposer, a CBA memory chip, one or more volatile memories, and an HBM stack. Figure 28 The completed processing of the core cross-sectional edge view.

[0023] Figure 30 This is a perspective view of an completed processing core according to an alternative embodiment of the present technology, including a processor mounted on an intermediary layer, a CBA memory chip, and one or more volatile memories.

[0024] Figure 31 An alternative embodiment of the present technology includes a processor mounted on an interposer, a CBA memory chip, and one or more volatile memories. Figure 30 The completed processing of the core cross-sectional edge view.

[0025] Figure 32 This is a perspective view of a completed processing core comprising a processor, a CBA memory chip, and one or more volatile memories according to an alternative embodiment of the present technology.

[0026] Figure 33 An alternative embodiment of the present technology includes a processor, a CBA memory chip, and one or more volatile memories. Figure 32 The completed processing of the core cross-sectional edge view.

[0027] Figure 34 This is a functional block diagram of a CBA memory chip coupled to a processor according to an embodiment of this technology.

[0028] Figure 35This is a perspective view of an HBF stack according to an embodiment of this technology.

[0029] Figure 36 This is a perspective view of a completed processing core including a processor and an HBF stack and an HBM stack, according to an alternative embodiment of the present technology.

[0030] Figure 37 It is based on the implementation scheme of this technology. Figure 36 The processing core is the cross-sectional edge view.

[0031] Figure 38 This is a perspective view of a completed processing core including a processor and a hybrid HBF stack, according to an alternative embodiment of the present technology.

[0032] Figures 39 to 40 This is an edge view of an example of a hybrid HBF stack according to different implementations of this technology. Detailed Implementation

[0033] The present technology will now be described with reference to the accompanying drawings, which, in embodiments, relate to a processing core comprising a processor integrated with high-bandwidth, low-latency memory. The processor may be, for example, a large-scale artificial intelligence (AI) processor, but it may be other types of dedicated processors including a graphics processing unit (GPU). The memory may include both non-volatile memory and volatile memory.

[0034] In a first aspect of the invention, the memory can be manufactured as a CBA (CMOS Bonded-to-Array) memory, which includes a NAND memory die coupled to a second semiconductor die, which can be a combination of CMOS logic circuitry and a volatile memory die. Specifically, particularly for large NAND memory dies, the CMOS logic circuitry requires only a portion of the second semiconductor die. Therefore, the remaining portion of the second semiconductor die can be used for low-latency volatile memory. In addition to providing high-bandwidth access to the memory, integrating volatile memory into the CBA memory also reduces the wear and tear that could occur during the large number of write operations originally required during the training of an AI processor.

[0035] In a second aspect of the invention, the cells of a NAND memory array (within CBA memory or otherwise) are partitioned to store different numbers of bits. The cells of the NAND memory array can be conventionally partitioned to hold one data bit (single-level cell or SLC), two data bits (multi-level cell or MLC), three data bits (triple-level cell or TLC), or four data bits (quadruple-level cell or QLC). SLC holds the least amount of data but also exhibits the least amount of wear during write operations throughout its lifetime. Conversely, QLC holds the largest amount of data but exhibits the highest amount of wear throughout its lifetime. According to various aspects of the present invention, the NAND memory array can be partitioned to include SLC, as well as at least one other type of cell, namely MLC, TLC, and / or QLC. SLC can be used primarily or exclusively during write-intensive training of the AI ​​processor to minimize wear on the memory during training. Thereafter, upon completion of the training period, other memory cells (MLC, TLC, and / or QLC) can be used primarily or exclusively during the use of the AI ​​processor.

[0036] In a third aspect of this technology, a memory stack comprising entirely non-volatile memory and a memory controller can be provided. Such a stack is referred to herein as a high-bandwidth flash (HBF) stack. In various embodiments, a processor may be mounted on a printed circuit board. One or more HBF stacks may then be mounted side-by-side with the processor and one or more HBM stacks. In one example, the HBF stack may be used by the processor for read operations, and the HBM stack may be used for write operations.

[0037] In a fourth aspect of this technology, a memory stack comprising a mixture of non-volatile memory and volatile memory, along with a memory controller, can be provided. Such a stack is referred to herein as a hybrid HBF stack. In various embodiments, a processor may be mounted on a printed circuit board. One or more hybrid HBF stacks may then be mounted side-by-side with the processor. The hybrid HBF stack may include different combinations of volatile and non-volatile memory dies, depending on the processor's capacity and bandwidth requirements. In one example, the non-volatile memory dies of the hybrid HBF stack may be used by the processor for read operations, and the volatile memory dies of the hybrid HBF stack may be used for write operations.

[0038] It should be understood that the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the invention to those skilled in the art. In fact, the invention is intended to cover alternatives, modifications, and equivalents of these embodiments, which are included within the scope and spirit of the invention as defined by the appended claims. Furthermore, numerous specific details are set forth in the following detailed description of the invention to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without such specific details.

[0039] The terms “top” and “bottom,” “upper” and “lower”, and “vertical” and “horizontal” as used herein, and their various forms, are by way of example and for illustrative purposes only, and are therefore not intended to limit the description of the technique, as the referenced items may be interchanged in position and orientation. Moreover, as used herein, the terms “substantially” and / or “about” mean that a specified dimension or parameter may vary within acceptable manufacturing tolerances for a given application. In one embodiment, acceptable manufacturing tolerances are ±0.15 mm or alternatively ±2.5% of a given dimension.

[0040] For the purposes of this disclosure, physical or electrical connections can be direct or indirect (e.g., via one or more other parts). In some cases, when the first element is referred to as a (physical or electrical) connection, attachment, mounting, or coupling to the second element, the first and second elements can be directly connected, attached, mounted, or coupled to each other or indirectly (physical or electrical) connected, attached, mounted, or coupled to each other. When the first element is referred to as a direct connection, attachment, mounting, or coupling to the second element, there is no intermediate element between the first and second elements (other than the possibility of adhesives or molten metal used to connect, attach, mount, or couple the first and second elements).

[0041] Now refer to Figure 1 Flowcharts and Figures 2 to 40 The diagram illustrates an implementation of this technology. In step 200, the first semiconductor wafer 100 can be processed as follows: Figure 2 The diagram shows a plurality of first semiconductor wafers 102. A first semiconductor wafer 100 may begin as an ingot of wafer material, which may be single-crystal silicon grown according to a Czochralski (CZ) or floating zone (FZ) process. However, in another embodiment, the first wafer 100 may be formed from other materials and by other processes.

[0042] Semiconductor wafer 100 can be cut from an ingot and onto a first main flat surface 104 and a second main flat surface 105 opposite to surface 104. Figure 4 Both surfaces are polished to provide a smooth surface. The first main surface 104 may undergo various processing steps to divide the wafer 100 into corresponding first semiconductor wafers 102 and form integrated circuits of the corresponding first semiconductor wafers 102 on and / or in the first main surface 104. Figure 2 Details of a single semiconductor wafer 102 are also shown, including patterns of microbump pads 106 and through regions 108 as explained below.

[0043] The processing of wafer 100 in step 200 may include forming an integrated circuit memory cell array 122 in a dielectric substrate including layers 124 and 126, such as Figure 4 The cross-sectional edge view is shown. A photomask can be used to transfer integrated circuit patterns for a single semiconductor wafer 102 during a photolithography process. The patterned wafer can then undergo various processes such as etching, ion implantation, and deposition to produce the actual semiconductor components and interconnects required to build the integrated circuits of semiconductor wafer 102. In an embodiment, the integrated circuit may be a memory cell array 122 formed as a 3D stacked memory structure having strings of memory cells formed as layers.

[0044] In various embodiments, the memory cell array 122 may be formed as NAND memory, such as, for example, BICS (Bit Cost Scalable) memory. Other types of memory are possible, including, for example, MRAM. In various embodiments, each memory cell in the array may be partitioned into SLC, MLC, TLC, or QLC. However, according to the following... Figures 13 to 18 As explained in the present technology, some memory cells in the memory cell may be partitioned into SLC, while other memory cells may be partitioned into one or more of MLC, TLC, or QLC. It should also be understood that the first semiconductor wafer 102 may be processed to include integrated circuits other than 3D stacked memory structures. A passivation layer 128 may be formed on top of the upper dielectric film layer 126.

[0045] Semiconductor processing is trending towards increasingly smaller semiconductor dies. In conventional semiconductor processing, a single photomask can include patterns for multiple semiconductor dies, and the photomask can be used to define hundreds (if not thousands) of semiconductor dies on a single wafer. Semiconductor wafer 102 is the opposite of this trend. Semiconductor wafer 102 can be the entire size of the photomask, and the photomask is used to form a relatively small number of semiconductor wafers on wafer 100. As explained below, the size of semiconductor wafer 102 can be, for example, 32 mm × 25 mm. However, it should be understood that in other embodiments, the size of semiconductor wafer 102 can vary, and in other embodiments, a single photomask can have patterns for more than one semiconductor wafer 102.

[0046] After forming the memory cell array 122, in step 204, internal electrical connections may be formed within the first semiconductor wafer 102. The internal electrical connections may include multilayer metal interconnects 130 and vias 132 formed sequentially through a dielectric film layer 126. As known in the art, the metal interconnects 130, vias 132, and dielectric film 126 may be formed one layer at a time, for example, using a damascene process employing photolithography and thin-film deposition processes. Photolithography processes may include, for example, patterning, plasma, chemical, or dry etching and polishing. Thin-film deposition processes may include, for example, sputtering and / or chemical vapor deposition. The metal interconnects 130 may be formed of various conductive metals, including, for example, copper and copper alloys known in the art, and the vias 132 may be lined and / or filled with various conductive metals, including, for example, tungsten, copper, and copper alloys known in the art.

[0047] For example in Figure 4 As seen, metal interconnects 130 and vias 132 may be formed to and through the memory cell array 122 to carry signals to and from the memory cell array 122. However, as noted, the semiconductor wafer 102 may include certain regions (referred to herein as through-hole regions 108) that do not contain memory cells or other integrated circuits. These regions 108 include TSVs 134. TSVs 134 may include metal interconnects and vias, and may be formed in the same manner as the metal interconnects 130 and vias 132 that pass through one or more dielectric layers as described above. Figure 2 and Figure 4 In the through-region 108, the TSV 134 and bump pad 106 are more densely packed within the through-region 108 compared to the interconnects 130, vias 132, and bump pads 106 outside the region 108. However, as explained below, the density of the TSV 134 and bump pads inside the through-region 108 can be the same as or less than the density of the interconnects 130, vias 132, and bump pads 106 outside the region 108.

[0048] In step 208, microbump pads 106 can be formed on the main flat surfaces 104 and 105 of the first semiconductor wafer 102. For example... Figure 2 and Figure 4 As shown, these bump pads can be formed on top of and / or bottom of vias 132 and TSVs 134. Also explained below, bump pads 106 are provided for transmitting signals to and from the semiconductor wafer 102. The bump pads can be etched into the passivation layer 128, and each bump pad 106 can be formed over a liner 136. As known in the art, bump pads 106 can be formed of, for example, copper, aluminum, and alloys thereof, and the liner 136 can be formed of, for example, titanium / titanium nitride stacks (e.g., Ti / TiN / Ti), but these materials can vary in other embodiments. Bump pads 106 and liner 136 can be applied by vapor deposition and / or electroplating techniques. Integrated circuit memory array 122 can be electrically connected to the bump pads 106 via metal interconnects 130 and vias 132.

[0049] Figure 2 The image shows a semiconductor wafer 102 on a wafer 100 and a bump pad 106 in a pattern on one of the semiconductor wafers 102. Figure 2 The number of first semiconductor wafers 102 shown on wafer 100 is for illustrative purposes, and wafer 100 may include more or fewer first semiconductor wafers 102 than shown in other embodiments. Similarly, the pattern and number of bump pads 106 on the first semiconductor wafers 102 are shown for illustrative purposes. Each first wafer 102 may include more bump pads 106 than shown in other embodiments, and may include various other patterns and densities of bump pads 106.

[0050] Before, after, or simultaneously with the formation of the first semiconductor wafer on wafer 100, the second semiconductor wafer 110 may be processed as follows: Figure 3 The multiple second semiconductor wafers 112 shown in step 210. Semiconductor wafers 110 may begin as single-crystal silicon ingots grown according to CZ, FZ, or other processes. The second semiconductor wafers 110 may have a first main surface 114 and a second main surface 115 opposite to surface 114. Figure 5 Both are cut and polished to provide a smooth surface. The first main surface 114 may undergo various processing steps to divide the second wafer 110 into corresponding second semiconductor wafers 112 and form integrated circuits of the corresponding second semiconductor wafers 112 on and / or in the first main surface 114. Figure 3 Details of a single semiconductor wafer 112 are also shown, including patterns of microbump pads 116 and through regions 108 as explained below.

[0051] In one embodiment, the second semiconductor wafer 112 may be processed to include forming on a substrate comprising, for example, Figure 5 The integrated circuit 142 is shown in the dielectric substrate of layers 144 and 146 in a cross-sectional edge view. The integrated circuit 142 can be configured as logic circuitry to control read / write operations of one or more integrated memory cell arrays 122. CMOS technology can be used to fabricate the logic circuitry, but in other embodiments, other technologies may be used. In other embodiments explained below, the second semiconductor wafer 112 may include other and / or additional integrated circuits. A passivation layer 148 may be formed on top of the upper dielectric film layer 146.

[0052] After forming the CMOS logic circuit 142, in step 204, internal electrical connections may be formed within the second semiconductor wafer 112. These internal electrical connections may include multilayer metal interconnects 150 and vias 152 formed sequentially through the dielectric film layer 146. The metal interconnects 150, vias 152, and dielectric film layer 146 may be formed in the same manner as the interconnects 130, vias 132, and dielectric film layer 126 described above for wafer 102.

[0053] For example in Figure 4 As seen, metal interconnects 150 and vias 152 can be connected to CMOS logic circuitry 142 to carry signals to and from logic circuitry 142. However, as noted, semiconductor wafer 112 may include through-regions 108 that do not contain CMOS logic circuitry or other integrated circuits. The size and pattern of through-regions 108 in semiconductor wafer 102 may match the size and pattern of through-regions 108 in semiconductor wafer 112. Through-regions 108 in wafer 112 may include TSVs 154. The number and pattern of TSVs 154 may match the number and pattern of TSVs 134 described above.

[0054] In step 208, microbump pads 116 can be formed on the main flat surfaces 114 and 115 of the second semiconductor wafer 112. For example... Figure 3 and Figure 5 As shown, these bump pads can be on top of and / or below vias 152 and TSVs 154. Similarly, as explained below, bump pads 116 are provided for transmitting signals to and from the semiconductor wafer 112. The bump pads can be etched into the passivation layer 148 and may include a liner 156. Bump pads 116 and liner 156 can be formed in the same manner as bump pads 106 and liner 136 described above. CMOS logic circuitry 142 can be electrically connected to bump pads 116 via metal interconnects 150 and vias 152.

[0055] Figure 3A semiconductor wafer 112 on wafer 110 and a bump pad 116 in a pattern on one of the semiconductor wafers 112 are shown. Figure 3 The number of second semiconductor wafers 112 shown on wafer 110 is for illustrative purposes, and wafer 110 may include more or fewer second semiconductor wafers 112 than shown in other embodiments. Similarly, the pattern and number of bump pads 116 on the second semiconductor wafers 112 are shown for illustrative purposes. Each second wafer 112 may include more bump pads 116 than shown in other embodiments, and may include various other patterns and densities of bump pads 116.

[0056] Once the fabrication of the first semiconductor wafer 102 and the second semiconductor wafer 112 is complete, the first semiconductor wafer 110 and the second semiconductor wafer 110 can be attached to each other in step 222, such that the corresponding memory wafer 102 is bonded to the CMOS logic circuit wafer 112. Each pair of bonded wafers 102, 112 is referred to herein as a CMOS bonded-to-array (CBA) memory wafer 160. For example, in Figure 6 An example of the completed CBA memory chip 160 is shown in a cross-sectional edge view. For bonding chips 102 and 112, the first semiconductor wafer 100 is flip-flop (relative to...). Figure 4 (see view), and the bump pads 106 and 116 of the corresponding pieces 102 and 112 are physically and electrically coupled to each other. As shown and indicated, the number and pattern of the bump pads 106 can match the number and pattern of the bump pads 116 such that the pads are aligned with each other when pieces 102 and 112 are coupled together. In an embodiment where the number and pattern of the bump pads 106 and 116 are asymmetrical about the vertical axis passing through the center of the piece, the number and pattern of the bump pads 106 can be a mirror image of the number and pattern of the bump pads 116 such that the pads 106 and 116 are aligned when the piece 102 is flipped.

[0057] The first semiconductor chip 102 and the second semiconductor chip 112 in the CBA memory chip 160 can be initially joined together by aligning the bump pads 106 and 116 on the respective chips 102 and 112 with each other. Thereafter, the bump pads 106 and 116 can be joined together using any of a variety of joining techniques, depending in part on the bump pad size and the bump pad spacing (i.e., the bump pad pitch). As explained below, the bump pad size and pitch can in turn be determined by the number of electrical interconnects required for the CBA memory chip 160.

[0058] As noted above, while non-volatile memory arrays such as semiconductor chip 102 offer large storage capacities, such as 2TB or greater, they also undergo degradation during write operations. Therefore, despite the advantage of high storage capacity, the use of non-volatile memory arrays during the training phase of an AI processor is not ideal. This technology addresses this problem by providing a CBA memory chip 160 that includes both non-volatile and volatile memory. While not as efficient in terms of storage capacity, volatile memory does not undergo the same degradation as non-volatile memory during write operations. Reference will now be made to... Figures 7 to 12 Describe this implementation scheme.

[0059] A characteristic of CMOS junction array semiconductor devices is that the required size of CMOS logic circuitry is smaller compared to the size of non-volatile memory. This is especially true for large CMOS junction array semiconductor devices such as CBA memory chip 160. According to various aspects of the present invention, the remaining space within the CMOS semiconductor chip 112 that is not needed by the logic control circuitry 142 can be utilized in volatile memory.

[0060] exist Figure 7 In the illustrated embodiment, the CMOS semiconductor wafer 112 is processed to include CMOS logic circuitry 142 in dielectric layers 144 and 146 as described above. The CMOS semiconductor wafer 112 includes a metallization layer 150, vias 152, and TSVs 154 (in the through region 108) as described above. However, the CMOS semiconductor wafer 112 also includes volatile memory 145, as described above. Figure 7 The volatile memory 145 is schematically shown in the cross-sectional view. It may be, for example, a DRAM memory cell, but in other embodiments, chip 112 may be configured to include other types of volatile memory, including, for example, SRAM and SDRAM.

[0061] The integrated circuit transistors and capacitors defining the volatile memory 145 may be formed at the same (or different) time as the integrated circuit transistors defining the logic circuit 142. Like the logic circuit 142 formed in the dielectric layer 146, the transistors and capacitors of the volatile memory can be formed in the dielectric layer 146 using photolithography. However, a different deposition and patterning process is used to define the volatile memory 145 compared to the logic circuit 142. As a result, a portion of the CMOS semiconductor wafer 112 is processed to include the logic circuit 142, while other portions of the CMOS semiconductor wafer 112 are processed to include the volatile memory 145.

[0062] Metallization layer 150 and via 152 can be used to electrically couple both logic circuitry 142 and volatile memory 145 to microbump pads 116 on at least a first flat surface 114 of semiconductor wafer 112 as described above. After wafer 112 is completed, pads 116 can be bonded to microbump pads 106 of first semiconductor wafer 102 as described above and below in more detail to complete the formation of CBA memory wafer 160. The bonding of pads 106 and 116 uses metallization layer 130, via 132 and / or TSV 134 in first semiconductor wafer 102 to electrically couple the volatile memory 145 of second semiconductor wafer 112 to first semiconductor wafer 102 and / or through the first semiconductor wafer to the volatile memory.

[0063] The amount of the first semiconductor chip 112 used for the volatile memory 145 can vary in different embodiments compared to the logic circuit 142. Figure 7 and Figure 8 The roughly equal division between logic circuit 142 and volatile memory 145 is shown. Figure 9 The cross-sectional view shows that the area used for logic circuitry 142 is larger than that for volatile memory 145. Furthermore, Figure 10 The cross-sectional view shows that the area used for volatile memory 145 is larger than that for logic circuitry 142. Although in Figures 7 to 10 In the diagram, logic circuitry 142 is shown on one side (left) and volatile memory 145 is shown on the other side (right). However, in other embodiments, the logic circuitry and volatile memory may be interleaved to a greater extent, for example, as shown in the diagram. Figure 11 The sectional view is shown.

[0064] Figures 7 to 11 The implementation includes a through region 108, which includes a TSV 154 and has no logic circuitry 142 and volatile memory 145. However, in Figure 12 In another embodiment shown in the cross-sectional view, some or all of the through regions 108 may be omitted, thereby leaving additional space for the formation of volatile memory 145 and / or logic circuitry 142.

[0065] The hybrid CMOS semiconductor chip 112, comprising both logic circuitry 142 and volatile memory 145, offers several advantages. First, logic circuitry 142 provides various control functions for both the non-volatile memory array 122 and the volatile memory 145. Additionally, the volatile memory array 145 provides a low-latency, high-bandwidth buffer or cache memory for use by the dedicated processor explained below. Furthermore, as noted in the background section, the training phase of an AI processor involves a very large number of reads / writes to the associated memory, which can degrade the non-volatile memory. However, volatile memory does not experience the same degradation under conditions of numerous write operations. Therefore, volatile memory 145 allows the CBA memory chip 160 to be used not only during the inference phase of the AI ​​processor explained below, but also extensively during the training phase of the AI ​​processor. In one example, the volatile memory 145 of the CBA memory chip 160 can provide approximately 10 GB of storage capacity for use during both the training and inference phases of the AI ​​processor. In another embodiment, the storage capacity of the volatile memory 145 may be greater than or less than 10 GB.

[0066] In another aspect of this technology, the non-volatile memory 122 may be additionally or alternatively customized to reduce wear and tear on the memory during the training phase of the AI ​​processor. Specifically, as noted above, non-volatile memory arrays are conventionally partitioned into units comprising one type of memory cell; namely, single-level cell (SLC), multi-level cell (MLC), three-level cell (TLC), or four-level cell (QLC). While QLC offers the highest storage capacity, these QLCs also suffer the greatest degradation during write operations due to the large amount of data written to them. Therefore, despite the advantage of high storage capacity, the use of QLC (and other multi-level cells) during the training phase of the AI ​​processor is not ideal. This technology addresses this problem by providing a hybrid non-volatile memory array comprising some cells partitioned as SLC and other cells partitioned as one or more of MLC, TLC, and QLC. Reference will now be made to... Figures 13 to 18 This aspect of the technology is described.

[0067] Figure 13 It is a cross-sectional view of a first semiconductor wafer 102 including a hybrid memory array 122, which includes a first memory array portion 122-1 partitioned as SLC and at least a second memory array portion 122-n partitioned as one or more of MLC, TLC and / or QLC. Figure 13 All other components of the semiconductor chip 102 shown can be manufactured as described above. Figure 14This is an illustration of a memory cell in the first memory array portion 122-1 and a memory cell in an example of the second memory array portion 122-n. As shown, portion 122-1 includes a single bit cell that allows 1 data bit, and portion 122-n (in this example) is partitioned into a QLC comprising four bit cells that allow 16 data bits.

[0068] Figure 15 A chip 102 is shown, comprising a hybrid memory array 122, which is attached to a second semiconductor chip 112 to form a completed CBA memory chip 160. In the illustrated embodiment, the second semiconductor chip 112 comprises only CMOS logic control circuitry 142. However, in another embodiment, chip 112 may alternatively be a hybrid CMOS semiconductor chip comprising both the logic control circuitry 142 described above and volatile memory 145.

[0069] exist Figure 13 and Figure 15 In this configuration, the hybrid memory array 122 is equally divided between the SLC memory array portion 122-1 and the memory array portion 122-n, which includes one or more of MLC, TLC, and QLC. Figure 16 In another embodiment shown in the cross-sectional view, the SLC memory array portion 122-1 may be smaller than the memory array portion 122-n. Figure 17 In another embodiment shown in the cross-sectional view, the SLC memory array portion 122-1 may be larger than the memory array portion 122-n.

[0070] While SLC memory array portion 122-1 may still experience degradation over time under write operations, it experiences less degradation than memory array portion 122-n. In one example, SLC memory array portion 122-1 provides 500GB of storage capacity and supports approximately 1.2 million write cycles. This storage capacity and wear cycles are sufficient to allow SLC memory array portion 122-1 of CBA memory chip 160 to fully support the AI ​​processor during its training phase. After the training phase is complete, SLC memory array portion 122-1 can still be used for the inference phase of the AI ​​processor, but even if portion 122-1 is not used after training, memory array portion 122-n of CBA memory chip 160 can provide 1.5TB or more of storage capacity, which is sufficient to support the AI ​​processor during the inference phase.

[0071] Figure 13 and Figures 15 to 17 The implementation includes a through region 108, which includes TSV 134 and does not have a memory array 122. However, in Figure 18In another embodiment shown in the cross-sectional view, some or all of the through regions 108 may be omitted, thereby leaving additional space for forming SLC memory array portions 122-1 and / or memory array portions 122-n, including MLC, TLC and / or QLC.

[0072] In the above context Figures 13 to 18 In the described embodiment of the hybrid memory array, a first semiconductor chip 102 comprising a hybrid memory array 122 is bonded to a second semiconductor chip 112 comprising CMOS logic circuitry 142 and a possible volatile memory 145. In another embodiment, a hybrid memory array 122 may be used alone, comprising... Figure 13 and Figures 16 to 18 The hybrid memory array 122 shown uses a semiconductor die 102 without a second semiconductor die 112. In this embodiment, a separate controller die, such as an ASIC, can be provided to control the operation of the hybrid memory array 122 using semiconductor die 102.

[0073] Figures 19 to 21 The perspective and sectional views will be used to illustrate different methods for bonding the first semiconductor die 102 and the second semiconductor die 112 together (according to any of the embodiments described above). Figure 19 In one embodiment shown in the perspective view, one or two sets of bump pads 106, 116 on the mating surfaces of the first piece 102 and the second piece 112 may include microbumps 164 applied to the surfaces of the pads 106 and / or 116. A small, controlled amount of solder, copper, bronze, gold, or other metal may be applied to the bump pads 106 and / or 116 of the pair of bump pads to be bonded. The respective bump pads can be coupled to each other via the microbumps 164 using, for example, thermal compression. In the example, the bump pads 106, 116 may be approximately 50 μm square. Similarly, Figure 19 The number and pattern of the bump pads 106 / 116 shown are for illustrative purposes only and may vary in other embodiments.

[0074] Instead of using microbumps 164, pads 106 and 116 of sheets 102 and 112 can be bonded to each other in a so-called Cu-Cu bonding process without solder or other added materials. Figure 20This example is illustrated in the perspective view. In the Cu-Cu bonding process, bump pads 106, 116 are controlled to be highly flat and formed in a highly controlled environment that is substantially free of environmental particles. Under these properly controlled conditions, bump pads 106, 116 are aligned and pressed against each other to form an inter-bonding based on oxide bonding. Such bonding can be formed at room temperature, but an annealing process can be performed to heat and cool the bump pads under controlled conditions to further improve the bonding. In embodiments using Cu-Cu bonding, bump pads 106, 116 can be approximately 5 μm square, and bumps 106, 116 can be spaced apart from each other with a pitch of 10 μm to 20 μm. The pads and / or pitch can be larger or smaller than the pads and / or pitch in other embodiments. Although this process is referred to herein as Cu-Cu bonding, the term can also be applied to cases where pads 106, 116 are formed of materials other than copper.

[0075] exist Figure 21 In another embodiment shown in the perspective view, Cu-Cu bonding can be enhanced by providing a film layer 166 on surface 104 of the first wafer 102 and / or on surface 114 of the second wafer 112. Such a film layer 166 is disposed around bump pads 106, 116. When the first wafer 102 and the second wafer 112 are placed together, the bump pads 106, 116 can be bonded to each other using oxide bonding and annealing, and the film layers 166 on the respective wafers can be bonded to each other using adhesion and / or surface tension. This type of bonding technique can be referred to as hybrid bonding. In embodiments using hybrid bonding, the bump pads 106, 116 can be approximately 5 μm square, and the bumps 106, 116 can be spaced apart from each other at a pitch of 5 μm to 10 μm. The pads and / or pitch can be larger or smaller than the pads and / or pitch in other embodiments.

[0076] As noted, once coupled together in step 222, the first semiconductor wafer 102 and the second semiconductor wafer 112 together form a CBA memory wafer 160. As is known, wafer 160 can be operationally tested in step 226, for example, using read / write and burnin operations. In step 228, wafer 160 can be diced from the coupled wafers 100 and 110. The above-described... Figure 6 , Figure 8 and Figure 15 In the cross-sectional edge view and Figure 22 and Figure 23An example of CBA memory chip 160 is shown in the edge view and perspective view. As shown, once coupled together, the bump pads 106 on surface 105 of chip 112 and the bump pads 116 on surface 115 of chip 102 can remain exposed. These exposed bump pads 106, 116 can be used as explained below. Similarly, Figure 22 and Figure 23 The views shown are merely illustrative examples. The number, pattern, and / or density of the bump pads 106, 116 shown may vary in other examples.

[0077] In one embodiment described above, membrane 166 ( Figure 21 The film can be disposed on the surface of either the first sheet 102 or the second sheet 112. If such a film is not initially provided, the space between the first and second sheets of the CBA memory chip 160 can be bottom-filled with epoxy resin or other resin or polymer 168. Figure 22 and Figure 23 The underfill material 168 can be applied as a liquid and then cured into a solid layer. This underfilling step protects the electrical connection between the first sheet 102 and the second sheet 112, and further secures the second sheet 112 to the first sheet 102. Various materials can be used as the underfill material 168, but in this embodiment, the underfill material can be Hysol epoxy resin from Henkel, a company with offices in California, USA.

[0078] As described above, the CBA memory chip 160 includes a through region 108. For example... Figure 2 and Figure 3 In the illustrated embodiment, the through region 108 includes a boundary surrounding the periphery of the sheet 160, and intersecting patterns extending horizontally and vertically through the center of the sheet 160. It should be understood that in other embodiments, the through region may include other patterns on the sheet 160. As noted, the memory array circuitry 122, logic circuitry 142, or volatile memory 145 is not present in the through region 108.

[0079] The bump pads 106 in the through-area 108 are used to transmit or transfer power signals, ground signals, and data signals to and from the processor via the CBA memory chip 160. In one embodiment, the through-area 108 surrounding the periphery of the chip 160 can be used for signal exchange between the processor and a high-bandwidth memory also mounted on the interposer layer via the chip 160. Given the large number of these connections, these peripheral through-areas 108 may have a width of approximately 1.25 mm, with 25 rows of bump pads spanning this width, the 25 rows of bump pads having a spacing of approximately 40 µm. The spacing of the bump pads along the length may be approximately 60 µm. In this embodiment, a cross pattern of the through-area 108 passing through the center of the chip 160 can be used for power signals and ground signals. These cross pattern through-areas 108 may have a width of approximately 500 µm, with 10 rows of bump pads spanning this width, the 10 rows of bump pads having a spacing of approximately 50 µm. The spacing of the bump pads along the length may be approximately 125 µm. Each of these dimensions is exemplary and may vary proportionally or disproportionately in other embodiments. It should also be understood that, in other embodiments, the portions of the through-area used for signals, power, and ground may also vary.

[0080] It should be understood that the size of the through area can be increased or decreased based on the requirements of the processing core. When more through connections are needed, the size of the through area can be increased and the number of direct connections between chip 160 and the processor can be reduced. When fewer through connections are needed (or more direct connections between chip 160 and the processor are needed), the size of the through area can be decreased and the number of direct connections between chip 160 and the processor can be increased.

[0081] Area 170 ( Figure 23 Region 170, comprising memory array circuitry 122, logic circuitry 142, and volatile memory 145, is located outside the through-region 108. In the illustrated embodiment, the through-region divides region 170 into four quadrants. Again, this is one of many possible configurations of region 170, including memory array circuitry 122, logic circuitry 142, and volatile memory 145. Although Figure 2 and Figure 3 For example, a higher density of microbump pads 106 is shown in through region 108 than in region 170, but the density of microbumps 106 in through region 108 may be less than or equal to the density of microbumps 106 in region 170.

[0082] As explained below, the CBA memory chip 160 can be mounted on a signal transmission medium such as a printed circuit board (PCB), substrate, or interposer, and the processor can be mounted on top of the CBA memory chip 160. The terms PCB, substrate, and interposer are used interchangeably herein and refer to means for electrically interconnecting one or more modules or circuits to each other (such as coupling the processor and / or CBA memory chip to one or more semiconductor memory dies). Furthermore, the use of one term in place of another does not imply a specific characteristic of the "signal transmission medium," such as substrate material, number of layers, etc. It will be understood by those skilled in the art that, for example, when the term interposer is used, it can also refer to a substrate or printed circuit board. The bump pads 116 in region 170 allow the processor to be directly coupled to the CBA memory chip 160, enabling the processor to perform read / write operations on the memory chip 160. Given the relatively large size of the CBA memory chip 160, there is ample space for all channels and electrical connections between the processor and the CBA memory chip 160.

[0083] In one implementation, the spacing or pitch between the bump pads 106 in region 170 can be from 2 μm to 50 μm, depending in part on the bonding technology used. Considering this spacing and the large surface area of ​​the CBA chip 160, this allows approximately 200,000 direct connections between chip 160 and the processor. In another implementation, the number of direct connections can be more or less than this. As described below, this allows for direct data transfer of high-bandwidth, wide-word data to and from the CBA memory chip 160. In another implementation, there can be more or fewer direct connections.

[0084] In step 230, the CBA memory chip 160 can be mounted on the interposer layer 172, such as... Figure 24 The perspective view is shown. The CBA memory chip 160 may include a first semiconductor die 102 and a second semiconductor die 112 processed according to any of the embodiments described above. The interposer 172 may be a signal transport medium comprising a plurality of conductive layers having vias and conductive patterns distributed between the dielectric layers. The interposer may be formed in a silicon wafer, diced to a size supporting the CBA memory chip 160 and a high-bandwidth memory stack, as explained below. In another embodiment, the CBA memory chip may be mounted to a corresponding interposer, with the interposer retaining an entire undicated portion of the wafer. The interposer 172 is used to transmit signals to and from the CBA memory chip 160 and the processor mounted thereon, as explained below. In another embodiment, other signal transport media may be used, including flexible tape, substrate, or printed circuit board.

[0085] The top surface of the interposer 172 may have a pattern of contact pads (not shown) that match the number and arrangement of bump pads 116 on the bottom surface 115 of the CBA memory chip 160. The CBA memory chip 160 can be physically and electrically coupled to the interposer 172 by matching the bump pads 116 on the surface 115 of the chip 160 with the contact pads on the upper surface of the interposer 172. The bonding between the bump pads 116 and the contact pads of the interposer can be accomplished using any of the methods described above for bonding the bump pads 116 and bonding pads 106 within the chip 160.

[0086] CBA memory chip 160 provides a large block of memory, such as 1 to 4 terabytes, close to the processor 175 described below. In various embodiments, one or more volatile memory chips 174 may be mounted on top of the CBA memory block in step 232, such as... Figure 25 As shown, a large high-speed / high-bandwidth memory block is provided near the processor 175. In this embodiment, each volatile memory chip 174 may have the same length and width (same footprint) as the CBA memory chip 160. Figure 25 In the illustrated embodiment, the volatile memory chip 174 may include chips 174-1, 174-2, 174-3, ..., 174-n. However, in different embodiments, there may be 1, 2, 3, 4 or more chips 174.

[0087] Figure 26 This is a cross-sectional view of an example of a volatile memory chip 174. The memory chip can be processed in wafer form to include an array 322 of integrated circuit memory cells formed in a dielectric substrate including layers 324 and 326. Integrated circuit patterns for a single semiconductor chip 174 can be transferred in a photolithography process using a photomask. The patterned wafer can then undergo various photolithography processes to produce the transistors, capacitors, and metal interconnects required to construct the volatile memory integrated circuit of chip 174. In various embodiments, the integrated circuit memory cell array 322 can be formed as DRAM. However, the memory cell array 322 can be a variety of other volatile memories, including, for example, SRAM and SDRAM. A passivation layer 328 can be formed on top of the upper dielectric film layer 326.

[0088] After forming the memory cell array 322, internal electrical connections can be formed within the volatile memory chip 174. These internal electrical connections may include multilayer metal interconnects 330 and vias 332 formed sequentially through the dielectric film layer 326. The metal interconnects 330 and vias 332 may be formed as described above with respect to metal interconnects 130 and vias 132. As discussed above with respect to the CBA chip 160, the volatile memory chip 174 may include through regions 308 without memory cells or other integrated circuits. These regions 308 include TSVs 334 and may be patterned and configured to match the TSVs 134 in the CBA chip 160. As discussed above, through regions 308 are provided to allow signals and voltages to pass through the volatile memory chip 174.

[0089] Microbump pads 306 can be formed on the main flat surfaces 304 and 305 of the volatile memory chip 174. These bump pads can be formed on the top and / or bottom of the vias 332 and TSVs 334. Microbump pads 306 can be formed in the same manner as the pads 106 described above and used for the same purpose. Although Figure 26 The pattern of the bump pad 306 is shown for illustrative purposes, but the pattern of the bump pad 306 and the number of bump pads 306 may vary in other embodiments.

[0090] The bump pad 306 on the bottommost volatile memory chip 174 is aligned with and bonded to the bump pad 106 on the topmost surface of the CBA memory chip 160. Furthermore, in the presence of multiple volatile memory chips 174, the bump pad 306 serves to bond the multiple volatile memory chips 174 to each other and electrically couple them to the CBA memory chip 160. As explained below, the volatile memory chip 174 may be omitted in other embodiments.

[0091] In step 234, the processor 175 may be mounted on top of one or more volatile memory chips 174 (or CBA memory chips 160, wherein chip 174 is omitted), as... Figure 27 The perspective view shows an integrated processor / memory core. In one embodiment, processor 175 may be a dedicated processor, such as a graphics processing unit (GPU) or artificial intelligence (AI) processor capable of performing parallel processing, complex graphics rendering, and / or other high-bandwidth, data-intensive tasks. Processor 175 may include multiple processing cores that enable processor 175 to execute multiple computational tasks simultaneously. In another embodiment, processor 175 may be other types of processors, such as a conventional central processing unit.

[0092] In this embodiment, processor 175 may have the same footprint as volatile memory chip 174 and CBA memory chip 160. The bottom surface of processor 175 may have a pattern of contact pads or microbumps (not shown) that match the number and arrangement of bump pads 306 on the top surface of the uppermost volatile memory chip 174. By mating the bump pads of the volatile memory chip with the contact pads on the bottom surface of processor 175, processor 175 can be physically and electrically coupled to the uppermost volatile memory chip 174. The bonding between the corresponding bump pads / microbumps of processor 175 and the uppermost volatile memory chip can be accomplished using any of the methods described above for bonding CBA memory chip 160.

[0093] In step 236, a high-bandwidth memory (HBM) stack 176 may be mounted around one or more sides of chips 160, 174 and processor 175, such as Figures 28 to 29 The perspective and sectional views are shown. In the embodiment, each HBM stack 176 includes one or more HBM dies 178 mounted on a dedicated HBM controller 180. The number of HBM dies 178 in each stack can vary. Each die 178 in the HBM stack can be a volatile memory, such as DRAM. The HBM stack provides high-speed, high-bandwidth, and low-power memory for fast data access to a dedicated high-performance processor, such as a GPU or AI processor that may include processor 175. The controller 180 is used to operate and communicate with the dies 178 in each HBM stack 176. Although in Figure 17 The diagram shows the controller 180 at the bottom of each stack, but in another embodiment, the controller 180 may be located elsewhere in the stack 176.

[0094] In the illustrated embodiment, three HBM stacks 176 are present on each of two opposing sides of wafers 160, 174, and processor 175. In other embodiments, more or fewer stacks may be present around more or fewer sides. Each die in the stacks 176 may be electrically coupled to each other using TSVs, and the bottom surface of the stacks 176 may have contact pads 182 on the interposer 172 in number and arrangement (one of which is on...). Figure 16 The pattern of contact pads (not shown) that match the numbering in the middle layer. Each stack 176 may be physically and electrically coupled to pads 182 on the interposer layer 172, as described above regarding other pad couplings.

[0095] Figure 28 A perspective view of the completed integrated processing core 184 is shown, including an integrated CBA memory chip 160 mounted on an interposer layer 172, one or more volatile memory chips 174, a processor 175, and an HBM stack 176. Figure 29This is a cross-sectional view showing the integrated processing core 184 with internal electrical connections. Figure 29 For example, bump pads such as bump pad 106 are shown between CBA memory chip 160 and volatile memory chip 174, and between volatile memory chip 174 and processor 175. The figure also shows bump pad 116 between CBA memory chip 160 and interposer 172. Traces 185 are also shown within interposer 172 for electrically coupling processor 175 to high-bandwidth memory stack 176 (through a through-area between CBA memory chip 160 and volatile memory chip 174). Vias 186 through interposer 172 are also shown, coupling to pads or bumps 187 on the bottom surface of interposer 172 for electrically coupling processing core 184 to a printed circuit board of a host device (not shown).

[0096] In the final step 238, the entire processing core 184 can be encapsulated in a molding compound. In some embodiments, the encapsulation step 238 can be omitted. Therefore, step 238... Figure 1 The figures are shown in dashed lines. Note that the CBA memory chip 160 chips 102 and 112, the volatile memory chip 174, the processor 175, and the high-bandwidth semiconductor die 178 are shown in the figures for illustrative purposes only, and the thicknesses of the corresponding chips, processors, and high-bandwidth semiconductor dies are not drawn to scale.

[0097] The above-described processing core 184 illustrates one example of a component; however, it should be understood that various substitutions and / or additions can be made to the processing core 184 in other embodiments. For example, in the above-described embodiment, the processing core 184 has two readily available sources of high-bandwidth volatile memory—HBM stack 176 and volatile memory chips 174. However, if a sufficient number of volatile memory chips 174 are provided (e.g., four chips of 100 gigabytes to 200 gigabytes), the HBM stack 176 can be partially or completely omitted. Therefore, step 236 of adding the HBM stack 176 is... Figure 1 It is shown in dashed lines. Figure 30 Perspective view and Figure 31 The cross-sectional view shows an embodiment in which the HBM stack 176 is completely omitted.

[0098] exist Figure 30 and Figure 31 In one implementation, the integrated processing core 184 is mounted on the intermediary layer 172. However, omitting the HBM stack 176 also allows the intermediary layer 172 to be omitted. Figure 32 Perspective and Figure 33This embodiment is illustrated in a cross-sectional view. In this embodiment, the CBA memory chip 160 can be configured as a fan-out package (used without a substrate), wherein the bottom surface of the microbump pads 116 of the CBA memory chip 160 is redistributed (e.g., in a redistribution layer) to directly electrically couple the processing core 184 to the printed circuit board of a host device (not shown). Omitting the interposer layer 172 provides advantages such as a smaller form factor and lower power requirements.

[0099] Figure 34 This is a functional block diagram illustrating further details of an embodiment of a processing core 184 comprising a CBA memory chip 102 having a memory array chip 102 and a CMOS logic chip 112. The memory array chip 102 of the CBA memory chip 160 may include a memory structure 360 ​​of memory cells (such as a memory cell array) and read / write circuitry 368. The CMOS logic chip 112 may include control logic circuitry 350. The memory structure 360 ​​is addressable by word lines via a row decoder 364 and by bit lines via a column decoder 366. The read / write circuitry 368 may include multiple sense blocks (sensor circuitry) that allow parallel reading or programming of a page of memory cells.

[0100] Multiple memory elements in memory structure 360 ​​can be configured such that they are connected in series or that each element is individually accessible. By way of non-limiting example, a flash memory system (NAND memory) in a NAND configuration typically comprises memory elements connected in series. A NAND string is an example of a group of transistors connected in series, including memory cells and select-gate transistors.

[0101] The NAND memory array can be configured such that the array consists of multiple memory strings, wherein a string consists of multiple memory elements that share a single bit line and are accessed as a group. Alternatively, the memory elements of memory structure 160 can be configured such that each element is individually accessible, for example, a NOR memory array. NAND and NOR memory configurations are exemplary, and memory elements can be configured in other ways.

[0102] The memory structure 360 ​​can be a two-dimensional (2D) structure or a three-dimensional (3D) structure. The memory structure 360 ​​may include one or more arrays of memory elements (also referred to as memory cells). The 3D memory array is arranged such that the memory elements occupy multiple planes or multiple memory device levels, thereby forming a three-dimensional structure (i.e., in the x, y, and z directions, where the z direction is substantially perpendicular to the main flat surface of the first semiconductor wafer 102, and the x and y directions are substantially parallel to the main flat surface).

[0103] The memory structure 360 ​​on the first chip 102 can be controlled by control logic circuitry 350 on the second chip 112. The logic control circuitry 350 may have circuitry for controlling and driving memory elements to perform functions such as programming and reading. The control circuitry 350 cooperates with read / write circuitry 368 to perform memory operations on the memory structure 360. In an embodiment, the control circuitry 350 may include a state machine 352, an on-chip address decoder 354, and a power control module 356. The state machine 352 provides chip-level control of memory operations. A memory area 353 may be provided for operating the memory array 360, such as programming parameters for different rows or other groups of memory cells. These programming parameters may include bit line voltages and verification voltages.

[0104] The on-chip address decoder 354 provides an address interface between the hardware addresses used by the host device or memory controller (as explained below) and the hardware addresses used by decoders 364 and 366. The power control module 356 controls the power and voltage supplied to the word lines and bit lines during memory operations. This power control circuitry may include drivers for word line layers in a 3D configuration, source-side select gates, drain-side select gates, and source lines. The source-side select gate is a gate transistor at the source terminal of the NAND string, and the drain-side select gate is a transistor at the drain terminal of the NAND string.

[0105] This technology offers several advantages. The various implementations described above address the degradation problem of non-volatile memory in training AI processors and provide different memory solutions that allow for both training and inference of AI processors.

[0106] Various implementations also provide storage solutions that meet the high-capacity, low-latency requirements of specialized processors such as AI processors and GPUs. For example, a larger size of non-volatile memory chips matching the size of processor 175 and volatile memory chips provides the processor with a larger memory capacity. In the example, this capacity could be approximately 2 terabytes, which is sufficient even for complex processors such as GPUs or AI processors.

[0107] Simultaneously, the large surface area of ​​the volatile memory chip 174, which is in direct contact with the processor 175, and the fine-pitch electrical connections on this area allow for a large number of direct electrical connections, resulting in high-bandwidth data transfer between the volatile memory chip 174 and the processor 175. In the example, the large number of direct electrical connections allows for wide-word data transfer between the volatile memory chip 174 and the processor 175, thereby providing, for example, 1024-bit data transfer between the volatile memory chip 174 and the processor 175. The same high bandwidth rate can be achieved between the processor 175 and the CBA memory chip 160, and between the processor 175 and the HBM stack 176. This high-bandwidth data transfer supports the parallel processing and high-performance requirements of complex processors such as GPUs or AI processors. Compared to conventional processing cores where non-volatile memory is located far from the processor, integrating the processor 175 directly on top of the large-surface-area volatile memory chip 174 and the CBA memory chip 160 further provides reduced power requirements and parasitic effects.

[0108] As another advantage, the TSV in the through area allows wide-word data transfer between the processor 175 and the HBM stack 176, thereby again supporting high-bandwidth data transfer between the processor 175 and the HBM stack 176.

[0109] In the above embodiments, the first wafer 100 and the second wafer 110 can be diced after forming and bonding the memory array wafer 102 and the CMOS logic circuit wafer 112. The formed CBA memory wafer 160 can then be bonded to the processor 175 as described above to form an integrated processing core. In another embodiment, instead of dicing one or both of wafers 100 and 110, the wafers can be used as a whole. For example, wafers 100 and 110 can be formed and bonded together to form a single large CBA memory wafer. Subsequently, multiple processors 175 can be bonded on top of the CBA memory wafer.

[0110] High-bandwidth flash stacks and hybrid high-bandwidth storage provide high-capacity, low-latency memory for AI processors. Wide flash stack

[0111] In the embodiments described above, processor 175 may be supported by an HBM stack 176 of DRAM memory. In another aspect of the technology, processor 175 may alternatively be supported by a non-volatile memory stack (referred herein to as a high-bandwidth flash (HBF) stack) or a memory stack including both volatile and non-volatile memory (referred herein to as a hybrid HBF stack). References below... Figures 35 to 40 Describe the details of these inventions.

[0112] Figure 35 This is a perspective view of a single HBF stack 400. Each stack 400 may include... Figure 35The stack 400 contains multiple semiconductor dies 402 designated as dies 402-1, 402-2, ..., 402-n. In one example, eight semiconductor dies 402 may be present in the stack 400, but other numbers of semiconductor dies may be present, including, for example, 1, 2, 4, 16, 32, and 64 dies. Each semiconductor die 402 in the HBF stack may be a non-volatile memory such as NAND. The NAND die 402 may be formed from a single wafer, or the NAND die may be formed from a memory cell array wafer and a CMOS logic circuit wafer to form a CMOS bonding array (CBA) memory die. Such a die may be similar to the CBA memory chip 160 described above, but may be smaller. Other types of non-volatile memory are possible, including MRAM.

[0113] Each die in the stack can be subdivided into memory structures 406, also referred to herein as planes 406. Each plane 406 on a given semiconductor die 402 may be aligned with a corresponding plane 406 in other semiconductor dies 402 in the stack 176. In one example, each die 402 may include 24 planes 406, but in other embodiments, more or fewer planes 406 may be present, including, for example, 36 planes and 64 planes.

[0114] In the example, each plane 406 on each die 402 can be accessed independently and in parallel with each other plane 406 on each die 402. To achieve this, each plane 406 has its own set of dedicated signal lines. Each of these signal lines is defined by one of TSVs 408, 410, and 412. TSVs 408 may extend in rows adjacent to each plane 406. TSVs 410 may extend in columns adjacent to each plane 406. Each semiconductor die 402 may also include a TSV channel 414, which includes TSVs 412. In the example, the TSV channel 414 is disposed in the middle portion of the semiconductor die 402, aligned along the columns and / or rows of the plane 406. In the example, the TSV channel 414 includes one thousand and twenty-four TSVs 412. However, in other embodiments, the number of TSVs 412 in the channel 414 may be higher or lower. Although channel 414 is shown as a grid including TSV 412, other patterns are possible. TSVs 408, 410, and 412 extend through each semiconductor die 402 in the stack 400, and they are coupled to a controller die 416 at the bottom of the stack. Within a given plane 406, TSVs 408, 410, and 412 pass through... Figure 35 The metallization layer, not shown in the diagram but described above, is used to couple to a separate memory array.

[0115] A set of TSVs 408, 410, and / or 412 can be coupled to the memory array of each plane 406. Since each plane 406 is associated with its own set of signal lines, data can be written directly to and / or read directly from each plane 406 independently and in parallel with each other plane 406. In the example, each set of signal lines may include eight I / O lines comprising eight separate signal lines. Although eight signal lines are shown and described, each set of signal lines can have any number of signal lines. For example, each set of signal lines can support up to two hundred and fifty-six (or more) lines / two hundred and fifty-six (or more) signals. Thus, wide word, high bandwidth is achieved through signal interconnection within the HBF stack 400.

[0116] Figure 36 It is a perspective view of the completed processing core 420, which includes the processor 175 directly mounted on the intermediary layer 172 and surrounded by a memory stack including both the HBM stack 176 (volatile memory) and the HBF stack 400 (non-volatile memory). Figure 37 It is a cross-sectional view through the processing core 420 and through the HBF stack 400. Figure 37 For example, a bump pad 422 is shown between the processor 175 and the interposer 172 to directly electrically and physically couple the processor 175 to the interposer 172. An electrical trace 424 is also shown within the interposer 172 for electrically coupling the processor 175 to the HBF stack 400. The bottom surface of the HBF stack 400 may include microbump pads 426 for physically and electrically coupling the HBF stack 400 to the interposer 172. A via 186 is also shown through the interposer 172, coupled to pads or bumps 190 on the bottom surface of the interposer 172 for electrically coupling the processing core 420 to a printed circuit board of a host device (not shown).

[0117] Processing core 420 may be, for example, an AI processing core. During the inference phase where the AI ​​processing core 420 provides a query response, numerous read operations are performed, requiring significant memory. The HBF stack 400 meets this memory requirement. When formulating a response to a query, processor 175 performs intermediate computations that are written to memory. These write operations can be performed on the HBM stack 176, thereby preventing the HBF stack 400 from degrading over time. Although Figure 36 Two HBM stacks 176 and four HBF stacks 400 are shown, but these numbers may vary relative to each other depending on the needs of the processor 175. As also noted, additional HBM stacks 176 and / or HBF stacks 400 may be provided around additional sides of the processor 175.

[0118] Traditionally, non-volatile memory stacks have not been used to support dedicated processors, such as processor 175, because non-volatile memory has high latency and is not fast enough. However, the features of this technology allow the HBF stack 400 to be formed entirely of non-volatile memory. This greatly increases the available storage capacity for processor 175 while meeting the bandwidth and low latency requirements of processor 175.

[0119] One reason why stack 400 can be formed entirely of non-volatile memory dies and still meet the high bandwidth requirements of the dedicated processor 175 is the parallelism of data operations occurring within stack 400. As noted above, each stack can form a plane 406 ( Figure 35 Each of these planes 400 can be accessed individually and in parallel. This greatly improves the speed at which data can be accessed from the HBF stack 400.

[0120] Another reason why the stack 400 can be formed entirely of non-volatile memory dies while still meeting the high bandwidth requirements of the dedicated processor 175 is the wide-word signal path used in the HBF stack 400. As described above, each group of signal lines may include eight I / O lines, which consist of eight separate signal lines. In other embodiments, up to two hundred and fifty-six (or more) lines / two hundred and fifty-six (or more) signals may be present. This results in low-latency, high-bandwidth data exchange between the HBF stack 400 and the processor 175 on the interposer layer 172.

[0121] Another reason why stack 176 can be formed entirely from non-volatile memory dies and still meet the high bandwidth requirements of dedicated processor 175 is the nature of AI and other dedicated processors. Processors such as AI processors are able to pre-fetch data from HBF stack 400. Specifically, while AI processor 155 can perform steps and calculations in nanoseconds, traditionally, NAND memory may require microseconds (1000 times slower) to locate and access requested data from its memory. This can result in high latency when processor 175 processes information.

[0122] However, when the processor 175 according to this technology receives a query, for example, consisting of multiple tokens, the processor will have a microsecond delay while processing the first token. However, while transmitting a request for data for the first token, the processor can also transmit requests for data for the second and subsequent tokens to the HBF stack 400. Therefore, the controller 416 of the HBF stack 400 can prefetch data associated with the second and subsequent tokens. This prefetched data can be stored in a buffer within the controller 416 or elsewhere readily accessible to the processor 175. When the processor has completed its processing and calculation of the first token and the data requests for the second and subsequent tokens are transmitted, the prefetched data for the second and subsequent tokens is transmitted. There is no need to wait the several microseconds that would otherwise (without prefetching) would take to access the data for each of the second and subsequent tokens from memory. This also significantly reduces the latency for accessing data from the HBF stack 400.

[0123] These features enable the HBM stack 176 to support the processor 175 as a high-capacity, high-bandwidth non-volatile memory device. For example, an HBF stack 400 entirely composed of non-volatile memory can have a storage capacity of two TB or more, which is significantly higher than current HBMs composed of volatile memory. Simultaneously, the HBF stack 400 entirely composed of non-volatile memory can have a bandwidth capability of at least 1.5 TB per second, which is sufficient to meet the bandwidth requirements of the processor 175. The storage capacity and bandwidth provided above are examples and may be higher or lower in other embodiments.

[0124] As noted, it is useful to arrange both HBF stacks and HBM stacks around the dedicated processor 175, so that the HBF stack can be used to support high-capacity read operations, and the HBM stack can be used to support intermediate write operations of the processor without degrading the memory. In another aspect of this technology, the HBF stack and HBM stack can be integrated together to form a hybrid HBF stack that has both volatile and non-volatile memory.

[0125] An example of this hybrid HBF stack 430 is as follows: Figure 38 As shown in the perspective view. Each hybrid stack 430 can be manufactured and assembled in the same manner as the HBF stack 400, and has the same planar surface and TSV as the HBF stack 400. However, the hybrid HBF stack 430 contains a mixture of both volatile memory dies such as DRAM and non-volatile memory dies such as NAND.

[0126] Figure 39The edge view illustrates one possible configuration of dies 432 within the hybrid HBF stack 430. In one example, some dies 432 (i.e., dies 432-1, 432-2, and 432-3) may be volatile memory dies, and the remaining dies 432 (i.e., dies 432-4, 432-5, 432-6, 432-7, and 432-8) may be non-volatile memory dies. A controller 434 is configured with circuitry and protocols specific to each memory type, enabling a single controller 434 to manage all dies in the hybrid HBF stack 430. In another embodiment, both volatile and non-volatile memory dies may be designed to follow the same protocol. Figure 40 The edge view illustrates one possible alternative configuration in which half of the dies 432 (i.e., dies 432-1, 432-2, 432-3, and 432-4) may be volatile memory dies, and the remaining dies 432 (i.e., dies 432-5, 432-6, 432-7, and 432-8) may be non-volatile memory dies. In various embodiments, the volatile dies may be located closer to the bottom of the stack.

[0127] The hybrid stack 430 can be used with processor 175, such as Figure 38 As shown, a processing unit 434 is provided. When a read operation is performed by processor 175, the processor can access one or more non-volatile memories in one or more hybrid HBF stacks. When a write operation is performed by processor 175, the processor can access one or more volatile memories in one or more hybrid HBF stacks. It should be understood that any combination of volatile and non-volatile memory dies can be used in the stack 430, depending on the storage capacity and access speed requirements of processor 175. In various embodiments, each hybrid HBF stack 430 has the same volatile and non-volatile memory composition. However, in other embodiments, two or more stacks 430 may have different volatile and non-volatile memory compositions.

[0128] In summary, examples of this technology relate to a semiconductor device comprising: a signal carrier medium; a processing core mounted on the signal carrier medium; and one or more high-bandwidth flash (HBF) memory stacks mounted on the signal carrier medium, each HBF memory stack comprising: a plurality of non-volatile memory dies and a controller die; wherein each HBF memory stack is electrically coupled to the processing core to provide high-bandwidth memory support for the processing core.

[0129] In another example, the technology relates to a semiconductor device comprising: a signal carrier medium; a processing core mounted on the signal carrier medium; and one or more hybrid high-bandwidth flash (HBF) memory stacks mounted on the signal carrier medium, each hybrid HBF memory stack comprising: a plurality of non-volatile memory dies, a plurality of volatile memory dies, and a controller die controlling I / O operations on the plurality of non-volatile memory dies in the stack and controlling I / O operations on the plurality of volatile memory dies in the stack; wherein each hybrid HBF memory stack is electrically coupled to the processing core to provide high-bandwidth memory support for the processing core.

[0130] In another example, the technology relates to a semiconductor device comprising: a signal carrier medium; a processing core mounted on the signal carrier medium; and a non-volatile flash memory component mounted adjacent to and electrically coupled to the processing core on the signal carrier medium for providing bandwidth support of at least 1.5 terabytes per second for the processing core and for providing storage capacity support of at least 2 terabytes for the processing core.

[0131] The foregoing detailed description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in accordance with the foregoing teachings. The described embodiments have been chosen to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention in various embodiments and with various modifications suitable for the particular intended use. The scope of the invention is intended to be defined by the appended claims.

Claims

1. A semiconductor device, the semiconductor device comprising: Signal transport medium; A processing core, which is mounted on the signal transport medium; and One or more high-bandwidth flash (HBF) memory stacks, said one or more high-bandwidth flash (HBF) memory stacks mounted on the signal carrier medium, each HBF memory stack comprising: Multiple non-volatile memory dies, and Controller chip; Each HBF memory stack is electrically coupled to the processing core to provide high-bandwidth memory support for the processing core.

2. The semiconductor device of claim 1, wherein the plurality of non-volatile memory dies in the stack of the one or more high-bandwidth flash memory stacks comprise NAND memory dies.

3. The semiconductor device of claim 1, wherein the plurality of non-volatile memory dies in the stack of the one or more high-bandwidth flash memory stacks include CBA memory dies, each CBA memory die including a NAND die coupled to a CMOS logic circuit die.

4. The semiconductor device of claim 1, wherein the stack in the one or more HBF memory stacks comprises two or more non-volatile memory dies.

5. The semiconductor device of claim 1, wherein the one or more HBF memory stacks comprise a plurality of HBF memory stacks adjacent to and surrounding the processing core.

6. The semiconductor device of claim 5, further comprising one or more high-bandwidth memory (HBM) stacks, each of the one or more HBM stacks comprising a plurality of volatile memory dies.

7. The semiconductor device of claim 6, wherein the one or more HBM stacks comprise a plurality of HBM stacks adjacent to and surrounding the processing core.

8. The semiconductor device of claim 1, wherein each non-volatile memory die in the HBF memory stack comprises a plurality of planes.

9. The semiconductor device of claim 8, wherein the HBF memory stack further comprises a plurality of signal lines, each plane of the HBF memory stack having its own set of dedicated signal lines among the plurality of signal lines.

10. The semiconductor device of claim 9, wherein the controller is configured to independently and in parallel access the plurality of planes in the non-volatile memory die via the plurality of signal lines.

11. The semiconductor device of claim 9, wherein the set of dedicated signal lines in each plane comprises eight to two hundred and fifty-six I / O signal lines.

12. A semiconductor device, the semiconductor device comprising: Signal transport medium; A processing core, which is mounted on the signal transport medium; One or more hybrid high-bandwidth flash (HBF) memory stacks, said one or more hybrid high-bandwidth flash (HBF) memory stacks mounted on the signal carrier medium, each hybrid HBF memory stack comprising: Multiple non-volatile memory dies, Multiple volatile memory dies, and A controller die that controls I / O operations on the plurality of non-volatile memory dies in the stack and controls I / O operations on the plurality of volatile memory dies in the stack. Each of the hybrid HBF memory stacks is electrically coupled to the processing core to provide high-bandwidth memory support for the processing core.

13. The semiconductor device of claim 12, wherein the one or more hybrid HBF memory stacks comprise a plurality of hybrid HBF memory stacks adjacent to and surrounding the processing core.

14. The semiconductor device of claim 12, wherein the processing core is an artificial intelligence (AI) processing core, and the write operation performed by the AI ​​processing core writes to the volatile memory die within the stack of the one or more hybrid HBF stacks, and the read operation performed by the AI ​​processing core reads from the non-volatile memory die within the stack.

15. A semiconductor device, the semiconductor device comprising: Signal transport medium; A processing core, which is mounted on the signal transport medium; and A memory component, mounted adjacent to and electrically coupled to the processing core on the signal carrier medium, and comprising at least one or more non-volatile memory dies for providing bandwidth support of at least 0.5 terabytes per second for the processing core and for providing storage capacity support of at least 256 gigabytes for the processing core.

Images