Integrated circuit assembly

Abstract

The embodiment of the invention provides an integrated circuit assembly. The integrated circuit assembly includes: a first wafer die formed with a plurality of arithmetic units for performing logical operations in parallel; a plurality of access control units are formed on the second wafer bare chip; a plurality of storage units are formed on the at least one third wafer bare chip; wherein the second wafer bare chip is stacked between the first wafer bare chip and the at least one third wafer bare chip; wherein each arithmetic unit corresponds to the plurality of access control units, each access control unit corresponds to at least one storage unit, and each arithmetic unit accesses the target storage unit through the access control unit corresponding to the target storage unit.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to an integrated circuit component. Background Art

[0002] Neural network models, such as Large Language Models (LLMs), are currently popular AI applications. However, efficient inference in neural network models places extremely high demands on storage bandwidth. This is especially true for the attention mechanism processing during the generation phase of LLMs, which consumes significant storage resources given the available computing resources.

[0003] Currently, accelerators such as graphics processing units (GPUs) offer sufficient computing performance but limited memory access bandwidth, making model inference extremely expensive. Techniques such as data quantization and model pruning can be used in some cases, but these come with a certain performance penalty. Therefore, an integrated circuit solution based on near-memory computing is needed to improve memory access bandwidth while maintaining computing performance. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides an integrated circuit component to solve the above-mentioned problem.

[0005] According to a first aspect of an embodiment of the present invention, an integrated circuit component is provided, comprising: a first wafer die, on which a plurality of operation units are formed, and the plurality of operation units are used to perform logical operations in parallel; a second wafer die, on which a plurality of access control units are formed; and at least one third wafer die, on which a plurality of storage units are formed; wherein the second wafer die is stacked between the first wafer die and the at least one third wafer die; wherein each operation unit corresponds to the plurality of access control units, each access control unit corresponds to at least one storage unit, and each operation unit accesses the target storage unit via the access control unit corresponding to the target storage unit.

[0006] In another implementation of the present invention, each operation unit sends at least one data access instruction to at least one access control unit, so that each access control unit accesses at least one storage unit corresponding to the access control unit in response to the corresponding data access instruction.

[0007] In another implementation of the present invention, a first inter-die wiring is formed between the first wafer bare die and the second wafer bare die, and each operation unit sends at least one data access instruction to at least one access control unit via the first inter-die wiring.

[0008] In another embodiment of the present invention, a second inter-chip wiring is formed between the second wafer bare die and the at least one third wafer bare die, and each access control unit responds to a corresponding data access instruction and accesses at least one storage unit corresponding to the access control unit via the second inter-chip wiring.

[0009] In another implementation of the present invention, the integrated circuit component further includes a plurality of shared cache units, which are formed in the first wafer die or the second wafer die, and each shared cache unit is shared by at least two computing units.

[0010] In another implementation of the present invention, each operation unit sends a data read instruction to the corresponding access control unit, so that the access control unit reads the first data from the corresponding at least one storage unit to the shared cache unit shared by the operation units.

[0011] In another implementation of the present invention, each operation unit reads second data from a shared cache unit shared by the operation unit, and sends a data write instruction to a corresponding access control unit, causing the access control unit to write the second data into at least one corresponding storage unit.

[0012] In another embodiment of the present invention, a first unit array is formed in the first wafer bare die, and the first unit array includes the multiple operation units and the multiple shared cache units. Each operation unit is connected to the shared shared cache unit via a first intra-chip wiring, and the first intra-chip wiring is connected to the multiple access control units through a first inter-chip wiring. The first inter-chip wiring is arranged between the first wafer bare die and the second wafer bare die.

[0013] In another implementation of the present invention, each of the plurality of operation units is provided with a local cache unit, and the local cache unit of each operation unit is connected to a shared cache unit shared by the operation unit through the first on-chip wiring.

[0014] In another implementation of the present invention, the first on-chip wiring includes multiple first wiring nodes corresponding to the multiple operation units, and a first wiring network interconnecting the multiple first wiring nodes, and the local cache unit of each operation unit is connected to the shared cache unit shared by the operation unit through the first wiring node corresponding to the operation unit.

[0015] In another implementation of the present invention, a second cell array is formed in the second wafer die, and the second cell array includes the multiple access control units and multiple shared cache units, and each access control unit corresponds to at least one shared cache unit.

[0016] In another implementation of the present invention, each access control unit is connected to the corresponding at least one shared cache unit via a second intra-chip wiring, and the multiple operation units are connected to the second intra-chip wiring via a first inter-chip wiring, and the first inter-chip wiring is arranged between the first wafer bare die and the second wafer bare die.

[0017] In another implementation of the present invention, the second on-chip wiring includes multiple second wiring nodes corresponding to the multiple access control units, and a second wiring network interconnecting the multiple second wiring nodes, and each access control unit is connected to at least one shared cache unit corresponding to the access control unit via the second wiring node corresponding to the access control unit.

[0018] According to a second aspect of an embodiment of the present invention, an integrated circuit component is provided, comprising: a first wafer die, on which a plurality of operation units and a plurality of access control units are formed, the plurality of operation units being used to perform logical operations in parallel, each operation unit corresponding to the plurality of access control units, the plurality of operation units and the plurality of access control units being connected via a third on-chip wiring; a second wafer die, on which a fourth on-chip wiring is formed; and at least one third wafer die, on which a plurality of storage units are formed; wherein the second wafer die is stacked between the first wafer die and the at least one third wafer die, each access control unit corresponds to at least one storage unit, and each operation unit accesses the target storage unit via the access control unit corresponding to the target storage unit.

[0019] In the solution of the embodiment of the present invention, a second wafer die having multiple access control units is stacked between a first wafer die having multiple computing units and at least one third wafer die having multiple storage units. That is, the spatial stacking method between the second wafer die, the first wafer die and the at least one third wafer die is conducive to achieving efficient parallel data communication, so that the access control unit and the computing unit and the storage unit can all achieve efficient data communication, thereby improving the storage access bandwidth while ensuring computing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1A and Figure 1B Some examples of wafer die configurations in integrated circuit components are shown.

[0022] Figure 2 Schematic diagram of the structure of an integrated circuit component according to some embodiments of the present invention.

[0023] Figure 3A 、 Figure 3B and Figure 3C for Figure 2 Schematic structural diagram of each wafer die of an embodiment.

[0024] Figure 4 for Figure 2 A block diagram of the structure of an integrated circuit component of an example of an embodiment.

[0025] Figure 5 、 Figure 6A and Figure 6B for Figure 2 A structural block diagram of an integrated circuit component of another example of an embodiment.

[0026] Figure 7A 、 Figure 7B 、 Figure 7C and Figure 7D Schematic diagram showing different examples of on-chip wiring.

[0027] Figure 8 Schematic structural diagrams of integrated circuit components according to other embodiments of the present invention.

[0028] Figure 9 for Figure 8 A block diagram of the structure of an integrated circuit component of an example of an embodiment. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and detailedly described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0030] In the following detailed description, reference is made to the accompanying drawings, which form a part of the detailed description and illustrate exemplary embodiments. In addition, it is understood that other embodiments may be utilized and structural and / or logical changes may be made without departing from the scope of the claimed subject matter. It should also be noted that directions and references (e.g., up, down, top, bottom, etc.) may be used merely to facilitate the description of features in the drawings. Therefore, the following detailed description is not to be construed in a limiting sense, and the scope of the claimed subject matter is limited solely by the appended claims and their equivalents.

[0031] In the following description, numerous details are set forth. However, it will be apparent to those skilled in the art that the embodiments herein may be practiced without these specific details. In some cases, well-known methods and apparatus are shown in block diagram form rather than in detail to avoid blurring the embodiments herein. References throughout this specification to "an embodiment" or "one embodiment" or "some embodiments" mean that the specific features, structures, functions, or characteristics described in conjunction with the embodiment are included in at least one embodiment herein. Therefore, the phrases "in an embodiment" or "in one embodiment" or "some embodiments" appearing throughout this specification do not necessarily refer to the same embodiment. In addition, in one or more embodiments, specific features, structures, functions, or characteristics may be combined in any suitable manner. For example, the first embodiment may be combined with the second embodiment in any case where the specific features, structures, functions, or characteristics associated with the two embodiments are not mutually exclusive.

[0032] As used in the description and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0033] The terms "coupled" and "connected," along with their derivatives, may be used herein to describe functional or structural relationships between components. It should be understood that these terms are not intended to be synonymous with each other. On the contrary, in certain embodiments, "connected" may be used to indicate that two or more elements are in direct physical, optical, or electrical contact with each other. "Coupled" may be used to indicate that two or more elements are in direct or indirect (with other intermediate elements between them) physical or electrical contact with each other, and / or that two or more elements collaborate or interact with each other (e.g., as in a cause-and-effect relationship).

[0034] As used herein, the terms "above," "below," "between," and "on" refer to the relative position of one component or material with respect to other components or materials where such physical relationship is significant. For example, in the context of materials, a material or materials positioned above or below another material may be in direct contact or may have one or more intervening materials. Moreover, a material positioned between two materials or materials may be in direct contact with both layers or may have one or more intervening layers. In contrast, a first material or materials "above" a second material or materials is in direct contact with the second material or materials. Similar distinctions apply in the context of component assembly.

[0035] As used throughout this description and in the claims, a list of items linked by the term "at least one of" or "one or more of" may mean any combination of the listed items. For example, the phrase "at least one of A, B, or C" may mean A; B; C; A and B; A and C; B and C; or A, B, and C.

[0036] The term "circuit" or "module" may refer to one or more passive and / or active components that are arranged to cooperate with each other to provide a desired functionality. The term "signal" may refer to at least one current signal, voltage signal, or magnetic signal. The terms "substantially," "close to," "approximately," "close to," and "approximately" generally mean within + / - 10% of a target value.

[0037] The specific implementation of the embodiment of the present invention is further described below with reference to the accompanying drawings of the embodiment of the present invention.

[0038] The wafer layer is composed of pure silicon (Si) and includes a front side and a back side. The front side of the wafer layer is the main working surface for chip production on the wafer. The front side usually has a specific orientation and lattice structure for growing or building transistors, circuits and other semiconductor devices thereon, that is, a wiring layer is provided on the front side. The back side of the wafer layer is also called the back surface or backing, which is opposite to the front side. The back side is usually flat and has no crystal structure, and is used to provide mechanical stability for supporting and handling the wafer. There are usually no circuits or devices on the back side, and special treatments or coatings can be performed to meet specific needs, such as enhancing adhesion or improving heat conduction.

[0039] 3D wafer-level packaging refers to an integrated circuit component consisting of two or more wafer layers.

[0040] Hybrid bonding (HB) is a method to obtain denser interconnections between stacked chips. The hybrid bonding process allows wafers to be stacked front-to-front.

[0041] Through-Silicon Vias (TSV) mainly functions as electrical extension and interconnection along the Z axis (the coordinate axis perpendicular to the plane of the wafer layer).

[0042] The redistribution layer (RDL) serves as the electrical extension and interconnect for the XY plane (the plane where the wafer layers reside). In advanced packaging technologies like FIWLP (Fan-In Wafer Level Package) and FOWLP (Fan-Out Wafer Level Package), RDL is a key technology. It is used to fan-in or fan-out IO pads, creating different types of wafer-level packages.

[0043] The specific implementation of the embodiment of the present invention is further described below with reference to the accompanying drawings of the embodiment of the present invention.

[0044] Figure 1A and Figure 1B Figure 1 shows some examples of wafer dies in integrated circuit assemblies. In order to increase the memory access bandwidth, memory access channels can be added. Figure 1A The example wafer die is provided with a processing element (PE) and an access control unit (ie, Ctrl). Figure 1B Various storage units (i.e., Mem) are set up in the wafer die. In this scheme, data transmission is achieved between each computing unit and each access control unit using on-chip wiring, which makes the access overhead large under the premise of a given wafer die area, and the storage access bandwidth still has room for improvement.

[0045] Alternatively, in other examples of increasing storage access channels, each access control unit and the storage unit can be set on the same wafer die, and each access control unit and each storage unit use on-chip wiring to achieve data transmission. In addition, the computing unit is set on another wafer die. In this solution, the computing performance of the computing unit can be increased to a certain extent. This means that the access overhead is still large under the premise of a given wafer die area, and there is still room for improvement in the storage access bandwidth.

[0046] Generally speaking, when the area of ​​the wafer die is large, for example, the area of ​​the wafer die can reach 800mm 2 The wafer die includes dozens of corresponding memory units (memory banks). Each computing unit needs to access the control unit to aggregate these stored input and output data. Regardless of whether the aggregation process is sent to any wafer die, it will bring a large bandwidth overhead, resulting in a decrease in the actual available area of ​​the chip.

[0047] To this end, embodiments of the present invention propose a series of solutions to improve storage access bandwidth while ensuring computing performance.

[0048] Figure 2 The integrated circuit assembly includes a first wafer die 210, a second wafer die 220, and at least one third wafer die 230. The second wafer die 220 is stacked between the first wafer die 210 and the at least one third wafer die 230.

[0049] It should be understood that the term "die" in this context refers to a die produced by a specific process within a wafer layer. For example, the first and second dies may be logic dies, while the third die may be memory dies.

[0050] Furthermore, the first wafer die 210 is formed with a plurality of operation units 211, such as Figure 3A As shown. Multiple operation units 221 are used to perform logical operations in parallel. The second wafer bare chip 220 is formed with multiple access control units 221, such as Figure 3B At least one third wafer die 230 is formed with a plurality of storage units 231, such as Figure 3C shown.

[0051] Each operation unit 211 corresponds to a plurality of access control units 221 , each access control unit 221 corresponds to at least one storage unit 231 , and each operation unit 211 accesses a target storage unit 231 via the access control unit 221 corresponding to the target storage unit 231 .

[0052] In the solution of the embodiment of the present invention, a second wafer die having multiple access control units is stacked between a first wafer die having multiple computing units and at least one third wafer die having multiple storage units. That is, the spatial stacking method between the second wafer die, the first wafer die and the at least one third wafer die is conducive to achieving efficient parallel data communication, so that the access control unit and the computing unit and the storage unit can all achieve efficient data communication, thereby improving the storage access bandwidth while ensuring computing performance.

[0053] Furthermore, in some examples of performing parallel data access, each computing unit sends at least one data access instruction to at least one access control unit, causing each access control unit to access at least one storage unit corresponding to the access control unit in response to the corresponding data access instruction. Thus, the at least one data access instruction can be executed in parallel by each access control unit, further improving storage access bandwidth.

[0054] In a further example of performing parallel data access, a first inter-die wiring is formed between the first wafer die and the second wafer die, and each computing unit sends at least one data access instruction to at least one access control unit via the first inter-die wiring. Thus, the first inter-die wiring is formed between the first wafer die and the second wafer die, fully utilizing the corresponding area between the surfaces of the first wafer die and the second wafer die for wiring, saving space in integrated circuit components and improving integration.

[0055] Furthermore, a second inter-die wiring is formed between the second die and at least one third die. Each access control unit responds to a corresponding data access instruction and accesses at least one storage unit corresponding to the access control unit via the second inter-die wiring. Thus, the second inter-die wiring is formed between the second die and at least one third die, fully utilizing the corresponding area between the surface of the second die and the surface of the at least one third die for wiring, saving space in integrated circuit components and improving integration.

[0056] Furthermore, the integrated circuit assembly further includes a plurality of shared cache units, for example, Figure 4 、 Figure 5 、 Figure 6A and Figure 6B The secondary cache L2 shown. A plurality of shared cache units are formed in the first wafer die or the second wafer die, and each shared cache unit is shared by at least two computing units. Figure 4 In the example of , the shared cache unit is set in the first cell array of the first wafer die. For another example, Figure 5 、 Figure 6A and Figure 6B In the example, the shared cache unit is set in the second cell array of the second wafer die. Thus, the shared cache unit is formed in the first wafer die or the second wafer die, making the integrated circuit component compatible with the software architecture including the compilation framework.

[0057] In addition, each of the multiple computing units is provided with a local cache unit, for example, Figure 4 、 Figure 5 、 Figure 6A and Figure 6B The first level cache L1 is shown. In some examples, the local cache unit is set in the corresponding computing unit. Alternatively, the local cache unit can also be set outside the computing unit corresponding to the computing unit.

[0058] Generally, each local cache unit 21 and each shared cache unit 22 can be connected using an on-chip wiring 23. In the first wafer die 210, the on-chip wiring 23 is an example of a first on-chip wiring. In the second wafer die 220, the on-chip wiring 23 is an example of a second on-chip wiring.

[0059] Furthermore, the first wafer die 210 and the second wafer die 220 are connected by first inter-die wiring. Specifically, the first inter-die wiring can be hybrid bonded via each hybrid bonding point on the corresponding surfaces of the first wafer die 210 and the second wafer die 220. For another example, the second wafer die 220 and the third wafer die 230 are connected by second inter-die wiring. Specifically, in the case of a third wafer die, the second inter-die wiring can be hybrid bonded via each hybrid bonding point on the corresponding surfaces of the second wafer die 220 and the third wafer die 230. In the case of multiple third wafer dies, for adjacently stacked second wafer dies 220 and third wafer dies 230, hybrid bonding connection can be performed through various hybrid bonding points on the corresponding surfaces between the second wafer dies 220 and the third wafer dies 230. For non-adjacently stacked second wafer dies 220 and third wafer dies 230, such a third wafer die 230 can be connected to the storage unit in the third wafer die through cross-wafer layer connection technology such as through silicon via (TSV) to the third wafer die 230 stacked adjacent to the second wafer die 220, and then connected to the second wafer die 220 through hybrid bonding. That is, the memory cells in any of the two adjacently stacked third wafer dies can be connected to the other third wafer die through hybrid bonding, and the two non-adjacently stacked third wafer dies can be connected by hybrid bonding through the middle wafer die of the two third wafer dies through silicon via technology and then through the bonding points on the contrasting surfaces of the two adjacently stacked third wafer dies. Figure 4 、 Figure 5 、 Figure 6A and Figure 6B In various examples, 3D-link (three-dimensional interconnection) is an example of inter-chip wiring.

[0060] Furthermore, the inter-chip wiring between the local cache unit and the shared cache unit includes wiring nodes and wiring network connections. As some examples, the wiring network can be a network on chip (Network on Chip, NoC), which can be used for on-chip communication architecture in integrated circuits, as an efficient alternative to the global bus structure for internal communication of integrated circuit chips, breaking down complex communication tasks into smaller, scalable network structures. For example, the network on chip is based on the principle of network interconnection, interconnecting functional modules such as computing units, access control units, cache units or storage units inside a chip or wafer die, and transmitting data through communication between network nodes, providing high bandwidth and low latency data transmission, while also having high reliability and scalability. Alternatively, as other examples of wiring networks, the wiring network can be a crossbar array, an electronic component structure for interconnection and cross communication, as an efficient alternative to the global bus structure for internal communication of integrated circuit chips. The crossbar array can be commonly used to implement multiplexing and crossbar switches. For example, the crossbar array can connect different components or circuits to achieve signal routing, data transmission and interaction between circuits. The crosspoint array interconnects functional modules such as arithmetic units, access control units, cache units, or storage units within a chip or wafer die, and transmits data through communication between network nodes. The crosspoint array achieves flexibility and scalability. By controlling the crosspoint switch state, the connection configuration between functional modules such as arithmetic units, access control units, cache units, or storage units can be achieved. The crosspoint array can be increased or decreased as needed to meet more or less interconnection requirements in the system. In addition, in the first wafer die, the wiring node is called the first wiring node, and the wiring network is called the first wiring network. In the second wafer die, the wiring node is called the second wiring node, and the wiring network is called the second wiring network.

[0061] In addition, the first wiring network and the second wiring network can be connected through the first inter-die wiring. For example, the first wiring network is connected to each bonding point of the first wafer die, and the second wiring network is connected to each bonding point of the second wafer die, and then connected to each bonding point of the second wafer die through each bonding point of the first wafer die.

[0062] For example, in Figure 4In the example shown in FIG, each processing unit PE is provided with a level 1 cache L1, and each level 1 cache L1 is connected to each level 2 cache L2 via a first intra-chip wiring 23. Each level 2 cache L2 in the first wafer die 210 is connected to each access control unit in the second wafer die 220 via a three-dimensional interconnect (3D link). Each access control unit in the second wafer die 220 is connected to a storage unit in the third wafer die 230 via a three-dimensional interconnect (3D link).

[0063] Without loss of generality, a first unit array is formed in the first wafer die, and the first unit array includes a plurality of computing units and a plurality of shared cache units. Figure 4 In the example, the shared cache unit is provided in the first wafer die. Furthermore, the first on-chip wiring includes a plurality of first wiring nodes corresponding to the plurality of computing units, and a first wiring network interconnecting the plurality of first wiring nodes. The local cache unit of each computing unit is connected to the shared cache unit shared by the computing unit via the first wiring node corresponding to the computing unit. As a result, the first wiring nodes facilitate highly localized data access, while the first wiring network facilitates highly global data access, effectively balancing computing performance and storage access bandwidth.

[0064] Furthermore, each computing unit is connected to the shared cache unit via a first intra-chip wiring, which is connected to the plurality of access control units via a first inter-chip wiring, and the first inter-chip wiring is disposed between the first wafer die and the second wafer die. Thus, the first intra-chip wiring improves the overall data transmission efficiency of each unit in the first unit array, and the first inter-chip wiring improves the overall data transmission efficiency between the first wafer die and the second wafer die, further improving the storage access efficiency of the integrated circuit assembly.

[0065] Furthermore, each of the multiple computing units is provided with a local cache unit, and the local cache unit of each computing unit is connected to a shared cache unit shared by the computing units via a first intra-chip wiring. This facilitates further compatibility with software architectures, including compilation frameworks. The local cache units have higher locality than the shared cache units, but lower globality than the shared cache units. The shared cache units, on the other hand, have higher locality than the individual storage units, but lower globality than the individual storage units. Consequently, data access with higher locality is achieved through intra-chip wiring, while data access with higher globality is achieved through inter-chip wiring, significantly balancing computing performance and storage access bandwidth.

[0066] In some examples of the read and write process of each computing unit, each computing unit sends a data read instruction to the corresponding access control unit, causing the access control unit to read the first data from the corresponding at least one storage unit into a shared cache unit shared by the computing units. This further achieves compatibility with the software architecture including the compilation framework, and implements the reading of the first data through the shared cache unit.

[0067] In other examples of the read and write process of each computing unit, each computing unit reads the second data from a shared cache unit shared by the computing units and sends a data write instruction to the corresponding access control unit, causing the access control unit to write the second data to at least one corresponding storage unit. This further achieves compatibility with software architectures including compilation frameworks, and implements reading and writing of the second data through the shared cache unit.

[0068] For example, in Figure 5 and Figure 6A In the example shown in FIG, each processing unit PE is provided with a level 1 cache L1. Each level 1 cache L1 is connected to a three-dimensional interconnect (3D link) via a first intra-chip wiring, and then to a second intra-chip wiring 23. In the second wafer die 220, the second intra-chip wiring 23 is connected to the level 2 cache L2 or the access control unit 221. Each access control unit 221 in the second wafer die 220 is connected to a storage unit in the third wafer die 230 via a three-dimensional interconnect (3D link).

[0069] Without loss of generality, a second cell array is formed in the second wafer die. The second cell array includes multiple access control units and multiple shared cache units, with each access control unit corresponding to at least one shared cache unit. Thus, the multiple access control units and the multiple shared cache units are arranged in the second cell array, and the number of access control units and the number of shared cache units is no greater than the number of computing units. This balances the cell arrangement efficiency between the first cell array and the second cell array, improving the compactness of the integrated circuit assembly.

[0070] Furthermore, each access control unit is connected to at least one corresponding shared cache unit via a second intra-chip wiring, and multiple computing units are connected to the second intra-chip wiring via a first inter-chip wiring, with the first inter-chip wiring being provided between the first wafer die and the second wafer die. Thus, local data access is configured between the access control unit and the shared cache unit, improving the access bandwidth between the computing unit or its local cache unit and the shared cache unit.

[0071] Furthermore, the second on-chip wiring includes a plurality of second wiring nodes corresponding to the plurality of access control units, and a second wiring network interconnecting the plurality of second wiring nodes. Each access control unit is connected to at least one shared cache unit corresponding to the access control unit via the second wiring node corresponding to the access control unit. Thus, the second wiring node is conducive to data access with higher locality, and the second wiring network is conducive to data access with higher globality, which greatly balances computing performance and storage access bandwidth. For example, Figure 6B As shown, the shared cache unit 22 and the access control unit 221 can be connected in parallel to the second wiring node, and the second wiring node is connected to the operation unit 211 (dashed part) in the first wafer die 210 through three-dimensional interconnection. At the same time, the access control unit 221 can be connected to each storage unit 231 in the third wafer die 230 through three-dimensional interconnection.

[0072] In some examples of the read and write process of each computing unit, each computing unit sends a data read instruction to the corresponding access control unit, causing the access control unit to read the first data from the corresponding at least one storage unit into a shared cache unit shared by the computing units. This further achieves compatibility with the software architecture including the compilation framework, and implements the reading of the first data through the shared cache unit.

[0073] In other examples of the read and write process of each computing unit, each computing unit reads the second data from a shared cache unit shared by the computing units and sends a data write instruction to the corresponding access control unit, causing the access control unit to write the second data to at least one corresponding storage unit. This further achieves compatibility with software architectures including compilation frameworks, and implements reading and writing of the second data through the shared cache unit.

[0074] The following will be combined Figure 7A 、 Figure 7B 、 Figure 7C and Figure 7D The following describes different examples of on-chip routing. The on-chip routing 23 of the embodiment of the present invention includes a first on-chip routing and a second on-chip routing. It should be understood that the first on-chip routing includes a first routing node 101 and a first routing network formed therefrom, and the second on-chip routing includes a second routing node 201 and a second routing network formed therefrom.

[0075] like Figure 7AAs shown, the first intra-chip wiring is arranged in the first unit array of the first wafer bare die 210. The first intra-chip wiring interconnects each operation unit 211 and each first wiring node 101. In some examples, the first unit array may have two array dimensions for interconnection, so as to achieve reliable wiring while ensuring the compactness of the devices of the first unit array. The first wiring nodes 101 are not adjacent to each other, and while maintaining the transmission efficiency of the first wiring network, they provide arrangement space for the operation units 211. For example, in each array dimension, the first wiring nodes 101 are arranged at intervals (that is, they are arranged at intervals of one array position, or they can be arranged at intervals of two or more array positions). In addition, the operation units 211 are arranged at other array positions. If the ratio between the number of first wiring nodes 101 and the number of operation units 211 is 1:1, then the first wiring nodes 101 are arranged at intervals of one array position.

[0076] like Figure 7B As shown, the second on-chip wiring is arranged in the second cell array of the second wafer bare die 220. The second wiring nodes 201 are not adjacent to each other, while maintaining the transmission efficiency of the second wiring network, providing arrangement space for access control units 221 and shared cache units 22. For example, in each array dimension, the second wiring nodes are arranged at intervals (i.e., arranged at intervals of one array position, or at intervals of two or more array positions). In addition, based on the ratio between the number of access control units 221 and the number of shared cache units 22, the access control units 221 and the shared cache units 22 are arranged at other array positions. For example, if the ratio between the number of second wiring nodes, the number of access control units 221, and the number of shared cache units 22 is 2:1:1, then adjacent shared cache units 22 or adjacent access control units 221 can be arranged at intervals of two array positions in each dimension, so that adjacent shared cache units 22 are evenly distributed in the second cell array, or adjacent access control units 221 are evenly distributed in the second cell array.

[0077] It should be understood that in order to form Figure 6A and Figure 6B The on-chip wiring of the first unit array requires forming a second wiring network between each second wiring node 201, which can be achieved by interconnecting each second wiring node 201. In addition, each first wiring node 101 in the first unit array can be connected by Figure 7B Alternatively, direct interconnection is not necessary, because when different computing units perform parallel processing, the frequency of accessing the shared cache unit is higher than direct access between the various computing units.

[0078] like Figure 7CAs shown, if the ratio between the number of first wiring nodes 101 and the number of computing units 211 is 1:1, a second wiring node can be provided in the other array dimension for every other array position in one array dimension. Accordingly, a computing unit can be provided in the other array dimension for every other array position in the other array dimension, so that the ratio between the number of first wiring nodes 101 and the number of computing units 211 is 1:1. Thus, by providing each first wiring node 101 along the other array dimension for every other array position in one array dimension, the distribution of the first wiring nodes 101 is ensured while improving the efficiency of the first wiring nodes 101.

[0079] like Figure 7D As shown, if the ratio between the number of second routing nodes 201, the number of access control units 221, and the number of shared cache units 22 is 2:1:1, each second routing node 201 can be arranged along the other array dimension for every array position along one array dimension, thereby forming a second routing network formed by each second routing node 201. Accordingly, for every array position along one array dimension, shared cache units 22 and access control units 221 are alternately arranged along the other array dimension, so that the spacing between adjacent access control units 221 along one array dimension is two array positions, and the spacing between adjacent access control units 221 along the other array dimension is four array positions. Furthermore, the spacing between adjacent shared cache units 22 along one array dimension is two array positions, and the spacing between adjacent shared cache units 22 along the other array dimension is four array positions. Thus, by arranging each second routing node 201 along the other array dimension for every array position along one array dimension, the distribution of each second routing node 201 is ensured while improving the efficiency of the second routing nodes.

[0080] It should be understood that hybrid bonding points between the first wafer die 210 and the second wafer die 220 can be implemented based on some or all routing nodes. For example, hybrid bonding can be performed on at least some of the first routing nodes 101 that correspond to at least some of the second routing nodes 201, forming inter-die routing. In some examples, all first routing nodes 101 are aligned with all second routing nodes 201. In other examples, for example, if the number of periodic array positions of the first routing nodes 101 is less than the number of periodic array positions of the second routing nodes 201, some first routing nodes 101 are aligned with all second routing nodes 201. In other examples, if the number of periodic array positions of the first routing nodes 101 is greater than the number of periodic array positions of the second routing nodes 201, all first routing nodes 101 are aligned with some second routing nodes 201. Furthermore, at least some of the second routing nodes 201 may also form corresponding bonding points on the hybrid bonding surface of the second wafer die 210 that is opposite the third wafer die 230. The third wafer die 230 includes multiple memory channels, each memory channel including multiple memory cells. Bonding points are formed on the hybrid bonding surface of the third wafer die 230, which is opposite to the second wafer die 210, and are opposite to the bonding points of the second wafer die 210. Each bonding point of the third wafer die 230 corresponds to a plurality of memory channels (e.g., in a one-to-one correspondence). In other words, the bonding points corresponding to the memory channels are electrically connected to all memory cells of the third wafer die 230.

[0081] Figure 8 Schematic structural diagrams of integrated circuit components according to other embodiments of the present invention. Figure 8 The integrated circuit components include:

[0082] A first wafer die 210 , a second wafer die 220 and at least one third wafer die 230 .

[0083] The first wafer die 210 is formed with multiple arithmetic units and multiple access control units. The multiple arithmetic units are used to perform logical operations in parallel. Each arithmetic unit corresponds to multiple access control units. The multiple arithmetic units and the multiple access control units are connected via third intra-chip wiring. The second wafer die 220 is formed with fourth intra-chip wiring. At least one third wafer die 230 is formed with multiple storage units.

[0084] The second wafer die is stacked between the first wafer die and at least one third wafer die, each access control unit corresponds to at least one storage unit, and each operation unit accesses the target storage unit via the access control unit corresponding to the target storage unit.

[0085] In the solution of the embodiment of the present invention, a second wafer die having a fourth intra-chip wiring is stacked between a first wafer die and at least one third wafer die having a plurality of storage units. That is, the spatial stacking method between the second wafer die, the first wafer die and the at least one third wafer die is conducive to achieving efficient parallel data communication. Therefore, each access control unit in the first wafer die can perform efficient data communication with each storage unit through the fourth intra-chip wiring, thereby improving the storage access bandwidth while ensuring computing performance.

[0086] like Figure 9 As shown, in the first wafer bare die 210, each operation unit PE is provided with a first-level cache unit L1, each first-level cache unit L1 is connected to each second-level cache unit L2 via a first intra-chip wiring, and each second-level cache unit is connected to each access control unit via a first intra-chip wiring. Each access control unit is connected to a fourth intra-chip wiring in the second wafer bare die 220 via a three-dimensional interconnect (an example of a first inter-chip wiring). The fourth intra-chip wiring in the second wafer bare die 220 is connected to a memory unit in the third wafer bare die 230 via a three-dimensional interconnect (an example of a third inter-chip wiring).

[0087] That is to say, Figure 9 Compared with the embodiment Figure 2 In various embodiments, each access control unit is disposed in the first cell array of the first wafer die, rather than in the second cell array of the second wafer die. The access control unit is connected to other cells in the same cell array using intra-chip wiring, and is connected to other cells not in the same cell array using inter-chip wiring.

[0088] Thus far, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0089] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0090] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0091] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. An integrated circuit assembly comprising: A first wafer die is formed with a plurality of operation units, wherein the plurality of operation units are used to perform logic operations in parallel; A second wafer die is formed with a plurality of access control units; at least one third wafer bare die having a plurality of memory cells formed thereon; wherein the second wafer die is stacked between the first wafer die and the at least one third wafer die; Each operation unit corresponds to the multiple access control units, each access control unit corresponds to at least one storage unit, and each operation unit accesses the target storage unit via the access control unit corresponding to the target storage unit.

2. The integrated circuit assembly according to claim 1, wherein: Each operation unit sends at least one data access instruction to at least one access control unit, so that each access control unit accesses at least one storage unit corresponding to the access control unit in response to the corresponding data access instruction.

3. The integrated circuit assembly of claim 2, wherein: A first inter-die wiring is formed between the first wafer bare die and the second wafer bare die, and each operation unit sends at least one data access instruction to at least one access control unit via the first inter-die wiring.

4. The integrated circuit assembly according to claim 2, wherein: A second inter-die wiring is formed between the second wafer bare die and the at least one third wafer bare die, and each access control unit accesses at least one storage unit corresponding to the access control unit via the second inter-die wiring in response to a corresponding data access instruction.

5. The integrated circuit assembly of claim 1 , wherein: The integrated circuit component further includes a plurality of shared cache units formed in the first wafer die or the second wafer die, and each shared cache unit is shared by at least two computing units.

6. The integrated circuit assembly of claim 5, wherein: Each operation unit sends a data read instruction to the corresponding access control unit, so that the access control unit reads the first data from the corresponding at least one storage unit to the shared cache unit shared by the operation units.

7. The integrated circuit assembly of claim 5, wherein: Each operation unit reads the second data from the shared cache unit shared by the operation unit, and sends a data write instruction to the corresponding access control unit, so that the access control unit writes the second data into the corresponding at least one storage unit.

8. The integrated circuit assembly of claim 5, wherein: A first unit array is formed in the first wafer die, and the first unit array includes the multiple operation units and the multiple shared cache units. Each operation unit is connected to the shared shared cache unit via a first intra-chip wiring. The first intra-chip wiring is connected to the multiple access control units through a first inter-chip wiring. The first inter-chip wiring is arranged between the first wafer die and the second wafer die.

9. The integrated circuit assembly of claim 5, wherein: The multiple operation units are respectively provided with a local cache unit, and the local cache unit of each operation unit is connected to the shared cache unit shared by the operation unit through the first on-chip wiring.

10. The integrated circuit assembly of claim 9, wherein: The first on-chip wiring includes multiple first wiring nodes corresponding to the multiple operation units, and a first wiring network interconnecting the multiple first wiring nodes. The local cache unit of each operation unit is connected to the shared cache unit shared by the operation unit through the first wiring node corresponding to the operation unit.

11. The integrated circuit assembly of claim 5, wherein: A second cell array is formed in the second wafer die. The second cell array includes the multiple access control units and multiple shared cache units. Each access control unit corresponds to at least one shared cache unit.

12. The integrated circuit assembly of claim 11, wherein: Each access control unit is connected to the corresponding at least one shared cache unit via a second intra-chip wiring, and the multiple operation units are connected to the second intra-chip wiring via a first inter-chip wiring, and the first inter-chip wiring is set between the first wafer bare die and the second wafer bare die.

13. The integrated circuit assembly of claim 12, wherein: The second on-chip wiring includes multiple second wiring nodes corresponding to the multiple access control units, and a second wiring network interconnecting the multiple second wiring nodes. Each access control unit is connected to at least one shared cache unit corresponding to the access control unit via the second wiring node corresponding to the access control unit.

14. An integrated circuit assembly comprising: A first wafer bare die is formed with a plurality of operation units and a plurality of access control units, wherein the plurality of operation units are used to perform logic operations in parallel, each operation unit corresponds to the plurality of access control units, and the plurality of operation units and the plurality of access control units are connected via a third on-chip wiring; The second wafer bare die is formed with a fourth intra-chip wiring; at least one third wafer bare die having a plurality of memory cells formed thereon; The second wafer die is stacked between the first wafer die and the at least one third wafer die, each access control unit corresponds to at least one storage unit, and each operation unit accesses the target storage unit via the access control unit corresponding to the target storage unit.

Images