Electronic device and distributed computing system

Abstract

Distributed computing systems and electronic devices are disclosed for implementing memory stacked computing (CMS) semiconductor hardware (e.g., dies, packages, etc.) having an architecture that integrates memory and computing on a die, implements short-range, high-bandwidth communication between components, and minimizes data traffic. The electronic device includes a first die comprising a first computing component, a second computing component, a first set of through-silicon vias (TSVs) associated with the first computing component and connecting the stacked dies of the electronic device, and a second set of TSVs associated with the second computing component and connecting the stacked dies of the electronic device. The electronic device includes a second die stacked on the first die. The second die may include a first memory bank module connected to the first computing component using the first set of TSVs and a second memory bank module connected to the second computing component using the second set of TSVs.

Description

Technical Field

[0001] Some aspects of embodiments of this disclosure generally relate to semiconductor circuits. Background Technology

[0002] High-bandwidth memory (HBM) is a high-performance memory system consisting of multiple three-dimensional (3D) stacked dynamic random-access memory (DRAM) dies. Various applications, such as deep neural networks and artificial intelligence (AI), may require significant computational and memory capabilities to train on diverse datasets and learn with high accuracy. For such applications, high memory bandwidth can be desirable. Memory bandwidth can be described from the perspectives of core bandwidth and bus bandwidth. As the number of stacked DRAM dies increases while sharing the same bus, bus bandwidth can become a limiting factor for memory performance.

[0003] The continued expansion of AI demands ever-increasing memory bandwidth and capacity. Attempting to scale memory bandwidth to be optimal for AI-based applications using a stacked HBM architecture (e.g., increasing memory bandwidth for larger neural networks and / or more complex AI computations) can lead to various drawbacks. For example, there may be shoreline limitations (e.g., limited pin count) and high power consumption associated with using physically large stacks of HBM to scale memory bandwidth. Additionally, the large amount of high-bandwidth data movement between stacked DRAM dies and on-chip processing elements (e.g., logic dies, host, etc.) can result in increased energy consumption. Furthermore, architectures that implement logic operations within the HBM core die can provide computational power but may threaten to exceed the thermal budget limitations associated with the die and / or package.

[0004] Therefore, a flexible system architecture is desired that utilizes independently operating distributed computing systems co-packaged beneath the DRAM die to reduce or minimize data traffic (e.g., reduce power consumption, eliminate physical common buses) and maintain thermal budget, while performing computation in a manner that provides what may be the best high-bandwidth, energy-efficient memory subsystem for AI-based applications.

[0005] The information disclosed in this background section is intended to enhance the understanding of the context of this disclosure, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0006] Some aspects of embodiments of this disclosure generally relate to memory stacked computing (CMS) semiconductor hardware (e.g., dies, packages, etc.) having an architecture that integrates memory and computing on a die, implements short-range, high-bandwidth communication between components, and minimizes data traffic. In some embodiments, an apparatus may include a first die comprising: a first computing component; a second computing component; a first set of through-silicon vias (TSVs) associated with the first computing component and connecting the stacked dies of the apparatus; and a second set of TSVs associated with the second computing component and connecting the stacked dies of the apparatus. The apparatus may include a second die stacked on the first die. The second die may include: a first memory bank module connected to the first computing component using the first set of TSVs; and a second memory bank module connected to the second computing component using the second set of TSVs.

[0007] In some embodiments, the apparatus may additionally include a third die stacked on the second die. The third die may include a third memory module connected to the first computing component using a first set of TSVs; and a fourth memory module connected to the second computing component using a second set of TSVs.

[0008] In some embodiments, a first memory storage module connected to a first computing component using a first set of TSVs can form a first processing element, the first processing element being configured to perform computational functions using data from the first memory storage module, and a second memory storage module connected to a second computing component using a second set of TSVs can form a second processing element, the second processing element being configured to perform computational functions using data from the second memory storage module.

[0009] In some embodiments, the apparatus may further include: a plurality of interconnect ports on the first die connecting the first processing element and the second processing element to one or more additional processing elements. The first computing component and the second computing component may include computing circuitry configured to perform computing functions, and the one or more additional processing elements may include computing components connected to at least one memory memory module from an array of memory memory modules, the at least one memory memory module from the array of memory memory modules being arranged on the first die stacked thereon and using corresponding TSV groups.

[0010] In some embodiments, for the one or more additional processing elements, their respective computing components may be configured to perform computing functions on data from their respective at least one memory storage module stacked thereon.

[0011] In some embodiments, the apparatus may additionally include: a controller programmed to perform data transfer between a first computing component and a first memory storage module, data transfer between a second computing component and a second memory storage module, and data transfer between a respective computing component of the one or more additional processing elements and its respective at least one memory storage module.

[0012] In some embodiments, the first group of TSVs and the second group of TSVs may be among multiple TSVs distributed in multiple regions of the first die.

[0013] In some embodiments, a first set of TSVs may include at least one of the plurality of TSVs in a first region of the first die that contacts the first computing component, and a second set of TSVs may include at least one of the plurality of TSVs in a second region of the first die that contacts the second computing component.

[0014] In some embodiments, the first processing element may additionally include a third memory storage module and be configured to perform computational functions using data from the third memory storage module, and the second processing element may additionally include a fourth memory storage module and be configured to perform computational functions using data from the fourth memory storage module.

[0015] In some embodiments, the stacked dies of the device may include a plurality of additional dies stacked on top of a third die.

[0016] In some embodiments, the apparatus may additionally include a dynamic random access memory (DRAM) die, and one or more of a second or third die may include a DRAM die.

[0017] In some embodiments, the computational function may include one or more of the following: matrix multiplication; dot product; activation function; or mathematical functions associated with artificial intelligence (AI) applications.

[0018] In some embodiments, an apparatus may include a first die, which may include: a plurality of TSVs distributed in a plurality of regions of the first die; a first computing component; a second computing component; and a first set of TSVs associated with the first computing component and connecting the stacked dies of the apparatus. The first set of TSVs may include one or more of the plurality of TSVs in a first region of the first die. The first die may include a second set of TSVs associated with the second computing component and connecting the stacked dies of the apparatus. The second set of TSVs may include one or more of the plurality of TSVs in a second region of the first die, different from the first region. The apparatus may include a second die stacked on the first die. The second die may include a first memory storage module connected to the first computing component using the first set of TSVs and a second memory storage module connected to the second computing component using the second set of TSVs.

[0019] In some embodiments, a first memory storage module connected to a first computing component using a first set of TSVs can form a first processing element, the first processing element being configured to perform computational functions using data from the first memory storage module, and a second memory storage module connected to a second computing component using a second set of TSVs can form a second processing element, the second processing element being configured to perform computational functions using data from the second memory storage module.

[0020] In some embodiments, the stacking of the device may include a plurality of additional dies stacked on top of a second die.

[0021] In some embodiments, the apparatus may additionally include a DRAM die, and one or more of the second die and the plurality of additional dies may include a DRAM die.

[0022] In some embodiments, the plurality of TSVs may be configured to transmit data stored on a DRAM die to a first die.

[0023] In some embodiments, the apparatus may additionally include: a plurality of interconnect ports configured on a first die for connecting a first processing element and a second processing element to the plurality of processing elements to implement a distributed computing system.

[0024] In some embodiments, a system may include an accelerator processor configured to perform computational functions. The accelerator processor may include a first die, which may include a first computing component, a second computing component, a first set of TSVs (Transmission Storage Vehicles) of stacked dies associated with and connecting the device to the first computing component, and a second set of TSVs of stacked dies associated with and connecting the device to the second computing component. The accelerator processor may include a second die stacked on the first die. The second die may include a first memory storage module connected to the first computing component using the first set of TSVs and a second memory storage module connected to the second computing component using the second set of TSVs. The system may include a memory storing instructions to be executed by the accelerator processor to perform computational functions. Attached Figure Description

[0025] In the following sections, aspects of the subject matter disclosed herein will be described with reference to exemplary embodiments shown in the accompanying drawings.

[0026] Figure 1A This is an implementation on a bare die according to some embodiments of this disclosure. Figure 1B A perspective view of an example memory-stacked computing (CMS) device of a distributed computing system of processing elements depicted in the figure.

[0027] Figure 1B The present disclosure describes some embodiments of having included Figure 1A An example of a microarchitecture processing element in a CMS device.

[0028] Figure 1C According to some embodiments of this disclosure Figure 1A The side view of the CMS device depicted in the image.

[0029] Figure 2A According to some embodiments of this disclosure Figure 1A The floor plan of the example CMS device is depicted in the figure.

[0030] Figure 2B This is a plan view of another example CMS device according to some embodiments of the present disclosure.

[0031] Figure 3 Another example CMS device with an architecture including a memory storage module and a 4-hi stack is depicted according to some embodiments of the present disclosure.

[0032] Figure 4 Another example CMS device with an architecture including a memory storage module and an 8-hi stack is depicted according to some embodiments of the present disclosure.

[0033] Figure 5 The present disclosure describes some embodiments having including Figure 3 Another example of a CMS device constructed from multiple CMS devices is described in the diagram.

[0034] Figure 6 This describes some embodiments of the present disclosure for use in Figure 1A A flowchart illustrating an example operation of a method for transmitting data within the architecture of a CMS device.

[0035] Figure 7 This is for implementation according to some embodiments of the present disclosure. Figure 1A A block diagram of an example electronic device depicting the hardware of a CMS device. Detailed Implementation

[0036] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the disclosure. However, those skilled in the art will understand that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid obscuring the subject matter disclosed herein.

[0037] Throughout this specification, references to "an embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment disclosed herein. Therefore, the phrases "in one embodiment," "in an embodiment," or "according to an embodiment" (or other phrases with similar meanings) throughout this specification do not necessarily all refer to the same embodiment. Furthermore, in some embodiments (e.g., in one or more embodiments), particular features, structures, or characteristics may be combined in any suitable manner. In this regard, as used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not to be construed as necessarily preferred or advantageous over other embodiments. Additionally, particular features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. Furthermore, depending on the context discussed herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. Similarly, hyphenated terms (e.g., "two-dimensional", "pre-defined", "pixel-specific", etc.) may occasionally be used interchangeably with their corresponding non-hyphenated versions (e.g., "two-dimensional", "pre-defined", "pixel-specific", etc.), and uppercase terms (e.g., "counter clock", "row select", "pixout", etc.) may be used interchangeably with their corresponding non-uppercase versions (e.g., "counter clock", "row select", "pixout", etc.). Such occasional interchangeability should not be considered inconsistent with each other.

[0038] Furthermore, depending on the context of this discussion, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. It should also be noted that the various figures shown and discussed herein (including component diagrams) are for illustrative purposes only and are not drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Additionally, reference numerals have been repeated between figures where deemed appropriate to indicate corresponding and / or similar elements.

[0039] The terminology used herein is for the purpose of describing some exemplary embodiments only and is not intended to limit the claimed subject matter. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are also intended to include the plural forms. It will also be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof.

[0040] It will be understood that when an element or layer is referred to as being "on" another element or layer, "connected to," or "bonded to" another element or layer, the element or layer may be directly on, directly connected to, or directly bonded to the other element or layer, or an intermediary element or intermediary layer may be present. Conversely, when an element is referred to as being "directly on" another element or layer, "directly connected to," or "directly bonded to" another element or layer, an intermediary element or intermediary layer is not present. The same notation always refers to the same element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0041] Unless explicitly defined as such, terms such as “first,” “second,” etc., as used herein are used as labels for nouns that follow them and do not imply any kind of ordering (e.g., spatial, temporal, logical, etc.). Furthermore, the same reference numerals may be used across two or more figures to refer to parts (components), components, blocks, circuits, units, or modules having the same or similar functions. However, such usage is merely for the sake of simplicity and ease of discussion; it does not imply that the construction or architectural details of such components or units are identical across all embodiments or that such commonly referenced parts (components) / modules are the only way some of the exemplary embodiments disclosed herein are implemented.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject pertains. It will also be understood that, unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and shall not be interpreted in an idealized or overly formalized sense.

[0043] As used herein, the term "module" refers to any combination of software, firmware, and / or hardware configured to provide the functionality described herein in conjunction with modules. For example, software may be embodied as a software package, code, and / or instruction set or instructions, and the term "hardware" as used in any implementation described again may include, for example, assemblies, hardwired circuitry, programmable circuitry, state machine circuitry, and / or firmware that store instructions executable by programmable circuitry, either individually or in any combination. Modules may be embodied collectively or individually as parts of circuitry that form a larger system, such as, but not limited to, integrated circuits (ICs), system-on-a-chip (SoCs), assemblies, etc.

[0044] In the field of computer technology, AI and other deep learning applications are becoming increasingly prevalent, and demand is currently booming. As AI computing applications grow, new hardware may be needed to enable new applications across fields such as image and speech classification, media analytics, healthcare, autonomous machines, and intelligent assistants. For example, in deep neural network algorithms, the size of the dataset may exceed the computing power of available hardware. Furthermore, as emerging AI applications become more widespread and complex, significant computational and memory capabilities are required to train on diverse datasets and learn with high accuracy. Additionally, as applications like high-performance computing (HPC) and graphics algorithms become data- and computationally intensive, energy efficiency and low latency may become increasingly desirable.

[0045] Some server computing environments can leverage configurations that allow computation to be performed physically close to the location where data resides. By implementing data movement over relatively short distances, higher bandwidth and reduced energy consumption can be achieved for computing systems performing more complex computations, such as AI applications. An example of a hardware architecture that can be modified to support computation (or logic) close to memory is high-bandwidth memory (HBM). HBM is a type of computer memory designed for high-speed data transfer and implemented through a stacked architecture of memory chips. For example, HBM is high-performance memory, which comprises dynamic random access memory (DRAM) dies stacked on top of each other in a manner that achieves high bandwidth while using less power with a small form factor.

[0046] The evolution of HBM, High Bandwidth Memory 2 (HBM2), can include, for example, up to 12 dies per stack and provides pin transfer rates of 2.4 GT / s (gigabit-per-second) or faster. HBM and HBM2 have been used to implement hardware (such as parallel accelerators) that provides the high bandwidth, high capacity, and computing power that AI applications may require. Other versions of HBM (e.g., HBM3, HBM3e, HBM4, etc.) offer even higher memory capacity, greater transfer rates, and more stacked DRAM dies. For simplicity, all versions of HBM will be referred to as "HBM" here.

[0047] HBM provides high-capacity memory through stacked memory chips; however, computation within the architecture is performed by a host or other external chip. The host may include, for example, a central processing unit (CPU) (such as a microprocessor), an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a field-programmable gate array (FPGA), etc. Therefore, HBM utilizes an asynchronous communication interface with the host. For example, in an HBM architecture, multiple stacked DRAM dies may be configured to share a global physical bus to support data movement between memory elements and computational elements (e.g., memory utilization components) (such as the host).

[0048] For higher HBM stack configurations (e.g., 8-stacked DRAM dies, 12-stacked DRAM dies, etc.), memory bus bandwidth utilization becomes increasingly important. For complex functions (such as AI and HPC), significant computational power and memory bandwidth are required, which may further increase the importance of bus bandwidth utilization in HBM architectures for these applications. As the demand for memory bandwidth increases, HBM architectures may not be able to fully utilize bus bandwidth due to factors such as the increasing power of GPUs. Furthermore, the asynchronous nature of communication on the global physical bus can improve performance but also make it difficult to handle complex logical operations. Additionally, because HBM architectures require data movement between stacked DRAM dies and the host (e.g., for computation), high-bandwidth data transfer over relatively long distances can increase power consumption in HBM systems.

[0049] To address the above and other limitations associated with certain memory architectures, the embodiments disclosed herein provide memory-stacked computing (CMS) semiconductor hardware (e.g., dies, packages, etc.) with different architectures that integrate memory and computing on a single die, enabling short-distance, high-bandwidth communication between components and minimizing data traffic. Therefore, the CMS hardware disclosed herein can provide memory devices with increasingly higher bandwidth, increased capacity, and high energy efficiency, optimized for complex processing functions such as AI applications.

[0050] The disclosed CMS hardware architecture can implement a die comprising a memory layer configured to support high-bandwidth memory and a compute layer configured to implement logic and / or computational functions, the compute layer being disposed adjacent to the memory layer (e.g., below the memory layer). The memory layer and compute layer can be interconnected (e.g., physically and / or communicatively connected) using vias (such as through-silicon vias (TSVs, or through-silicon vias)) that "support computational functions and / or capabilities implemented close to the memory on the die and eliminate (e.g., the use of a global physical bus required in HBM architectures)" throughout the die distribution. In one or more embodiments, the memory layer of the CMS architecture can comprise a stacked "layer" of one or more memory bank modules. In one or more embodiments, the stacked memory bank modules in the memory layer of the HBMC hardware architecture can be implemented as stacked DRAM dies. Therefore, the CMS hardware architecture disclosed herein can implement integrated high-bandwidth memory and active computing devices instead of passive memory devices (e.g., unlike existing HBM architectures which require a separate host for processing). As disclosed herein, the disclosed CMS hardware enables several advantages, such as improved speed and performance for data processing tasks by eliminating latency in data transmission (e.g., latency associated with a separate computing SoC, latency associated with a global physical bus, etc.), thereby enabling real-time analytics and faster decision-making, which improves the overall efficiency of hardware utilized in complex AI applications.

[0051] In one or more embodiments, various configurations can be used to implement the CMS hardware architecture. For example, CMS dies may be configured to include a different number of multiple stacked memory bank modules. Additionally, in one or more embodiments, the CMS hardware architecture may provide a semiconductor package that can include a different number of multiple interconnected CMS dies in a manner that supports a wide range of bandwidth and / or capacity capabilities for memory. Therefore, the embodiments disclosed herein provide a flexible and / or adaptable CMS hardware architecture that enables memory and active computing devices to be scaled and / or optimized as desired to fully support specific AI applications.

[0052] Furthermore, the embodiments disclosed herein may provide processing elements that can serve as “building blocks” for a larger architecture of CMS hardware. For example, the CMS die may include an array of processing elements (e.g., multiple processing elements interconnected on the die via interconnect ports), the size and / or dimensions of which can be flexibly adjusted to scale and / or optimize based on applicable AI capabilities. In one or more embodiments, the processing element may have a microarchitecture including computational logic and at least one memory bank module, the computational logic being arranged below the memory bank module and having a form factor substantially similar to the memory bank module in a manner that “supports processing power close to memory (e.g., the memory bank module) while minimizing data traffic and operating within an appropriate thermal budget.” The microarchitecture for the processing element is also described herein with respect to the framework of CMS hardware.

[0053] Figure 1A This is a perspective view of an example of a CMS device (or HBMC device) 100 as disclosed herein. Figure 1A In the example, the architecture of CMS device 100 implements integrated high-bandwidth memory and computing functions and / or capabilities on the hardware of semiconductor die (or bare die) 105. Figure 1C According to some embodiments of this disclosure Figure 1A The image depicts a side view of the CMS device 100. As used herein, "die" can refer to a semiconductor device containing functional circuitry and / or integrated circuits (ICs) for performing functions that can be created (e.g., monolithically produced) on a wafer of a semiconductor material such as silicon. In the architecture of the CMS device 100, die 105 may be arranged as a "base die" (also referred to as a "first die"), which may be positioned in the lower layer (e.g., bottom) of the architecture and implements the computing layer 110 (and computing components in the computing layer 110). As the base die, die 105 may have one or more additional dies disposed adjacent to it (e.g., a memory storage module 116 implemented as a memory die stacked on the base die 105). In one or more embodiments, the architecture for the CMS device 100 may be described as an N×N (or N×M) array comprising smaller processing elements 150. (Refer to...) Figure 1B A more detailed description of the microarchitecture of processing element 150 is provided. For example... Figure 1A As depicted, the example CMS device 100 can be configured as a 4×4 array of processing elements 150 implemented on a substrate die using a die 105 (although other array sizes may be available). Figure 1AAn example architecture for a CMS device 100 is depicted, which may include (indicated by horizontal dashed lines) two layers and external components 121 and 122. The two layers may be a computing layer 110 and a memory layer 115. The computing layer 110 is implemented on a die 105 (e.g., the die 105 is a substrate die that includes computing components (computing circuitry 151 containing the computing layer 110)). The memory layer 115 is physically disposed adjacent to the computing layer 110 (e.g., disposed on top of the computing layer 110) and is connected to the computing layer using a plurality of vias (shown as TSV 152) distributed throughout the die 105. Therefore, each processing element 150 in the array may have a memory layer 115 (including memory bank modules 116), a computing layer 110 (containing computing components corresponding to specific memory bank modules 116 stacked thereon), and an associated TSV 152 arranged in such a way as to provide memory and computing components to each processing element 150 of the array (connecting the computing components to the corresponding memory bank modules stacked thereon). In other words, the computing layer 110, implemented on a die 105 (base die) adjacent to (below) the memory layer 115, includes computing components configured to perform computing functions on data transferred from the memory layer 115 and specifically from the corresponding memory bank modules 116 stacked on the memory layer 115. For the computing layer 110 and the memory layer 115, each layer may be implemented as a corresponding die (e.g., each die produced from a single wafer), so the CMS device 100 may include multiple adjacent or stacked dies. Figure 1A In the example, the die implementing the memory layer 115 can be divided into an array of separate memory storage modules 116 that can be directly accessed by computing elements arranged adjacent to (e.g., directly below) the computing layer 110, which is implemented by the die 105 at the substrate.

[0054] CMS device 100 includes a plurality of TSVs 152 distributed across multiple regions of die 105 to provide physical and / or communication connections between a plurality of processing elements 150, and to directly interconnect the compute layer 110 and memory layer 115 of each processing element 150 to support compute functionality. The TSVs 152 are vertically positioned between individual elements of the compute layer 110 and memory layer 115, thereby enabling “vertical” communication directly between separate memory bank modules 116 and one or more compute elements(s), each of the processing elements 150 being arranged below the compute layer 110 (e.g., memory bank modules 116 stacked on top of the corresponding compute elements). For example, the CMS device 100 may have an architecture including (e.g., disposed at a substrate in the lower layer of the architecture) a die 105 as a first die and at least a second die disposed on the first die (e.g., one or more additional dies stacked on top of the die 105), the first die implementing a computing layer 110, and the at least a second die implementing a memory layer 115. The first die may include a plurality of TSVs 152 and (e.g., corresponding to two individual processing elements 150 in an array) at least a first computing component and a second computing component, the plurality of TSVs 152 being distributed across the die 105 (e.g., sets of individual TSVs 152 are arranged in different regions / parts of the die 105), and at least the first computing component and the second computing component (in the computing layer 110 of the architecture) are implemented on the first die. The second die may include (e.g., corresponding to two individual processing elements 150 in an array) at least a first memory storage module 116 and a second memory storage module 116. The first memory storage module can be connected to the first computing component using a first set of TSVs 152 associated with it. For example, the first set of TSVs 152 can be associated with the first computing component by being located closer to it (e.g., contacting, connecting to, etc.). The second memory storage module can be connected to the second computing component using a second set of TSVs 152 associated with it. For example, the second set of TSVs 152 can be associated with the second computing component by being located closer to it (e.g., contacting, connecting to, etc.). The second set of TSVs associated with the second computing component may be different from the TSVs 152 corresponding to the first set of TSVs 152. Thus, the TSVs 152 are configured to transfer data between a die 105 at the lower computing layer 110 of the architecture and at least a second die that may be stacked on top of it in the higher memory layer 115 of the architecture (e.g., transferring data stored on the memory storage module 116 to a corresponding computing component arranged below), with each set of TSVs 152 acting for each of the individual processing elements 150 (and their components).

[0055] Additionally, by arranging the processing elements 150 within the array, such that multiple interconnect ports 154 in the die 105 (substrate die) are horizontally positioned between the multiple processing elements 150, physical and / or communication connections between the multiple processing elements 150 can be provided. This enables the CMS device 100 to function as a network of distributed nodes (e.g., each node having memory and computing capabilities) with the die 105 as the substrate die. Although Figure 1A A CMS device 100 is shown as a 4×4 array of processing elements 150 configured to be implemented on a die 105 (substrate die). However, embodiments of this disclosure are not limited thereto, and according to various embodiments, additional (or fewer) processing elements 150 may be included (e.g., as a larger array and / or an additional layer having processing elements 150). Additionally, according to various embodiments, additional memory layers 115 (e.g., 8, 12, 16, 24, 32, etc.) may be included in the architecture of the CMS device 100, the additional memory layers being implemented as additional dies (disposed on top of the die 105 at the substrate), as disclosed herein.

[0056] In one or more embodiments, the configuration of die 105 (e.g., the substrate die and higher-level elements implemented thereon) can be considered as a “core” CMS die design for the various CMS devices and / or hardware architectures disclosed herein. That is, die 105 can be used as a “core” standalone module that can be repeatedly connected to ultimately assemble relatively large and / or complex CMS devices and / or hardware architectures. For example, below... Figures 4 to 6 Examples of CMS devices described in more detail may have an architecture comprising multiple modularly repeating CMS “core” dies 105 in a larger assembly with different configurations. Thus, in one or more embodiments, die 105 may be mounted on a circuit board along with other circuitry, external components, and / or semiconductor devices to form CMS device 100. In some embodiments, CMS device 100 may be implemented as a hardware device, or a combination of hardware and / or software components. In some embodiments, die 105 may be included in a semiconductor microchip and / or semiconductor package, which may include additional dies, additional circuitry, additional external components, additional semiconductor devices, additional external pins, additional pads, additional electrical connections, etc., that may be encapsulated in a protective package. In one or more embodiments, die 105 may include additional circuitry for supporting aspects and / or functions of memory layer 115 and / or computing layer 110. In one or more embodiments, CMS device 100 may be implemented as a high-bandwidth and / or high-efficiency hardware component of a computer processor (such as a CPU, GPU, neural network processor (NPU), and / or accelerator processor). For example, refer to Figure 7 An example of a CMS device 100 implementing an accelerator processor is described in more detail, which can be used for data-intensive tasks including AI applications.

[0057] Memory layer 115 may be configured to include circuitry for multiple memory bank modules 116. As referred to herein, a "memory bank module" may refer to physical circuitry and / or logic units on hardware (such as a die) that allow parallel access to and manipulation of data, enabling high-bandwidth and high-capacity memory capabilities, as disclosed herein. Figure 1A As shown, the memory layer 115 can be organized as an array of memory storage modules 116, so that the memory storage modules 116 can be accessed simultaneously, thereby achieving a faster data transfer rate. Figure 1A The CMS device 100 can be configured as a 4×4 array of memory "banks," with a total of 16 memory bank modules 116 implemented on the die 105. In other words, Figure 1A The CMS device 100 can be configured as a 4×4 array of processing elements 150, wherein each processing element 150 can be configured to include a corresponding memory storage module 116 implemented thereon. In one or more embodiments, the architecture of the CMS device can be flexible and / or scalable by implementing different numbers of stacked memory storage modules 116 in the memory layer 115. For example, Figure 3 Another example of a CMS device 300 is depicted, wherein the memory layer 315 is clearly configured to include a 4-hi memory bank stack, wherein each processing element 350 has a "to be connected with" Figure 1A The HBMC device 100 implemented in the "single memory stack" provides increased memory bandwidth and / or capacity compared to a "vertically stacked" four memory bank modules 316. In one or more embodiments, the memory bank modules 316 may be implemented as DRAM dies. Thus, a 4-hi memory bank stack may be implemented as four DRAM dies vertically stacked on top of each other. Although one or more exemplary embodiments of the present disclosure have been described with reference to the CMS device architecture, those skilled in the art will readily understand that many modifications to the exemplary embodiments are possible without substantially departing from the aspects of the present disclosure. Therefore, all such modifications (e.g., adjustments to the number of vertically stacked DRAM dies that may be implemented in the CMS hardware architecture (e.g., more and / or fewer layers of memory)) are intended to be included within the scope of this disclosure.

[0058] As will be described in more detail here, Figures 4 to 6Examples of different CMS device architectures are described, in which different configurations of the CMS device (e.g., different numbers of memory bank stacks and / or different numbers of dies) enable the hardware to be tuned and / or optimized to (e.g., based on AI applications) provide the desired memory and / or computing power that is deemed necessary and / or appropriate.

[0059] Refer again Figure 1A The memory layer 115 may be disposed adjacent to the compute layer 110 on the die 105 at the substrate (e.g., on top of the compute layer 110), and the memory layer 115 may be physically and / or communicatively connected to the compute layer 110 using vias (shown as TSV 152). As previously described, the architecture of the example HBMC device 100 may be configured as a 4×4 array of a plurality of processing elements 150. Each processing element 150 may have a corresponding vertical connector using TSV 152 (e.g., interconnecting the memory layer 115 and the compute layer 110), so that by utilizing the die 105 as the substrate die to arrange a plurality of connected processing elements 150, a plurality of TSV 152 may be distributed across a plurality of regions of the die 105 (e.g., as opposed to the TSV location in a concentrated region of the die). As disclosed herein, aspects of the CMS device 100 and / or hardware architecture may involve distributing a plurality of TSV 152 across the die configuration. By implementing multiple processing elements 150 (e.g., including corresponding compute and stacked DRAM) on a die (e.g., die 105), the architecture may include multiple TSV regions distributed across various areas of the die region (instead of centrally arranging all TSVs on the die). Thus, the disclosed CMS hardware utilizes different arrangements of TSVs 152 distributed across die 105 (or multiple stacked dies) to implement short-range (or short-distance) high-bandwidth interconnects between the stacked memory bank module 116 and the underlying compute modules (in the compute layer 110) in a manner that reduces power consumption and alleviates the need for a global physical bus and / or global addressability. Additionally, each processing element 150 may have a corresponding horizontal connector using interconnect ports 154 (e.g., interconnecting multiple processing elements 150 on die 105). Therefore, by utilizing distributed TSV 152, interconnect ports 154, and software (e.g., in contrast to a global physical bus), multiple processing elements 150 within a 4×4 array can have physical and / or communication connections between them to support data transfer and / or communication between a distributed network of processing elements 150 on die 105. Thus, the computing layer 110 can consist of computing components within “building blocks” of processing elements 150 used for hardware, which are repeated 16 times on die 105. This can be referred to as an array of building blocks or an array of processing element building blocks. In other words, in Figure 1AIn the example, each of the 16 processing elements 150 in the 4×4 array has a memory bank module 116 on top of the computing component and corresponding to the computing component (within the memory layer 115), which is disposed on the die 105 directly below the memory bank module 116 (within the computing layer 110). Although Figure 1A An array of 16 building blocks is depicted, but this disclosure is not limited thereto, and the array may include more or fewer building blocks (e.g., 6, 8, 9, 12, or 25 building blocks, for example). See also Figure 1B More detailed descriptions are provided regarding the structure and function of the computing components that may be included in processing element 150. In one or more embodiments, computing layer 110 may be implemented as a logic die. Thus, in one or more embodiments, CMS device 100 may be implemented as a stack of DRAM dies on top of each other and a logic die disposed below the DRAM die stack.

[0060] Computation layer 110 may be configured to include computing circuitry to implement computing functions, processing, and / or computing-related capabilities, enabling CMS device 100 to function as a high-bandwidth memory and active computing device. Computation layer 110 may be configured to implement various capabilities that are identical (or substantially similar) to those of memory-utilizing components, including but not limited to: host; CPU; GPU; NPU; ASIC; field-programmable gate array (FPGA); etc. For example, the computing circuitry of computation layer 110 may be configured to receive data from memory layer 115 and perform general-purpose or special-purpose logic functions on the data, which may be specific to machine learning and / or AI applications that may have specific high-bandwidth requirements. In one or more embodiments, computation layer 110 may be configured to perform basic input / output (I / O) operations and / or control operations related to communication and / or data transfer with other components, such as memory bank module 116.

[0061] Furthermore, the compute layer 110 may include multiple interconnect ports 154, which provide physical and / or communication connections between multiple processing elements 150 on die 105. For example, each processing element 150 is configured to include one or more interconnect ports 154, which may be used as “horizontal” connections to adjacent processing elements 150 in an example 4×4 array configuration. By arranging multiple processing elements 150 (each with a corresponding interconnect port 154) in the array, a mesh network of interconnect ports 154 between the processing elements 150 can be distributed in the compute layer 110, thereby enabling communication between the processing elements 150 in the array (e.g., nodes in a mesh network). The network of interconnect ports 154 on die 105 (distributed in the compute layer 110) allows multiple processing elements 150 to communicate with each other on die 105 (and with connected devices), thereby forming a unified “compute network” system of multiple processing elements 150 to jointly implement compute functions, processing, and / or compute-related capabilities. In one or more embodiments, the computing layer 110 may include a plurality of ingress / egress inter-die ports (interconnect ports) 154, which may be configured as horizontal electrical connections between processing elements 150 that can be distributed across separate dies. For example, one or more ingress / egress inter-die ports 154 disposed on die 105 may be connected to (e.g., in...) Figure 5 In a multi-die package, other processing elements 150 are implemented on separate and adjacent dies. An inlet / outlet die port 154 can provide physical and / or communication interconnection for die 105 and other processing elements 150 on dies in a manner that supports inter-die communication that may be involved in a large-scale multi-die distributed processing system.

[0062] like Figure 1A As seen, the hardware architecture of CMS device 100 places the computing component (within computing layer 110) physically adjacent to and close to the associated memory component (within memory layer 115) on die 105 (e.g., placing the computing component below and close to the memory component), which reduces the distance associated with the transfer of data used to perform computational functions. CMS device 100 utilizes different architectures implementing the active computing layer 110 below memory layer 115 to provide processor near-memory capabilities that can improve memory bandwidth and performance for specific hardware (such as machine learning accelerators) while reducing power consumption associated with die 105. Machine learning and / or AI-based algorithms can benefit from the lower latency and improved memory throughput achieved by the architecture of CMS device 100, as these applications may require intensive bandwidth and computational efficiency for training and prediction.

[0063] Figure 1AThe CMS device 100 is shown to include components (e.g., external components) connected to die 105 and implementing aspects and / or functions related to the high-bandwidth memory and / or computing capabilities of the CMS device 100. In an example, the external components may include a controller 121 and an input / output interface 122. The controller 121 may be configured to perform functions related to the control of the CMS device 100 (which may include function fallback) and additional computational (e.g., high-precision mathematics) operations that can be performed by the CMS device. In one or more embodiments, the controller 121 may be configured to utilize Reduced Instruction Set Computing (RISC)-V to perform basic operations and control the behavior of the CMS device 100 hardware and / or software (such as executing instructions and / or processing data). In one or more embodiments, the controller 121 may be implemented as a processor, microprocessor, CPU, etc., for die 105. In one or more embodiments, the controller 121 may be programmed to perform data transfers between computing components and memory storage modules, and to perform data transfers between their respective computing components and their respective at least one memory storage module for one or more additional processing elements.

[0064] Input / output (I / O) interface 122 may be configured to implement functions related to data and / or component interfaces on die 105 in a manner that enables efficient communication between die 105 and peripheral devices that can be connected to die 105. In one or more embodiments, I / O interface 122 may be configured to implement encryption / description, ASIC-related functions (e.g., protecting model IP, user privacy, and / or dedicated / application-specific end-user programs), dedicated I / O-related functions, etc. In one or more embodiments, I / O interface 122 may be implemented as a Peripheral Component Interconnect Fast (PCIe) interface, and the circuitry of HBMC device 100 may be implemented on a PCI-E compatible board.

[0065] Figure 1B An example microarchitecture for processing element 150 is depicted. Figure 1B As seen herein, processing element 150 may be configured to include one or more components in a microarchitecture, including but not limited to: memory storage module 116; computing circuitry 151; TSV 152; static memory 153; and one or more interconnect ports 154. As used herein, the terms “computing” and / or “computing circuitry” may refer to computing hardware resources (e.g., circuits, IC modules, logic units, CPUs, GPUs, dedicated hardware, etc.) that can be utilized in computing (such as training, retraining, and / or running AI models and / or algorithms).

[0066] The memory bank module 116 may be implemented in the form of dynamically volatile computer memory that utilizes electricity to maintain data and has relatively high capacity and / or speed, as is considered suitable for providing high-bandwidth memory and data storage capabilities for the CMS device 100. The memory bank module 116 may be implemented as DRAM, static random access memory (SRAM), magnetoresistive random access memory (MRAM) or other volatile or non-volatile memory, one or more memory cells, circuitry, and / or other memory components considered suitable and / or appropriate for arrangement in a bank, array, and / or stacked memory configuration. In one or more embodiments, the memory bank module 116 is implemented as a DRAM die. Therefore, in one or more embodiments, the stack of memory bank modules 116 in memory layer 115 may be implemented as a plurality of vertically stacked DRAM dies.

[0067] As previously described, the microarchitecture of processing element 150 can also be described as having a memory layer 115 and a compute layer 110 disposed adjacent to (e.g., below) the memory layer 115. The compute layer 110 of processing element 150 may include several components including compute circuitry 151, TSV 152, static memory 153, and one or more interconnect ports 154, which can work together to implement a distributed computing system directly beneath a stack of memory bank modules 116 in memory layer 115.

[0068] The computing circuit 151 may be a circuit configured to implement the computing functions and / or capabilities of the CMS device 100. As implied above, the computing circuit 151 may perform computing operations and / or functions via memory storage modules, memory circuitry, and / or memory devices substantially adjacent to the processing element 150. Figure 1B As seen, the computing circuitry 151 (in computing layer 110) performs computing functions and is arranged directly below and connected to the memory storage module 116 (in memory layer 115) in a manner that integrates (e.g., embeds) the computing power as physically close to the memory hardware of the CMS device 100 (e.g., close to the memory storage module 116) and / or within the memory hardware of the CMS device 100 (e.g., on a single die). Figure 1B In the example, the memory storage module 116 has direct access to the computing circuitry 151 arranged below.

[0069] Therefore, due to proximity, high-bandwidth data transfer between the memory bank module 116 and the computing circuitry 151 can be supported over short distances in a manner that minimizes latency, reduces energy consumption, and minimizes the distance associated with data transfer (e.g., data transfer between memory and separate processing units of processing element 150). For example, the computing circuitry 151 may be configured to receive data from the memory bank module 116 and perform one or more computational functions, which may be mathematical operations that may involve AI and / or machine learning tasks. In one or more embodiments, the computing circuitry 151 may perform all and / or part of the computational functions, instructions, and / or tasks associated with machine learning operations, including but not limited to: matrix multiplication; dot product; activation functions used in neural networks, etc. In other words, computational functions may include one or more of matrix multiplication; dot product; activation functions; and mathematical functions associated with artificial intelligence applications. In one or more embodiments, computing circuitry 151 may be configured to include logic for supporting various controller functions of the CMS device 100 (such as data flow management, scheduling operations, and coordination of data transfer between memory bank module 116 and other components of processing element 150) in a manner that can increase throughput and / or reduce latency of the CMS device 100. Therefore, by directly connecting computing circuitry 151 to memory bank module 116 using TSV 152 and utilizing software to transfer data between computing circuitry 151 and memory bank module 116 (e.g., in contrast to a global physical bus in HBM), processing element 150 can provide memory and integrated, active computing components that can significantly improve the performance and / or energy efficiency of the CMS device 100 for data-intensive tasks (e.g., AI applications).

[0070] TSV 152 can be configured as a high-performance interconnect between memory layer 115 (and components in memory layer 115) and compute layer 110 (and components in compute layer 110). As disclosed herein, TSV 152 can pass through die 105 to provide vertical electrical connections (vias) forming high-bandwidth interconnects for 3D stacked CMS dies, devices, and / or packages. In one or more embodiments, TSV 152 can be vertical wiring connecting different layers of microchips and / or stacked dies, thereby allowing direct die-to-die communication. In one or more embodiments, multiple TSVs 152 can be implemented as data TSVs and / or power TSVs, with data TSVs configured to transmit data signals between different layers and / or dies, and power TSVs configured to transmit power signals and ground connections, thereby ensuring efficient power delivery to different portions of the die or stacked dies. For example, TSV 152 can provide communication connections between multiple stacked DRAM dies in memory layer 115 and compute dies in compute layer 110. Therefore, TSV 152 can enable short-range and high-bandwidth data transfer between the memory bank module 116 and the computing circuitry 151 of the CMS device 100 in a manner that reduces latency and / or power consumption (e.g., by eliminating the global data bus, eliminating long-distance and / or high-power data movement from the memory stack to the SoC for computing in conventional HBM).

[0071] Processing element 150 may be configured to include static memory 153. Static memory 153 may be implemented as, for example, SRAM, which retains data as long as power is supplied and has relatively fast access times and low latency (e.g., relative to memory bank module 116), as is considered suitable for providing cache and / or high-speed register capabilities for CMS device 100. Memory 153 may be implemented as SRAM, one or more memory cells, circuitry, and / or other memory components considered suitable and / or appropriate for operation with computing circuitry 151. In one or more embodiments, memory 153 is implemented as a single-port (e.g., a single data port for reading and writing at a time) SRAM.

[0072] Processing element 150 may be configured to include one or more interconnect ports 154. In one or more embodiments, the interconnect ports 154 may be implemented as horizontal electrical connections for inlet / outlet connections to one or more other processing elements 150. For example, processing element 150 may utilize interconnect ports 154 as physical connections to one or more processing elements positioned to... Figure 1AIn a 4×4 array configuration, processing elements 150 are adjacent on die 105. Furthermore, arranging multiple processing elements 150 together on die 105 may include distributing multiple interconnect ports 154 across die 105 (e.g., forming a “mesh” network for communication between processing elements 150). For example, each of the processing elements 150 arranged in a 4×4 array on die 105 may have a corresponding interconnect port 154 that horizontally connects each processing element 150 to one or more adjacent processing elements 150 on die 105. The interconnect ports 154 are configured to support inter-processor connectivity and / or communication between the multiple processing elements 150. Thus, multiple interconnected processing elements 150 can collectively serve as a distributed network of compute nodes connected via interconnect ports 154 on die 105. In one or more embodiments, the microarchitecture of the processing elements 150 may have variations in the number and / or configuration of one or more interconnect ports 154 that are considered suitable for a particular architecture of the CMS core die, device, and / or package.

[0073] As implied above, AI workloads can involve large datasets and may require thousands of computations. Therefore, processing element 150 can be utilized as a core unit that provides memory and computational processing power "in a manner suitable for or optimized for scalability and (e.g., parallelism of performing multiple operations simultaneously)," ultimately making CMS device 100 (consisting of an array of processing elements 150) efficient hardware for AI applications.

[0074] Figure 2A This is a two-dimensional (2D) top view of the example CMS die (example CMS device) 200. The architecture, components, and functions of the CMS die 200 are basically similar to those described above. Figure 1A and Figure 1B The previously described CMS "core" die 105. Therefore, for the sake of brevity, some details of the CMS die 200 may be disregarded. Figure 2A repeat. Figure 2A The CMS die 200 is shown to have an architecture including multiple stacked memory bank modules 216, with multiple TSVs 252 distributed among the multiple stacked memory bank modules 216. The TSVs 252 can be distributed within the CMS die 200 to achieve short-range, high-bandwidth interconnects between the stacked memory bank modules 216 and the underlying compute modules in a manner that "reduces power consumption and mitigates the need for a global physical bus and / or global addressability." In one or more embodiments, in addition to the physical bus structure (e.g., see...), Figure 2B In addition to or replacing the physical bus structure (e.g., see...) Figure 2BCMS devices can utilize software to manage data movement (e.g., message passing) between processing elements (e.g., compute, memory, etc.) in their distributed computing systems. For example, communication between multiple processing elements on a die can be controlled and / or guided by software and implemented through physical connections between processing elements established via interconnect ports.

[0075] Figure 2B Another example architecture of a CMS device 260 according to some embodiments is depicted. In one or more embodiments, components of the CMS device 260 may be implemented on a single die. The CMS device 260 may be configured to include a first plurality of stacked memory bank modules 261a corresponding to an underlying logic module 262a and a second plurality of stacked memory bank modules 261b corresponding to an underlying logic module 262b, the first plurality of stacked memory bank modules 261a and the second plurality of stacked memory bank modules 261b being arranged on opposing regions 260a, 260b of the die. In one or more embodiments, the plurality of stacked memory bank modules 261a, 261b may be implemented as a plurality of stacked DRAM dies, and the logic modules 262a, 262b may be implemented as logic dies disposed below the corresponding memory bank modules 261a, 261b.

[0076] like Figure 2BAs seen herein, a first plurality of stacked memory modules 261a and logic modules 262a are arranged in a region 260a on one side of the die (e.g., near the first periphery and / or edge of the die), a second plurality of stacked memory modules 261b and logic modules 262b are arranged in a region 260b on the opposite side of the die (e.g., near the second periphery of the die), and a plurality of TSVs 263 are located in a concentrated position on the die between the first plurality of stacked memory modules 261a and logic modules 262a and the second plurality of stacked memory modules 261b and logic modules 262b. For example, as disclosed herein, the CMS device 260 may include an "on-die" host that can be configured to perform computing functions. The host may be located substantially close to the center of the die, for example, physically close to the region of the die including the plurality of TSVs 263 between the two sides of the stacked memory modules 261a, 261b. Therefore, CMS device 260 may include multiple physical buses 264 to transfer data from multiple stacked memory bank modules 261a, 261b and logic modules 262a, 262b to a host (e.g., for computational processing) using a centrally located TSV 263. Thus, data that may be stored in one of the first plurality of stacked memory bank modules 261a and / or logic modules 262a may have to travel a distance from region 260a to be received by the TSV 263 for transfer to a separate host for computational processing. As implied above, CMS device 200 (e.g., see...) Figure 2A TSVs are configured to reduce the use of the global physical bus and are distributed across the die and utilize communication software (e.g., messaging) for data.

[0077] Figure 3 Another example is depicted of a CMS device 300 that can be configured to implement the high-bandwidth memory and computing power disclosed herein. For general description, the CMS device 300 can be described as CMS device 100 (e.g., see...). Figure 1A A high-capacity variant of the CMS device 300 is also implemented on a single die 305. The architecture, components, and functions of the CMS device 300 are substantially similar to those described above. Figure 1A and Figure 1B The CMS device 100 was described previously. Therefore, for the sake of brevity, some details of the CMS device 300 may be omitted. Figure 3 Repeat. However, Figure 3 The architecture of the CMS device 300 shown may include a 4-hi (i.e., four-layer) stack configuration comprising multiple vertically stacked memory storage modules 316, instead of the CMS device 100 (see, for example, see...). Figure 1A The memory storage module 116 used in the "single stack" is a type of memory storage unit. Although Figure 3 The architecture of a CMS device 300 with four vertically stacked memory bank modules is shown, but embodiments of the present disclosure are not limited thereto, and according to some embodiments, the CMS device 300 may include less than, equal to, or greater than Figure 3 The diagram shows a number of multi-layered, vertically stacked memory modules.

[0078] like Figure 3 As shown, the CMS device 300 can be configured as a 4×4 array of processing elements 350. Each processing element 350 may include four memory storage modules 316 stacked on top of each other in a memory layer 315, and a computing layer 310 (and components in the computing layer 310) is disposed below the 4-hi stack of memory storage modules 316. In other words, the example CMS device 300 may include an N×M (e.g., 4×4) array of processing elements 350, wherein each processing element 350 includes a 4-hi stack of memory storage modules 316, and a corresponding computing layer 310 is disposed below it. Thus, as with reference to the... Figure 1A Compared to the described CMS device 100, the CMS device 300 may have an architecture including increased memory hardware, thereby scaling the memory bandwidth and capacity of the CMS device 300 for, for example, improved performance and / or more complex AI applications. In one or more embodiments, the 4-hi stack of the memory bank module 316 may be implemented as four stacked DRAM dies. Therefore, in one or more embodiments, the CMS device 300 may include a 4-hi stack of multiple stacked DRAM dies further stacked on top of the logic dies.

[0079] Figure 4 Another example is depicted of a CMS device 400 that can be configured to implement the high-bandwidth memory and computing power disclosed herein. For general description, the CMS device 400 can be described as CMS device 100 (e.g., see...). Figure 1A ) and CMS device 300 (e.g., see Figure 3 An increased capacity variant of the CMS device 400. The architecture, components, and functions of the CMS device 400 are basically similar to those described above. Figure 1A and Figure 1B The CMS device 100 was previously described. Therefore, for the sake of brevity, the details of the CMS device 400 will not be referred to again. Figure 4 Discussion. However, Figure 4 The architecture of the CMS device 400 shown may include an 8-hi stack configuration comprising multiple (e.g., eight) vertically stacked memory storage modules 416, rather than a “single stack” of memory storage modules 116 utilized in the CMS device 100 (e.g., see [link]). Figure 1AIn a 4×4 array of processing elements 450 in the CMS device 400, each processing element 450 may include eight memory bank modules 416 stacked on top of each other in a memory layer 415, and a corresponding compute layer 410 (and components in the compute layer 410) is disposed below the 8-hi stack of memory bank modules 416. In other words, the CMS device 400 may include a 4×4 array of processing elements 450, wherein each processing element 450 includes an 8-hi stack of memory bank modules 416, and a corresponding compute layer 410 is disposed below it. In one or more embodiments, the 8-hi stack of memory bank modules 416 may be implemented as eight stacked DRAM dies. Therefore, in one or more embodiments, the CMS device 400 may include an 8-hi stack of multiple stacked DRAM dies further stacked on top of logic dies.

[0080] Figure 5 Another example is depicted of a CMS device 500 that can be configured to implement the high-bandwidth memory and computing power disclosed herein. The architecture of the CMS device 500 may include modular assemblies of CMS devices (e.g., 100, 300, 400). For example, the CMS device 500 may be implemented as a multi-die semiconductor package that includes multiple modularly connected CMS devices 300 (e.g., see [link to relevant documentation]) in a manner that provides more advanced distributed computing. Figure 3 A 4×4 array of CMS devices 300. Each processing element on the corresponding CMS device 300 may include an inlet / outlet inter-die port configured to support horizontal electrical connections to other processing elements that may be distributed on individual dies. Thus, by arranging multiple CMS devices 300, each including multiple inlet / outlet inter-die ports, inter-die communication is supported across a multi-die semiconductor package comprising a 4×4 array of CMS devices 300 with multiple modular connections.

[0081] For example, based on the size / scale, architecture, and / or desired application of device 500, the architecture of CMS device 500 can be configured to have each layer consisting of a single die, such as Figure 5As shown in the figure. However, in one or more embodiments, the CMS device 500 may be a package comprising layers (e.g., units (e.g., 100) of individuals packaged together on an interposer and / or PCB) that are larger than a single die, and may also have additional management / communication components within the package. Thus, the CMS device 500 may include multiple sub-packages connected within a single package, such as an interposer that includes a bridge for signals and power between the sub-packages and the package substrate. Each sub-package in the CMS device 500 may be implemented as a modular (e.g., independently designed and manufactured) functional circuit block and then assembled together on the interposer, or may be a package as a sub-unit further packaged on the interposer, and each package in the CMS device 500 may offer several advantages compared to a conventional monolithic system-on-a-chip (SoC).

[0082] The architecture, components, and functions of the individual components of the CMS device 500 are as described above. Figure 1A The previously described CMS device 100 and / or reference Figure 3 The CMS device 300 described earlier is the same as (or substantially similar to) the CMS device 500. Therefore, for the sake of brevity, some details of the CMS device 500 may not be referred to here again. Figure 5 Discussion. However, Figure 5 The architecture of the CMS device 500 can be a larger assembly of multiple smaller CMS "core" dies (e.g., HBMC device 300), which are modularly repeated and connected in a manner that scales the CMS device 500 to even higher performance, higher capacity variants (e.g., via inlet / outlet die ports). In some embodiments, the CMS device 500 can be implemented as a semiconductor microchip and / or semiconductor package, which may include additional dies, additional circuitry, additional external components, additional semiconductor devices, additional external pins, additional pads, additional electrical connections, etc., that can be encapsulated in a protective package.

[0083] Therefore, as disclosed herein, CMS dies, devices, and packages can be utilized to provide high-bandwidth memory and computing power "in a manner that can be modularly designed and scaled to optimize based on application complexity and processing requirements." For example, CMS device 100 (e.g., see...) Figure 1A It can be used as a low-cost variant of an AI-based processor for devices, and the CMS device 500 can be used as a high-performance, high-capacity "super chip" variant for large-scale AI-based applications such as image and voice classification.

[0084] Figure 6 This describes some embodiments of the present disclosure for use in Figure 1AA flowchart illustrating an example operation of a method for transmitting data within the architecture of a CMS device 100. For example, Figure 6 Various operations in method 600 are illustrated, which can be implemented by software to support, manage and / or control, for example, via multiple TSV 152 (e.g., see...). Figure 1A The software facilitates communication (including data transfer) between various components within the architecture of the CMS device 100, which is initiated by [the system / entity]. In one or more embodiments, the software may include logic and / or instructions implemented by one or more components of the CMS device 100, such as controller 121, memory storage module 116, computing circuitry 151, static memory 153, etc. In one or more embodiments, the software may include logic and / or instructions implemented external to the CMS device 100, such as a host, CPU, controller, etc. Although [the software / entity]... Figure 6 Various operations in a method according to some embodiments are illustrated, but the embodiments of this disclosure are not limited thereto, and the method may include additional or fewer operations according to various embodiments without departing from the spirit and scope of the embodiments of this disclosure.

[0085] Additionally, to support communication between various components of the HBMC device 100 that may be involved in performing in-memory computing functions, software (e.g., replacing the functionality of the physical bus) can be used. In one or more embodiments, the CMS device 100 may be configured to utilize software to perform several operations, including but not limited to: controlling data movement and / or transmission; implementing message passing; performing data preparation and layout; performing task scheduling and synchronization; performing memory controller functions; initiating computation; and managing memory layout.

[0086] Reference Figure 6Method 600 may include one or more of the following operations: Allocable memory storage modules (operation 605). In an operational example, CMS device 100 may be employed by a computer device to perform computational functions that may be relevant to an AI application (such as multiplying two vectors). The data that may be involved in the computational function may not initially be located within the memory layer 115 of CMS device 100. For example, data representing vectors A and B to be multiplied may be stored in components and / or devices (such as CPU, main memory, GPU, etc.) outside CMS device 100. Therefore, messages and / or instructions may be passed to multiple memory storage modules 116 within the memory layer 115 of CMS device 100 to allocate at least one memory storage module 116 for the computational function, such as allocating memory storage module 116 to store data during the execution of the computational function. In one or more embodiments, messages and / or instructions may be dispatched via TSV to a plurality of memory storage modules 116 in an N×N array of processing element 150 to allocate (e.g., corresponding to processing element 150) selected memory storage modules 116. In one or more embodiments, allocating memory storage modules 116 may involve reserving, allocating, and / or accessing specific portions of memory within memory storage modules 116 for a specific purpose or task, thereby allowing for organized and efficient use of available memory resources. In some embodiments, allocating memory storage modules 116 may be dynamic, wherein memory storage modules 116 are allocated and / or reconfigured based on changing workload requirements. Alternatively, in one or more embodiments, allocation may be static, wherein memory storage modules 116 are predefined and / or pre-allocated for specific tasks.

[0087] Data can be transferred to an allocated memory storage module 116 (operation 610). Messages, instructions, and / or data can be passed to memory storage modules 116 within the memory layer 115 of the CMS device 100 to control the movement of data to the allocated memory storage modules 116 for computational functions. For example, data representing vectors A and B can be transferred (e.g., from a CPU outside the HBMC device 100) to the allocated memory storage module 116 to store data during the execution of vector multiplication. In one or more embodiments, messages and / or instructions can be dispatched via TSV 152 to memory storage modules 116 in an N×N array of processing element 150 to transfer data to and / or store it in the allocated memory storage modules 116 (e.g., corresponding to processing element 150). Therefore, the allocation of memory storage module 116 and / or efficient data transfer for computational functions (e.g., vectors A and B for vector multiplication) to the allocated memory storage module 116 can be synchronized and / or controlled via software (e.g., in the absence of a bus).

[0088] A computation function can be triggered (operation 615). Messages and / or instructions can be passed to (e.g., of processing element 150) the computation circuitry 151 corresponding to memory storage module 116 to initiate and / or control the circuitry to perform the computation function. For example, computation circuitry 151 can be configured to perform computations related to performing vector multiplication of vectors A and B, wherein data representing the vectors is stored in memory storage module 116 during the operation. In one or more embodiments, messages, instructions, and / or data can be passed between computation circuitry 151 and the corresponding memory storage module 116 via TSV 152. Memory storage module 116 and / or computation circuitry 151 can be configured to ensure that processing instructions are sent to the corresponding computation circuitry 151, which (e.g., on the same processing element) is arranged below the specific memory storage module 116 storing data related to those instructions. In one or more embodiments, an on-die controller (or other component) can implement management of data transfer between processing elements arranged on the die and / or (on each processing element) layers. Therefore, the controller and / or related software can be configured to control the communication of "processing instructions between memory storage modules 116 used in operation to the appropriate corresponding computing circuits 151".

[0089] Computational circuitry 151 performs computational functions and is disposed directly beneath memory bank module 116. Furthermore, the computational circuitry 151, connected to memory bank module 116 via TSV 152, provides physical and communication interconnects "in a manner that integrates computational capabilities physically close to the memory hardware of CMS device 100 (e.g., close to memory bank module 116) and / or within the memory hardware of CMS device 100 (e.g., on a single die)" to improve performance and energy efficiency (e.g., eliminating physical buses, reducing data movement, etc.). Therefore, the execution of computational functions (e.g., vector multiplication) can be triggered, executed, and / or controlled by computational circuitry 151 via software (e.g., in the absence of a bus).

[0090] The result of the computational function can be transferred to the (allocated) memory storage module (operation 620). After the computational circuit 151 performs the operation related to the computational function, messages, data, and / or instructions can be passed to the memory storage module 116. For example, the computational circuit 151 can perform a computation to generate the result of a vector multiplication of vector A and vector B. In one or more embodiments, messages, instructions, and / or data can be passed between the computational circuit 151 and the corresponding memory storage module 116 via TSV 152 to transfer the result of the computational function to and / or store it in the memory storage module. Thus, the execution of the computational function (e.g., vector multiplication) can be performed by the memory storage module 116 via software (e.g., in the absence of a bus) "in a manner that reduces data movement, lowers energy consumption, and improves the efficiency of tasks that may involve large amounts of memory access (such as AI applications)" and the result can be obtained.

[0091] Figure 7 This is based on some embodiments of the present disclosure, for example, utilizing a CMS device (e.g., see...). Figure 1A This is a block diagram of an electronic device that implements a parallel accelerator for AI applications. For example, electronic device 701 may be configured to implement generated AI applications (such as AI applications that create new content (e.g., text, images, music, videos, etc.) based on patterns learned from large datasets). Therefore, processor 720 may include an accelerator processor, such as an auxiliary processor 723 configured to optimize for AI-related computational and other processing tasks. The CMS device may be physically integrated into the auxiliary processor 723 in a manner that reduces (or minimizes) latency and / or improves (or maximizes) bandwidth (e.g., included in a processor package). In some embodiments, the CMS device may be attached to and / or integrated with other components of electronic device 700 (other than and / or replacing the auxiliary processor 723) (including, but not limited to, the main processor 721, memory 730, etc.). In some embodiments, electronic device 700 may be implemented as a distributed computing system.

[0092] As an operational example, electronic device 701 can perform complex training processes for large-scale models associated with the generated AI application. For example, electronic device 701 can utilize auxiliary processor 723 as an accelerator processor to process large matrices of input data, weights, and / or activations associated with training (one or more) models. Auxiliary processor 723, including a CMS device, can provide data transfer "related to rapidly accessing and / or updating data (e.g., parameters, training data, etc.) and / or performing computational tasks (e.g., matrix multiplication, reading / writing computation results, etc.) associated with training a model with reduced latency" in a manner that improves the overall performance and / or efficiency of electronic device 701 in implementing the AI ​​application.

[0093] Reference Figure 7 Electronic device 701 in electronic device 700 can communicate with electronic device 702 via a first network 798 (e.g., a short-range wireless communication network), or with electronic device 704 or server 708 via a second network 799 (e.g., a long-range wireless communication network). Electronic device 701 can communicate with electronic device 704 via server 708. Electronic device 701 may include processor 720, memory 730, input device 750, sound output device 755, display device 760, audio module 770, sensor module 776, interface 777, haptic module 779, camera module 780, power management module 788, battery 789, communication module 790, SIM card 796, and / or antenna module 797. In one embodiment, at least one component (e.g., display device 760 or camera module 780) may be omitted from electronic device 701, or one or more other components may be added to electronic device 701. A portion of the components may be implemented as a single integrated circuit (IC). For example, a sensor module 776 (e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) may be embedded in a display device 760 (e.g., a display).

[0094] The processor 720 can execute software (e.g., program 740) to control the integration of the electronic device 701 into at least one other component (e.g., hardware or software component) of the processor 720, and can perform various data processing or calculations.

[0095] As at least part of data processing or computation, processor 720 can load commands or data received from another component (e.g., sensor module 776 or communication module 790) into volatile memory 732, process the commands or data stored in volatile memory 732, and store the resulting data in non-volatile memory 734. Processor 720 may include a main processor 721 (e.g., a central processing unit or application processor (AP)) and an auxiliary processor 723 (e.g., a graphics processing unit (GPU), image signal processor (ISP), sensor hub processor, or communication processor (CP)) that can operate independently of or in conjunction with the main processor 721. Additionally or optionally, auxiliary processor 723 may be adapted to consume less power than the main processor 721 or to perform specific operations. Auxiliary processor 723 may be implemented separately from or as part of the main processor 721.

[0096] The auxiliary processor 723 (instead of the main processor 721) can control at least a portion of the functions or states associated with at least one component of the electronic device 701 (e.g., display device 760, sensor module 776, or communication module 790) when the main processor 721 is inactive (e.g., in sleep mode), or control, together with the main processor 721, at least a portion of the functions or states associated with at least one component of the electronic device 701 (e.g., display device 760, sensor module 776, or communication module 790) when the main processor 721 is active (e.g., executing an application). The auxiliary processor 723 (e.g., an image signal processor or a communication processor) can be implemented as part of another component (e.g., camera module 780 or communication module 790) functionally associated with the auxiliary processor 723.

[0097] The memory 730 may store various data used by at least one component of the electronic device 701 (e.g., processor 720 or sensor module 776). The various data may include, for example, software (e.g., program 740) and input or output data for software-related commands. The memory 730 may include volatile memory 732 or non-volatile memory 734. In one or more embodiments, the non-volatile memory 734 may include internal memory 736 and external memory 738.

[0098] The program 740 may be stored as software in the memory 730 and may include, for example, an operating system (OS) 742, middleware 744 and / or application program 746.

[0099] Input device 750 can receive commands or data from outside electronic device 701 (e.g., a user) that will be used by another component of electronic device 801 (e.g., processor 720). Input device 750 may include, for example, a microphone, mouse, or keyboard.

[0100] The sound output device 755 can output sound signals to the outside of the electronic device 701. The sound output device 755 may include, for example, a speaker or a receiver. The speaker can be used for general purposes (such as playing multimedia or recording), and the receiver can be used to receive incoming calls. The receiver may be implemented separately from the speaker or as part of the speaker.

[0101] Display device 760 can visually provide information to the outside of electronic device 701 (e.g., to a user). Display device 760 may include, for example, a display, a holographic device, or a projector, and may include control circuitry for controlling a corresponding one of the display, holographic device, and projector. Display device 760 may include touch circuitry adapted to detect touch, or may include sensor circuitry (e.g., a pressure sensor) adapted to measure the intensity of the force caused by touch.

[0102] The audio module 770 can convert sound into electrical signals and vice versa. The audio module 770 can obtain sound via the input device 750, or can output sound via the sound output device 755 or headphones of the external electronic device 702 directly (e.g., wired) or wirelessly connected to the electronic device 701.

[0103] Sensor module 776 can detect the operating state of electronic device 701 (e.g., power or temperature) or the environmental state outside electronic device 701 (e.g., user state). Sensor module 776 can then generate an electrical signal or data value corresponding to the detected state. Sensor module 776 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biosensor, temperature sensor, humidity sensor, and / or illuminance sensor.

[0104] Interface 777 may support one or more specified protocols for electronic device 701 to be directly (e.g., wired) or wirelessly coupled to external electronic device 702. Interface 777 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0105] Connection terminal 778 may include a connector through which electronic device 701 can be physically connected to external electronic device 702. Connection terminal 778 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0106] The haptic module 779 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through tactile or kinesthetic sensation. The haptic module 779 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0107] Camera module 780 can capture still or moving images. Camera module 780 may include one or more lenses, an image sensor, an image signal processor, or a flash. Power management module 788 can manage the power supplied to electronic device 701. Power management module 788 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0108] The battery 789 can supply power to at least one component of the electronic device 701. The battery 789 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0109] Communication module 790 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 701 and external electronic devices (e.g., electronic device 702, electronic device 704, or server 708), and can support communication via the established communication channel. Communication module 790 may include one or more communication processors that can operate independently of processor 720 (e.g., AP), and can support direct (e.g., wired) or wireless communication. Communication module 790 may include wireless communication module 792 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) and / or wired communication module 794 (e.g., local area network (LAN) communication module or power line communication (PLC) module). A corresponding communication module among these communication modules can communicate via a first network 798 (e.g., a short-range communication network such as Bluetooth). TMThe communication module 792 communicates with external electronic devices via a second network 799 (e.g., a remote communication network such as a cellular network, the Internet, or a computer network such as a LAN or a WAN). These various types of communication modules can be implemented as a single component (e.g., a single IC) or as multiple components that are separate from each other (e.g., multiple ICs). The wireless communication module 792 can use user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 796 to identify and verify the electronic device 701 in the communication network (e.g., the first network 798 or the second network 799).

[0110] Antenna module 797 can transmit or receive signals or power to or from the outside of electronic device 701 (e.g., external electronic device). Antenna module 797 may include one or more antennas. Communication module 790 (e.g., wireless communication module 792) can select at least one of one or more antennas suitable for a communication scheme used in a communication network (such as a first network 798 or a second network 799). Signals or power can then be transmitted or received between communication module 790 and external electronic device via at least one selected antenna.

[0111] Commands or data may be sent or received between electronic device 701 and external electronic device 704 via server 708 integrated into a second network 799. Each of electronic devices 702 and 704 may be a device of the same or different type as electronic device 701. All or some of the operations to be performed at electronic device 701 may be performed at one or more of external electronic devices 702, 704, or 708. For example, if electronic device 701 is required to automatically perform a function or service, or in response to a request from a user or another device, electronic device 701 may request one or more external electronic devices to perform at least a portion of the function or service, rather than performing the function or service itself, or electronic device 701 may request one or more external electronic devices to perform at least a portion of the function or service in addition to performing the function or service. One or more external electronic devices receiving the request may perform at least a portion of the requested function or service or additional functions or services related to the request, and transmit the result of the execution to electronic device 701. Electronic device 701 may provide a result as at least part of a response to the request, with or without further processing. For this purpose, cloud computing, distributed computing, or client-server computing technologies can be used, for example.

[0112] The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits, or in computer software, firmware, or hardware that includes the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer storage medium for execution by or control of operations by a data processing device. Optionally or additionally, the program instructions may be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof, or may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. Furthermore, although the computer storage medium is not a propagation signal, it may be a source or destination of computer program instructions encoded in an artificially generated propagation signal. Computer storage media may also be one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices), or may be included within one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). Furthermore, the operations described herein can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0113] While this specification may contain numerous specific implementation details, these details should not be construed as limiting the scope of any claimed subject matter, but rather as descriptions of features specific to particular embodiments. Certain features described in this specification within the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations, and even initially claimed in this way, one or more features from a claimed combination may, in some cases, be removed from that combination, and the claimed combination may be for sub-combinations or variations thereof.

[0114] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in a sequential sequence, or requiring all shown operations to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0115] Therefore, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the appended claims. In some cases, the actions set forth in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0116] As those skilled in the art will recognize, the innovative concepts described herein can be modified and varied across a wide range of applications. Therefore, the scope of the claimed subject matter should not be limited to any particular exemplary teachings discussed above, but rather is instead defined by the appended claims.

Claims

1. An electronic device comprising: The first nude film, including: First computing component, Second computing component, The first set of through-silicon vias (TSVs) is associated with the first computing component and connects the stacked dies of the electronic device. A second set of through-silicon vias (TSVs) is associated with the second computing component and connects the stacked dies of the electronic device; and The second die, stacked on top of the first die, includes: A first memory storage module is connected to a first computing component using a first set of through-silicon vias (TSVs), and The second memory module is connected to the second computing component using a second set of through-silicon vias.

2. The electronic device according to claim 1, further comprising: A third die, stacked on top of the second die, and includes: A third memory module is connected to a first computing component using a first set of through-silicon vias; and The fourth memory module is connected to the second computing component using a second set of through-silicon vias.

3. The electronic device according to claim 2, wherein, A first processing element is formed by a first memory storage module connected to a first computing component using a first set of through-silicon vias (TSVs), and the first processing element is configured to perform computational functions using data from the first memory storage module. A second processing element is formed by a second memory storage module connected to a second computing component using a second set of TSVs (TSVs), and the second processing element is configured to perform computational functions using data from the second memory storage module.

4. The electronic device according to claim 3, further comprising: The first die has multiple interconnect ports that connect the first processing element and the second processing element to one or more additional processing elements, wherein the first computing component and the second computing component include computing circuitry configured to perform computing functions, and the one or more additional processing elements include computing components connected to at least one memory memory module from an array of memory memory modules, the at least one memory memory module from the array of memory memory modules being disposed on the first die and using a group of corresponding through-silicon vias.

5. The electronic device according to claim 4, wherein, For the one or more additional processing elements, their respective computing components are configured to perform computing functions on data from their respective at least one memory storage module stacked thereon.

6. The electronic device according to claim 5, further comprising: The controller is programmed to perform data transfer between a first computing component and a first memory storage module, data transfer between a second computing component and a second memory storage module, and data transfer between a respective computing component and its respective at least one memory storage module for the one or more additional processing elements.

7. The electronic device according to claim 5, wherein, The first group of through-silicon vias and the second group of through-silicon vias are distributed among multiple through-silicon vias in multiple regions of the first die.

8. The electronic device according to claim 7, wherein, The first set of through-silicon vias includes at least one of the plurality of through-silicon vias in a first region of the first die contacting the first computing component, and the second set of through-silicon vias includes at least one of the plurality of through-silicon vias in a second region of the first die contacting the second computing component.

9. The electronic device according to claim 3, wherein, The first processing element additionally includes a third memory storage module and is configured to perform computational functions using data from the third memory storage module, and the second processing element additionally includes a fourth memory storage module and is configured to perform computational functions using data from the fourth memory storage module.

10. The electronic device according to claim 2, wherein, The stacked dies of the electronic device include multiple additional dies stacked on top of a third die.

11. The electronic device according to claim 2, further comprising: Dynamic random access memory (DRAM) die, wherein one or more of the second and third dies include a dynamic random access memory die.

12. The electronic device according to claim 3, wherein, The computational functions include one or more of matrix multiplication, dot product, activation functions, and mathematical functions associated with artificial intelligence applications.

13. An electronic device comprising: The first nude film, including: Multiple through-silicon vias are distributed in multiple regions of the first die. First computing component, Second computing component, A first set of through-silicon vias (TSVs) is associated with a first computing component and connects stacked dies of the electronic device, wherein the first set of TSVs includes one or more of the plurality of TSVs in a first region of the first die, and A second set of through-silicon vias (TSVs), associated with a second computing component and connecting stacked dies of the electronic device, wherein the second set of TSVs includes one or more of the plurality of TSVs in a second region of the first die, different from the first region; and The second die, stacked on top of the first die, includes: A first memory storage module is connected to a first computing component using a first set of through-silicon vias (TSVs), and The second memory module is connected to the second computing component using a second set of through-silicon vias.

14. The electronic device according to claim 13, wherein, A first processing element is formed by a first memory storage module connected to a first computing component using a first set of through-silicon vias (TSVs), and the first processing element is configured to perform computational functions using data from the first memory storage module. A second processing element is formed by a second memory storage module connected to a second computing component using a second set of TSVs (TSVs), and the second processing element is configured to perform computational functions using data from the second memory storage module.

15. The electronic device according to claim 13, wherein, The stacked dies of the electronic device include a plurality of additional dies stacked on a second die.

16. The electronic device of claim 15, further comprising: Dynamic random access memory (DRAM) die, wherein the second die and one or more of the plurality of additional dies include a DRAM die.

17. The electronic device according to claim 16, wherein, The plurality of through-silicon vias are configured to transfer data stored on the dynamic random access memory die to the first die.

18. The electronic device of claim 14, further comprising: Multiple interconnect ports are located on the first die and are configured to connect the first and second processing elements to the multiple processing elements to implement a distributed computing system.

19. The electronic device of claim 13, further comprising: The controller is programmed to perform data transfer between the first computing component and the first memory storage module, and data transfer between the second computing component and the second memory storage module.

20. A distributed computing system, comprising: Accelerator processors, configured to perform computational functions, include: A first die includes: a first computing component; a second computing component; a first set of through-silicon vias (TSVs) associated with the first computing component and connecting the stacked dies of the accelerator processor; and a second set of TSVs associated with the second computing component and connecting the stacked dies of the accelerator processor. A second die, stacked on the first die, and including: a first memory module connected to a first computing component via a first set of through-silicon vias (TSVs); and a second memory module connected to a second computing component via a second set of TSVs; and The memory stores instructions to be executed by the accelerator processor to perform computational functions.

Images