Storage and calculation integrated structure and chip

Abstract

The invention provides a storage and calculation integrated structure and a chip, and belongs to the technical field of integrated circuits, and the storage and calculation integrated structure comprises at least one near storage core grain which is used for providing large-capacity storage; the in-memory computing unit is used for providing computing processing, and the near-memory core grains provide cache space for data and / or instructions processed by the in-memory computing unit; the at least one logic processing core particle is used for loading the data and / or instructions stored in the near memory core particle into the in-memory computing unit, and updating and storing the data and / or instructions processed by the in-memory computing unit into the near memory core particle; and the logic processing core particle is integrated with the near memory core particle and the in-memory computing unit through hybrid bonding and / or TSV (Through Silicon Via). The storage and calculation integrated structure is applied to high-performance calculation, the high-performance calculation requirement and the large-capacity storage requirement can be met at the same time, the high-performance calculation of the system is maximized, and support is provided for the development of the calculation power technology from the hardware structure.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit technology and relates to heterogeneous integration technology that combines near-memory computing and in-memory computing, specifically a memory computing structure and chip. Background Technology

[0002] With the rapid development of computing technologies such as the digital economy and artificial intelligence, higher requirements are being placed on chips. They not only need to meet the needs of high-performance computing, but also the needs of large-capacity storage. However, the existing integrated circuit structure has the problems of "memory wall" and "power wall".

[0003] See Figure 1 In existing integrated circuit architectures, a "memory wall" exists during data / instruction transfer between near-memory computing (memory units) and the CPU (Central Processing Unit). This results in a "power wall" where the energy consumption for reading and writing data / instructions in near-memory computing (memory units) is far greater than the energy consumption for processing data / instructions by the CPU. To address the "memory wall" and "power wall" issues, researchers have developed in-memory computing and near-memory computing. In-memory computing, as a novel computing architecture, aims to directly distribute computational tasks from the CPU to memory computing, thereby reducing data transfer and complex memory management, significantly improving computational efficiency and performance. Near-memory computing integrates storage units and computing units, aiming to reduce data transmission latency between storage units and computing units, improve data processing efficiency, reduce overall energy consumption, and enhance chip performance.

[0004] In-memory computing is a technology that performs computational tasks directly within the storage unit, avoiding frequent data transfers and complex memory management. In-memory computing achieves the highest energy efficiency, but its capacity is limited. Near-memory computing encapsulates the data processing unit and the storage unit together, thereby reducing the data / instruction transfer path between the storage unit and the data processing unit, achieving high I / O density, thus increasing memory bandwidth and reducing access latency. See Table 1 for a performance comparison between in-memory computing and near-memory computing. As shown in Table 1, in-memory computing has extremely high computational performance, with an energy efficiency of 10–500 TOPS / W, but its storage capacity is limited, ranging from 1KB to 10MB; near-memory computing has extremely high storage capacity, ranging from 100MB to 100GB, but its computational performance is limited, less than 20 TOPS / W.

[0005] Table 1: Performance Comparison of In-Memory Computation and Near-Memory Computation

[0006] type Calculate energy efficiency Storage capacity Process constraints Structural constraints In-memory computing 10~500TOPS / W 1KB~10MB Partial logic process compatibility Logical structure is limited Near-memory computing Less than 20 TOPS / W 100MB~100GB Logic process compatibility Logical structure is limited

[0007] Therefore, while existing in-memory computing can meet the demands of high-performance computing, its limited storage capacity cannot satisfy the need for large-capacity storage. Similarly, while existing near-memory computing can meet the needs of large-capacity storage, its limited computing performance cannot meet the demands of high-performance computing. Thus, the inability of current chips to simultaneously meet the requirements of high-performance computing and large-capacity storage has become a bottleneck in the development of computing power technology, hindering its further advancement from a hardware perspective. Summary of the Invention

[0008] In view of the technical problem described in the background section above, existing chips cannot simultaneously meet the requirements of high-performance computing and large-capacity storage, which hinders the further development of computing technology from the perspective of hardware structure. In order to address this technical problem, the present invention proposes an in-memory computing structure and chip.

[0009] This invention utilizes heterogeneous integration technology to integrate near-memory chips and in-memory computing units into a single structure, supporting an in-memory computing architecture that supports near-memory computing. It features a standard memory interface and combines the standard server compatibility of in-memory computing with the high computing efficiency of near-memory computing. This architecture can be applied to high-performance computing, allowing the advantages of both in-memory and near-memory computing structures to complement each other. It combines extremely high computing efficiency with extremely high storage capacity, simultaneously meeting the needs of high-performance computing and large-capacity storage, and maximizing the high-performance computing of the system. This provides support for the development of computing technology from a hardware structure perspective.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0011] An in-memory computing architecture, comprising:

[0012] At least one near-memory chip is used to provide high-capacity storage;

[0013] And at least one in-memory computing unit for providing computing processing, wherein the near-memory chip provides cache space for the data and / or instructions processed by the in-memory computing unit;

[0014] And at least one logic processing chip, used to load data and / or instructions stored in the near memory chip into the in-memory computing unit, and to update and store the data and / or instructions processed by the in-memory computing unit into the near memory chip;

[0015] The logic processing core is integrated with the near-memory core and the in-memory computing unit through hybrid bonding and / or TSV.

[0016] Furthermore, the in-memory computing unit is located within the logic processing core;

[0017] The logic processing core includes a computing / processing unit; the near-memory core includes a memory unit.

[0018] The in-memory computing unit and the computing / processing unit are respectively connected to the memory unit via hybrid bonding.

[0019] Further specified, the in-memory computing unit is disposed within the in-memory computing chip;

[0020] The near-memory chip includes a memory unit, and the logic processing chip includes a computing / processing unit;

[0021] The memory units, in-memory computing units, and computing / processing units are integrated with each other through hybrid bonding and / or TSV.

[0022] Further specifying, the logic processing chip also includes an external interface for providing an external signal transmission channel; the external interface includes a standard memory module interface or a standard memory chip interface, the standard memory module interface includes a PCIE interface or a CXL interface; the standard memory chip interface includes any one of the following: DDR series interface, LPDDR series interface, GDDR series interface or HBM series interface.

[0023] Furthermore, the computing / processing unit also includes an in-memory computing controller;

[0024] The in-memory computing controller is used to switch the loading mode, reading mode, and computing mode of data and / or instructions of the in-memory computing unit.

[0025] Furthermore, the memory unit also includes a memory manager, which is located on the near-memory chip;

[0026] The memory manager is used to manage storage access to memory units, including refresh, activation, prefetching, and protocol conversion.

[0027] A chip includes a primary packaging substrate and the aforementioned in-memory computing structure, wherein the primary packaging substrate is connected to the in-memory computing structure via primary pads.

[0028] A chip includes multiple in-memory computing structures as described above, wherein the multiple in-memory computing structures are arranged in an array.

[0029] Multiple computing / processing units of the aforementioned in-memory computing architecture are interconnected to form an on-chip network or a standard global memory access bus.

[0030] Or multiple memory units of the aforementioned in-memory computing structure are interconnected to form an on-chip network or a standard global memory access bus;

[0031] Multiple in-memory computing architectures share a single external interface, and the on-chip network or standard global memory access bus is connected to the external interface.

[0032] Further, adjacent in-memory computing architectures share a memory unit, or adjacent in-memory computing architectures share an in-memory computing unit, or adjacent in-memory computing architectures share a computing / processing unit.

[0033] Further, it also includes: a command / address encoding module connected to an external interface, wherein the command / address encoding module is used to manage the working status and address selection of each in-memory computing architecture;

[0034] The chip also includes a data switching circuit for bridging the connection between various in-memory computing structures and external interfaces.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention discloses an in-memory computing architecture that integrates at least one near-memory core and at least one in-memory computing unit into a single structure using heterogeneous integration technology. The in-memory computing unit is integrated into a logic processing core. The logic processing core can directly load data and / or instructions stored in the near-memory core into the in-memory computing unit, and update and store data and / or instructions processed by the in-memory computing unit back into the near-memory core. This allows the in-memory computing architecture to simultaneously possess the standard server compatibility of in-memory computing and the high computational efficiency advantages of near-memory computing. Applying this in-memory computing architecture to high-performance computing can simultaneously meet the demands of high-performance computing and large-capacity storage, maximizing the system's high-performance computing capabilities and providing support for the development of computing technology from a hardware architecture perspective.

[0037] This invention discloses an in-memory computing architecture, which further includes a logic processing chip. The logic processing chip can load data and / or instructions stored in the near-memory chip into the in-memory computing unit, and update and store the data and / or instructions processed by the in-memory computing unit into the near-memory chip. Leveraging the advantage of short data link distances in heterogeneous integration, the logic processing chip loads data and / or instructions stored in the near-memory chip into the in-memory computing unit and updates and stores the data and / or instructions processed by the in-memory computing unit into the near-memory chip. Furthermore, considering the regional organization within a large model, the loaded data / instructions have a high reuse rate—that is, loaded once and used multiple times—thus reducing the energy consumption of data / instruction transmission. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the connection between a near-memory computing system and a CPU in the prior art.

[0039] Figure 2This is a schematic diagram of the in-memory computing structure of the present invention. Figure 1 ;

[0040] Figure 3 This is a schematic diagram of the in-memory computing structure of the present invention. Figure 2 ;

[0041] Figure 4 for Figure 2 Integrated circuits formed by in-memory computing architecture;

[0042] Figure 5 A schematic diagram showing the integration of logic processing chips into in-memory computing chips;

[0043] Figure 6 A schematic diagram showing the integration of in-memory computing chips into logic processing chips;

[0044] Figure 7 The in-memory computing structure of this invention is an integrated circuit formed using 3D heterogeneous integration technology;

[0045] Figure 8 This is a schematic diagram of the in-memory computing structure of the present invention, in which both the near-memory core and the in-memory computing core are two layers.

[0046] Figure 9 The in-memory computing structure of this invention is an integrated circuit formed using 2.5D heterogeneous integration technology;

[0047] Figure 10 This invention refers to an integrated circuit formed by the distributed array of multiple in-memory computing structures.

[0048] Figure 11 for Figure 10 A schematic diagram of two adjacent in-memory computing architectures sharing the same memory unit;

[0049] Figure 12 for Figure 10 A schematic diagram of two adjacent in-memory computing architectures sharing the same in-memory computing unit;

[0050] Figure 13 for Figure 10 A schematic diagram of two adjacent in-memory computing architectures sharing the same computing / processing unit;

[0051] Figure 14 This diagram illustrates the connection between external interfaces and multiple in-memory computing architectures.

[0052] Figure 15 This is a schematic diagram showing the connection between an in-memory computing architecture and an external interface.

[0053] Explanation of reference numerals in the attached figures:

[0054] 1-Near-memory chip, 101-Memory unit, 1011-Memory manager, 2-In-memory computing chip, 201-In-memory computing unit, 3-TSV, 4-Hybrid bonding bond, 5-Logic processing chip, 501-External interface, 502-Computing / processing unit, 5021-In-memory computing controller, 6-Primary pad, 7-Primary package substrate, 8-Secondary pad, 9-Secondary package substrate, 10-On-chip network or standard global memory access bus. Detailed Implementation

[0055] To make the technical solution of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of protection, but merely to illustrate selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] See Figure 2 and Figure 3 This invention provides an in-memory computing architecture, including at least one near-memory core 1, at least one in-memory computing unit 201, and at least one logic processing core 5. The near-memory core 1 is used to provide large-capacity storage; the in-memory computing unit 201 is used to provide computing processing, and the near-memory core 1 provides cache space for the data and / or instructions processed by the in-memory computing unit 201; the logic processing core 5 is used to load the data and / or instructions stored in the near-memory core 1 into the in-memory computing unit 201, and to update and store the data and / or instructions processed by the in-memory computing unit 201 into the near-memory core 1; the logic processing core 5 is integrated with the near-memory core 1 and the in-memory computing unit 201 through hybrid bonding and / or TSV, that is, the near-memory core 1, the in-memory computing unit 201, and the logic processing core 5 are integrated through heterogeneous integration technology. This invention integrates the near-memory core 1, the in-memory computing unit 201, and the logic processing core 5 using heterogeneous integration technology (hybrid bonding and / or TSV) to form an in-memory computing structure. This in-memory computing structure supports the in-memory computing architecture of near-memory computing, has a standard memory interface, and combines the standard server compatibility of in-memory computing with the high computing energy efficiency of near-memory computing. This allows the advantages of in-memory computing and near-memory computing to complement each other, resulting in both extremely high computing energy efficiency and extremely high storage capacity, further meeting the hardware requirements of large-scale high-performance computing application scenarios.

[0057] The logic processing core 5 organizes the memory data (training weights) in the near-memory core 1 and updates it in the in-memory computing core 2. Taking advantage of the short data link distance (10um level) and low memory access energy consumption (1pj / bit level) after heterogeneous integration, the computing parameters (data / instructions) in the near-memory core 1 are loaded into the in-memory computing core 2 for calculation. At the same time, combined with the regional organization in the large model, the loaded data / instructions have a high reuse rate, that is, they are loaded once and used many times, which reduces the energy consumption of data / instruction transmission.

[0058] Applying the in-memory computing architecture of this invention to high-performance computing can simultaneously meet the requirements of high-performance computing and large-capacity storage, and maximize the high-performance computing of the system, thus providing support for the development of computing technology from a hardware architecture perspective.

[0059] The near-memory core 1 in this invention includes 100MB to 100GB of distributed large-capacity memory to meet the huge storage capacity requirements of large model parameters. Specifically, the large-capacity memory can be SRAM, DRAM, or Flash, etc. The in-memory computing core 2 in this invention includes 1KB to 10MB of in-memory computing units 201 to improve the computing efficiency of the computing system. The in-memory computing units 201 are disposed on the in-memory computing core 2.

[0060] The in-memory computing core 2, the near-memory core 1, and the logic processing core 5 are stacked sequentially from top to bottom to enhance the storage access capability of data and / or instructions between the near-memory core 1 and the logic processing core 5. This is more suitable for large models in the near-memory core 1 where data / instructions are frequently updated, such as the training computation of AI models. Alternatively, the near-memory core 1, the in-memory computing core 2, and the logic processing core 5 are stacked sequentially from top to bottom to enhance the update and storage access capability of data / instructions between the in-memory computing core 2 and the logic processing core 5, thereby improving the high-energy-efficiency computing performance of the in-memory computing core 2. In this case, the relationship between the logic processing core 5 and the in-memory computing core 2 is closer, which is more suitable for applications where data reuse in the near-memory core 1 is stronger, such as the inference computation of AI models.

[0061] In a preferred embodiment of the present invention, the in-memory computing unit 201 is disposed within the logic processing chip 5; the logic processing chip 5 includes a computing / processing unit 502; the near-memory chip 1 includes a memory unit 101; the in-memory computing unit 201 and the computing / processing unit 502 are respectively connected to the memory unit 101 by hybrid bonding.

[0062] See Figure 2 and Figure 4In another preferred embodiment of the present invention, the near-memory chip 1 includes a memory unit 101, which provides large-capacity storage; an in-memory computing unit 201, which provides high-energy-efficiency computing; and a logic processing chip 5, which includes a computing / processing unit 502, which implements complex calculations and control processes that are difficult for the in-memory computing unit 201 to implement. The memory unit 101, the in-memory computing unit 201, and the computing / processing unit 502 are integrated with each other through hybrid bonding and / or TSV. The memory unit 101 also includes a memory manager 1011, which is located on the near-memory chip 1 or the logic processing chip 5. The memory manager 1011 manages the storage access of the memory unit 101, including refresh, activation, prefetching, and protocol conversion. Specifically, in Figure 4 In this configuration, the memory manager 1011 is connected to the computing / processing unit 502 via TSV 3, the memory unit 101 is connected to the in-memory computing unit 201 via hybrid bonding key 4, and the in-memory computing unit 201 is connected to the computing / processing unit 502 via hybrid bonding key 4. Specifically, TSV 3 connects the front and back sides of two cores (near-memory core 1, in-memory computing core 2, or logic processing core 5) in the intermediate layer stack, and hybrid bonding key 4 interconnects two adjacent cores (near-memory core 1, in-memory computing core 2, or logic processing core 5).

[0063] The in-memory computing unit 201 in this invention includes, but is not limited to, a standard in-memory computing unit 201 implemented in analog, mixed-signal, or digital modes using SRAM, flash, RRAM, MRAM, PCM, and DRAM as media; it can also be a part of a standard in-memory computing unit 201 using SRAM, DRAM, and RRAM as media. In particular, except for the memory cell 101, the digital, mixed-signal, and analog circuits of the in-memory computing unit 201 are designed in the in-memory computing chip 2. At the same time, the storage cell design or multiplexing design of the in-memory computing unit 201 is in the memory cell 101 of the near-memory chip 1, which can further reduce the energy and time overhead of moving data / instructions from the near-memory chip 1 to the in-memory computing unit 201. The in-memory computing unit 201 is used for basic calculations of weights and inputs. High computing energy efficiency is achieved by heterogeneously integrating the in-memory computing unit 201 and the memory cell 101.

[0064] The computing / processing unit 502 of this invention also includes an in-memory computing controller 5021, which is used to provide switching between the data and / or instruction loading mode, the reading mode, and the computing mode of the in-memory computing unit 201. The in-memory computing controller 5021 can be a controller (such as an embedded MCU) and / or a reconfigurable calculator (such as a CGRA) and / or an ASIC and / or an FPGA, etc., used alone, or in combination (such as a computing / processing unit 502 including an MCU and a CGRA), or used in conjunction (such as the computing / processing unit 502 being an MCU and a CGRA respectively).

[0065] See Figure 5 The logic processing chip 5 can be integrated into the in-memory computing chip 2. The logic processing chip 5 directly accesses the memory unit 101 in the near-memory chip 1 through three-dimensional heterogeneous integration technology, so as to achieve faster storage of intermediate calculation results and data updates.

[0066] See Figure 6 The in-memory computing chip 2 can be integrated into the logic processing chip 5. The medium of the in-memory computing chip 2 may be RRAM, MRAM, etc. The technical effects are: the computing / processing unit 502 has a close relationship with the memory unit 101 and the in-memory computing unit 201, which simplifies the stack structure, reduces production costs, and improves production yield.

[0067] In this invention, the logic processing chip 5 also includes an external interface 501, which provides an external signal transmission channel to realize the external communication function of the in-memory computing structure and to realize AI model calculation and data communication between modules. The external interface 501 includes a standard memory module interface or a standard memory chip interface. The standard memory module interface includes a PCIE interface and a CXL interface. The standard memory chip interface includes DDR series interfaces, LPDDR series interfaces, GDDR series interfaces and HBM series interfaces.

[0068] This invention may include multiple near-memory chips 1 and multiple in-memory computing chips 2, which are stacked from top to bottom. The multiple near-memory chips 1 and multiple in-memory computing chips 2 share a single logic processing chip 5. The number of near-memory chips 1 and multiple in-memory computing chips 2 can be 2, 3, 4, or even more. The specific number is set according to the application environment, and this invention does not impose a specific limitation.

[0069] See Figure 8 The present invention includes two near-memory chips 1 and two in-memory computing chips 2, which are stacked sequentially from top to bottom. The two near-memory chips 1 and the two in-memory computing chips 2 share a single logic processing chip 5.

[0070] Additionally, it should be noted that the positions of the near-memory core 1 and the in-memory computing core 2 in this invention can be interchanged. For example, one or more near-memory cores 1 can be placed between two in-memory computing cores 2, or one or more in-memory computing cores 2 can be placed between two near-memory cores 1, or all in-memory computing cores 2 can be located on top of the near-memory core 1, or all near-memory cores 1 can be located on top of the in-memory computing core 2. By adjusting the positions of the near-memory core 1 and the in-memory computing core 2, the latency and bandwidth of the computing / processing unit 502 accessing the in-memory computing unit 201 can be optimized, which is beneficial for applications with large data interaction between in-memory computing structures.

[0071] Increasing the number of one or more types of chips (near-memory chip 1, in-memory computing chip 2, or logic processing chip 5) in an in-memory computing architecture falls within the scope of protection of this invention. This invention can design in-memory computing architectures according to different application tasks. It can expand memory capacity by increasing the number of layers of near-memory chip 1; expand in-memory computing density by increasing the number of layers of in-memory computing chip 2; and improve the capabilities of special operations, data distribution processing, or computation management by increasing the number of layers of logic processing chip 5.

[0072] The present invention also provides a chip, including a primary packaging substrate 7 and the aforementioned in-memory computing structure, see [link to relevant documentation]. Figure 5 , Figure 6 and Figure 7 The chip uses 3D integration technology for packaging, integrating the near-memory chip 1, the in-memory computing chip 2, and the logic processing chip 5 through hybrid bonding technology. A primary pad 6 is set on the back of the logic processing chip 5, and the primary packaging substrate 7 is connected to the in-memory computing structure through the primary pad 6.

[0073] The present invention also provides a chip, comprising a primary packaging substrate 7, secondary pads 8, a secondary packaging substrate 9, and the aforementioned in-memory computing structure, see [link to relevant documentation]. Figure 9 The chip uses 2.5D integration technology for packaging. After the near-memory chip 1 and the in-memory computing chip 2 are integrated using hybrid bonding technology, secondary pads 8 are provided on the back of both the in-memory computing chip 2 and the logic processing chip 5. The secondary pads 8 are connected to the secondary packaging substrate 9. The logic processing chip 5 is also connected to the secondary packaging substrate 9 through the secondary pads 8. Primary pads 6 are provided on the back of the secondary packaging substrate 9 and are connected to the primary packaging substrate 7 through the primary pads 6. The secondary packaging substrate 9 is a silicon substrate.

[0074] See Figure 10The present invention also provides a chip comprising multiple in-memory computing (IMC) structures as described above, arranged in an array. In each IMC structure, a memory cell 101 is disposed in a near-memory chip 1, an in-memory computing unit 201 is disposed in an in-memory computing chip 2, and a computing / processing unit 502 is disposed in a logic processing chip 5. The computing / processing units 502 of the multiple IMC structures are interconnected to form an on-chip network or a standard global memory access bus 10, where the computational data input and exchange capabilities are maximized. Alternatively, the memory cells 101 of the multiple IMC structures can be interconnected to form an on-chip network or a standard global memory access bus 10, which optimizes the sharing of input data for the in-memory computing unit 201. Multiple IMC structures share a single external interface 501. The on-chip network or standard global memory access bus 10 is used not only to interconnect the multiple IMC structures but also to connect to the external interface 501, providing a data link between the external interface 501 and each IMC structure.

[0075] See Figure 11 In the multiple in-memory computing structures distributed in the array in this invention, adjacent in-memory computing structures share a memory unit 101, that is, two or more in-memory computing units 201 and two or more computing / processing units 502 all share a memory unit 101.

[0076] See Figure 12 In the multiple in-memory computing structures distributed in the array of the present invention, adjacent in-memory computing structures share an in-memory computing unit 201, that is, two or more memory units 101 and two or more computing / processing units 502 all share an in-memory computing unit 201.

[0077] See Figure 13 In the multiple in-memory computing structures distributed in the array of the present invention, adjacent in-memory computing structures share a computing / processing unit 502, that is, two or more memory units 101 and two or more in-memory computing units 201 all share a computing / processing unit 502.

[0078] Therefore, this invention can address different computation scheduling and memory requirements by increasing data interaction between two adjacent in-memory computing structures. Specifically, it can allow two or more in-memory computing structures to share a single memory unit 101 to achieve computational data sharing, or allow two or more in-memory computing structures to share a single in-memory computing unit 201 to achieve data storage sharing; or allow two or more in-memory computing structures to share a single computing / processing unit 502 to achieve more complex computation and control processes.

[0079] See Figure 14In the chip of this invention, when the external interface 501 is used as a standard memory module interface, the external interface and the in-memory computing structure are typically interconnected via an on-chip network. This overcomes the global synchronization constraints of the chip, enabling high-performance computing of large models. When the external interface 501 is used as a standard memory chip interface, the external interface 501 and the in-memory computing structure are typically interconnected via a standard memory access bus. Its advantage is that it allows the device to have compatibility with standard memory chips, such as enabling the implementation of DDR series memory modules that support in-memory computing.

[0080] The chip of this invention also includes a command / address encoding module connected to an external interface 501, which is used to manage the operating status and address selection of each in-memory computing architecture. Figure 14 In this architecture, multiple in-memory computing units are connected to the status structure and external interface 501 via a command / address encoding module. These units are also connected to external interface 501 via a data switching circuit. When external interface 501 functions as a standard memory chip interface, the status monitoring interface is the same as the monitoring interface for the standard memory chip interface, such as the SPD interface on a DDR series module. Reusing this interface allows the in-memory computing units to switch between two operating modes (in-memory computing control mode and standard external storage access mode). The command / address encoding module manages the operating status and address selection of each in-memory computing unit; the data switching circuit module bridges the data bus between each in-memory computing unit and the standard memory chip interface.

[0081] The chip of this invention also includes a data switching circuit for bridging the connection between each in-memory computing architecture and the external interface 501. See also Figure 15The external interface 501 of the in-memory computing architecture is linked from the data switching circuit module and the command / address encoding circuit module to the global standard memory access bus of the in-memory computing architecture. The memory manager 1011 is located on the computing / processing unit 502. The near-memory interface is linked to the memory unit 101, and the in-memory computing interface is linked to the in-memory computing unit 201. Both the near-memory interface and the in-memory computing interface are connected to the memory access switching circuit, which is connected to the data switching circuit. The memory access switching circuit is connected to the command / address encoding module through the working state control circuit. The memory access switching circuit is connected to the memory manager 1011 on the computing / processing unit 502, and the computing / processing unit 502 is connected to the command / address encoding module. The in-memory computing controller 5021 has the ability to perform calculations and processing, as well as the ability to access data between the memory unit 101 and the in-memory computing unit 201. It is mainly implemented in the form of MCU or ASIC. The memory unit 101 has the versatility of calculation and processing, while the in-memory computing unit 201 has higher performance dedicated calculation and processing capabilities. The memory manager 1011 has functions such as refreshing, activating, prefetching, and protocol conversion for memory units 101 and in-memory computing units 201.

[0082] The working state control circuit extracts the working state instructions of the in-memory computing structure from the command / address encoding module, and controls the storage access switching circuit to switch the data path of the memory unit 101 and the in-memory computing unit 201 to the in-memory processing state of the computing / processing unit 502, or the standard external memory access state of the global data switching circuit.

[0083] The storage access switching circuit, on the one hand, responds to the input of the working state control circuit to switch the internal memory processing path and the external memory access path of the memory unit 101 and the memory-in-memory computing unit 201; on the other hand, in the internal memory processing state, it can also respond to the memory manager 1011 to realize direct data exchange between the memory-in-memory computing unit 201 and the memory unit 101.

[0084] It should be noted that the data switching circuit, operating state control circuit, and memory access switching circuit mentioned above are all existing circuits. The data switching circuit is composed of a tri-state gate array to achieve bidirectional interconnection between multiple in-memory computing circuits, the data bus, and the external interface 501. The operating state control circuit parses the instructions from the command / address encoding module to generate control signals for the memory access switching circuit, thereby enabling mode switching between in-memory computing control mode and standard external memory access mode. The memory access switching circuit receives control signals output by the operating state control circuit to achieve mode switching between in-memory computing control mode and standard external memory access mode. In the in-memory computing control mode, the tri-state gate array blocks the data and command / address bus bridge output by the memory manager 1011 to the interface of the in-memory computing unit 201, thus connecting it to the memory unit 101, enabling the external interface 501 to read and write to the memory unit 101. In the standard external memory access mode, the data and command / address bus bridge outside the near-memory computing structure is blocked from the near-memory interface, thus connecting to the memory unit, enabling the in-memory computing controller 5021 to read and write to the memory unit.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A storage-computing integrated architecture, characterized in that, include: At least one near-memory chip (1) is used to provide high-capacity storage; And at least one in-memory computing unit (201) for providing computing processing, wherein the near-memory chip (1) provides cache space for the data and / or instructions processed by the in-memory computing unit (201); And at least one logic processing chip (5) for loading data and / or instructions stored in the near memory chip (1) into the in-memory computing unit (201), and updating and storing the data and / or instructions processed by the in-memory computing unit (201) into the near memory chip (1); The logic processing chip (5) is integrated with the near-memory chip (1) and the in-memory computing unit (201) through hybrid bonding and / or TSV (3).

2. The in-memory computing architecture according to claim 1, characterized in that, The in-memory computing unit (201) is disposed within the logic processing core (5); The logic processing core (5) includes a computing / processing unit (502); the near-memory core (1) includes a memory unit (101); The in-memory computing unit (201) and the computing / processing unit (502) are respectively connected to the memory unit (101) via hybrid bonding.

3. The in-memory computing architecture according to claim 1, characterized in that, The in-memory computing unit (201) is disposed within the in-memory computing chip (2); The near-memory chip (1) includes a memory unit (101), and the logic processing chip (5) includes a computing / processing unit (502); The memory unit (101), the in-memory computing unit (201), and the computing / processing unit (502) are integrated with each other through hybrid bonding and / or TSV (3).

4. A storage-computing integrated structure according to claim 2 or 3, characterized in that, The logic processing chip (5) also includes an external interface (501) for providing an external signal transmission channel; the external interface (501) includes a standard memory module interface or a standard memory chip interface, the standard memory module interface includes a PCIE interface or a CXL interface; the standard memory chip interface includes any one of the following: DDR series interface, LPDDR series interface, GDDR series interface or HBM series interface.

5. The in-memory computing architecture according to claim 4, characterized in that, The computing / processing unit (502) also includes an in-memory computing controller (5021); The in-memory computing controller (5021) is used to switch the loading mode, reading mode and computing mode of data and / or instructions of the in-memory computing unit (201).

6. The in-memory computing architecture according to claim 5, characterized in that, The memory unit (101) further includes a memory manager (1011) located on the near-memory chip (1); The memory manager (1011) is used to manage the storage access of the memory unit (101), and the storage access includes refresh, activation, prefetching and protocol conversion.

7. A chip, characterized in that, It includes a primary packaging substrate (7) and the in-memory computing structure according to any one of claims 1-6, wherein the primary packaging substrate (7) is connected to the in-memory computing structure via a primary pad (6).

8. A chip, characterized in that, It includes multiple in-memory computing structures as described in any one of claims 4-6, wherein the multiple in-memory computing structures are distributed in an array; Multiple computing / processing units (502) of the aforementioned in-memory computing architecture are interconnected to form an on-chip network or a standard global memory access bus (10). Or multiple memory units (101) of the in-memory computing structure are interconnected to form an on-chip network or a standard global memory access bus (10); Multiple in-memory computing structures share a single external interface (501), and the on-chip network or standard global memory access bus (10) is connected to the external interface (501).

9. The chip according to claim 8, characterized in that, Adjacent memory computing architectures share a memory unit (101), or adjacent memory computing architectures share an in-memory computing unit (201), or adjacent memory computing architectures share a computing / processing unit (502).

10. The chip according to claim 9, characterized in that, Also includes: A command / address encoding module connected to an external interface (501), the command / address encoding module being used to manage the working status and address selection of each in-memory computing architecture; The chip also includes a data switching circuit for bridging the connection between each in-memory computing structure and the external interface (501).

Images