A hybrid heterogeneous storage-compute integrated acceleration architecture and content-aware quantization method

Abstract

The application provides a hybrid heterogeneous computing and storage integrated acceleration architecture and a content-aware quantization method. The architecture comprises a content-aware quantization module for differentiating precision quantization of input and weight data based on data importance and outputting configuration information; a non-volatile memory computing and storage integrated circuit comprising a precision adaptation interface for storing static data and dynamically adjusting sampling precision according to the configuration information to perform multiply-accumulate operations; and an embedded dynamic random access memory computing and storage integrated circuit for dynamic data storage and high-frequency update calculation, and fusing the calculation results of the non-volatile memory computing and storage integrated circuit through an interaction layer. The application can realize differentiated adaptation of static and dynamic data, reduce data handling overhead and redundant invalid calculation, effectively slow down the performance degradation of the non-volatile memory computing and storage integrated circuit device and improve the hardware energy efficiency, and balance the calculation precision of large models and hardware reliability.

Description

Technical Field

[0001] This invention relates to the field of in-memory computing AI chip technology, and in particular to a hybrid heterogeneous in-memory computing acceleration architecture and a content-aware quantization method. Background Technology

[0002] With the widespread application of large-scale artificial intelligence models in various fields, the surge in parameter size and the increase in computational complexity have posed severe challenges to the computing power, energy efficiency, and storage capacity of hardware platforms. In scenarios involving large language model inference and neural network training, the system needs to simultaneously process massive static model parameters and frequently updated dynamic state matrices and intermediate values. The inherent property of separating storage and computing units under the traditional von Neumann architecture leads to significant power consumption and latency when data is transmitted across units. The resulting storage wall bottleneck severely restricts the efficient deployment of large-scale intelligent models.

[0003] To alleviate the aforementioned bottlenecks, in-memory computing architectures deeply integrate computing logic with storage units, enabling data to be processed directly at the storage location, effectively reducing data transfer overhead. Current in-memory computing architecture designs heavily rely on the technological iteration of storage devices; SRAM, eDRAM, and new non-volatile memories such as FeFET and ReRAM have been widely used in related designs. Existing solutions attempt to leverage the physical characteristics of different media to adapt to the storage and computing needs of differentiated data. For example, eDRAM's high endurance, low power consumption for read / write operations, and high area efficiency are used to process frequently updated dynamic data, or the high density and non-volatile characteristics of non-volatile memories are used to achieve efficient storage of static weights.

[0004] However, existing in-memory computing acceleration solutions still face limitations in adapting to the diverse computational needs of large models. Due to the lack of effective hardware-software co-design, the hardware architecture struggles to dynamically allocate resources based on the importance characteristics of the model data. This results in storage resources failing to accurately match the significant differences in storage density and update frequency between static weights and dynamic data, leading to low storage utilization and redundant computations. Furthermore, when performing calculations with fixed precision, high-frequency voltage inputs accelerate the degradation of the polarization or resistive switching characteristics of novel non-volatile storage devices, making it difficult for the system to maintain computational accuracy while simultaneously ensuring operational energy efficiency and long-term hardware reliability. Summary of the Invention

[0005] This invention provides a hybrid heterogeneous in-memory computing acceleration architecture and a content-aware quantization method to solve the defects of existing in-memory computing architectures, such as the inability to meet the needs of differentiated data due to the single storage medium, and the invalid computational redundancy and performance degradation of storage devices caused by fixed computational precision. This enables the accelerated deployment of large-scale intelligent models with high energy efficiency and high reliability.

[0006] This invention provides a hybrid heterogeneous in-memory computing acceleration architecture, comprising: The content-aware quantization module is used to perform differential precision quantization on input data and weight data based on data importance characteristics, and generate precision configuration information; The non-volatile memory-based computing circuit includes a precision adaptation interface for storing static data and performing parallel multiply-accumulate operations. The non-volatile memory-based computing circuit dynamically adjusts the sampling precision of the readout circuit based on the precision configuration information through the precision adaptation interface to generate corresponding static data calculation results. An embedded dynamic random access memory (DRAM) in-memory computing circuit is used to store dynamic data and perform high-frequency update calculation tasks. The embedded dynamic random access memory in-memory computing circuit includes an interaction layer, which is used to perform fusion calculations on the static data calculation results and the high-frequency update calculation results of the embedded dynamic random access memory in-memory computing circuit itself.

[0007] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the non-volatile memory in-memory computing circuit includes: A cross-array structure, wherein each cross point of the cross-array structure corresponds to a non-volatile storage unit, and the cross-array structure is used to perform calculations directly within the storage array; A driving circuit, connected to the cross array structure, is used to convert input data or programming data into corresponding electrical signals and apply them to the word lines of the cross array structure. The readout circuit is connected to the cross array structure and controlled by the precision adaptation interface, and is used to dynamically adjust the sampling precision according to the precision configuration information; An accumulator, used to accumulate the data after analog-to-digital conversion by weights; A data buffer is used to temporarily store the static data calculation results and transmit them to the interaction layer.

[0008] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the non-volatile memory in-memory computing circuit is configured to have mutually independent programming states and computing states: In the programming state, the driving circuit is used to write the quantized weight data into the cross array structure to map the weight data to the physical state of the non-volatile memory cell; In the computation state, the driving circuit is used to convert the input data into a voltage signal and apply it to the word lines of the cross array structure, so that each of the non-volatile memory cells generates a corresponding drain current according to the physical state, and the readout circuit accumulates the current to complete the multiplication-accumulation operation.

[0009] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the embedded dynamic random access memory in-memory computing circuit includes: The array layer consists of multiple embedded dynamic random access memory blocks. Each embedded dynamic random access memory block contains a storage array composed of storage cells. Eight adjacent storage cells in the same row are used to store an eight-bit dynamic data and are connected to the local addition tree within the embedded dynamic random access memory block to complete the initial accumulation of the multiply-accumulate operation. The driver layer integrates a word line decoder, a bit line driver circuit, and a readout circuit. The readout circuit includes a sensitive amplifier, which is used to convert the readout analog signal into a digital signal. The driver layer is used to perform timing control and data driving of the embedded dynamic random access memory in-memory computing circuit. The interaction layer includes a data buffer and interface circuits, which are used to temporarily store the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself, and to perform fusion calculations on the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself.

[0010] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the unit structure of the embedded dynamic random access memory in-memory computing circuit includes a two-transistor embedded dynamic random access memory unit or a five-transistor embedded dynamic random access memory unit, and the two-transistor embedded dynamic random access memory unit and the five-transistor embedded dynamic random access memory unit are compatible with the same architecture. The dual-transistor embedded dynamic random access memory unit adopts a dual-transistor structure and relies on the parasitic capacitance of the gate of the storage transistor to realize data storage, and achieves read-write separation through separate read-write ports and control lines. The five-transistor embedded dynamic random access memory unit adds three auxiliary transistors to the two-transistor embedded dynamic random access memory unit to receive input data and perform multiplication calculations within the unit. During active inference, the embedded dynamic random access memory in-memory computing circuit utilizes the high-frequency update operation of dynamic data as the implicit refresh cycle of the storage unit to achieve operation without additional refresh cycles.

[0011] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the content-aware quantification module uses a lightweight statistical standard to calculate the importance index. The calculation expression of the lightweight statistical standard is the ratio of the maximum absolute value of a single set of data to the root mean square value of that set of data. Furthermore, the content-aware quantification module divides the data into categories based on the comparison results between the importance index and the preset threshold of the corresponding dimension: High-importance data exceeding the preset threshold is quantized using eight-bit precision; and low-importance data less than or equal to the preset threshold is quantized using four-bit precision. When processing the low-importance data, the non-volatile memory-in-memory computing circuit reduces the number of computation cycles and thus lowers the frequency at which the non-volatile memory cell receives high-voltage input.

[0012] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the content-aware quantization module performs quantization in the following dimensions respectively: For the same batch of input data fed into the non-volatile memory-in-memory computing circuit, a first importance index is calculated and quantized on a per-input data token basis; and, For static data stored in the non-volatile memory-in-memory computing circuit, a second importance index is calculated and quantized on a weighted output channel basis.

[0013] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the non-volatile memory in-memory computing circuit processes four-bit precision data using a bit-serial input method, inputting one bit of input data per clock cycle, and the four-bit precision data only needs four clock cycles to complete the input, so as to mitigate the polarization or resistive switching characteristic degradation of the non-volatile memory cell.

[0014] According to the hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention, the quantized weight data is stored in the cross array structure, wherein four-bit precision weights are stored by four consecutive non-volatile memory cells, and eight-bit precision weights are stored by eight consecutive non-volatile memory cells.

[0015] This invention also provides a content-aware quantization method, implemented using any of the above-described hybrid heterogeneous in-memory computing acceleration architectures, the method comprising: Based on the importance characteristics of the data to be processed, the first importance index and the second importance index are calculated using lightweight statistical standards for the input data token and the weight output channel, respectively. The first importance index and the second importance index are compared with the first preset threshold and the second preset threshold, respectively. Based on the comparison results, four-bit or eight-bit quantization precision is dynamically allocated, and precision configuration information is generated. The precision configuration information is synchronized to the precision adaptation interface of the non-volatile memory in-memory computing circuit so that the non-volatile memory in-memory computing circuit can dynamically adjust the sampling precision of the readout circuit. The computation cycle of the non-volatile memory in-memory computing circuit is controlled according to the precision configuration information. By reducing the number of input cycles for low-importance data, the frequency at which the non-volatile memory cell receives high-voltage input is reduced.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content-aware quantization method as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the content-aware quantization method as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the content-aware quantization method as described above.

[0019] This invention provides a hybrid heterogeneous in-memory computing acceleration architecture and a content-aware quantization method. The content-aware quantization module performs differentiated precision quantization on input and weight data, and a hybrid heterogeneous hardware architecture composed of a non-volatile memory in-memory computing circuit and an embedded dynamic random access memory in-memory computing circuit processes the data collaboratively. On one hand, it utilizes the physical characteristics of different storage media to adapt to the differentiated needs of long-term static data storage and high-frequency dynamic data updates, breaking the resource utilization limitations of a single storage medium and significantly reducing the power consumption and latency caused by data transfer between storage and computing units. On the other hand, through deep collaboration between the precision configuration information generated by the quantization module and the precision adaptation interface of the non-volatile memory in-memory computing circuit, it achieves dynamic matching between hardware computing precision and data importance characteristics. This effectively reduces unnecessary computational redundancy and improves computing energy efficiency while dynamically adjusting the sampling precision of the readout circuit. This reduces unnecessary computational overhead, lowers the wear and tear on non-volatile storage devices, and extends the overall cycle life of the devices. Finally, through fusion computation implemented through the interaction layer, it achieves coordinated optimization of computing precision, hardware energy efficiency, and device reliability while ensuring the accuracy of large-scale intelligent model calculations. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1This is a schematic diagram illustrating the problems of existing in-memory computing architectures.

[0022] Figure 2 This is a schematic diagram of the overall hybrid heterogeneous in-memory computing acceleration architecture provided by the present invention.

[0023] Figure 3 This is a schematic diagram of the structure of the non-volatile memory-in-memory computing circuit provided by the present invention.

[0024] Figure 4 This is a schematic diagram of the embedded dynamic random access memory in-memory computing circuit provided by the present invention.

[0025] Figure 5 This is a schematic diagram illustrating the precision mapping and effect of the content-aware quantization module provided by the present invention.

[0026] Figure 6 This is a flowchart illustrating the content-aware quantification method provided by the present invention.

[0027] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The present invention will now be described in detail with reference to the accompanying drawings. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. In the description of the present invention, unless otherwise stated, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B, and C includes: A existing alone, B existing alone, A and B existing simultaneously, A and C existing simultaneously, B and C existing simultaneously, and A, B, and C existing simultaneously. In the present invention, " / " means "or". For example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0030] Current in-memory computing (CIM) acceleration solutions, such as T-PIM (Heo et al., JSSC 2022), primarily employ a single storage medium (e.g., SRAM). While offering high read / write speeds, their scalability is limited by low storage density and high static leakage current when dealing with large-scale intelligent models (LLMs). Another approach, such as HRAMTran (Zhang et al., ICCAD 2025), attempts to combine SOT MRAM with SRAM to utilize MRAM for storing static weights. However, existing technologies still fail to address the limitations of single-medium storage in meeting diverse data requirements, the ineffective computations caused by fixed precision, and the durability degradation of novel non-volatile devices (e.g., FeFET, ReRAM) during high-frequency read / write operations. In particular, fixed-precision in-memory computations consume high bit width resources regardless of data importance, resulting not only in energy efficiency losses but also accelerating the degradation of device polarization or resistive switching characteristics due to frequent high-voltage inputs.

[0031] like Figure 1 In the existing technologies shown, on the one hand, traditional storage media have poor adaptability to data characteristics, failing to meet differentiated data needs and resulting in low resource utilization. Single-media CIM architectures, such as SRAM-based CIM architectures, have low storage density and high bit-line charging / discharging power consumption, failing to meet the different storage and computing needs of static and dynamic data. Figure 1 As shown in (a), static weights / model parameters require high-density, low-leakage, and long-term retention storage characteristics, while dynamic state matrices, training intermediate values, and other data require high-endurance, low-power, and frequent read / write storage characteristics. Although the FeFET / ReRAM-based CIM architecture can achieve high-density, low-leakage storage of static weights, the new devices themselves have insufficient endurance and are prone to degradation with repeated read / write operations, making it unable to support high-frequency updates of dynamic state matrices, training intermediate values, and other data. Hybrid storage media solutions require flexible combinations of multiple storage media, placing high demands on data scheduling capabilities, leading to low storage resource utilization and hardware redundancy. To address this shortcoming, this invention aims to provide a hybrid heterogeneous architecture with multiple storage media options, breaking the limitations of a single storage medium and supporting flexible combinations of multiple storage media such as SRAM, FeFET, and ReRAM. This allows different storage media to be adapted to the storage and computing needs of static and dynamic data respectively, improving storage resource utilization, reducing hardware power consumption, and solving the problem of poor storage media compatibility.

[0032] On the other hand, rigid control of computational precision leads to ineffective computational redundancy and may accelerate device performance degradation. Existing in-memory computing architectures generally use fixed bit-width precision, such as INT8, for computation, lacking hardware-software co-design. They fail to deeply integrate software-level quantization strategies with the hardware architecture and do not optimize for the non-uniform distribution, significant values, and redundant data characteristics of intelligent models. This results in a large amount of low-importance data still occupying high-bit-width computational resources, causing ineffective computation and wasted power. Simultaneously, new memory devices such as FeFET and ReRAM inherently suffer from performance degradation after repeated read / write operations. High-frequency computation causes these devices to frequently receive high-voltage inputs, further accelerating the degradation of their polarization / resistive switching characteristics, shortening hardware lifespan, and failing to balance computational performance and device reliability. Figure 1 As shown in (b). To address this shortcoming, this invention aims to introduce a content-aware hybrid precision quantization mechanism that integrates software and hardware. By deeply adapting the quantization strategy at the software level to the hardware architecture, and considering the non-uniform distribution of data, the precision of the input data and weight output channels is dynamically adjusted to reduce invalid computation and power consumption waste. At the same time, it slows down the performance degradation of new devices, extends hardware lifespan, and achieves a balance between computational precision, energy efficiency, and device reliability, fully leveraging the advantages of software and hardware collaboration.

[0033] In view of this, the present invention proposes a hybrid heterogeneous in-memory computing acceleration architecture and content-aware quantization method. By constructing a system that deeply integrates non-volatile memory in-memory computing circuit (NVM CIM), embedded dynamic random access memory in-memory computing circuit (eDRAM CIM), and content-aware quantization module, it utilizes the high-density non-volatile adaptability of NVM to store static weights and the high endurance read-write separation characteristics of eDRAM to adapt to the high-frequency updates of dynamic data. At the same time, it uses a lightweight statistical standard κ at the software level to evaluate the importance of input data instructions (Token) and weight output channels, dynamically triggering 4-bit and 8-bit mixed precision calculation logic. At the hardware level, it reduces the calculation cycle and voltage input frequency of low-importance data through precision adaptation interface and bit serial input mechanism. Thus, it significantly improves in-memory computing energy efficiency, ensures the inference accuracy of large models, and effectively extends the lifespan of new non-volatile storage devices from the underlying software and hardware interaction level. It achieves a high degree of synergistic optimization of static and dynamic data, high precision and long lifespan, and software quantization and hardware execution.

[0034] It is worth noting that the hybrid heterogeneous in-memory computing acceleration architecture proposed in this invention includes two core circuits: one is an in-memory computing circuit based on non-volatile memory (hereinafter referred to as NVM CIM, also known as weighted in-memory computing circuit WCIM in the architecture logic), used to store static weights; the other is an in-memory computing circuit based on embedded dynamic random access memory (hereinafter referred to as eDRAM CIM, also known as state in-memory computing circuit SCIM in the architecture logic), used to store dynamic data.

[0035] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0036] In some specific embodiments of the present invention, this solution provides a hybrid heterogeneous in-memory computing acceleration architecture, including: The content-aware quantization module is used to perform differential precision quantization on input data and weight data based on data importance characteristics, and generate precision configuration information; The non-volatile memory-based computing circuit includes a precision adaptation interface for storing static data and performing parallel multiply-accumulate operations. The non-volatile memory-based computing circuit dynamically adjusts the sampling precision of the readout circuit based on the precision configuration information through the precision adaptation interface to generate corresponding static data calculation results. An embedded dynamic random access memory (DRAM) in-memory computing circuit is used to store dynamic data and perform high-frequency update calculation tasks. The embedded dynamic random access memory in-memory computing circuit includes an interaction layer, which is used to perform fusion calculations on the static data calculation results and the high-frequency update calculation results of the embedded dynamic random access memory in-memory computing circuit itself.

[0037] Specifically, please refer to Figure 2 The hybrid heterogeneous in-memory computing acceleration architecture provided in this embodiment of the invention employs a heterogeneous system consisting of a weight calculation circuit based on non-volatile memory (NVM) (labeled WCIM in the figure) and a state calculation circuit based on embedded dynamic random access memory (eDRAM) (labeled SCIM in the figure). Specifically, WCIM utilizes the non-volatility of NVM to ensure storage density, while SCIM utilizes the high durability of eDRAM to ensure high-frequency updates of dynamic data.

[0038] See Figure 2 The configuration information generated by the content-aware quantization module controls the NVM CIM (i.e., WCIM in the diagram) and the eDRAM CIM (i.e., SCIM in the diagram), respectively. The output of WCIM is connected to the input of the interaction layer of SCIM to realize the fusion of static weight calculation results and dynamic state matrix.

[0039] More specifically, the content-aware quantization module, as a software layer, analyzes data characteristics and generates precision configuration information to guide hardware behavior. Based on data importance, it generates precision configuration information and sends it to WCIM and SCIM respectively to regulate their computational behavior. WCIM and SCIM, as heterogeneous hardware layers, are used to process static and dynamic data respectively. Finally, the computation results are fused through an interaction layer within SCIM. WCIM and SCIM are connected via interaction logic, and the final computation results converge to the global input / output registers.

[0040] In some possible embodiments of the present invention, the non-volatile memory-in-memory computing circuit includes: A cross-array structure, wherein each cross point of the cross-array structure corresponds to a non-volatile storage unit, and the cross-array structure is used to perform calculations directly within the storage array; A driving circuit, connected to the cross array structure, is used to convert input data or programming data into corresponding electrical signals and apply them to the word lines of the cross array structure. The readout circuit is connected to the cross array structure and controlled by the precision adaptation interface, and is used to dynamically adjust the sampling precision according to the precision configuration information; An accumulator, used to accumulate the data after analog-to-digital conversion by weights; A data buffer is used to temporarily store the static data calculation results and transmit them to the interaction layer.

[0041] Please see Figure 3 , Figure 3 This is a schematic diagram of the Non-Volatile Memory Computing In-Memory (NVM CIM) circuit. The NVM CIM includes a cross-array structure, driver circuitry, readout circuitry (SA), accumulator, and data buffer. The NVM CIM module is primarily used for storing static data such as model weights and for parallel computing tasks.

[0042] In this embodiment, the NVM CIM circuit includes a cross-array structure, a driving circuit, a readout circuit, an accumulator, and a data buffer. Each cross-point of the cross-array structure is equipped with a non-volatile memory cell, i.e., an NVM cell in the figure. The NVM cell can be selected from FeFET, ReRAM, PCM, or MRAM according to application requirements. The driving circuit receives the quantized input data and converts it into a voltage signal applied to the NVM cell array. The NVM cells in the cross-array store static weights. In computation mode, the readout circuit SA senses the bit line current and converts it into a digital signal. The accumulator accumulates the partial sum generated by the bit serial input, and the final result is temporarily stored in the data buffer, which then transmits it to the subsequent SCIM module or interaction layer.

[0043] In some possible embodiments of the present invention, the non-volatile memory-in-memory computing circuit is configured to have mutually independent programming and computing states: In the programming state, the driving circuit is used to write the quantized weight data into the cross array structure to map the weight data to the physical state of the non-volatile memory cell; In the computation state, the driving circuit is used to convert the input data into a voltage signal and apply it to the word lines of the cross array structure, so that each of the non-volatile memory cells generates a corresponding drain current according to the physical state, and the readout circuit accumulates the current to complete the multiplication-accumulation operation.

[0044] Specifically, to adapt to different work requirements, the NVM CIM circuit is configured to have mutually independent programming and calculation states: In programming mode, the main task is to write the model weights.

[0045] In some possible embodiments of the present invention, the quantized weight data is stored in the cross array structure, wherein four-bit precision weights are stored by four consecutive non-volatile memory cells, and eight-bit precision weights are stored by eight consecutive non-volatile memory cells.

[0046] Specifically, to optimize storage resource utilization and simplify control logic, the quantized weight data employs a contiguous physical mapping scheme during writing. Specifically, a four-bit precision weight data is stored in four physically contiguous non-volatile memory cells, while an eight-bit precision weight is stored in eight consecutive cells. The driving circuit is responsible for converting this quantized weight data into precise programming voltages or currents, mapping them to the physical states of the memory cells, such as the polarization of a FeFET or the resistance value of a ReRAM.

[0047] In some possible embodiments of the present invention, the non-volatile memory in-memory computing circuit uses a bit serial input method to process four-bit precision data, inputting one bit of input data per clock cycle, and the four-bit precision data only needs four clock cycles to complete the input, so as to mitigate the polarization or resistive switching characteristic degradation of the non-volatile memory cell.

[0048] Specifically, in computation mode, to achieve coordination with the content-aware quantization module, the driving circuit adopts a bit-serial input method. When input data is received, the driving circuit inputs one bit per clock cycle. Therefore, for four bits of low-importance data, only four clock cycles are needed to complete one multiplication-accumulation input process. This mechanism can significantly reduce the total frequency and duration of high-voltage input received by the storage unit, thereby effectively mitigating the performance degradation of the device caused by electrical stress.

[0049] In each cycle, an input voltage is applied to the word line. The storage cell generates a drain current corresponding to its weight based on its physical state. The currents on the same word line are naturally superimposed according to Kirchhoff's laws, and then sensed and converted by the readout circuit. The readout circuit is connected to the precision adaptation interface (SA) and dynamically adjusts the sampling precision according to configuration information. Finally, the accumulator accumulates the data sampled in each cycle according to the weights and temporarily stores the calculation result in a data buffer, ready to be transmitted to the interaction layer.

[0050] In some possible embodiments of the present invention, see Figure 4 , Figure 4 This is a schematic diagram of the embedded dynamic random access memory (eDRAM) computing circuit, illustrating the hierarchical design of the eDRAM CIM from cell to array. Figure 4 (a) is a two-tube, i.e., 2T unit. Figure 4 (b) is a five-tube, i.e., 5T unit. Figure 4 (c) shows the array layer structure of the eDRAM CIM. The embedded dynamic random access memory computing circuit (eDRAM CIM) is mainly responsible for the storage, high-frequency updates, and computation of dynamic data, such as state matrices and training intermediate values. Its overall architecture adopts a layered design, including an array layer, a driver layer, and an interaction layer.

[0051] The array layer consists of multiple embedded dynamic random access memory blocks. Figure 4 (c) consists of eDRAM blocks. Each embedded dynamic random access memory block contains a storage array composed of storage cells. To efficiently process data, eight adjacent storage cells in the same row are used to store an eight-bit dynamic data and are connected to the local addition tree within the embedded dynamic random access memory block to complete the initial multiplication and accumulation operation within the block, thereby reducing the data traffic of the global bus.

[0052] The driver layer integrates a word line decoder, namely the write bit line, write word line, bit line driver circuit, and readout circuit as shown in the figure. The readout circuit includes a sensitive amplifier, which is used to convert the readout analog signal into a digital signal. The driver layer is used to execute the timing control and data driving of the embedded dynamic random access memory computing circuit.

[0053] The interaction layer includes a data buffer and an interface circuit, which are used to temporarily store the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself, and to perform fusion calculation on the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself.

[0054] Overall, the eDRAM block is controlled by the bit line driver circuit. After the data is read out, it is initially accumulated through a local addition tree. Then, through a sensitive amplifier, updater, and accumulated value interface, high-frequency updates of dynamic data and result fusion are achieved.

[0055] In some possible embodiments of the present invention, the unit structure of the embedded dynamic random access memory in-memory computing circuit includes a two-transistor embedded dynamic random access memory unit or a five-transistor embedded dynamic random access memory unit, and the two-transistor embedded dynamic random access memory unit and the five-transistor embedded dynamic random access memory unit are compatible with the same architecture. The dual-transistor embedded dynamic random access memory unit adopts a dual-transistor structure and relies on the parasitic capacitance of the gate of the storage transistor to realize data storage, and achieves read-write separation through separate read-write ports and control lines. The five-transistor embedded dynamic random access memory unit adds three auxiliary transistors to the two-transistor embedded dynamic random access memory unit to receive input data and perform multiplication calculations within the unit.

[0056] Specifically, see still Figure 4 This embodiment provides two structures. Figure 4 (a) A 2T eDRAM cell consisting of a write transistor MG and a storage transistor MS; Figure 4 (b) A 5T eDRAM cell with an auxiliary transistor added to a 2T base. The structure of the memory cell can be flexibly selected as a two-transistor (2T) eDRAM cell or a five-transistor (5T) eDRAM cell, both of which are compatible with the same architecture. The 2T cell has a simple structure and achieves efficient read-write separation operation through separate read and write ports. The 5T cell adds a transistor to the 2T base and can directly perform multiplication calculations within the cell. Since dynamic data updates are frequent in scenarios such as large language model inference, this high-frequency update operation can directly serve as the implicit refresh cycle of the eDRAM cell, thereby achieving operation without additional refresh cycles during active inference and ensuring the continuity of data processing.

[0057] The driver layer integrates timing control and data driving circuits. The interaction layer contains data buffers and interface circuits, which are used to temporarily store the static data calculation results transmitted by NVM CIM and their own dynamic data calculation results, and finally complete the fusion calculation of the two.

[0058] In some possible embodiments of the present invention, the content-aware quantification module uses a lightweight statistical standard to calculate an importance index, and the calculation expression of the lightweight statistical standard is the ratio of the maximum absolute value of a single set of data to the root mean square value of that set of data. Furthermore, the content-aware quantification module divides the data into categories based on the comparison results between the importance index and the preset threshold of the corresponding dimension: High-importance data exceeding the preset threshold is quantized using eight-bit precision; and low-importance data less than or equal to the preset threshold is quantized using four-bit precision.

[0059] Furthermore, the content-aware quantization module performs quantization in the following dimensions: For the same batch of input data fed into the non-volatile memory-in-memory computing circuit, a first importance index is calculated and quantized on a per-input data token basis; and, For static data stored in the non-volatile memory-in-memory computing circuit, a second importance index is calculated and quantized on a weighted output channel basis.

[0060] The content-aware quantization module is the core of the hardware and software collaboration in this invention. To accurately capture the importance of data, the content-aware quantization module uses the lightweight statistical standard к (ByteDance, 2025) as the criterion for judging data significance. It can quickly calculate the distribution characteristics of the data and distinguish significant data from redundant data. The calculation expression is as follows: (1) Where x represents all weight data within this channel, This represents the maximum absolute value of all data in a single set of input data tokens or a single weighted output channel. The root mean square value (k) represents the data set. A larger k value indicates a more uneven distribution of data, more prominent significance values, and higher data importance; a smaller k value indicates a more uniform distribution of data, higher redundancy, and lower data importance. By pre-setting a reasonable threshold, which can be dynamically adjusted according to the actual application scenario and model type, the data is divided into high-importance data and low-importance data, and quantified with different precision for each.

[0061] Specifically, based on the comparison between the indicator and the preset threshold, the content-aware quantization module dynamically divides the data into: high-importance data quantized with eight-bit precision and low-importance data quantized with four-bit precision.

[0062] The content-aware quantization module is deeply adapted to NVM CIM and eDRAM CIM modules. Its core purpose is to solve problems such as invalid computational redundancy and NVM device performance degradation caused by fixed-precision computation in existing in-memory computing architectures. Through software-level quantization strategies, it dynamically matches the hardware computational precision to achieve synergistic optimization of computational precision, energy efficiency and device reliability.

[0063] See Figure 5 , Figure 5 This is a schematic diagram illustrating the precision mapping and effect of the perceptual quantization module, where... Figure 5 (a) illustrates the physical mapping of two-dimensional quantization, where input data such as the Token dimension and weight data such as the output channel dimension are assigned 4-bit or 8-bit precision respectively according to their importance, and are coordinated in matrix multiplication through the accumulation direction. Figure 5 (b) A direct comparison of the technical effects shows that, with this solution, 8 bits are used only for highly important data, maintaining computational accuracy; while 4 bits are widely used for less important data, reducing computational overhead. Furthermore, since 4-bit precision requires only four input cycles, the number of input cycles is reduced, directly lowering the frequency at which the NVM unit receives high-voltage input, thus protecting hardware lifespan.

[0064] The content-aware quantization module provided in this embodiment performs differentiated precision quantization on the input data and the weighted output channel data based on the importance characteristics of the data. Instead of using a fixed bit width precision, it dynamically allocates two precisions: 4 bits and 8 bits. This ensures the accuracy of the calculation of core data, reduces the computational overhead of low-importance data, and slows down the performance degradation of NVM devices, thus achieving the core goal of software and hardware co-optimization.

[0065] To achieve precise control, quantization is performed in two dimensions: for input data, quantization is performed in units of input data tokens; for static data, quantization is performed in units of weighted output channels.

[0066] The generated precision configuration information is synchronized to the precision adaptation interface of NVM CIM, guiding it to dynamically adjust the sampling precision of the readout circuit and control the number of calculation cycles of the drive circuit. When processing data deemed to be of low importance, by reducing the number of calculation cycles, the frequency at which non-volatile memory cells receive high-voltage input is directly reduced, achieving synergistic optimization of precision, energy efficiency, and device reliability.

[0067] To verify the technical effectiveness of this invention, simulation tests have been conducted. At a frequency of 200MHz, the NVM part (WCIM) and eDRAM part (SCIM) of this architecture can achieve peak computing power of 2.457 GOPS and 2.453 GOPS, respectively, with corresponding energy efficiencies of 219.14 TOPS / W and 204.78 TOPS / W. The results show that the hardware and software co-design scheme proposed in this invention is feasible and has high performance, and can effectively support the efficient deployment of large-scale intelligent models.

[0068] The proposed hybrid heterogeneous in-memory computing acceleration architecture employs a hybrid heterogeneous architecture of NVM CIM and eDRAM CIM to achieve separate storage and collaborative computing of static and dynamic data. This breaks the limitations of single storage media, adapts to multiple types of non-volatile memories, and addresses the differentiated needs of high-density static data storage and high-frequency dynamic data updates. This hybrid heterogeneous architecture includes an NVM CIM module and an eDRAM CIM module. The NVM CIM module can use various non-volatile memories such as FeFET, ReRAM, PCM, and MRAM, depending on the application scenario, to handle static data storage and MAC operations. The eDRAM CIM module handles dynamic data storage, high-frequency updates, and other operations. The two modules work together to complete the full hybrid computing process.

[0069] In some specific embodiments of the present invention, such as Figure 6 As shown, this solution provides a content-aware quantization method, implemented using any of the above-described hybrid heterogeneous in-memory computing acceleration architectures. The method includes the following steps: Step S100: Based on the importance characteristics of the data to be processed, calculate the first importance index and the second importance index for the input data token and the weight output channel respectively using lightweight statistical standards; Step S200: Compare the first importance index and the second importance index with the first preset threshold and the second preset threshold respectively, dynamically allocate four-bit or eight-bit quantization precision according to the comparison results, and generate precision configuration information. Step S300: Synchronize the precision configuration information to the precision adaptation interface of the non-volatile memory in-memory computing circuit, so that the non-volatile memory in-memory computing circuit can dynamically adjust the sampling precision of the readout circuit. Step S400: Control the calculation cycle of the non-volatile memory in-memory computing circuit according to the precision configuration information, and reduce the frequency of high voltage input received by the non-volatile memory unit by reducing the number of input cycles for low-importance data.

[0070] In some possible embodiments of the present invention, the step of dynamically allocating four-bit or eight-bit quantization precision according to the comparison result specifically includes: If the first importance index is greater than the first preset threshold, the corresponding input data token is quantized using eight bits; otherwise, it is quantized using four bits. If the second importance index is greater than the second preset threshold, then eight-bit quantization is used for the corresponding weight output channel; otherwise, four-bit quantization is used.

[0071] See you again Figure 5 (a) The dual-dimensional quantization implementation process provided in this embodiment of the invention is specifically divided into two parts: input token quantization and weight output channel quantization. The two parts are implemented independently but are adapted to the hardware module in a coordinated manner.

[0072] (1) Input data quantization: For the same batch of data input to NVM CIM, the importance index of each token is first calculated using the above standard к. If the calculated index result is greater than the threshold, the data is determined to be high-importance data (containing core feature information), and quantization is performed with 8-bit precision to ensure the accuracy of core data calculation and avoid inference and training errors caused by insufficient quantization precision. If it is less than the threshold, it is determined to be low-importance data (high redundancy, less impact on calculation results), and quantization is performed with 4-bit precision to reduce the data bit width overhead and the number of input cycles, thereby reducing the use of NVM devices and alleviating device degradation problems. The input data quantization process is deeply coordinated with the hardware module: The quantized 4-bit / 8-bit data is transmitted to the NVM CIM module through the interface, and the precision configuration information (4-bit / 8-bit) is synchronized with the precision adaptation interface, so that the hardware module dynamically adjusts the precision parameters of the drive circuit and readout circuit to match the software quantization precision. After low-importance data is quantized with 4 bits, it only takes 4 clock cycles to send it to the NVM CIM module to complete the calculation. Compared with 8-bit quantization, the calculation cycle is reduced by 50%, which significantly reduces the frequency of NVM devices receiving high-voltage input, slows down the degradation of polarization / resistivity switching characteristics of new NVM devices such as FeFET and ReRAM, extends device life, and reduces power consumption waste caused by invalid calculations.

[0073] (2) Weighted output channel quantization: For the static weighted data stored in the NVM CIM module, the importance index of each output channel is calculated in units of weighted output channels. Because the saliency values ​​of weight output channels in linear attention and neural network calculations of large-scale intelligent models are relatively dispersed, κ can be used to accurately identify high-importance and low-importance channels: if the calculated κ values ​​are significantly different from the weighted output channels, then the significant values ​​of weighted output channels are significantly different from the weighted output channels. If the value is greater than the threshold, the output channel is a high-importance channel, using 8-bit precision quantization to ensure the accuracy of core weight calculation; if it is less than the threshold, the output channel is a low-importance channel, using 4-bit precision quantization to save storage resources and computational overhead in the NVM CIM module. The quantization of the weight output channel is deeply adapted to the NVM CIM module: the quantized weight data is written to the NVM cross array through the programming stage. 4-bit weights can be stored in 4 consecutive NVM storage cell blocks, and 8-bit weights can be stored in 8 consecutive NVM storage cell blocks. No modification to the overall structure of the NVM CIM module is required; precision adaptation can be achieved solely through software configuration. Using 4-bit quantization for low-importance channels saves 50% of NVM storage resources and reduces current consumption during MAC operations, further improving hardware energy efficiency.

[0074] The quantization effect of the perceptual quantization method of the present invention is as follows: Figure 5 As shown in (b), the main improvements are in data distribution adaptation and hardware performance enhancement. High-importance input data and weighted output channels employ 8-bit quantization to retain core data features and ensure no significant loss in model calculation accuracy. Low-importance data uses 4-bit quantization, effectively reducing computation cycles and storage overhead, achieving significant resource savings. This reduces the computation cycle of the NVM CIM module, slows performance degradation of new NVM devices, extends hardware lifespan, reduces power consumption, and achieves triple optimization of accuracy, energy efficiency, and reliability. Furthermore, this method offers good flexibility, dynamically adjusting thresholds to adapt to different scenarios and models, and works collaboratively with NVM CIM and eDRAM CIM modules, fully demonstrating the advantages of hardware-software synergy.

[0075] This invention proposes a content-aware hybrid precision quantization method that dynamically allocates 4-bit / 8-bit quantization precision based on data importance, accurately distinguishing between high-importance and low-importance data. This ensures computational accuracy while reducing unnecessary computation and device losses. The quantization process uses a lightweight statistical standard κ to determine data importance, presets dynamic thresholds to divide high / low importance data, and performs differentiated quantization of input data and weighted output channels using 4-bit / 8-bit methods respectively. Furthermore, the quantization process is deeply integrated with the hardware module, synchronizing quantization precision configuration information to the precision adaptation interface of the NVM CIM module. This achieves dynamic matching between software quantization and hardware computation. Low-importance data is quantized to 4 bits to reduce computation cycles and mitigate NVM device degradation, fully leveraging the performance advantages of the hybrid heterogeneous architecture to achieve triple optimization of computational accuracy, energy efficiency, and device reliability.

[0076] In some specific embodiments of the present invention, this solution also provides a hybrid heterogeneous in-memory computing data processing method, implemented using the hybrid heterogeneous in-memory computing acceleration architecture described in any of the above embodiments, the method comprising: The content-aware quantization module performs differential precision quantization on input data and weight data based on data importance characteristics, generating precision configuration information. Static data is stored and parallel multiply-accumulate operations are performed through the precision adaptation interface of the non-volatile memory in-memory computing circuit; and based on the precision configuration information, the sampling precision of the readout circuit is dynamically adjusted through the precision adaptation interface to generate the corresponding static data calculation results. The embedded dynamic random access memory (DRAM) in-memory computing circuit stores dynamic data and performs high-frequency update calculation tasks. The embedded DRAM in-memory computing circuit includes an interaction layer, through which the static data calculation results are fused with the high-frequency update calculation results of the embedded DRAM in-memory computing circuit itself.

[0077] Compared with existing technologies, the hybrid heterogeneous in-memory computing acceleration architecture, content-aware quantization method, and hybrid heterogeneous in-memory computing data processing method provided by this invention have the following advantages: This invention offers enhanced storage media adaptability, overcoming the inherent limitations of single-media architectures and significantly improving resource utilization. Existing single-media in-memory architectures cannot simultaneously address the differentiated needs of static and dynamic data. SRAM-based architectures suffer from low storage density and high power consumption, while NVM-based architectures cannot support high-frequency updates. Hybrid media architectures exhibit poor flexibility, ineffective hardware-software co-design, and are designed for single scenarios, failing to adapt to high-frequency update scenarios such as neural network training. This invention employs a hybrid heterogeneous architecture combining NVM and eDRAM, supporting flexible selection of various NVM devices such as FeFET and ReRAM. NVM adapts to static data, while eDRAM adapts to dynamic data. Furthermore, eDRAM offers flexible 2T / 5T unit configurations, allowing for matching storage and computing requirements according to specific scenarios. This invention can adapt to the accuracy and power consumption requirements of different scenarios without modifying the overall architecture. Relying on the collaborative working mode of NVM CIM and eDRAMCIM (such as the implicit refresh of eDRAM and the low power standby of NVM), it can flexibly meet the needs of various scenarios such as large language model inference and neural network training, greatly expanding the scope of application. In the future, it can also be adapted to larger-scale intelligent models by expanding the in-memory macrocells and adjusting the quantization threshold, making it more scalable.

[0078] With strong hardware-software synergy, controllable computational accuracy, reduced redundant and ineffective computations, and significantly extended device lifespan, this invention fully leverages the architecture's performance potential. Current state-of-the-art fixed-precision hybrid in-memory computing architectures employ fixed INT8 precision, failing to utilize data redundancy, resulting in numerous ineffective computations. Furthermore, high-frequency fixed-precision computations accelerate NVM device degradation and fail to achieve deep synergy between software quantization and hardware architecture. Simply optimizing either hardware or software alone cannot fully unleash the performance of the in-memory computing architecture. This invention utilizes a content-aware quantization method, deeply adapted to NVM CIM and eDRAM CIM modules. It dynamically allocates 4-bit / 8-bit precision based on data importance, ensuring accuracy for high-importance data and reducing overhead for low-importance data. This reduces ineffective computations by 50%, while simultaneously reducing input to NVM devices, slowing down device polarization / resistive switching characteristic degradation, and extending hardware lifespan, achieving a triple optimization of accuracy, energy efficiency, and reliability.

[0079] The data processing method and hardware design proposed in this invention have been designed, implemented, and tested through simulation, ensuring the feasibility and high performance of the algorithm and architecture design. This invention achieves peak computing power of 2.457 GOPS (WCIM) and 2.453 GOPS (SCIM) at a frequency of 200MHz, with energy efficiency of 219.14 TOPS / W (WCIM) and 204.78 TOPS / W (SCIM). This invention can simultaneously support dynamic and static data processing of large-scale intelligent models, and utilizes in-memory computing mechanisms to reduce the frequency of frequent data communication between storage and computing units.

[0080] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communications bus 840. The processor 810 can call logic instructions in the memory 830 to execute a content-aware quantization method. This method includes: calculating a first importance index and a second importance index based on the importance characteristics of the data to be processed, using lightweight statistical standards for the input data token and weighted output channel, respectively; comparing the first importance index and the second importance index with a first preset threshold and a second preset threshold, respectively; dynamically allocating four-bit or eight-bit quantization precision according to the comparison results, and generating precision configuration information; synchronizing the precision configuration information to the precision adaptation interface of the non-volatile memory-in-memory computing circuit, so that the non-volatile memory-in-memory computing circuit dynamically adjusts the sampling precision of the readout circuit; and controlling the computation cycle of the non-volatile memory-in-memory computing circuit according to the precision configuration information, thereby reducing the frequency of high-voltage input received by the non-volatile memory unit by reducing the number of input cycles for low-importance data.

[0081] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the content-aware quantization method provided by the above methods. The method includes: calculating a first importance index and a second importance index based on the importance characteristics of the data to be processed, respectively, for the input data token and the weight output channel using lightweight statistical standards; comparing the first importance index and the second importance index with a first preset threshold and a second preset threshold, respectively, dynamically allocating four-bit or eight-bit quantization precision according to the comparison results, and generating precision configuration information; synchronizing the precision configuration information to the precision adaptation interface of the non-volatile memory in-memory computing circuit, so that the non-volatile memory in-memory computing circuit can dynamically adjust the sampling precision of the readout circuit; controlling the calculation cycle of the non-volatile memory in-memory computing circuit according to the precision configuration information, thereby reducing the frequency of high-voltage input received by the non-volatile memory unit by reducing the number of input cycles for low-importance data.

[0083] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the content-aware quantization method provided by the above methods. The method includes: calculating a first importance index and a second importance index based on the importance characteristics of the data to be processed, respectively, for the input data token and the weighted output channel using lightweight statistical standards; comparing the first importance index and the second importance index with a first preset threshold and a second preset threshold, respectively, dynamically allocating four-bit or eight-bit quantization precision according to the comparison results, and generating precision configuration information; synchronizing the precision configuration information to the precision adaptation interface of the non-volatile memory-in-memory computing circuit, so that the non-volatile memory-in-memory computing circuit dynamically adjusts the sampling precision of the readout circuit; controlling the calculation cycle of the non-volatile memory-in-memory computing circuit according to the precision configuration information, thereby reducing the frequency of high-voltage input received by the non-volatile memory unit by reducing the number of input cycles for low-importance data.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hybrid heterogeneous in-memory computing acceleration architecture, characterized in that, include: The content-aware quantization module is used to perform differential precision quantization on input data and weight data based on data importance characteristics, and generate precision configuration information; The non-volatile memory-based computing circuit includes a precision adaptation interface for storing static data and performing parallel multiply-accumulate operations; the non-volatile memory-based computing circuit dynamically adjusts the sampling precision of the readout circuit through the precision adaptation interface based on the precision configuration information to generate corresponding static data calculation results. An embedded dynamic random access memory (DRAM) in-memory computing circuit is used to store dynamic data and perform high-frequency update calculation tasks. The embedded DRAM in-memory computing circuit includes an interaction layer, which is used to fuse the static data calculation results with the high-frequency update calculation results of the embedded DRAM in-memory computing circuit itself.

2. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 1, characterized in that, The non-volatile memory-in-memory computing circuit includes: A cross-array structure, wherein each cross point of the cross-array structure corresponds to a non-volatile storage unit, and the cross-array structure is used to perform calculations directly within the storage array; A driving circuit, connected to the cross array structure, is used to convert input data or programming data into corresponding electrical signals and apply them to the word lines of the cross array structure. The readout circuit is connected to the cross array structure and controlled by the precision adaptation interface, and is used to dynamically adjust the sampling precision according to the precision configuration information; An accumulator, used to accumulate the data after analog-to-digital conversion by weights; A data buffer is used to temporarily store the static data calculation results and transmit them to the interaction layer.

3. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 2, characterized in that, The non-volatile memory-based computing circuit is configured to have independent programming and computing states: In the programming state, the driving circuit is used to write the quantized weight data into the cross array structure to map the weight data to the physical state of the non-volatile memory cell; In the computation state, the driving circuit is used to convert the input data into a voltage signal and apply it to the word lines of the cross array structure, so that each of the non-volatile memory cells generates a corresponding drain current according to the physical state, and the readout circuit accumulates the current to complete the multiplication-accumulation operation.

4. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 1, characterized in that, The embedded dynamic random access memory in-memory computing circuit includes: The array layer consists of multiple embedded dynamic random access memory blocks. Each embedded dynamic random access memory block contains a storage array composed of storage cells. Eight adjacent storage cells in the same row are used to store an eight-bit dynamic data and are connected to the local addition tree within the embedded dynamic random access memory block to complete the initial accumulation of the multiply-accumulate operation. The driver layer integrates a word line decoder, a bit line driver circuit, and a readout circuit. The readout circuit includes a sensitive amplifier, which is used to convert the readout analog signal into a digital signal. The driver layer is used to perform timing control and data driving of the embedded dynamic random access memory in-memory computing circuit. The interaction layer includes a data buffer and interface circuits, which are used to temporarily store the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself, and to perform fusion calculations on the static data calculation results and the calculation results of the embedded dynamic random access memory in-memory computing circuit itself.

5. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 4, characterized in that, The unit structure of the embedded dynamic random access memory in-memory computing circuit includes a two-transistor embedded dynamic random access memory unit or a five-transistor embedded dynamic random access memory unit, and the two-transistor embedded dynamic random access memory unit and the five-transistor embedded dynamic random access memory unit are compatible with the same architecture. The dual-transistor embedded dynamic random access memory unit adopts a dual-transistor structure and relies on the parasitic capacitance of the gate of the storage transistor to realize data storage, and achieves read-write separation through separate read-write ports and control lines. The five-transistor embedded dynamic random access memory unit adds three auxiliary transistors to the two-transistor embedded dynamic random access memory unit to receive input data and perform multiplication calculations within the unit. During active inference, the embedded dynamic random access memory in-memory computing circuit utilizes the high-frequency update operation of dynamic data as the implicit refresh cycle of the storage unit to achieve operation without additional refresh cycles.

6. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 1, characterized in that, The content-aware quantification module uses a lightweight statistical standard to calculate the importance index. The calculation expression of the lightweight statistical standard is the ratio of the maximum absolute value of a single set of data to the root mean square value of that set of data. Furthermore, the content-aware quantification module divides the data into categories based on the comparison results between the importance index and the preset threshold of the corresponding dimension: Highly important data exceeding the preset threshold are quantized using eight-bit precision. Furthermore, low-importance data that is less than or equal to the preset threshold is quantized using four-bit precision; When processing the low-importance data, the non-volatile memory-in-memory computing circuit reduces the number of computation cycles and thus lowers the frequency at which the non-volatile memory cell receives high-voltage input.

7. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 6, characterized in that, The content-aware quantization module performs quantization in the following dimensions respectively: For the same batch of input data input to the non-volatile memory-in-memory computing circuit, the first importance index is calculated and quantized in units of input data tokens; as well as, For static data stored in the non-volatile memory-in-memory computing circuit, a second importance index is calculated and quantized on a weighted output channel basis.

8. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 3, characterized in that, The non-volatile memory-in-memory circuit uses a bit-serial input method to process four-bit precision data. One bit of input data is input in each clock cycle, and the four-bit precision data only needs four clock cycles to complete the input, so as to slow down the polarization or resistive switching characteristic degradation of the non-volatile memory cell.

9. The hybrid heterogeneous in-memory computing acceleration architecture according to claim 3, characterized in that, The quantized weight data is stored in the cross array structure, wherein four-bit precision weights are stored in four consecutive non-volatile memory cells, and eight-bit precision weights are stored in eight consecutive non-volatile memory cells.

10. A content-aware quantization method, characterized in that, The method is implemented using the hybrid heterogeneous in-memory computing acceleration architecture as described in any one of claims 1 to 9, and includes: Based on the importance characteristics of the data to be processed, the first importance index and the second importance index are calculated using lightweight statistical standards for the input data token and the weight output channel, respectively. The first importance index and the second importance index are compared with the first preset threshold and the second preset threshold, respectively. Based on the comparison results, four-bit or eight-bit quantization precision is dynamically allocated, and precision configuration information is generated. The precision configuration information is synchronized to the precision adaptation interface of the non-volatile memory in-memory computing circuit so that the non-volatile memory in-memory computing circuit can dynamically adjust the sampling precision of the readout circuit. The computation cycle of the non-volatile memory in-memory computing circuit is controlled according to the precision configuration information. By reducing the number of input cycles for low-importance data, the frequency at which the non-volatile memory cell receives high-voltage input is reduced.

Images