In-memory computing architecture supporting native transpose
By using a shared addition tree and a reconfigurable routing module, the data transport and hardware area issues of matrix transpose operations in traditional in-memory computing architectures are solved, achieving efficient native transpose computation and reducing latency and power consumption.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SEMICON TECH INNOVATION CENT(BEIJING) CORP
- Filing Date
- 2026-01-07
- Publication Date
- 2026-04-21
AI Technical Summary
Traditional in-memory computing architectures face huge data transfer overhead and latency when performing matrix transpose operations, and also have large hardware area overhead and low hardware utilization.
It adopts a shared addition tree and a reconfigurable routing module, and controls the data path at the input of the adder through the MUX module to realize the switching between horizontal and vertical accumulation modes. It shares a set of addition trees to support native transpose calculation.
It effectively reduces hardware area overhead, improves hardware utilization, and achieves efficient native transpose computing without the need to move data outside the array, thus reducing latency and power consumption.
Smart Images
Figure CN121900723A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of in-memory computing technology, and in particular relates to an in-memory computing architecture that supports native transposition. Background Technology
[0002] In-memory computing (CIM) technology effectively overcomes the "memory wall" bottleneck in traditional von Neumann architectures by performing computations within a memory array, demonstrating extremely high energy efficiency in computationally intensive tasks such as deep learning. However, many modern neural networks (such as Transformers) and signal processing tasks require frequent matrix transpose operations, and implementing transpose operations in traditional CIM architectures typically faces significant challenges.
[0003] 1. High data transfer overhead: Traditional methods require reading data into on-chip cache (SRAM) or off-chip memory (DRAM) for transposition, and then writing it back to the CIM array, resulting in huge data transfer overhead and latency;
[0004] 2. High hardware area cost: To support in-memory transposition, existing technologies typically employ a method of configuring two sets of adder trees (horizontal and vertical) to support bidirectional multiplication and accumulation. This method requires an additional set of adder trees, resulting in a significant increase in area overhead. Furthermore, the two sets of adder trees cannot be used simultaneously most of the time, leading to low hardware utilization.
[0005] In summary, existing technologies have resulted in significant data transfer overhead and latency, as well as substantial area overhead and low hardware utilization. To address these issues, this invention provides an in-memory computing architecture that supports native transposition. Summary of the Invention
[0006] The purpose of this invention is to provide an in-memory computing architecture that supports native transposition, so as to solve the problems existing in the prior art.
[0007] To achieve the above objectives, this invention provides an in-memory computing architecture that supports native transposition. This architecture, without significantly increasing area overhead, efficiently supports both horizontal and vertical accumulation modes through a shared addition tree and reconfigurable routing. Specifically, it includes:
[0008] Multiple in-memory computation subarrays: configured as two-dimensional arrays (e.g., supplementary arrays). Figure 2 (8 subarrays in the array).
[0009] A hierarchical shared adder tree: A hierarchical shared adder tree consists of multiple levels of adders used to hierarchically accumulate the computation results of subarrays.
[0010] A set of reconfigurable routing modules (MUX): MUXs are set at the input of at least one adder level to select the data path based on the operating mode.
[0011] The addition tree described in this invention is unique and shared. By controlling the reconfigurable routing module (MUX), this architecture can achieve both horizontal and vertical accumulation. This invention avoids the huge area cost associated with configuring a dual addition tree to achieve transpose, and in particular, avoids doubling the number of low-order adders. This solution only adds a small-area MUX module at the adder input, significantly saving chip layout area while providing efficient native transpose calculation capabilities.
[0012] Optionally, multiple in-memory computing subarrays are configured as a two-dimensional array according to a preset configuration strategy.
[0013] Optionally, the specific process of horizontal accumulation is as follows: the MUX routes the output of the horizontally adjacent upper-level functional block (subarray or low-order adder) to the input of the current level adder for accumulation.
[0014] Optionally, the specific process for vertical accumulation or transpose mode is as follows: the MUX routes the outputs of vertically adjacent parent function blocks to the inputs of the same set of current-level adders for accumulation. In this way, the present invention uses only one addition tree and flexibly reconstructs the data flow through the MUX to simultaneously support horizontal and vertical accumulation, thereby achieving native transpose calculation.
[0015] The technical effects of this invention are as follows:
[0016] 1. This invention avoids the huge area overhead associated with configuring two sets of adder trees (especially a large number of low-order adders) to achieve transposition by using a shared adder tree design. The solution only adds a MUX module (reconfigurable routing module) with a small area overhead at the adder input.
[0017] 2. This invention, through the flexible control of the reconfigurable routing module (MUX), can support both horizontal and vertical accumulation using only one addition tree, achieving efficient native transpose calculation without having to move data outside the array.
[0018] 3. This design concept remains applicable when the number of subarrays increases and higher-level adders are required. Even if two sets of adders are used at higher levels, the area cost is far less than that of replicating a large number of adders at lower levels, resulting in a significant overall layout advantage.
[0019] In summary, this invention is applicable to the field of integrated circuit design (Design Service-Layout), especially for in-memory computing (CIM) chip design that requires frequent matrix multiplication and matrix transpose operations, such as artificial intelligence, deep learning (e.g., Transformer networks), and signal processing. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0022] Figure 1 This is a schematic diagram of a traditional in-memory computing array structure;
[0023] Figure 2 This is a schematic diagram of the hierarchical layout of the adder tree in an embodiment of the present invention;
[0024] Figure 3 This is a schematic diagram of the hierarchical layout of the adder tree in an embodiment of the present invention;
[0025] Figure 4 This is a schematic diagram of the level 2 layout of the adder tree in an embodiment of the present invention. Detailed Implementation
[0026] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.
[0027] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.
[0028] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.
[0029] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.
[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0031] Example 1
[0032] like Figure 1 - Figure 4 As shown, this embodiment provides an in-memory computing architecture that supports native transposition, including:
[0033] 1) Two-dimensional in-memory computation subarray: Configure multiple in-memory computation subarrays, such as organized as 2x2 or other dimensional arrays;
[0034] 2) Hierarchical shared addition tree: Set up a unique and shared hierarchical addition tree (including first-level adder 5b, second-level adder 6b, third-level adder 7b, etc.) to perform hierarchical accumulation of subarray results;
[0035] 3) Reconfigurable Routing Module (MUX): A MUX is set at the input of the adder level. Two operating modes are achieved by controlling the MUX.
[0036] In horizontal accumulation mode, MUX routes the output of the horizontally adjacent parent function block to the input of the current level adder.
[0037] In vertical accumulation (transpose) mode, the MUX routes the outputs of vertically adjacent upper-level function blocks to the inputs of the same set of current-level adders, and even swaps the two sets of inputs in the middle MUX module.
[0038] This embodiment eliminates the redundant adder tree in traditional solutions (especially the area waste caused by doubling the number of low-order adders) by reusing the same set of adder hardware to process horizontal and vertical data flows. While ensuring the integrity of the computing function, it greatly reduces the layout area occupied and improves the chip's area efficiency. It solves the problem of area surge caused by configuring bidirectional adder trees when the existing in-memory computing architecture processes tasks that require matrix transposition, as well as the latency and power consumption problems caused by data out-of-memory transposition.
[0039] like Figure 2 As shown, this embodiment illustrates a CIM architecture comprising eight in-memory compute subarrays. Level 0 (as shown in the attached diagram) Figure 2 The basic unit (as shown) is a CIM subarray (e.g., 16x4 SRAM) and a first-stage adder. The first-stage adder (adder 5b) processes the addition of two 4-bit signed numbers. In horizontal accumulation mode, adder 5b processes subarray data from the same row; in vertical accumulation mode, adder 5b processes subarray data from the same column.
[0040] Level 1: such as Figure 3 As shown, the four level 0 function blocks (labeled #0, #1, #2, #3) are organized into a 2x2 array. Two adders 6b and the associated MUX are placed on the convergence path of these blocks.
[0041] Horizontal accumulation mode ( Figure 3 (Middle left diagram): The MUX configuration data flow routes the outputs of level 0 block #0 and #0 adder 5b (horizontally adjacent) to the first adder 6b (labeled #0). The other #1, #2, and #3 have the same connection relationship. Vertical accumulation mode ( Figure 3 (Right-middle diagram): The output (vertically) of adder 5b marked as 0 in level 0 block is routed to the first ADDER6b (#0). Simultaneously, the outputs (vertically adjacent) of adders 5b marked as 1, 2, and 3 in level 0 are routed to the 2nd, 3rd, and 4th adders 6b (marked as #1, 2, and 3), respectively.
[0042] Level 2: such as Figure 4 As shown, the outputs of the two level 1 modules (top and bottom) are routed to a set of adder & MUX modules (containing adder 7b).
[0043] Transpose implementation: The middle MUX module can swap the two sets of inputs. This allows the level 2 adder (adder7b) to accumulate inputs from both the horizontally adjacent Stage 1 module (horizontal accumulation) and the vertically adjacent Level 1 module (vertical accumulation).
[0044] This implementation's design philosophy remains applicable even when the number of subarrays increases further, such as when higher-level adders like adders 8b and 9b are needed. Since the number of high-level adders like adders 8b and 9b is small, even using two sets of adders at the highest level (one horizontal and one vertical) results in a significantly lower area cost compared to replicating a large number of adders at lower levels (e.g., levels 0, 1, 2). The core advantage of this implementation lies in its significant saving of layout area through sharing the low- and middle-level adder trees.
[0045] In summary, this embodiment avoids the significant area overhead associated with configuring two sets of adder trees (especially a large number of low-order adders) for transposition by using a shared adder tree design. Furthermore, through the flexible control of the reconfigurable routing module (MUX), only one adder tree is needed to support both horizontal and vertical accumulation, achieving efficient native transposition computation without needing to move data off-array. Therefore, this embodiment is applicable to the field of integrated circuit design (Design Service-Layout), particularly for in-memory computing (CIM) chip designs that require frequent matrix multiplication and transposition operations, such as those used in artificial intelligence, deep learning (e.g., Transformer networks), and signal processing.
[0046] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An in-memory computing architecture supporting native transposition, characterized in that, include: Multiple in-memory computing subarrays are provided; A hierarchical shared addition tree, comprising multiple adders at different levels, is used to perform hierarchical accumulation of the computation results of the subarray; A reconfigurable routing module is set at the input of each level adder and is used to select the data transmission path according to the working mode; the working mode includes horizontal accumulation and vertical accumulation.
2. The in-memory computing architecture supporting native transposition according to claim 1, characterized in that, Multiple in-memory computing subarrays are configured as two-dimensional arrays according to a preset configuration strategy.
3. The in-memory computing architecture supporting native transposition according to claim 1, characterized in that, The horizontal accumulation process specifically includes: The reconfigurable routing module routes the output of the horizontally adjacent upper-level in-memory computation subarray to the input of the current-level adder for accumulation.
4. The in-memory computing architecture supporting native transposition according to claim 1, characterized in that, The vertical accumulation process specifically includes: The reconfigurable routing module routes the outputs of the vertically adjacent upper-level in-memory computation subarrays to the inputs of the same group of current-level adders for accumulation.