Adaptive architecture for near-memory computing for shared inactive in-memory computing devices

Abstract

The invention relates to an adaptive architecture for near-memory computing of shared inactive in-memory computing devices. A hardware accelerator includes a plurality of functional circuits, a stream switch, one or more data reconfiguration units coupled to the plurality of functional circuits via the stream switch to stream data to the functional circuits of the plurality of functional circuits, and one or more in-memory computing (IMC) clusters coupled to the plurality of functional circuits via the stream switch to stream data to the functional circuits of the plurality of functional circuits. And stream-receiving data from a functional circuit of the plurality of functional circuits, one or more in-memory computing clusters coupled with the stream switch. In operation, inactive in-store computing devices of at least one subset of the one or more in-store computing clusters are accessible by at least one subset of the one or more data reconstruction units via a memory interface independent of the flow switch, the memory is configured to serve as at least part of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

Description

Technical Field

[0001] This disclosure generally relates to hardware architectures for accelerating artificial intelligence (AI) workloads, and more specifically, to the field of stream-based hardware accelerators. Background Technology

[0002] Various computer vision, speech recognition, and signal processing applications can benefit from the use of learning / inference machines, which are capable of rapidly performing hundreds, thousands, or even millions of concurrent operations. The learning / inference machines discussed in this disclosure fall under the categories of machine learning, artificial intelligence, neural networks, probabilistic inference engines, accelerators, and other related technologies. Such learning / inference machines may incorporate or utilize artificial neural networks (ANNs), for example, to process large amounts of data and achieve adaptive “learning” by fusing closely related features in the data, making broad predictions about the data, and optimizing predictions based on reliable conclusions and new fusion results. In most cases, neural networks consist of multiple “layers,” with different types of computation performed within each layer. Hardware accelerators (such as neural processing units (NPUs)) are often used to accelerate the processing of large amounts of data by artificial neural networks. Summary of the Invention

[0003] Performing inference operations on a complete artificial neural network (such as a convolutional neural network (CNN)) on a resource-constrained platform, while meeting the throughput requirements of real-time applications, is a computationally intensive task, typically requiring dedicated hardware accelerators based on highly parallel architectures. Furthermore, a major bottleneck for CNNs lies in their frequent memory accesses, which account for a very high percentage of total power consumption. To avoid excessive data transfer, solutions based on in-memory computing (IMC) devices have emerged, which can perform data processing directly within memory. However, IMC devices generally perform worse than standard memory because they require additional circuitry for computation. Therefore, there is a trade-off between cost and performance, and the advantages of an IMC architecture are only truly realized when this trade-off is met (e.g., only when using multiple large-capacity memory units). Unfortunately, the geometry of convolutional kernels varies significantly between layers, making it difficult to perfectly map CNN models to available resources in the most efficient way. This means that during runtime, some IMC devices may be inactive, leading to reduced hardware utilization (i.e., the "dark silicon" phenomenon).

[0004] In view of the above, in some embodiments of the present disclosure, the hardware accelerator includes multiple functional circuits, a stream switch, one or more data reconstruction units, and one or more in-memory computing (IMC) clusters; wherein, the one or more data reconstruction units are coupled to the multiple functional circuits via the stream switch to stream data to and from the functional circuits of the multiple functional circuits; the one or more in-memory computing clusters are coupled to the stream switch; in operation, inactive in-memory computing devices of at least a subset of the one or more in-memory computing clusters can be accessed by at least a subset of the one or more data reconstruction units via a memory interface independent of the stream switch to serve as at least a portion of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

[0005] In some embodiments, the system includes a host device and a hardware accelerator. The hardware accelerator includes multiple functional circuits, a stream switch, one or more data reconstruction units, and one or more in-memory compute (IMC) clusters; wherein the one or more data reconstruction units are coupled to the multiple functional circuits via the stream switch to stream data to and from the functional circuits; the one or more in-memory compute clusters are coupled to the stream switch; in operation, inactive in-memory compute devices of at least a subset of the one or more in-memory compute clusters are accessible by at least a subset of the one or more data reconstruction units via a memory interface independent of the stream switch to serve as at least a portion of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

[0006] In some embodiments, a method includes: streaming data between one or more data reconstruction units of a hardware accelerator and multiple functional circuits of the hardware accelerator via a streaming switch, wherein one or more in-memory computing (IMC) clusters are coupled to the streaming switch; and enabling at least one subset of the one or more data reconstruction units to access inactive in-memory computing devices of at least one subset of the one or more in-memory computing clusters via a memory interface independent of the streaming switch, for use as at least a portion of tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units. Attached Figure Description

[0007] One or more embodiments will be described below with reference to the accompanying drawings.

[0008] Figure 1 This is a functional block diagram of an electronic device or system to which the embodiments of this disclosure may be applied.

[0009] Figure 2A-2CTo illustrate a block diagram of integrating in-memory computing sharing in an exemplary hardware accelerator, some embodiments are described.

[0010] Figure 3A and Figure 3B Examples of implementations of the technology disclosed herein for different use cases are shown, which correspond to some embodiments.

[0011] Figure 4 An exemplary implementation of in-memory computation with dedicated configuration registers is shown, which is consistent with some embodiments of the technology disclosed herein. Detailed Implementation

[0012] The following description and accompanying drawings illustrate certain specific details to provide a thorough understanding of the various embodiments disclosed. However, those skilled in the art will understand that the disclosed embodiments can be implemented in various combinations, whether or not they include one or more of these specific details, or in combination with other methods, components, devices, materials, etc. In other instances, well-known structures or components (including, but not limited to, interfaces, power supplies, physical component layouts, convolution accelerators, multiply-accumulate (MAC) circuitry, control or configuration registers, bus systems, etc.) relevant to the context of this disclosure are not shown or described to avoid unnecessarily obscuring the description of the embodiments. Furthermore, the various embodiments can be methods, systems, devices, computer program products, etc.

[0013] Throughout this specification, claims, and drawings, unless the context otherwise requires, the following terms shall have the meanings applicable herein. The term “this document” refers to the specification, claims, and drawings relating to this application. The phrases “in one embodiment,” “in another embodiment,” “in various embodiments,” “in some embodiments,” “in other embodiments,” and their variations refer to one or more features, structures, functions, limitations, or characteristics of this disclosure, and are not limited to the same or different embodiments unless the context otherwise requires. The term “or” as used herein is an inclusive “or” operator, equivalent to the phrases “A or B, or both” or “A or B or C, or any combination thereof,” and lists containing additional elements are treated similarly. The term “based on” is not exclusive and allows for reliance on additional features, functions, aspects, or limitations not described, unless the context otherwise requires. Furthermore, throughout this specification, the meanings of “an,” “a,” and “the” include both singular and plural.

[0014] Performing full artificial neural network (ANN) inference (e.g., convolutional neural network (CNN)) on resource-constrained platforms, aiming to meet the throughput requirements of real-time applications, can be a computationally intensive task, typically handled by dedicated hardware accelerators based on highly parallel architectures. Furthermore, a major bottleneck of CNNs lies in their frequent memory accesses, which contribute significantly to power consumption. To avoid excessive data throughput, [the text abruptly ends here, likely due to an incomplete sentence or missing information].

[0015] Solutions based on In-Memory Capabilities (IMCs) have emerged, which perform data processing directly in memory. However, IMCs typically perform worse than standard memory because they require additional circuitry to perform computations. Therefore, there may be a trade-off between cost and performance that allows architectures using IMCs to truly function effectively (e.g., only when using multiple large memory modules). Unfortunately, the geometry of convolutional kernels can vary significantly between layers, making it difficult to map CNN models perfectly to available resources in the most efficient way. This means that some IMCs may be inactive during runtime, leading to reduced hardware utilization (dark silicon).

[0016] Shared memory typically means that concurrent access can clog bandwidth and requires complex hardware for bus arbitration and caching (e.g., managing whether cache locations are dirty or locked). The advantage of typical TCM is its speed and ease of management because it is not shared with other units. This mode is called Near-Memory Computing (NMC). However, a limitation of NMC is that TCM cannot be used when coupled units are inactive.

[0017] Embodiments of this disclosure provide an adaptive architecture for sharing inactive IMCs with other units in an NPU subsystem that typically features a TCM (e.g., used as a temporary buffer or register file). A dedicated configuration register can be used to select (e.g., at runtime) the number of devices in the IMC pool available for use as TCMs by other coupled units within the NPU. No additional hardware is required in the coupled unit, as it only needs to acquire or otherwise determine the programmed number of IMCs used as TCMs to define the buffer space. Neural network mapping tools or mechanisms can then be implemented to ensure that sharing IMCs does not result in conflicts or data corruption.

[0018] Figure 1This is a functional block diagram of an embodiment of an electronic device or system 100 to which the described embodiments can be applied. System 100 includes one or more processing cores or circuits 102. Processing core 102 may include, for example, one or more processors, state machines, microprocessors, programmable logic circuits, discrete circuits, logic gates, registers, etc., and various combinations thereof. The processing core can control the overall operation of system 100, the execution of applications by system 100 (e.g., a program that uses a convolutional neural network (CNN) to classify images), etc.

[0019] System 100 includes one or more memories 104, such as one or more volatile and / or non-volatile memories, which may store all or part of instructions and data related to, for example, control of system 100, applications executed by system 100, and operations. The one or more memories 104 may include a memory array that can be shared by one or more processes executed by system 100 during operation.

[0020] System 100 may include one or more sensors 160 (e.g., image sensors, audio sensors, accelerometers, pressure sensors, temperature sensors, etc.), one or more interfaces 170 (e.g., wireless communication interfaces, wired communication interfaces, etc.), and other circuitry 180 (which may include antennas, power supplies, one or more built-in self-test (BIST) circuits, etc.) and a main bus system 190. The main bus system 190 may include one or more data, address, power, and / or control buses coupled to the various components of system 100.

[0021] System 100 also includes one or more hardware accelerators 110 (e.g., NPUs) that accelerate the performance of one or more operations during operation, such as those related to implementing CNNs or other ANNs. As shown, hardware accelerator 110 includes one or more functional circuits (e.g., convolution accelerators, pooling layer modules, active units, direct memory access (DMA) units, etc.). Hardware accelerator 110 also includes a stream switch 155 or other interconnects and one or more IMC units 150. In some embodiments, an IMC unit includes an IMC cluster consisting of multiple IMC devices constrained by IMC control logic for accessing and operating the IMC. In some embodiments, the stream switch 155 uses a protocol shared by all components or circuits connected to it.

[0022] Embodiments of this disclosure allow the IMC to be shared with any unit capable of using TCM (hereinafter referred to as a "data reconstruction unit"). The selection of the data reconstruction unit coupled to the IMC depends on the use case and the type of operation the IMC is capable of performing. In some embodiments, for neural network applications, efficient performance synergy can be achieved when an inactive IMC (e.g., dedicated to matrix-vector multiplication (MVM)) is shared with a DMA or similar unit (e.g., a lightweight unit dedicated to tensor reconstruction that can benefit from predetermined access patterns), the basic function of which is to move data between functional circuits or units of the hardware accelerator 110. This is at least partly due to the architecture of neural networks: some layers do not require data storage at all (e.g., pointwise convolution), while others are highly reusable, so the amount of memory dedicated to computation in the IMC and the amount of memory dedicated to TCM can be adjusted, updated, or otherwise determined at runtime. The combination of IMC and NMC provides NPU designers with additional degrees of freedom, opening up new opportunities for hardware-software co-design optimization, scalability enhancement, and hardware specialization for high-end or low-end products.

[0023] Figure 2A-2C This is a block diagram illustrating the integration of IMC sharing in the example hardware accelerator 110. Figure 2A An example hardware accelerator 110 (e.g., an NPU subsystem as shown) is illustrated with or without integrated IMC sharing. As described above, the hardware accelerator 110 includes one or more functional circuits 210 (e.g., including a data reconstruction unit as shown) and one or more IMC units 150 (e.g., including an IMC convolutional unit as shown), which are coupled to a stream switch 155 via connections. Data is streamed from the stream switch 155 to the reconstruction unit 210 and stored in one of its TCMs. The reconstructed data is then streamed to two IMCs in computation mode within the IMC convolutional unit 150. The remaining IMC in the IMC convolutional unit 150 is inactive, resulting in underutilization of hardware resources.

[0024] Figure 2B An example hardware accelerator 110 with integrated IMC sharing functionality is demonstrated. Figure 2A In contrast, data reconstruction unit 220 is not equipped with any TCM, but it can access inactive IMCs to store input data via a dedicated memory interface. In various embodiments, the dedicated memory interface is independent of the stream switch 155 or the main bus system 190; in other words, the dedicated memory interface is dedicated to data communication between reconstruction unit 220 and the applicable IMCs in IMC unit 150. In this way, the present disclosure avoids the cost of an additional TCM and improves hardware utilization.

[0025] In some embodiments, the data reconstruction unit has both a dedicated TCM and access to inactive IMCs. In some embodiments, the data reconstruction unit has multiple dedicated memory interfaces that can access multiple IMC units or clusters. In some embodiments, an IMC unit or cluster can allow multiple data reconstruction units to access its IMC. These choices can be determined during the design phase based on the target performance, specifications, constraints, or market segmentation of the final product.

[0026] For example, Figure 2C Two IMC-sharing variants are demonstrated and integrated into an example hardware accelerator. In one variant (left), the hardware accelerator (e.g., an NPU subsystem) comprises two or more IMC units or clusters, and data reconstruction units can access the IMC of each IMC unit through their respective dedicated memory interfaces, while also being equipped with at least one dedicated TCM. In another variant (right), the hardware accelerator (e.g., an NPU subsystem) comprises at least two different types or entities of data reconstruction units (e.g., one of which is a pooling unit), all of which can access the IMC of the same IMC unit or cluster.

[0027] Figure 3A and 3B This section showcases various implementation examples of the currently disclosed technology applicable to different use cases. For instance, at design time, the NPU subsystem includes at least one IMC cluster consisting of eight IMC devices, two DMAs with access to external memory, and a data reconstruction unit that accesses the IMC cluster and its IMC controller. For example, the IMCs can perform multiplication and accumulation (MAC) operations. The IMC controller can coordinate the IMC devices to perform basic convolution operations, which are typically necessary during the inference phase of a CNN model. At runtime, a neural network mapping tool creates a series of processing elements to accelerate the convolutional layer 0 of the CNN model. In the two DMAs, DMA0 fetches live data from external memory to the data reconstruction unit, which then transmits a transposed version of the input tensor to the IMC controller to perform convolution;

[0028] DMA1 stores the generated output tensor back to external memory. Here, the kernel of layer 0 has already been stored in the IMC.

[0029] like Figure 3A As shown, according to use case 1, the kernel of convolutional layer 0 in a CNN model can be placed into four IMCs. The mapping tool then programs the processing chain to allocate inactive IMCs (memory modes) as buffer space for data reconstruction units. Even though the total buffer space allocated in this way is greater than the minimum space required for data reconstruction units to transpose active data, the system can still benefit from lower latency because DMA0 can prefetch new data for the next operation.

[0030] like Figure 3B As shown, according to use case 2, the kernel of convolutional layer 1 in the CNN model can hold one IMC-M0, which is programmed to operate in computation mode. The remaining 7 inactive IMCs (memory modes) are used as buffers by the data reconstruction unit.

[0031] like Figure 3B As shown in use case 3, the second-layer convolutional kernel of the CNN model can be populated with seven IMCs (M0 to M6), which are programmed to operate in computation mode. The remaining inactive IMC (memory mode) is used as a buffer by the data reconstruction unit.

[0032] like Figure 3B As shown, according to use case 4, convolutional layer 3 of the CNN model uses a 1x1 kernel, meaning no transposition is required. All IMCs are in compute mode. The mapping tool removes and disables the data reconstruction unit from the processing chain to avoid power consumption caused by switching.

[0033] like Figure 3B As shown, according to use case 5, layer 4 of the CNN model is not a convolutional layer. Only the transpose operation needs to be performed. The mapping tool programs all IMCs in memory mode for the data reconstruction unit to use as a buffer, and removes and disables the IMC controller from the processing chain to avoid switching overhead.

[0034] Figure 4 Example implementations of an IMC with dedicated configuration registers according to various embodiments of the present disclosure are illustrated. As shown, the dedicated configuration registers can be used to change the operating mode of the IMC. The dedicated configuration registers can switch the IMC between memory mode and compute mode. When the IMC is in compute mode, it is used by the associated IMC controller to perform certain operations, therefore... Figure 4 The multiplexer shown is programmed with a dedicated configuration register to select the IMC controller. When the IMC is in memory mode, it can be idle or used as a buffer by other data reconstruction units. The dedicated configuration register is used to program the multiplexer to select from applicable data reconstruction units (e.g., IP0, IP1, etc.). Neural network mapping tools can make an idle IMC in memory mode usable as a TCM by programming the dedicated configuration register accordingly.

[0035] In some embodiments, the IMC device may be based on SP / DP-RAM technology and have multiple write / read ports (e.g., 1W2R, 2W1R, etc.). Typically, due to technological limitations or significant area cost trade-offs, only a fixed set of IMC geometries (memory depth multiplied by memory width) can be used. If a legacy update is required, the design can be modified to change the TCM interface in case of geometric mismatches. To avoid redesigning existing units or modules and to improve scalability and modularity, adapter-level implementations can be adopted.

[0036] For example, a module (e.g., between a dedicated memory interface of a data reconstruction unit and the corresponding IMC unit to be coupled) can adjust for memory type mismatches (SPRAM vs. DPRAM IMC in previous designs) and memory width mismatches (e.g., 128 bits vs. 256 bits). This module adjusts the SP signal in the input to write data to the IMC and adjusts the DP signal in the output to read data from the IMC. Internal data width conversion can also be performed to meet unit requirements. Other embodiments can have any combination of type or width mismatches (e.g., DP2SP, DP2SP with width mismatch, SP2SP with width mismatch, etc.).

[0037] In some embodiments, the hardware accelerator includes a plurality of functional circuits; a stream switch; one or more data reconstruction units coupled to the plurality of functional circuits via the stream switch to stream data to and from the functional circuits in the plurality of functional circuits; and one or more in-memory computing (IMC) clusters coupled to the stream switch, wherein in operation, inactive IMC devices of at least a subset of the one or more IMC clusters are accessible to at least a subset of the one or more data reconstruction units via a memory interface independent of the stream switch to serve as at least a portion of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

[0038] In some embodiments, the memory interface is independent of the system bus coupled to the hardware accelerator.

[0039] In some embodiments, a dedicated configuration register is used to select a number of inactive IMC devices as at least a portion of the TCM at runtime.

[0040] In some embodiments, one or more data reconstruction units are configured to move data between functional circuits. In some embodiments, one or more data reconstruction units include a direct memory access (DMA) unit.

[0041] In some embodiments, a subset of one or more data reconstruction units includes a single data reconstruction unit that can access inactive IMC devices of multiple IMC clusters that act as at least a portion of a TCM dedicated to that single data reconstruction unit.

[0042] In some embodiments, a subset of one or more data reconstruction units includes more than one data reconstruction unit, each of which can access inactive IMC devices in the same IMC cluster, the inactive IMC devices serving as at least a portion of the corresponding TCM dedicated to each of the more than one data reconstruction unit.

[0043] In some embodiments, the hardware accelerator is a neural processing unit (NPU).

[0044] In some embodiments, inactive IMC devices of one or more IMC clusters operate in memory mode, and active IMC devices of one or more IMC clusters operate in compute mode.

[0045] In some embodiments, the system includes a host device and a hardware accelerator. The hardware accelerator includes: a plurality of functional circuits; a stream switch; one or more data reconstruction units coupled to the plurality of functional circuits via the stream switch for transmitting data streams to or from the functional circuits; and one or more in-memory computing (IMC) clusters coupled to the stream switch. In operation, inactive IMC devices of at least a subset of the one or more IMC clusters can be accessed by at least a subset of the one or more data reconstruction units via a memory interface independent of the stream switch, to serve as at least a portion of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

[0046] In some embodiments, the memory interface is independent of the system bus coupled to the hardware accelerator.

[0047] In some embodiments, a dedicated configuration register is used to select a number of inactive IMC devices as at least a portion of the TCM at runtime.

[0048] In some embodiments, one or more data reconstruction units are configured to move data between functional circuits. In some embodiments, one or more data reconstruction units include a direct memory access (DMA) unit.

[0049] In some embodiments, a subset of one or more data reconstruction units includes a single data reconstruction unit that can access inactive IMC devices of multiple IMC clusters that act as at least a portion of a TCM dedicated to that single data reconstruction unit.

[0050] In some embodiments, a subset of one or more data reconstruction units includes more than one data reconstruction unit, each of which can access inactive IMC devices in the same IMC cluster, the inactive IMC devices serving as at least a portion of the corresponding TCM dedicated to each of the more than one data reconstruction unit.

[0051] In some embodiments, the hardware accelerator includes a neural processing unit (NPU).

[0052] In some embodiments, inactive IMC devices of one or more IMC clusters operate in memory mode, and active IMC devices of one or more IMC clusters operate in compute mode.

[0053] In some embodiments, a method includes moving data between one or more data reconstruction units of a hardware accelerator and multiple functional circuits of the hardware accelerator via a stream switch, wherein one or more in-memory computing (IMC) clusters are coupled to the stream switch; and providing access to inactive IMC devices of at least one subset of the one or more IMC clusters for use as at least a portion of tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units via a memory interface independent of the stream switch.

[0054] In some embodiments, the method includes using a dedicated configuration register to select a number of inactive IMC devices as at least a portion of the TCM.

[0055] Some embodiments may take the form of or include a computer program product. For example, according to one embodiment, a computer-readable medium is provided that includes a computer program or other computer-executable instructions adapted to perform one or more of the methods or functions described above. The medium may be a physical storage medium, such as a read-only memory (ROM) chip, or a disk, such as a digital versatile optical disc (DVD-ROM), optical disc (CD-ROM), hard disk, memory, network, or portable media article, readable by a suitable drive or via a suitable connection, including one or more barcodes or other related codes encoded on one or more such computer-readable media and readable by a suitable reading device.

[0056] Furthermore, in some embodiments, some or all of the methods and / or functions may be implemented or provided in other ways, such as at least in part by firmware and / or hardware, including but not limited to one or more application-specific integrated circuits (ASICs), digital signal processors, discrete circuits, logic gates, standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and devices employing RFID technology and various combinations thereof.

[0057] The various embodiments described above can be combined to provide further embodiments. These and other modifications can be made to the embodiments based on the detailed description above. Generally, the terminology used in the following claims should not be construed as limiting the claims to the specific embodiments disclosed in the specification and claims, but should be understood to include all possible embodiments and the full scope of the equivalents conferred by these claims. Therefore, the claims are not limited by this disclosure.

Claims

1. A hardware accelerator, comprising: Multiple functional circuits; Flow switch; One or more data reconstruction units are coupled to the plurality of functional circuits via the stream switch to stream data to and stream data to the functional circuits in the plurality of functional circuits. as well as One or more in-memory compute IMC clusters are coupled to the stream switch; In operation, inactive in-memory computing devices of at least one subset of the one or more in-memory computing clusters can be accessed by at least one subset of the one or more data reconstruction units via a memory interface independent of the stream switch, to serve as at least a portion of a tightly coupled memory TCM dedicated to at least one of the one or more data reconstruction units.

2. The hardware accelerator of claim 1, wherein the memory interface is independent of the system bus coupled to the hardware accelerator.

3. The hardware accelerator of claim 1, wherein a dedicated configuration register is used at runtime to select a number of the inactive in-memory computing devices to be used as at least a portion of the TCM.

4. The hardware accelerator of claim 1, wherein the one or more data reconstruction units are configured to move data between the functional circuits.

5. The hardware accelerator of claim 4, wherein the one or more data reconstruction units include a direct memory access (DMA) unit.

6. The hardware accelerator of claim 1, wherein the subset of the one or more data reconstruction units includes a single data reconstruction unit that accesses inactive IMC devices of more than one IMC cluster, the inactive IMC devices serving as at least a portion of the TCM dedicated to the single data reconstruction unit.

7. The hardware accelerator of claim 1, wherein the subset of the one or more data reconstruction units comprises more than one data reconstruction unit, each data reconstruction unit accessing inactive IMC devices of the same IMC cluster, the inactive IMC devices serving as at least a portion of each TCM of each of the more than one data reconstruction units.

8. The hardware accelerator according to claim 1, wherein the hardware accelerator is a neural processing unit (NPU).

9. The hardware accelerator of claim 1, wherein the inactive IMC devices of the one or more IMC clusters operate in memory mode, and the active IMC devices of the one or more IMC clusters operate in compute mode.

10. A system comprising: Host equipment; as well as Hardware accelerator, the hardware accelerator comprising: Multiple functional circuits; Flow switch; One or more data reconstruction units are coupled to the plurality of functional circuits via the stream switch to stream data to and stream data to the functional circuits in the plurality of functional circuits; and One or more in-memory compute (IMC) clusters are coupled to the flow switch; wherein, in operation, inactive IMC devices of at least a subset of the one or more IMC clusters are accessible by at least a subset of the one or more data reconstruction units via a memory interface independent of the flow switch, to serve as at least a portion of a tightly coupled memory (TCM) dedicated to at least one of the one or more data reconstruction units.

11. The system of claim 10, wherein the memory interface is independent of the system bus coupled to the hardware accelerator.

12. The system of claim 10, wherein a dedicated configuration register is used at runtime to select a number of the inactive IMC devices to be used as at least a portion of the TCM.

13. The system of claim 10, wherein the one or more data reconstruction units are configured to move data between the functional circuits.

14. The system of claim 13, wherein the one or more data reconstruction units include a direct memory access (DMA) unit.

15. The system of claim 10, wherein the subset of the one or more data reconstruction units includes a single data reconstruction unit that accesses inactive IMC devices of more than one IMC cluster, the inactive IMC devices serving as at least a portion of the TCM dedicated to the single data reconstruction unit.

16. The system of claim 10, wherein the subset of the one or more data reconstruction units comprises more than one data reconstruction unit, each data reconstruction unit accessing inactive IMC devices of the same IMC cluster, the inactive IMC devices serving as at least a portion of the respective TCMs of each of the more than one data reconstruction unit.

17. The system of claim 10, wherein the hardware accelerator includes a neural processing unit (NPU).

18. The system of claim 10, wherein the inactive IMC devices of the one or more IMC clusters operate in memory mode, and the active IMC devices of the one or more IMC clusters operate in compute mode.

19. A method comprising: Data is streamed between one or more data reconstruction units of a hardware accelerator and multiple functional circuits of the hardware accelerator via a stream switch, wherein one or more in-memory computing (IMC) clusters are coupled to the stream switch. as well as Access to inactive IMC computing devices of at least one subset of the one or more IMC clusters is provided to at least one subset of the one or more data reconstruction units via a memory interface independent of the stream switch, for use as at least a portion of a tightly coupled memory TCM dedicated to at least one of the one or more data reconstruction units.

20. The method of claim 19, further comprising using a dedicated configuration register to select a number of the inactive IMC devices to be used as at least a portion of the TCM.

Images