Cache-Coherent In-Memory Computing Interface for Latency Bottlenecks

The patent segments the memory system into multiple independent function-in-memory circuits (FIM0-FIM7) that can operate in parallel on different data segments. Each FIM circuit processes specific portions of data independently, allowing simultaneous computations across multiple units. This segmentation enables the system to achieve high throughput by processing multiple data elements concurrently, effectively hiding latency through parallel execution.

The patent implements preliminary action by pre-fetching operands into on-chip buffers before computation begins. The system prepares data in advance by loading operands from external memory into local buffer memory within each FIM circuit, so that when computation is initiated, the data is already available. This eliminates wait time during operation and ensures continuous processing, thereby reducing the impact of latency on overall computational efficiency.

The patent implements universality by designing function-in-memory circuits that can perform multiple operations including arithmetic computations, logical operations, and data transformation functions. Each FIM circuit is equipped with configurable functional units that can be programmed to execute different operations on the same hardware infrastructure. This multi-functionality allows a single integrated memory-processor unit to replace multiple separate components, increasing bandwidth while managing complexity through consolidation rather than proliferation of dedicated circuits.

The patent introduces intermediary control logic and interface circuits that mediate between the external memory controller and the internal function-in-memory circuits. These intermediary components handle protocol conversion, data routing, and coordination between multiple FIM units, simplifying the overall system complexity by providing a standardized interface layer. The intermediary structures manage the increased complexity of integrated operations without exposing it to external systems, thereby maintaining manageable device complexity while achieving high bandwidth.

The patent implements feedback mechanisms where each function-in-memory circuit reports its operation status, data modification states, and completion signals back to a central coherence controller. This feedback enables the system to track which memory locations have been modified by which FIM circuit, allowing the coherence controller to maintain accurate cache state information. The feedback loop ensures that parallel operations do not compromise cache coherence by providing real-time visibility into data state changes across all FIM units.

The patent introduces a cache coherence controller as an intermediary component that manages coherence protocols across multiple parallel FIM circuits. This intermediary controller coordinates memory access requests, handles cache line invalidation, and ensures that all FIM circuits operate with consistent view of shared data. By placing a dedicated coherence management layer between the parallel FIM units and the memory subsystem, the system maintains cache coherence without sacrificing parallel processing throughput.