Memory Sub-System Virtualized Bus In-Memory ML Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory sub-systems experience latency issues due to the need for external buses and interfaces to transmit data, machine learning models, and intermediate data between memory components and machine learning processors, which hampers the performance of machine learning operations.
Innovation Solution
Implementing internal logic within memory components to perform machine learning operations, eliminating the need for external machine learning processors and reducing data transmission latency by using digital logic or resistor arrays integrated within the memory components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If external buses and interfaces are used to transmit data between memory components and machine learning processors, then connectivity and data transmission capability are improved, but latency and transmission time increase
Solution Approach 1:
The patent merges the machine learning processor functionality directly into the memory component by integrating digital logic circuits that can perform machine learning operations internally. This eliminates the need for separate external machine learning processors and reduces data transmission latency by performing computations within the memory subsystem itself.
Solution Approach 2:
The patent introduces internal logic and digital logic circuits as intermediaries within the memory component to perform machine learning operations. These internal logic elements act as mediators that process data locally without requiring external processing devices, thereby reducing transmission time and latency.
2Power
If separate external machine learning processors are used, then processing capability is improved, but device complexity and data transmission requirements increase
Solution Approach 1:
The patent combines the machine learning processing functionality with the memory component by integrating digital logic circuits directly into the memory device. This merging reduces system architecture complexity by eliminating separate external processors and simplifying the overall system structure while maintaining processing capability.
Solution Approach 2:
The memory component is designed to perform multiple functions: it serves as both data storage and machine learning processing unit. The integrated digital logic enables the memory device to execute machine learning operations independently, making it a multi-functional component that reduces the need for separate dedicated processing hardware.
3Ease of operation
If data is transmitted through external interfaces for machine learning operations, then data access flexibility is improved, but transmission time and operational latency increase
Solution Approach 1:
The memory component performs self-service by executing machine learning operations internally using integrated digital logic circuits. This eliminates the need to wait for external processors to become available or for data to be transmitted through external interfaces, thereby reducing operational latency while maintaining data access flexibility.
Solution Approach 2:
The patent enables preliminary processing actions to be performed within the memory component itself before data needs to be accessed or transmitted externally. The internal digital logic can prepare and process data locally in advance, reducing the time required for subsequent external transmissions and operations.
Data Source
AI summary
A memory component includes a memory region to store a machine learning model and input data and another memory region to store host data from a host system. A controller can be coupled to the memory component and can include in-memory logic to perform a machine learning operation by applying the machine learning model to the input data to generate an output data. A bus can receive additional data from the host system and a decoder can receive the additional data from the bus and can transmit the additional data to the other memory region or the in-memory logic of the controller based on a characteristic of the additional data.