Near Data Processor Resource Allocation for Bandwidth Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face challenges with memory capacity scarcity and bandwidth constraints, particularly in deep learning-based systems where embedding operations are primarily performed by the host device.
Innovation Solution
The proposed solution involves a computing system that includes a memory system with near data processors (NDPs) configured to perform operations on raw data, and a host device that determines resources for further operations based on bandwidth and data size ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If embedding operations are performed primarily by the host device, then processing capability is maintained, but bandwidth constraints and memory capacity scarcity worsen
Solution Approach 1:
The patent segments the data processing function by introducing a near data processor (NDP) that operates independently from the host device. The NDP is configured to perform embedding operations on raw data locally within the memory system, separating the embedding function from the host device while maintaining processing capability. This segmentation allows the host device to focus on higher-level tasks while the NDP handles data preparation operations.
Solution Approach 2:
The patent introduces an intermediary component - the near data processor (NDP) - that acts as a mediator between the memory system and the host device. The NDP receives raw data from the memory system, performs embedding operations, and provides processed data to the host device. This intermediary approach enables distributed processing and reduces the bandwidth burden on the host-device memory interface.
2Adaptability or versatility
If embedding operations are performed by the host device, then processing flexibility is maintained, but bandwidth constraints increase
Solution Approach 1:
The patent segments the data processing workload by assigning embedding operations to a dedicated near data processor (NDP) within the memory system. This segmentation allows the host device to maintain flexibility in executing high-level applications while the NDP handles specific data preparation tasks, thereby distributing the bandwidth requirements and reducing congestion on the host-device interface.
Solution Approach 2:
The patent adds a new dimension to the system architecture by introducing a near data processor that operates in parallel with the host device. This dimensional addition creates a multi-level processing hierarchy where the NDP handles embedding operations locally, and the host device handles application-level operations, enabling simultaneous operations that improve overall system throughput and bandwidth utilization.
3Speed
If near data processing is implemented, then bandwidth constraints are alleviated, but device complexity increases
Solution Approach 1:
The patent extracts the embedding operation functionality from the host device and places it into a dedicated near data processor (NDP) within the memory system. This extraction simplifies the host device's workload while concentrating the embedding processing capability in a specialized component. The NDP is designed with specific functionality for embedding operations, which reduces the overall system complexity by creating clear functional boundaries and dedicated processing units.
Data Source
AI summary
The present disclosure relates to a computing system. The computing system may include a memory system including a plurality of memory devices configured to store raw data and a near data processor (NDP) configured to receive the raw data by a first bandwidth from the plurality of memory devices and generate intermediate data by performing a first operation on the raw data, and a host device coupled to the memory system by a second bandwidth and determining a resource to perform a second operation on the intermediate data based on a bandwidth ratio and a data size ratio.