Storage Device Channel Access Optimization for ML Workloads

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Storage devices experience a decrease in speed for reading and writing user data when used to store both machine learning data and user data, due to the integration of machine learning functions.

Innovation Solution

A storage device with a memory controller and multiple nonvolatile memory channels, where the controller determines the least frequently accessed channel for accessing weight data, allowing for efficient processing of machine learning requests without compromising user data access speed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the storage device stores both machine learning data and user data, then the machine learning function is supported, but the read/write speed of user data decreases

Engineering Contradiction:

Improvemachine learning function supportVSAvoiduser data read/write speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The storage device is divided into multiple channels (first channel and second channel), each capable of independent data access. The memory controller segments the data access paths to allow simultaneous operations on different data types without mutual interference, thus maintaining user data read/write speed while supporting machine learning functions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The memory controller dynamically determines which channel to access based on real-time access frequency analysis. When machine learning operations require weight data, the controller identifies the least frequently accessed channel and routes the access through that channel, adapting the access pattern dynamically to prevent performance degradation.

Inventive Principle:

Principle #15Dynamics

2Productivity

If the storage device accesses weight data frequently for machine learning operations, then the machine learning inference is improved, but the user data access speed is compromised

Engineering Contradiction:

Improvemachine learning inference performanceVSAvoiduser data access speed

Core Design Contradiction:

ProductivityVSSpeed

Solution Approach 1:

The memory controller incorporates a feedback mechanism that monitors and analyzes access frequencies of different channels. Based on this feedback, the controller adjusts its data access routing decisions, selecting channels with lower access frequencies for weight data retrieval to minimize impact on overall system performance and maintain user data access speed.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system dynamically adjusts access patterns by determining the least frequently accessed channel in real-time. This dynamic adaptation allows the system to optimize machine learning operations without compromising user data access performance, as the routing decision changes based on current system state.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11526287B2Storage device and operating method of storage device

Publication Date: 2022.12.13 SAMSUNG ELECTRONICS CO LTD

AI summary

A storage device is provided including a memory controller having a neural processing unit (NPU); a first nonvolatile memory (NVM) connected to the memory controller through a first channel; and a second NVM connected to the memory controller through a second channel. The first NVM stores first weight data for the NPU and the second stores second weight data for the NPU. The memory controller is configured to determine one of the first and second channels that is less frequently accessed upon receiving an inference request from the neural processor, and access a corresponding one of the first weight data and the second weight data using the determined one channel.