Shared Non-Volatile Memory for Machine Learning Data Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems face inefficiencies in machine-learning operations due to limitations in processing resources and memory access speeds, particularly when using non-volatile memory, which hinders direct processing and concurrent access by multiple resources.
Innovation Solution
Implementing a shared non-volatile memory space for both general-purpose processing resources and graphics processing resources, allowing direct and concurrent access to machine-learning data structures, and utilizing storage-class memory with high read/write speeds to facilitate efficient execution of machine-learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If non-volatile memory is used for storing machine-learning data structures, then data persistence and reduced power consumption are achieved, but memory access speeds and direct processing capability deteriorate
Solution Approach 1:
The system segments memory into volatile and non-volatile portions, with the non-volatile memory further divided into a file system portion and a machine-learning data structure portion. This segmentation allows optimized access paths where frequently accessed machine-learning data structures can be served from faster volatile memory or cache, while less frequently accessed data resides in non-volatile memory, thus balancing speed and persistence requirements.
Solution Approach 2:
The patent introduces a memory controller and cache memory as intermediaries between the non-volatile memory and processing resources. The memory controller manages data transfer and access requests, while the cache memory acts as a buffer that stores frequently accessed machine-learning data structures, thereby mediating between the slow non-volatile memory and the fast processing units to improve effective access speed without sacrificing persistence.
2Reliability
If separate memory spaces are used for general-purpose processing and graphics processing, then resource isolation and stability are improved, but memory utilization efficiency and switching times deteriorate
Solution Approach 1:
The patent merges the memory spaces for general-purpose processing resources and graphics processing resources into a unified non-volatile memory structure. Both types of processing resources can access the same machine-learning data structures stored in the non-volatile memory, eliminating the need to copy or transfer data between separate memory spaces. This unified approach reduces switching times and improves memory utilization efficiency while maintaining reliable access through the memory controller's arbitration mechanisms.
3Speed
If traditional volatile memory is used for machine-learning operations, then processing speed is improved, but power consumption and data persistence deteriorate
Solution Approach 1:
The system performs preliminary actions by pre-loading machine-learning data structures into the non-volatile memory before processing operations begin. The data is organized and staged in the non-volatile memory in advance, allowing processing resources to access it directly without requiring continuous power to maintain volatile memory states. This preliminary preparation enables fast access patterns similar to volatile memory while maintaining the power efficiency and persistence of non-volatile memory.
Data Source
AI summary
Examples include a system comprising a non-volatile memory to store a machine-learning data structure. Examples access the machine-learning data structure with a first processing resource, and examples access the machine-learning data structure with a second processing resource, which includes at least one graphics processing core, such that the machine-learning data structure is a shared memory space of the first processing resource and the second processing resource.