Smart Memory Handling for Machine Learning Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for processing data in machine learning networks, particularly convolutional neural networks (CNNs), are inefficient due to limitations in memory usage at individual nodes and challenges in handling three-dimensional data, which hampers the growth of machine learning applications like autonomous machines.
Innovation Solution
The implementation of a smart memory handling and data management system that allows for efficient GPU memory handling by joining multiple large data tables and converting two-dimensional data into three-dimensional data, enabling dense memory layouts and efficient re-layout of data for parallel processing across multiple autonomous machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data processing is performed at a single node, then memory access is simplified, but processing capacity is limited and bandwidth requirements increase
Solution Approach 1:
The patent divides the data processing task across multiple nodes in a distributed system. Each node processes a portion of the data independently, and the results are aggregated. This segmentation enables parallel processing, increasing overall productivity while distributing the memory management complexity across multiple nodes rather than concentrating it at a single node.
Solution Approach 2:
The patent transforms two-dimensional data structures into three-dimensional data structures by adding a depth dimension. This dimensional transformation enables more efficient data layout and access patterns in the context of distributed processing, allowing for better memory utilization and reduced bandwidth requirements across the network.
2Productivity
If two-dimensional data is processed directly, then memory layout is simple, but processing efficiency for CNN operations is reduced
Solution Approach 1:
The patent converts two-dimensional data into three-dimensional data by introducing a depth dimension that corresponds to the channel dimension in convolutional neural networks. This transformation aligns the data layout with the computational requirements of CNN operations, enabling more efficient processing while the system manages the increased complexity through automated memory handling mechanisms.
Solution Approach 2:
The patent changes the parameter representation of data from two-dimensional to three-dimensional format. This parameter transformation optimizes how data is stored and accessed in memory, improving processing efficiency for CNN operations by enabling better cache utilization and memory access patterns, while the complexity is managed through systematic transformation protocols.
3Productivity
If multiple large data tables are joined, then data processing capability increases, but memory requirements at individual nodes increase
Solution Approach 1:
The patent segments large data tables across multiple nodes in the distributed system. Each node stores and processes only a portion of the data, reducing the memory requirements at individual nodes while maintaining the ability to process the complete dataset through coordinated operations across the network.
Solution Approach 2:
The patent introduces an intermediary memory management system that coordinates data access and processing across multiple nodes. This intermediary layer handles the complexity of joining data tables distributed across nodes, enabling data processing capability to increase without requiring each individual node to hold the complete dataset in memory.
Data Source
AI summary
A mechanism is described for facilitating memory handling and data management in machine learning at autonomous machines. A method of embodiments, as described herein, includes detecting multiple tables associated with multiple neural networks at multiple autonomous machines, where each of the multiple tables include an index. The method may further include combining the multiple tables and multiple indexes associated with the multiple tables into a single table and a single index, respectively, where the single table is communicated to the multiple autonomous machines to allow simultaneous processing of one or more portions of the single table using one or more memory devices and one or more processors of one or more of the multiple autonomous machines.