ML Model Fusion Graph for Low-DRAM Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge is to minimize power consumption in mobile devices by reducing dynamic random access memory (DRAM) accesses and increasing cache accesses in accelerated processing units (APUs) used in consumer devices that are always on and connected.
Innovation Solution
A method and device that convert a portion of a machine learning model into a directed acyclic graph, determining a shortest path with minimal DRAM accesses by analyzing the total amount of DRAM accesses between fusions, and optimizing data storage to use cache instead of DRAM where possible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If external memory (DRAM) access is increased to meet operation requirements, then computational functionality is improved, but power consumption increases
Solution Approach 1:
The machine learning model is segmented into multiple fusions, and the computation graph is divided into different execution paths. By segmenting the computational tasks, the system can selectively execute only necessary operations and avoid unnecessary DRAM accesses, thus reducing power consumption while maintaining computational functionality.
Solution Approach 2:
The system performs preliminary analysis to convert the computation graph into a directed acyclic graph (DAG) and pre-calculates the shortest path before execution. This preliminary action identifies the optimal execution path that minimizes DRAM accesses, allowing the system to prepare the most energy-efficient execution plan in advance.
2Use of energy by moving object
If cache access is increased to reduce DRAM access, then power consumption is reduced, but memory access efficiency may deteriorate
Solution Approach 1:
The patent introduces an intermediary mechanism (the shortest path algorithm on DAG) that mediates between cache and DRAM access. This intermediary optimizes the data flow by identifying which data should be kept in cache and which requires DRAM access, thereby maintaining memory access efficiency while reducing overall power consumption through intelligent cache utilization.
3Measurement precision
If the machine learning model is fully executed with all fusions, then computational accuracy is improved, but the complexity of memory access optimization increases
Solution Approach 1:
The system employs feedback mechanisms by continuously analyzing the computation graph, converting it to DAG format, and using shortest path algorithms to determine optimal execution paths. This feedback loop ensures that the system maintains computational accuracy by executing necessary operations while simultaneously optimizing memory access patterns to reduce complexity and power consumption.
4Use of energy by moving object
If DRAM access is minimized by using cache, then power consumption is reduced, but the amount of data that can be processed is limited
Solution Approach 1:
The system dynamically adapts its memory access strategy by converting the computation graph to a DAG and calculating the shortest path based on actual execution requirements. This dynamic approach allows the system to flexibly utilize cache for frequently accessed data while automatically transitioning to DRAM access when larger data volumes are needed, thereby maintaining data processing capacity while minimizing power consumption.
Data Source
AI summary
A method for optimizing memory access based on a machine learning model is provided. The method includes converting a portion of the machine learning model corresponding to an operation requirement of a hardware into a directed acyclic graph, wherein the portion of the machine learning model comprises multiple fusions, and the directed acyclic graph comprises a plurality of vertexes and a plurality of directed edges, wherein the edge ei,j is the edge from the vertex Vi to vertex Vj, and the value of the edge ei,j is set to indicate a total amount of DRAM accesses from an input of fusioni to an output of fusionj-1, and, where i, j are positive integers and j is larger than i; and determining a shortest path, wherein the shortest path represents the path from a starting vertex to a destination vertex with a smallest total amount of DRAM accesses.