Neural Network Memory Planning Across Primary and Secondary Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory management in artificial neural networks, particularly on embedded hardware, is inefficient due to increased memory requirements caused by skip connections, leading to reduced available memory for computation.
Innovation Solution
A memory plan is determined using a list of memory areas with specified lifetimes, allowing efficient assignment to primary and secondary memories to minimize data transfers and optimize memory usage, utilizing a satisfiability modulo theories solver to minimize memory area transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory areas are allocated for skip connections in neural networks, then the network can perform operations with skip connections, but the memory requirements increase and reduce available memory for computation
Solution Approach 1:
The patent segments memory management by introducing a list of memory areas with specified lifetimes and dividing memory allocation into primary and secondary memory. This allows the system to manage memory more efficiently by tracking which memory areas are needed for skip connections versus computation, thereby reducing overall memory requirements while maintaining neural network operation capability.
Solution Approach 2:
The patent performs preliminary memory planning by creating a list of memory areas with specified lifetimes before executing operations. This preliminary action allows the system to determine in advance which memory areas are needed for skip connections and which are needed for computation, enabling optimized memory allocation that reduces overall memory requirements.
2Quantity of substance
If multiple memories are used simultaneously, then memory overhead is reduced and available memory for computation increases, but the complexity of memory management increases
Solution Approach 1:
The patent implements self-service memory management through automated memory planning. The system automatically determines which memory areas should be in primary memory versus secondary memory by analyzing the list of memory areas with specified lifetimes. This automation handles the complexity of managing multiple memories simultaneously, reducing memory overhead while maintaining computational efficiency.
Solution Approach 2:
The patent changes the parameters of memory management by introducing specified lifetimes for memory areas and using satisfiability modulo theories solvers to optimize memory allocation. This parameter-based approach allows the system to automatically adjust memory allocation strategies, managing multiple memories efficiently without increasing operational complexity.
3Productivity
If memory areas are transferred between primary and secondary memory, then memory usage is optimized, but the number of data transfers increases
Solution Approach 1:
The patent applies partial action by transferring only the necessary memory areas between primary and secondary memory based on their lifetimes and operational requirements. The satisfiability modulo theories solvers determine the minimum necessary transfers, avoiding excessive data movement while still achieving optimized memory usage efficiency.
Solution Approach 2:
The patent performs preliminary determination of memory area assignments before actual data transfer occurs. By analyzing the list of memory areas with specified lifetimes in advance, the system pre-determines which areas need to be transferred and when, reducing unnecessary transfers and optimizing both memory usage efficiency and data transfer time.
Data Source
AI summary
An apparatus and computer-implemented method for determining a memory plan for executing operations, in particular of an artificial neural network. A list of memory areas required for executing the operations is created, wherein, depending on the list, it is determined for the operations which memory areas must be present in a first memory for executing the particular operation and which memory areas may be present in a second memory during the execution of the particular operation, wherein the memory plan is determined depending on whether a memory area must be present in the main memory for execution or may be present in the secondary memory.