On-Device Neural Network Training Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training neural network models on-device face challenges due to limited computing resources and data, making it difficult to personalize services effectively while ensuring user privacy and reducing service costs.
Innovation Solution
An electronic apparatus that optimizes memory usage by dividing learning steps into forward propagation, gradient calculation, and derivative calculation steps, determining an execution order, and allocating memory based on tensor usage patterns to share resources and minimize memory allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network model training is performed on a server using large amounts of data, then model accuracy and personalization are improved, but user privacy security deteriorates and service costs increase
Solution Approach 1:
The patent extracts the training process from the server environment and relocates it to the user's terminal device. By extracting only the necessary model structure parameters from the server and performing training locally using user's own data, the system achieves personalization without transmitting sensitive personal data to the server, thus resolving the contradiction between model accuracy and user privacy security
Solution Approach 2:
The patent enables the user's terminal device to perform self-training of the neural network model using its own computational resources and locally stored data. This self-service approach allows the device to personalize the model independently without relying on server resources or transmitting data externally, thereby improving model accuracy while maintaining privacy security
2Measurement precision
If neural network model training is performed on a server, then model personalization is improved, but service costs deteriorate
Solution Approach 1:
The patent extracts the training execution from server infrastructure and relocates it to user terminal devices. By extracting only the model architecture definitions from the server and performing actual training computations locally, the system achieves model personalization without incurring additional server computing costs, thus resolving the contradiction between personalization and service costs
Solution Approach 2:
The patent enables each user's terminal device to perform self-training using its own computational resources. This eliminates the need for expensive server-based personalized training services, allowing users to achieve model personalization independently and reducing overall service costs while maintaining personalization quality
3Productivity
If computing resources are increased for on-device training, then training efficiency is improved, but device resource limitations deteriorate
Solution Approach 1:
The patent segments the training process into distinct phases: model structure parameter acquisition from server, local data preparation, and distributed training execution across multiple layers. By segmenting the training workflow and utilizing available device resources in a structured manner, the system improves training efficiency without overwhelming device resources, thus resolving the contradiction between training efficiency and device resource limitations
Data Source
AI summary
An electronic apparatus may include a memory configured to store data related to a neural network model and at least one processor configured to divide a learning step performed through a plurality of layers of the neural network model into a plurality of steps including a forward propagation step, a gradient calculation step, and a derivative calculation step, and determine an execution order of the plurality of steps, obtain first information regarding in which step of a plurality of steps according to the determined execution order a plurality of sensors used in the plurality of layers are used, based on the determined execution order, integrate the determined execution order based on the first information and second information regarding whether tensors used in neighboring layers from among the plurality of layers are able to be shared, allocate the data to the plurality of tensors by minimizing a region of the memory for allocating data corresponding to the plurality of tensors, based on the integrated execution order, and train the neural network model according to the integrated execution order using the plurality of tensors and the data allocated to the plurality of tensors. Various other embodiments are possible to be implemented.