Adaptive Deep Learning Model Inference on Mobile Terminals

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep learning model inference on mobile devices faces challenges in balancing computation resources with inference speed and accuracy, particularly when resources are limited, and there is a risk of personal information leakage when performed on cloud servers.

Innovation Solution

A dynamic resource adaptive deep learning model inference method that determines computing resource information on a mobile terminal, transforms a basic deep learning model into an adaptive model based on allocable resources, and performs inference using the adaptive model, reducing the number of layers and minimizing resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a basic deep learning model with many layers is used for inference on a mobile terminal, then inference accuracy is improved, but computing resource consumption increases beyond available resources

Engineering Contradiction:

Improveinference accuracyVSAvoidcomputing resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies dynamics by making the deep learning model adaptive to changing resource conditions. The processor dynamically determines available computing resources and transforms the basic model into an adaptive model with modified architecture (pruned layers, reduced nodes, adjusted channels) that matches current resource availability. This allows the system to optimize between accuracy and resource usage in real-time based on actual device state.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes structural parameters of the deep learning model including number of layers, nodes per layer, channels in convolutional layers, and filter counts. By adjusting these parameters based on available computing resources, the system transforms a fixed basic model into a flexible adaptive model that can operate efficiently within resource constraints while maintaining acceptable inference accuracy.

Inventive Principle:

Principle #35Parameter changes

2Use of energy by moving object

If the number of layers in a deep learning model is reduced to decrease computation, then resource usage is improved, but inference accuracy deteriorates

Engineering Contradiction:

Improvecomputation resource usageVSAvoidinference accuracy

Core Design Contradiction:

Use of energy by moving objectVSMeasurement precision

Solution Approach 1:

The patent systematically adjusts model parameters including reducing layer counts, pruning nodes, decreasing channel dimensions, and reducing filter numbers. These parameter changes are performed adaptively based on available computing resources, allowing the model to achieve lower resource consumption while maintaining the highest possible accuracy given the resource constraints.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The model transformation is dynamic rather than static. The processor evaluates current resource availability and transforms the basic model accordingly, creating an adaptive model whose architecture is optimized for the specific resource conditions. This dynamic adaptation allows the system to achieve better accuracy-resource trade-offs compared to fixed lightweight models.

Inventive Principle:

Principle #15Dynamics

3Productivity

If user data is transmitted to a cloud server for deep learning inference, then inference performance is improved, but personal information leakage risk increases

Engineering Contradiction:

Improveinference performanceVSAvoidpersonal information leakage risk

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent extracts the deep learning inference capability from external cloud servers and implements it directly within the mobile terminal's processor. By taking out the inference function and embedding it in the device, the system eliminates the need to transmit user data to external servers, thereby preventing personal information leakage while maintaining inference performance through on-device processing.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The mobile terminal performs deep learning inference independently using its own processor and memory resources. The device serves itself by hosting the adaptive deep learning model locally, eliminating dependency on external cloud services and ensuring that sensitive user data remains within the device throughout the inference process.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20240211721A1Adaptive deep learning inference method and apparatus, and storage medium storing instructions to perform adaptive deep learning inference method

Publication Date: 2024.06.27 KOREA ADVANCED INST OF SCI & TECH

AI summary

There is provided a method for inferring a result data corresponding to an input data using an adaptive deep learning model in a mobile terminal including a memory and a processor. The method comprises determining computing resource information of the mobile terminal; determining a basic deep learning model stored in the memory of the mobile terminal; generating the adaptive deep learning model by transforming the basic deep learning model based on allocable resources determined with reference to the computing resource information of the the mobile terminal, wherein the adaptive deep learning model has a number of layers less than the basic deep learning model; and inputting the input data into the adaptive deep learning model in the mobile terminal to determine the inferred result data to be outputted from the adaptive deep learning model.