ML Model Resource Evaluation for Constrained Devices

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Resource-constrained devices face challenges in implementing machine learning models due to limited processing power, memory, and battery life, and delegating tasks to the cloud increases complexity and cost, with network bottlenecks being a performance hindrance.

Innovation Solution

The method involves evaluating and tuning machine learning models and their associated computer-readable instructions to optimize resource usage on resource-constrained devices by identifying necessary computing resources such as memory and processing cycles, and applying alterations through a problem-solution matrix to ensure compliance with technical and commercial constraints.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If machine learning models are implemented on resource-constrained devices, then processing speed and response time improve, but device complexity and cost increase

Engineering Contradiction:

Improveprocessing speedVSAvoiddevice complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The patent segments the machine learning model into multiple layers and evaluates each layer's resource requirements separately. This allows the system to identify which specific layers consume the most memory and processing power, enabling targeted optimization rather than treating the entire model as a monolithic block.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes parameters such as data precision (e.g., converting from 32-bit floating point to 8-bit integers) and model architecture configurations to reduce resource consumption. These parameter adjustments allow the model to run on resource-constrained devices while maintaining acceptable performance levels.

Inventive Principle:

Principle #35Parameter changes

2Loss of time

If machine learning models are implemented on resource-constrained devices, then response time improves, but memory usage increases

Engineering Contradiction:

Improveresponse timeVSAvoidmemory usage

Core Design Contradiction:

Loss of timeVSQuantity of substance

Solution Approach 1:

The patent loads only the necessary portions of the model into memory at any given time, rather than requiring the entire model to reside in memory simultaneously. This partial loading approach reduces peak memory usage while still enabling fast inference by keeping critical layers readily accessible.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent implements a hierarchical memory structure where frequently accessed model layers are stored in faster, smaller memory, while less frequently accessed layers are stored in slower, larger storage. This nested arrangement allows the system to achieve fast response times for critical operations while managing overall memory consumption efficiently.

Inventive Principle:

Principle #7Nested doll (Nesting)

3Power

If machine learning models are implemented on resource-constrained devices, then processing power requirements increase, but device cost decreases

Engineering Contradiction:

Improveprocessing powerVSAvoiddevice cost

Core Design Contradiction:

PowerVSDevice complexity

Solution Approach 1:

The patent performs model evaluation and optimization offline before deployment on the resource-constrained device. Resource requirements are calculated and model parameters are adjusted in advance, allowing the device to operate with reduced processing power requirements without sacrificing model performance during actual use.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces computationally intensive operations with simplified algorithms or pre-computed lookup tables. For example, complex matrix multiplications are replaced with optimized kernels or pre-calculated values, reducing the processing power required while maintaining inference accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Power

If cloud-based machine learning is used, then processing power requirements are reduced, but network dependency increases

Engineering Contradiction:

Improveprocessing powerVSAvoidnetwork dependency

Core Design Contradiction:

PowerVSReliability

Solution Approach 1:

The patent introduces an intermediate evaluation layer that assesses the device's actual resource capabilities and determines the appropriate model configuration to deploy. This intermediary assessment mechanism allows the system to balance between local processing (reducing network dependency) and cloud-based processing (reducing device power requirements) based on real-time conditions.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11551147B2Evaluating resources used by machine learning model for implementation on resource-constrained device

Publication Date: 2023.01.10 KONINKLIJKE PHILIPS NV

AI summary

The present disclosure is directed to methods and apparatus for evaluating resources that would be used by machine learning model(s) for purposes of implementing the machine learning model(s) on resource-constrained devices. For example, in one aspect, a plurality of layers in a machine learning model may be identified. A plurality of respective output sizes corresponding to the plurality of layers may be calculated. Based on the plurality of output sizes, a maximum amount of volatile memory used for application of the machine learning model may be estimated and compared to a volatile memory constraint of a resource-constrained computing device. Output indicative of a result of the comparing may be provided at one or more output components.