Low-Power ML Vision Processing via Image Patch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning (ML) techniques for vision processing consume significant power, posing challenges for low-power, embedded, and real-time applications, particularly in mobile devices and edge computing.
Innovation Solution
The development of low-power machine-learning (LP-ML) algorithms that utilize machine-learned patch features, optimized for embedded hardware environments, and combine with image signal processor (ISP) and hardware acceleration to reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional neural network-based ML models are used for vision processing, then detection accuracy is improved, but power consumption increases significantly
Solution Approach 1:
The patent segments the image processing task by dividing the image into multiple patches and processing them independently through the ML model. This allows the system to process only relevant regions at high resolution while reducing overall computational load, thereby maintaining detection accuracy for critical areas while lowering power consumption across the entire image processing pipeline.
Solution Approach 2:
The patent applies partial action by processing only certain patches of the image through the full ML pipeline rather than the entire image. Less critical regions may use simplified processing or lower resolution analysis, while critical regions receive full ML processing. This selective approach maintains necessary detection accuracy while significantly reducing overall power consumption.
2Measurement precision
If high-resolution image processing is performed, then detection precision is improved, but computational complexity increases
Solution Approach 1:
The patent divides the high-resolution image into multiple smaller patches that are processed independently. Each patch requires less computational resources to analyze while maintaining the ability to detect fine details within that specific region. This segmentation approach allows high detection precision in critical areas without requiring the entire system to handle the full computational burden of the complete high-resolution image.
Solution Approach 2:
The patent applies different processing qualities to different regions of the image based on their importance. Critical regions containing potential objects of interest receive full high-resolution ML processing for maximum detection precision, while less critical background regions use simplified or lower-resolution processing. This local quality differentiation maintains necessary detection precision where needed while reducing overall computational complexity.
3Speed
If real-time processing is implemented, then response speed is improved, but power consumption increases
Solution Approach 1:
The patent segments the image into patches that can be processed in parallel, significantly reducing the time required to complete analysis compared to sequential full-image processing. This parallel processing of divided patches enables real-time response speeds while keeping power consumption manageable by distributing computational tasks across multiple smaller, more efficient processing units rather than overloading a single high-power processor.
Solution Approach 2:
The patent implements periodic processing where the system analyzes image patches at strategically determined intervals rather than continuously processing every frame at full resolution. This periodic action maintains real-time responsiveness for critical detections while allowing the system to enter lower-power states between processing cycles, thereby achieving real-time performance without sustained high power consumption.
Data Source
AI summary
Systems, computer programs, devices, and methods that enable ML-based vision processing for low-power, embedded, and/or real-time applications. In one exemplary embodiment, smart glasses use classifiers that are based on machine-learned (ML) patch relationships. The ML patch features are determined during an offline training process. The ML patch features are grouped into weak classifiers, strong classifiers, and detectors to progressively improve prediction accuracy. An object detection architecture uses triggering logic, search management, and a classification neural network to enable event-based searching, interest-based searching, and/or dynamic search control. In some cases, pre-processing may also be used to minimize the neural network complexity (e.g., pre-processing for scaling, rotations, translations, etc.).