On-Device Computer Vision Filtering for Lightweight Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art object detectors are computationally expensive and complex, making them challenging for use on local devices like smartphones, and existing methods for improving precision and recall in object detection often increase complexity further.
Innovation Solution
Implement a two-stage approach using a lightweight initial filter to classify image regions, sending only regions of interest to a more specialized model, and masking or cropping regions without objects to reduce computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained to recognize objects, actions, and scenes, then computer vision performance is improved, but model size and computational requirements increase
Solution Approach 1:
The patent segments the machine learning model into a teacher model (large, high-accuracy model used for training) and a student model (small, efficient model deployed on-device). The teacher model processes images to generate training data, which is then used to train the compact student model that runs on mobile devices with limited resources.
Solution Approach 2:
The patent creates a simplified copy of the large teacher model by training a student model on distilled knowledge from the teacher. The student model replicates the essential vision recognition capabilities while occupying significantly less memory and requiring fewer computational resources for inference on mobile devices.
2Measurement precision
If large machine learning models are deployed on mobile devices, then vision recognition accuracy is improved, but memory usage and processing power requirements increase
Solution Approach 1:
The system separates the computationally intensive training phase (performed on server using large teacher model) from the inference phase (performed on mobile device using small student model). This segmentation allows high accuracy to be achieved during training while keeping on-device processing requirements minimal.
Solution Approach 2:
The student model is self-sufficient once trained, requiring minimal processing power to run on the mobile device. The model performs object, action, and scene recognition independently without needing continuous connection to external computing resources, enabling efficient on-device operation.
3Difficulty of detecting and measuring
If comprehensive object recognition is implemented, then detection capability is improved, but on-device storage requirements increase
Solution Approach 1:
The patent creates a compressed knowledge copy from the large teacher model that fits within mobile device storage constraints. The student model captures the essential detection capabilities for objects, actions, and scenes in a compact form factor, enabling comprehensive recognition without requiring gigabytes of additional storage space.
Data Source
Figure 1a
Figure 1B
Figure 1C
AI summary
The present disclosure is directed to object and/or character recognition for use in applications such as computer vision. Advantages of the present disclosure include lightweight functionality that can be used on devices such as smart phones. Aspects of the present disclosure include a sequential architecture where a lightweight machine-learned model can receive an image, detect whether an object is present in one or more regions of the image, and generate an output based on the detection. This output can be applied as a filter to remove image data that can be neglected for more memory intensive machine-learned models applied downstream.