On-Device Image Classification via Hierarchical Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing methods on portable devices are resource-intensive, leading to shortened battery life and high bandwidth consumption, as they often require transmitting images to cloud-based servers for processing, which increases latency and communication costs.
Innovation Solution
A system that identifies interest points in images using a filter, gradient, and pool module, extracts features, and classifies them on-device, filtering out less-relevant data to transmit only relevant images to a remote server for further processing, employing a hierarchically-pipelined architecture with specialized processing elements for efficient energy use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If images are transmitted to cloud-based servers for processing, then image classification accuracy is improved, but bandwidth consumption and communication costs increase
Solution Approach 1:
The image classification system is segmented into two parts: a lightweight on-device classifier that performs initial filtering, and a cloud-based server that handles only the filtered subset of images. This segmentation allows most images to be classified locally without transmission, reducing bandwidth consumption while maintaining overall classification accuracy through the combination of both classifiers.
Solution Approach 2:
The on-device image classifier performs preliminary classification of images before they are transmitted to the server. By pre-filtering images based on local classification results, the system identifies and transmits only those images that require further processing, thereby reducing the quantity of data transmitted while ensuring that potentially important images are not missed.
2Measurement precision
If images are transmitted to cloud-based servers for processing, then image classification accuracy is improved, but latency increases
Solution Approach 1:
The classification process is segmented into immediate on-device classification and deferred cloud-based classification. The on-device classifier provides immediate results for most images, eliminating transmission latency for those cases. Only images requiring cloud processing experience the full latency, but this is minimized by the filtering effect of the on-device classifier.
Solution Approach 2:
The on-device classifier performs preliminary classification immediately on captured images without transmission delay. This preliminary action provides fast classification results for the majority of images, reducing overall system latency while the server handles the remaining images in the background.
3Measurement precision
If resource-intensive image processing is performed on portable devices, then image classification capability is improved, but energy consumption increases
Solution Approach 1:
The computational workload is segmented between the portable device and the cloud server. The on-device classifier uses simplified algorithms that consume minimal energy, while the energy-intensive cloud-based server handles the complex processing for filtered images. This segmentation allows the portable device to maintain classification capability while conserving energy resources.
Solution Approach 2:
The on-device classifier performs partial classification action on all images using lightweight algorithms, then transfers only the necessary remaining work to the server. This partial action approach provides sufficient classification capability for battery-powered operation while avoiding the excessive energy consumption of full-scale image processing on the portable device.
4Measurement precision
If all images are transmitted to the server for processing, then classification accuracy is improved, but false positives decrease
Solution Approach 1:
The on-device image classifier serves as an intermediary between image capture and server processing. It filters images before transmission, creating a selective gateway that reduces the volume of transmitted data while maintaining reliability by ensuring that images requiring server processing are properly identified and sent for accurate classification.
Data Source
AI summary
Examples of the disclosure enable efficient processing of images. In some examples, one or more interest points are identified in an image. One or more features are extracted from the identified interest points using a filter module, a gradient module, a pool module, and/or a normalizer module. The extracted features are aggregated to generate one or more vectors. Based on the generated vectors, it is determined whether the extracted features satisfy a predetermined threshold. Based on the determination, the image is classified such that the image is configured to be processed based on the classification. Aspects of the disclosure facilitate conserving memory at a local device, reducing processor load or an amount of energy consumed at the local device, and/or reducing network bandwidth usage between the local device and the remote device.