MobileDeepPill CNN for Pill Recognition on Smartphones
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pill identification systems through image analysis are inaccurate due to variability in image attributes such as white balance, focus, orientation, and ambient conditions, and are resource-intensive, making them unreliable and inaccessible on mobile devices without cloud connectivity.
Innovation Solution
The development of a small-footprint deep learning system, MobileDeepPill, using convolutional neural networks (CNNs) that can operate on mobile devices to recognize pills from unconstrained images, incorporating a multi-CNN architecture and a teacher-student learning paradigm for model compression, allowing for accurate identification without external computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing pill identification systems use complex specialized systems to improve accuracy, then measurement precision is improved, but device complexity increases and reliability decreases
Solution Approach 1:
The patent replaces complex mechanical and optical systems (enclosed chambers, laser imaging devices, predetermined checkerboard backgrounds) with a software-based deep learning system that runs on mobile device cameras and processors. The convolutional neural network processes images directly from the mobile camera, eliminating the need for specialized hardware while maintaining or improving identification accuracy through learned features from training data.
2Measurement precision
If existing systems increase complexity to handle image variability, then measurement precision is improved, but ease of operation deteriorates
Solution Approach 1:
The system automatically handles all image processing and pill identification tasks without requiring user intervention for calibration, background setup, or parameter adjustment. The deep learning model self-adapts to various imaging conditions by learning from diverse training data, eliminating the need for users to understand or control technical parameters like white balance, focus, or orientation.
3Measurement precision
If existing systems rely on cloud connectivity for processing, then measurement precision is improved, but reliability deteriorates when cloud is unavailable
Solution Approach 1:
The patent segments the pill identification system into two independent components: a training phase that can use cloud resources to build the deep learning model, and an inference phase that runs entirely on the mobile device. This segmentation allows the system to achieve high accuracy through comprehensive training while ensuring reliable operation offline by executing the trained model locally without cloud dependency.
4Measurement precision
If existing systems use resource-intensive processing, then measurement precision is improved, but loss of energy increases
Solution Approach 1:
The patent performs the computationally intensive model training and feature extraction in advance during an offline training phase, producing a compact trained model that can be deployed to mobile devices. The pre-trained model contains learned features and parameters that enable accurate pill identification with minimal real-time computation, significantly reducing energy consumption during actual use while maintaining high precision.
Data Source
AI summary
A system and method is provided that utilizes deep learning, including convolutional neural networks, to identify subject objects in unconstrained user images such as unknown pills. An image of, e.g., a pill, may be captured and subsequently processed using deep learning models to identify the pill. The deep learning models may be optimized to have a small footprint (in terms of computational and memory resources) suitable for a resource-limited device such as a smartphone while retaining a high object recognition accuracy. Each such model may also be run on modified versions of the unconstrained image, for example on color, greyscale, and gradient images, to focus the models on different distinguishing features of the object.