Local Image Tagging via Deep Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for enabling visual search on mobile devices require significant computing resources, leading to scalability and cost issues, and raise privacy concerns due to the need for network-based image processing and storage.
Innovation Solution
Implementing deep convolutional neural networks (DCNN) and knowledge graphs on mobile devices for local image tagging and processing, allowing for ephemeral and non-ephemeral content management, enabling secure and private visual search without network access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If network-based image processing and storage is used, then visual search functionality is enabled, but computing resources and costs increase significantly
Solution Approach 1:
Instead of processing images on remote servers (traditional approach), the patent inverts the architecture by implementing deep convolutional neural networks directly on mobile devices. This allows the device to perform image tagging and visual search locally, eliminating the need for continuous network communication and reducing server computing resources.
2Productivity
If network-based image processing is used, then visual search is enabled, but privacy concerns arise due to network access requirements
Solution Approach 1:
The patent extracts the image processing functionality from the network environment and embeds it directly in the mobile device. By taking out the deep learning models and processing capabilities from remote servers and placing them locally, the system enables visual search without requiring images to be transmitted over the network, thus eliminating privacy risks associated with network-based processing.
3Ease of operation
If deep convolutional neural networks are implemented on mobile devices, then local image processing is enabled, but device complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-training deep convolutional neural networks on large datasets before deploying them to mobile devices. The complex model training and optimization are performed in advance on powerful servers, and only the finalized models are transferred to devices. This allows mobile devices to perform complex image processing without requiring users to manage the complexity of model training and updates.
4Object-affected harmful factors
If images are processed and stored locally, then privacy is improved, but storage space requirements increase
Solution Approach 1:
The patent extracts only the essential processing capabilities (deep learning models) from remote servers and embeds them locally, while maintaining a minimal local storage footprint. The system processes images locally for tagging and search without requiring extensive local storage of raw image data, thus achieving privacy protection without excessive storage requirements.
Data Source
AI summary
Systems, methods, devices, media, and computer-readable instructions are described for local image tagging and processing in a resource-constrained environment such as a mobile device. In some embodiments, characteristics associated with images are used to determine whether to store content (e.g., images and video clips) as ephemeral content or non-ephemeral content. Based on the determination, the image is stored in a non-ephemeral camera roll storage of the mobile device, or an ephemeral local application storage. Additional storage operations such as encryption or backup copying may additionally be determined and performed based on the analysis of the content. In some embodiments, such images may be indexed, sorted, and searched based on the image tagging operations used to characterize the content.