Smartphone Visual Processing via Context Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in enabling smartphones to autonomously understand and respond to user environment stimuli, particularly in identifying visual inputs and determining appropriate actions, due to resource constraints and the complexity of the visual world.
Innovation Solution
The implementation of an Intuitive Computing Platform (ICP) that employs image processing, context information, and cloud-based resources to progressively understand visual stimuli, using baubles and recognition agents to provide user interactions and responses, while managing resources and user interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cloud-based image recognition algorithms are applied to understand visual stimuli, then understanding accuracy is improved, but processing time and energy consumption increase significantly
Solution Approach 1:
The system performs preliminary local processing on the mobile device to identify and extract key visual features before submitting to cloud services. This preliminary action filters out unnecessary data and prepares pre-processed information, reducing the time required for cloud-based analysis while maintaining understanding accuracy.
Solution Approach 2:
The image recognition task is segmented into multiple stages: local feature extraction on the device, cloud-based algorithmic analysis, and result integration. This segmentation allows parallel processing and optimizes the use of both local and remote resources, reducing overall processing time.
2Productivity
If comprehensive image processing is performed locally on the smartphone, then processing speed is improved, but device energy consumption and resource usage increase
Solution Approach 1:
Different processing operations are performed at different locations based on their resource requirements. Simple, energy-efficient operations are executed locally on the device, while computationally intensive tasks are offloaded to cloud services. This local quality differentiation optimizes the balance between processing speed and energy consumption.
Solution Approach 2:
A context engine acts as an intermediary that manages the division of processing tasks between local and cloud resources. It monitors device state, user context, and task requirements to dynamically determine which operations should be performed locally versus remotely, optimizing energy usage while maintaining productivity.
3Adaptability or versatility
If multiple recognition agents and services are deployed to handle diverse visual stimuli, then system versatility is improved, but device complexity increases
Solution Approach 1:
A single context engine provides universal management for multiple recognition agents and services. Instead of deploying separate complex systems for each function, the context engine serves as a multi-functional platform that coordinates diverse visual recognition tasks, reducing overall system complexity while maintaining versatility.
Solution Approach 2:
The system implements feedback mechanisms where the context engine continuously monitors recognition outcomes and adjusts resource allocation, service selection, and processing strategies. This feedback loop enables the system to adapt to diverse visual stimuli dynamically without requiring pre-configured complex pathways for every possible scenario.
4Loss of information
If extensive context information is collected and processed, then user intent understanding is improved, but information processing overhead increases
Solution Approach 1:
The context engine extracts only the most relevant context information needed for understanding user intent, rather than processing all available data. It selectively takes out and processes key contextual elements based on the current task and user state, reducing information processing overhead while maintaining understanding accuracy.
Data Source
AI summary
A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.