Intuitive Computing Platform Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in enabling smart phones to autonomously understand and respond to their environment, particularly in identifying visual stimuli and inferring user desires, due to limitations in processing power and resource management, leading to inefficient and time-consuming solutions such as cloud-based image recognition or human intervention.
Innovation Solution
The implementation of an Intuitive Computing Platform (ICP) that employs local and cloud-based processing, using recognition agents and context information to progressively understand visual stimuli, allocate resources efficiently, and provide user-relevant responses, with features like baubles for interactive visual objects and dynamic resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cloud-based image recognition is used to identify visual stimuli, then measurement precision is improved, but loss of time increases significantly
Solution Approach 1:
The system performs preliminary local processing of image data on the mobile device before submitting to cloud services. This includes initial analysis, filtering, and preparation of visual stimuli, which reduces the complexity of cloud processing and accelerates overall recognition time while maintaining accuracy
Solution Approach 2:
The image recognition process is divided into multiple segments: local preprocessing on the device, selective cloud-based analysis for specific regions of interest, and local post-processing. This segmentation allows parallel execution of tasks and reduces the time-critical path by handling only essential portions in the cloud
2Reliability
If extensive cloud processing is performed for environment understanding, then reliability of user desire inference is improved, but use of energy increases
Solution Approach 1:
The system performs partial cloud processing only when necessary, based on local analysis results and context. Instead of always submitting full images for cloud analysis, it selectively processes only regions of interest or sends simplified data representations, reducing energy consumption while maintaining sufficient inference reliability
Solution Approach 2:
The system dynamically adjusts processing parameters such as image resolution, analysis depth, and cloud submission frequency based on available energy levels, network conditions, and user context. This allows the system to maintain reliable user desire inference when energy is abundant while conserving power when battery levels are low
3Adaptability or versatility
If multiple recognition agents are deployed for comprehensive environment analysis, then adaptability to different visual stimuli is improved, but device complexity increases
Solution Approach 1:
The system employs a hierarchical architecture where a single universal local processing agent handles multiple types of visual stimuli (images, video frames, screenshots) using common preprocessing techniques. Specialized recognition agents are selectively activated based on the type of stimulus detected, avoiding the need to maintain all specialized agents simultaneously active and reducing overall system complexity
Solution Approach 2:
The system dynamically loads and activates specific recognition agents based on the detected visual stimulus type and current processing needs. Instead of running all recognition agents continuously, it switches between them as needed, and can dynamically allocate cloud processing resources for specialized analysis only when required, thereby managing complexity while maintaining versatility
4Loss of time
If local processing is performed on the mobile device, then loss of time is reduced, but use of energy increases
Solution Approach 1:
The processing workload is segmented between local and cloud operations. Time-critical operations such as initial image capture, basic filtering, and rapid response generation are performed locally on the mobile device to minimize latency. Less time-critical but computationally intensive operations are delegated to the cloud, optimizing the balance between response time and energy consumption
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.