Handheld UPI Object Identification with Spatial Audio Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems on handheld devices are expensive, difficult to operate, and pose a safety risk due to user distraction, while simplified AR applications lack intuitive interaction with objects of interest in the surroundings.
Innovation Solution
A universal pointing and interacting (UPI) device that identifies and interacts with objects of interest using a handheld device, incorporating a camera, LIDAR, and sensors to provide information and control functions, enabling intuitive interaction and navigation for users, including the blind or visually impaired.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If full scale AR systems with HMD are used, then information provision about objects in surroundings is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent extracts the core AR functionality from complex HMD systems and implements it using a handheld device with camera, GPS, and basic processing capabilities. The system captures images, identifies objects using database matching, and provides information through the handheld device screen, eliminating the need for expensive head-mounted displays while maintaining object identification and information provision capabilities.
Solution Approach 2:
The patent uses a camera to capture visual copies of objects in the surroundings, then matches these visual copies against database entries to identify objects. This copying approach allows the system to recognize and provide information about objects without requiring complex sensors or processing hardware, thereby reducing system complexity while maintaining effective information provision.
2Device complexity
If simplified AR applications on handheld devices are used, then device complexity is reduced, but interaction intuitiveness deteriorates
Solution Approach 1:
The patent merges the camera, GPS receiver, object identification software, and information display into a single handheld device that users already carry. This integration maintains simplicity while improving interaction intuitiveness by providing contextual information about surroundings directly on the device screen, allowing natural pointing and viewing gestures without requiring separate equipment.
Solution Approach 2:
The system automatically captures images through the camera, processes them to identify objects by matching against databases, retrieves relevant information, and displays it to the user without requiring manual intervention. This self-service approach makes the interaction intuitive as the system autonomously performs all steps from object detection to information provision, reducing the cognitive load on the user.
3Loss of information
If handheld devices are held at eye level for navigation, then navigation information visibility is improved, but user safety deteriorates due to distraction
Solution Approach 1:
The patent transitions from two-dimensional screen display to three-dimensional spatial audio feedback. The system uses the device's speaker to play audio messages that convey navigation directions and object information, allowing users to keep the device in their hand or pocket rather than holding it at eye level. This dimensional change from visual to auditory feedback maintains information accessibility while eliminating the safety hazard of road distraction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables comfortable and intuitive interaction with objects in the surroundings, providing accurate location, orientation, and navigation assistance, enhancing safety and usability for users, particularly those with visual impairments.
Implementation Method 1
camera
Implementation Method 2
LIDAR
Implementation Method 3
The accuracy of the estimation of the location parameters may improve in urban and indoor environments by triangulating wireless signals such as cellular, Wi-Fi, Bluetooth, or dedicated radio-beacons
Implementation Method 4
The azimuth is estimated by a magnetometer that measures the earth magnetic field
Implementation Method 5
the orientation (pitch and roll) is estimated by an accelerometer that measures the earth gravity vector
Data Source
AI summary
This invention describes providing information of objects in the surroundings by a handheld device using a universal pointing and interacting device. The universal pointing and interacting (UPI) device is configured to identify objects in the surroundings and to interact with the handheld device such as a smartphone or a tablet. The handheld device is further configured to provide visual information on its screen or audio information by its loudspeaker or by a connected earpiece or headphones.