Wearable Device Object Recognition and Speech Query System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Wearable augmented reality devices often provide irrelevant information to users, leading to user fatigue, as they continuously supply data not needed by the user.
Innovation Solution
A wearable device equipped with a camera, microphone, and controller that identifies objects within the user's field of vision, stores information about these objects, and provides relevant information to the user upon request, either through explicit query or when the user is flustered, using a combination of artificial neural networks and multi-modal deep learning for emotion recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the wearable device continuously provides augmented reality information to the user, then the user receives comprehensive data about the environment, but the user experiences fatigue due to receiving irrelevant information
Solution Approach 1:
The device performs preliminary actions by continuously monitoring and identifying objects in the user's field of vision, pre-processing image data through object recognition algorithms, and storing relevant object information before the user actually requests it. This preparation enables the system to quickly provide relevant information when needed while filtering out irrelevant data proactively.
Solution Approach 2:
The system incorporates feedback mechanisms by detecting user queries through the microphone and sensor signals indicating user state (such as flustered state), then dynamically adjusting information provision based on this feedback. The device learns from user interactions to determine what information is actually needed, creating a closed-loop system that adapts to user requirements.
2Object-affected harmful factors
If the wearable device provides information only when explicitly queried, then user fatigue is reduced, but the device fails to provide necessary information when the user is flustered or unable to query
Solution Approach 1:
The device implements self-service by autonomously monitoring sensor signals to detect when the user is in a flustered state or otherwise unable to formulate a query. Upon detecting such states, the system automatically provides relevant object information without requiring explicit user initiation, ensuring continuous support regardless of user capacity.
Solution Approach 2:
The system uses feedback from sensor signals (such as biometric data indicating emotional or physical state) to dynamically adjust its information provision behavior. When sensors indicate the user is flustered, the system interprets this feedback and automatically provides relevant information, creating an adaptive response to user needs.
3Loss of information
If the wearable device continuously monitors and processes image and speech data, then relevant information can be provided on demand, but energy consumption increases
Solution Approach 1:
The device employs periodic action by processing image and speech data selectively rather than continuously. The system activates intensive processing only when triggered by specific events such as detected speech queries or sensor signals indicating user need, otherwise entering a lower-power monitoring state. This periodic processing pattern maintains information retrieval capability while significantly reducing average energy consumption.
4Loss of information
If the wearable device provides detailed object information continuously, then the user has access to comprehensive data, but the device complexity increases
Solution Approach 1:
The system applies the extraction principle by selectively extracting only the relevant object information needed for specific user queries or situations. Rather than continuously providing all available data, the device extracts and provides only the specific attributes and details pertinent to the current context, reducing information overload while maintaining comprehensive underlying data availability.
Data Source
AI summary
A method of controlling a wearable device, and which includes obtaining, by a camera of the wearable device, an image including at least a partial field of vision of a user; identifying, by a controller of the wearable device, an object appearing in the image obtained by the camera at a first time point; storing information about the identified object in a memory; detecting, via a microphone of the wearable device, a speech of a user using the wearable device at a second time point; identifying, via the controller, a word included in the detected speech of the user corresponds to the object identified in the image; retrieving, via the controller, the information about the object from the memory based on the identified word included in the detected speech; and outputting the retrieved information about the object.