Guidance Display Device Using Action Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional guidance information display devices for users operating instruments often fail to provide timely and relevant information, as they rely solely on the object being watched by the user, without considering the instrument's state or the user's intentions, leading to ineffective guidance.
Innovation Solution
A guidance information display device that uses a head-mounted display and portable terminal connected via a network to detect user actions and instrument states, providing tailored guidance by analyzing camera images and user intentions, and displaying relevant information at the appropriate time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If guidance information is displayed only when the user watches the object, then the display timing is simple to determine, but the guidance information is not displayed at the suitable timing and is not effective
Solution Approach 1:
The system performs preliminary detection of user actions (picking up objects, opening instruments) before the actual guidance is needed. By detecting these precursor actions and pre-loading or pre-positioning guidance information, the system ensures that relevant guidance is displayed at the optimal moment when the user actually needs it, rather than waiting for the user to watch the object.
Solution Approach 2:
The system continuously monitors user actions and instrument states, using this feedback to dynamically determine when to display guidance information. By establishing a feedback loop that tracks user behavior patterns and instrument operational states, the system can identify the most appropriate timing for guidance display based on actual usage context.
2Loss of information
If only the watched object is used to determine information to be displayed, then the system is simple to operate, but it is not possible to determine the information to be provided when there are multiple types of instruments
Solution Approach 1:
The system merges multiple information sources including the watched object, detected user actions, and instrument state data to comprehensively determine the guidance information to display. By combining these different data streams, the system can accurately identify which guidance is relevant even when multiple instrument types are involved.
Solution Approach 2:
The system dynamically adjusts the guidance information based on real-time changes in user actions and instrument states. Rather than using a static approach based solely on the watched object, the system continuously updates the displayed guidance to match the current operational context, ensuring relevance across different instrument types and usage scenarios.
3Loss of information
If the same guidance information is displayed regardless of instrument state, then the display process is simple, but the suitable information changes according to the state of the instrument
Solution Approach 1:
The system applies local quality by providing different guidance information tailored to specific instrument states and user actions. Rather than using a uniform guidance approach, the system customizes the displayed information based on the local context of what the user is doing and what state the instrument is in, ensuring each guidance message is precisely appropriate for that moment.
Solution Approach 2:
The system changes guidance parameters based on instrument state variations. By monitoring changes in instrument operational parameters and user action parameters, the system dynamically adjusts the displayed guidance information to match the current state, ensuring accuracy without requiring complex manual intervention.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is a guidance information display device, including: a display unit; a camera unit; an action detecting unit to detect a specific action of a user; and a control unit to instruct the display unit to display guidance information when the specific action is detected by the action detecting unit and a specific apparatus to be operated is shot by the camera unit, the displayed guidance information corresponding to a combination of the detected specific action and the specific apparatus.