AR Glasses Control for Prosthetic Hands and Robotic Arms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for controlling neuroprosthetic hands and robotic arms are not intuitive and require laborious learning, leading to potential incorrect operations.
Innovation Solution
A method using camera tracking with markers and augmented reality (AR) glasses to control robotic devices, where head movements and line of sight are used to generate commands, providing intuitive and precise control by recognizing predefined movements and reducing the risk of unintentional actuation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If EMG signals or brain-machine interface are used to control neuroprosthetic hands, then control capability is achieved, but the control method becomes complex and requires laborious learning
Solution Approach 1:
The patent replaces complex physiological signal-based control (EMG, brain-machine interface) with optical tracking using cameras and markers. The camera system tracks markers on the prosthetic device and translates visual position data into control commands, substituting biological signal processing with optical-mechanical tracking that is more intuitive and easier to operate.
Solution Approach 2:
The patent introduces markers as intermediary elements between the user and the prosthetic control system. These visual markers serve as a mediator that the camera can detect and translate into control signals, creating a simple visual-language interface that avoids the complexity of direct physiological signal interpretation while maintaining intuitive control.
2Reliability
If conventional control methods are used, then device actuation is possible, but the risk of unintentional actuation and incorrect operations increases
Solution Approach 1:
The patent implements visual feedback by continuously displaying the camera's view and marker positions to the user. This feedback loop allows the user to see exactly what the system detects, confirm proper marker recognition, and verify that control commands are being generated from intentional gestures, thereby reducing unintentional actuation while maintaining intuitive control through visual confirmation.
3Ease of operation
If EMG-based actuation is used, then prosthetic hand movement is achieved, but the learning curve is steep and user-friendly control is reduced
Solution Approach 1:
The patent uses visual copying by having the camera track and replicate the spatial position and movement of markers attached to or near the prosthetic device. The system copies the visual appearance and motion of the device itself as the control reference, making the control mechanism immediately understandable without requiring learning of abstract physiological signal patterns.
Data Source
AI summary
A method for controlling a device, in particular a prosthetic hand or a robotic arm, includes using an operator-mounted camera to detect at least one marker positioned on or in relation to the device. Starting from the detection of the at least one marker, a predefined movement of the operator together with the camera is detected and is used to trigger a corresponding action of the device. The predefined movement of the operator is detected in the form of a line of sight by means of camera tracking. A system for controlling a device, in particular a prosthetic hand or a robotic arm, includes a pair of AR glasses adapted to detect the at least one marker and to detect the predefined movement of the operator.