Vision-Based Prosthetic Control With AI Intent Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-human interface (MHI) systems for controlling prosthetics are complex, require extensive user training, and suffer from inaccuracies in executing movements.
Innovation Solution
An MHI system utilizing computer vision, pattern recognition, AI, and LIDAR technology to non-invasively predict a user's intention by examining the field of view and measuring the location of targeted objects, enabling control of external movement devices like prosthetics through a wearable display and external movement devices without implantable components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If implantable electrodes are used to record neural signals, then control accuracy of prosthetics is improved, but device complexity and invasiveness increase
Solution Approach 1:
The patent extracts the control interface from the implantable domain to the external wearable domain. Instead of using implantable electrodes to read neural signals, the system uses external cameras to capture visual field information and external sensors to detect head movements, thereby eliminating the need for complex implantable components while maintaining control capability.
Solution Approach 2:
The patent introduces an intermediary AI system that translates external visual and motion data into control commands for the prosthetic. This AI intermediary processes camera images to identify objects and combines them with head movement sensor data to generate accurate control signals, replacing the direct neural signal reading approach.
2Adaptability or versatility
If implantable electrodes and complex neural decoding are used, then prosthetic control capability is improved, but ease of operation deteriorates due to extensive training requirements
Solution Approach 1:
The AI system performs self-learning and adaptation by processing visual field data and head movement patterns. The system automatically identifies objects in the user's visual field and learns the relationship between head movements and intended prosthetic actions, eliminating the need for extensive manual training programs.
Solution Approach 2:
The patent replaces complex neural decoding algorithms with a simpler vision-based object recognition system combined with head movement detection. This substitution uses computer vision and pattern recognition to identify objects and infer intent, which is more intuitive and requires less training than neural signal interpretation.
3Extent of automation
If traditional MHI systems are used, then prosthetic control is achieved, but reliability deteriorates due to inaccuracies in executing movements
Solution Approach 1:
The system continuously captures visual field information through cameras and head movement data through sensors, providing real-time feedback to the AI control system. This feedback loop allows the system to monitor the environment and adjust control commands dynamically, improving the reliability of movement execution.
Solution Approach 2:
The AI system preliminarily identifies objects in the visual field and predicts user intent before executing prosthetic movements. By pre-processing visual data and anticipating user goals, the system can plan and execute movements more accurately and reliably.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system provides faster, more reliable, and accurate control of prosthetics with a user-friendly interface that learns from mistakes, improving over time, and eliminates the need for invasive components.
Implementation Method 1
The MHI system can measure the location of the targeted subject/object using a measuring means (e.g., using Light Detection and Ranging (LIDAR) technology)
Data Source
AI summary
Machine-human interface (MHI) systems for control of powered external movement devices, such as prosthetics (e.g., prosthetic hands and/or arms), are provided, as well as methods of using the same. The efficient MHI systems leverage features of computer vision and pattern recognition to examine the subjects and/or objects within the field of view of a user of the system, and then uses artificial intelligence and/or machine learning to guess the user's intention. Once the user acknowledges the guessed intention, the MHI system can measure the location of the targeted subject/object using a measuring means (e.g., using Light Detection and Ranging (LIDAR) technology) and then coordinate the movement of the external movement device (e.g., prosthetic arm and/or hand).