Computer Vision Hand Grasp Control for Non-Invasive FES
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neuromuscular stimulation systems face challenges in accurately decoding volitional intent for complex hand movements due to the complexity of brain neural activity and muscle signal interference, particularly in cases of paralysis, leading to invasive procedures and unreliable EMG readings.
Innovation Solution
Utilizing computer vision to analyze video of the hand and object to determine the necessary hand actions for manipulation, combined with BCI, gaze monitoring, and EMG/EEG triggers, to control functional electrical stimulation (FES) devices for non-invasive upper limb reanimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If EMG signals are used to decode volitional intent for hand movements, then muscle contraction detection is improved, but signal reliability deteriorates due to muscle signal interference and paralysis
Solution Approach 1:
The patent introduces computer vision as an intermediary system that captures visual information about hand movements and objects, serving as a mediator between the user's intent and the FES control system. This bypasses the unreliable EMG signals by using an alternative sensing modality (visual) to infer hand action intent, thereby resolving the contradiction between detection capability and signal reliability in paralyzed patients
Solution Approach 2:
The patent replaces the biological electrical signaling system (EMG) with an optical sensing system (computer vision). By substituting the mechanical/biological measurement approach with an optical one, the system achieves reliable intent detection without being constrained by muscle signal interference or paralysis-related EMG degradation
2Measurement precision
If cortical implants are used to decode neural signals for intended actions, then intent detection accuracy is improved, but invasiveness increases
Solution Approach 1:
The patent uses computer vision as a non-invasive intermediary that indirectly infers hand action intent from visual observations of hand-object interactions. This mediator approach achieves intent detection without requiring direct neural signal acquisition through invasive cortical implants, thereby maintaining accuracy while eliminating surgical intervention
Solution Approach 2:
The patent creates a visual copy or representation of the hand's intended action through computer vision analysis of hand-object interactions. Instead of directly accessing neural signals, the system observes and interprets the visual manifestation of intent (hand movements toward objects), providing a non-invasive alternative that captures the essential information needed for FES control
3Quantity of substance
If high-density EMG electrodes are used to capture muscle signals, then signal quantity is improved, but device complexity increases
Solution Approach 1:
The patent replaces the complex array of high-density EMG electrodes with a simpler computer vision system. The intermediary visual sensing approach captures sufficient information about hand actions without requiring numerous physical electrodes, thereby reducing device complexity while maintaining adequate signal quantity for control purposes
Solution Approach 2:
The patent substitutes the mechanical electrode array system with an optical sensing system. This replacement eliminates the need for multiple physical contact points and complex electrode positioning, significantly simplifying the device architecture while still providing sufficient data for decoding hand action intent
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables precise and non-invasive control of hand movements for daily activities by accurately determining hand actions through computer vision, reducing cognitive load and avoiding invasive procedures.
Implementation Method 1
a video camera arranged to acquire video of a hand of a person and of an object
Implementation Method 2
proximity of the hand to the object measured by a proximity sensor
Data Source
AI summary
An assistance method for assisting a person in grasping or otherwise manipulating an object includes receiving video of a hand of the person and of an object. An intent to grasp the object is identified based on proximity of the hand to the object in the video or as measured by a proximity sensor, or using gaze tracking, or based on measured neural activity of the person. The object and the hand in the video are analyzed to determine an object grasping action for grasping or otherwise manipulating the object. An actuator is controlled to cause the hand to perform the determined hand action for grasping or otherwise manipulating the object.