Video-Based Gait Analysis Using Spatio-Temporal Joint Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional gait assessment methods are subjective, intrusive, or fail to capture the complex biomechanical dependencies between body joints, lacking integration of spatial and temporal relationships in human motion analysis.
Innovation Solution
A digital system using video-trained spatio-temporal graph neural networks (ST-GNNs) for automated gait analysis, incorporating spatial-temporal graph convolutional networks (ST-GCNs) to analyze video data, capturing both spatial connectivity and temporal dynamics of human motion, with optional integration of inertial data and muscle activation parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If wearable sensors are used for gait assessment, then objective measurement is improved, but device complexity and ease of operation deteriorate due to precise placement requirements and intrusiveness
Solution Approach 1:
The patent replaces wearable mechanical sensors with a computer vision system that uses cameras to capture and analyze gait data. This substitution eliminates the need for precise sensor placement while maintaining objective measurement capabilities through automated image processing and machine learning algorithms.
Solution Approach 2:
The system creates a digital representation (copy) of the patient's gait pattern by capturing video data and processing it through pose estimation algorithms. This digital model allows for objective analysis without requiring physical sensors on the patient's body, thereby improving ease of operation while maintaining measurement precision.
2Ease of operation
If traditional visual observation is used for gait assessment, then ease of operation is improved, but measurement precision deteriorates due to subjectivity and lack of quantitative rigor
Solution Approach 1:
The patent replaces subjective visual observation with an automated computer vision system that uses machine learning models to objectively quantify gait parameters. This substitution maintains ease of operation (simple video capture) while dramatically improving measurement precision through quantitative analysis of joint positions and movements.
Solution Approach 2:
The system enables self-service gait assessment by automatically analyzing gait patterns without requiring a trained therapist for each evaluation. The automated pose estimation and gait analysis algorithms perform the assessment independently, maintaining ease of operation while providing precise, reproducible measurements.
3Device complexity
If independent signal analysis is used for movement data, then device complexity is reduced, but measurement precision deteriorates due to failure to capture biomechanical dependencies between joints
Solution Approach 1:
The patent merges the analysis of individual joint movements with the analysis of inter-joint relationships by processing movement data as an integrated system. The machine learning model simultaneously captures independent joint positions and the biomechanical dependencies between joints, improving measurement precision without significantly increasing device complexity.
Solution Approach 2:
The system adds a new dimension of analysis by incorporating spatial relationships and temporal dynamics into the movement data processing. Instead of analyzing joints independently, the system analyzes joints in the context of their spatial arrangements and temporal sequences, capturing the full complexity of biomechanical dependencies while maintaining manageable system complexity through efficient algorithms.
Data Source
AI summary
A method for pose and gait classification and motion prediction using spatio-temporal relationships between body joints includes capturing a sequence of images or video frames of a subject; applying a neural network-based pose estimation algorithm to the sequence of images or video frames to detect landmark positions of anatomical joints; constructing a spatio-temporal graph from the detected landmark positions of the one or more anatomical joints, wherein nodes of the spatio-temporal graph correspond to the anatomical joints and the landmark positions, spatial edges of the spatio-temporal graph represent anatomical connections between the anatomical joints within a single image or frame, and temporal edges connect the one or more anatomical joints across successive images or frames of the sequence of images or video frames; and inputting the constructed spatio-temporal graph into a spatio-temporal graph convolutional network (ST-GCN) to classify pose and gait patterns and predict motion or stability states.