Vehicle Person Classification and Gesture Recognition Using Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems in vehicles, particularly for autonomous or semi-autonomous vehicles, are resource-intensive due to the use of three-dimensional image processing and machine learning techniques, which can be slow and require extensive training data, making real-time gesture recognition challenging.
Innovation Solution
A system utilizing a convolutional neural network architecture that treats time sequences of pose-estimation as images, allowing for faster training and real-time inference, while maintaining accuracy, by using pose-estimation and a graph convolutional network, and employing skeleton-based action techniques to identify and track gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three-dimensional image processing and machine learning techniques are used to identify objects and gestures, then measurement precision and reliability are improved, but device complexity and resource consumption increase
Solution Approach 1:
The system segments the image processing task by first extracting pose estimation (skeletal representation) from images, then using this segmented skeletal data as input to a machine learning model for gesture recognition. This segmentation separates the complex 3D image processing into manageable stages: pose extraction followed by gesture classification, reducing overall system complexity while maintaining accuracy.
Solution Approach 2:
The pose estimation (skeletal representation) serves as an intermediary between the raw image data and the gesture recognition model. Instead of feeding complex 3D image data directly into the machine learning model, the system uses pose estimation as an intermediate representation that captures essential body configuration information, simplifying the input for gesture classification.
2Measurement precision
If machine learning techniques are used for image processing, then measurement precision is improved, but training time and productivity are worsened
Solution Approach 1:
The system extracts pose estimation (skeletal data) from the images as a separate intermediate representation before feeding data to the machine learning model. This extraction step simplifies the input data structure, allowing the model to focus on learning gesture patterns from already-processed pose information rather than learning from raw pixel data, thereby reducing training time while maintaining recognition accuracy.
3Measurement precision
If three-dimensional image processing is used, then measurement precision is improved, but resource consumption increases
Solution Approach 1:
The system creates a simplified copy of the essential information from the images through pose estimation (skeletal representation). Instead of processing the full 3D image data, the system generates a simplified skeletal copy that captures the necessary body configuration information, significantly reducing computing resource consumption while preserving the information needed for accurate gesture recognition.
Data Source
AI summary
A method includes receiving image data that includes at least two images of an environment associated with a vehicle, identifying at least one person of interest in the image data, and generating, using a pose estimation model and the image data, a representation of the person of interest. The method also includes determining at least one characteristic associated with the at least two images of the image data and providing, to a machine learning model, at least the representation of the person of interest and the at least one characteristic associated with the at least two images of the image data. The method also includes receiving, from the machine learning model, a gesture prediction indicating a predicted gesture being made by the person of interest, and causing the vehicle to take at least one action based on the gesture prediction.