Gesture-Based Human Following With Pose Authentication for Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in safely navigating environments with humans, particularly in detecting and responding to human gestures for control and following modes.
Innovation Solution
A gesture-based system that uses image sensors and machine learning models to detect humans, authenticate users through specific gestures, and enable follow modes by recognizing follow gestures, allowing the vehicle to track and follow humans while maintaining a safe distance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture-based authentication and control systems are implemented, then ease of operation is improved, but device complexity increases due to machine learning models and image processing requirements
Solution Approach 1:
The patent introduces an intermediary processing system that includes image sensors, machine learning models, and pose estimation algorithms. This intermediary layer translates complex computer vision processing into simple gesture recognition, allowing the vehicle to understand human intentions without requiring direct complex interaction. The intermediary system handles the computational complexity internally while presenting a simple gesture-based interface to users.
2Adaptability or versatility
If human following capability is added to autonomous vehicles, then adaptability to different operating modes is improved, but safety risks increase due to potential unauthorized control
Solution Approach 1:
The patent implements preliminary authentication through gesture recognition before enabling follow mode. The system requires the human to perform specific authenticated gestures that are verified by the machine learning model before the vehicle enters follow mode. This preliminary action ensures that only authorized individuals can activate the follow capability, preventing unauthorized control while maintaining adaptability.
Solution Approach 2:
The system continuously monitors and tracks the human's pose and gestures in real-time during follow mode. By providing continuous feedback through pose estimation and gesture detection, the system can verify ongoing authentication and detect unauthorized actions. This feedback mechanism allows the vehicle to maintain safety by detecting when the authenticated human is no longer present or when unauthorized gestures are performed.
3Speed
If real-time gesture recognition is implemented, then responsiveness to human commands is improved, but measurement precision requirements increase for accurate pose detection
Solution Approach 1:
The patent segments the gesture recognition process into distinct stages: image capture, pose estimation, gesture classification, and command execution. By dividing the complex real-time recognition task into smaller sequential segments, the system can process each stage with appropriate precision requirements. The machine learning model processes images in segments (detecting key points, estimating poses, classifying gestures) which allows for optimized computation at each stage while maintaining overall real-time performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for gesture-based systems for human following are described. For example, a method includes detecting a human in a first image captured using an image sensor connected to a vehicle; inputting the first image to a first machine learning model to obtain a first pose of the human; comparing the first pose to pose parameters for an authentication gesture; authenticating the human based on a match to enable gesture commands from the human; inputting a second image captured using the image sensor to the first machine learning model to obtain a second pose of the human; comparing the second pose to pose parameters for a follow gesture; commencing a follow mode based on a match between the second pose and the follow gesture occurring after authentication of the human; and controlling the vehicle to follow the human responsive to being in the follow mode.