Lidar Gesture Recognition for Autonomous Vehicle Ride Hailing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles (AVs) are unable to recognize traditional hailing gestures, such as raising an arm, from pedestrians, limiting their ability to provide on-demand transportation services to individuals who are not familiar with or do not have access to internet-connected devices.
Innovation Solution
The AV is equipped with a gesture recognition component that uses lidar sensor data to generate a skeletal model of pedestrians and determine if they are making a hailing gesture by comparing the model to a reference model, either through a weighted sum of distances or a neural network analysis, and initiates a pickup maneuver upon confirmation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AVs use conventional summoning methods requiring internet-connected devices, then service can be provided to tech-savvy users, but accessibility is limited for people without such devices or knowledge
Solution Approach 1:
The system enables pedestrians to hail the AV directly through natural gestures (raising an arm) that the lidar sensor system detects and interprets. The AV autonomously responds to the gesture by determining rider intent and initiating pickup maneuvers, eliminating the need for pedestrians to use internet-connected devices or apps.
Solution Approach 2:
The patent replaces the digital/mechanical interaction system (internet-connected devices, apps, and buttons) with an optical sensing system (lidar) that detects physical human gestures. The lidar sensor system captures three-dimensional point cloud data to identify arm-raising motions, translating physical gestures into digital signals for vehicle response.
2Measurement precision
If AVs are equipped with gesture recognition capabilities using lidar data processing and skeletal model generation, then recognition accuracy for hailing gestures is improved, but computational complexity and processing requirements increase
Solution Approach 1:
The gesture recognition system processes lidar data by segmenting the point cloud into individual pedestrian detections, then further segmenting each pedestrian's body into skeletal components (joints and members). This hierarchical segmentation allows the system to focus computational resources on relevant features while maintaining high recognition accuracy.
Solution Approach 2:
The system creates a skeletal model as a simplified copy or representation of the actual pedestrian geometry captured by lidar. This skeletal abstraction retains the essential pose information needed for gesture recognition while reducing computational complexity compared to processing full point cloud data.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables AVs to respond to pedestrians making hailing gestures, allowing them to stop and allow passengers to board, thereby expanding the accessibility of on-demand transportation services to a broader population.
Implementation Method 1
a lidar sensor system that outputs lidar data based on sensor signals received from the at least one lidar sensor, where the lidar data is indicative of positions of objects in the driving environment
Data Source
AI summary
An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a gesture recognition component to determine, based upon lidar sensor data, whether a pedestrian in a driving environment of the AV is performing a hailing gesture. The AV can be configured to initiate a pickup maneuver to approach the pedestrian in response to determining that the pedestrian is performing a hailing gesture.