User Device Sensor Fusion for Autonomous Vehicle Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost and technical challenges of autonomous vehicle (AV) ridehail and rideshare services due to the complexity and expense of AV sensors, which hinder their ability to safely navigate environments.
Innovation Solution
Supplementing AV sensors with data from user devices, such as smartphones, using trained supplementary sensors and calibration profiles to enhance perception and navigation capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If AV sensors are made more complex and expensive to improve environmental perception, then navigation safety is improved, but cost and device complexity increase
Solution Approach 1:
The patent combines AV onboard sensors with user device sensors (smartphone cameras, microphones, GPS) to create a unified sensing system. The user device acts as a supplementary sensor node that extends the AV's environmental perception capabilities without requiring the AV to carry all sensing functions internally.
Solution Approach 2:
The system enables user devices to serve multiple functions: they act as supplementary cameras for visual perception, microphones for audio detection, GPS receivers for location data, and processing units for sensor data analysis. This multi-functionality reduces the need for dedicated specialized sensors on the AV.
2Reliability
If AV sensors are made more complex to improve environmental perception, then navigation safety is improved, but service cost increases
Solution Approach 1:
User devices that participants already possess (smartphones, tablets) are leveraged to provide sensing capabilities. The system utilizes existing hardware resources in the field rather than requiring the AV operator to provision and maintain specialized sensor equipment, thereby reducing service costs.
Solution Approach 2:
Instead of equipping each AV with expensive dedicated sensors, the system creates virtual copies of sensing capabilities by utilizing the sensors already present in user devices. The user device sensors serve as functional copies that supplement the AV's native sensor suite.
3Ease of manufacture
If user device sensors are used to supplement AV sensors, then cost is reduced, but data accuracy and reliability may be affected
Solution Approach 1:
The system implements feedback mechanisms where the AV's onboard sensors continuously verify and calibrate data from user device sensors. Discrepancies between AV sensor readings and user device sensor readings trigger recalibration or data rejection, ensuring maintained accuracy standards.
Solution Approach 2:
The system creates a composite sensing system that combines data from multiple sensor sources (AV onboard sensors and user device sensors) with different characteristics. By fusing data from heterogeneous sensors through calibration profiles and weighting algorithms, the system achieves robust measurement accuracy that leverages the strengths of each sensor type.
Data Source
AI summary
One or more embodiments can include identifying a personal sensor on a user device, communicating a library of object recognition profiles for the personal sensor to the user device, capturing personal sensor data related to an environment, using the library of object recognition profiles for the personal sensor to use the captured personal sensor data to identify objects and/or features in the environment, and using the identified objects and/or features in the environment to supplement data collected by the autonomous vehicle's sensors. The library of object recognition profiles can be created by identifying the personal sensor on the user device, training a personal perception module to identify objects and/or other features in an environment setting based on data captured by the personal sensor, and using the training from the personal perception module to create the library of object recognition profiles for the personal sensor.