Surfel Map Localization for Sensor-Blind Regions in Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in detecting objects outside the detectable range or field of view of their sensors, such as lidar systems, which limits their ability to make accurate long-range localization predictions and plan for unexpected objects.
Innovation Solution
The use of surfel maps combined with camera image data allows the vehicle to predict the state of regions outside the detectable range by comparing image data with existing surfel data, enabling the system to identify unexpected objects and plan accordingly, even when within the lidar's field of view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If detection sensors (lidar) are used to detect objects, then reliable detection results are obtained for objects in limited detectable region, but objects outside the detectable region (too far away or outside field of view) cannot be detected
Solution Approach 1:
The patent uses a surfel map as an intermediary data structure to bridge the gap between limited sensor detection and broader environmental awareness. The surfel map stores preprocessed environmental information from multiple sources and time points, allowing the system to infer the presence and state of objects outside the current sensor field of view by comparing surfel data with camera images, thereby extending effective detection coverage without requiring physical sensors in all directions.
Solution Approach 2:
The system performs preliminary actions by pre-building and maintaining surfel maps of the environment using data from previous time points and multiple vehicles. This advance preparation of environmental models allows the autonomous vehicle to predict and plan for objects that are currently outside sensor range, enabling proactive detection and planning rather than reactive response to newly detected objects.
2Area of stationary object
If lidar sensor system covers wider field of view to detect more regions, then detection coverage is improved, but computational complexity increases
Solution Approach 1:
The patent segments the environmental data processing by dividing the scene into discrete surfel elements, each representing a small surface patch with associated properties. This segmentation allows the system to process environmental information in manageable units rather than handling entire point clouds or images at once, reducing computational complexity while maintaining comprehensive coverage through efficient data structures and algorithms.
Solution Approach 2:
The system creates simplified copies of environmental data in the form of surfel representations, which are computationally efficient approximations of the full sensor data. These surfel copies store only essential information (position, orientation, surface properties) rather than complete raw sensor measurements, allowing rapid comparison and processing while preserving the ability to detect and reason about objects across wide fields of view.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.