LiDAR Point Selection via Image Segmentation Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing techniques face challenges in accurately disaggregating and classifying overlapping objects in 2D space for transformation into 3D space, particularly in autonomous vehicle navigation, due to occlusions and close proximities between image objects.
Innovation Solution
The use of machine-learning models, such as neural networks, to segment images and select specific points for processing with LiDAR data, enabling accurate distance estimations by placing bounding boxes and image masks to isolate objects and associate classification labels, thereby reducing processing power and improving object recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image processing techniques are used to identify objects in 2D space, then object detection can be performed, but accurate disaggregation and classification of overlapping objects becomes difficult due to occlusions and close proximities
Solution Approach 1:
The patent applies segmentation by dividing the image into multiple segments or regions, each potentially containing a single object or a portion of an object. This is achieved through machine learning models that predict segmentation masks, which separate overlapping objects in 2D space before transformation to 3D, thereby resolving occlusions and enabling accurate classification of each object independently.
Solution Approach 2:
The patent transforms objects from 2D image space to 3D space using depth information from LiDAR data. By adding the depth dimension, objects that appear overlapping in 2D can be separated in 3D, allowing accurate disaggregation and classification. The system selects points from segmented 2D regions and associates them with 3D LiDAR points to reconstruct object positions in three-dimensional space.
2Measurement precision
If machine-learning models are used to segment images and select specific points for LiDAR processing, then distance estimation accuracy is improved, but processing power requirements increase
Solution Approach 1:
The patent extracts only the necessary information from the image by selecting specific points within segmented regions rather than processing entire images or all pixels. The machine learning model identifies and extracts key points from segmented object regions, which are then matched with LiDAR points for distance estimation. This selective extraction reduces the volume of data requiring complex processing while maintaining accuracy.
Solution Approach 2:
The patent applies partial action by processing only certain portions of the image (segmented regions containing objects of interest) rather than the entire image. The system selectively applies machine learning segmentation and LiDAR matching only to relevant regions identified in the image, reducing overall processing power requirements while maintaining distance estimation accuracy for critical objects.
3Ease of operation
If bounding boxes are used to identify image regions, then object locations can be identified, but accurate separation of overlapping objects is achieved
Solution Approach 1:
The patent replaces conventional bounding box methods with segmentation-based object separation. Instead of using single rectangular bounding boxes that encompass entire objects including occluding portions, the system generates segmentation masks that precisely outline object boundaries. This allows overlapping objects to be separated at pixel level, enabling accurate identification of each object's true location and shape even when they overlap in the image.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the accuracy of distance estimations and object classification in 3D space, particularly for autonomous vehicles, by effectively handling occlusions and reducing processing power, leading to improved navigation and guidance systems.
Implementation Method 1
selecting a second set of points from a second image based on a second set of depth values associated with the first set of points... processing the selected points with Light Detection and Ranging (LiDAR) data
Data Source
AI summary
The subject disclosure relates to techniques for selecting points of an image for processing with LiDAR data. A process of the disclosed technology can include steps for receiving an image comprising a first image object and a second image object, processing the image to place a bounding box around the first image object and the second image object, and processing an image area within the bounding box to identify a first image mask corresponding with a first pixel region of the first image object and a second image mask corresponding with a second pixel region of the second image object. Systems and machine-readable media are also provided.