LiDAR Object Detection Using Virtual Data Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional LiDAR-based deep learning object detection systems face performance degradation due to class imbalance in training data, where models focus on dominant classes, leading to poor detection reliability for less represented classes and unrealistic data augmentation issues with existing ground-truth sampling techniques.
Innovation Solution
A method that uses a multi-head deep learning network with a shared backbone, where virtual objects are added to the original datasets based on object-associated position information, and the Dynamic Weight Average (DWA) method is applied to calculate a weighted sum of losses for each class, ensuring balanced learning and realistic data augmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a multi-head deep learning network with shared backbone is used to detect multiple object classes, then detection coverage is improved, but gradient imbalance occurs causing poor performance on minority classes
Solution Approach 1:
The patent applies preliminary action by performing data augmentation through virtual object addition before the training process begins. Virtual objects of minority classes are synthesized and inserted into training datasets in advance, ensuring balanced class representation before model training starts. This pre-processing step prevents gradient imbalance from occurring during training, allowing the multi-head network to learn all classes equally without requiring complex gradient balancing mechanisms during the training process itself.
2Quantity of substance
If ground-truth sampling technique is used to add virtual objects, then class imbalance is reduced, but objects are added in unrealistic positions reducing performance improvement
Solution Approach 1:
The patent applies local quality by making the data augmentation process context-aware and location-specific. Instead of uniformly distributing virtual objects throughout the scene, the system determines appropriate placement locations based on local environmental context - such as placing pedestrians on sidewalks or roads, vehicles on roads, and considering occlusion relationships with existing objects. This localized, context-sensitive approach ensures that virtual objects are added in realistic positions that match the local scene characteristics, thereby maintaining detection performance while still addressing class imbalance.
3Quantity of substance
If virtual objects are added in arbitrary positions to balance class numbers, then class imbalance is addressed, but unrealistic data augmentation limits performance improvement
Solution Approach 1:
The patent introduces an intermediary component - the location determination module - that acts as a mediator between the virtual object generation process and the final scene composition. This intermediary evaluates environmental context, identifies suitable placement locations, and ensures that virtual objects are inserted at realistic positions before being added to the training dataset. This intermediary layer maintains the balance of class numbers while simultaneously ensuring the realism of data augmentation, resolving the contradiction between quantity balance and manufacturing ease (realism).
Data Source
AI summary
The machine learning method for a LiDAR-based deep learning object perception apparatus comprises preparing an original dataset of a LiDAR point cloud; acquiring virtual object datasets from a point cloud database; adding the virtual object datasets to the original dataset based on an association of position information for each object to acquire a learning dataset; and training the LiDAR-based deep learning object perception apparatus using the learning dataset.