Adjustable Virtual Camera Model for Autonomous Driving Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous driving systems rely on costly and error-prone sensors, such as radar and Lidar, and lack accuracy in detecting and classifying moving and stationary objects, leading to deficient performance.
Innovation Solution
A vision-based machine learning model that uses image sensors to detect objects by projecting them into a virtual camera space, with separate branches for vulnerable road users and non-vulnerable road users, reducing sensor complexity and enhancing accuracy through end-to-end training and separate processing of VRU and non-VRU objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If costly sensors like radar and Lidar are used, then detection capability is improved, but system cost and complexity increase
Solution Approach 1:
The patent creates a virtual camera space that copies and transforms real-world sensor data into an alternative representation. By projecting 3D point cloud data into a virtual camera coordinate system, the system replicates the detection functionality of complex sensors using processed image data from simpler cameras, achieving similar measurement precision without the cost and complexity of radar and Lidar hardware
Solution Approach 2:
The patent replaces mechanical sensor systems (radar and Lidar hardware) with a computational approach using vision-based machine learning models. Instead of relying on electromagnetic wave transmission and reception mechanisms, the system uses image processing algorithms and neural networks to detect and classify objects, substituting physical sensing mechanisms with information processing methods
2Measurement precision
If multiple sensor types are integrated, then detection accuracy is improved, but system cost increases
Solution Approach 1:
The patent makes the vision-based machine learning model perform multiple detection functions that would traditionally require different sensor types. The single vision system simultaneously detects vulnerable road users, non-vulnerable road users, and classifies them with high accuracy, replacing the need for separate radar and Lidar systems with one multi-functional vision processing system
Solution Approach 2:
The patent merges the detection and classification functions for different object types (VRU and non-VRU) into a unified processing framework. By combining multiple detection tasks into a single end-to-end trained model, the system achieves accurate detection of various objects using only image sensors, reducing the total quantity of sensors needed while maintaining or improving detection accuracy
3Measurement precision
If separate processing branches for VRU and non-VRU are implemented, then classification accuracy is improved, but model complexity increases
Solution Approach 1:
The patent segments the object detection and classification process into separate branches for vulnerable road users and non-vulnerable road users. This segmentation allows each branch to specialize in detecting and classifying specific object types, improving classification accuracy by focusing computational resources on particular categories while maintaining an integrated overall system architecture
Data Source
AI summary
Systems and methods for a vision-based machine learning model for autonomous driving with adjustable virtual camera. An example method includes obtaining images from a multitude of image sensors positioned about a vehicle. Features associated with the images are determined, with the features being output based on a forward pass through a first portion of a machine learning model. The features are projected into a vector space associated with a virtual camera at a particular height. The projected features are aggregated with other projected features associated with prior images. A plurality of objects which are positioned according to the virtual camera are determined.