Automated Driving Data Acquisition Through Learned Scene Representations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The collection of data from a fleet of vehicles for the development of automated driving systems is hindered by bandwidth and storage limitations, as well as the immense need for post-processing to extract relevant datasets, leading to inefficient and costly data transmission.
Innovation Solution
A method involving the learning of a multidimensional scene representation from a set of sensor data frames, which is then used to transmit compressed data, enabling the reconstruction or rendering of sensor data frames at a remote server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If raw sensor data is collected and transmitted from a fleet of vehicles, then data completeness and information quality are improved, but bandwidth requirements and transmission costs increase significantly
Solution Approach 1:
The patent extracts only the essential scene representation information from raw sensor data using neural radiance fields, separating the core semantic content from redundant raw data. This allows transmission of compressed scene representations that retain critical information while dramatically reducing bandwidth requirements.
Solution Approach 2:
The patent transforms raw sensor data into a different parameter space using neural radiance field representations. By changing the data representation from raw sensor readings to learned scene parameters, the system achieves compact encoding that reduces transmission volume while preserving essential scene information for reconstruction.
2Quantity of substance
If sensor data is compressed to reduce transmission size, then bandwidth requirements are reduced, but information loss occurs during compression and decompression
Solution Approach 1:
The patent creates a learned scene representation that acts as a compact copy of the essential scene information. This learned representation can be transmitted efficiently and then used to reconstruct or render sensor data frames, providing a faithful copy of the critical scene content without transmitting the full raw data.
Solution Approach 2:
The patent transitions from transmitting data in the raw sensor dimension to transmitting in the learned representation dimension. By encoding scene information in a compressed latent space and enabling rendering in the original sensor space, the system achieves loss-efficient compression through dimensional transformation.
3Loss of information
If all sensor data from the fleet is uploaded to centralized servers, then data availability for processing is improved, but storage requirements and post-processing complexity increase
Solution Approach 1:
The patent extracts only the necessary scene representation from complete sensor data before transmission. By separating essential scene information from redundant raw data at the source, the system reduces the volume of data requiring centralized storage and post-processing while maintaining availability of critical information.
Solution Approach 2:
The patent performs preliminary processing at the vehicle端 by learning scene representations from raw sensor data before transmission. This preliminary action of encoding scene information into compact representations reduces the burden on centralized servers for both storage and subsequent processing operations.
4Productivity
If compression techniques are applied to sensor data, then transmission efficiency is improved, but the ability to reconstruct complete sensor data frames is degraded
Solution Approach 1:
The patent creates a learned scene representation that serves as a compact copy capable of rendering complete sensor data frames. This learned copy maintains the essential information needed for high-quality reconstruction or rendering of sensor frames, achieving both compression and fidelity.
Solution Approach 2:
The patent enables bidirectional transformation between compressed learned representation space and original sensor data space. By establishing this dimensional bridge through neural radiance fields, the system achieves efficient compression while preserving the ability to reconstruct complete sensor data frames with high quality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to, among other things, a computer-implemented method (100) performed in a vehicle. The method (100) comprising: obtaining (5102) a set of sensor data frames depicting a scene in a surrounding environment of the vehicle from at least two points-of-view; forming (S108) a learned scene representation of the depicted scene, based on the obtained set of sensor data frames, wherein the learned scene representation is a multidimensional representation of the depicted scene; and transmitting (5110) information indicative of the learned scene representation to a remote server for subsequent rendering of sensor data frames based on the learned scene representation.