UAV Scene Compression With Generative Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Unmanned aerial vehicles (UAVs) face challenges in efficiently conveying detailed aerial images for navigation and obstacle avoidance due to the high data volume, which is costly and resource-intensive, necessitating a more efficient method to communicate and process scene data for safe and intelligent navigation.
Innovation Solution
A generative neural network (GNN) is trained on a sparse set of aerial images captured from different vantage points, encoding a volumetric representation of the scene, allowing for efficient communication of scene data to a backend datacenter, and enabling generation of novel views and improved localization, obstacle avoidance, and decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If detailed aerial images are conveyed to backend datacenter for scene modeling, then navigation and obstacle avoidance accuracy is improved, but bandwidth consumption and resource usage increase significantly
Solution Approach 1:
The patent extracts only the essential scene information needed for navigation and obstacle avoidance by training a neural network on sparse aerial images. Instead of transmitting all raw image data, the system extracts and transmits only the compressed scene representation that contains the necessary spatial and contextual information, significantly reducing data volume while maintaining modeling accuracy.
Solution Approach 2:
The patent transforms the scene representation from raw image data to a compressed neural network parameter format. By changing the parameter representation from pixels to learned features, the system achieves efficient compression that reduces bandwidth consumption while preserving the essential geometric and semantic information needed for safe navigation.
2Loss of information
If more aerial images are captured from multiple vantage points, then scene model completeness is improved, but data transmission cost and processing resources increase
Solution Approach 1:
The patent applies partial action by using a sparse set of aerial images rather than capturing images from every possible vantage point. The neural network is trained on this partial data set, which is sufficient to reconstruct accurate novel views and achieve complete scene coverage without the excessive resource consumption of capturing and processing all possible images.
Solution Approach 2:
The patent transitions from the image space domain to the neural network parameter space, adding a new dimension for scene representation. This dimensional transformation allows the system to achieve complete scene coverage through learned representations that generalize across different viewpoints, reducing the need for exhaustive image capture from multiple vantage points.
3Measurement precision
If full-resolution aerial images are transmitted to backend, then image quality for analysis is improved, but bandwidth requirements and transmission time increase
Solution Approach 1:
The patent creates a compressed copy of the scene data in the form of trained neural network parameters. Instead of transmitting the original high-resolution images, the system transmits the learned parameter representations that can be used to reconstruct images and analyze scenes, achieving both image quality and transmission efficiency through this parameter-based copying approach.
Data Source
AI summary
A technique performed by UAV delivery system includes: arriving by a UAV of over a destination area; capturing a plurality of aerial images of a scene at the destination area with an onboard camera system of the UAV while flying above the destination area, wherein the aerial images capture the scene from a plurality of UAV vantage points offset from each other; optimizing weights of a generative neural network (GNN) using at least some of the aerial images as a training dataset to encode a volumetric representation of the scene into the GNN, wherein the weights are optimized by an onboard processing system of the UAV; and communicating the GNN with the weights optimized to a backend datacenter in communication with the UAV to transmit the volumetric representation of the scene over which the UAV flew without transmitting the aerial images themselves to the backend datacenter.