Auxiliary CNN Input Channels for Efficient Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation techniques for autonomous driving systems are computationally demanding and inefficient, particularly when using multiple cameras, and require costly multiple forward and backward passes, while end-to-end cost volume methods are non-trivial and difficult to implement on hardware.
Innovation Solution
Combining odometry information with image data to form a single input component structured according to the stride of a neural network, reducing the number of channels and overhead by incorporating multiple auxiliary values into a single component, such as X, Y, Z positions and yaw, to enhance depth estimation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple auxiliary information sets are provided as distinct components to the neural network, then the neural network can access complete auxiliary data, but memory consumption and processing operations increase
Solution Approach 1:
The patent combines multiple auxiliary information sets (odometry data, camera intrinsic data, extrinsic data) into a single auxiliary component that is processed together with image data through convolution operations. This merging reduces the number of separate data streams from multiple distinct components to one unified component, thereby reducing memory consumption and processing overhead while maintaining access to all necessary auxiliary information for depth estimation
2Reliability
If multiple auxiliary information sets are provided as distinct components to the neural network, then the neural network can access complete auxiliary data, but processing operations and power consumption increase
Solution Approach 1:
The patent merges multiple auxiliary information sets into a single auxiliary component that undergoes convolution operations with image data. This unified processing approach reduces the number of separate processing pipelines, decreasing computational operations and power consumption while maintaining depth estimation accuracy through the preserved spatial and contextual relationships in the combined component
3Reliability
If auxiliary information is provided in traditional multi-component format, then data completeness is maintained, but hardware implementation becomes complex and non-trivial
Solution Approach 1:
The patent combines multiple auxiliary information sets into a single auxiliary component that can be processed using standard convolutional neural network operations. This unified structure simplifies hardware implementation by eliminating the need for complex multi-stream processing architectures, making the system more suitable for deployment on autonomous driving hardware while maintaining complete auxiliary data for accurate depth estimation
Data Source
AI summary
An example device for processing image data includes a memory configured to store image data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: obtain an image to be processed; obtain a first auxiliary value for the image and a second auxiliary value for the image; generate an input component including the first auxiliary value and the second auxiliary value arranged in a pattern according to a stride of a neural network; and provide the image and the input component to the neural network.