Distributed CNN Sensor Fusion for Lower Bandwidth Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing early fusion networks in autonomous driving systems face challenges with high memory and processing demands due to centralized processing, inefficient use of edge processing, and complex programming models, which can be addressed by distributing convolutional neural network layers across edge and data-network processors.
Innovation Solution
Distribute convolutional neural network layers for preprocessing and feature extraction at edge sensors and gateway nodes, reducing data transmission to a centralized processor, and using specialized ASICs for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single centralized neural network processes all sensor data, then detection accuracy is improved, but memory requirements and processing throughput increase significantly
Solution Approach 1:
The patent divides the centralized neural network into distributed neural networks deployed across multiple edge sensors. Each edge sensor executes a portion of the convolutional neural network layers locally, segmenting the processing task from the centralized processor. This segmentation reduces the memory burden on any single device while maintaining the early fusion detection accuracy benefits.
2Measurement precision
If a single centralized neural network processes all sensor data, then detection accuracy is improved, but processing throughput requirements increase
Solution Approach 1:
The patent segments the computationally intensive convolutional neural network layers across distributed edge sensors, allowing parallel processing of sensor data at the edge. This distributes the processing throughput requirement across multiple devices rather than concentrating it in a single centralized processor, reducing the peak throughput demand on the central system.
3Measurement precision
If raw sensory data is transmitted to a centralized processor, then early fusion detection accuracy is maintained, but network bandwidth requirements increase
Solution Approach 1:
The patent applies preliminary action by executing convolutional neural network preprocessing and feature extraction at the edge sensors before data transmission. This preliminary processing at the source reduces the volume of data that needs to be transmitted over the network, as only processed features rather than raw sensor data are sent to the centralized processor.
4Adaptability or versatility
If general-purpose processors are used for centralized processing, then system flexibility is maintained, but processing efficiency decreases
Solution Approach 1:
The patent introduces specialized neural network processors or accelerators as intermediary components between the general-purpose centralized processor and the edge sensors. These intermediary specialized processors handle the computationally intensive neural network operations efficiently, while the general-purpose processor maintains system flexibility and coordination.
5Measurement precision
If all neural network layers are centralized, then detection accuracy is improved, but edge processing efficiency is reduced
Solution Approach 1:
The patent segments the neural network architecture by deploying convolutional layers at edge sensors and reserving fully connected layers for the centralized processor. This segmentation enables edge processing of the most computationally intensive and data-volume-dependent convolutional operations, improving edge processing efficiency while maintaining overall detection accuracy through centralized fusion.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An early fusion network is provided that reduces network load and enables easier design of specialized ASIC edge processors through performing a portion of convolutional neural network layers at distributed edge and data-network processors prior to transmitting data to a centralized processor for fully-connected/deconvolutional neural networking processing. Embodiments can provide convolution and downsampling layer processing in association with the digital signal processors associated with edge sensors. Once the raw data is reduced to smaller feature maps through the convolution-downsampling process, this reduced data is transmitted to a central processor for further processing such as regression, classification, and segmentation, along with feature combination of the data from the sensors. In some embodiments, feature combination can be distributed to gateway or switch nodes closer to the edge sensors, thereby further reducing the data transferred to the central node and reducing the amount of computation performed there.