Deep Learning Accelerator Sensor Fusion with RAM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion technologies for Artificial Neural Networks (ANNs) face challenges in reducing energy consumption and computation time, particularly when processing data from multiple sensors, leading to inefficiencies in generating accurate and timely outputs.
Innovation Solution
An integrated circuit with a Deep Learning Accelerator (DLA) and random access memory is designed to perform sensor fusion using ANNs, optimizing matrix computations and coordinating timing of intermediate results from different sensors, thereby reducing overlapping processing and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data from multiple sensors is processed using traditional sensor fusion technologies, then accurate outputs can be generated, but energy consumption increases and computation time extends
Solution Approach 1:
The patent divides the sensor fusion processing into separate deep learning networks for each sensor type (e.g., camera network, radar network, lidar network). Each network independently processes data from its corresponding sensor, eliminating the need for a single large network to process all sensor data simultaneously. This segmentation reduces the computational load and energy consumption while maintaining output accuracy through coordinated processing of intermediate results.
2Measurement precision
If sensor data from multiple sensors is processed using traditional sensor fusion technologies, then accurate outputs can be generated, but computation time extends
Solution Approach 1:
The patent performs preliminary processing of sensor data through separate deep learning networks before final fusion. Each sensor's data is pre-processed independently to extract relevant features and generate intermediate results, which are then coordinated and combined. This preliminary action reduces the complexity of the final fusion step and accelerates overall computation time while preserving accuracy.
3Use of energy by moving object
If multiple deep learning networks process sensor data separately, then energy consumption is reduced, but coordination of intermediate results becomes complex
Solution Approach 1:
The patent introduces a coordination mechanism that acts as an intermediary between separate deep learning networks. This coordinator collects intermediate results from various sensor networks, aligns their timing and spatial references, and integrates them into a unified output. This intermediary structure simplifies the complexity of coordinating multiple networks while maintaining energy efficiency through their independent operation.
4Ease of manufacture
If traditional processing architectures are used, then implementation is straightforward, but data access bottlenecks reduce efficiency
Solution Approach 1:
The patent transitions from traditional sequential processing architecture to a parallel distributed architecture where multiple deep learning networks operate simultaneously on different sensor data streams. This dimensional change in processing architecture allows concurrent execution of multiple processing tasks, eliminating data access bottlenecks and improving overall processing efficiency while maintaining implementation feasibility through modular design.
Data Source
AI summary
Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory. The random access memory is configured to store a plurality of inputs from a plurality of sensors respective, parameters of an Artificial Neural Network, and instructions executable by the Deep Learning Accelerator to perform matrix computation to generate outputs of the Artificial Neural Network, including first outputs generated using the sensors separately and a second output generated using a combination of the sensors.