Activation Bypass and Weight Pruning for Vehicle Perception Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vehicle perception systems using neural networks face inefficiencies in processing resources due to the need for performing multiplication and accumulation operations on zero or near-zero activation values and the overhead of storing weights for all neurons, which consumes significant computational and memory resources.
Innovation Solution
Implementing an activation bypass for values below a threshold, which skips the multiplication and accumulation operation, and performing weight pruning based on thresholds to reduce the number of synapses and neurons, thereby optimizing processing resources and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiplication and accumulation operations are performed on all activation values including zero or near-zero values, then computational completeness is maintained, but processing resources are significantly consumed
Solution Approach 1:
The patent extracts and removes zero or near-zero activation values from the computational pipeline by implementing an activation bypass mechanism. These values are identified and diverted before reaching the multiplication and accumulation units, eliminating unnecessary computations while preserving non-zero values that contribute to the neural network output.
Solution Approach 2:
The patent applies partial action by selectively performing multiplication and accumulation operations only on non-zero activation values. Instead of processing all activation values uniformly, the system performs computations partially - only where needed - by bypassing zero values and their associated weight multiplications, thereby reducing computational overhead.
2Reliability
If weights for all neurons are stored in memory, then model accuracy is maintained, but memory resources are significantly consumed
Solution Approach 1:
The patent extracts and removes weights associated with zero or near-zero activation values from memory storage. By identifying neurons that produce zero activations and removing their corresponding weights from both memory and the computational graph, the system significantly reduces memory requirements while maintaining the functional integrity of the neural network.
Solution Approach 2:
The patent discards weights corresponding to zero or near-zero activation values, effectively removing them from the model. This pruning process discards redundant parameters that contribute minimally to the output, recovering memory space while preserving the essential computational pathways that maintain model accuracy.
3Measurement precision
If all synapses and neurons are maintained in the neural network, then computational accuracy is preserved, but the number of processing operations increases
Solution Approach 1:
The patent extracts and removes synapses connected to neurons that produce zero or near-zero activations. By identifying and eliminating these redundant connections from the computational graph, the system reduces the total number of synapses and neurons that require processing, thereby simplifying the network structure while preserving accuracy-critical pathways.
Solution Approach 2:
The patent changes the structural parameters of the neural network by pruning synapses and neurons based on their activation patterns. This parameter change involves modifying the network topology itself - removing edges and nodes that contribute minimally to the output - thereby reducing computational complexity while maintaining the essential functional parameters that ensure accuracy.
Data Source
AI summary
In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.