NPU Post-Processing Circuit for Low-Latency Bounding Box Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units face inefficiencies in data transmission and power consumption due to the need for external CPUs to perform post-processing operations, leading to delays and increased latency in neural network computations.
Innovation Solution
Incorporation of a post-processing unit within the neural processing unit (NPU) to perform operations such as non-maximum suppression and class selection directly on the data output, reducing the need for external processing and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If post-processing operations are performed using external CPUs and memory access, then processing flexibility is maintained, but processing time increases and power consumption rises
Solution Approach 1:
The patent merges the post-processing unit with the neural processing unit to form an integrated processing system. The post-processing unit is directly coupled to the neural processing unit, allowing convolution operations and post-processing operations to be performed in close proximity without data transmission over buses. This integration eliminates the need for external memory access during post-processing, thereby reducing power consumption and increasing processing speed.
Solution Approach 2:
The patent introduces an intermediate buffer memory located between the neural processing unit and post-processing unit. This buffer temporarily stores convolution output data, enabling the post-processing unit to access data locally without requiring external memory accesses. The buffer acts as an intermediary that reduces both power consumption and access latency.
2Productivity
If data is transmitted over buses between processing units, then system modularity is maintained, but processing delays occur
Solution Approach 1:
The patent combines the neural processing unit and post-processing unit into an integrated system where data can be passed directly between units without traversing external buses. This direct coupling eliminates transmission delays while maintaining the functional separation and modularity of the two processing units.
Solution Approach 2:
The patent implements a buffer memory that pre-loads convolution output data before post-processing operations begin. This preliminary action ensures that data is readily available when needed, eliminating waiting time and processing delays during the post-processing stage.
Data Source
AI summary
According to one example of the present disclosure, the neural processing unit may comprise a processing element array configured to perform operations of a neural network model and a post-processing unit configured to process data output from the processing element array. The post-processing unit includes a first computation circuit that extracts a subset of classes for each bounding box by comparing class scores of classes and a second computation circuit configured to extract one or more bounding boxes by comparing a class confidence score of each bounding box with a threshold confidence score.