NPU Post-Processing for Bounding Box Filtering and Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) rely on external central processing units (CPUs) for post-processing operations like filtering and deduplication, leading to significant latency and power consumption due to memory access delays.
Innovation Solution
A post-processing unit (PPU) is integrated with the NPU to perform these operations efficiently, reducing the need for external CPU involvement and minimizing memory access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If post-processing operations are performed by external CPU using external memory, then device complexity is reduced, but processing speed and power efficiency deteriorate due to memory access latency
Solution Approach 1:
The patent merges the post-processing unit with the neural processing unit by integrating shared memory resources and interconnect structures. The post-processing unit accesses bounding box data directly from the NPU's output buffer through the existing interconnect network, eliminating the need for separate external memory access. This consolidation reduces memory access latency while maintaining manageable device complexity through resource sharing.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of a shared buffer memory and interconnect network that mediates data transfer between the NPU and post-processing unit. This intermediary structure allows efficient data exchange without requiring full external memory access, reducing latency while keeping the overall system architecture manageable.
2Device complexity
If post-processing operations are performed by external CPU using external memory, then device complexity is reduced, but power consumption increases due to frequent memory access
Solution Approach 1:
The patent merges the post-processing unit with the neural processing unit by integrating shared memory resources and interconnect structures. The post-processing unit accesses bounding box data directly from the NPU's output buffer through the existing interconnect network, eliminating the need for separate external memory access. This consolidation reduces memory access latency while maintaining manageable device complexity through resource sharing.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of a shared buffer memory and interconnect network that mediates data transfer between the NPU and post-processing unit. This intermediary structure allows efficient data exchange without requiring full external memory access, reducing latency while keeping the overall system architecture manageable.
3Speed
If post-processing unit is integrated with NPU, then processing speed and power efficiency improve, but device complexity increases
Solution Approach 1:
The patent merges the post-processing unit with the neural processing unit by integrating shared memory resources and interconnect structures. The post-processing unit accesses bounding box data directly from the NPU's output buffer through the existing interconnect network, eliminating the need for separate external memory access. This consolidation reduces memory access latency while maintaining manageable device complexity through resource sharing.
Solution Approach 2:
The patent implements multi-functionality where the shared memory and interconnect structures serve both the NPU and post-processing unit. The same memory resources are used for different purposes (training data storage, bounding box output, post-processing input), reducing the need for dedicated components and thereby limiting the increase in device complexity.
4Ease of operation
If post-processing operations use external memory, then ease of operation is maintained, but processing time increases due to memory access latency
Solution Approach 1:
The patent merges the post-processing unit with the neural processing unit by integrating shared memory resources and interconnect structures. The post-processing unit accesses bounding box data directly from the NPU's output buffer through the existing interconnect network, eliminating the need for separate external memory access. This consolidation reduces memory access latency while maintaining manageable device complexity through resource sharing.
Data Source
AI summary
According to one example of the present disclosure, a post-processing unit may be provided. The post-processing unit may be implemented in register transfer level (RTL) code and designed to interface with a neural processing unit (NPU) configured for object detection computations of a neural network model. The post-processing unit may include a processing unit configured to filter a plurality of bounding boxes transmitted from the NPU and output only those that satisfy a particular condition and one or more input registers configured to store data output from the processing unit.