Deep Learning Accelerator Object Detection Without NMS Post-Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection systems using Artificial Neural Networks (ANNs) face challenges in achieving high performance comparable to Graphics Processing Units (GPUs) while minimizing energy consumption and computation time, particularly in image processing tasks.
Innovation Solution
Implementing an object detector using a Deep Learning Accelerator (DLA) with cross stage partial networks for feature extraction and fusion, combined with minimum cost assignment in object classification, and utilizing a compiler to generate instructions for efficient matrix computations, reducing reliance on CPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If object detection is implemented using traditional CPUs or GPUs, then detection performance can be achieved, but energy consumption and computation time increase significantly
Solution Approach 1:
The patent segments the object detection task into distinct functional modules: backbone network for feature extraction, neck network for feature fusion, and head network for object classification. Each module is implemented as a separate processing stage in the deep learning accelerator, allowing independent optimization and efficient resource utilization, thereby achieving high detection performance with reduced energy consumption compared to monolithic GPU implementations.
Solution Approach 2:
The patent introduces a specialized deep learning accelerator as an intermediary device between the input image data and the object detection output. This accelerator includes dedicated hardware components such as convolutional neural network processors, fully connected layer processors, and feature fusion units that are specifically designed to optimize neural network computations, reducing both energy consumption and computation time compared to general-purpose CPUs or GPUs.
2Productivity
If object detection is implemented using traditional CPUs or GPUs, then detection functionality is provided, but computation time increases
Solution Approach 1:
The patent implements preliminary action through the backbone network that performs feature extraction from input images before the main detection process. The extracted features are stored and reused in subsequent processing stages, eliminating the need to reprocess raw pixel data multiple times. This preliminary feature extraction significantly reduces computation time in the neck and head networks while maintaining detection performance.
Solution Approach 2:
The patent ensures continuity of useful action through the neck network's feature fusion mechanism, which continuously integrates features from multiple backbone network stages. The fused features are immediately passed to the head network for object classification without interruption, creating a continuous processing pipeline that maximizes computational efficiency and reduces overall detection time compared to discrete, interrupt-driven processing in traditional systems.
3Measurement precision
If cross stage partial networks are used for feature extraction and fusion, then detection accuracy improves, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the complex cross-stage partial network into modular components: backbone modules for feature extraction at different resolutions, neck modules for feature fusion, and head modules for object prediction. Each module has a specialized function and can be independently configured, making the overall complex network manageable and implementable in hardware while maintaining high detection accuracy.
Solution Approach 2:
The patent utilizes another dimension by implementing multi-scale feature extraction in the backbone network, processing images at different resolutions simultaneously. The neck network then fuses features across these dimensional scales, allowing the system to detect objects of varying sizes effectively. This dimensional approach increases detection accuracy without proportionally increasing device complexity, as the additional processing occurs in parallel rather than sequentially.
4Measurement precision
If minimum cost assignment is used in object classification, then detection accuracy improves, but computation complexity increases
Solution Approach 1:
The patent implements self-service through the minimum cost assignment mechanism in the head network, where each detected object automatically assigns itself to the most appropriate class based on computed costs. The system calculates assignment costs between detected objects and ground truth labels, then automatically selects the minimum cost assignment without external intervention. This self-service approach improves classification accuracy while keeping the computation complexity manageable through efficient cost calculation algorithms.
Data Source
AI summary
Systems, devices, and methods related to an object detector and a deep learning accelerator are described. For example, a computing apparatus has an integrated circuit device with the deep learning accelerator configured to execute instructions generated by a compiler from a description of an artificial neural network of the object detector. The artificial neural network includes a first cross stage partial network to extract features from an image and a second cross stage partial network to combine the features to identify a region of interest in the image showing an object. The artificial neural network uses a technique of minimum cost assignment in assigning a classification to the object and thus avoids post processing of non-maximum suppression.