NPU Feature Map Transceiving With Selective Enhancement Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies lack effective methods for image analysis by machines, particularly in the context of high-definition and ultra-high-definition video processing and the expanding applications of artificial intelligence.
Innovation Solution
A neural processing unit (NPU) is developed to decode and encode video and feature maps, incorporating processing elements that can selectively process enhancement layers based on available bandwidth and machine analysis tasks, adjusting the number of layers according to transmission channel conditions and feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all enhancement layers are decoded and processed, then image analysis quality is improved, but processing complexity and bandwidth requirements increase
Solution Approach 1:
The patent segments the enhancement layers into multiple groups (first group and second group) with different processing priorities. The first group contains layers essential for basic image analysis tasks, while the second group contains layers for enhanced quality. This segmentation allows the system to process only necessary layers based on task requirements, reducing processing complexity while maintaining adequate image analysis quality.
Solution Approach 2:
The patent implements partial action by selectively decoding only the first group of enhancement layers when bandwidth is limited or processing capacity is constrained. This partial processing approach provides sufficient quality for many machine analysis tasks without the full computational overhead of processing all enhancement layers, effectively balancing quality and complexity.
2Measurement precision
If all enhancement layers are transmitted over the network, then image quality is improved, but bandwidth consumption increases
Solution Approach 1:
The patent extracts and transmits only the essential first group of enhancement layers over the network when bandwidth is constrained, separating them from the second group of enhancement layers that are processed locally or skipped. This extraction approach reduces network bandwidth consumption while maintaining sufficient image quality for machine analysis tasks.
Solution Approach 2:
The patent applies partial action by transmitting only a subset of enhancement layers (the first group) rather than all layers. This partial transmission reduces bandwidth consumption significantly while providing adequate image quality for many practical machine analysis applications, avoiding the excessive bandwidth requirement of transmitting complete high-resolution data.
3Measurement precision
If the NPU processes all enhancement layers, then analysis accuracy is improved, but processing time increases
Solution Approach 1:
The patent segments enhancement layers into priority groups, allowing the NPU to process only the first group of enhancement layers for time-sensitive machine analysis tasks. This segmentation enables the system to achieve acceptable analysis accuracy within limited time constraints by focusing computational resources on the most critical layers.
Solution Approach 2:
The patent implements partial processing by decoding and analyzing only the first group of enhancement layers when processing time is constrained. This partial action approach provides sufficient analysis accuracy for real-time or near-real-time machine analysis applications without the time penalty of processing all enhancement layers.
Data Source
AI summary
A neural processing unit (NPU) for decoding video and/or feature map may include at least one processing element (PE) for an artificial neural network (ANN), the at least one PE to receive and decode a bitstream. The bitstream is received in units of frames, and one frame includes a weight for an ANN model, data of a base layer, and data of a plurality of enhancement layers. An NPU for encoding video and/or feature map may include at least one processing element (PE) for an artificial neural network (ANN), the at least one PE to encode an input video or feature map and to transmit the encoded input video or feature map as a bitstream. The at least one PE transmits the bitstream in units of frames, and one frame includes a weight for an ANN model, data of a base layer, and data of a plurality of enhancement layers.