Compressed Video Recognition Across Edge-Cloud CNN Stages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual computing approaches inefficiently utilize resources, leading to high latency, inaccuracy, and inflexibility due to reliance on cloud processing, which consumes significant bandwidth and requires full decompression before analysis.
Innovation Solution
A visual fog computing system that leverages both edge and cloud resources to perform visual computing tasks, utilizing a multi-domain convolutional neural network (CNN) for efficient processing and storage, enabling flexible design and scalable ad-hoc queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If images and video are compressed and transmitted to the cloud for processing, then bandwidth is saved during transmission, but full decompression is required before any processing which increases latency and consumes more bandwidth
Solution Approach 1:
The patent segments the visual processing pipeline into multiple domains (compressed domain, decompressed domain, feature domain) that can operate independently or in combination. This allows partial processing in the compressed domain without full decompression, reducing latency while maintaining bandwidth efficiency.
Solution Approach 2:
The system performs preliminary processing operations directly on compressed visual data before full decompression. By executing certain analytics and processing tasks in advance on the compressed domain, the system reduces the amount of data that needs to be fully decompressed and transmitted, thereby reducing both latency and bandwidth consumption.
2Measurement precision
If full decompression is performed before processing, then processing accuracy is improved, but bandwidth is wasted transmitting insignificant visual data
Solution Approach 1:
The patent applies different processing qualities to different parts of the visual data pipeline. Critical processing tasks that require high accuracy are performed in the decompressed or feature domain, while less critical tasks are performed in the compressed domain. This localized quality approach ensures accuracy where needed while saving bandwidth overall.
Solution Approach 2:
The system dynamically changes processing parameters based on the domain and task requirements. By adjusting the level of decompression and processing intensity according to the specific analytical task, the system optimizes the balance between accuracy and bandwidth consumption for each processing operation.
3Power
If cloud processing is used for all visual tasks, then processing power is sufficient, but resource utilization is inefficient and the system is inflexible
Solution Approach 1:
The patent segments the processing architecture into edge devices, fog nodes, and cloud resources. This segmentation allows visual processing tasks to be distributed across multiple levels of the hierarchy, enabling efficient resource utilization by performing appropriate tasks at the appropriate level rather than centralizing all processing in the cloud.
Solution Approach 2:
The system creates a universal processing framework that can handle multiple types of visual analytics tasks across different domains (compressed, decompressed, feature). This multi-functional architecture allows the same system to efficiently perform various processing tasks using appropriate resources, improving overall resource utilization and system flexibility.
4Device complexity
If a single-domain processing approach is used, then system complexity is reduced, but processing accuracy and reliability are compromised
Solution Approach 1:
The patent merges multiple processing domains (compressed domain processing, decompressed domain processing, and feature domain processing) into a unified multi-domain CNN framework. This combination allows the system to leverage the strengths of each domain while maintaining a cohesive architecture, thereby improving reliability without excessive complexity.
Solution Approach 2:
The system implements a nested architecture where compressed domain processing operates within the broader framework of decompressed domain processing, which in turn operates within the feature domain processing layer. This nested structure allows progressive refinement of processing accuracy while maintaining manageable system complexity through hierarchical organization.
Data Source
AI summary
In one embodiment, an apparatus comprises a communication interface and a processor. The communication interface is to communicate with a plurality of devices. The processor is to: receive compressed data from a first device, wherein the compressed data is associated with visual data captured by sensor(s); perform a current stage of processing on the compressed data using a current CNN, wherein the current stage of processing corresponds to one of a plurality of processing stages associated with the visual data, and wherein the current CNN corresponds to one of a plurality of CNNs associated with the plurality of processing stages; obtain an output associated with the current stage of processing; determine, based on the output, whether processing associated with the visual data is complete; if the processing is complete, output a result associated with the visual data; if the processing is incomplete, transmit the compressed data to a second device.