Semantic Video Compression for Compute Offloading Over Impaired Links
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cellular connectivity faces challenges such as bad coverage, low throughput, and dynamic network load, making it difficult to provide high-fidelity services to lightweight devices like XR glasses, which are compute-intensive and power-needy, especially for tasks like object detection and gesture recognition.
Innovation Solution
Implementing channel-aware semantic coding (CASC) that combines semantic source coding and channel coding to adapt to channel conditions, using multi-priority transmission (MPT) and hierarchical modulation simultaneous transmission (HMST) to optimize power consumption and bandwidth usage, and separating encoded data into high and low priority parts for efficient transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If compute-intensive tasks are performed locally on lightweight devices, then processing capability is improved, but power consumption increases
Solution Approach 1:
The patent segments video data into different priority levels (high priority and low priority portions) based on semantic importance. High priority portions containing critical semantic information are transmitted with higher reliability, while low priority portions are transmitted with lower reliability or compressed more aggressively. This segmentation allows the system to offload processing to remote devices while maintaining acceptable performance with reduced power consumption compared to full local processing.
Solution Approach 2:
The patent dynamically changes encoding parameters such as compression ratio, priority assignment, and transmission reliability based on channel conditions and device capabilities. When channel conditions are poor or device power is limited, the system adjusts parameters to transmit less data with higher compression. When conditions improve, it transmits more detailed data, thereby optimizing the trade-off between processing capability and power consumption.
2Use of energy by moving object
If uncompressed or lightly compressed source data is offloaded to remote devices, then local power consumption is reduced, but bandwidth consumption and transmit power increase
Solution Approach 1:
The patent dynamically adjusts compression parameters based on channel conditions. When channel capacity is limited or transmit power is constrained, the system increases compression ratio to reduce bandwidth consumption. When channel conditions improve, it reduces compression to maintain higher fidelity, thereby optimizing the trade-off between local power savings and bandwidth consumption.
Solution Approach 2:
The patent divides video data into priority-based segments and applies different compression strategies to each segment. High priority portions are compressed less to preserve critical information, while low priority portions are compressed more aggressively. This selective segmentation reduces overall bandwidth consumption while maintaining essential visual quality, allowing lightweight devices to offload processing without excessive bandwidth usage.
3Quantity of substance
If high compression is applied to video data, then bandwidth usage is reduced, but inference fidelity at the decoder decreases
Solution Approach 1:
The patent applies different quality levels to different portions of video data based on their semantic importance. High priority portions containing critical semantic information (such as key objects or actions) are encoded with higher quality and less compression, while low priority portions (such as background or less important details) are encoded with lower quality and higher compression. This local quality differentiation reduces overall bandwidth usage while preserving inference fidelity for the most important visual elements.
4Productivity
If multiple priority levels are transmitted simultaneously, then transmission efficiency is improved, but device complexity increases
Solution Approach 1:
The patent segments video data into priority levels and transmits them through different channel coding schemes. High priority data uses more robust coding with higher redundancy, while low priority data uses less robust coding with lower redundancy. This segmentation enables simultaneous transmission of multiple priority levels with different reliability requirements, improving overall transmission efficiency by matching coding resources to data importance while managing encoding complexity through standardized priority categories.
Data Source
AI summary
Methods and systems for semantic encoding by a user equipment (UE) are configured for determining an expected power consumption for encoding video data that includes semantic features, the semantic features representing a meaning of information represented in video frames of the video data; encoding one or more video frames of the video data using a selected a semantic representation of one or more video frames of the video data, the semantic representation being selected based on the expected power consumption that is determined; and transmitting the encoded video data including the semantic representation.