AI Upscaling Streaming Terminal Joint DNN
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data streaming technologies face challenges in maintaining Quality of Service (QoS) for real-time image data transmission, especially when network conditions change, and there is a need for adaptive streaming methods that leverage artificial intelligence (AI) encoded data using deep neural networks (DNNs to optimize image quality and resolution.
Innovation Solution
A method and apparatus for streaming AI-encoded data using DNNs, where a server transmits multiple quality versions of image content, and a terminal determines whether to perform AI upscaling or downscaling based on received AI data, adjusting the image quality according to network conditions and terminal capabilities, using jointly trained upscaling and downscaling DNNs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If AI upscaling is performed on the terminal side using jointly trained DNNs, then image quality is improved, but device complexity increases
Solution Approach 1:
The server performs preliminary AI downscaling on the original image before transmission, preparing a compressed version that can be efficiently transmitted. The terminal then performs AI upscaling to reconstruct the image. This preliminary action at the server side reduces the complexity burden on the terminal while maintaining image quality through coordinated pre-processing.
Solution Approach 2:
The patent introduces an intermediary mechanism where AI data (including DNN model parameters and intermediate representation data) is transmitted from the server to the terminal along with the image data. This intermediary AI data enables the terminal to perform accurate AI upscaling without requiring the full complexity of the original processing, thus improving image quality while managing terminal complexity.
2Adaptability or versatility
If multiple quality versions of image content are transmitted, then adaptability to network conditions is improved, but loss of information increases
Solution Approach 1:
The patent segments the image content into multiple quality versions (e.g., high quality, medium quality, low quality) and transmits them separately. The terminal selects the appropriate version based on network conditions. This segmentation allows adaptive transmission without requiring all versions to be sent simultaneously, reducing information loss while maintaining adaptability.
Solution Approach 2:
The patent changes the quality parameter of the transmitted image data based on network conditions. Instead of transmitting all quality versions with equal detail, the system adjusts the quality parameter dynamically - transmitting higher quality versions when bandwidth is available and lower quality versions when bandwidth is constrained. This parameter adjustment reduces unnecessary data transmission overhead while maintaining adaptability.
3Measurement precision
If AI data is transmitted along with image content, then image reconstruction accuracy is improved, but quantity of substance increases
Solution Approach 1:
The patent extracts only the essential AI data needed for reconstruction from the complete AI processing pipeline. Instead of transmitting all intermediate data and full model parameters, the system extracts and transmits only the critical AI data (such as key model parameters or compressed representation data) that is sufficient for accurate image reconstruction at the terminal. This extraction reduces data transmission volume while maintaining reconstruction accuracy.
Data Source
AI summary
A terminal for receiving streaming data may receive information of a plurality of different quality versions of an image content; request, based on the information, a server for a version of the image content from among the plurality of different quality versions of the image content; when the requested version of the image content and artificial intelligence (AI) data corresponding to the requested version of the image content are received, determines whether to perform AI upscaling on the received version of the image content, based on the AI data; and based on a result of the determining whether to perform AI upscaling, performs AI upscaling on the received version of the image content through a upscaling deep neural network (DNN) that is trained jointly with a downscaling DNN of the server.