Digital Media Up-Sampling Using Scene Segmentation and GANs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current digital media content up-sampling methods result in resolution defects and continuity artifacts when scaling digital media to higher pixel resolutions, especially when converting from lower resolution standards like SD to HD or UHD, due to inefficient processing and lack of scene-based analysis.
Innovation Solution
An automated up-sampling technique using a service provider computer that segments media files into scenes, detects and recognizes facial features, and employs generative adversarial networks (GANs) trained on metadata and multiple pixel-resolution standards to up-sample media files in parallel, minimizing artifacts by processing foreground and background regions separately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If simple scaling is used to up-sample digital media content, then processing time is reduced, but resolution defects and continuity artifacts are introduced
Solution Approach 1:
The patent segments the media content into scenes based on metadata analysis, and further divides each scene into foreground and background regions. This segmentation allows the system to apply different up-sampling techniques to different regions, improving image quality while managing processing complexity through parallel processing of independent regions.
Solution Approach 2:
The patent applies different up-sampling methods to different regions of the media content. Foreground regions (containing important visual elements like faces and objects) receive specialized up-sampling treatment, while background regions use different processing. This local quality approach minimizes artifacts in critical areas while maintaining overall efficiency.
2Productivity
If automated up-sampling is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The system performs preliminary analysis of metadata to identify scene boundaries and content characteristics before actual up-sampling begins. This preliminary action enables automated decision-making about processing parameters and reduces the complexity of real-time processing by pre-determining processing strategies.
Solution Approach 2:
The patent introduces an intermediary processing layer that analyzes metadata and coordinates the up-sampling process. This intermediary component simplifies the overall system architecture by centralizing the decision-making logic and managing the complexity of coordinating multiple processing operations.
3Manufacturing precision
If scene-based processing is used, then image quality is improved, but processing time increases
Solution Approach 1:
By segmenting media into scenes and further dividing scenes into foreground and background regions, the system enables parallel processing of independent regions. This segmentation strategy improves image quality through region-specific processing while reducing overall processing time through concurrent operations.
Solution Approach 2:
The patent introduces a temporal dimension to the processing approach by analyzing metadata to identify scene boundaries and character appearances over time. This temporal analysis enables the system to process media in parallel temporal segments, improving both image quality and processing efficiency.
Data Source
AI summary
Techniques for automated up-sampling of media files are provided. In some examples, a title associated with a media file, a metadata file associated with the title, and the media file may be received. The media file may be partitioned into one or more scene files, each scene file including a plurality of frame images in a sequence. One or more up-sampled scene files may be generated, each corresponding to a scene file of the one or more scene files. An up-sampled media file may be generated by combining at least a subset of the one or more up-sampled scene files. Generating one or more up-sampled scene files may include identifying one or more characters in a frame image of the plurality of frame images, based at least in part on implementation of a facial recognition algorithm including deep learning features in a neural network.