Recursive Decoder Segmentation for High-Resolution Image Detail
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital visual media segmentation systems are inflexible and inaccurate, particularly when dealing with high-resolution images, as they struggle to accurately segment fine-grained details and require significant memory and computing resources.
Innovation Solution
A neural network with a recursive decoder, including a deconvolution branch and a refinement branch with hierarchical point-wise refining blocks, is used to generate refined segmentation masks for high-resolution digital visual media items, employing a low-resolution segmentation model and a high-resolution refinement model within a framework pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional segmentation systems are used to segment high-resolution images, then segmentation masks can be generated, but the systems are inflexible and inaccurate at segmenting fine-grained details
Solution Approach 1:
The patent divides the segmentation task into two distinct models: a low-resolution segmentation model for coarse segmentation and a high-resolution refinement model for fine-grained detail segmentation. This segmentation of the processing task enables each model to specialize in specific resolution levels, improving both accuracy and flexibility.
Solution Approach 2:
The patent introduces a resolution dimension by processing images at multiple resolutions simultaneously. The low-resolution model handles the overall structure while the high-resolution model refines fine-grained details, adding the resolution dimension to the segmentation approach and enabling accurate segmentation at high resolutions without sacrificing flexibility.
2Measurement precision
If conventional systems process high-resolution images, then segmentation can be performed, but extensive memory and computing resources are required
Solution Approach 1:
By segmenting the segmentation task into low-resolution and high-resolution models, the system processes different portions of the image at appropriate resolutions. The low-resolution model handles the bulk of the computation for coarse segmentation, while the high-resolution model only processes fine-grained details, reducing overall computational resource requirements.
Solution Approach 2:
The system applies high-resolution processing only where necessary - for fine-grained detail segmentation - rather than processing the entire high-resolution image through a single high-resolution model. This partial application of computational resources to specific tasks reduces overall energy consumption while maintaining accuracy where needed.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilizes a neural network having a hierarchy of hierarchical point-wise refining blocks to generate refined segmentation masks for high-resolution digital visual media items. For example, in one or more embodiments, the disclosed systems utilize a segmentation refinement neural network having an encoder and a recursive decoder to generate the refined segmentation masks. The recursive decoder includes a deconvolution branch for generating feature maps and a refinement branch for generating and refining segmentation masks. In particular, in some cases, the refinement branch includes a hierarchy of hierarchical point-wise refining blocks that recursively refine a segmentation mask generated for a digital visual media item. In some cases, the disclosed systems utilize a segmentation refinement neural network that includes a low-resolution network and a high-resolution network, each including an encoder and a recursive decoder, to generate the refined segmentation masks.