Resolution-Switchable Segmentation Networks for Multi-Resolution Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) for image segmentation suffer from significant accuracy drops when applied to test frames with resolutions different from the training frames, leading to increased storage and computation requirements, and necessitate individual model training for each target frame resolution, causing delays and inefficiencies.
Innovation Solution
A computer model for image segmentation that uses shared convolutional layer parameters across multiple image sizes, combined with size-specific normalization layers, employs mixed-resolution parallel training, ensemble predictions, and distillation losses to adapt to different input resolutions, reducing the need for multiple models and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If individual models are trained for each target frame resolution, then segmentation accuracy is maintained, but storage requirements and training costs increase significantly
Solution Approach 1:
The patent applies universality by training a single segmentation model to handle multiple target frame resolutions (e.g., 480p, 720p, 1080p) rather than training separate models for each resolution. The model uses shared convolutional layers that can process inputs of varying resolutions and produce appropriate segmentation outputs, thereby reducing storage requirements while maintaining accuracy across different resolutions.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the number of deconvolutional layers and their parameters based on the target frame resolution. When the target resolution changes, the model selectively activates or deactivates specific deconvolutional layers, effectively changing the model's parameter configuration to match the required output resolution without requiring complete model retraining.
2Measurement precision
If individual models are trained for each target frame resolution, then segmentation accuracy is maintained, but computation requirements increase
Solution Approach 1:
The patent applies universality by training a single segmentation model to handle multiple target frame resolutions (e.g., 480p, 720p, 1080p) rather than training separate models for each resolution. The model uses shared convolutional layers that can process inputs of varying resolutions and produce appropriate segmentation outputs, thereby reducing storage requirements while maintaining accuracy across different resolutions.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the number of deconvolutional layers and their parameters based on the target frame resolution. When the target resolution changes, the model selectively activates or deactivates specific deconvolutional layers, effectively changing the model's parameter configuration to match the required output resolution without requiring complete model retraining.
3Measurement precision
If models are retrained when target frame resolution is modified, then accuracy for new resolution is ensured, but inference delay increases
Solution Approach 1:
The patent applies preliminary action by pre-training a single model with shared convolutional layers that can handle multiple resolutions. The model is prepared in advance with the capability to adapt to different target frame resolutions through its architectural design, eliminating the need for retraining when resolution changes occur during inference.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the number of deconvolutional layers and their parameters based on the target frame resolution. When the target resolution changes, the model selectively activates or deactivates specific deconvolutional layers, effectively changing the model's parameter configuration to match the required output resolution without requiring complete model retraining.
4Measurement precision
If size-specific model parameters are retrieved for each resolution, then correct segmentation is achieved, but processing time increases
Solution Approach 1:
The patent applies universality by training a single segmentation model to handle multiple target frame resolutions (e.g., 480p, 720p, 1080p) rather than training separate models for each resolution. The model uses shared convolutional layers that can process inputs of varying resolutions and produce appropriate segmentation outputs, thereby reducing storage requirements while maintaining accuracy across different resolutions.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the number of deconvolutional layers and their parameters based on the target frame resolution. When the target resolution changes, the model selectively activates or deactivates specific deconvolutional layers, effectively changing the model's parameter configuration to match the required output resolution without requiring complete model retraining.
Data Source
AI summary
A computer model for object segmentation in images may be used for multiple input image sizes with shared convolutional layer parameters to be applied across application of the multiple image sizes. The model can also include size-specific parameters for one or more size-dependent layers, such as a normalization layer. The model may be trained with mixed-resolution training images in parallel in which the training image is resized to multiple sizes and the resulting predictions may learn the respective parameters in parallel based on an ensemble prediction as well as distillation from higher to lower resolution input image predictions.