Image Processing Device for High-Resolution Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks used in semantic segmentation are not invariant to geometric deformation, leading to inaccurate object class estimation for high-resolution images not present in the training dataset, and retraining with high-resolution images is costly and inefficient.
Innovation Solution
An image processing device that generates low-resolution images from input images using shift amounts corresponding to the training image resolution, applies semantic segmentation, and upsamples the results to achieve accurate label assignment for high-resolution images without requiring retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the resolution of input image is reduced to the resolution of training image, then the trained model can process the image, but the image quality and recognition detail are degraded
Solution Approach 1:
The input image is divided into multiple patches, each processed separately by the trained model. This allows the model to handle images of any resolution by processing them in manageable segments, avoiding the need to reduce the overall image resolution while maintaining quality.
Solution Approach 2:
The problem is solved by transitioning from a single-resolution processing approach to a multi-scale patch-based approach. By processing patches at the original resolution and then aggregating results, the system maintains high-resolution output quality while being compatible with models trained on specific resolutions.
2Measurement precision
If retraining the model with high-resolution images is performed, then the model can accurately process high-resolution images, but the training cost and time are significantly increased
Solution Approach 1:
Instead of retraining the model with high-resolution images (which would be time-consuming), the method copies and processes multiple patches from the high-resolution input image through the existing trained model. This approach achieves accurate high-resolution processing without the need for expensive retraining.
Solution Approach 2:
The input image is pre-processed into patches before being fed to the model. This preliminary segmentation allows the existing model to be used effectively on high-resolution images without modification, avoiding the need for retraining while maintaining accuracy.
3Measurement precision
If the trained model processes high-resolution images directly, then the label estimation accuracy is improved, but the model fails due to resolution mismatch with training data
Solution Approach 1:
The high-resolution input image is segmented into multiple patches that match the resolution expected by the trained model. This segmentation allows the model to process high-resolution content reliably by breaking it into manageable pieces that fit its training specifications.
Solution Approach 2:
Patches serve as an intermediary between the high-resolution input image and the trained model. By processing through this intermediate representation, the system maintains model stability and reliability while still achieving accurate label estimation for the full high-resolution image.
Data Source
AI summary
Labels can be accurately identified even for an image with a resolution not used in training data. Based on an input image, a resolution of the input image, and a resolution of a training image used for training a trained model of assigning labels to pixels of an image, a plurality of low-resolution images are generated from the input image by using a plurality of shift amounts for a pixel correspondence between the input image and the respective low-resolution images with a resolution corresponding to the training image, the low-resolution images are input to the trained model, a plurality of low-resolution label images is output in which pixels of the respective low-resolution images are assigned labels, and a label image is output in which labels for pixels of the input image are obtained, based on the shift amounts used for generating the low-resolution images and the low-resolution label images.