Image Harmonization for Cross-Domain Deep Learning Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models trained on specific source domain images face performance degradation when applied to images with appearance variations due to factors like capture protocol, dose usage, and exposure settings, making it difficult and costly to adapt them effectively.
Innovation Solution
Image harmonization techniques that decompose input images into sub-images, harmonize them with corresponding reference sub-images based on calculated statistical measures, and reconstruct them to resemble the reference images, using methods like energy band decomposition and segmentation, with optional training of harmonization models to generate ground-truth data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image harmonization is performed to improve model accuracy, then training time increases
Solution Approach 1:
The patent applies preliminary action by performing image harmonization during the pre-processing phase before model training begins. The system harmonizes images from multiple sources and creates a unified training dataset in advance, so that when training starts, the data is already optimized. This separates the harmonization task from the training process, allowing the model to train faster on pre-harmonized data without sacrificing accuracy improvements.
2Adaptability or versatility
If multiple image sources are combined to increase data diversity, then data processing complexity increases
Solution Approach 1:
The patent merges multiple image sources and their associated metadata into a unified data structure that maintains diversity while simplifying processing. The system combines images from different sources (e.g., satellite imagery, aerial photos, ground-level images) and harmonizes their coordinate systems and attribute spaces, creating a single consistent dataset that preserves the benefits of multiple sources without requiring complex separate processing pipelines for each source.
Solution Approach 2:
The patent introduces an intermediary harmonization layer that acts as a mediator between multiple image sources and the final training model. This intermediary component standardizes and harmonizes data from diverse sources before it reaches the model, absorbing the complexity of handling multiple sources in a centralized manner and presenting a simplified, consistent data interface to the training algorithm.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for optimizing deep learning model performance using image harmonization as a pre-processing step comprises decomposing, by a system operatively coupled to a processor, an input image into sub-images. The method further comprises harmonizing the sub-images with corresponding reference sub-images of at least one reference image based on two or more different statistical values respectively calculated for the sub-images and the corresponding reference-sub images, resulting in transformation of the sub-images into modified sub-images images. The modified sub-images can be combined into a harmonized image having a more similar appearance to the at least one reference image relative to the input image. Harmonized images and/or modified sub-images generated using these techniques can be used as ground-truth training samples for training one or more deep learning model to transform input images with appearance variations into harmonized images.