Cross-Modality Image Resolution Enhancement via Joint Histograms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing techniques fail to enhance the resolution of an image in one modality based on information from a dataset in another modality, limiting the ability to improve clarity in images with different resolutions.
Innovation Solution
The method involves spatially registering datasets from different modalities, computing a joint histogram to estimate missing pixel intensities, and using an objective function that prioritizes consistency with the higher-resolution dataset and smoothness, with iterative methods like annealing to optimize pixel values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If interpolation or superresolution techniques are used to enhance resolution within the same modality, then image clarity improves, but the technique cannot leverage information from other modalities to improve resolution
Solution Approach 1:
The patent merges information from multiple modalities by computing a joint histogram that combines pixel intensity distributions from both the first modality (lower resolution) and second modality (higher resolution). This integration allows the lower-resolution image to benefit from the higher-resolution data through probabilistic modeling, effectively combining the strengths of both modalities to enhance resolution.
Solution Approach 2:
The patent introduces a joint probability distribution as an intermediary between the two modalities. This probabilistic model serves as a mediator that translates information from the higher-resolution modality to enhance the lower-resolution modality, allowing indirect information transfer and resolution enhancement without direct pixel-to-pixel mapping.
2Measurement precision
If multiple images are obtained for superresolution, then higher resolution can be achieved, but the complexity of the process increases due to registration and alignment requirements
Solution Approach 1:
The patent changes the approach from spatial domain operations to probability distribution domain operations. Instead of performing complex geometric registrations and alignments of multiple images, the method transforms the problem into computing and manipulating joint probability distributions of pixel intensities, simplifying the processing while achieving resolution enhancement.
3Stability of the object's composition
If texture synthesis is used to fill holes in images, then visual consistency is improved, but the method cannot propagate actual high-resolution detail from another modality
Solution Approach 1:
The patent uses the joint histogram computation as a feedback mechanism that continuously refines the probability distribution of pixel values. By iteratively updating the probability distributions based on observed pixel pairs from both modalities, the method ensures that the synthesized high-resolution details are consistent with the actual statistical relationships between modalities, rather than relying on generic texture models.
Data Source
AI summary
The dataset describing an entity in a first modality and of a first, high resolution is used to enhance the resolution of a dataset describing the same entity in a second modality of a lower resolution. The two data sets of different modalities are spatially registered to each other. From this information, a joint histogram of the values in the two datasets is computed to provide a raw analysis of how the intensities in the first dataset correspond to intensities in the second dataset. This is converted into a joint probability of possible intensities for the missing pixels in the low resolution dataset as a function of the intensities of the corresponding pixels in the high-resolution dataset to provide a very rough estimate of the intensities of the missing pixels in the low resolution dataset. Then, an objective function is defined over the set of possible new values that gives preference to datasets consistent with (1) the joint probability distributions, (2) the existing values in the low resolution dataset, and (3) smoothness throughout the data set. Finally, an annealing or similar iterative method is used to minimize the objective function and find an optimal solution over the entire dataset.