Multi-Modal Image Enhancement Using Time-Adjacent Remote Sensing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote sensing imagery often exhibits sparsity in modality, spectral band, location, and time due to limitations such as satellite revisiting periods and cloud coverage, making it challenging to obtain images satisfying specific queries.
Innovation Solution
An adaptive multi-modal regression and generation technique using intra-modal and inter-modal models to generate enhanced images based on time-adjacent data, employing neural networks and generative adversarial networks for image enhancement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If remote sensing imagery is collected with fixed satellite revisiting periods and spectral bands, then data acquisition is systematic and manageable, but image sparsity occurs in modality, spectral band, location, and time
Solution Approach 1:
The system pre-collects and stores multiple modalities of remote sensing imagery data (including optical, SAR, and other types) in advance for the same geographic region. When a query is received, the pre-stored time-adjacent images can be immediately retrieved and processed, eliminating the need to wait for specific satellite revisiting periods and reducing image sparsity.
Solution Approach 2:
The system introduces an image enhancement model as an intermediary that synthesizes target modality images from source modality images. This intermediary model bridges the gap between available remote sensing data and requested data, enabling generation of images for queried modalities, spectral bands, and locations even when direct observations are unavailable.
2Loss of information
If multiple remote sensing satellites with different modalities are used to reduce image sparsity, then data availability improves, but system complexity increases
Solution Approach 1:
The image enhancement model is designed with multi-functionality to handle multiple modalities (optical, SAR, etc.), spectral bands, and geographic regions universally. A single unified model architecture can process different types of remote sensing data and generate enhanced images across various conditions, reducing the need for separate specialized systems for each satellite or modality combination.
Solution Approach 2:
The system changes the parameters of the image enhancement model dynamically based on the input data characteristics and query requirements. By adjusting model parameters rather than changing the entire system architecture, the system can adapt to different satellite data types and conditions without increasing structural complexity.
3Device complexity
If traditional image processing methods are used without time-adjacent data, then processing simplicity is maintained, but image quality and enhancement capability deteriorate
Solution Approach 1:
The system applies partial action by using only the necessary time-adjacent images and model components required for enhancement, rather than processing all available data. This selective approach maintains processing efficiency while achieving improved image quality through targeted enhancement of specific regions and features.
Data Source
AI summary
According to one aspect of the present disclosure, a method of image enhancement is provided. The method may include receiving, by at least one processor, a query for an enhanced image of a queried region obtained by a remote sensor at a queried time. The method may include obtaining, by the at least one processor, an image dataset associated with the queried region from the remote sensor. In response to the image dataset including a plurality of image data associated with the queried region obtained by the remote sensor, the method may include generating, by the at least one processor, the enhanced image of the queried region using an intra-modal regression model.