Satellite Image Reconstruction Using GANs for Cloud Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Satellite imagery is frequently obscured by clouds, leading to incomplete and inaccurate image data, limiting its utility in applications such as environmental monitoring and mapping, as traditional inpainting methods struggle to reconstruct large occluded areas accurately.
Innovation Solution
A machine learning model, specifically a generative adversarial network (GAN) trained with partial observation data, utilizes both satellite optical and radar data to generate accurate, cloud-free imagery by simulating occlusions using masks and incorporating neural attention mechanisms to capture contextual information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional inpainting methods are used to reconstruct occluded areas, then the process is simple and fast, but the reconstruction accuracy is poor especially for large occluded areas
Solution Approach 1:
The patent introduces a machine learning model as an intermediary between the occluded input image and the reconstructed output. This model acts as a mediator that learns the complex mapping from occluded to complete images through training on paired datasets, achieving high reconstruction accuracy without requiring complex manual inpainting algorithms
Solution Approach 2:
The patent performs preliminary training of the machine learning model on large datasets of occluded and corresponding clear images before actual reconstruction. This preliminary learning phase enables the model to capture patterns and relationships that improve reconstruction accuracy when deployed, separating the complex learning process from the actual inpainting operation
2Measurement precision
If satellite optical images are used for monitoring, then the spatial resolution is high, but the images are frequently obscured by clouds
Solution Approach 1:
The patent uses cloud masks that identify occluded regions and transforms this harmful obstruction into a beneficial training signal. By masking cloud-covered areas and training the model to reconstruct these regions, the harmful effect of clouds becomes a useful mechanism for teaching the model what clear imagery should look like, improving both reliability and maintaining high spatial resolution
Solution Approach 2:
The patent segments the satellite image into occluded and unoccluded regions using cloud masks. This segmentation allows the model to focus computational resources on reconstructing only the problematic occluded areas while preserving the high-resolution unoccluded regions, maintaining overall image quality and availability
3Measurement precision
If machine learning models are trained with complete images, then the model learns accurate features, but it cannot handle occluded input during deployment
Solution Approach 1:
The patent inverts the traditional training approach by training the model to generate complete images from occluded inputs, rather than learning from complete images. This is achieved by creating training pairs where clear images are artificially occluded and then using the model to reconstruct them, enabling the model to handle occluded deployment inputs while learning accurate features from ground truth clear images
4Ease of manufacture
If uniform sampling is used for training dataset creation, then the process is simple, but bias is introduced when certain land cover types dominate
Solution Approach 1:
The patent applies different sampling strategies to different regions of the satellite image based on local characteristics. Areas with dominant land cover types or frequent occlusions are oversampled, while other areas are sampled at lower rates. This local quality adjustment ensures the training dataset is representative of all important scenarios without requiring complex manual curation
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Examples disclosed herein are related to using a machine learning model to generate image data. One example provides a system, comprising one or more processors, and storage comprising instructions executable by the one or more processors to obtain image data comprising an image with unoccluded features, apply a mask to the unoccluded features in the image to form partial observation training data comprising a masked region that obscures at least a portion of the unoccluded features, and train a machine learning model comprising a generator and a discriminator at least in part by generating image data for the masked region and comparing the image data generated for the masked region to the image with unoccluded features.