Turbulence Correction via Overlapping Video Tile Mosaicing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Atmospheric turbulence causes image degradation in video data, particularly affecting the quality of images captured by aircraft-mounted cameras, leading to blurring and reduced image value for operators and processing equipment, with existing technologies struggling to effectively correct low-frequency conditions and maintain uniformity across the full Field of View (FOV).

Innovation Solution

A system and method for image turbulence correction that involves demosaicing consecutive image frames into overlapping tiles, determining displacement for spatial alignment, converting tiles into the frequency domain for iterative processing, and mosaicing them back into a single frame with turbulence degradation correction, ensuring precise frame registration and stitching to maintain image quality across the FOV.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional temporal filters are applied to correct image degradation, then temporal filtering performance is improved, but spatial uniformity across the Field of View deteriorates due to non-congruent digital manipulation of individual subset images

Engineering Contradiction:

Improvetemporal filtering performanceVSAvoidspatial uniformity across FOV

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The image is divided into multiple overlapping subset images or tiles that are processed independently through temporal filtering, then recombined through mosaicing with gain and level matching to maintain spatial uniformity across the full field of view

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different gain and level adjustments are applied to different tiles or subsets of the image to compensate for non-uniformities introduced during independent processing, ensuring each region maintains appropriate quality characteristics

Inventive Principle:

Principle #3Local quality

2Reliability

If individual subset images are processed independently to correct turbulence, then temporal filtering effectiveness is improved, but blocky artifacts appear in the larger mosaic video

Engineering Contradiction:

Improveturbulence correction effectivenessVSAvoidimage continuity in mosaic

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The full field of view is divided into multiple overlapping tiles that are processed independently for turbulence correction, then stitched together with blending techniques that smooth transitions at boundaries to eliminate blocky artifacts

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple independently processed tiles are merged into a single cohesive mosaic image through stitching algorithms that ensure continuity and smooth transitions at tile boundaries, preventing visible block artifacts

Inventive Principle:

Principle #5Merging (Combining)

3Manufacturing precision

If global frame registration is used for image optimization, then overall image quality is improved, but processing complexity increases and precise results across full FOV are difficult to achieve

Engineering Contradiction:

Improveoverall image qualityVSAvoidprocessing complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

Instead of processing the entire frame globally, the image is segmented into smaller tiles that are registered and optimized independently, reducing computational complexity while maintaining precision through localized processing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Processing is applied selectively to individual tiles rather than uniformly across the entire frame, allowing optimized processing resources to focus on regions requiring correction while reducing overall computational burden

Inventive Principle:

Principle #16Partial or excessive action

4Reliability

If low frequency conditions are corrected using conventional methods, then some image degradation is improved, but image artifacts appear due to inadequate spatial compensation and stitching

Engineering Contradiction:

Improvelow frequency condition correctionVSAvoidimage artifact free output

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

Different spatial compensation strategies are applied to different regions and frequency components, with specialized handling for low frequency content that prevents artifact generation during the stitching process

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

An intermediary processing stage is introduced between individual tile correction and final mosaicing that performs gain and level matching to smooth transitions and eliminate artifacts at tile boundaries

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10521883B1Image turbulence correction using tile approach

Publication Date: 2019.12.31 RAYTHEON CO

AI summary

System and method for image turbulence correction includes: receiving a plurality of consecutive image frames; demosaicing previous, current and preceding image frames into a plurality of same size overlapping video tiles; determining a displacement of each of the video tiles; converting the video tiles of the current image frame, the previous image frame, and the plurality of preceding image frames into a frequency domain; iteratively processing the video tiles of the previous image frame, the current image frame and the plurality of preceding image frames for turbulence correction in the frequency domain; converting the turbulence corrected video tiles into a spatial domain, wherein the converted turbulence corrected video tiles form a single video frame tile including turbulence degradation correction; and mosaicing the single video frame tiles including turbulence degradation correction together to generate a full field of view turbulence correct video stream.