3D LADAR and Multispectral Image Registration via Bundle Adjustment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Multispectral images from commercial earth observation satellites often exhibit registration errors with 3D data from LADAR, leading to uncertainties in object location and requiring time-consuming manual registration, which is seldom performed accurately.

Innovation Solution

A system that automatically registers 3D data with multispectral images using photogrammetric bundle adjustment, extracting ground control points and colorizing 3D geodetic coordinates with registered electro-optical data to reduce registration errors and generate accurate color images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual registration of LADAR and multispectral images is performed, then registration accuracy can be improved, but time consumption increases significantly

Engineering Contradiction:

Improveregistration accuracyVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary automated registration by extracting ground control points and performing bundle adjustment before final manual refinement. This preliminary action establishes an accurate initial alignment that reduces the time required for manual correction while maintaining high registration accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an automated registration algorithm as an intermediary between raw LADAR data and multispectral images. This intermediary process uses feature extraction, ground control point identification, and bundle adjustment to bridge the two data sources, reducing both time consumption and improving accuracy compared to direct manual registration.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Loss of time

If automated registration algorithms are used, then time consumption is reduced, but registration accuracy deteriorates due to uncorrelated geometric support data errors

Engineering Contradiction:

Improvetime consumptionVSAvoidregistration accuracy

Core Design Contradiction:

Loss of timeVSMeasurement precision

Solution Approach 1:

The system implements iterative feedback loops where the automated registration algorithm processes the data, evaluates registration quality metrics, and refines the alignment through multiple passes. The bundle adjustment process uses feedback from residual errors to progressively improve registration accuracy while maintaining automated operation.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent dynamically adjusts registration parameters during the automated process, including ground control point selection criteria, optimization weights, and geometric constraints. By changing parameters adaptively based on data quality and registration progress, the system maintains high accuracy while operating automatically.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If LADAR data is used alone, then 3D geometric information is obtained, but color information and spectroradiometric data are lost

Engineering Contradiction:

Improve3D geometric accuracyVSAvoidcolor and spectroradiometric information

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent merges LADAR point cloud data with multispectral image data through automated registration. The LADAR provides accurate 3D geometric structure while the multispectral images contribute color and spectroradiometric information. The bundle adjustment process integrates these complementary data sources into a unified colored point cloud that preserves both geometric precision and spectral information.

Inventive Principle:

Principle #5Merging (Combining)

4Loss of information

If multispectral images are used alone, then color and spectral information are obtained, but accurate 3D location data and depth information are lost

Engineering Contradiction:

Improvecolor and spectral informationVSAvoid3D location accuracy

Core Design Contradiction:

Loss of informationVSMeasurement precision

Solution Approach 1:

The system combines multispectral imagery with LADAR depth data to create a unified dataset. The multispectral images provide rich color and spectral information while LADAR contributes precise 3D location and depth measurements. Through automated registration and bundle adjustment, these data sources are merged to produce accurately located colored point clouds that retain both spectral richness and geometric precision.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS9275267B2System and method for automatic registration of 3D data with electro-optical imagery via photogrammetric bundle adjustment

Publication Date: 2016.03.01 RAYTHEON CO

AI summary

Accurate automatic registration and fusion of LADAR (from laser detection and ranging) and EO (electro-optical) data from different sensors provides additional analysis and exploitation value beyond what each data set provides on its own. Such data sets often exhibit significant misregistration due to uncorrelated geometric errors between or among two or more sensors. One or more automatic algorithms achieve superior registration as well as algorithms for fusing the data in three dimensions (3D). The fused data can provide multi-image colorization for change detection, automatic generation of surface relief colorization, interactive and/or automatic filtering of 3D vegetation points for LADAR foliage penetration analysis, automatic surface orientation determination for improved spectroradiometric exploitation, and other benefits that cannot be achieved by the LADAR or EO data alone.