Image Registration to 3D Point Sets Using Synthetic Intermediary

Solution Overview

Problem

Current image registration techniques face challenges in accurately aligning two-dimensional images with three-dimensional point sets, especially due to inaccuracies in geometric metadata and the presence of voids in synthetic image data, which affects cross-sensor fusion and other applications.

Innovation Solution

An edge-based, two-step registration method is employed, involving coarse and fine registration processes to identify tie points and control points, using a synthetic image generated by projecting 3D point sets into an image space, and interpolating void pixels to enhance alignment accuracy across different sensor modalities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If current image registration techniques are used to align 2D images with 3D point sets, then the registration process can be completed, but the alignment accuracy deteriorates due to geometric metadata inaccuracies and voids in synthetic image data

Engineering Contradiction:

Improvealignment accuracyVSAvoidregistration robustness

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The registration process is divided into two distinct stages: coarse registration that establishes initial alignment using available geometric metadata, and fine registration that refines alignment by matching features between the 2D image and synthetic image generated from the 3D point set. This segmentation allows the system to handle metadata inaccuracies and voids by progressively improving alignment rather than requiring perfect initial conditions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A synthetic image is generated as an intermediary representation of the 3D point set in the 2D image space. This synthetic image serves as a mediator that bridges the 2D image and 3D point set, allowing feature matching to be performed in a common space while handling voids and metadata inaccuracies through the two-step registration process.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If geometric metadata is used for initial alignment, then the registration process can start, but alignment accuracy deteriorates due to metadata inaccuracies

Engineering Contradiction:

Improveregistration initiationVSAvoidalignment accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

Geometric metadata is used to perform preliminary coarse registration that establishes an initial alignment between the 2D image and 3D point set. This preliminary action enables the registration process to start without requiring perfect alignment, while the subsequent fine registration step corrects the inaccuracies introduced by the metadata.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The registration process is divided into two distinct stages: coarse registration that establishes initial alignment using available geometric metadata, and fine registration that refines alignment by matching features between the 2D image and synthetic image generated from the 3D point set. This segmentation allows the system to handle metadata inaccuracies and voids by progressively improving alignment rather than requiring perfect initial conditions.

Inventive Principle:

Principle #1Segmentation

3Adaptability or versatility

If synthetic image data is generated from 3D point sets, then image registration can be performed, but the quality deteriorates due to voids in the synthetic image data

Engineering Contradiction:

Improvecross-sensor fusion capabilityVSAvoidregistration accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

A synthetic image is generated as an intermediary representation of the 3D point set in the 2D image space. This synthetic image serves as a mediator that bridges the 2D image and 3D point set, allowing feature matching to be performed in a common space while handling voids and metadata inaccuracies through the two-step registration process.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The registration process dynamically adapts to the quality of the synthetic image data by using the two-step approach: coarse registration handles areas with voids using geometric metadata, while fine registration refines alignment in areas where the synthetic image data is reliable. This dynamic adaptation maintains versatility across different sensor modalities while preserving accuracy where possible.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11127145B2Image registration to a 3D point set

Publication Date: 2021.09.21 RAYTHEON CO

AI summary

Discussed herein are devices, systems, and methods for image processing. A method can include generating a synthetic image based on a two-dimensional (2D) image of the geographical region, performing a coarse registration to grossly, register the synthetic image to the 2D image, and performing a fine registration following the coarse registration to improve the registration between the synthetic image and the 2D image.