UAV Multi-Camera Image Deviation Correction via Feature Mapping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Unmanned aerial vehicles (UAVs) with multiple cameras face challenges in capturing highly realistic panoramic images due to vibrations caused by external disturbances, leading to deviations and inconsistencies between frame images, which are exacerbated by the difficulty in stabilizing multiple cameras using traditional gimbal systems.

Innovation Solution

An image processing method that acquires and compares frame images from multiple cameras, determines deviations, calculates correction values, and corrects the images to synthesize a highly accurate panoramic image without the need for firm fixation of cameras, using a system comprising a deviation determination unit, correction value calculation unit, and frame image correction unit.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of stationary object

If multiple cameras are mounted on a UAV to capture panoramic images, then the coverage area and definition are improved, but vibrations cause deviations and inconsistencies between frame images

Engineering Contradiction:

Improvecoverage areaVSAvoidimage consistency

Core Design Contradiction:

Area of stationary objectVSReliability

Solution Approach 1:

The patent uses feature point mapping between frame images to detect deviations caused by vibrations, then calculates correction values based on these detected deviations to correct the images, creating a feedback loop that maintains image consistency despite UAV vibrations

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent calculates correction values by analyzing parameter differences (feature point mappings) between frame images, then applies these parameter corrections to compensate for vibration-induced deviations and maintain image consistency

Inventive Principle:

Principle #35Parameter changes

2Stability of the object's composition

If traditional gimbal systems are used to stabilize cameras, then image stability is improved, but the system complexity and weight increase

Engineering Contradiction:

Improveimage stabilityVSAvoidsystem complexity

Core Design Contradiction:

Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The patent replaces mechanical stabilization systems (gimbals) with an image processing approach that uses feature point mapping and correction value calculation to achieve stability, eliminating complex mechanical components

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent extracts the stabilization function from the mechanical gimbal system and implements it through software-based image correction, separating the stabilization function from physical camera mounting

Inventive Principle:

Principle #2Taking out (Extraction)

3Reliability

If cameras are firmly fixed to reduce vibration effects, then image consistency is improved, but the ability to absorb UAV vibrations is reduced

Engineering Contradiction:

Improveimage consistencyVSAvoidvibration absorption

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent implements feedback-based correction where feature point mappings detect vibration effects and correction values are calculated and applied to compensate for these effects, maintaining image consistency without rigid fixation

Inventive Principle:

Principle #23Feedback

4Manufacturing precision

If projective transformation is performed using feature point mappings, then distortion is removed and images are projected onto one plane, but errors in mapping cause deviations and strange lines at joins

Engineering Contradiction:

Improveimage alignmentVSAvoidmapping accuracy

Core Design Contradiction:

Manufacturing precisionVSMeasurement precision

Solution Approach 1:

The patent performs preliminary correction by calculating correction values based on feature point mappings before final image synthesis, preventing mapping errors from causing deviations and strange lines in the panoramic image

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces correction values as an intermediary between feature point mapping and final image projection, mediating the transformation process to eliminate errors and produce smooth seamless panoramas

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11856298B2Image processing method, image processing device, image processing system, and program

Publication Date: 2023.12.26 NIPPON TELEGRAPH & TELEPHONE CORP

AI summary

An image processing method according to the present invention includes the steps of acquiring a first frame image and a second frame image, determining, based on a difference between the first frame image at a first time and the second frame image at the first time, whether or not a deviation has occurred therebetween, calculating a correction value based on an amount of deviation when it has been determined that a deviation has occurred, correcting the first frame image and the second frame image at the first time based on the correction value, and synthesizing the corrected first frame image and the corrected second frame image.