Aerial Refueling Camera Control Using 3D Keypoint Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated aerial refueling techniques face challenges in accurately and efficiently conducting refueling operations due to difficulties in using cameras for precise control.
Innovation Solution
A system that includes a camera on a tanker aircraft to generate 2D images of refueling operations, identifies keypoints, projects them to 3D space, compares them to a stored 3D model to produce a confidence value, and automatically controls the refueling process based on this value.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automated aerial refueling control is implemented using camera-based systems, then safety and automation level are improved, but measurement precision and control accuracy deteriorate due to difficulty in accurately conducting refueling operations
Solution Approach 1:
The system creates a 3D digital model (copy) of the receiver aircraft and compares it with real-time camera images to determine alignment accuracy. This virtual model serves as a reference for precise measurement without requiring direct physical measurement tools, resolving the contradiction between automation and precision.
Solution Approach 2:
The patent transitions from 2D camera images to 3D spatial understanding by projecting image points into three-dimensional space and comparing them with a 3D model of the receiver aircraft. This dimensional transformation enables accurate depth and position measurement that cannot be obtained from 2D images alone, thereby improving measurement precision while maintaining automation.
2Device complexity
If 2D camera images are used for refueling control, then device complexity is reduced, but measurement precision deteriorates due to inability to accurately represent 3D spatial relationships
Solution Approach 1:
The system projects 2D image points into 3D space using known camera parameters and geometric relationships. This allows the simple camera device to extract precise three-dimensional position information, resolving the contradiction between device simplicity and measurement precision by adding computational dimensionality rather than physical complexity.
Solution Approach 2:
The patent introduces a 3D digital model of the receiver aircraft as an intermediary between the 2D camera images and the control system. This virtual model serves as a mediator that translates simple image data into precise spatial measurements, enabling accurate control without complex measurement devices.
3Measurement precision
If keypoint projection to 3D space is implemented, then measurement precision is improved, but device complexity and computational requirements increase
Solution Approach 1:
The system pre-establishes a 3D digital model of the receiver aircraft and pre-calculates the relationship between camera image coordinates and 3D world coordinates. This preliminary preparation reduces real-time computational requirements during actual refueling operations, resolving the contradiction between measurement precision and computational complexity.
Solution Approach 2:
The patent uses a pre-created 3D digital copy of the receiver aircraft as a reference model. This virtual copy contains all necessary geometric information, eliminating the need for complex real-time 3D reconstruction and reducing computational burden while maintaining high measurement precision.
Data Source
AI summary
A method for performing automated refueling operations includes receiving a two-dimensional (2D) image of a device associated with in-flight refueling operations between a receiver aircraft and a tanker aircraft, identifying keypoints on the device in the 2D image, projecting the keypoints to three-dimensional (3D) space to produce 3D keypoints, comparing the 3D keypoints to a previously stored 3D model of the device to produce a confidence value, and automatically controlling the refueling operations in response to the confidence value.