Monocular Fuel Receptacle Pose Refinement for Aerial Refueling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Aerial refueling currently relies on highly skilled human operators, which is costly and requires additional expensive components like stereoscopic vision systems or LIDAR, and there is a need for automated solutions that can accurately estimate the position and pose of aircraft fuel receptacles using a single camera.
Innovation Solution
The system receives a video frame of the aircraft, generates an initial rendering, refines position and pose estimates using deep learning-based neural networks, and controls the aerial refueling boom to engage the fuel receptacle, enabling automated refueling without the need for a human operator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If automated aerial refueling is implemented using a single camera with deep learning-based pose estimation, then device complexity and cost are reduced, but measurement precision of fuel receptacle position and pose may be insufficient
Solution Approach 1:
The system transitions from 2D image coordinates to 3D pose estimation by generating initial 3D position and pose estimates from 2D video frames, then refining these estimates through iterative optimization that incorporates rendering comparisons across multiple dimensions
Solution Approach 2:
A differentiable renderer is introduced as an intermediary component that generates synthetic renderings of the fuel receptacle based on estimated pose parameters. This renderer acts as a bridge between the monocular camera input and the pose estimation output, enabling accurate measurement through rendering-comparison refinement
2Measurement precision
If stereoscopic vision with dual cameras or LIDAR is used, then measurement precision of range and position is improved, but device complexity and cost increase
Solution Approach 1:
The system extracts depth and pose information from monocular video frames by removing the need for dedicated active sensing components like LIDAR or stereoscopic camera pairs. The pose estimation algorithm extracts all necessary spatial information from standard video input
Solution Approach 2:
The system creates a virtual copy of the fuel receptacle through 3D modeling and rendering. This digital twin is then used for comparison and refinement against the actual visual input, enabling accurate pose estimation without physical measurement sensors
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Aspects of the disclosure provide fuel receptacle position/pose estimation for aerial refueling (derived from aircraft position and pose estimation). A video frame (200), showing an aircraft (110) to be refueled, is received from a single camera. An initial position/pose estimate (506) is determined for the aircraft, which is used to generate an initial rendering (418) of an aircraft model (416). The video frame and the initial rendering are used to determine refinement parameters (44) (e.g., a translation refinement and a rotational refinement) for the initial position/pose estimate, providing a refined position/pose estimate (508) for the aircraft. The position/pose (622) of a fuel receptacle (116) on the aircraft is determined, based on the refined position/pose estimate for the aircraft, and an aerial refueling boom (104) may be controlled to engage the fuel receptacle. Examples extract features from the aircraft in the video frame and the aircraft model rendering, and use a deep learning neural network (NN) to determine the refinement parameters.