Refueling Boom 3D Pose Estimation from 2D Keypoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current aerial refueling techniques face challenges in accurately and efficiently conducting refueling operations due to the complexity of tracking the refueling boom and boom tip in three-dimensional space using two-dimensional camera images.
Innovation Solution
The method involves receiving a two-dimensional image from a camera, determining 2D keypoints of the refueling boom, calculating a 6 degree-of-freedom (6DOF) pose using 2D and 3D keypoints, optimizing 3D keypoints associated with moveable boom components, and estimating the position of the refueling boom tip based on the 6DOF pose and optimized keypoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D camera images are used to track the refueling boom, then the system complexity is reduced, but the measurement precision of the boom tip position deteriorates
Solution Approach 1:
The patent applies dimensionality change by transforming 2D camera image coordinates into 3D spatial coordinates through a predefined 3D point model of the refueling boom. The system establishes correspondence between 2D keypoints detected in camera images and their corresponding 3D keypoints in the boom's coordinate system, enabling accurate 3D position estimation while maintaining the simplicity of 2D image capture.
2Productivity
If traditional 2D to 3D transformation methods are used, then the processing speed is maintained, but the reliability of boom position estimation deteriorates
Solution Approach 1:
The patent employs parameter changes by optimizing 3D keypoints associated with moveable components of the refueling boom in response to boom control parameters. This optimization process refines the correspondence between 2D and 3D keypoints, improving the reliability of position estimation while maintaining real-time processing capability through efficient computational methods.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed herein is methods, systems, and aircraft for performing image analysis for aiding refueling operations. A tanker aircraft includes a camera, a refueling boom, a camera configured to generate a two-dimensional (2D) image of the refueling boom, a processor, and non-transitory computer readable storage media storing code. The code is executable by the processor to perform operations including receiving the two-dimensional (2D) image from the camera, determining 2D keypoints of the refueling boom located within the 2D image based on a predefined point model of the refueling boom, determining a 6 degree-of-freedom (6DOF) pose using the 2D keypoints and the corresponding three-dimensional (3D) space 3D keypoints, optimizing 3D keypoints associated with moveable components of the refueling boom in response to a plurality of boom control parameters to produce optimized 3D keypoints, and estimating a position of a tip of the refueling boom based on the 6DOF pose.