Aerial Refueling Boom Tip Position Estimation Using Monocular Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Aerial refueling currently requires highly skilled human operators and expensive equipment like stereoscopic vision systems or LIDAR, which increases costs and complexity.
Innovation Solution
The use of computer vision techniques, including monocular vision with deep learning algorithms, to estimate the position and pose of a fuel receptacle and boom tip, enabling automated aerial refueling by determining keypoints in video frames and controlling the refueling boom to engage the receptacle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereoscopic vision with dual cameras or LIDAR is used, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent uses a single camera to capture video frames and creates a 2D projection of the 3D fuel receptacle and boom tip positions. Instead of using multiple cameras or LIDAR to directly measure 3D positions, the system captures a 2D image and processes it through computer vision algorithms to estimate the required positions, effectively using a simple copy (2D image) to represent complex 3D spatial information
Solution Approach 2:
The patent replaces complex mechanical/optical measurement systems (dual cameras, LIDAR) with a computational approach using a single camera and computer vision algorithms. The mechanical system of multiple sensors is substituted with an information-processing system that uses image processing and keypoint detection to achieve the same measurement function
2Measurement precision
If stereoscopic vision or LIDAR is used, then measurement precision is improved, but cost increases
Solution Approach 1:
The patent replaces expensive, complex measurement equipment (stereoscopic vision systems, LIDAR) with a single, inexpensive camera. The system uses affordable computer vision software and processing algorithms instead of costly hardware, significantly reducing the overall system cost while maintaining the capability to estimate fuel receptacle and boom tip positions
3Reliability
If human operators are used, then operational reliability is improved, but device complexity increases
Solution Approach 1:
The patent implements an automated system where the computer vision algorithm independently detects keypoints, estimates positions and poses, and controls the boom engagement without human intervention. The system serves itself by using the camera feed to automatically make refueling decisions, eliminating the need for human operators to interpret visual information and control the boom
Solution Approach 2:
The patent replaces the human operator's visual system and manual control mechanisms with an automated computer vision and control system. The human cognitive processing and manual manipulation are substituted with automated image processing algorithms and robotic control, eliminating the need for operator accommodation while maintaining operational reliability
Data Source
AI summary
Aspects of the disclosure provide fuel receptacle and boom tip position and pose estimation for aerial refueling. A video frame is received and within the video frame, aircraft keypoints for an aircraft to be refueled are determined. Based on at least the aircraft keypoints, a position and pose of a fuel receptacle on the aircraft is determined. Within the video frame, a boom tip keypoint for a boom tip of an aerial refueling boom is also determined. Based on at least the boom tip keypoint, a position and pose of the boom tip is determined. Based on at least the position and pose of the fuel receptacle and the position and pose of the boom tip, the aerial refueling boom is controlled to engage the fuel receptacle. Some examples overlay projections of an aircraft model on displayed video for a human observer.