Monocular Fuel Receptacle Localization for Automated Boom Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current aerial refueling methods rely heavily on human operators, which are costly and require additional expensive components like stereoscopic vision, lidar, or radar, and lack efficient automation for precise control of the refueling process.
Innovation Solution
An automated system using computer vision that receives video frames from a single camera, identifies aircraft and refueling boom positions, generates segmentation masks, and controls the refueling boom to engage the fuel receptacle, with optional fusion of proximity sensor data for improved accuracy and safety monitoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If human operators are used for aerial refueling, then operational flexibility and decision-making capability are improved, but operational cost and device complexity increase
Solution Approach 1:
The system enables the refueling process to be performed autonomously by the aircraft and boom system themselves, using automated computer vision-based localization and control algorithms to replace human operators in the refueling execution phase, thereby reducing operational costs while maintaining safety through continuous automated monitoring
Solution Approach 2:
The patent replaces the mechanical human operator system with an automated computer vision system that uses monocular video feeds, segmentation masks, and algorithmic processing to detect fuel receptacles and control the refueling boom, eliminating the need for expensive stereoscopic vision systems, LiDAR, or radar while achieving comparable or superior precision
2Measurement precision
If stereoscopic vision with dual cameras is used, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts and eliminates the unnecessary complexity of dual-camera stereoscopic vision systems, LiDAR, and radar components, retaining only a single monocular camera system that achieves equivalent localization precision through advanced computer vision algorithms including segmentation masks and perspective projection models
Solution Approach 2:
The system creates a virtual 3D representation of the fuel receptacle and aircraft geometry through segmentation masks and perspective projection from a single camera, effectively copying the depth information that would traditionally require multiple sensors, thereby achieving accurate localization without the hardware complexity of stereoscopic systems
3Productivity
If automated control is implemented, then productivity is improved, but measurement precision requirements increase
Solution Approach 1:
The system implements continuous real-time feedback through the monocular vision system that tracks the relative positions of the boom tip and fuel receptacle, using segmentation masks and perspective projection to provide automated position information that enables precise control adjustments while maintaining high operational efficiency
Solution Approach 2:
The single monocular camera system performs multiple functions simultaneously including fuel receptacle detection, aircraft geometry estimation, boom tip positioning, and collision avoidance monitoring, thereby achieving high measurement precision requirements for automated control without requiring multiple specialized sensors that would reduce productivity
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Aspects of the disclosure provide solutions for automated air-to-air refueling (A3R) and assisted air-to-air refueling. Examples include: receiving a video frame; identifying an aircraft to be refueled from within the video frame; segmenting the video frame to generate a segmentation mask matching the aircraft within the video frame; based on at least the segmentation mask, determining a position of a fuel receptacle on the aircraft; determining a position of a boom tip of the aerial refueling boom; and controlling the aerial refueling boom to engage the fuel receptacle based on at least the position of the fuel receptacle and the position of the boom tip.