Receiver Aircraft Keypoint Monitoring for Automated Boom Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional air-to-air refueling operations heavily rely on human input and judgment, which can lead to inefficiencies, increased costs, and safety risks due to the reliance on human operators.
Innovation Solution
Implementing a computer-automated vision-based system using machine learning (ML) to detect keypoint locations on a receiver aircraft, convert them to 3D coordinates, and use closed-loop control logic to automate the refueling process, with real-time error monitoring to ensure accuracy and safety.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a computer-automated vision-based system using machine learning is implemented to detect keypoint locations and automate the refueling process, then productivity and reliability are improved, but device complexity increases
Solution Approach 1:
The patent replaces manual mechanical operations with an automated vision-based system. Machine learning algorithms detect keypoint locations on the receiver aircraft and automatically generate boom control commands, substituting the human operator's mechanical judgment and control actions with computational processes. This substitution improves productivity by enabling faster, more consistent operations while managing complexity through software-based solutions.
2Reliability
If real-time error monitoring based on keypoint rate of change is implemented to ensure safety and accuracy, then reliability is improved, but use of energy and computational resources increases
Solution Approach 1:
The patent implements real-time error monitoring by continuously comparing detected keypoint locations with predicted keypoint positions based on rate of change calculations. This feedback mechanism detects deviations that may indicate safety issues or tracking errors, allowing the system to alert operators or correct course. The feedback loop operates at standard video frame rates, providing timely safety monitoring without requiring excessive computational resources by using efficient differential calculations between consecutive frames.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method of operating an aerial refueling aircraft includes an aircraft controller receiving a time-series of images of a receiver aircraft from an aircraft camera system and, using these images, determines an aircraft type and predefined keypoint locations for the receiver aircraft. Using a position estimation machine learning algorithm, the controller modifies each aircraft image to place keypoints at the aircraft keypoints on the receiver aircraft. The controller determines a rate of change for each keypoint in each image and, using these keypoint rates of change, predicts keypoint locations for the keypoints in each image. The controller determines a system confidence value by comparing the keypoints in each image with its corresponding predicted keypoint location. Responsive to determining that the system confidence value exceeds a threshold confidence value, the controller controls movement of the aircraft's refueling boom in accordance with a predefined boom control protocol. (Fig. 3)