Virtual Laser Pointer for Aerial Refueling Contact Prediction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Aerial refueling operations face challenges in accurately predicting the point of contact between a refueling boom and a receiver aircraft due to movement, poor lighting, and interpreting 3D information from 2D camera images, which can lead to difficulties in deploying the boom and managing turbulence.

Innovation Solution

A system utilizing a data-driven analyzer trained with supervised learning to estimate the point of contact based on image features, overlaying a virtual laser pointer indication on the video stream to aid the operator in predicting the correct deployment of the refueling boom.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If an operator uses images from a camera to direct the refueling boom to dock with the receptacle, then the operator can control the boom deployment, but it becomes difficult to accurately predict the point of contact due to movement, poor lighting, and interpreting 3D information from 2D images

Engineering Contradiction:

Improvepoint of contact prediction accuracyVSAvoiddifficulty in interpreting 2D camera images for 3D point of contact

Core Design Contradiction:

Measurement precisionVSDifficulty of detecting and measuring

Solution Approach 1:

A virtual laser pointer is introduced as an intermediary visual aid that projects an indication of the estimated point of contact directly onto the video stream of the receiver aircraft. This mediator translates the complex 3D prediction problem into a simple 2D visual overlay that the operator can easily interpret and use to guide boom deployment.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the operator's manual interpretation of 2D camera images with an automated data-driven analyzer that uses machine learning to estimate the point of contact. This substitution transforms the mechanical/visual interpretation task into an automated computational process, improving accuracy while reducing operator cognitive load.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If the operator manually directs the refueling boom using camera images, then the operator has control over the deployment, but the complexity of the targeting operation increases due to the need to interpret 2D images for 3D positioning

Engineering Contradiction:

Improveease of controlling boom deploymentVSAvoidcomplexity of the targeting operation

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The data-driven analyzer performs self-service by automatically analyzing camera images and computing the estimated point of contact without requiring manual intervention from the operator. The system serves itself by processing the visual data and generating the virtual laser pointer indication, freeing the operator to focus on controlling the boom deployment based on the provided guidance.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the complex manual targeting operation with an automated data-driven analysis system. The machine learning model substitutes for the operator's cognitive processing of 2D images into 3D spatial understanding, significantly reducing the perceived complexity of the operation while maintaining or improving accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Measurement precision

If a data-driven analyzer is used to estimate the point of contact, then the accuracy of point of contact prediction is improved, but the system complexity increases due to the need for machine learning models and video stream processing

Engineering Contradiction:

Improvepoint of contact estimation accuracyVSAvoidcomplexity of the data processing system

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The virtual laser pointer serves as a simplifying intermediary that bridges the complex data processing system and the operator. By overlaying the point of contact indication directly on the video stream, it transforms complex computational output into an intuitive visual format, making the system's complexity transparent to the user while maintaining high measurement precision.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The data-driven analyzer performs multiple functions: it processes camera images, estimates the point of contact, and generates the virtual laser pointer indication all in one integrated system. This multi-functionality consolidates what could be separate complex subsystems into a single unified component, managing overall system complexity while delivering precise measurements.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11459116B2Virtual laser pointer as a point of contact indicator for machine learning assisted aerial refueling or other targeting

Publication Date: 2022.10.04 THE BOEING CO

AI summary

An example system includes a processor and a non-transitory computer-readable medium having stored therein instructions that are executable to cause the system to perform various functions. The functions include: (i) acquiring an image of a first aerial vehicle, the image depicting an object of a second aerial vehicle prior to contact between the object and a surface of the first aerial vehicle; (ii) providing the image as input to a data-driven analyzer that is trained in a supervised setting with example images for determining a predetermined point of contact between the object and the surface of the first aerial vehicle; (iii) determining, based on an output of the data-driven analyzer corresponding to the input, an estimated point of contact between the object and the surface of the first aerial vehicle; and (iv) providing the estimated point of contact to a display system.