Monocular EO/IR Camera Cloud Detection for UAS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Unmanned Aircraft Systems (UAS) and Remotely Piloted Aircraft (RPA) lack the capability to sense and avoid clouds and other air traffic using passive electro-optical (EO) or infra-red (IR) sensors, limiting their operation in civil airspace due to size, weight, and power constraints, and existing methods for 3D cloud reconstruction are not suitable for monocular cameras or real-time applications.
Innovation Solution
A novel method using monocular EO/IR cameras for autonomous cloud detection and avoidance, processing image sequences to extract 3D cloud information and plan collision-free paths, incorporating incremental estimation of 3D structure and uncertainty, and clustering features for real-time cloud reconstruction and path planning, adaptable to grayscale, color, and stereo camera configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Weight of moving object
If passive EO/IR sensors are used for cloud detection, then size, weight and power constraints are satisfied, but cloud detection and avoidance capability is lost
Solution Approach 1:
The patent replaces active radar sensing with passive EO/IR camera-based optical sensing for cloud detection. The system uses image processing algorithms to extract cloud information from passive optical images, substituting mechanical/electromagnetic active sensing with optical passive sensing combined with computational methods.
Solution Approach 2:
The patent transforms 2D image data from passive EO/IR cameras into 3D cloud formation information through computational processing. By changing the dimensional parameter representation and using multiple image sequences, the system recovers depth information necessary for cloud avoidance without requiring active sensing hardware.
2Reliability
If radar is used for sense and avoid capability, then cloud detection capability is improved, but size, weight and power constraints are violated
Solution Approach 1:
The patent enables passive EO/IR cameras to serve dual purposes: their original mission function (surveillance, reconnaissance, or observation) and cloud detection for compliance with Visual Flight Rules. This multi-functionality eliminates the need for separate dedicated cloud sensing hardware.
Solution Approach 2:
The patent creates a computational model or representation of cloud formations based on processing passive optical images. Instead of directly sensing clouds with specialized hardware, the system creates a digital replica of cloud structures through image processing and 3D reconstruction algorithms.
3Device complexity
If monocular camera is used for cloud detection, then device complexity is reduced, but 3D cloud reconstruction capability is lost
Solution Approach 1:
The patent recovers 3D information from 2D monocular images by introducing temporal dimension through image sequences. The system uses temporal coherence and motion parallax across multiple frames to infer depth, effectively adding a time dimension to compensate for the lack of spatial stereo information.
Solution Approach 2:
The patent introduces computational algorithms as an intermediary between the monocular camera and cloud detection. Image processing algorithms, feature tracking, and 3D reconstruction methods act as intermediaries to extract depth and spatial information that is not directly available from single-camera 2D images.
4Reliability
If real-time cloud detection is implemented, then operational safety is improved, but processing time and computational load increase
Solution Approach 1:
The patent performs preliminary processing of image sequences to pre-identify potential cloud regions and track features across frames. By preparing and pre-processing data in advance, the system reduces the computational burden during critical real-time decision-making moments, enabling faster cloud avoidance responses.
Data Source
AI summary
A computerized aircraft system, such as an unmanned aircraft system (UAS) is provided with a cloud detection system. The UAS includes a monocular electro-optic or infra-red camera which acquires consecutively, in real time, a plurality of images within a field of view of the camera. The system identifies feature points in each of the consecutive images, and generates macro representations of cloud formations (3D representations of the clouds) based on tracking of the feature points across the plurality of images. A cloud avoidance system takes in nominal own-ship waypoints, compares those waypoints to the 3D cloud representations and outputs modified waypoints to avoid the detected clouds.