Onboard Deep Learning for SAR Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object recognition algorithms have historically underperformed in identifying objects in synthetic aperture radar imagery due to noise statistics and require significant computational resources, leading to inefficiencies and logistical challenges when applied to airborne vehicles, where they often rely on ground-based processing and EO sensors are preferred over SAR sensors.
Innovation Solution
Implementing deep learning techniques, specifically deep neural networks with normalization, sampling, data augmentation, foveation, and cascade architectures, onboard airborne vehicles with low power GPUs or FPGAs to process SAR imagery, enabling real-time object recognition and eliminating the need for ground station communication links.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional object recognition algorithms are applied to SAR imagery, then object identification can be performed, but the accuracy is insufficient due to noise statistics in SAR data
Solution Approach 1:
The patent transforms SAR imagery from its native complex-valued form to magnitude and phase components, then applies deep learning algorithms that operate on these transformed parameters. This parameter transformation enables the algorithm to better handle SAR noise statistics and achieve reliable object identification.
Solution Approach 2:
The patent replaces traditional hand-crafted feature extraction methods with deep learning-based automatic feature learning. The deep neural network automatically learns robust features from SAR imagery, substituting the manual feature engineering approach that failed to achieve sufficient accuracy due to SAR noise characteristics.
2Measurement precision
If high performance computing clusters are used for object recognition, then recognition accuracy improves, but space and power requirements increase significantly
Solution Approach 1:
The patent extracts and implements only the essential deep learning inference functionality on compact GPU/FPGA devices, separating the computationally intensive training phase (performed offline) from the resource-constrained onboard inference phase. This extraction enables accurate object recognition without requiring full HPC cluster resources onboard.
Solution Approach 2:
The patent uses pre-trained deep learning models that were trained on ground-based HPC systems, then deploys these trained models to lightweight onboard devices. The knowledge learned during offline training is copied to the onboard system, enabling accurate inference with minimal onboard computational resources.
3Power
If object recognition is performed on ground stations via wireless downlink, then computational resources are sufficient, but communication delays and reliability problems occur
Solution Approach 1:
The patent performs deep learning model training and feature extraction in advance on ground-based systems, then deploys the trained models to onboard devices. This preliminary action enables the onboard system to perform inference independently without requiring real-time communication with ground stations, eliminating communication delays.
Solution Approach 2:
The patent enables airborne vehicles to perform object recognition independently using onboard deep learning implementations. The system serves itself by processing SAR imagery locally without requiring external ground station support, achieving real-time autonomous object identification.
4Measurement precision
If EO sensors are used instead of SAR sensors, then object recognition performance is better, but the system cannot operate in cloudy conditions or at night
Solution Approach 1:
The patent transforms SAR imagery parameters (magnitude and phase) and applies deep learning algorithms specifically optimized for SAR data characteristics. This parameter transformation and algorithm optimization enable SAR-based object recognition to achieve performance comparable to EO sensors while maintaining SAR's advantage of all-weather, day-night operational capability.
Data Source
AI summary
The present invention is directed to systems and methods for detecting objects in a radar image stream. Embodiments of the invention can receive a data stream from radar sensors and use a deep neural network to convert the received data stream into a set of semantic labels, where each semantic label corresponds to an object in the radar data stream that the deep neural network has identified. Processing units running the deep neural network may be collocated onboard an airborne vehicle along with the radar sensor(s). The processing units can be configured with powerful, high-speed graphics processing units or field-programmable gate arrays that are low in size, weight, and power requirements. Embodiments of the invention are also directed to providing innovative advances to object recognition training systems that utilize a detector and an object recognition cascade to analyze radar image streams in real time. The object recognition cascade can comprise at least one recognizer that receives a non-background stream of image patches from a detector and automatically assigns one or more semantic labels to each non-background image patch. In some embodiments, a separate recognizer for the background analysis of patches may also be incorporated. There may be multiple detectors and multiple recognizers, depending on the design of the cascade. Embodiments of the invention also include novel methods to tailor deep neural network algorithms to successfully process radar imagery, utilizing techniques such as normalization, sampling, data augmentation, foveation, cascade architectures, and label harmonization.