Event-Camera Navigation Control for Agile UAV Obstacle Avoidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cameras are inadequate for agile unmanned aerial vehicles (UAVs) due to their low data rate, which limits their ability to perform fast and safe navigation in cluttered environments, as they provide data at a rate of 30-60 Hz, making them unsuitable for agile navigation and obstacle avoidance.
Innovation Solution
The use of event cameras, which provide a stream of data representing pixel location, timestamp, and polarity, allowing for a compressed representation that is processed by a neural network model trained using reinforcement learning to generate control actions for the UAV, enabling faster and more efficient obstacle avoidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CMOS based cameras are used for perception, then high resolution data is provided, but the data rate is limited to 30-60 Hz which is insufficient for agile navigation
Solution Approach 1:
The patent transitions from conventional frame-based camera operation (30-60 Hz) to event-based asynchronous operation, fundamentally changing the temporal sampling parameter. Event cameras generate data only when changes occur, enabling effectively unlimited data rates while maintaining spatial resolution, thus resolving the contradiction between measurement precision and productivity.
2Productivity
If event cameras are used to increase data rate, then perception speed is improved, but computational load increases
Solution Approach 1:
The patent extracts and processes only the most relevant features from the high-rate event camera data stream using a simplified neural network architecture. By focusing computation on essential collision avoidance features rather than processing all event data comprehensively, the system maintains high perception data rate while managing computational load within onboard constraints.
3Loss of time
If fast processing algorithms are used for real-time control, then response time is reduced, but accuracy may be compromised
Solution Approach 1:
The neural network model is trained in advance using reinforcement learning on simulated environments, preparing optimal control policies beforehand. During real-time operation, the pre-trained model rapidly processes event camera data and generates control commands without requiring complex real-time computation, thus achieving both low latency and high accuracy by shifting computational burden to the training phase.
Data Source
AI summary
A computer implemented method for controlling a system moving through an environment includes receiving a stream of event data from an event camera, the stream of event data representing a pixel location, a time stamp, and a polarity for each event detected by the event camera. A compressed representation of the stream of data is generated. The compressed representation is provided to a neural network model trained on prior compressed representations using reinforcement learning to learn actions for controlling the system. A control action is generated via the neural network model to control the movement of the system.