Stereo Vision Thermal Tracking for Continuous 3D Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for monitoring flying objects, such as birds and UAVs, face challenges in managing large datasets and are computationally intensive, making long-term, continuous 3D tracking inefficient, especially at remote offshore wind energy locations where species-specific data is lacking and nocturnal species pose difficulties.
Innovation Solution
The method involves stereo vision processing of thermal video frames to create composite motion track images, matching points between stereo pairs, and generating depth maps, allowing for real-time 3D tracking and species identification, with reduced file size and computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video frames are processed individually for 3D tracking, then measurement precision is improved, but productivity deteriorates due to computationally intensive processing
Solution Approach 1:
The patent combines multiple video frames into a single composite motion track image by integrating temporal information across frames. This merging approach maintains 3D tracking precision while significantly reducing computational load by processing one composite image instead of multiple individual frames, thereby improving processing speed and productivity.
Solution Approach 2:
The patent transitions from processing individual 2D video frames to creating composite motion track images that incorporate temporal dimension. By adding the time dimension to the spatial information, the system achieves accurate 3D tracking while reducing the number of processing operations required, thus resolving the contradiction between precision and productivity.
2Measurement precision
If thermal cameras operate at high frame rates, then measurement precision is improved, but use of energy worsens due to increased data volume
Solution Approach 1:
The patent extracts only the essential motion track information from thermal video frames by creating composite motion track images that contain peak pixel values across multiple frames. This extraction process removes redundant data while preserving detection accuracy, thereby reducing the energy required for data processing and transmission.
Solution Approach 2:
Instead of processing every frame at full resolution and frame rate, the patent applies partial processing by selecting only the peak pixel values from sequences of frames to create composite images. This partial action approach maintains sufficient detection accuracy while dramatically reducing the data volume and associated energy consumption.
3Measurement precision
If stereo vision processing is applied to video frames, then measurement precision is improved, but device complexity worsens
Solution Approach 1:
The patent performs preliminary action by creating composite motion track images from video frames before applying stereo vision processing. This pre-processing step simplifies the subsequent stereo matching by providing condensed motion information, thereby reducing the overall system complexity while maintaining depth measurement accuracy.
Solution Approach 2:
The composite motion track image serves as an intermediary between the raw video frames and the stereo vision processing algorithm. This intermediary representation simplifies the data structure and reduces the computational complexity of stereo matching, making the overall system more manageable while preserving measurement precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient, continuous, and real-time 3D monitoring of flying objects, reducing data volume and computational requirements, and facilitating species identification based on flight patterns and behaviors, even at night and in challenging conditions.
Implementation Method 1
for each of a stereo pair of thermal cameras providing thermal video
Data Source
AI summary
Described herein are sensing methods, sensor systems, and non-transitory, computer-readable, storage media having programs for long-duration, continuous monitoring of flying objects during the day or the night and regardless of weather conditions. The methods and systems are computationally efficient and can provide compact, three-dimensional representations of motion from the observed object. A 3D track of the flying object can be generated from a point-matched pair of stereo composite motion track images and not directly from the videos, wherein each composite motion track image is based on a composite of a plurality of video frames composited in part according to video frame numbers.