Camera Tracking via Commanded Motion and Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current real-time camera tracking systems using pan/tilt mechanisms with computer vision techniques are limited by computational inefficiencies, requiring wide field-of-view, low resolution images to track moving objects effectively, which results in reduced image quality and accuracy, especially when objects move significantly across the frame.
Innovation Solution
The integration of commanded motion with optical flow to derive object motion, using a Kalman filter to predict object locations and control the pan and tilt unit, allowing for real-time tracking with a narrow field-of-view, high-resolution image capture, by classifying background and noise points and utilizing a CMOS color camera with a separate graphics processing unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a wide field of view camera is used to track moving objects in real-time, then the computational requirements are reduced and tracking is feasible with current hardware, but the image resolution and quality deteriorate
Solution Approach 1:
The system dynamically adjusts the camera field of view based on object position and motion. The camera transitions from a wide field of view during initial acquisition to a narrow field of view during active tracking, allowing high resolution images while maintaining real-time tracking capability through coordinated pan-tilt movements
Solution Approach 2:
The system adds temporal dimension to the tracking process by using predictive algorithms (Kalman filter) to forecast object position. This allows the camera to proactively reposition before the object leaves the current frame, maintaining high resolution tracking without requiring continuously wide field of view
2Measurement precision
If a narrow field of view camera is used to capture high resolution images, then the image quality improves, but the computational complexity increases and real-time tracking becomes difficult
Solution Approach 1:
The system performs preliminary actions by predicting object future position using Kalman filter before the object actually moves out of frame. This proactive approach reduces the computational burden of real-time image processing by minimizing the frequency of full-frame searches, as the narrow field of view is strategically repositioned in advance
Solution Approach 2:
The system implements feedback mechanisms where object detection results from high-resolution narrow field of view images are fed back to update the predictive model. This closed-loop control optimizes the balance between computational complexity and tracking accuracy by adjusting prediction parameters based on actual object behavior patterns
3Measurement precision
If template matching is used to track objects, then the system can identify specific objects, but the system fails when object appearance changes
Solution Approach 1:
The system replaces traditional template matching mechanics with physics-based motion prediction mechanics. Instead of relying on visual pattern recognition that fails when appearance changes, the system uses Kalman filter to predict object position based on motion dynamics, making tracking robust to appearance variations while maintaining identification accuracy
4Productivity
If general purpose motion detection methods are used, then the system can detect moving objects, but the background must be feature rich which reduces the object's prominence in the image
Solution Approach 1:
The system extracts and isolates the object of interest from the background by using narrow field of view imaging combined with predictive positioning. This extraction approach eliminates the need for feature-rich backgrounds, as the object is directly targeted and tracked in isolation, improving detection accuracy without requiring complex background differentiation
Data Source
AI summary
A new apparatus and method for tracking a moving object with a moving camera provides a real-time, narrow field-of-view, high resolution and on target image by combining commanded motion with an optical flow algorithm for deriving motion and classifying background. Commanded motion means that movement of the pan, tilt and zoom (PTZ) unit is “commanded” by a computer, instead of being observed by the camera, so that the pan, tilt and zoom parameters are known, as opposed to having to be determined, significantly reducing the computational requirements for tracking a moving object. The present invention provides a single camera pan and tilt system where the known pan and tilt rotations are used to calculate predicted optical flow points in sequential images, so that resulting apparent movement can be subtracted from the movement determined by an optical flow algorithm to determine actual movement, following by use of a Kalman filter algorithm to predict subsequent locations of a determined moving object and command the pan and tilt unit to point the camera in that direction.