Spherical Video Object Detection With Projection Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manually framing wide field of view videos to identify objects is difficult and time-consuming, especially in spherical videos where determining which spatial extent contains interesting views is challenging.
Innovation Solution
The system generates multiple perspective projections of spherical videos, performs object detection in each projection, adjusts detection scores based on object proximity to boundaries, filters out redundant detections, and tracks objects across projections to accurately identify and present relevant content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple perspective projections are generated for object detection, then object detection accuracy is improved, but device complexity increases
Solution Approach 1:
The spherical video is divided into multiple perspective projections, each covering a specific field of view (e.g., 120-130 degrees). Object detection is performed independently on each projection, which simplifies the detection task compared to processing the entire spherical video at once. This segmentation approach improves detection accuracy while managing system complexity through distributed processing.
Solution Approach 2:
The system transitions from processing spherical video in its native spherical coordinate system to multiple 2D perspective projections. This dimensional transformation allows standard 2D object detection algorithms to be applied effectively, improving accuracy by leveraging proven 2D detection techniques while the overlapping projections ensure complete coverage of the spherical space.
2Measurement precision
If object detection is performed in multiple perspective projections, then object identification accuracy is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by generating multiple perspective projections with overlaps before object detection. The overlap regions are strategically designed to ensure that objects near boundaries are captured in multiple projections, allowing for more accurate detection and score aggregation without requiring excessive processing of redundant regions.
Solution Approach 2:
Instead of processing the entire spherical video multiple times, the system creates multiple 2D projection copies of the spherical content. Each projection is a simplified 2D representation that can be processed efficiently by standard object detection algorithms, reducing overall processing time while maintaining detection accuracy through the use of multiple views.
3Reliability
If detection scores are modified based on proximity to boundaries, then detection reliability is improved, but processing complexity increases
Solution Approach 1:
The system applies local quality adjustment by modifying detection scores based on the local position of detected objects within each projection. Objects closer to the boundaries of a projection receive different score adjustments compared to objects in the center. This local adjustment improves reliability by accounting for the fact that boundary objects may be partially visible or detected with lower confidence, while adding minimal complexity through simple distance-based scoring modification.
Data Source
AI summary
A wide field of view video is split into multiple perspective projections, with individual perspective projections providing a two-dimensional view of a spatial extent of the wide field of view video. Object detection is performed within individual perspective projections to determine the placement of the objects within individual perspective projections. The placement of the objects are projected back into the wide field of view video to merge the detections. Redundant detection are filtered out and the remaining detections are used to perform object tracking in the wide field of view video.