Monocular Moving Object Detection via Parallel CNN Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting moving objects in monocular video streams struggle to accurately identify objects moving independently of the observer, especially when the camera is in motion, due to over-segmentation of rigid objects and under-segmentation of nearby moving objects.
Innovation Solution
A system comprising an object-feature extractor and a motion-feature extractor, both based on trained convolutional neural networks, processes monocular video data to generate class-agnostic object features and motion features, respectively, which are combined to produce segmentation masks identifying moving objects, even when only a part of the object is moving, using a motion-cost-volume calculator and a hybrid task cascade head.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional video-segmentation methods treat objects as rigid bodies and identify groups of pixels undergoing the same rigid motion, then the detection system can process video data efficiently, but it over-segments the scene by identifying individual parts of non-rigid objects as separate objects
Solution Approach 1:
The patent divides the object detection task into two independent segmentation processes: rigid object detection and non-rigid object detection. Each process uses specialized neural network architectures optimized for its target object type, allowing accurate segmentation of both rigid and non-rigid objects without mutual interference, thereby resolving the over-segmentation problem while maintaining processing efficiency
Solution Approach 2:
The patent introduces dynamic modeling capabilities by using pose estimation networks for non-rigid objects that can capture temporal variations in object configuration. This allows the system to track and segment non-rigid objects across frames by learning their dynamic patterns, preventing over-segmentation while maintaining computational efficiency through temporal coherence
2Device complexity
If conventional video-segmentation methods treat nearby objects moving along similar trajectories as a single object, then the system reduces computational complexity, but it under-segments the scene and fails to identify individual objects
Solution Approach 1:
The patent applies segmentation to the detection task by creating separate detection pipelines for rigid and non-rigid objects. This division allows each pipeline to specialize in detecting specific object types with distinct motion patterns, enabling accurate separation of nearby objects that would otherwise be confused by a unified detection approach
Solution Approach 2:
The patent employs local quality by using object-specific feature extraction and classification mechanisms within each detection pipeline. The rigid object detector and non-rigid object detector use different feature representations and decision criteria tailored to their respective object types, enabling precise discrimination between nearby objects with similar trajectories
3Device complexity
If the system uses a single detection model for both rigid and non-rigid objects, then the system structure is simplified, but it cannot accurately detect non-rigid objects whose parts move in different manners
Solution Approach 1:
The patent segments the detection system into specialized sub-systems: a rigid object detection network and a non-rigid object detection network. Each sub-system is optimized for its specific object type with appropriate architectural choices, achieving high adaptability to different object types while managing complexity through modular design
Solution Approach 2:
The patent incorporates dynamic modeling specifically for non-rigid objects using pose estimation networks that can capture temporal variations in object configuration. This dynamic approach allows the system to adapt to the varying structures of non-rigid objects across different frames, enhancing versatility without requiring a completely different system architecture
4Device complexity
If the system processes monocular video data without distinguishing camera motion from object motion, then the processing pipeline remains simple, but it cannot disambiguate ego motion from actual object motion when the camera moves
Solution Approach 1:
The patent extracts and separately processes camera motion information from the video sequence using ego-motion estimation techniques. By isolating the camera motion component, the system can then focus on detecting object motion relative to the camera, significantly improving motion detection accuracy while adding only moderate computational complexity
Solution Approach 2:
The patent changes the reference frame parameters by transforming object motion detection from the camera-centered frame to an ego-motion-compensated frame. This parameter transformation allows the detection algorithms to operate on motion-residual data where camera motion has been removed, improving accuracy without requiring fundamentally more complex processing
Data Source
Figure 1~2B
Figure 3~4B
Figure 5~6
AI summary
Systems and methods detect moving objects in image data of a monocular video stream using a trained CNN-based object-feature extractor (2) and a trained CNN-based motion-feature extractor (4) operating in parallel branches on the input video stream, with the feature sets produced by the features extractors being concatenated and supplied to a detection head (6). The object-feature extractor (2) is trained to generate output comprising class-agnostic features. The motion-feature extractor (4) is trained to generate output including first (backward) motion-features (Mt―1) relating to motion between a subject image frame and the previous image frame and second (forward) motion-features (Mt+1) relating to motion between the subject image frame and the next image frame, based on respective backward and forward motion cost volumes.