3D Voxel Segmentation for Overlapping Objects in Video Surveillance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image and video processing techniques for object segmentation in video surveillance systems face challenges in segmenting connected or overlapping objects in 2D frames without depth information, leading to unreliable results due to changing parameters like luminance and motion, and require significant processing power to handle high volumes of live streamed data.
Innovation Solution
A segmentation system that utilizes a processor and memory to detect camera motion, convert 2D frames into 3D cloud points, and apply voxelization and 3D connected component labeling to segment objects, leveraging depth and color data to distinguish valid voxels and discard noisy frames, enabling real-time processing and accurate object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image processing techniques are used for object segmentation in 2D video frames, then object detection can be performed, but segmentation becomes unreliable due to changing parameters like luminance and motion
Solution Approach 1:
The patent transforms 2D video frames into 3D point clouds by introducing depth information through time-based analysis. By converting spatial coordinates (x, y) into three-dimensional coordinates (x, y, z) where z represents depth derived from temporal changes, the system enables reliable object segmentation in 3D space, overcoming the limitations of 2D segmentation受限于 changing luminance and motion parameters
Solution Approach 2:
The system performs preliminary actions by detecting camera motion and compensating for it before object segmentation. The method identifies and removes frames containing camera motion artifacts, then performs segmentation on stabilized frames, ensuring reliable object detection before final segmentation occurs
2Ease of operation
If manual detection of human activities is performed in video surveillance systems, then detection can be done with simple observational skills, but it becomes highly difficult with increase in number of surveillance cameras
Solution Approach 1:
The patent replaces manual mechanical observation with automated computer-based image processing and analysis systems. The system uses algorithms to automatically detect objects, segment them in 3D space, and identify human activities, eliminating the need for manual monitoring of multiple cameras while significantly improving productivity and detection accuracy
3Measurement precision
If segmentation of individual overlapping components is performed to improve object detection accuracy, then detection precision improves, but it requires a lot of processing power
Solution Approach 1:
By performing segmentation in 3D space rather than 2D, the patent naturally separates overlapping objects that occupy different depth positions. The 3D point cloud representation allows objects to be distinguished based on their spatial coordinates (x, y, z), reducing the computational complexity required for separating overlapping components compared to 2D segmentation methods
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is a method for segmenting a plurality of objects from a two-dimensional (2D) video captured through a depth camera and an RGB/G camera. The method comprises detecting camera motion in each 2D frame of the plurality of 2D frames from the 2D video and generate a first set of 2D frames without any camera motion. The method further comprises generating a plurality of cloud points for the first set of 2D frames corresponding to each pixel associated a 2D frames in the first set of 2D frames. The method further comprises generating a 3D grid comprising a plurality of voxels. The method further comprises determining valid voxels and an invalid voxels in the 3D grid. Further, a 3D connected component labeling technique is applied on to the set of valid voxels to segment the plurality of objects in the 2D video.