Video Object Tracking Using Spatiotemporal Neural Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision systems for smart driving assistance face challenges in efficiently and robustly detecting and tracking objects from video inputs, requiring complex and computationally intensive neural networks.
Innovation Solution
A method involving the formation of spatiotemporal data volumes from video frames, slicing these volumes into spatiotemporal images, and enhancing features of interest using a neural network, particularly a convolutional neural network, trained in the spatiotemporal domain with synthesized data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complex neural networks are used for object detection and tracking, then detection accuracy and robustness are improved, but computational complexity and processing time increase
Solution Approach 1:
The patent transforms 2D image frames into a 3D spatiotemporal data volume by adding the temporal dimension. This dimensional transformation allows the neural network to process spatial and temporal information simultaneously, improving detection accuracy while enabling simpler network architecture through the inherent temporal context that reduces computational complexity
Solution Approach 2:
The patent performs preliminary feature extraction and creates a spatiotemporal data volume before feeding data to the neural network. This preliminary organization of data in the spatiotemporal domain prepares the information in a way that simplifies subsequent neural network processing, allowing for simpler network structures while maintaining high detection accuracy
2Productivity
If traditional image processing methods are used, then processing speed is maintained, but detection robustness and accuracy deteriorate
Solution Approach 1:
The patent merges multiple image frames into a single spatiotemporal data volume structure, combining spatial and temporal information. This merging allows the neural network to process multiple frames simultaneously with a single computational operation, maintaining processing speed while significantly improving detection robustness through temporal context awareness
Solution Approach 2:
The patent creates a computational copy of the video data in the form of a spatiotemporal data volume. This copy allows the neural network to perform complex analysis without repeatedly processing individual frames, maintaining processing efficiency while improving detection robustness through the structured representation
3Reliability
If large annotated datasets are used for training, then model accuracy is improved, but data preparation time and cost increase
Solution Approach 1:
The patent changes the fundamental parameter of data representation from 2D images to 3D spatiotemporal data volumes. This parameter change allows the model to learn temporal patterns and relationships, improving accuracy with smaller datasets because the spatiotemporal structure itself provides valuable contextual information that reduces the need for large amounts of annotated data
Data Source
AI summary
A method and a computing device for object detection and tracking from a video input are described. The method and the computing device may be used to, for example, track objects of interest, such as lane markings, in traffic. A plurality of frames corresponding to a video may be analyzed in a spatiotemporal domain by a neural network. The neural network may be trained using data synthesized in the spatiotemporal domain.