Video Appearance Infilling With Diffusion Models for Occluded Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud computing architectures face challenges in latency, availability, bandwidth usage, data privacy, and network security when processing large volumes of data from remote locations, particularly in real-time applications and AI/ML workloads, and transmission of sensor data over wireless links is slow and expensive, while artifacts in video data obscure object tracking.
Innovation Solution
Implementing edge computing units with trained machine learning models that refine object tracking models in real-time to overcome occlusions by artifacts, infilling obscured object pixels to maintain visibility and optionally removing artifacts from video frames, using ruggedized systems for harsh environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If centralized processing architecture is used, then data can be processed at remote locations, but latency increases and real-time processing capability deteriorates
Solution Approach 1:
The patent divides the processing system into distributed edge computing units deployed at remote locations, each capable of autonomous real-time processing. This segments the centralized processing function into multiple distributed nodes, enabling local real-time operations without waiting for centralized processing, thus reducing latency while maintaining processing reliability.
2Productivity
If sensor data is transmitted over wireless communication links, then data can be sent from remote locations, but transmission speed decreases and bandwidth usage increases
Solution Approach 1:
The patent performs preliminary processing and data filtering at the edge computing units before transmission to centralized systems. By pre-processing the data locally to extract only essential information and reduce data volume, the system minimizes bandwidth consumption during transmission while maintaining processing productivity.
3Measurement precision
If artifacts are present in video data, then environmental context is captured, but object tracking accuracy deteriorates
Solution Approach 1:
The patent introduces AI-based artifact removal as an intermediary processing step between capturing video data and performing object tracking. The artifact removal model acts as a mediator that eliminates occluding artifacts from the video frames, creating a cleaned-up version of the scene that allows accurate object tracking while preserving the original environmental context.
4Loss of time
If edge computing units are deployed at remote locations, then processing latency is reduced, but device complexity and system cost increase
Solution Approach 1:
The patent designs edge computing units as universal, multi-functional devices that can perform various processing tasks (object detection, artifact removal, data filtering) using a single integrated AI model framework. This universality reduces system complexity by eliminating the need for multiple specialized devices while maintaining low latency processing capabilities through centralized model deployment.
Data Source
AI summary
Disclosed are systems and methods to track an object in a scene even when the object is partially or wholly obscured by an artifact, such as another object. As the tracked object moves through the scene, frames of video are processed and used to refine and tune a diffusion model that predicts an appearance of the tracked object in future frames of the video as well as the appearance of artifacts in the frames of the video. Frames of the video may then be enhanced to illustrate the tracked object as if the tracked object were visible through the artifact.