Machine-Learning Video Inference with Parallel CPU-GPU Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for digital video processing using machine-learning techniques are resource-intensive and computationally expensive, suffering from inefficiencies such as high computing costs, latency, and inflexibility due to the fan-out approach that requires multiple CPU/GPU containers and additional components.
Innovation Solution
A video inference system utilizing improved architectures that process digital videos by extracting frames on CPUs and inferencing on GPUs, eliminating the need for job distributors and single video aggregators, and processing each video as a single unit, thereby reducing latency and overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional fan-out approach with multiple CPU/GPU containers is used, then digital videos can be processed, but computing resources are excessive and costs are high
Solution Approach 1:
The patent merges multiple CPU/GPU container operations into a unified processing architecture where a single GPU service handles inferencing for multiple video processing pipelines. This consolidation eliminates redundant computing resources while maintaining video processing throughput by sharing the GPU resource across multiple CPU-based preprocessing and postprocessing operations.
Solution Approach 2:
The GPU service is designed with universal functionality to serve multiple video processing tasks simultaneously. A single GPU instance performs inferencing for different video streams and processing pipelines, making the computing resource multi-functional rather than dedicated to single-purpose containers, thereby reducing total resource requirements.
2Productivity
If conventional fan-out approach with job distributors and video aggregators is used, then videos can be processed, but system complexity increases
Solution Approach 1:
The patent extracts and eliminates the intermediate components of job distributors and video aggregators from the processing pipeline. By allowing CPU services to directly interact with the GPU service through standardized interfaces, the system removes unnecessary complexity layers while preserving the core video processing capability through direct communication pathways.
Solution Approach 2:
Instead of using a centralized job distributor to allocate tasks and a video aggregator to collect results (traditional top-down approach), the patent inverts the architecture to allow individual CPU processing services to directly request and receive GPU inferencing services. This bottom-up approach simplifies the system by eliminating the need for centralized coordination components.
3Reliability
If conventional processing methods are used, then videos can be inferenced, but processing time and latency are excessive
Solution Approach 1:
The patent implements preliminary action by performing CPU-based preprocessing operations (such as video decoding, frame extraction, and data preparation) before GPU inferencing. This separation allows the GPU to receive pre-prepared data ready for immediate processing, reducing idle time and overall latency while maintaining processing accuracy through proper preprocessing.
Solution Approach 2:
The architecture enables continuous useful action by allowing multiple CPU processing services to operate in parallel with the GPU service. While the GPU is performing inferencing on one video stream, other CPU services can simultaneously prepare data for subsequent inferencing tasks, eliminating idle periods and maintaining continuous productive operation across the system.
Data Source
AI summary
This disclosure describes one or more implementations of a video inference system that utilizes machine-learning models to efficiently and flexibly process digital videos utilizing various improved video inference architectures. For example, the video inference system provides a framework for improving digital video processing by increasing the efficiency of both central processing units (CPUs) and graphics processing units (GPUs). In one example, the video inference system utilizes a first video inference architecture to reduce the number of computing resources needed to inference digital videos by analyzing multiple digital videos utilizing sets of CPU/GPU containers along with parallel pipeline processing. In a further example, the video inference system utilizes a second video inference architecture that facilitates multiple CPUs to preprocess multiple digital videos in parallel as well as a GPU to continuously, sequentially, and efficiently inference each of the digital videos.