Streaming Video Processing via Instruction Graph Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video processing systems incur significant latency as they wait for the entire video to be uploaded before processing can begin, leading to delayed availability of the video for uploaders and users, especially for larger video files.
Innovation Solution
Implementing a system where video processing can start as soon as the video begins uploading, with each stage of the processing pipeline receiving and processing portions of the video as they are streamed, allowing for incremental processing and reduced latency by using a video processor with components like a front end, config engine, controller, and services that manage instruction graphs and metadata storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the system waits for the entire video to be uploaded before processing, then processing accuracy is ensured, but latency increases significantly
Solution Approach 1:
The system performs preliminary actions by starting video processing as soon as the first portion of the video is received, before the entire video is uploaded. The front end initiates processing workflows, creates instruction graphs, and activates services in advance, allowing processing to overlap with the upload process and significantly reducing latency while maintaining processing accuracy through controlled data flow management.
Solution Approach 2:
The video processing system divides the video into multiple portions and processes them segment by segment as they are uploaded. The instruction graph architecture allows different services to process different segments of the video in parallel, with each service receiving and processing specific portions independently, thereby reducing overall latency while ensuring complete processing accuracy.
2Loss of time
If processing starts early during upload, then latency is reduced, but system complexity increases
Solution Approach 1:
The instruction graph serves as an intermediary data structure that manages the complexity of coordinating multiple services processing video portions in parallel. It acts as a mediator between the front end and various processing services, organizing data flow and dependencies without requiring complex point-to-point coordination logic, thereby enabling early processing while managing system complexity through structured abstraction.
Solution Approach 2:
The front end is designed as a universal component that handles multiple functions: receiving video uploads, initiating processing workflows, managing instruction graphs, and coordinating services. This multi-functional design consolidates complexity into a single component rather than distributing it across multiple specialized components, reducing overall system complexity while enabling early processing initiation.
3Productivity
If parallel processing of video portions is implemented, then productivity increases, but coordination complexity increases
Solution Approach 1:
The system implements feedback mechanisms where services report their processing status and data consumption back to the front end through the instruction graph structure. This feedback allows the front end to dynamically coordinate data flow between parallel processing services, ensuring each service receives the correct video portions without conflicts, thereby enabling high throughput while managing coordination complexity through structured feedback loops.
Data Source
AI summary
In one embodiment, a method includes accessing data identifying a video for processing and mapping an instruction graph to the video based on one or more predetermined criteria and the data identifying the video. The instruction graph describes a workflow for processing the video, and the workflow includes a series of services that each provide particular processing of the video. The method includes calling the services in series to process the video based on the instruction graph. The video streams to each service in series as the video processes. A downstream service may begin processing one or more portions of the video after an upstream service has finished processing the one or more portions of the video and before the upstream service has finished processing the video in its entirety.