Live Video Filtering via Client-Server Frame Offloading

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mobile devices lack the computational power to apply complex filtering operations to live video streams, limiting their ability to provide sophisticated filtering capabilities for camera views.

Innovation Solution

A client-server communication system is implemented, where the client device transmits video frames to a server for processing, and the server generates filter processing messages that are used by the client to apply filters to the video stream, allowing for the application of sophisticated filters to live video streams by distributing the computational load.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If complex filtering operations are applied to live video streams on mobile devices, then filtering capability is improved, but computational power requirements increase beyond mobile device capabilities

Engineering Contradiction:

Improvefiltering capabilityVSAvoidcomputational power requirements

Core Design Contradiction:

Adaptability or versatilityVSPower

Solution Approach 1:

The system segments the filtering process into two parts: the mobile device captures and transmits video frames, while the server performs the computationally intensive filtering operations. This division allows complex filters to be applied without overloading the mobile device's processor.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A server acts as an intermediary between the mobile device and the filtering process. The server receives video frames from the mobile device, applies complex filters, and returns filtered frames, thereby enabling sophisticated filtering capabilities without requiring the mobile device to have sufficient computational power.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If video frames are transmitted to a server for processing, then filtering capability is improved, but network communication overhead is introduced

Engineering Contradiction:

Improvefiltering capabilityVSAvoidnetwork communication overhead

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system establishes a persistent connection and prepares the communication channel in advance. Video frames are transmitted in a streamlined manner with pre-allocated buffers and pre-established protocols, reducing the overhead of repeated connection setup and negotiation.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If successive video frames are processed iteratively, then real-time filtering is achieved, but processing latency increases

Engineering Contradiction:

Improvereal-time filteringVSAvoidprocessing latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system processes video frames at regular intervals corresponding to the video frame rate. By batching frames and processing them in periodic cycles rather than individually, the system achieves real-time filtering while minimizing per-frame latency through efficient batch processing on the server.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS10863210B2Client-server communication for live filtering in a camera view

Publication Date: 2020.12.08 FUSION INC

AI summary

Provided are mechanisms and processes for performing live filtering in a camera view via client-server communication. In one example, a first video frame in a raw video stream is transmitted from a client device to a server. The client device receives a filter processing message associated with the first video frame that includes filter data for applying a filter to the first video frame. A processor at the client device creates a filtered video stream by applying the filter to a second video frame that occurs in the video stream later than the first video frame.