Skeleton Detection Tracking Client-Server Video Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional algorithms for skeleton detection in video streams are not fast enough to run in real-time on mobile computing devices, especially when dealing with complex filtering operations and self-occlusions, limiting their ability to process live video streams effectively.
Innovation Solution
A client-server communication system is implemented where a client device receives skeleton detection messages from a remote server, allowing for the identification and tracking of skeletons across video frames, with the server performing initial detection and the client performing client-side processing to maintain real-time capabilities and reduce processing load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional skeleton detection algorithms are used on mobile devices, then real-time processing is achieved, but detection precision and handling of complex poses deteriorate
Solution Approach 1:
The system segments the skeleton detection task between server and client device. The server performs computationally intensive skeleton detection on video frames and returns results to the mobile device, enabling both high precision and real-time performance by dividing the workload across different processing environments
Solution Approach 2:
The patent introduces an intermediary communication system between the server and client device. The server acts as an intermediary that processes video frames and returns skeleton data, while the client device receives and applies this data locally, resolving the contradiction between processing power and real-time response
2Measurement precision
If complex filtering operations based on skeleton detection are applied to live video streams, then visual effect quality is improved, but computing resource consumption increases beyond mobile device capabilities
Solution Approach 1:
The patent extracts the computationally intensive skeleton detection and filtering operations from the mobile device and relocates them to a server environment. The mobile device only performs lightweight tasks such as receiving skeleton data and applying pre-computed filters, dramatically reducing power consumption while maintaining high visual quality
Solution Approach 2:
The system creates a simplified copy of the skeleton detection functionality on the client device, receiving pre-processed skeleton data from the server. This allows the mobile device to apply visual filters based on skeleton information without performing full skeleton detection locally, reducing computing resource requirements
3Measurement precision
If skeleton detection is performed on every video frame, then detection accuracy is improved, but processing time and energy consumption increase
Solution Approach 1:
The system implements periodic skeleton detection where the server processes video frames at optimized intervals rather than continuously analyzing every frame. This periodic processing maintains adequate detection accuracy while significantly reducing processing time and energy consumption compared to frame-by-frame analysis
Data Source
AI summary
Provided are mechanisms and processes for performing skeleton detection and tracking via client-server communication. In one example, a server transmits a skeleton detection message that includes position data for a skeleton representing the structure of an object depicted in a first video frame in a raw video stream at a client device. Based on the initial position data, a processor identifies intervening position data for the skeleton in one or more intervening video frames that are temporally located after the first video frame in the raw video stream. A filtered video stream is then presented by altering the raw video stream based at least in part on the first position data and the intervening position data.