Tele-Robot WebRTC Stream Forking for Edge Visual Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional tele-robot systems lack standard mechanisms for real-time multimedia conferencing that efficiently stream live feeds to edge devices for processing, as existing WebRTC connections do not meet real-time requirements and occupy camera resources, hindering effective visual analytics for multiple users.
Innovation Solution
A method and system utilizing Web Real-Time Communication (WebRTC) to establish edge-inclusive real-time multimedia tele-conferencing by forming peers with unique type identifiers, enabling peer-to-peer connections between tele-robots and edge devices for live stream processing and visual analytics, while maintaining seamless multimedia conferencing with human participants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If WebRTC is used for real-time multimedia conferencing, then real-time communication is achieved, but camera resources are occupied and cannot be streamed to edge devices
Solution Approach 1:
The system segments the camera resource usage by creating separate peer-to-peer connections: one WebRTC connection for multimedia conferencing and another dedicated connection for streaming to the edge device. This segmentation allows the same camera resource to be utilized in parallel for both conferencing and edge processing without mutual interference.
Solution Approach 2:
The patent introduces a signaling server as an intermediary that coordinates multiple peer-to-peer connections. The signaling server manages the establishment of separate connections for conferencing and edge device streaming, enabling the tele-robot to maintain multiple simultaneous communication channels without resource conflict.
2Reliability
If conventional WebSocket connection is used for robot-to-edge device communication, then connection is established, but real-time behavior requirements are not met
Solution Approach 1:
The patent replaces the WebSocket connection mechanism with a WebRTC peer-to-peer connection mechanism. WebRTC is specifically designed for real-time communication with lower latency and better temporal synchronization, making it suitable for real-time video streaming and processing between the tele-robot and edge device.
3Extent of automation
If tele-robot performs computationally intensive intelligent processes locally, then embodied intelligence is achieved, but energy and computing capacity constraints are exceeded
Solution Approach 1:
The patent extracts computationally intensive processing tasks from the tele-robot and relocates them to an edge device. The tele-robot captures video feed and streams it to the edge device, which performs the heavy computational workload for intelligent processes such as object detection, tracking, and scene understanding, thereby conserving the tele-robot's limited energy and computing resources.
Solution Approach 2:
The system creates a copy of the video feed stream and directs it to the edge device for processing. Instead of the tele-robot performing all computations, a duplicate stream is sent to the edge device which then performs the intensive processing and sends back results, effectively distributing the computational burden.
4Adaptability or versatility
If multiple human users join tele-conferencing session, then multi-user collaboration is achieved, but only one user can be remote operator at a time
Solution Approach 1:
The patent implements a multi-functional tele-conferencing system where the same infrastructure supports both conferencing and edge device communication. The system allows multiple users to participate in the session simultaneously, with dynamic assignment of operator privileges. The signaling server manages user roles and permissions, enabling seamless transition between observer and operator modes without requiring separate systems.
Data Source
AI summary
The present disclosure relates to methods and systems for establishing an edge-inclusive real-time multimedia tele-conferencing using tele-robots. Conventional techniques employ a WebSocket connection for the robot-to-edge device (R2E) connection which may not meet expected real-time behaviour. The present disclosure provides a resource-friendly application protocol using WebRTC that enables the same tele-robotic WebRTC session to fork the live feed of the tele-robot for consumption by the edge device. According to the present disclosure, a peer-to-peer connection is established between the plurality of peers by the cloud signaling server, based on the type of the unique type identifier. Then, the live stream of the tele-robot, in the tele-conferencing session, is forked based on the peer-to-peer connection established (i) between the tele-robot and the broadcaster, or (ii) between the tele-robot and the edge device, to obtain a stream forked from the live stream, which is then used for various visual analytics.