Live Video Text Search Through WebRTC Media Server Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently and securely streaming live video from devices like UAVs over public and private networks for real-time computer vision tasks, with limitations in hardware, memory, processing, and network security, leading to inaccurate object detection and classification.
Innovation Solution
A system that decodes live video into frames, sends them to image classification services for metadata generation, and stores this metadata in a search engine, allowing authenticated users to search and annotate video content, with secure protocols like WebRTC and SRTP to minimize latency and firewall issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video content is streamed over public or private networks from devices like UAVs, then remote access and sharing of video content is enabled, but network security restrictions and firewall issues increase latency and reduce reliability
Solution Approach 1:
The patent introduces a media server as an intermediary component that receives video content from capture devices, processes it through parallel pipelines including decoding and encoding, and redistributes it to client devices. This mediator architecture enables secure network transmission by handling protocol conversions and firewall traversals centrally, improving reliability while maintaining remote accessibility.
2Measurement precision
If real-time video processing and analysis is performed, then object detection and classification accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The patent divides video processing into separate parallel pipelines: one for decoding video content, another for encoding/transmitting, and a third for analysis by computer vision services. This segmentation allows object detection and classification to occur concurrently with video transmission rather than sequentially, reducing processing latency while maintaining detection accuracy through dedicated analysis resources.
3Measurement precision
If trained neural networks are deployed on server systems for image classification, then classification accuracy improves, but system complexity and resource requirements increase
Solution Approach 1:
The media server is designed as a multi-functional system that handles video capture, decoding, encoding, transmission, and coordination with computer vision services all through a single unified architecture. This universal server reduces overall system complexity by consolidating multiple functions into one platform rather than requiring separate specialized systems for each function.
4Reliability
If video content is securely transmitted over networks with authentication protocols, then security and access control improve, but communication overhead and latency increase
Solution Approach 1:
The system performs authentication and authorization checks in advance before video content transmission begins. Client devices and media servers establish secure connections and verify permissions beforehand, allowing subsequent video data transmission to proceed with minimal additional security overhead, thus reducing communication latency while maintaining strong security controls.
Data Source
AI summary
A server receives and rebroadcasts live streaming video content from a video capture device, such as a mobile phone or unmanned surveillance vehicle. The server includes a media server configured to stream selected video content to a client device, a video analysis system configured to analyze the live video content and generate object detection data, a storage system configured to store the generated object detection data and an identifier of the associated live video content, and a search engine configured to receive a text-based search request, search the object detection data stored in the storage system for relevant search results, and generate a list of live and stored video content associated with the relevant search results.