Video Analytics System with Edge Caching for Network Outages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in efficiently processing and analyzing large volumes of video data stored in data lakes, particularly in providing real-time and interactive insights while ensuring network connectivity and optimal cost-benefit for clients and service providers.
Innovation Solution
A method and system for performing video analytics by collecting data from multiple sources following a predefined streaming protocol, caching content during network downtime, and segregating data into real-time, micro-batch, or batch sources for processing, including real-time and batch analytics, and displaying insights in a human-readable format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If video data is stored in a data lake with flat architecture, then storage capacity and data retention are improved, but system complexity and difficulty of data management increase
Solution Approach 1:
The patent segments the flat data lake architecture into hierarchical layers (hot, warm, cold storage zones) with different retention policies and access patterns. This segmentation maintains the large storage capacity of the data lake while reducing complexity through structured organization and automated data lifecycle management.
Solution Approach 2:
The patent introduces data lakeshore and data lakeview as intermediary layers between the raw data lake and analytics workloads. These intermediaries pre-process, catalog, and manage data quality, reducing the complexity of querying and analyzing raw data while preserving the full storage capacity of the underlying data lake.
2Speed
If real-time analytics are performed on video data, then insight speed and decision-making capability are improved, but processing resource consumption and cost increase
Solution Approach 1:
The patent applies local quality by performing different types of analytics at different levels: real-time analytics on recent hot data, near-real-time analytics on warm data, and batch analytics on cold data. This approach provides timely insights where needed while conserving processing resources on historical data where full real-time performance is not critical.
Solution Approach 2:
The patent implements partial real-time analytics by processing only the most recent and relevant video data streams in real-time, while using batch processing for historical data. This partial action approach delivers sufficient insight speed for critical decisions without the excessive resource consumption of processing entire data lakes in real-time.
3Loss of information
If network connectivity is maintained for continuous data streaming, then data freshness and analytics accuracy are improved, but system reliability during network outages deteriorates
Solution Approach 1:
The patent implements preliminary action by having data collectors buffer and queue video data locally before transmission to the data lake. This preliminary buffering ensures that data continues to be captured and stored even during network outages, maintaining data freshness upon network recovery while ensuring system reliability during connectivity issues.
Solution Approach 2:
The patent provides beforehand cushioning by implementing redundant data collection paths and local caching mechanisms at edge devices. These cushioning measures ensure that data collection and initial processing continue during network outages, protecting against information loss while maintaining system operation without continuous network connectivity.
Data Source
AI summary
A method, system and computer program product for performing video analytics on content, by collecting content (video data) from a plurality of sources, wherein the content pursues a streaming protocol; and performing at least one of storing the content in a local repository for downtime recording wherein on negative determination of a network connection, wherein servers directly coupled to a plurality of sources cache content until the network connection retains normalcy or performing analytics at least one of a real-time insight or an interactive insight or a batch insights on the content, and displaying to the user a resulting insight wherein the resulting insights are in a human readable form.