Edge DPU Video Data Chunking and Metadata Staging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack an efficient method for processing and storing video data, particularly in managing retention and staging metadata to optimize storage types and retention periods, leading to inefficiencies in data retrieval and compliance with storage requirements.
Innovation Solution
A data processing unit (DPU) at an edge device processes video data into chunks, generates retention and staging metadata, and associates it with the video data chunks, storing them in appropriate storage tiers based on the metadata, allowing for efficient storage and retrieval while ensuring compliance with retention policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video data is stored without processing into chunks and metadata generation, then storage operations are simpler, but data retrieval efficiency and compliance management deteriorate
Solution Approach 1:
The patent divides video data into discrete chunks and generates metadata for each chunk, including retention metadata specifying retention periods and staging metadata specifying storage tier information. This segmentation enables efficient retrieval and compliance management while maintaining systematic organization of the data.
2Adaptability or versatility
If all video data is stored in the same storage type, then storage management is simpler, but storage optimization and compliance with retention policies deteriorate
Solution Approach 1:
The patent assigns different storage tier metadata to different video data chunks based on their specific retention requirements and access patterns. Hot data is directed to fast storage tiers while cold data is directed to slower, more cost-effective tiers, optimizing storage resource utilization while maintaining compliance with retention policies.
3Reliability
If video data retention periods are not tracked, then data management is simpler, but compliance with retention policies deteriorates
Solution Approach 1:
The patent generates retention metadata for each video data chunk at the time of ingestion, specifying the retention period and storage tier requirements. This preliminary action ensures compliance is built into the storage system from the outset, enabling automatic compliance verification and reducing the complexity of ongoing compliance management.
Data Source
AI summary
A method for processing video data is performed by a data processing unit (DPU). The method includes obtaining, by the DPU of an edge device, video data; processing the video data to obtain video data chunks and indexing attributes; generating retention and staging metadata based on the video data chunks and the video processing engine outcomes, and the retention and staging metadata specifies retention and staging information associated with the video data chunks; associating the retention and staging metadata with the video data chunks; and storing the retention and staging metadata and the video data chunks in appropriate storages based on the retention and staging metadata.