Video Data Storage System with Dynamic Hot Value Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video cloud storage systems face inefficiencies in resource utilization and retrieval due to high redundancy in video data storage and increased pressure on index servers, leading to increased costs and reduced retrieval efficiency.
Innovation Solution
A video data storage system with an index server and multiple storage servers that dynamically manage video data backup based on operation hot values, using a distributed indexing structure to selectively back up and store video data, optimizing storage and retrieval processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all video data is backed up using specified redundant copies, then data security is improved, but storage space and bandwidth resources are wasted
Solution Approach 1:
The system changes the parameter of backup strategy from fixed redundant copies to dynamic hot value-based backup. By calculating operation hot values based on actual access patterns, the system determines backup levels dynamically, ensuring data security for important videos while avoiding unnecessary bandwidth consumption for less important videos.
Solution Approach 2:
The system enables video data to 'self-determine' its backup level through the hot value calculation mechanism. Videos automatically receive appropriate backup treatment based on their actual operational importance, eliminating the need for manual specification of redundant copies and optimizing resource allocation.
2Ease of operation
If the index server records all video data index information, then data retrieval is enabled, but the index server becomes overloaded and retrieval efficiency decreases
Solution Approach 1:
The system segments the indexing function by introducing a separate indexing module that operates independently from the index server. This module pre-processes video data and generates index information, which is then stored in the distributed storage system. The index server only needs to query this pre-generated index information, significantly reducing its computational burden and improving retrieval efficiency.
Solution Approach 2:
The indexing module performs preliminary indexing operations before data retrieval requests are made. By pre-generating and storing index information in the distributed storage system, the system prepares retrieval data in advance, eliminating the need for the index server to process all video data during retrieval operations.
3Reliability
If redundant copies of all video data are stored, then data security is improved, but resource utilization efficiency deteriorates
Solution Approach 1:
The system changes the backup parameter from fixed redundant copies to dynamic hot value-based replication. Videos with higher operation hot values receive more backup copies, while videos with lower hot values receive fewer or no backups. This dynamic parameter adjustment optimizes both data security and resource utilization efficiency simultaneously.
Solution Approach 2:
Different backup levels are applied to different video data based on their local characteristics (operation hot values). Important videos receive enhanced protection with multiple backups, while less important videos use minimal or no backup, creating a differentiated quality of redundancy that optimizes overall system resource utilization.
Data Source
AI summary
The present disclosure provides a video data storage system, an operation method thereof, and a retrieval server. The video data storage system according to present disclosure includes an index server and at least two storage servers for storing video data. The index server stores a first mapping relationship between a camera identification code and a storage server identification code. Each of the storage server stores a second mapping relationship among a camera identification code, a time range of video data and a physical location of video data in the storage server. The operation method includes: making, by the index server, statistics of an operation hot value of video data operated by all clients, and determining the operation hot value of the video data is greater than a predetermined first threshold; if yes, sending, by the index server, a backup instruction to at least one storage server, to instruct the storage server to back up the video data and update the second mapping relationship. The present disclosure can protect the hottest data using a minimum amount of capacity and bandwidth.