URL Index Database Storage Optimization via Schema Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current P2SP multi-source download systems face inefficiencies due to large storage requirements and reduced query efficiency caused by numerous URLs corresponding to the same hash value, leading to disk space issues and decreased resource query efficiency.
Innovation Solution
The solution involves analyzing URLs for the same file under the same domain name, separating variable and constant parts, replacing variable parts with universal characters, and storing mapping relationships between file hash values and URL schema hash values in the URL index database, thereby reducing the number of records and improving storage and query efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all URLs corresponding to the same hash value are stored in the URL index database, then the database contains complete download resource information, but the database size increases and query efficiency decreases
Solution Approach 1:
The patent merges multiple URLs that correspond to the same hash value into a single index record. Instead of storing each URL separately, the system combines them into one record that represents all download sources for a given file, thereby reducing database size while maintaining access to complete resource information.
Solution Approach 2:
The patent creates a universal index structure where a single URL index record serves multiple URLs with the same hash value. This multi-functional index record can represent and provide access to multiple download sources simultaneously, improving query efficiency without sacrificing completeness.
2Reliability
If all URLs corresponding to the same hash value are stored in the URL index database, then the database contains complete download resource information, but the database storage space increases
Solution Approach 1:
The patent merges multiple URLs that correspond to the same hash value into a single index record. Instead of storing each URL separately, the system combines them into one record that represents all download sources for a given file, thereby reducing database size while maintaining access to complete resource information.
Solution Approach 2:
The patent changes the parameter representation by storing a consolidated view of URLs rather than individual URL instances. The index record stores aggregated information about multiple URLs (such as the hash value and a representative URL or URL list), reducing the quantity of stored data while preserving essential download resource information.
Data Source
AI summary
The present invention discloses a data storage method, a data downloading method, and a related apparatus. The data storage method includes: receiving a storage request containing a uniform resource locator (URL) for a storage operation; acquiring the URL in the storage request; determining whether a URL schema corresponding to the URL is in a storage, the URL schema corresponding to different URLs for a same file under a same domain name; when the URL schema is in the storage, completing the storage operation without storing the URL; when the URL schema is not in the storage, determining a hash value of a file corresponding to the URL and a hash value of the URL schema, and storing mapping relationship information between the file hash value and the URL schema hash value in the URL index database. The storage resource usage of the URL index database is reduced.