URL Policy Enforcement Using Cache and Bloom Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Firewalls and security devices face challenges in efficiently enforcing policies based on URL information due to the vast number of URLs, making it difficult to match rules effectively.
Innovation Solution
A policy enforcement appliance uses a combination of caching, bloom filters, and database queries to efficiently evaluate URLs by transforming them into hashes, using a multi-stage process involving caches, bloom filters, and databases to determine access permissions based on URL categories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database querying is used for URL matching, then accuracy is maintained, but processing speed deteriorates due to the sheer volume of URLs
Solution Approach 1:
The patent segments the URL matching process into three distinct stages: (1) cache lookup for frequently accessed URLs, (2) bloom filter checking for probabilistic filtering, and (3) database querying for definitive verification. This segmentation allows each stage to handle specific types of URL queries efficiently, improving overall speed while maintaining accuracy through the progressive refinement of the matching process.
Solution Approach 2:
The patent introduces intermediate data structures (cache and bloom filter) between the URL and the final database query. The cache acts as a fast intermediary for known URLs, while the bloom filter serves as a probabilistic intermediary that can quickly eliminate non-matching URLs before they reach the database, thus speeding up processing without sacrificing final accuracy.
2Reliability
If comprehensive URL databases are maintained for accurate policy enforcement, then reliability is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing hash values of URLs in both the cache and bloom filter before they are needed for matching. This preliminary preparation allows the system to quickly check URLs during policy enforcement without performing complex computations in real-time, thus maintaining reliability while reducing processing complexity during operation.
Solution Approach 2:
The patent creates simplified copies of URL data in the form of hash values stored in the cache and bloom filter. These copies contain essential matching information without the full complexity of the original URL database, allowing fast probabilistic matching while the complete database remains available for final verification when needed.
3Measurement precision
If all URLs are checked against the database, then accuracy is maintained, but processing time increases significantly
Solution Approach 1:
The patent applies partial action by checking URLs against the cache and bloom filter before performing the complete database query. This partial checking process quickly eliminates many non-matching URLs without requiring a full database search, thus maintaining matching precision for relevant URLs while significantly reducing the average lookup time across all URLs.
Solution Approach 2:
The system performs preliminary filtering using the cache and bloom filter to identify URLs that likely require full database verification. This preliminary action prevents unnecessary database queries for URLs that can be quickly resolved or eliminated by the intermediate structures, thereby maintaining precision for critical matches while reducing overall processing time.
Data Source
AI summary
Enforcing a policy based at least in part on URL information is disclosed. A uniform resource locator (URL) is received. A portion of the URL, or a transformation thereof, is matched against a bloom filter. Based on a result of the match, a first query is performed. A policy is enforced based at least in part on a category received as a result of a second query. In some cases, the first and second query are the same.