Web Filtering System with Local Database Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional web filtering systems face inefficiencies in categorizing URLs for access control, leading to increased communication with lookup servers and higher response times due to the need for frequent server lookups, especially when local determinations fail.
Innovation Solution
A web filtering system that employs a dual-database approach, where a lookup client performs local determinations using a partial database and caches results, and only resorts to server lookups when local categorization fails, utilizing flags to optimize matching and reduce redundant data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a complete database is stored locally for URL categorization, then local determination accuracy is improved, but memory usage increases
Solution Approach 1:
The database is segmented into two parts: a complete database stored on the server and a reduced database stored locally on the client. The reduced database contains only frequently accessed or most important URL categories, enabling fast local lookups while keeping memory usage manageable. This segmentation resolves the contradiction by allowing local determination for common cases without storing the entire database locally.
Solution Approach 2:
The local database is optimized with specific quality characteristics - it contains pre-selected, high-priority URL categories that are most relevant for local determination. This local quality approach ensures that the reduced database has the right data for common cases, improving local determination accuracy while maintaining small size. The server retains the complete database for comprehensive coverage when local lookup fails.
2Measurement precision
If server lookup is performed for every URL categorization request, then determination accuracy is improved, but response time increases
Solution Approach 1:
The system performs preliminary action by pre-populating the local reduced database with frequently accessed URL categories before the client needs them. This allows the client to immediately determine categories for common URLs without contacting the server, significantly reducing response time for typical cases while maintaining accuracy through the pre-cached data.
Solution Approach 2:
The local reduced database acts as an intermediary between the client and the server. For common URL categorization requests, the local database mediates by providing immediate answers without involving the server, thus reducing response time. Only when the local database cannot determine a category does the system escalate to the server, maintaining a balance between speed and accuracy.
3Measurement precision
If the local database is updated frequently to maintain accuracy, then determination precision is improved, but communication overhead with the server increases
Solution Approach 1:
The local reduced database is updated periodically rather than frequently or continuously. The system establishes a scheduled update mechanism that refreshes the local database at predetermined intervals, reducing communication overhead with the server while maintaining sufficient determination precision. This periodic action resolves the contradiction by balancing freshness of data with reduced communication frequency.
Data Source
AI summary
A web filtering system efficiently controls whether to permit access to a website. The web filtering system includes a first database and a first computer in the server side, and a second database and a second computer in the client side. The first database stores URLs of websites and categories corresponding to the URLs, while the second database stores periodically delivered records constituting part of all records stored in the first database. When an access request to a website occurs, the second computer first tries to determine a category corresponding to a URL of the website by using the second database. Then, when the category is not determined here, the first computer executes determination by using the first database. Thus, the web filtering system minimizes the communication amount between the server side and the client side and shortens the response time required for the control of access requests to websites.