Multi-Layered URL Filtering with Dynamic Blocklist Updates

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional blocklist systems are ineffective in detecting unknown malicious URLs, allowing cyber attackers to evade detection by modifying URL configurations, thereby posing a significant threat to web users and systems.

Innovation Solution

A multi-layered URL filtering system that combines a first layer of blocklist comparison with a second layer of machine learning analysis to predict malicious URLs, updating the blocklist dynamically with identified malicious URLs for broader protection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a blocklist system is used to filter URLs, then known malicious URLs can be blocked, but unknown malicious URLs cannot be detected

Engineering Contradiction:

Improvedetection accuracyVSAvoidability to detect new threats

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The filtering system is divided into two independent layers: a blocklist-based first layer for known threats and a machine learning-based second layer for unknown threats. Each layer operates independently and contributes to the overall filtering decision, allowing the system to leverage both exact matching and pattern recognition capabilities.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The machine learning model acts as an intermediary between the blocklist and the final filtering decision. It analyzes URL patterns and characteristics to predict maliciousness, bridging the gap between known blocklist entries and unknown malicious URLs by identifying subtle patterns that indicate potential threats.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If only blocklist comparison is performed, then filtering is fast, but detection capability is limited to known URLs

Engineering Contradiction:

Improvefiltering speedVSAvoidmalicious URL detection precision

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system applies partial machine learning analysis by first checking the blocklist for quick rejection of known threats, then applying more computationally intensive machine learning analysis only to URLs that pass the first layer. This selective approach maintains overall system speed while improving detection precision for unknown threats.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If machine learning analysis is applied to all URLs, then detection accuracy improves, but system complexity and processing time increase

Engineering Contradiction:

Improvemalicious URL detection reliabilityVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The blocklist check serves as a preliminary action that quickly filters out known malicious URLs before they reach the machine learning analysis stage. This preliminary filtering reduces the number of URLs requiring complex machine learning analysis, thereby reducing overall system complexity and processing time while maintaining high detection reliability.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11916875B2System and method for multi-layered rule learning in URL filtering

Publication Date: 2024.02.27 UAB 360 IT

AI summary

Systems and method for URL filtering are provided herein. In some embodiments, a system includes a processor programmed to receive a URL request to access a resource associated with the URL; perform a first layer of URL filtering by comparing the URL to a blocklist of malicious URLs; determine that the URL does not match a URL on the blocklist; perform a second layer of filtering by applying a machine learning algorithm to analyze the URL to predict whether the URL is malicious; and generate and transmit a URL filter determination that the URL is malicious and update the blocklist to include the URL.