Parallel DNS Record Detection Using Shared Bloom Filter
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face computational intensity in efficiently detecting newly observed domains, which are potentially malicious, as they require tracking a large volume of newly registered domain names in a short period.
Innovation Solution
A computer-implemented method using multiple processors to apply a common filter to a stream of DNS data points, identifying new records by generating keys and using a Bloom filter to determine if records are in a set, allowing for parallel processing and scalability to quickly detect new domain name records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a single processor tracks newly observed domains using a Bloom filter, then detection accuracy is maintained, but processing speed decreases and the system becomes a bottleneck
Solution Approach 1:
The patent divides the single processor into multiple processors (first processor, second processor, etc.), each capable of independently applying the Bloom filter to DNS data points. This segmentation allows parallel processing of domain name lookups, significantly increasing detection speed while maintaining the same detection logic in each processor instance
Solution Approach 2:
Multiple processors are merged into a coordinated system that shares a common Bloom filter data structure. The processors work in parallel but coordinate through the shared filter, combining their individual processing capabilities into a unified high-performance detection system that maintains data consistency
2Productivity
If multiple processors are used to process DNS data in parallel, then processing speed increases, but race conditions and deadlocks may occur when updating the Bloom filter
Solution Approach 1:
A message queue is introduced as an intermediary between the multiple processors and the Bloom filter update mechanism. Processors send detection results and filter update requests through the message queue, which serializes access to the Bloom filter. This allows parallel processing to continue while preventing race conditions and deadlocks during filter updates
Solution Approach 2:
The system maintains continuous parallel processing by allowing processors to independently evaluate DNS data points against the Bloom filter without blocking each other. The message queue ensures continuous updates to the Bloom filter without interrupting the parallel detection workflow, maintaining system productivity while ensuring data consistency
Data Source
AI summary
In an embodiment, a computer-implemented method detects updates to a domain name system record system. In the method, a stream of data points is received with each data point describing a record of a domain name system. For respective data points, a processor is selected from a plurality of processors. The processors are each configured to apply a data point against a common filter that assesses whether the data point is in a set. At the selected processor, the respective data point is applied to the common filter to determine whether the record is included in the set. When the record is determined not to be included in the set, a message is provided to indicate that the domain name system includes a new record, and the common filter is updated to include the data point in the set.