Distributed Hash Table Forensic Search via Linear Orbits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network administrators face challenges in performing forensic investigations on isolated machines within a network due to communication barriers, and traditional file search methods in distributed systems are inefficient, relying on centralized databases that burden servers and network bandwidth.
Innovation Solution
A method is introduced for sending similarity search queries through linear communication orbits in a distributed system, allowing machines to identify and report files that meet predefined similarity criteria with respect to a target document, and establishing direct duplex connections for secure and efficient data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a centralized database is used to store an index of all files throughout the distributed system, then file search capability is improved, but server burden and network bandwidth consumption increase significantly
Solution Approach 1:
The patent divides the centralized index storage into distributed hash tables (DHTs) at each node in the overlay network. Each node maintains a local DHT that stores file metadata and routing information, eliminating the need for a single centralized database. This segmentation distributes the storage and processing burden across all nodes, reducing server burden while maintaining file search capability through distributed query resolution.
2Ease of operation
If remote servers communicate directly with machines within the network for forensic investigations, then investigation capability is improved, but network security is compromised due to firewall restrictions
Solution Approach 1:
The patent introduces an overlay network as an intermediary layer between remote servers and machines within the private network. This overlay network operates above the physical network infrastructure and uses virtual channels for communication. Remote servers can initiate forensic investigations by sending queries through the overlay network, which routes messages through trusted nodes without requiring direct firewall penetration, thus maintaining both investigation capability and network security.
3Adaptability or versatility
If traditional centralized file search methods are used in distributed systems, then search functionality is provided, but network bandwidth is consumed for crawling and re-crawling the entire network
Solution Approach 1:
The patent implements preliminary indexing where file metadata is pre-processed and stored in the distributed hash tables at each node. When a search query is initiated, the system queries the DHT structure which already contains organized file information, rather than crawling the entire network. This preliminary organization of data enables efficient search operations with minimal network bandwidth consumption, as queries are resolved through hash-based lookup rather than exhaustive scanning.
Data Source
AI summary
In a distributed system, each of N machines receives a similarity search query through a linear communication orbit. The similarity search query includes token identifiers corresponding to tokens in a target document. Each machine, in response, identifies files that meet predefined similarity criteria with respect to the target document. Subsequent to receiving the similarity search query, the machine generates a first report, including a count of files stored at the machine that meet the predefined similarity criteria with respect to the target document, and/or information identifying a set of files that meet the predefined similarity criteria with respect to the target document; and sends the first report to a server through the linear communication orbit. The server produces a merged report presenting information with respect to files at a set of machines, including the N machines, that meet the predefined similarity criteria with respect to the target document.