Network Asset Hashing for Real-Time Event Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for asset correlation in computer networks face challenges in processing high volumes of network events due to the overhead of database queries and locking mechanisms, making it difficult to achieve the necessary throughput for real-time monitoring and action in systems like firewalls and IDS.
Innovation Solution
The use of locality-sensitive hashing (LSH) techniques, including character embedding and min-hash, to generate hashed signatures of network assets, enabling efficient association with assets in an asset catalog without the need for database queries, allowing for rapid and accurate identification of implicated assets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional database query methods are used for asset correlation, then asset identification accuracy is maintained, but processing speed and throughput deteriorate due to query overhead and locking mechanisms
Solution Approach 1:
The system performs preliminary actions by generating hashed signatures of all assets in the network and storing them in a hash table before monitoring events. When an event occurs, the system immediately hashes the involved asset's signature and performs a direct lookup in the pre-built hash table, eliminating the need for real-time database queries and locking operations. This preliminary preparation enables O(1) constant-time asset identification.
Solution Approach 2:
The invention creates a simplified copy of the asset catalog in the form of a hash table containing only hashed signatures rather than complete asset records. This copy structure enables rapid comparison and matching operations without the overhead of querying the full database, while still providing sufficient information for accurate asset identification through hash matching.
2Productivity
If real-time monitoring of high-volume network traffic is implemented, then detection capability is improved, but system resource consumption and processing complexity worsen
Solution Approach 1:
The system extracts only the essential identifying feature (hashed signature) from each asset for storage and comparison purposes, rather than maintaining complete asset metadata in memory during monitoring operations. This extraction reduces the data volume that must be processed in real-time while preserving the ability to accurately identify assets involved in events.
Solution Approach 2:
The invention transforms asset identification from a complex multi-parameter database query operation into a simple hash value comparison operation. By changing the parameter representation from structured asset records to hashed signatures, the system reduces processing complexity from O(N) database queries to O(1) hash table lookups, enabling high-throughput real-time monitoring.
3Loss of time
If complete asset metadata is stored and queried for each event, then asset association accuracy is maintained, but processing time increases due to extensive database operations
Solution Approach 1:
The system replaces the mechanical database query system with a hash-based lookup system. Instead of performing sequential or indexed database queries that require multiple I/O operations and locking mechanisms, the system uses hash function computation followed by direct memory access to retrieve asset associations, dramatically reducing processing time while maintaining reliability through the deterministic nature of hash matching.
Data Source
AI summary
Techniques for associating assets related to events detected in at least one computer network with respective assets in an asset catalog for the at least one computer network. The techniques include: while monitoring activity on the at least one computer network, obtaining information about an event related to a first asset, the information specifying computer network addressing information for the first asset; generating a signature of the first asset from the computer network addressing information; generating a hashed signature of the first asset by applying a locality sensitive hashing (LSH) technique to the signature; associating the first asset with at least one asset in the asset catalog using the hashed signature of the first asset and at least one hashed signature of the at least one asset in the asset catalog; and outputting information identifying the at least one asset with which the first asset was associated.