Network Asset Hashing for High-Throughput Event Association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for asset correlation in computer networks face challenges in processing large volumes of network events due to high overhead from database queries and locking mechanisms, making it difficult to achieve the necessary throughput for real-time monitoring and action-taking systems like firewalls and IDS systems.
Innovation Solution
The use of locality-sensitive hashing (LSH) techniques, including character embedding and min-hash, to generate hashed signatures of network assets, allowing for efficient and rapid association with assets in an asset catalog, reducing the need for database queries and improving processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional database query methods are used for asset correlation, then asset association can be achieved, but processing overhead increases and throughput decreases
Solution Approach 1:
The patent pre-generates hashed signatures for all assets in the asset catalog before processing network events. This preliminary action allows the system to avoid performing database queries during real-time event processing, as the hashed signatures enable direct comparison and matching without requiring complex database lookups. The pre-computed hashed signatures are stored in an efficient data structure that allows rapid retrieval and comparison.
Solution Approach 2:
The patent creates hashed signature copies of asset identifiers (such as IP addresses, hostnames, or other asset identifying information) that can be rapidly compared without accessing the original database. These hashed copies serve as lightweight representations that preserve the ability to uniquely identify and match assets while dramatically reducing the computational overhead of database queries and locking mechanisms.
2Reliability
If real-time network monitoring is implemented, then security threats can be detected, but processing large volumes of events becomes computationally expensive
Solution Approach 1:
The patent transforms asset identifiers into hashed signature representations, changing the parameter form from raw asset information (IP addresses, hostnames) to compressed hash values. This parameter transformation maintains the uniqueness and identifiability of assets while significantly reducing the computational resources required for comparison and matching operations during real-time threat detection.
3Reliability
If database locking mechanisms are used for asset correlation, then data consistency is maintained, but processing speed decreases
Solution Approach 1:
The system performs asset signature hashing in advance, before real-time event processing begins. This preliminary computation of hashed signatures eliminates the need for database locking during event processing, as the hashed representations can be compared and matched without requiring exclusive access to the asset catalog database, thereby maintaining data consistency while enabling high-speed processing.
Data Source
AI summary
Techniques for verifying correctness of associations between assets related to events detected in at least one computer network and assets in an asset catalog for the at least one computer network. The techniques include obtaining information specifying a first asset and a first set of assets with which the first asset was previously associated; generating a signature of the first asset from the computer network addressing information for the first asset; generating a hashed signature by applying a locality sensitive hashing (LSH) technique to the signature; associating the first asset with a second set of assets in the asset catalog using the hashed signature and at least one hashed signature of the at least one asset in the asset catalog; and when it is determined that the second set of includes the first set, outputting an indication that the first asset was correctly associated with the first set of assets.