Security Event Clustering Using Vector Embeddings Across Data Types
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustering techniques for data analytics are resource-intensive and specific to particular data types, limiting their effectiveness in identifying relationships between data items.
Innovation Solution
A computer-implemented method using neural network-generated vector embeddings to evaluate semantic distances between data records, identifying similar records based on concentration of distances, and clustering computer systems with common security occurrences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing clustering techniques are used to identify relationships between data items, then clustering effectiveness for specific data types is achieved, but resource consumption increases and adaptability to diverse data types decreases
Solution Approach 1:
The patent transforms data records into vector embeddings, changing the parameter representation from raw data to dense vector space. This transformation enables efficient similarity computation through vector operations while maintaining clustering effectiveness across diverse data types including text, media, and structured data.
Solution Approach 2:
The patent creates a universal clustering approach that handles multiple data types (text, media, structured data) through a single vector embedding framework. The system generates vector embeddings for different data types and applies the same similarity measurement and clustering algorithms, eliminating the need for type-specific processing pipelines.
2Reliability
If existing clustering techniques are used to identify relationships between data items, then clustering effectiveness for specific data types is achieved, but the method's versatility across different data types decreases
Solution Approach 1:
The patent creates a universal clustering approach that handles multiple data types (text, media, structured data) through a single vector embedding framework. The system generates vector embeddings for different data types and applies the same similarity measurement and clustering algorithms, eliminating the need for type-specific processing pipelines.
Solution Approach 2:
The patent introduces vector embeddings as an intermediary representation between raw data of various types and the clustering algorithm. This intermediary transformation enables diverse data types to be processed uniformly through vector space operations, improving both versatility and computational efficiency.
3Measurement precision
If traditional clustering methods are applied to security event data, then clustering accuracy for specific event types is achieved, but processing time increases
Solution Approach 1:
The patent transforms security event records into vector embeddings, changing the parameter representation from raw structured data to dense vector space. This transformation enables efficient similarity computation through vector operations while maintaining clustering effectiveness across diverse data types.
Solution Approach 2:
The patent replaces traditional distance computation mechanics with vector embedding-based similarity measures. Instead of computing distances in high-dimensional feature spaces, the system uses dot products and cosine similarities in dense vector spaces, significantly reducing computational complexity and processing time.
Data Source
Figure 1~2
Figure 3~4
AI summary
A computer implemented method of clustering computer systems in a plurality of systems to identify computer systems being subject to a common security occurrence, each computer system generating data records corresponding to security events in respect the systems, the method comprising: receiving a set of one or more data records associated with each computer system, each record including a sequence of data fields; generate a vector embedding for each data field in each record; evaluate a distance of each vector embedding from a reference vector as an indicator of semantic distance; identifying similar data records based on a measure of a degree of similarity of the distances of vector embeddings for each record; defining a cluster of computer systems including computer systems associated with the similar data records for applying protective measures to the computer systems in the cluster.