Semantic String Distance for Computer Alert Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustering algorithms for computer system alerts, such as machine learning algorithms, fail to accurately differentiate between semantically similar and dissimilar strings, leading to inefficient grouping and slow down security analysis, especially when pseudorandom substrings or timestamps are present, which impedes the identification of meaningful patterns and hampers response to potential security threats.
Innovation Solution
A tailored string distance calculation algorithm that considers semantically-similar parts in strings by using different weights for length, entropy, and specific matching characteristics, providing more accurate distances to clustering algorithms for effective cluster generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional distance functions (e.g., Levenshtein distance) are used for string clustering, then the clustering process can be performed, but the accuracy of distinguishing semantically similar strings is poor
Solution Approach 1:
The patent transforms the string comparison problem from character-level operations to semantic-unit-level operations. By segmenting strings into semantic units (words, phrases, or meaningful tokens) and comparing these units rather than individual characters, the system achieves better differentiation between semantically similar strings. This parameter change from character distance to semantic unit distance resolves the contradiction by improving both measurement precision and clustering reliability.
2Productivity
If machine learning clustering algorithms are used on security alerts, then clustering can be performed, but the system speed is slowed down due to inability to effectively reduce repetitive alerts
Solution Approach 1:
The patent segments the alert processing workflow into two distinct stages: (1) a fast pre-filtering stage using the tailored string distance function to quickly identify and group obviously similar alerts, and (2) a secondary analysis stage for remaining alerts. This segmentation enables early elimination of repetitive alerts through efficient string matching, thereby improving overall processing speed and reducing analysis time without sacrificing clustering quality.
3Adaptability or versatility
If traditional string distance functions are used, then the clustering algorithm can process all strings uniformly, but it fails to account for semantically-similar parts in strings
Solution Approach 1:
The patent applies local quality by treating different parts of strings differently during comparison. Instead of uniform character-level weighting, the system identifies and weights semantic units (such as key terms, file names, command parameters) based on their local importance. This allows the clustering algorithm to be more adaptable to semantic variations while maintaining precision in detecting meaningful similarities, resolving the contradiction between flexibility and precision.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and software can be used to clustering strings. In some aspects, a method includes: obtaining a pair of string elements, wherein the pair of string elements comprises a first string element and a second string element, the first string element comprises one or more first strings, the second string element comprises one or more second strings; separating the one or more first strings into a set of first string terms and the one or more second strings into a set of second string terms; generating one or more string term distances based on the set of first string terms and the set of second string terms; generating a distance between the first string element and the second string element based on the one or more string term distances; and providing the distance to a clustering algorithm to generate a plurality of clusters.