Deep Learning Pipeline for Duplicate Crash Report Clustering
Solution Overview
Problem
Existing duplicate crash report detection methods are inefficient due to high computational resource consumption and limited throughput when dealing with large-scale crash bucketing tasks, particularly when stack traces have little overlap in functions, making it difficult to identify and group duplicate reports effectively.
Innovation Solution
A deep learning pipeline is employed to process stack traces, converting them into feature vectors using a frame2vec model and a deep metric model trained with circle loss and large-margin softmax, enabling efficient clustering of crash reports into groups representing common root causes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional similarity measurement methods (trace alignment, edit distance) are used to detect duplicate crash reports, then detection accuracy is maintained, but computational resource consumption increases and throughput decreases
Solution Approach 1:
The patent transforms the crash report data from traditional text representation into vector embeddings using deep learning models (stack trace embedding model and crash report embedding model). This parameter transformation enables the use of efficient vector similarity computations (cosine similarity) instead of computationally intensive text processing methods (trace alignment, edit distance), thereby improving throughput while maintaining detection accuracy
Solution Approach 2:
The patent replaces the mechanical text processing system (character-by-character or token-by-token comparison, alignment algorithms) with a neural network-based embedding system. The deep learning models automatically learn meaningful representations of stack traces and crash reports, substituting the need for complex mechanical comparison algorithms and achieving both high throughput and accuracy
2Measurement precision
If traditional similarity measurement methods are used, then detection accuracy is maintained, but consumption of technical resources (processors, memory) increases
Solution Approach 1:
The patent changes the representation parameters of crash reports from detailed text structures to compact vector embeddings. This parameter compression reduces the computational complexity of similarity calculations from O(n*m) for text comparison to O(d) for vector comparison (where d is the embedding dimension), significantly reducing processor and memory consumption while preserving detection accuracy through the semantic information captured in the embeddings
3Productivity
If stack trace alignment is performed to speed up similarity measurement, then throughput is improved, but resource consumption increases and alignment itself consumes time
Solution Approach 1:
The patent performs preliminary action by pre-computing vector embeddings for stack traces using the stack trace embedding model before the actual duplicate detection process. These pre-computed embeddings are stored and reused for multiple comparisons, eliminating the need for repeated alignment operations and reducing both alignment processing time and resource consumption during the detection phase
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving a set of crash reports, determining a set of trace vectors by processing a set of stack traces through a first DL model, each trace vector in the set of trace vectors being a multi-dimensional vector representation of a stack trace of a respective crash report provided from the set of stack traces, generating a set of feature vectors by processing the set of trace vectors through a second DL model, each feature vector being a multi-dimensional vector representation of a stack trace of a respective crash report, and clustering each crash report in the set of crash reports into a group of a set of groups based on comparing feature vectors of respective crash reports, each group representative of a root cause resulting in respective crashes of the software system represented in one or more crash reports.



