Automated Bug Labeling for Machine Learning Code Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for training machine learning algorithms to detect bugs in code are inefficient due to the time-consuming and costly process of manually labeling commits as introducing bugs or not, which limits the accuracy and scope of the training data.
Innovation Solution
A system and method that automatically classify lines of code, commits, and code files as introducing bugs or not by analyzing commit history and issue tracking systems, using data structures like line change lists and commit labels to generate labels for training a bug detection algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of commits is used to train machine learning algorithms, then the training data can be accurately labeled as introducing bugs or not, but the process is time-consuming and costly, limiting the efficiency and scale of training
Solution Approach 1:
The system uses automatic classification algorithms to label commits themselves without requiring manual developer intervention. The classification is performed by analyzing commit patterns, code changes, and historical data, allowing the system to self-label large volumes of commits efficiently while maintaining consistent labeling standards across the entire codebase.
Solution Approach 2:
The patent replaces the manual mechanical process of developer review and labeling with an automated computational system. Machine learning models and classification algorithms analyze commit data, identify bug-introducing patterns, and automatically assign labels, substituting human effort with algorithmic processing that can handle much larger volumes of data.
2Reliability
If manual code review is performed to determine likely effects in source code, then accurate bug labels can be assigned, but the process is cumbersome and limits the amount of training data available
Solution Approach 1:
The system performs classification on a subset of commits with high confidence first, then uses those results to bootstrap further classification. By focusing computational resources on clearly identifiable bug-introducing commits and using them to train initial models, the system can progressively label larger portions of the codebase without requiring exhaustive manual review of every single commit.
Solution Approach 2:
The patent applies preliminary automated filtering and classification to identify candidate bug-introducing commits before more detailed analysis. By pre-processing commits to identify those with high probability of introducing bugs based on simple heuristics (e.g., removing comments only, changing variable names), the system can prioritize these for detailed labeling while skipping obvious non-bug commits.
Data Source
AI summary
Provided are a computer program product, system, and method for classifying code as introducing a bug or not introducing a bug to train a bug detection algorithm. For each commit in a commit history of code changes to the code base, a determination is made of lines of code changed by the commit. For each line of code changed by the commit, a determination is made as to whether the commit is for a bug fix. A determination is made as to whether a previous commit changing the line of code changed by the commit for the bug fix in response to determining that the commit is for the bug fix. Indication is made that the previous commit introduced a bug. The algorithm is trained to classify changes to lines of code by commits indicated as having introduced a bug as bug introducing commits.