ML Post-Processing for Static Analysis Bug Severity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing static analysis tools are unsound and imprecise, leading to false negatives and false positives in code quality assessments, limiting their effectiveness in determining the actual quality of software projects.
Innovation Solution
Implementing a machine learning-based post-processing system that accesses bug datasets from static analysis tools, selects relevant features, and uses trained ML classifiers to estimate the severity of reported issues, thereby improving the accuracy of code quality assessments by distinguishing true positives from false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If static analysis tools are used to assess code quality, then code quality assessment can be performed automatically, but the assessment results contain false positives and false negatives reducing reliability
Solution Approach 1:
The patent introduces machine learning models as intermediary components between static analysis tools and code quality assessment results. These ML models process the raw bug reports from static analysis tools and filter out false positives and negatives, thereby improving the reliability of the final assessment without eliminating automated analysis
Solution Approach 2:
The system implements feedback mechanisms where ML models learn from the results of static analysis tools and continuously improve their filtering capability. The system uses historical data and patterns from multiple static analysis runs to refine its understanding of true positives versus false positives, creating a self-improving assessment pipeline
2Reliability
If multiple static analysis tools are used to improve assessment coverage, then more bugs can be detected, but the complexity of processing and analyzing results increases
Solution Approach 1:
The patent merges the results from multiple static analysis tools into a unified assessment framework. Instead of processing each tool's results separately, the system combines them and applies a single ML-based filtering layer that handles false positives and negatives across all tools, reducing overall system complexity while maintaining comprehensive coverage
3Reliability
If formal verification is used to ensure high code quality, then code quality can be guaranteed, but the cost of specification and verification becomes prohibitively high
Solution Approach 1:
The patent employs inexpensive machine learning models that can be rapidly trained and deployed as alternatives to expensive formal verification methods. These ML models provide sufficient quality assurance for practical purposes at a fraction of the cost of formal verification, making high-reliability assessment accessible for regular software development cycles
Data Source
AI summary
A preferably cloud-based platform supports improvements in software development by assessing quality of source code files, for example, when files are pushed to a software repository. Various static analysis tools are executed on a source file, and the resulting bug reports, which reflect native features of the tools are assembled in a dataset. The bug dataset is enhanced by adding additional features that are not natively generated by the static analysis tool. An ML classifier is trained to predict a selected bug feature, and the classifier is used to update the bug dataset to include estimated values of the selected feature. In an embodiment, post-processing analysis of bug report datasets applies machine learning methods to predict the “severity” of bug reports, an indication of whether they are likely to be true or false. Further, a report of code quality can be returned based on the severity predictions.