Source Code Differential Pruning for Low-Noise Vulnerability Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating datasets of vulnerable and remediated source code face challenges such as high noise levels due to commit IDs used for both functional and non-functional issues, and the scalability limitations of supervised learning techniques for large datasets.
Innovation Solution
Implementing differential pruning using unsupervised and supervised learning to segregate relevant data from irrelevant data by analyzing commit descriptions and utilizing machine learning models to identify and classify sentences based on scores, generating an auxiliary dataset of relevant vulnerabilities and remediations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If commit IDs are used for both functional and non-functional issues in dataset generation, then data coverage is improved, but noise level increases
Solution Approach 1:
The patent segments the dataset generation process by separating functional issues from non-functional issues through differential pruning. It divides commit data into different categories and applies selective pruning to remove noise while preserving relevant vulnerability and remediation information, thus maintaining data coverage while reducing noise levels.
Solution Approach 2:
The patent extracts and removes irrelevant commit data (noise) from the dataset through differential pruning techniques. By identifying and taking out non-functional commit information while retaining functional vulnerability-related commits, it achieves noise reduction without sacrificing essential data coverage for training machine learning models.
2Measurement precision
If supervised learning techniques are used for data labeling, then labeling accuracy is improved, but scalability is limited for large datasets
Solution Approach 1:
The patent applies differential pruning as a preliminary action before supervised learning. By pre-processing the dataset to remove noise and irrelevant commits, it reduces the burden on supervised learning models, enabling them to scale more effectively to large datasets while maintaining high labeling accuracy through cleaner input data.
Solution Approach 2:
The patent introduces differential pruning as an intermediary step between raw data collection and supervised learning. This intermediary process prepares the data by removing noise, allowing supervised learning to operate more efficiently on cleaned datasets, thus improving both scalability and maintaining accuracy.
3Reliability
If differential pruning is applied to segregate relevant data, then data quality is improved, but processing complexity increases
Solution Approach 1:
The patent implements self-service through automated differential pruning that uses machine learning models to automatically identify and remove noise from commit data. The system serves itself by autonomously performing data segmentation and filtering without requiring extensive manual intervention, thus improving data quality while managing processing complexity through automation.
Data Source
AI summary
In some examples, source code differential pruning-based dataset creation may include receiving source code that includes at least one vulnerability and at least one remediation that remediates the at least one vulnerability, extracting at least one remediated section, and identifying each sentence of the remediated section. A plurality of clusters may be generated based on an analysis of each identified sentence of the remediated section to determine a score with respect to a specified cluster that includes the identified sentence. Further, a determination may be made as to whether the score is greater than a specified threshold. Each identified sentence for which the score is greater than the specified threshold may be designated as a relevant sentence. An auxiliary dataset may be generated based on a plurality of relevant sentences and include at least one relevant vulnerability and at least one relevant remediation that remediates the relevant vulnerability.