Fine-Tuned Language Models for Explainable Bug Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly large language models, are challenging to analyze and lack explainability, making it difficult to generate natural language explanations for code correlations with bug reports due to their complexity and size.
Innovation Solution
A method and system utilizing a fine-tuned language model that enhances bug localization explanations through code summarization, achieved by executing explanation and summarization similarity models to filter high-quality samples, and then training the model with these samples to improve understanding and generate better explanations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language models are used for bug localization, then prediction accuracy is improved, but explainability deteriorates due to model complexity and size
Solution Approach 1:
The patent segments the bug localization task into two distinct components: (1) a large language model that performs accurate code-change prediction, and (2) a separate explanation generation model that produces human-understandable rationales. This segmentation allows each component to be optimized independently, maintaining prediction accuracy while improving explainability through dedicated explanation generation mechanisms.
Solution Approach 2:
The patent introduces an intermediary explanation generation model that acts as a mediator between the large language model's predictions and the user's understanding needs. This intermediary translates the model's internal decision-making processes into natural language explanations, bridging the gap between accurate but opaque predictions and human-comprehensible reasoning.
2Measurement precision
If large language models are used for bug localization, then prediction accuracy is improved, but natural language explanation generation becomes challenging
Solution Approach 1:
The patent divides the system into a prediction component and an explanation generation component. The explanation generation model is specifically designed and trained to produce natural language explanations, making the explanation generation process easier and more effective while the prediction component maintains high accuracy through the large language model.
Solution Approach 2:
The explanation generation model serves as an intermediary that converts the technical predictions into user-friendly natural language. This intermediary handles the complexity of translation from model internals to human language, making the overall system easier to operate and understand without compromising prediction accuracy.
3Reliability
If pre-trained language models are fine-tuned for bug localization, then task-specific performance is improved, but training data requirements increase
Solution Approach 1:
The patent merges the training processes of two models: the pre-trained language model is fine-tuned on bug localization datasets while the explanation generation model is trained on the same data. This combined approach allows both models to benefit from the available training data, improving task-specific performance without requiring separate large-scale datasets for each component.
Solution Approach 2:
The patent makes the training data multi-functional by using the same bug localization datasets to train both the prediction model and the explanation generation model. This universal use of training data maximizes the value extracted from limited datasets, improving task-specific performance without proportionally increasing data requirements.
Data Source
AI summary
A method implements the use of large language models to generate bug localization explanations enhanced by code summarization. The method includes executing an explanation similarity model using a training report and a training explanation to generate an explanation score for an explanation sample. The method further includes filtering multiple explanation samples using multiple explanation scores to generate a set of filtered explanation samples. The method further includes executing a summarization similarity model using source code and a description to generate a summarization score for a summarization sample including the source code and the description. The method further includes filtering multiple summarization samples using multiple summarization scores to generate a set of filtered summarization samples. The method further includes training a language model using the set of filtered explanation samples and the set of filtered summarization samples to generate a fine-tuned model.