Code Coverage Quality Estimator Using Neural Network Risk Factors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing code coverage measurement methodologies are inadequate as they rely on subjective percentage values, failing to ensure that the right code is thoroughly tested, especially in black box testing, and do not provide a comprehensive quality assessment.
Innovation Solution
A neural network-based system that generates risk factors for each code element by training on historical data, including suggestive data and error severity, to estimate the quality of code coverage by evaluating the likelihood and severity of errors in executed and unexecuted code elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional percentage-based code coverage measurement is used, then the testing thoroughness can be quantified, but the measurement is subjective and does not ensure that the right code is tested
Solution Approach 1:
The patent transforms the单一 parameter of code coverage percentage into multiple parameters including risk factors, error severity ratings, and quality scores. This multi-parameter approach allows for more precise and reliable measurement by considering not just the quantity of code executed but also the quality and risk associated with each code element.
Solution Approach 2:
The patent introduces neural networks as an intermediary system that processes code execution data and automatically generates risk assessments and quality metrics. This intermediary eliminates subjectivity in manual evaluation and provides consistent, data-driven measurements of testing quality.
2Reliability
If 100% code coverage is achieved, then complete code testing is accomplished, but the process is very time consuming and inefficient
Solution Approach 1:
The patent applies local quality by differentiating between high-risk and low-risk code elements. Instead of treating all code equally, the system assigns different importance weights to different code segments based on their risk factors, allowing testers to focus resources on critical areas while maintaining adequate coverage of less critical areas.
Solution Approach 2:
The patent implements partial action by determining that achieving complete coverage of all code elements is unnecessary. Instead, the system identifies the minimum necessary coverage required to achieve acceptable testing quality by focusing on high-risk areas, thereby reducing testing time and resources while maintaining reliability.
3Loss of information
If comprehensive code coverage analysis is performed, then detailed testing information is obtained, but the complexity of the testing process increases
Solution Approach 1:
The patent merges multiple testing metrics (code coverage percentage, risk factors, error severity, execution frequency) into a single integrated quality score. This consolidation maintains comprehensive information while simplifying the interpretation and presentation of results, reducing the perceived complexity for users.
4Ease of operation
If black box testing methodology is used, then the testing process is simplified, but the internal code quality assessment becomes subjective and unreliable
Solution Approach 1:
The patent implements self-service by enabling the testing system to automatically assess its own quality metrics through neural network analysis. The system autonomously generates risk factors and quality scores based on execution data, eliminating the need for manual subjective evaluation while maintaining the simplicity of black box testing methodology.
Data Source
AI summary
A method for estimating a quality of code coverage of a test is described. The method includes training a neural network, using the neural network to generate a risk factor for each code element, and determining a coverage quality based on risk factors of executed code elements and risk factors of unexecuted code elements. The neural network is trained by inputting suggestive data as input and error severity data as output. Suggestive data may be data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element. A coverage quality can be determined based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test.