Binary Code Test Impact Analysis for Regression Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software development, large regression test suites for web UI and REST API applications take a long time to execute, making it difficult for QA or IT integrators to obtain quick feedback on changes, especially when they lack access to the source code and can only work with binary files.
Innovation Solution
A method and system that collect test coverage data from executed tests, calculate a baseline report correlating tests with binary code blocks, determine binary code changes, and identify a subset of tests to execute for verifying changes, using a coverage collector and test impact analysis engine to analyze binary code changes without access to source code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire test suite is executed to verify changes, then reliability of verification is improved, but time consumption increases
Solution Approach 1:
The patent segments the entire test suite into smaller subsets based on binary code change analysis. By dividing tests into groups associated with specific binary code blocks, only the relevant subsets needing verification are executed, rather than running the complete test suite. This segmentation maintains verification reliability for changed areas while reducing overall execution time.
Solution Approach 2:
The patent applies partial action by executing only the necessary portion of the test suite rather than the entire suite. By identifying and running only those tests associated with modified binary code blocks, the system performs sufficient verification for the changed areas without the excessive time cost of executing all tests, achieving optimal balance between reliability and efficiency.
2Measurement precision
If source code access is required for impact analysis, then test selection accuracy is improved, but ease of operation deteriorates
Solution Approach 1:
The patent creates a copy or representation of source code information in the form of binary code blocks that can be analyzed without accessing the actual source code. By working with binary representations and metadata that capture the essential structure and change information, the system achieves test selection accuracy comparable to source code analysis while eliminating the need for source code access, thus improving ease of operation.
Data Source
AI summary
System and method for testing changes to binary code of a computer program include: collecting test coverage data from an executed set of tests of an original computer program; calculating a baseline report containing correlations between the executed set of tests and blocks of binary code of the original computer program; determining binary code changes between the original computer program and a modified version of the computer program; identifying one or more tests to be executed for verifying the binary code changes.