Predictive Build Quality Assessment Using Source Code Graphs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional approaches to measuring enterprise software build quality are inadequate for predicting future performance behavior in production environments, as they rely on insufficient testing methods and metrics that can be flawed or manipulated, and do not guarantee functional stability.

Innovation Solution

A multistage process that compares the performance of a candidate software build with clusters of previous builds, using performance data and source code graphs to identify similar builds, and employs a graph convolutional neural network to predict production environment performance, thereby informing decision-making on deployment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional testing methods (unit, integration, functional tests) are used to measure build quality, then source code quality can be judged, but production environment performance cannot be reliably predicted

Engineering Contradiction:

Improvebuild quality measurement accuracyVSAvoidproduction environment performance prediction

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The system performs preliminary actions by deploying builds to a test environment before production, collecting performance data in advance, and creating performance profiles that predict production behavior. This preliminary performance measurement in a controlled test environment resolves the contradiction by providing reliable production performance predictions without relying solely on conventional source code testing.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If regression test suites are expanded to cover larger codebases, then integration errors can be detected sooner, but processing time increases to hours

Engineering Contradiction:

Improveintegration error detectionVSAvoidregression test processing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

Instead of running complete regression test suites that take hours, the system applies partial action by using performance profile comparisons and clustering algorithms that analyze only the most relevant performance characteristics. This selective approach maintains integration error detection capability while reducing processing time from hours to a fraction of that duration.

Inventive Principle:

Principle #16Partial or excessive action

3Ease of manufacture

If code coverage metrics are used to evaluate testing completeness, then CICD threshold criteria can be met, but actual production performance relevance is low

Engineering Contradiction:

Improvetesting process automationVSAvoidproduction performance indication

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The system changes the evaluation parameters from static code coverage metrics to dynamic performance-based metrics. By measuring actual performance characteristics in the test environment and comparing them against historical production performance data, the system transforms the evaluation from a superficial code coverage check to a meaningful production performance prediction, resolving the contradiction between ease of automation and reliability.

Inventive Principle:

Principle #35Parameter changes

4Reliability

If immediate last production build comparison is used, then some quality issues can be identified, but historical build issues are not protected against

Engineering Contradiction:

Improvebuild quality comparisonVSAvoidhistorical build performance data

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The system performs preliminary action by maintaining a comprehensive database of historical performance data from multiple previous builds, not just the immediate last build. This historical performance information is prepared in advance and used for comparison with current builds, preventing the loss of valuable historical information while maintaining reliable build quality comparison.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11847446B2Predictive build quality assessment

Publication Date: 2023.12.19 CREDIT SUISSE (AG) CAYMAN ISLANDS BRANCH

AI summary

Predictive build quality assessment is disclosed, e.g., to predict performance of a candidate software build prior to deploying the candidate software build in a production environment. An example method can include comparing performance of a candidate software build with performance data associated with clusters of previous builds. A cluster having similar performance to the candidate software build can be selected. A source code graph for the candidate software build can be compared to source code graphs of builds included in the selected cluster. A previous build in the cluster, which is associated with a source code graph that is similar to the candidate software build's source code graph, can be selected. Individual performance data associated with the selected previous build can be inferred to be the predicted performance of the candidate software build, thereby informing decision making regarding further testing, modification, and/or deployment of the candidate software build into production.