Adaptive Integration Testing via Author Analytics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Continuous integration processes are hindered by lengthy integration testing procedures, which consume significant time and resources without ensuring the integrity and functionality of the project.

Innovation Solution

A computer-implemented method that uses analytics data to simulate error rates when omitting certain testing stages, determining a reduced testing procedure based on risk thresholds, thereby streamlining the integration process while maintaining project integrity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If comprehensive testing procedures are performed on every commit, then project integrity and functionality are ensured, but integration time increases significantly

Engineering Contradiction:

Improveproject integrityVSAvoidintegration time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system dynamically changes the testing parameters (which tests to execute) based on the author's historical performance data. High-performing authors have fewer tests executed on their commits, while lower-performing authors trigger more comprehensive testing, thus adapting the testing regimen to individual contributor reliability

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

Instead of performing all possible tests on every commit, the system performs only the necessary subset of tests based on risk assessment. For commits from trusted authors with proven track records, fewer tests are executed, while commits from less reliable authors undergo more extensive testing

Inventive Principle:

Principle #16Partial or excessive action

2Measurement precision

If extensive testing stages are executed for every commit, then error detection capability is improved, but integration productivity decreases

Engineering Contradiction:

Improveerror detection capabilityVSAvoidintegration productivity

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The testing regime parameters are dynamically adjusted based on the author's historical error rates and performance metrics. The system modifies which testing stages are executed and with what depth, balancing error detection needs against integration speed requirements

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The testing procedure is made dynamic rather than static. The system continuously learns from historical data and adjusts the testing intensity in real-time based on the author's current performance, creating an adaptive testing strategy that optimizes both detection capability and productivity

Inventive Principle:

Principle #15Dynamics

3Stability of the object's composition

If all testing stages are performed uniformly for every commit, then consistency in quality assurance is maintained, but resource consumption increases

Engineering Contradiction:

Improvequality assurance consistencyVSAvoidresource consumption

Core Design Contradiction:

Stability of the object's compositionVSLoss of energy

Solution Approach 1:

Instead of applying uniform testing quality across all commits, the system applies differentiated testing strategies tailored to each author's performance characteristics. High-performing authors receive streamlined testing while maintaining adequate quality checks, while lower-performing authors trigger more rigorous testing protocols

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system changes the testing parameters dynamically based on author performance data, adjusting the scope, depth, and type of testing executed. This allows the system to maintain quality assurance consistency through data-driven decision-making while reducing unnecessary resource consumption on low-risk commits

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11080180B2Integration times in a continuous integration environment based on statistical modeling

Publication Date: 2021.08.03 KYNDRYL INC

AI summary

A computer-implemented method may include: receiving a request to integrate a commit; obtaining analytics data of an author that developed the commit; executing a simulation using the analytics data of the author as inputs to the simulation; obtaining results from the simulation, wherein the results indicate error rates when one or more testing stages are omitted from a testing procedure of the commit; comparing the results of the simulation with a threshold; determining, by the computing the device, the testing procedure based on the comparing, wherein the testing procedure identifies the one or more testing stages that are omitted and one or more testing stages that are included in the testing procedure; and outputting information regarding the determined testing procedure, wherein the outputting causes an integration server to test the commit in accordance with the testing procedure as part of an integration process for integrating the commit to a project.