Test case generator built into data-integration workflow editor

Abstract

Embodiments presented herein provide techniques for generating test cases for transformation modules that are part of a larger data-integration workflow for an extract, transfer, and load (ETL) project. A test case generated according to the present disclosure can be executed independently of the application that generated it and independently of data sources and destinations referenced in the data-integration workflow. To achieve this independence, the test case can include code that simulates data sources and destinations that interact with the transformation module in the data-integration workflow. Furthermore, the test case can be a unit test based on a unit testing framework and can be compatible with a selected software framework.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims benefit of U.S. Provisional Patent Application No. 62 / 395,179, entitled “Test Case Generator Built into Data-Integration Workflow Editor,” filed Sep. 15, 2016; and U.S. Provisional Patent Application No. 62 / 402,880, entitled “Test Case Generator Built into Data-Integration Workflow Editor,” filed Sep. 30, 2016, both of which are assigned to the assignee hereof and the contents of which are hereby incorporated by reference.BACKGROUNDField[0002]The present disclosure generally relates to generating unit tests for elements of a data-integration workflow. More specifically, the present disclosure provides techniques for a data-integration workflow editor to generate code to test user-selected program modules within a user-selected software framework.Related Art[0003]Technology for gathering and storing data has greatly improved in recent decades. In particular, data collection rates, data access speeds, and data storag...

AI Technical Summary

Benefits of technology

The present patent relates to techniques for generating unit tests for data-integration workflows, which are used for extracting data from different sources, transforming the data, and loading it into a single data repository for statistical analysis and predictive modeling. The techniques involve creating a graphical model of the data transformation workflow and allowing the user to select icons for different modules within the workflow. The user can also request that the test case be compatible with a specific software framework. The code generator creates a test case for the selected module and executes it using the software framework. The test case compares actual output data with target output data and indicates if the test succeeded or failed. The techniques help manage software for ETL projects effectively through continuous integration and ensure the correct structure and operation of components within the data-integration workflow.

Problems solved by technology

As a result, large-scale data analysis that would have been impractical a few decades ago is now possible.

Images