Cloud Workload Capture for Regression Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently testing new features or programs due to the risk of errors or bugs, especially in fast-moving cloud database services where there is a short time window for testing before release.
Innovation Solution
The implementation of systems, methods, and devices for capturing client workloads in database systems, allowing for rigorous real-world testing of features or programs on actual database data and queries, while ensuring security and resource isolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If new features or programs are released quickly in cloud database services, then productivity is improved, but the risk of errors or bugs increases due to insufficient testing time
Solution Approach 1:
The system performs preliminary testing of new features or programs by capturing and executing client workloads before the features are released to production. This advance testing allows identification and fixing of errors or bugs prior to release, resolving the contradiction between fast release cycles and reliability by ensuring thorough validation happens beforehand rather than risking post-release failures
2Reliability
If rigorous real-world testing is performed on actual database data and queries, then reliability is improved, but the time and resources required for testing increase
Solution Approach 1:
The system uses self-service by capturing actual client workloads that are already being executed on the database and reusing them for testing purposes. Instead of requiring separate test data generation and manual test case creation, the system automatically leverages the existing production workload as test input, achieving rigorous real-world testing without proportionally increasing time or resource requirements
3Measurement precision
If client workloads are captured and reused for testing new features, then measurement precision is improved by using real-world data, but the complexity of managing workload capture and isolation increases
Solution Approach 1:
The system introduces an intermediary layer that captures client workloads and manages their execution for testing purposes. This intermediary component handles the complexity of workload capture, storage, and reexecution, while providing isolation mechanisms to prevent testing from affecting production. By centralizing these functions in a dedicated intermediary system, the complexity is managed in one place rather than being distributed throughout the entire database system
Data Source
AI summary
A system to identify optimal cloud resources for executing workloads. The system deduplicates historical client queries based on a workload selection configuration to determine a grouping of historical client queries. The system generates a workload based on at least a portion of the grouping of historical client queries. The system repeatedly executes a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run. The system, in response to determining that there is no performance difference, identifies one or more sets of decreased resources of the cloud environment. The system re-executes the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment.