Test Service Manager for Storage Command Routing and Shedding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data center systems face challenges in testing changes to production storage services without disrupting production operations, as test storage commands can impact the performance and overload production services, leading to potential degradation of storage system performance.

Innovation Solution

Implementing a test service manager that routes test storage commands through a production service environment without interrupting production storage commands, using test identifiers to differentiate and manage test storage commands, and shedding or queuing test commands when production services experience overload, ensuring uninterrupted production processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If test storage commands are processed in the production service environment, then testing can be performed in the actual production environment, but production services may experience overload and performance degradation

Engineering Contradiction:

Improvetesting capabilityVSAvoidproduction service performance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system segments storage commands into two distinct categories: production storage commands and test storage commands. This segmentation is achieved through the use of different identifiers (production identifiers vs. test identifiers) that allow the storage processing service to differentiate and route commands appropriately. By dividing the command processing into separate streams, the system enables testing functionality while preserving production service performance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The storage processing service acts as an intermediary that receives both production and test storage commands, analyzes their types based on identifiers, and routes them to appropriate processing paths. This intermediary component enables coexistence of testing and production operations by mediating between test commands and the underlying storage resources, preventing test commands from directly impacting production service performance.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If test storage commands are routed through production services, then integration testing is improved, but production services may be interrupted or overloaded

Engineering Contradiction:

Improveintegration testing accuracyVSAvoidproduction processing throughput

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system dynamically adjusts its behavior based on the type of storage command received. The storage processing service continuously monitors incoming commands, identifies their type through identifier analysis, and dynamically routes them to appropriate processing paths. This dynamic approach allows the system to maintain high production throughput while seamlessly integrating test operations without fixed interference patterns.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

Different processing qualities are applied to different command types within the same storage processing service. Production storage commands receive prioritized processing with guaranteed throughput, while test storage commands are processed with appropriate routing to test storage resources. This local quality differentiation ensures that testing accuracy is maintained without compromising overall production productivity.

Inventive Principle:

Principle #3Local quality

3Loss of time

If test commands are processed concurrently with production commands, then testing efficiency is improved, but resource contention and overload occur

Engineering Contradiction:

Improvetesting timeVSAvoidservice load

Core Design Contradiction:

Loss of timeVSStress or pressure

Solution Approach 1:

The system segments the storage resource pool into production storage resources and test storage resources, allowing concurrent processing without resource contention. The storage processing service routes production commands to production resources and test commands to test resources based on identifier analysis, enabling parallel execution while preventing load mixing and resource contention.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system creates a copy of the production storage service environment specifically for test operations. Test storage services are instantiated as copies that mirror production functionality, allowing comprehensive testing while isolating test loads from production resources. This copying approach enables efficient concurrent testing without impacting production service stress levels.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS10521258B2Managing test services in a distributed production service environment

Publication Date: 2019.12.31 AMAZON TECH INC

AI summary

Systems and methods are described for a storage processing service that processes multiple storage commands. The storage processing service uses tags from test storage commands to determine whether a test storage service is to be instantiated that reflects a corresponding production service. Test storage commands with the same tag are tested on that test service. Additionally, the storage processing service determines a strategy for testing processes on production services when the storage system is overloaded. In one embodiment, the test service manager can determine to stop testing processes for a period of time, and issue a shed command that queues or sheds test storage commands. Advantageously, a shed command, while active at a storage processing service, may alleviate the overload on production services. The test service manager can continue to monitor the storage system to determine whether the overload continues to exist.