Dynamic Node Segmentation for Safe Infrastructure Software Rollouts

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The rollout of infrastructure software in production datacenters is risky due to hardware diversity and unknown user workloads, making it challenging to ensure compatibility and safety, as existing test infrastructures may not fully cover real-world conditions, leading to potential disruptions and compliance issues.

Innovation Solution

The system dynamically matches cloud service requests to clusters with nodes running test versions of infrastructure software, allowing for isolated validation within subsets of nodes, ensuring that issues are contained and validated independently by development teams, and enabling parallel and isolated testing across different hardware and software configurations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If infrastructure software is rolled out across the entire production datacenter, then software update coverage is improved, but system reliability deteriorates due to potential bugs impacting all nodes

Engineering Contradiction:

Improvesoftware update coverageVSAvoidsystem reliability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides the production datacenter into multiple isolated subsets or slices of nodes. Software updates are deployed to specific subsets rather than the entire datacenter, allowing parallel updates across different segments while containing potential bugs to only the affected subset. This segmentation enables continuous operation of other subsets unaffected by update issues.

Inventive Principle:

Principle #1Segmentation

2Loss of time

If test infrastructure is used to validate software changes, then validation speed is improved, but validation fidelity deteriorates due to hardware diversity not being covered

Engineering Contradiction:

Improvevalidation speedVSAvoidvalidation fidelity

Core Design Contradiction:

Loss of timeVSMeasurement precision

Solution Approach 1:

The patent performs preliminary validation by deploying software updates to a subset of production nodes before full rollout. This allows real-world validation on actual production hardware configurations, identifying compatibility issues with specific hardware types before they affect the entire datacenter. The subset acts as a testbed within the production environment itself.

Inventive Principle:

Principle #10Preliminary action

3Speed

If software updates are deployed to all nodes simultaneously, then deployment speed is improved, but bug impact worsens affecting all user workloads

Engineering Contradiction:

Improvedeployment speedVSAvoidbug impact

Core Design Contradiction:

SpeedVSObject-affected harmful factors

Solution Approach 1:

The patent segments the node population into multiple subsets that can be updated in parallel. Each subset is isolated from others, so if a bug is introduced in an update deployed to one subset, the harm is contained to only that subset while other subsets continue operating normally. This maintains overall system availability and limits user impact.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent deploys updates to only a partial subset of nodes initially rather than all nodes. This partial deployment allows the system to proceed with updates without waiting for complete validation, maintaining productivity while limiting exposure. Once validated on the subset, updates can be safely expanded to remaining nodes.

Inventive Principle:

Principle #16Partial or excessive action

4Reliability

If validation is performed on every production cluster, then validation coverage is improved, but system complexity worsens due to coordination overhead

Engineering Contradiction:

Improvevalidation coverageVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the validation process by assigning different subsets of nodes within clusters to different development teams. Each team validates their changes independently on their assigned subset, eliminating the need for complex coordination across all clusters. This segmentation simplifies the validation orchestration while maintaining comprehensive coverage across diverse hardware configurations.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11366744B2Partitioning and orchestrating infrastructure software deployments for safety and agility across diverse configurations and hardware types

Publication Date: 2022.06.21 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Systems, methods and computer program products are described herein that can be used to help achieve a safe rollout of software in a production datacenter environment. In accordance with certain embodiments, cloud services requests from certain users of a cloud services system (e.g., users that are authorized to receive cloud services via computing devices running test versions of infrastructure software) are dynamically matched to clusters (groups of commonly-managed computing devices called nodes) that are capable of providing the requested services on nodes running test versions of infrastructure software. Within such clusters, the requested services are provided to the users on a subset of cluster nodes that run a test version of an infrastructure software component, while the remaining cluster nodes are not running the particular test version.