Versioned Schema Loading via Compute Service Manager

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data platforms face challenges in updating applications while they are in use, requiring methodologies that allow for seamless updates, fine-grained security controls, and protection of consumer data and intellectual property, while enabling global distribution and monetization of applications.

Innovation Solution

The proposed solution involves a data platform architecture that includes a compute service manager, execution platform, and metadata database, allowing for in-database application updates, fine-grained security controls, and global distribution of applications, with features like versioning, pinning, and asynchronous upgrades to minimize disruption and ensure security.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Duration of action of stationary object

If applications are updated while in use, then continuous operation is maintained, but security control and intellectual property protection become more difficult

Engineering Contradiction:

Improvecontinuous operationVSAvoidsecurity control

Core Design Contradiction:

Duration of action of stationary objectVSReliability

Solution Approach 1:

The application is divided into multiple versions (current version and new version) that coexist in the database. The compute service manager segments the update process into discrete versioned schemas, allowing selective activation of new versions while maintaining the current version for ongoing operations. This segmentation enables security controls to be applied independently to each version.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The compute service manager acts as an intermediary between the application versions and the execution platform. It manages version registration, activation, and deactivation, controlling which version executes when. This intermediary layer provides fine-grained security control over version transitions and execution without disrupting continuous application operation.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If versioned schemas are loaded dynamically, then update flexibility is improved, but system complexity increases

Engineering Contradiction:

Improveupdate flexibilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The versioned schema mechanism provides universal functionality for managing multiple application versions within a single database system. The same infrastructure handles version registration, activation, deactivation, and execution for all applications, reducing the need for application-specific update management code and simplifying the overall system despite the added versatility.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system automatically manages versioned schemas through the compute service manager, which handles version registration, validation, activation, and deactivation without requiring manual intervention. The system self-services the complexity of version management, providing update flexibility while abstracting the complexity from users and applications.

Inventive Principle:

Principle #25Self-service

3Reliability

If fine-grained security controls are implemented, then intellectual property protection is improved, but ease of operation decreases

Engineering Contradiction:

Improveintellectual property protectionVSAvoidease of operation

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The compute service manager implements feedback mechanisms by monitoring which version is currently active and automatically managing version transitions. The system provides feedback about version status and handles security control decisions automatically, reducing the operational burden on users while maintaining strong intellectual property protection through version-specific access control.

Inventive Principle:

Principle #23Feedback

4Reliability

If multiple application versions are maintained, then update rollback capability is improved, but resource consumption increases

Engineering Contradiction:

Improveupdate rollback capabilityVSAvoidresource consumption

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system maintains multiple application versions in the database but allows for selective deactivation and removal of obsolete versions. The compute service manager manages the lifecycle of versions, enabling rollback by reactivating previous versions while providing mechanisms to discard and recover resources from versions that are no longer needed, balancing rollback capability with resource consumption.

Inventive Principle:

Principle #34Discarding and recovering

Data Source

PatentUS20240311136A1Loading procedures of versioned schema

Publication Date: 2024.09.19 SNOWFLAKE INC

AI summary

A versioned schema of a data platform. A process of maintaining a call stack of executing objects of an application package having a versioned schema includes calling, by a first procedure executed by one or more processors, a second procedure of a versioned application instance, and determining, by the first procedure, a version of the second procedure based on a call context. In response to determining that the version of the second procedure is not in the call context, the first procedure determines a current version of the versioned application package adds the current version to the call context as the version of the second procedure.