Credential-less External Stages for Cloud Data Warehouse Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional external stages in cloud data warehouses require secret security credentials for access, leading to security vulnerabilities and limited control over access permissions, allowing potential data exfiltration and restricted usage to single file paths.
Innovation Solution
Credential-less external stage objects are created, allowing access to cloud storage without exchanging security credentials, enabling fine-grained control over access permissions and usage across multiple file paths, managed by the cloud storage provider.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If secret security credentials are exchanged for external stage access, then data warehouse can access external storage locations, but security vulnerabilities arise leading to potential credential exposure and unauthorized access
Solution Approach 1:
The patent introduces an intermediary authentication mechanism where the data warehouse service assumes the external storage account's identity through token-based authentication. Instead of directly exchanging secret credentials between systems, the data warehouse service acts as an intermediary that temporarily assumes the external account's identity to access storage locations, eliminating the need for credential sharing while maintaining access functionality
Solution Approach 2:
The patent extracts the secret credentials from the authentication process by implementing a credential-less authentication model. The external storage account's credentials are never exposed or shared with the data warehouse service. Instead, the system uses token-based authentication where the data warehouse service assumes the external account's identity without ever handling or storing the actual secret credentials
2Ease of operation
If external stages are created with broad access permissions, then data loading and unloading is simplified, but fine-grained control over access permissions is lost allowing potential data exfiltration
Solution Approach 1:
The patent implements local quality by enabling fine-grained, path-specific access permissions for external stages. Instead of applying uniform access controls across all storage locations, the system allows administrators to define different permission levels for different file paths and storage locations. This enables the data warehouse to access specific directories or files with appropriate permissions while restricting access to other areas, providing both operational simplicity and granular control
3Object-affected harmful factors
If external stages are limited to single file path usage, then security control is simplified, but versatility is reduced preventing reuse across multiple file paths
Solution Approach 1:
The patent implements universality by enabling external stages to be reused across multiple file paths and storage locations. A single external stage definition can specify multiple allowed paths, and the authentication mechanism automatically applies the appropriate permissions based on the specific path being accessed. This eliminates the need to create separate external stages for each file path while maintaining security through path-specific permission validation
4Adaptability or versatility
If account administrators have limited ability to prohibit external stage creation, then member autonomy is preserved, but data exfiltration risk increases
Solution Approach 1:
The patent implements preliminary anti-action by enabling account administrators to pre-configure external stages with restricted access permissions before members can use them. Administrators can define which external storage locations are accessible, what path-specific permissions apply, and which members can create or use external stages. This preliminary configuration prevents members from creating unauthorized external stages that could lead to data exfiltration, while still allowing autonomous use within the defined boundaries
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A storage integration object is created in a database of a data warehouse system. The storage integration object identifies a storage location in a storage platform of a cloud storage provider system and a cloud identity object maintained by a network-based data warehouse system. The cloud identity object is associated, at the storage platform of the cloud storage provider, with a proxy identity object granted permission to access the storage location. An external stage object is created based on the storage integration object. The external stage object identifies the storage location and includes an association with the storage integration object. A command to load or unload data at the storage location is received. In response to the command, the data is loaded or unloaded, via the proxy identity object, at the storage location using the external stage object.