Distributed Data Processing with Sandboxed Hermetic Language Runtimes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing customer data platforms (CDPs) face issues with applying business logic uniformly across applications, leading to unexpected results and resource contention due to centralized execution, which creates a single point of failure.
Innovation Solution
Implementing a hermetic and deterministic programming language, such as STARLARK, to execute customer business logic in a sandboxed runtime environment, allowing local execution and safe access to resources, while preventing unauthorized access and excessive use, and enabling uniform execution across multiple applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If centralized execution of data processing logic is used, then uniform application of business logic across applications is achieved, but resource contention and single point of failure occur
Solution Approach 1:
The patent divides the centralized execution model into distributed execution across multiple consumer services. Each service runs its own sandboxed runtime environment, allowing business logic to be executed locally rather than centralized. This segmentation eliminates the single point of failure while maintaining uniform execution through standardized sandboxed environments.
Solution Approach 2:
The patent introduces a sandboxed runtime environment as an intermediary between the business logic and the consumer services. This intermediary ensures uniform execution by providing a standardized, isolated environment that enforces consistent behavior across all services while preventing resource contention through controlled access to system resources.
2Adaptability or versatility
If business logic is created for one application, then that application can execute it, but unexpected results occur when applied to different applications
Solution Approach 1:
The patent applies local quality by allowing each consumer service to have its own sandboxed runtime environment with customized constraints and resource limits. This enables business logic to be adapted to specific application needs while maintaining reliability through the standardized sandboxed execution model that prevents unexpected behavior across different applications.
Solution Approach 2:
The patent changes the execution parameters by introducing sandboxed runtime environments with configurable resource constraints, isolation levels, and access controls. This allows business logic to be executed with different parameters for different applications while ensuring consistent and predictable behavior through the standardized sandboxed model.
3Reliability
If sandboxed runtime environment is used for hermetic execution, then safety and resource control are improved, but execution overhead increases
Solution Approach 1:
The patent applies preliminary action by pre-configuring sandboxed runtime environments with defined resource constraints, isolation policies, and access controls before execution. This preparation work is done in advance, allowing the actual business logic execution to proceed efficiently without incurring overhead during runtime, as the safety mechanisms are already in place.
Data Source
AI summary
In one embodiment, a computer system comprises one or more central processing units (CPUs) that are communicatively coupled to a system clock, one or more network interfaces, and one or more database interfaces; digital electronic main memory that is communicatively coupled to the one or more CPUs and storing one or more sequences of stored program instructions which, when executed using the one or more CPUs, cause the one or more CPUs to execute a plurality of different consumer services of a SaaS-based data analytics platform, each of the consumer services hosting an instance of a sandboxed runtime for a hermetic and deterministic programming language; user function storage that is communicatively coupled to one of the database interfaces and storing a plurality of different user functions, each of the user functions having been programmed using the programming language, each of the user functions being stored in association with a reference to a destination table of a destination database; each of the consumer services being programmed to initiate a data ingestion process; load a copy of a user function from the user function storage to the sandboxed runtime that is associated with the particular consumer service; using the sandboxed runtime local to the particular consumer service, execute the user function over records directed to the destination table identified in the reference; filter the records or write new records resulting from the function to the destination table.