Custom Code Execution via Proxy Isolation on Data Platforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users creating customised code for cloud-based data processing platforms often face challenges as their code may not conform to the platform's technical requirements or security policies, leading to potential interference or corruption of other data resources, and existing solutions like manual assistance or software guidance tools are not foolproof.
Innovation Solution
A code creation tool that receives user-entered code, provides debugging aids for syntax, logical, and runtime errors, and commits the code to a repository with an identifier mapping to an execution environment, ensuring isolation and compliance with platform requirements, and a code execution tool that executes the code via a proxy to authorized services, preventing adverse impacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users are allowed to create and execute customised code on the data processing platform, then user autonomy and productivity are improved, but the reliability and security of the platform deteriorate due to potential code non-compliance
Solution Approach 1:
A code validation service acts as an intermediary between users and the data processing platform. This service validates user-submitted code against platform requirements, security policies, and data resource protocols before execution is permitted. The validation service includes automated testing capabilities that simulate code execution in isolated environments to detect potential harmful effects without risking the actual platform.
Solution Approach 2:
The system performs preliminary validation and testing of user code before it is executed on the live platform. Code is first submitted through a validation interface where it undergoes automated checks for syntax errors, logical errors, and runtime errors. Only after successful validation is the code committed to a repository and made executable on the platform, preventing non-compliant code from causing harm.
2Manufacturing precision
If manual assistance is provided to users for code creation, then code quality and compliance are improved, but the device complexity and loss of time increase
Solution Approach 1:
The code creation tool provides automated debugging aids that enable users to self-diagnose and self-correct errors in their code. The system includes integrated syntax checkers, logical error detectors, and runtime error predictors that guide users through fixing issues without requiring manual intervention from platform administrators or support staff.
Solution Approach 2:
The validation service provides immediate feedback to users about code quality issues, compliance problems, and potential errors. The system analyzes submitted code and returns detailed reports identifying specific issues with line numbers and suggestions for correction, enabling users to iteratively improve their code until it meets platform requirements.
3Ease of operation
If software guidance tools are provided to users, then ease of operation is improved, but the reliability deteriorates as the tools are not foolproof
Solution Approach 1:
The code validation process is segmented into multiple independent validation stages: syntax validation, logical error detection, runtime error prediction, and security compliance checking. Each stage independently verifies specific aspects of the code, and all stages must pass before code is permitted to execute. This multi-layered approach ensures comprehensive validation without relying on a single imperfect tool.
4Productivity
If code execution is permitted without validation, then productivity is improved, but harmful factors increase due to potential code errors
Solution Approach 1:
The system performs all necessary code validation, testing, and verification steps before code is permitted to execute on the platform. Automated validators check for syntax errors, logical errors, and runtime errors in advance. Only code that passes all validation checks is committed to the repository and made executable, preventing erroneous code from being deployed while maintaining efficient execution of validated code.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of executing computer-readable code for interaction with one or more data resources on a data processing platform is disclosed, wherein the method is performed using one or more processors. The method may comprise receiving a request message including an identifier identifying executable code stored in a data repository. Another operation may comprise_determining, using the identifier, an execution environment mapped to the executable code. Another operation may comprise_executing the identified executable code using the determined execution environment. A further operation may comprise_passing requests made with the executable code to one or more data resources via a proxy. Also disclosed is a method of creating customised computer-readable code for interaction with one or more data resources on a data processing platform, wherein the method is performed using one or more processors. This method may comprise receiving, through a code creation tool, user entered computer-readable code, committing the entered code to a data repository and creating an identifier which maps to the committed code and to an execution environment for running the committed code on the data processing platform.