External Access System for Data Processing Platform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud computing platforms face challenges in allowing customers to perform specific data processing tasks using their own applications or code on datasets stored within the platform, due to limitations such as size constraints, confidentiality issues, or licensing restrictions.
Innovation Solution
A method and system that enable external access to a data processing platform, allowing customers to perform data processing tasks externally using their own applications or code, by coordinating build requests between the platform and an external system, and managing dataset transfers and metadata handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If customers use their own applications or code externally to perform data processing tasks on datasets stored in the platform, then flexibility and control are improved, but data security and licensing compliance may be compromised
Solution Approach 1:
The patent introduces an external access system as an intermediary layer between the data processing platform and external systems. This mediator manages build requests, coordinates dataset transfers, and handles metadata exchange, allowing customers to execute their own applications externally while the system maintains control over data security and licensing compliance through coordinated access management
2Adaptability or versatility
If customers upload their own applications or code to the platform, then customization is improved, but platform size constraints and storage requirements worsen
Solution Approach 1:
The patent extracts the application execution environment from the platform's internal storage constraints by allowing customers to run their applications and code externally on their own infrastructure. The platform only stores necessary metadata and coordinates with external systems, removing the burden of storing and managing large customer applications within platform storage limits
3Ease of operation
If the platform provides comprehensive data processing capabilities internally, then integration is improved, but complexity of the platform worsens
Solution Approach 1:
The patent creates a universal external access interface that can handle multiple types of data processing tasks through a standardized build request mechanism. The co-ordinator function manages diverse external systems and applications through a common protocol, allowing the platform to provide comprehensive capabilities without increasing internal complexity by maintaining a simplified coordination layer
4Adaptability or versatility
If the platform allows external access to datasets, then functionality is improved, but control over data usage and licensing worsens
Solution Approach 1:
The patent implements a feedback mechanism where the external access system receives build requests from the platform, executes tasks externally, and returns results with metadata to the platform. This closed-loop communication allows the platform to maintain control over data usage and licensing by receiving confirmation that external operations completed successfully and by tracking what data was accessed and for what purpose
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus, and a method, performed by one or more processors are disclosed. The method may comprise receiving a build request associated with performing an external data processing task on a first data set, the first data set being stored in memory associated with a data processing platform to be performed at a system external to the data processing platform. The method may also comprise generating a task identifier for the data processing task, and providing, in association with the task identifier, the first data set to an agent associated with the external system with an indication of the data processing task, the agent being arranged to cause performance of the task at the external system, to receive a second data set resulting from performance of the task, and to provide the second data set and associated metadata indicative of the transformation. The method may also comprise receiving the second data set and metadata from the agent associated with the external system and storing the second data set and associated metadata.