Data Processing Arbitration Between Local and Cloud Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The inefficient storage, transfer, and processing of large amounts of data in bioinformatics systems, particularly in managing data operations between local and remote computing resources, lead to performance lag and resource wastage, especially when handling sensitive and private information.
Innovation Solution
A data processing controller that automatically and intelligently arbitrates between executing data processing operations locally and remotely using local and cloud computing clusters, determining the suitability based on data sensitivity and resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data processing operations are performed using remote cloud computing clusters, then processing capacity and scalability are improved, but data security and privacy protection deteriorate
Solution Approach 1:
The patent segments data processing operations into different categories (sensitive vs. non-sensitive) and routes them to different computing resources accordingly. Sensitive data operations are performed locally on-premises, while non-sensitive operations are performed remotely on cloud clusters, achieving both security and scalability
Solution Approach 2:
The patent implements local quality by performing data processing operations locally for sensitive data while utilizing remote resources for non-sensitive data. This creates different processing qualities based on data sensitivity requirements, ensuring security where needed while maintaining scalability where appropriate
2Power
If data is transferred to remote computing resources for processing, then computational resources are improved, but network bandwidth consumption and transfer time increase
Solution Approach 1:
The patent applies partial action by transferring only the necessary data to remote resources for processing rather than transferring all data. This minimizes network bandwidth consumption while still utilizing remote computational resources effectively for the specific processing tasks required
3Productivity
If manual arbitration between local and remote computing resources is implemented, then resource optimization is improved, but system complexity and user workload increase
Solution Approach 1:
The patent implements self-service by enabling the system to automatically determine whether to perform data processing operations locally or remotely based on data sensitivity and resource availability. This eliminates manual arbitration complexity while maintaining optimal resource utilization through automated decision-making
4Object-affected harmful factors
If all data processing operations are performed locally, then data security is improved, but processing capacity and scalability deteriorate
Solution Approach 1:
The patent segments processing operations by data sensitivity level, performing sensitive operations locally to maintain security while routing non-sensitive operations to remote cloud resources to achieve scalability and increased processing capacity
Data Source
AI summary
A data processing architecture controls data processing arbitration between a service provider, a local computing cluster, and a remote computing cluster. The architecture receives, by a data processing controller, a request to perform one or more data processing operations for a batch of data and determines, by the data processing controller, whether the batch of data includes private information. The data processing controller selects a computing cluster from a plurality of computing clusters based on a result of determining whether the batch of data includes private information, the plurality of computing clusters comprising a local cluster and a cloud cluster. The data processing controller instructs the selected computing cluster to access the batch of data and perform the one or more data processing operations.