Cloud Data Pipeline for Cryo-EM Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current cloud computing services face challenges in efficiently processing and transferring large volumes of data generated by research experiments, such as cryo-electron microscopy, which results in significant time delays and slows down research.

Innovation Solution

A scalable cloud-based data processing and computing platform is developed, which includes methods for receiving synchronization requests, determining files in a staging location, generating data transfer filters, and transferring files to destination computing devices, optimizing data transfer and processing times.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Power

If large volume data is transferred to cloud computing services for processing, then computing power and processing capability are improved, but data transfer time increases significantly

Engineering Contradiction:

Improvecomputing powerVSAvoiddata transfer time

Core Design Contradiction:

PowerVSLoss of time

Solution Approach 1:

The patent segments the data transfer process by implementing a data pipeline that divides data into manageable chunks and processes them through multiple stages (data generation, staging, filtering, transfer, and processing). This segmentation allows parallel processing and optimizes transfer efficiency, reducing overall transfer time while maintaining access to cloud computing power.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by pre-processing data in a staging location before cloud transfer. Data is prepared, filtered, and organized in advance using the data transfer filter, which identifies and prioritizes critical data elements. This preliminary preparation reduces the complexity and time of actual cloud transfer operations.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If all generated data is uploaded to cloud storage, then data availability for processing is improved, but upload time and research productivity are reduced

Engineering Contradiction:

Improvedata availabilityVSAvoidresearch productivity

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements local quality by applying different handling strategies to different data elements. The data transfer filter identifies and prioritizes critical data for immediate cloud transfer while allowing less critical data to be processed locally or transferred with lower priority. This selective approach ensures data availability for processing while maintaining research productivity by avoiding unnecessary transfer delays.

Inventive Principle:

Principle #3Local quality

3Speed

If data is processed locally on workstations or computer clusters, then processing speed is maintained, but computing power and processing capacity are limited

Engineering Contradiction:

Improveprocessing speedVSAvoidcomputing capacity

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent introduces an intermediary data pipeline system that bridges local processing environments and cloud computing resources. The pipeline includes staging areas and transfer filters that mediate between local data generation and cloud processing, enabling seamless integration of local processing speed with cloud computing capacity. This intermediary system allows researchers to maintain fast local preprocessing while accessing unlimited cloud computing power for intensive analysis.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12254347B2Data pipeline

Publication Date: 2025.03.18 REGENERON PHARMACEUTICALS INC

AI summary

A scalable cloud-based data processing and computing platform to support a large volume data pipeline.