Virtual Object Generation for Remote Monolithic File Analytics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional Extract, Transform, and Load (ETL) processes for analytics consume excessive network resources and time, and traditional analytics engines are unable to handle monolithic files, leading to inefficiencies and limitations in data analysis.

Innovation Solution

The system generates virtual objects representing data objects within remote monolithic files, allowing these objects to be exposed to an analytics engine without copying them, making them appear locally stored, enabling analytics engine operations even on files it cannot natively open.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional ETL processes are used to copy data sets to analytics engines, then analytics can be performed on the data, but network resources are excessively consumed and processing time is prolonged

Engineering Contradiction:

Improveanalytics processing speedVSAvoidnetwork resource consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent creates virtual copies of data objects from remote monolithic files rather than physically copying the actual data. Virtual objects are generated that reference the original data location, allowing analytics engines to access remote data without network transfer of the underlying data bytes, thus eliminating ETL copying overhead while maintaining analytics functionality

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent introduces a virtualization layer that acts as an intermediary between remote monolithic files and analytics engines. This layer generates virtual objects that mediate access to the original data, allowing analytics operations to proceed on virtual representations without requiring physical data movement across the network

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If traditional ETL processes are used for analytics, then data can be loaded into analytics engines, but up to 80% of analytics job time is consumed by the ETL process itself

Engineering Contradiction:

Improveanalytics job throughputVSAvoidETL process duration

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent performs preliminary virtualization of data objects before analytics processing begins. By pre-generating virtual objects that represent the data structure and location information, the system prepares access pathways in advance, eliminating the need for time-consuming ETL extraction and loading steps during actual analytics job execution

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If traditional analytics engines are used, then analytics operations can be performed on local data, but the engines are unable to open or read remote monolithic files

Engineering Contradiction:

Improveanalytics engine compatibilityVSAvoidinability to process monolithic files

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent creates virtual copies of data objects from within remote monolithic files without requiring the analytics engine to natively open or parse the monolithic file format. The virtualization layer extracts and represents individual data objects virtually, allowing traditional analytics engines to process these virtual objects as if they were local files while actually accessing remote monolithic data sources

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent introduces a virtualization intermediary that translates between the monolithic file structure and individual data objects. This mediator layer handles the complexity of accessing data within monolithic files, presenting simplified virtual object interfaces to analytics engines that cannot natively process monolithic formats, thus extending engine compatibility without modifying the engines themselves

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9864790B1Systems and methods for facilitating analytics on data sets stored in remote monolithic files

Publication Date: 2018.01.09 COHESITY INC

AI summary

The disclosed computer-implemented method for facilitating analytics on data sets stored in remote monolithic files may include (1) identifying, within a secondary storage system, a secondary copy of a data set duplicated from a primary copy of the data set stored in a primary storage system, (2) generating a set of virtual objects that represent at least a portion of the secondary copy of the data set, (3) exposing the set of virtual objects to a remote analytics engine via a network such that the set of individual data objects appears to be stored locally on the remote analytics engine, and then (4) enabling the remote analytics engine to perform at least one analytics job on the set of individual data objects by way of the set of virtual objects via the network. Various other methods, systems, and computer-readable media are also disclosed.