Virtual Objects for Remote Analytics Data Access

Solution Overview

Problem

Traditional Extract, Transform, and Load (ETL) processes for analytics consume high network resources and account for up to 80% of analytics job time, leading to excessive resource consumption and prolonged processing times.

Innovation Solution

Facilitating analytics on remotely stored data sets by exposing virtual objects to a remote analytics engine without copying the data, making the data appear locally stored to enable efficient analytics job execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional ETL processes are used to copy data sets to analytics engines, then analytics jobs can be performed on the data, but network resource consumption increases and processing time is prolonged

Engineering Contradiction:

Improveanalytics job execution speedVSAvoidnetwork resource consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent creates virtual copies (virtual objects) of data stored in secondary storage systems and exposes these virtual objects to remote analytics engines. This allows analytics engines to access and process data without physically copying the actual data over the network, thereby maintaining analytics functionality while eliminating the network resource consumption and time overhead of traditional ETL data copying processes

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent introduces a virtual object layer as an intermediary between the secondary storage system and the analytics engine. This virtual object layer acts as a mediator that allows analytics engines to interact with remotely stored data as if it were locally available, eliminating the need for direct data copying while maintaining the appearance of local data access

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If data is copied from primary storage to analytics engine via traditional ETL, then analytics can be performed, but up to 80% of analytics job time is consumed by the ETL process itself

Engineering Contradiction:

Improveanalytics job completion rateVSAvoidETL process duration

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent creates virtual copies (virtual objects) of data stored in secondary storage systems and exposes these virtual objects to remote analytics engines. This allows analytics engines to access and process data without physically copying the actual data over the network, thereby maintaining analytics functionality while eliminating the network resource consumption and time overhead of traditional ETL data copying processes

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent pre-creates virtual objects that represent data in secondary storage systems before analytics jobs are executed. By having these virtual objects ready in advance and exposed to analytics engines, the system eliminates the need for time-consuming ETL data copying operations at the start of each analytics job, reducing ETL duration from 80% of total job time to near-zero

Inventive Principle:

Principle #10Preliminary action

3Loss of energy

If virtual objects are exposed to remote analytics engine without copying data, then network resource consumption decreases, but data must appear to be stored locally for analytics to work

Engineering Contradiction:

Improvenetwork resource consumptionVSAvoidvirtual object exposure mechanism

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The patent introduces a virtual object layer as an intermediary between the secondary storage system and the analytics engine. This virtual object layer acts as a mediator that allows analytics engines to interact with remotely stored data as if it were locally available, eliminating the need for direct data copying while maintaining the appearance of local data access

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates virtual copies (virtual objects) of data stored in secondary storage systems and exposes these virtual objects to remote analytics engines. This allows analytics engines to access and process data without physically copying the actual data over the network, thereby maintaining analytics functionality while eliminating the network resource consumption and time overhead of traditional ETL data copying processes

Inventive Principle:

Principle #26Copying

Data Source

PatentUS10528602B1Systems and methods for facilitating analytics on remotely stored data sets

Publication Date: 2020.01.07 COHESITY INC

AI summary

The disclosed computer-implemented method for facilitating analytics on remotely stored data sets may include (1) identifying, within a secondary storage system, a secondary copy of a data set duplicated from a primary copy of the data set stored in a primary storage system, (2) generating a set of virtual objects that represent at least a portion of the secondary copy of the data set, (3) exposing the set of virtual objects to a remote analytics engine via a network such that the portion of the secondary copy of the data set appears to be stored locally on the remote analytics engine, and then (4) enabling the remote analytics engine to perform at least one analytics job on the portion of the secondary copy of the data set by way of the set of virtual objects via the network. Various other methods, systems, and computer-readable media are also disclosed.