Unlock instant, AI-driven research and patent intelligence for your innovation.

Computation reuse in analytics job service

An operation and computer technology, applied in computing, program control design, special data processing applications, etc., can solve the problems of waste of cluster resources and adverse effects of cluster operation costs.

Pending Publication Date: 2020-12-04
MICROSOFT TECH LICENSING LLC
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Simply evaluating the same job subexpression multiple times wastes cluster resources, which has an adverse effect on the operational cost of the cluster

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Computation reuse in analytics job service
  • Computation reuse in analytics job service
  • Computation reuse in analytics job service

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Various techniques related to automatically detecting and reusing overlapping computations in an analytics job service are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects. Furthermore, it should be understood that functionality described as being performed by certain system components may be performed by multiple components. Similarly, for example, a component may be configured to perform functionality described as being performed by multiple components.

[0027] The present disclosure supports ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Described herein is a system and method for detecting and reusing overlapping computations. Overlapping subgraphs of the query are determined using a normalized signature for a particular subgraph that identifies a particular subgraph across recurring instances of data. A normalized signature for each overlapping subgraph for the determined overlapping subgraphs of the query is provided. For eachoverlapping subgraph determined to be materialized: whether or not the particular subgraph has been materialized is determined using a precise signature corresponding to a normalized signature of theparticular overlapping subgraph. The precise signature identifies a particular subgraph corresponding to the normalized signature within a particular recurring instance of data. When the particular subgraph has not been materialized, the subgraph is materialized and used to respond to the query. When the particular subgraph has been materialized, the materialized subgraph is used to respond to thequery.

Description

Background technique [0001] Shared analytics clusters have become the de-facto way for large organizations to analyze and gain insights into their data. Clusters typically include tens of thousands of machines, store exabytes of data, and support thousands of users collectively running hundreds of thousands of batch jobs per day. [0002] In the case of a shared analytics cluster, significant overlap can be observed in the computations performed by submitted jobs. Simply computing the same job subexpression multiple times wastes cluster resources, which has an adverse effect on the operational cost of the cluster. Contents of the invention [0003] This article describes a system for automatically reusing overlapping computations in an analysis job service: A computer includes a processor and a memory having stored thereon computer-executable instructions that, when executed by the processor, cause the computer to: Receive a query; determine overlapping subgraphs for the q...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/2453
CPCG06F16/2393G06F16/24539G06F9/4881G06F16/24542
Inventor A·金达尔H·帕特尔乔石狄杰明M·K·巴格尹致诚
Owner MICROSOFT TECH LICENSING LLC