Real-time online distributive calculation frame implementing method

A technology of distributed computing and implementation method, which is applied in the direction of software simulation/interpretation/simulation, program control device, etc. It can solve the problems of load balancing granularity that cannot be detailed at the JVM level, unnecessary JVM, and high startup overhead, so as to improve horizontal expansion capacity, increased throughput, and reduced start-up stress

Active Publication Date: 2015-06-24
DINGLI COMM
View PDF2 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Thereby have caused following defective in the prior art: (1) in MapReduce, Job starts to load slowly, because each Job needs to start a plurality of JVM processes on a plurality of servers, and the more servers start up overhead, the slower it is to start
(2) At present, MapReduce only supports Task multiplexing JVM. When it is necessary to continuously process multiple data analysis services on massive data, the overhead of repeatedly starting the JVM is obvious and unnecessary.
(3) More broadly, for distributed Java programs on clusters, the granularity of load balancing cannot be as fine as the JVM level

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time online distributive calculation frame implementing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0023] Such as figure 1 A method for realizing a real-time online distributed computing framework includes the following steps:

[0024] Step 101, share JVM resources in the MapReduce address space of JVM; JVM resources include stack, middleware for loading stack and application class for running stack; system stack is used to load class files in the life cycle of JVM operation; application class system stack It is used to build application class objects for shared JVM resources during the life cycle of the JVM; the non-system stack is used to build memory areas for the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A real-time online distributive calculation frame implementing method includes steps of 101, sharing JVM (java virtual machine) resources in a MapReduce address space of JVM, wherein the JVM resources include stacks, middleware downloading the stacks and applications operating the stacks; 102, resetting the JVM by the shared JVM resources, to be specifically, initializing the JVM, reusing the middleware and/or sharing the applications; 103, implementing online distributive calculation frame by resetting the JVM. In an affair processing environment of a large-scale cluster operation system, the durable reusable Java virtual machine can accelerate processing speed of Java applications. The durable reusable JVM can increase input and output of affair processing in these environments by providing the following functions, and startup time of MapReduce tasks used for processing massive data is greatly shortened.

Description

technical field [0001] The invention relates to the technical field of data processing and analysis in computer cloud computing, in particular to a method for realizing a real-time online distributed computing framework. Background technique [0002] With the rapid development of network communication technology, massive data analysis services in the field of cloud computing have been vigorously applied. In the field of cloud computing, massive data processing and analysis services and other distributed Java programs mainly use the persistent and reusable Java Virtual Machine (Java Virtual machine, hereinafter referred to as JVM) improves the throughput and speed of transaction processing in a cluster environment, and at the same time greatly reduces the startup time of MapReduce tasks for massive data processing, improves the efficiency of the system during continuous operation, and improves the system query communication CDR and KPI The query efficiency of data such as and...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/455
Inventor 张明浩王旭王耘陆金红
Owner DINGLI COMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap