Resource management method and system for data analysis and readable storage medium

A resource management system and data analysis technology, which can be used in electrical digital data processing, program startup/switching, program control design, etc., and can solve the problems of not supporting resource isolation and dynamic scheduling of resources.

Active Publication Date: 2018-05-15
WEBANK (CHINA)
View PDF3 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The main purpose of the present invention is to provide a data analysis resource management method, system, and readable storage medium, aiming at solving the problem that the existing data analysis resource management method does not support resource isolation and resource dynamic scheduling in multi-user scenarios technical problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Resource management method and system for data analysis and readable storage medium
  • Resource management method and system for data analysis and readable storage medium
  • Resource management method and system for data analysis and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0047] The solution of the embodiment of the present invention is mainly: after the resource manager receives the code execution request sent by the scheduler, the resource manager detects according to the execution request whether the user corresponding to the execution request has a parser used; if the user does not have the parser that can be reused, the resource manager will obtain the first resource usage of the user parser and the second resource usage of the big data platform situation, and the total resource usage of the big data platform, and determine whether the user has applied for a new The application authority of the parser; if the user has the application authority, assign a new parser to the user according to the application authority, and send the identification information corresponding to the new p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a resource management method and system for data analysis and a readable storage medium. The method comprises the steps that a resource manager detects that corresponding usershave no reusable parser according to a code execution request which is sent by a scheduler, the resource use condition of the users and the total resource use condition of a big data platform are acquired, and according to the resource use condition of the users, the total resource use condition and resources which are applied by the users, whether or not the users have application permissions for applying new parsers is determined; if yes, the new parsers are distributed to the users according to the application permissions, and identification information of the new parsers is sent to the scheduler; the scheduler sends an execution request to the big data platform through a remote service manager and the new parsers according to the received identification information, so that the big data platform executes codes which are input by the users according to the execution request. According to the resource management method for data analysis, the resources among different users are separated through the parsers, and the resources are dynamically distributed for the users.

Description

technical field [0001] The present invention relates to the technical field of data analysis, in particular to a data analysis resource management method, system and readable storage medium. Background technique [0002] Currently, the commonly used analysis software is SAS (Statistical Analysis System), which is a modular and integrated large-scale application software system. It consists of dozens of dedicated modules, and its functions include data access, data storage and management, application development, graphics processing, data analysis, report compilation, operations research methods, econometrics and forecasting, etc., and supports standard SQL (Structured Query Language (Structured Query Language) language operates on data, and can make simple lists to more complex statistical reports. In the Apache ecosystem, Zeppelin and Hue are currently open source visual data analysis tools. Zeppelin is a web-based notebook that provides interactive data analysis, making ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48
CPCG06F9/4843
Inventor 卢道和邸帅尹强
Owner WEBANK (CHINA)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap