Big data platform data operation and maintenance method and system

A big data platform and data technology, applied in the computer field, can solve problems such as complex operations, batch jobs cannot well support intelligent repair, and data job maintenance cannot be solved.

Pending Publication Date: 2019-07-16
杭州数澜科技有限公司
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If the maintenance is performed directly in the server background, the operation and maintenance personnel need to have the operating authority and experience of the server and special personnel are required to do this
In addition, this kind of operation and maintenance is complex, inefficient, and difficult to respond quickly
If maintenance is performed through conventional click operations, its operational flexibility is not high enough, it is not suitable for various job repair scenarios, and it cannot well support its intelligent repair for batch jobs
An example of the data operation and maintenance method in the prior art is the bank data processing operation scheduling system and its method taught by the Chinese patent application publication number CN106156956A; The problem of data job maintenance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data platform data operation and maintenance method and system
  • Big data platform data operation and maintenance method and system
  • Big data platform data operation and maintenance method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] The content of the present invention will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are discussed only to enable those of ordinary skill in the art to better understand and thus realize the content of the present invention, and not to imply any limitation on the scope of the present invention.

[0031] As used herein, the term "including" and its variants are to be interpreted as open-ended terms that mean "including but not limited to." The term "based on" is to be read as "based at least in part on." The terms "one embodiment" and "one embodiment" are to be read as "at least one embodiment." The term "another embodiment" is to be read as "at least one other embodiment."

[0032] The different embodiments of the present invention can freely select the time period to repair the data of the job; can perform automatic dependent execution or independent parallel execution according to the time dimension; c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a big data platform data operation and maintenance method and system. The system includes an execution agent for executing an instance of a task for processing data; a scheduler for allocating instances of the task to the execution agent; a database which is used for storing the information of the instances and the task scheduling information related to the instances; and adata source that stores the data to be processed while the instance is running.

Description

Technical field [0001] The present invention relates to computer technology, in particular to a method and system for data operation and maintenance of a big data platform capable of efficient operation and maintenance. Background technique [0002] With the continuous growth of the amount of enterprise data and the increasing importance of enterprise data, there is a demand for continuous and effective mining of enterprise data. At this time, the demand for business data processing is increasing, and the demand for developing corresponding data operations is also increasing. In addition, there are complex dependencies between jobs. When the cluster fails, batch jobs need to be repaired. However, since there are many jobs, work efficiency will be low when performing manual repairs for each job. When a job fails or its code has been modified, the job needs to be repaired. [0003] In addition, in a big data platform, there will inevitably be a large number of data operations tha...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/21
CPCG06F16/21
Inventor 张翔
Owner 杭州数澜科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap