Service dual-active implementation method and system of big data platform

A technology of big data platform and implementation method, which is applied in the field of active-active implementation method and system of big data platform business, which can solve the problems that data cannot be read and written at the same time, and batch processing and data query cannot be performed at the same time, so as to ensure timeliness Effect

Inactive Publication Date: 2016-09-07
AGRICULTURAL BANK OF CHINA
View PDF4 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to propose a method and system for implementing active-active business on a big data platform to solve the problem in the ...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Service dual-active implementation method and system of big data platform
  • Service dual-active implementation method and system of big data platform
  • Service dual-active implementation method and system of big data platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0039] The present invention is a processing system for the coexistence of efficient batch processing and real-time online query of massive data, and fundamentally needs to solve the problem of time and resource conflicts between batch processing and online real-time query.

[0040] The present invention is based on the column-stored MPP database architecture, realizes the incremental synchronization of binary data node-to-node among isomorphic (same number of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a service dual-active implementation method and system of a big data platform. The method comprises that a Hadoop cluster massive file processing cluster obtains and stores source files and preprocesses source data in the source files; a first cluster processes the data, thus obtaining data processing results and carries out node blocking to the data processing results; related data of the data processing results in the first cluster is synchronized to a second cluster by a synchronization tool; the first cluster and the second cluster are equipped with same numbers of nodes; and node-to-node increment synchronization is carried out between the first cluster and the second cluster by binary data. According to the MPP database architecture based on column storage provided by the invention, the node-to-node increment synchronization is carried out between the homogeneous clusters by the binary data; through transmission of compressed increment file blocks, the batch inquiry is separated; moreover, the online inquiry timeliness is ensured; and the problems of time and resource conflicts between batch processing and online real time inquiry are solved.

Description

technical field [0001] The present invention relates to the technical field of Internet and cloud computing, and more specifically, to a method and system for realizing active-active business of a big data platform. Background technique [0002] With the continuous development of the Internet and cloud computing technology, more and more services need to use cluster services to better meet user needs. On the one hand, service clustering enables users to get a better user experience, such as fast access, and at the same time, service clustering can provide services for more users at the same time; on the other hand, service clustering can improve system availability , to reduce the harm caused by system failure to the business. [0003] At present, with the continuous advancement of big data construction, on the basis of better solutions to data integration and processing, as the demand for data in the operation process increases, the service capabilities of big data platfor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F17/30
CPCH04L67/1097G06F16/182H04L67/1095
Inventor 周頔谢友珉陈宏晓张鸿郑东去游黎
Owner AGRICULTURAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap