Distributed database storage method, system, device and storage medium

A distributed and database technology, applied in the field of big data processing, can solve the problems of reducing staff efficiency, low data transmission rate, and low support for near real-time scenarios, and achieve the effect of improving data processing efficiency

Inactive Publication Date: 2019-11-05
JIANGSU MANYUN SOFTWARE TECH CO LTD
View PDF7 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, because this solution does not support corresponding data processing, the data transmission rate is relatively low, and it

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed database storage method, system, device and storage medium
  • Distributed database storage method, system, device and storage medium
  • Distributed database storage method, system, device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0049] Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a distributed database storage method, a system, a device and a storage medium. The method comprises the steps: obtaining data storage configuration data, wherein the data storage configuration data comprises the information of a synchronization table in a kafka message queue and the information of a landing table in a distributed database; receiving to-be-stored data from the synchronization table in the kafka message queue by a Flink distributed data flow engine; and inserting the to-be-stored data into a landing table of the distributed database by the Flink distributed data flow engine. By adopting the scheme of the invention, the time of a data processing link is extremely compressed. A multi-aggregation-node and multi-source service scene is supported. The useis humanized, and the data processing and analysis efficiency is greatly improved.

Description

technical field [0001] The present invention relates to the technical field of big data processing, in particular to a method, system, equipment and storage medium for storing distributed databases. Background technique [0002] With the rapid development of the Internet and Internet technology, the data generated every day is growing at an exponential rate. The processing and analysis of these large amounts of data has great application value. With the increase of real-time data, the traditional offline data calculation has become more and more difficult. It is becoming more and more difficult to meet the needs of analysis, so stream computing is more and more widely used. [0003] Greenplum, a large-scale parallel data analysis engine, is a database based on PostgreSQL. It has the characteristics of resource sharing, high concurrency, and fast data processing. In today's complex and diverse data processing scenarios, it cannot be distributed and expanded compared to mySQL ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/27G06F16/23G06F16/25
CPCG06F16/23G06F16/254G06F16/27
Inventor 张宁董延峰
Owner JIANGSU MANYUN SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap