PostgreSQL database synchronization method and PostgreSQL database synchronization system

A technology for synchronizing systems and databases, applied in the field of big data, it can solve the problems of complex business layer logic, difficult data synchronization in EalsticSearch database, and poor maintainability.

Pending Publication Date: 2021-08-24
杭州每刻科技有限公司
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The above-mentioned data synchronization process is complicated to implement. The data maintenance in the search engine ElasticSearch library depends on the business logic of the application layer, and the logic of the business layer is generally more complicated. The data synchronization...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • PostgreSQL database synchronization method and PostgreSQL database synchronization system
  • PostgreSQL database synchronization method and PostgreSQL database synchronization system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The following description serves to disclose the present invention to enable those skilled in the art to carry out the present invention. The preferred embodiments described below are only examples, and those skilled in the art can devise other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents and other technical solutions without departing from the spirit and scope of the present invention.

[0035] It can be understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element The quantity can be multiple, and the term "a" cannot be understood as a limitation on the quantity.

[0036] Please combine figure 1 , the present invention provides a PostgreSQL database synchronization method and system, wherein th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a PostgreSQL database synchronization method and a PostgreSQL database synchronization system. The method comprises the following steps: acquiring a WAL log of a PostgreSQL database; configuring and registering a Kafka connector, and connecting the PostgreSQL database and the Kafka cluster; monitoring the WAL log, and storing WAL log data obtained through monitoring in a Kafka part; consuming the top data in the Kafka cluster, encapsulating the acquired top data into data in a Json format, and synchronizing the encapsulated data in the Json format into an ElasticSearch database. According to the method and the system, a partition algorithm is adopted, partition keys can be flexibly defined through configuration parameters, and databases with the same characteristics can be synchronized into the same partition for ordered consumption and storage of signing data.

Description

technical field [0001] The invention relates to the field of big data, in particular to a PostgreSQL database synchronization method and system. Background technique [0002] At present, the data maintenance in the ElasticSearch library of the search engine is to change the PostgreSQL data of the main database through the application layer business code, and at the same time call the relevant interface of ElasticSearch to synchronize the data to ElasticSearch, so as to realize the relative synchronization between the main data of PostgreSQL and the data of ElasticSearch. The above-mentioned data synchronization process is complicated to implement. The data maintenance in the search engine ElasticSearch library depends on the business logic of the application layer, and the logic of the business layer is generally more complicated. The data synchronization script of the ElasticSearch library must be injected into each related business code; Poor maintainability. With the deve...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/23G06F16/25G06F16/27G06F9/445
CPCG06F16/2365G06F16/27G06F16/258G06F9/44526
Inventor 丁佳军
Owner 杭州每刻科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap