Transaction log processing method and system for distributed database

A processing method and distributed technology, applied in the field of log processing, can solve problems such as uncertain order, scattered transaction logs, uncertain length of transaction execution process, etc., to achieve the effect of reducing complexity and improving efficiency

Pending Publication Date: 2022-03-04
上海沄熹科技有限公司
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Due to the uncertain length of the transaction execution process, in the case of high concurrent requests, the transaction logs recorded by each computing node are scattered and in an uncertain order from a global perspective, which makes it difficult for subsequent data backup, data recovery and Data synchronization and other operations bring a lot of inconvenience

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Transaction log processing method and system for distributed database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0051] combined with figure 1 , the present embodiment proposes a transaction log processing method of a distributed database, the processing process including:

[0052]Each node of the distributed database accepts the client request and executes the transaction. After the transaction is executed, each node writes its own transaction log into the transaction log file. The transaction log contains the transaction ID and transaction time stamp. The transaction ID is A randomly generated unique value, the transaction timestamp is the time when the transaction starts, that is, the system time of the host where the node is located when the transaction starts;

[0053] The log collector monitors transaction log files, collects incremental logs and sends them to message queue 1;

[0054] Log processor 1 consumes transaction log messages from message queue 1, and writes them into message queue 2 after performing global sorting;

[0055] The log processor II consumes globally ordered...

Embodiment 2

[0066] combined with figure 1 , the present embodiment proposes a transaction log processing system of a distributed database, and its structure involves each node of the distributed database, a log collector, a message queue 1, a log processor I, a message queue 2, and a log processor II;

[0067] Each node of the distributed database accepts the client request and executes the transaction. After the transaction is executed, each node writes its own transaction log into the transaction log file. The transaction log contains the transaction ID and transaction time stamp. The transaction ID is A randomly generated unique value, the transaction timestamp is the time when the transaction starts, that is, the system time of the host where the node is located when the transaction starts;

[0068] The log collector monitors transaction log files, collects incremental logs and sends them to message queue 1;

[0069] Log processor 1 consumes transaction log messages from message queu...

Embodiment 1 Embodiment 2

[0080] For Embodiment 1 and Embodiment 2, what needs to be supplemented is:

[0081] ①The node of the distributed database refers to each instance process that the distributed database actually runs. Each instance process can independently accept client requests, and control transaction execution and record transaction logs.

[0082] ②The transaction log refers to the transaction log recorded by each node of the distributed database. The transaction log of each node only records the log of the transaction executed by the node;

[0083] ③ The log collector refers to the application program that collects transaction logs. The log collector will monitor the transaction log file. When a transaction log is appended to the file, the log collector will collect the incremental transaction log and send it to the message queue. Each node's transaction log is sent to a separate topic.

[0084] ④ Message queue 1 / message queue 2 is used to receive and temporarily store transaction log me...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a transaction log processing method and system of a distributed database, and relates to the technical field of log processing, and the processing process comprises the following steps: each node of the distributed database receives a client request, executes a transaction, and writes a transaction log recorded by each node into a transaction log file after the execution of the transaction is completed; a log collector monitors a transaction log file, collects an incremental log and sends the incremental log to a message queue 1; the log processor I consumes transaction log messages from the message queue 1, and the transaction log messages are written into a message queue 2 after being globally sorted; and the log processor II consumes the globally ordered transaction log messages from the message queue 2 and writes the globally ordered transaction log messages into a transaction log file, and the transaction log file centrally stores globally ordered transaction logs. According to the method, centralized storage and global ordered arrangement of the transaction logs can be realized, so that the functions of data backup, data recovery, data synchronization and the like can be conveniently realized by using the transaction logs subsequently.

Description

technical field [0001] The invention relates to the technical field of log processing, in particular to a transaction log processing method and system of a distributed database. Background technique [0002] Most of the current distributed databases use a multi-copy mechanism, and the Raft consistency protocol is used between multiple copies to ensure data consistency. The multi-copy mechanism itself can provide rich disaster recovery capabilities, so most distributed databases basically do not record transaction logs. Some distributed databases record transaction logs, but since each computing node is peer-to-peer, user requests can be processed in parallel and transaction logs can be recorded. Transaction logs generally must contain transaction ID (id) and timestamp (ts), abbreviated as txn i (id i , ts i ), where the id is generated randomly to ensure uniqueness, and ts is mainly related to the system time of each node. Generally, each node will have a certain time e...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30G06F16/23G06F16/27G06F9/54
CPCG06F11/3065G06F11/3006G06F16/2358G06F16/27G06F9/546G06F2201/80G06F2201/835G06F2209/548
Inventor 江磊陈磊
Owner 上海沄熹科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap