Large transaction-oriented fragmentation execution method and device based on row change

An execution method and transaction technology, applied in the computer field, can solve the problems of low probability of large transactions, waste of resources, and low memory configuration in the operating environment, so as to improve synchronization efficiency, ensure integrity, and reduce memory configuration requirements. Effect

Active Publication Date: 2020-05-22
DAREWAY SOFTWARE
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, when a large transaction is encountered, a large transaction causes a large number of row changes. For example, a transaction changes tens of millions or hundreds of millions of rows of data. At this time, if all row changes are read in at one time, the operating environment needs to be very High memory configuration, but under normal circumstances, the operating environment does not have such a high memory configuration. Even if such a high memory configuration is conditional, the probability of large transactions occurring is very small. In this case, resources will be wasted

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large transaction-oriented fragmentation execution method and device based on row change
  • Large transaction-oriented fragmentation execution method and device based on row change
  • Large transaction-oriented fragmentation execution method and device based on row change

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0041] The present disclosure provides a large transaction-oriented sharding execution method based on row changes. This embodiment includes the following steps:

[0042] S1: Obtain the row change data of the synchronized database, convert it into a row change object and push it to the row change queue;

[0043] S2: Extract the row change object from the row change queue, convert multiple row change objects into a transaction object and push it to the transaction queue;

[0044] S3: Fragment the large transaction object, obtain the fragmented transaction object from the transaction queue, parse it into a corresponding SQL statement and execute it in the second database, and the second database synchronously updates the row change data.

[0045] Such as Figure 2-5 As shown, the meanings of the nouns in the figure are explained as follows:

[0046] CURRENT_MEM_SIZE: the total memory usage of all unexecuted row changes in the running environment;

[0047] MAX_MEM_SIZE: The ma...

Embodiment 2

[0072] The disclosure provides a large-scale transaction-oriented shard execution device based on row changes, including a row change push module, a transaction push module, and a transaction execution module;

[0073] The row change push module, transaction push module, and transaction execution module are executed in parallel.

[0074] The row change push module is responsible for converting the row change into Row (row change object) and pushing it to RowQueue (row change queue);

[0075] RowQueue is responsible for storing the pushed row changes;

[0076] The transaction push module is responsible for obtaining Row from RowQueue and converting it into Transaction (transaction object) and pushing it to TransQueue (transaction queue);

[0077] TransQueue is responsible for storing transactions to be executed;

[0078] The transaction execution module is responsible for obtaining Transaction from TransQueue and parsing it into corresponding SQL statements for execution.

Embodiment 3

[0080] The present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the program, a large transaction-oriented analysis based on row changes is implemented. The slice performs the steps described in the method.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a large transaction-oriented fragmentation execution method and device based on row change, and the method comprises the steps: obtaining row change data of a synchronized database, converting the row change data into row change objects, and pushing the row change objects to a row change queue; extracting the row change objects from the row change queue, converting the plurality of row change objects into a transaction object, and pushing the transaction object to a transaction queue; and fragmenting the large transaction object, obtaining the fragmented transaction object from the transaction queue, analyzing the fragmented transaction object into a corresponding sql statement, executing the sql statement in a second database in a fragmented manner, and synchronously updating the line change data by the second database A plurality of line change objects are combined into a large transaction object, the large transaction object is fragmented, and the large transaction object is analyzed into sql statement fragments for execution; and line change pushing, transaction pushing and transaction execution are processed in parallel, so that the real-time performance of line change data synchronization is ensured, and the synchronization efficiency is improved.

Description

technical field [0001] The present disclosure relates to the field of computer technology, and in particular to a large transaction-oriented fragmentation execution method and device based on row change. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] Row changes, that is, data row changes caused by database transaction execution. A row change includes the user to which the row belongs, the table to which the row belongs, the type of change (delete, add, or update), and the row data before and after the change. The data synchronization between the two databases is realized by reading all the row changes of the synchronized database, and then converting the row changes into corresponding SQL statements to be executed on another database; at the same time, in order to ensure the integrity of the transaction, select a Read all the row chang...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/2455G06F9/46
CPCG06F16/2308G06F16/2455G06F9/466Y02D10/00
Inventor 钱进张世栋于秋波徐喆郭斌赵永光巫双果宋婷婷张雪卫
Owner DAREWAY SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap