Data transfer node, method and system and computer readable storage medium

A transfer node and data reading technology, applied in the field of data transfer, can solve the problems that logstash occupies high memory and CPU, messages are easily lost, and the avalanche effect of the destination system, so as to ensure high efficiency and avoid reading bottlenecks

Pending Publication Date: 2021-05-14
CHINA UNIONPAY
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Data transfer has a strong demand in distributed systems, especially the scenario where traditional relational databases are transferred to distributed file systems (HDFS) is more complicated
However, in the process of data transfer, there are often unequal read and write performances between the source and destination, which leads to the problem that the destination cannot perform

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data transfer node, method and system and computer readable storage medium
  • Data transfer node, method and system and computer readable storage medium
  • Data transfer node, method and system and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0037]An exemplary embodiment of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be restricted herein. Instead, it is provided to provide more thoroughly understood the present disclosure, and can communicate the scope of the disclosure to those skilled in the art.

[0038]In the description of the present application embodiment, it should be understood that terms such as "including" or "having" are intended to indicate the features, numbers, steps, behaviors, components, part, or combinations thereof, and not It is intended to exclude the possibility of one or more other features, numbers, steps, behaviors, components, part, or combinations thereof.

[0039]Unless otherwise stated, " / " means or means, for example, A / B can represent a or b; "and / ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data transfer node, method, system and device and a computer readable storage medium, and the node comprises a data reading scheduler which is used for reading to-be-transferred data from a source database and putting the to-be-transferred data into a built-in queue of a memory; a queue scheduler which is used for acquiring the to-be-transferred data from the internal memory built-in queue and writing the to-be-transferred data into the target database; a memory built-in queue which comprises a reading queue and a reading buffer queue, the reading queue is used for receiving the data to be transferred from the data reading scheduler, and the reading buffer queue is used for providing the data to be transferred for the queue scheduler, wherein each time when the reading queue is fully written and the reading buffer queue is in an emptying state, the current reading queue is switched to the reading buffer queue, and the current reading buffer queue is switched to the reading queue. By means of the data transfer node, read-write separation can be achieved, high efficiency of data transfer is guaranteed, and the problem of read bottleneck is avoided.

Description

technical field [0001] The invention belongs to the field of data transfer, and in particular relates to a data transfer node, a method, a system and a computer-readable storage medium. Background technique [0002] This section is intended to provide a background or context for implementations of the invention that are recited in the claims. The descriptions herein are not admitted to be prior art by inclusion in this section. [0003] Data transfer has a strong demand in distributed systems, especially the scenario where traditional relational databases are transferred to distributed file systems (HDFS) is more complicated. However, in the process of data transfer, there are often unequal read and write performances between the source end and the destination end, which leads to the problem that the destination end cannot perform data transfer efficiently due to the system avalanche effect caused by excessive writing pressure or the source end is slow to read. At present,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/21G06F16/27
CPCG06F16/214G06F16/27
Inventor 唐真周之敏汤茂安张佳安
Owner CHINA UNIONPAY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap