Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and system of distributed data loading scheduling

A distributed data and distributed system technology, applied in the field of distributed data loading and scheduling, can solve problems such as low data loading efficiency, and achieve the effects of improving system throughput, ensuring high reliability, and improving efficiency

Active Publication Date: 2013-07-03
CHINA MOBILE COMM GRP CO LTD
View PDF5 Cites 17 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a method and system for distributed data loading scheduling aiming at the defect of low data loading efficiency of distributed systems in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system of distributed data loading scheduling
  • Method and system of distributed data loading scheduling
  • Method and system of distributed data loading scheduling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

[0061] like image 3 As shown, the distributed data loading scheduling system of the present invention includes:

[0062] More than one Source node (source loading node): deployed on the data source side, each source loading node is coupled with a source data node in a distributed data source (which can be a distributed database or a distributed file system), and is used to obtain Encapsulate the data in the source data node and send it to one or more corresponding destination loading nodes through the network;

[0063] More than one Dest node (destination loading node): deployed on the storage side, each destination loading node is coupled with a storage node in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a system of distributed data loading scheduling. The method comprises collecting the running status information of a source loading node and a purpose loading node in real time through a control node, and generating configuration updating information and sending the configuration updating information down to the source loading node and the purpose loading node through the control node according to the running status information. The control node is arranged in a distributed system. The method and the system of the distributed data loading scheduling adopt a method of dynamic topology generation and automatic scheduling. Due to the fact that a flexible configuration updating strategy is adopted, the load balancing and the high reliability of the system are guaranteed, the throughput rate of the system is improved, and the efficiency of data loading is improved.

Description

technical field [0001] The invention relates to a distributed system in the computer field, in particular to a method and system for distributed data loading scheduling. Background technique [0002] In the existing distributed data loading system, its data loading methods mainly include: [0003] Random configuration: such as figure 1 As shown, the source data node and the storage node are configured one-to-one. Before data loading, a combination of a source data node and a storage node is randomly selected. After the data loading starts, the relationship between each source data node and each storage node The connection relationship is static. [0004] High reliability configuration: such as figure 2 As shown, before data loading, assign two or more storage nodes to each source data node and send data at the same time, one storage node can receive data from multiple source data nodes; after data loading starts, each source data The connection relationship between the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/803H04L29/08H04L12/24
Inventor 高丹钱岭齐骥郭磊涛周大
Owner CHINA MOBILE COMM GRP CO LTD