Unlock instant, AI-driven research and patent intelligence for your innovation.

Data loading method and device, computer equipment and storage medium

A computer program and data technology, applied in the Internet field, can solve problems such as bottlenecks in the performance of the main node server, affecting data synchronization efficiency, and low data synchronization efficiency, so as to shorten data synchronization time, improve data synchronization efficiency, and avoid performance bottlenecks. Effect

Inactive Publication Date: 2018-08-03
SUZHOU LANGDONG NET TEC CO LTD
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the data warehouse must perform data synchronization through the master node. The performance of the master node server will affect the data synchronization efficiency, and there is a bottleneck in the performance of the master node server. As the amount of data increases, the time to synchronize data also increases exponentially.
That is, the traditional method of data warehouse data transmission through the master node will have the problem of low data synchronization efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data loading method and device, computer equipment and storage medium
  • Data loading method and device, computer equipment and storage medium
  • Data loading method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

[0030] The data loading method provided by this application can be applied to such as figure 1 The shown application environment includes a database 100 and a cluster, and the cluster includes a data node 120 and a master node 130, wherein the database 110 is a data set organized, stored, managed and built on a computer storage device according to a data structure. The cluster includes a group of data nodes 120 and master nodes 130. The master nodes 130 and data nodes 120 in the cluster communicate through a high-speed network, and each data node cooperates with each other...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a data loading method and system, computer equipment and a storage medium. The method comprises the following steps of: obtaining an operation instruction of an executable file; obtaining information of data nodes in a cluster according to the operation instruction of the executable file, wherein the information of the data nodes comprises a quantity of the data nodes; andloading to-be-distributed data to corresponding data nodes according to the quantity of the data nodes. According to the data loading method and device, the computer equipment and the storage medium,the technical effects of enhancing the data synchronization efficiency and shortening the data synchronization time can be realized, and the performance bottlenecks of main node servers can be avoided.

Description

technical field [0001] The present application relates to the technical field of the Internet, in particular to a data loading method, device, computer equipment and storage medium. Background technique [0002] With the development of the Internet industry, massive amounts of information and data have been accumulated. In order to meet the needs of processing massive data, big data solutions using distributed database systems have emerged. With the help of distributed computing ideas, big data clusters use massively parallel processing architectures to realize database-based distributed data storage and parallel computing in software. Distributed big data cluster is mainly composed of three parts: data master node (master host), data node (segment host), and high-speed network (interconnect). [0003] In the traditional technology, when the source data in the relational database is transmitted to the data warehouse of the cluster, first, the data master node receives the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/254G06F16/27G06F16/284
Inventor 何鑫
Owner SUZHOU LANGDONG NET TEC CO LTD