Unlock instant, AI-driven research and patent intelligence for your innovation.

Data loading method, device and system

A data loading and loading machine technology, applied in the Internet field, can solve the problem of low data loading efficiency across computer rooms, and achieve the effect of improving data loading efficiency and saving resource occupation.

Active Publication Date: 2019-06-18
BEIJING QIHOO TECH CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In view of this, the present invention proposes a data loading method, device and system, the main purpose of which is to solve the problem of low data loading efficiency across computer rooms

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data loading method, device and system
  • Data loading method, device and system
  • Data loading method, device and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0131] In addition to creating a certain number of processes for writing to each group of loaders, the central control server in the host room can also create multiple threads under each process to speed up the speed of writing data fragments, among which One thread is responsible for writing to at least one loader. In the process of actually writing data shards through threads, there can be multiple implementation methods, including:

[0132] a) One loader corresponds to one thread. Taking a process as an example, the process creates multiple threads with the same number of loaders according to the number of loaders it is responsible for, and writes data slices to each loader in parallel through multiple threads to improve the process speed.

[0133] b) A group of loaders corresponds to one thread, that is, a process creates multiple threads with the same number of loaders according to the number of loader groups it is responsible for, and writes data fragments to each group...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data loading method, device and system, and relates to the technical field of the internet. The problem that the computer room striding data loading efficiency is low is solved. The method comprises the steps that block data in a transferring device are sent to a main computer room, the main computer room comprises a plurality of sets of loading machines, the number of the sets of the loading machines is equal to the number of auxiliary computer rooms, and the number of the loading machines in each set is equal to the number of secondary servers in the corresponding auxiliary computer room; the block data are written in the corresponding sets of loading machines through the main computer room respectively; the block data in the loading machines are written in the secondary servers corresponding to the loading machines in the auxiliary computer rooms. The data loading method, device and system are mainly applied to computer room striding updating of mass data.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a data loading method, device and system. Background technique [0002] Based on the development of Internet cloud technology, big data has attracted more and more attention. With the increase of data mining dimensions and the increase of covered users, the amount of valuable data mined is also increasing. In the actual process, in the face of massive amounts of valuable data, in order to improve user experience and meet system performance, the data will be deployed in multiple computer rooms or multiple databases to meet the query needs of different users. For example, a large amount of data can usually be stored in an external database (such as MySQL, Hbase) and updated regularly. [0003] In the prior art, for the scenario of data loading in multiple computer rooms, it is necessary to place the data blocks to be loaded on the disk of the springboard, and the spr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/2458G06F16/27G06F16/33G06F16/903
CPCG06F16/2471G06F16/27G06F16/3331G06F16/903
Inventor 曹柏余王颖曹雪林邓宇洪春晓
Owner BEIJING QIHOO TECH CO LTD