Data processing method and device

A data processing and data source technology, applied in the computer field, can solve the problems of low computing efficiency in distributed data systems, achieve the effects of reducing maintenance costs and expansion costs, improving computing efficiency, and improving flexibility

Inactive Publication Date: 2018-01-05
HUAWEI TECH CO LTD
View PDF8 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The present invention provides a data processing method and device to solve the problem of low computational efficiency of distributed data systems in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device
  • Data processing method and device
  • Data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them.

[0046] Embodiments of the present invention provide a data processing method and device to solve the problem of low computing efficiency in distributed data systems in the prior art. Wherein, the method and the controller of the present invention are based on the same inventive concept. Since the principles of the method and the device to solve problems are similar, the implementation of the device and the method can be referred to each other, and the repetition will not be repeated.

[0047] In the embodiment of the present invention, the distributed data system includes multiple node servers, wherein the first node server among the multiple n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device for solving a problem that the distributed data system in the prior art is low in computing efficiency. The method comprises the following steps: a node server can acquire to-be-processed data from at least one data source in the procedure of performing the data processing on a data processing instruction; the acquired to-be-processed issplit, and split multiple partial to-be-processed data and a query plane corresponding to the data processing instruction are distributed to multiple node servers (comprising the mentioned node server) to realize the distributed correlation computation. Therefore, the computing pressure of the to-be-processed data processed by one node server is decomposed to multiple node servers to realize the horizontal expansion of the computing capacity of the node server; since the node server can acquire the to-be-processed data from multiple data sources, the computing capacity thereof is not limited by the I/O capacity of each data source, and the computing efficiency of the distributed data system is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data processing method and device. Background technique [0002] With the advent of the era of big data, users store data differently according to the needs of specific applications, that is, store data in multiple different databases. Due to the historical reasons of the application system construction and the characteristics of the data, the types of databases in the data layer of the application system are different. The types of databases can include Oracle databases, DB2 databases, and non-relational databases (NoSQL databases). However, different databases are usually maintained in different physical devices, which leads to low computational efficiency when performing associated calculations on data in different databases. [0003] Distributed databases can use multi-node storage and computing technologies, making it possible for application systems to achieve ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 蒋文
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap