Large-scale data processing system and method

A large-scale data and processing system technology, applied in the direction of input/output to record carrier, etc., can solve the problem that the network bandwidth system memory cannot meet the needs of large-scale data processing, shorten calculation time, improve system efficiency, and optimize data transmission The effect of the process

Active Publication Date: 2015-12-16
INSPUR BEIJING ELECTRONICS INFORMATION IND
View PDF4 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the existing data processing systems and methods are limited by the performance of n

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large-scale data processing system and method
  • Large-scale data processing system and method
  • Large-scale data processing system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to make the purpose, technical solution and advantages of the present invention more clear, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

[0021] The steps shown in the flowcharts of the figures may be performed in a computer system, such as a set of computer-executable instructions. Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0022] The large-scale data processing system involved in the embodiment of the present invention is applied to a mixed heterogeneous cluster of a central processing unit and a graphics processing unit, specifically, a computer cluster or a server cluster,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a large-scale data processing system and method, which are applied to a hybrid heterogeneous cluster of central processors and graphics processor. The large-scale data processing system comprises at least one shared storage node, a control node and multiple calculating nodes, the shared storage node, the control node and the multiple calculating nodes are connected via a high-speed network, the control node overall plans and allocates basic data blocks in a data packet to the calculating nodes, to ensure load balance of the calculating nodes, the calculating nodes directly read corresponding basic data blocks from the shared storage node for calculating and return computed results to the control node, the data transmission process is optimized, and the calculating time is shortened, so that the performance of the existing system is fully used, the efficiency of the entire system is greatly improved, such situations as insufficient network bandwidth, small internal memory capacity and the like are not limited, and the requirements of high-performance applications for processing large-scale data are satisfied.

Description

technical field [0001] The invention relates to a large-scale data processing technology, in particular to a large-scale data processing system and method. Background technique [0002] At present, in the era of data explosion, there are more and more information data, and people's requirements for information data processing capabilities are also getting higher and higher. Not only oil exploration, weather forecast, aerospace defense, scientific research, etc. require high-performance computing, but also financial , Government informatization, education, enterprises, online games, artificial intelligence and other wider fields have rapidly increased demand for high-performance computing, and computing speed is particularly important for high-performance computing. [0003] In the prior art, high-performance computing is developing toward multi-core and many-core, and a heterogeneous collaborative computing mode of a central processing unit (Central Processing Unit, referred...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06
Inventor 高永虎张清
Owner INSPUR BEIJING ELECTRONICS INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap