Computing system and computing method for big data processing

A technology of big data processing and computing system, applied in the fields of electrical digital data processing, special data processing applications, computing, etc.

Active Publication Date: 2014-01-01
INST OF INFORMATION ENG CAS
View PDF4 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is to provide a computing system and computing method for

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Computing system and computing method for big data processing
  • Computing system and computing method for big data processing
  • Computing system and computing method for big data processing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The principles and features of the present invention are described below in conjunction with the accompanying drawings, and the examples given are only used to explain the present invention, and are not intended to limit the scope of the present invention.

[0033] like figure 1 As shown, this embodiment provides a computing system for big data processing, which runs on multiple computing nodes, and includes three layers of modules from bottom to top, which are respectively the bottom layer module 1, the middle layer module 2 and the highest layer module. layer module 3 , and the middle layer module 2 further includes a message transmission module 21 and a calculation model module 22 .

[0034] The bottom module 1 adopts HDFS to store data, and the stored data includes input data set, intermediate result data set and output result data set.

[0035] The message transmission module 21, which is used to transfer messages between computing model modules running on differe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a computing system and computing method for big data processing. The computing system comprises a bottommost-layer module, a middle-layer module and a topmost-layer module sequentially from bottom to top. The middle-layer module comprises a message transmission module and computation model modules. A Hadoop distributed file system is adopted in the bottommost-layer module, and the bottommost-layer module is used for storing data. The message transmission module is used for transmitting messages between the computation model modules operating at different computing nodes. The computation model modules operating at the different computing nodes work cooperatively according to the messages transmitted by the message transmission module and establish specific types of computation models respectively to process the data. The topmost-layer module is used for providing programmatic interfaces for the computation models, combining computation expressed by the different computation models in a serial mode and enabling the different computation models to share the data based on an internal storage flow line mode through setting at the same time. According to the computing system and computing method for big data processing, application programs can be written in one system through the multiple computation models, and more complex problems can be solved.

Description

technical field [0001] The invention relates to the field of large number processing, in particular to a computing system and computing method based on hybrid programming. Background technique [0002] In recent years, with the rapid development of social informatization, data has shown explosive growth in scientific research, industrial production, commerce and the Internet. At present, the data in many applications has rapidly developed from TB (terabyte) level to PB (petabyte) level or even higher. Computational frameworks for big data processing have become a hot topic. At present, the open source Hadoop system has been widely used in the industry. Although the MapReduce model provided by Hadoop is simple and easy to use, its calculation model has limitations and limited expressive ability. When solving complex problems such as iterative calculation and graph analysis, it is difficult to map the algorithm to the MapReduce model, and the development workload is heavy. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/182
Inventor 王鹏韩冀中王伟平孟丹张云
Owner INST OF INFORMATION ENG CAS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap