Memory computing-based customizable multimode big data processing system

A big data processing and memory computing technology, applied in the system field of big data technology field, can solve the problems of unable to support massive data processing, difficult to support batch processing, graph processing, and single processing mode, so as to improve data processing efficiency , Efficient and fast iterative analysis capabilities, and the effect of improving support performance

Inactive Publication Date: 2016-10-12
NO 32 RES INST OF CHINA ELECTRONICS TECH GRP
View PDF5 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, it only supports real-time processing of data, and the supported processing mode is single, which makes it difficult to support processing requirements such as batch processing and graph processing. In addition, the size of the data it processes is limited to the cluster memory capacity, and it cannot support massive data whose data volume exceeds the memory range. deal with

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Memory computing-based customizable multimode big data processing system
  • Memory computing-based customizable multimode big data processing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] The present invention will be described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0037] The present invention provides a customizable multi-mode big data processing system based on memory computing. The present invention provides multi-mode synthesis such as batch processing, real-time data flow calculation, real-time interactive query analysis, efficient and fast iterative analysis capability graph calculation, and complex logic processing. It has the advantages of large-scale data processing programming support capabilities, fine-grained job scheduling capabilities, flexible custom...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a memory computing-based customizable multimode big data processing system. The system comprises a data storage layer module, a memory-based data sharing and management layer module, a memory computing-based universal execution framework layer module and an access interface layer module, wherein a distributed memory abstraction mechanism, a position-sensing scheduling mechanism and a distributed hybrid column storage mechanism of data are adopted. According to the system, massive data storage is constructed; a cluster concurrent computing-oriented memory data management and sharing framework is provided, and an efficient and customized big data multimode universal processing framework is provided; batch processing and real-time data stream computing are supported; and the support is provided for flexible analysis and deep utilization of the data.

Description

technical field [0001] The invention relates to a system in the field of big data technology (basic platform), in particular to a customizable multi-mode big data processing system based on memory computing. Background technique [0002] Big data processing technology is the realization of the calculation mode of massive data processing, based on big data storage and management technology, and is the core support of big data analysis, mining and application. With the diversified development of massive data processing requirements, big data processing technology is required to be able to provide data processing functions of various processing modes such as batch processing, stream processing, and graph processing under a unified general processing framework, and to have comprehensive data processing functions; at the same time, the big data processing requirements of various industries and fields are developing towards fine-grained and customizable directions. On the one hand...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/2219G06F16/2237G06F16/25
Inventor 冯伟王金华
Owner NO 32 RES INST OF CHINA ELECTRONICS TECH GRP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap