Big data process modeling analysis engine

A technology of process modeling and data analysis, applied in digital data processing, structured data retrieval, database management systems, etc., can solve the problems of narrow user groups, cumbersome front-end page display, difficulty in meeting application requirements, etc., and achieve efficient integration The effect of the process

Active Publication Date: 2016-05-04
JIANGSU DAWN INFORMATION TECH CO LTD
View PDF3 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] The big data analysis tool software in the prior art is basically based on localized processing, storing the data in the local hard disk, and using a relational database to process and analyze a small amount of data, but for massive data, it cannot be effectively processed and analyzed
[0003] Moreover, the front-end page display of the data analysis tools in the prior art is too cumbersome and requires professional data administrators to operate, while ordinary users are beyond the reach, so the target user groups are relatively narrow. In addition, the current data processing and analysis uses relational data. The efficiency of massive data processing is low, and it is difficult to meet the application requirements

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data process modeling analysis engine
  • Big data process modeling analysis engine
  • Big data process modeling analysis engine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0030] Such as figure 1 As shown, a big data process modeling analysis engine, including interface layer, task scheduling layer, platform layer;

[0031] Platform layer: mainly based on the Hadoop platform, the present invention mainly uses yarn, spark and hdfs in the Hadoop platform as the basic function support of the invention; the operation of each data analysis process task based on the present invention requires the cooperation of the three to complete the resources Scheduling, allocation and other work, yarn is a general resource management system that can provide unified resource management and scheduling, spark is a general parallel computing framework, hdfs is a highly fault-tolerant system; hdfs is very suitable for large-scale data sets The application on the network realizes the purpose of reading file system data in a streaming m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a big data process modeling analysis engine. The big data process modeling analysis engine comprises an interface layer, an application logic layer, a data analysis algorithm layer and a platform layer, wherein the interface layer is used to carry out data analysis modeling operations during analytical processing of massive data, so as to produce a data analysis model; a task scheduling layer is used to analyze the data analysis model and retrieve a corresponding algorithm package to establish an executable data analysis task; the platform layer is used to calculate and store resources, so as to finally execute the task and obtain a result. The big data process modeling analysis engine provided by the invention has the advantages that based on the Spark design concept, data analysis operation steps and processes of a user are analyzed after the user carries out processed data modeling analysis operations, Spark is then called through processes, and the result is finally output after memory processing of all the data analysis operation steps of the user, so that efficient integrated processes are achieved.

Description

technical field [0001] The invention belongs to the field of fast analysis of big data, and in particular relates to a big data process modeling analysis engine. Background technique [0002] The big data analysis tool software in the prior art is basically based on localized processing, storing the data in the local hard disk, and using a relational database to process and analyze a small amount of data, but for massive data, it cannot be effectively processed and analyzed . [0003] Moreover, the front-end page display of the data analysis tools in the prior art is too cumbersome and requires professional data administrators to operate, while ordinary users are beyond the reach, so the target user groups are relatively narrow. In addition, the current data processing and analysis uses relational data. The efficiency of massive data processing is low, and it is difficult to meet the application requirements. Contents of the invention [0004] Purpose of the invention: A...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/25G06F16/254
Inventor 朱亚洲朱俊秀丁章明陈骥洲
Owner JIANGSU DAWN INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap