Marking and explaining system and method for large-data analysis model

Abstract

The invention discloses a marking and explaining system and method for a large-data analysis model. The marking and explaining system comprises a model marking unit, a model managing unit, a model explaining unit, a model compiling unit and a user unit. The model marking unit is used for marking model metadata; the model managing unit is used for examining and approving the model, metadata managing and indexing; the model explaining unit is used for analyzing user operation and DAG relationship diagram converting; the model compiling unit is used for building the model dependency relationship, downloading a dependent library and jointly compiling; the user unit is used for identifying user identification and design achieving, managing and submitting of a user to the model. By means of the marking and explaining system and method, marking and explaining of the model in the large-data analysis process are achieved, operation when the user creates operation is convenient, and the calling process and the explaining process of the model are quickened.

Description

technical field [0001] The invention relates to a system and method for marking and explaining big data analysis models. Background technique [0002] In recent years, with the continuous development of big data processing technology, various big data analysis platforms have emerged, and they have gradually begun to support the processing of various types of big data tasks. In their model libraries Various types of function models such as Mahout, Pig, Crunch, and DataFu are integrated, and users can construct big data processing tasks through various models in the model library, and create and register scheduling transactions. However, the configuration files of these jobs need to be written by the users themselves, and the compilation and analysis of the files also need to be submitted to the remote server, which often takes a lot of time. There is a lack of a visual and local platform, and the job design and analysis can be easily carried out by dragging and dropping. The...

AI Technical Summary

Problems solved by technology

This invention makes the workflow engine system more scalable in a distributed environment, but saves the entire job process as a template, which lacks flexibility

[0005] The job parsing function implemented by existing big data model patents or tagging and explaining methods mostly relies on stored job templates and preset XML definition files in terms of user job interpretation, especially in terms of job templates, which are simply based on The current job calls the job template without considering the processing when the user submits a job different from the existing template

Images