Universal big data model configuration and analysis method

An analysis method and big data technology, applied in the field of big data analysis, can solve problems such as timeliness requirements, inability to notify relevant personnel in time, and lack of analysis model sharing function.

Pending Publication Date: 2020-06-26
CHINA NAT OFFSHORE OIL CORP +1
View PDF5 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] First, it does not have integrated configuration, analysis and display functions
[0005] Second, there is no early warning push function
For some big data analysis results that require timeliness and need to be processed immediately, relevant personnel cannot be notified in time
[0006] Third, there is no analysis model sharing function

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Universal big data model configuration and analysis method
  • Universal big data model configuration and analysis method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0032] A general big data model configuration and analysis method, big data model configuration and analysis process as attached figure 1 shown.

[0033] 1. Dataset construction

[0034] For enterprise data (structured and unstructured), construct enterprise data sets, establish business data sets according to business classification, unify master data standards, apply unified data sources and data standards for data, and train and verify big data models , Select the corresponding data through the data set in the application analysis.

[0035] 2. Algorithm library construction

[0036] 1. Algorithm library establishment method

[0037] Develop a unified processing class for commonly used big data algorithms such as linear regression, logistic regression, random forest regression, random forest classification, clustering algorithm, decision tree classification, decision tree regression, neural network, etc., and specify the calling parameters of each method And instructions...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a universal big data model configuration and analysis method. According to the method, the modes of data set construction, algorithm library construction, model template configuration, background scheduling, early warning pushing and the like are adopted, data sets, cleaning rules, algorithm libraries (algorithms and parameters), potential factors (target factors), algorithm models and the like are managed in a unified mode, and tasks are executed automatically (or regularly) by configuring a scheduling execution scheme for a training / prediction model. The scheduling center is responsible for executing a big data analysis task and carrying out big data analysis processing on historical / real-time data. The early warning pushing center pushes early warning information, and the configuration center is visualized, so that visualized display of big data analysis results is realized. The model management method realizes bidirectional sharing of the analysis model, exports the analysis model, and provides a model interface specification for an external system to use; an analysis model of an external system is imported and configured to realize the application of anexternal model.

Description

technical field [0001] The invention is applied to big data analysis and is a general big data model configuration and analysis method. Background technique [0002] In conventional big data analysis, the usual analysis process is for specific applications, preparing data, manual data cleaning, writing and calling corresponding algorithm codes, parameter selection, training, etc. In big data analysis, the functions of each step must be redone every time the work is done, starting from scratch. In this way, the workload of data preparation and data cleaning is heavy, and the code development is highly repetitive, resulting in a large amount of repetitive development work, long development cycle, high work cost, and low work efficiency. In addition, it is difficult for business personnel to carry out big data analysis without the cooperation of software developers. [0003] Some existing methods have the following functional defects: [0004] First, it does not have integra...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/901G06F16/904
CPCG06F16/901G06F16/904
Inventor 李明江万欢刘敏辛国安郑毅黄小非刘欢杜宏伟闫宾李新建
Owner CHINA NAT OFFSHORE OIL CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap