Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A design method of etl framework for safety monitoring business analysis

A business analysis and security-oriented technology, applied in the field of data extraction-transformation-loading framework design, can solve problems such as concurrency limitation and scalability

Active Publication Date: 2019-06-28
HOHAI UNIV +1
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Purpose of the invention: In order to solve the problems of concurrency limitation and scalability when the traditional ETL framework processes this type of business data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A design method of etl framework for safety monitoring business analysis
  • A design method of etl framework for safety monitoring business analysis
  • A design method of etl framework for safety monitoring business analysis

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0014] Below in conjunction with specific embodiment, further illustrate the present invention.

[0015] figure 1 Design a method structure diagram for the ETL framework oriented to security monitoring business analysis. The ETL framework design method for safety monitoring business analysis includes data extraction module, data conversion module and data loading module.

[0016] The data extraction module refers to the process of obtaining source data from multiple heterogeneous data sources, by using the XML adapter to confirm the data source and using the incremental extraction technology engine for data extraction.

[0017] The data conversion module is the link connecting the data extraction process and the data loading process. In this process, the two components of "data processing engine" and "Redis storage engine" are used for data conversion.

[0018] Data loading module, the loading of data mainly includes two loading types: initial loading and incremental loading...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an ETL (Extract Transform and Load) framework design method for safety monitoring business analysis, which comprises data extraction, data transformation and data loading. The data extraction is completed by the steps of obtaining source data from a plurality of heterogeneous data sources, affirming the data sources through utilizing an XML (Extensible Makeup Language) adapter and utilizing a timestamp to carry out incremental extraction; the data transformation process is a link for connecting the data extraction process and the data loading process, and data transformation is carried out in the process by utilizing two components, namely a data processing engine and an Redis storage engine; during data loading, data loading mainly includes two loading types, namely initial loading and incremental loading, the initial loading mainly utilizes an incremental loading engine to process, and the incremental loading utilizes a way of combining a batch loading engine and a real-time loading engine to process. According to the method, high unity of heterogeneous data is implemented for laying a foundation for the subsequent decision support work.

Description

technical field [0001] The invention relates to an ETL framework design method for safety monitoring business analysis, in particular to a design of a data extraction-transformation-loading framework for safety monitoring business analysis in the field of dam engineering safety monitoring, to realize heterogeneous data for safety monitoring high degree of unity. Background technique [0002] Dams are an important infrastructure for national economic construction and social development in our country. Improving the safety awareness and management level of dams is of great significance to play the role of dams. In order to ensure the safety of the dam during the preparation, construction and operation stages, the general dam safety monitoring system is equipped with a dam strong earthquake monitoring system and a reservoir water regime monitoring system according to the different dam types and geological conditions. , stress and strain monitoring system and special monitoring...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/25G06F16/21
CPCG06F16/21G06F16/254
Inventor 毛莺池贾必聪艾永平易魁许峰肖海斌周晓峰王龙宝邱小弟卢吉陈豪李然周健李洪波张鹏
Owner HOHAI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com