Big data conversion method and system based on adapter

A transformation method and big data technology, applied in the field of data processing, can solve the problems of inability to perform seamless exchange and the inability to exert data application value, and achieve the effect of reducing the cost of data collection and analysis, reducing logic errors, and improving exchange efficiency

Inactive Publication Date: 2017-11-03
上海赢华软件科技有限公司
View PDF0 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are so many heterogeneous data that cannot be seamlessly exchanged with each other, forming data islands, and unable to exert the application value of data
Now, the challenge we face is how to deal with massive amounts of unstructured and uncertain data and extract business value from these data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data conversion method and system based on adapter

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] In the current big data, there have been various data formats for a long time, and data exchange cannot be performed. However, there are problems of high cost and low efficiency for data maintenance and upgrading.

[0045]Scenario 1: After years of accumulation, commercial banks have accumulated a lot of data, including business data, customer data, risk data, performance data, accounting data, etc. The formats of these data are various, including Flat files and transactions from the core system Data files, batch files, CSV files, pdf files, etc., and some image files include both structured data and unstructured data. Data cannot be exchanged, forming data barriers and affecting work efficiency. Therefore, there is an urgent need for a technology or tool to enable batch conversion of various heterogeneous data, so as to achieve the goal of seamless data exchange and data sharing.

[0046] Scenario 2: The value potential of industrial big data applications is huge. How...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a big data conversion method and system based on an adapter. The big data conversion method based on the adapter comprises the following steps that: receiving a heterogeneous data conversion request, carrying out analysis processing on the heterogeneous data, and configuring the heterogeneous data on the basis of an SQL (Structured Query Language) script; carrying out cache processing on the heterogeneous data subjected to the cache processing, and carrying out operation stage configuration on the configured heterogeneous data; carrying out conversion processing on the heterogeneous data subjected to the cache processing, storing the converted heterogeneous data, and carrying out operation stage operation on the heterogeneous data subjected to the operation stage configuration; and displaying the stored heterogeneous data. By use of the method, a batch data conversion script set is automatically executed, conversion script execution result information is stored, and then, an application analysis report is generated. When the method is adopted, logic errors generated when the heterogeneous data is converted can be reduced, conversion efficiency among the heterogeneous data is improved, and data collection and analysis cost of the heterogeneous data can be lowered.

Description

technical field [0001] The invention relates to the field of data processing, in particular to an adapter-based big data conversion method and system. Background technique [0002] In recent years, with the rapid development and popularization of computer and information technology, the scale of industrial application systems has expanded rapidly, and the data generated by industrial applications has grown explosively. Industry / enterprise big data that can easily reach hundreds of terabytes or even dozens to hundreds of PBs has far exceeded the processing capabilities of existing traditional computing technologies and information systems. Therefore, it is necessary to seek effective big data processing technologies, methods and means It has become an urgent need in the real world. The current total data volume of Baidu has exceeded 1000PB, and the webpage data that needs to be processed every day reaches 10PB~100PB; the cumulative transaction data volume of Taobao is as hig...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/904
Inventor 朱宇尧
Owner 上海赢华软件科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap