Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method for checking conversion field types between data sources

A technology of field type and verification method, which is applied in the verification field of converting field types between data sources, and can solve problems such as data conversion not reaching the expected goal, type mismatch, garbled characters, etc.

Active Publication Date: 2020-10-30
浪潮卓数大数据产业发展有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the pipeline design interface, during the conversion process of different data sources, due to the different field types, the type mismatch may result in abnormal data, garbled characters, etc. during the transmission process, resulting in the data conversion not reaching the expected goal

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for checking conversion field types between data sources

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0042] The source library of the data source passes through the converter (replacing fields, intercepting, etc.), or through the filter (Like, equal, etc.), if the number of source libraries is greater than 1, it passes through the aggregator, and then reaches the mapping If the types of field mapping do not match each other, data transmission may fai...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention particularly relates to a method for checking conversion field types between data sources. According to the invention, the method comprises the steps: enabling the fields in a data source library and fields in a data source target library to be mutually mapped in a mapper, and verifying a mapping result; if the data source library and the data source target library belong to the samedata source, determining which field type the mapping result should correspond to; generating a special database to store the detailed conversion verification, and generating a verification table according to the verification result, so that the conversion is more accurate and convenient, and the generation of abnormal data is reduced to the greatest extent. According to the verification method for the conversion field type between the data sources, the data transmission failure caused by mismatching of field mapping types between the data source library and the data source target library canbe avoided, the conversion is more accurate and convenient, the generation of abnormal data is reduced as much as possible, and the efficiency of rapid migration of a large amount of data is guaranteed.

Description

technical field [0001] The invention relates to the technical field of data aggregation, in particular to a verification method for converting field types between data sources. Background technique [0002] Data aggregation, or ETL, loads data from different business systems into the data warehouse. There are many methods of data aggregation. According to the transmission method of data aggregation, it can be divided into file transfer, data extraction, content crawling, and message push. [0003] The data aggregation system is the software formed by the product packaging of distributed data pipeline tools, mainly including cluster monitoring, pipeline design, pipeline management, task monitoring and other functions. In the pipeline design interface, during the conversion process of different data sources, due to the different field types, the type mismatch may result in abnormal data, garbled characters, etc. during the transmission process, resulting in the data conversio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/242G06F16/25G06F16/28
CPCG06F16/254G06F16/2433G06F16/284
Inventor 王帅徐宏伟
Owner 浪潮卓数大数据产业发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com