Method for checking conversion field types between data sources

A technology of field type and verification method, which is applied in the verification field of converting field types between data sources, and can solve problems such as data conversion not reaching the expected goal, type mismatch, garbled characters, etc.

Active Publication Date: 2020-10-30
浪潮卓数大数据产业发展有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the pipeline design interface, during the conversion process of different data sources, due to the different field types, the type mismatch may

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for checking conversion field types between data sources

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0041] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0042] The source database of the data source is processed by a converter (replacement of fields, interception, etc.), or after processing by filters (Like, equal, etc.), if the number of source databases is greater than one, it will pass through the aggregator and then arrive at the mapping If the type of field mapping does not match each other, it may...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention particularly relates to a method for checking conversion field types between data sources. According to the invention, the method comprises the steps: enabling the fields in a data source library and fields in a data source target library to be mutually mapped in a mapper, and verifying a mapping result; if the data source library and the data source target library belong to the samedata source, determining which field type the mapping result should correspond to; generating a special database to store the detailed conversion verification, and generating a verification table according to the verification result, so that the conversion is more accurate and convenient, and the generation of abnormal data is reduced to the greatest extent. According to the verification method for the conversion field type between the data sources, the data transmission failure caused by mismatching of field mapping types between the data source library and the data source target library canbe avoided, the conversion is more accurate and convenient, the generation of abnormal data is reduced as much as possible, and the efficiency of rapid migration of a large amount of data is guaranteed.

Description

technical field [0001] The invention relates to the technical field of data aggregation, in particular to a verification method for converting field types between data sources. Background technique [0002] Data aggregation, or ETL, loads data from different business systems into the data warehouse. There are many methods of data aggregation. According to the transmission method of data aggregation, it can be divided into file transfer, data extraction, content crawling, and message push. [0003] The data aggregation system is the software formed by the product packaging of distributed data pipeline tools, mainly including cluster monitoring, pipeline design, pipeline management, task monitoring and other functions. In the pipeline design interface, during the conversion process of different data sources, due to the different field types, the type mismatch may result in abnormal data, garbled characters, etc. during the transmission process, resulting in the data conversio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/242G06F16/25G06F16/28
CPCG06F16/254G06F16/2433G06F16/284
Inventor 王帅徐宏伟
Owner 浪潮卓数大数据产业发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap