A verification method for converting field types between data sources

A field type and verification method technology, which is applied in the verification field of converting field types between data sources, can solve problems such as data conversion not reaching the expected goal, garbled characters, and type mismatch, so as to avoid data transmission failure, accurate conversion, and The effect of guaranteeing efficiency

Active Publication Date: 2022-07-05
浪潮卓数大数据产业发展有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the pipeline design interface, during the conversion process of different data sources, due to the different field types, the type mismatch may result in abnormal data, garbled characters, etc. during the transmission process, resulting in the data conversion not reaching the expected goal

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A verification method for converting field types between data sources

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] In order to make those skilled in the art better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0042] The source library of the data source is processed by a converter (field replacement, interception, etc.) or a filter (Like, equal, etc.), if the number of source libraries is greater than 1, it will go through the aggregator and then arrive at the mapping If the types of field mappings do not match each other, it may cause data transmission failure, so the verification fu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention particularly relates to a verification method for converting field types between data sources. In the verification method for converting field types between data sources, the mapper maps the fields in the data source source library and the data source target library to each other, and verifies the mapping results; if the data source source library and the data source target library The database belongs to the same data source, and the mapping result should be the field type corresponding to the field type; generate a special database to store the verification of the detailed conversion, and generate a verification table according to the verification result, so as to make the conversion more efficient. Accurate, convenient, and minimize the generation of abnormal data. The verification method for converting field types between data sources can avoid data transmission failure caused by mismatching field mapping types between the data source source library and the data source target library, making the conversion more accurate and convenient, and reducing abnormal data to the greatest extent possible. The generation of large amounts of data ensures the efficiency of rapid migration of large amounts of data.

Description

technical field [0001] The invention relates to the technical field of data aggregation, in particular to a verification method for converting field types between data sources. Background technique [0002] Data aggregation, or ETL, loads data from different business systems into a data warehouse. There are many ways of data aggregation. According to the transmission mode of data aggregation, it can be divided into file transmission, data extraction, content crawling, and message push. [0003] The data aggregation system is a software formed by the productized packaging of distributed data pipeline tools, mainly including cluster monitoring, pipeline design, pipeline management, task monitoring and other functions. In the pipeline design interface, in the process of conversion of different data sources, due to the different field types, there may be abnormal data and garbled characters generated due to type mismatches during the transmission process, resulting in data conv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/242G06F16/25G06F16/28
CPCG06F16/254G06F16/2433G06F16/284
Inventor 徐宏伟王帅
Owner 浪潮卓数大数据产业发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap