Automatic data processing method and system, computer equipment and readable storage medium

A processing method and data technology, applied in the field of data processing, can solve the problems of missing dictionaries of manufacturers, high error rate and high error rate, and achieve the effect of reducing the workload of manual verification and quick search.

Active Publication Date: 2020-04-21
SANMENG TECH CO LTD
View PDF8 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, in the current early stage of data governance, the standardized mapping of data dictionaries of various types of data is mostly through national/industry standards, and the metadata is manually mapped to the corresponding standard dictionary one by one, so as to realize the link between metadata and standard data. conversion, which consumes a lot of manpower and has a high error rate
On the other hand, each implementation is based on the original data dictionar

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic data processing method and system, computer equipment and readable storage medium
  • Automatic data processing method and system, computer equipment and readable storage medium
  • Automatic data processing method and system, computer equipment and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0026] see figure 1 , figure 1 The flow chart of an embodiment of the automatic data processing method of the present invention is shown, which includes:

[0027] S101. Scan each data table in the metadata separately, and extract field information of each field in each data table to generate a data dictionary in an intermediate process.

[0028] The field information includes the role of the field in the data table, data quality, field name translation, field content specific field identification, field data type and other information, but is not limited thereto. Among them, the field name translation is mainly based on pinyin and English translation.

[0029] Therefore, the present invention obtains the field information of each field in each data table by sca...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an automatic data processing method, which comprises the following steps of: respectively scanning each data table in metadata, and extracting field information of each field in each data table to generate a data dictionary of an intermediate process; adjusting the data dictionary, and carrying out association between tables to extract a repetition relationship and a mapping relationship; and constructing a data view according to the repetitive relationship and the mapping relationship, updating the data dictionary according to translation information of each field in the data view, and performing anomaly detection on metadata. The invention further discloses an automatic data processing system, computer equipment and a computer readable storage medium. By the adoption of the method and device, automatic data processing is achieved, and standard unified management and effective execution are truly achieved by effectively carding the data.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a data automatic processing method, a data automatic processing system, computer equipment and a computer-readable storage medium. Background technique [0002] With the continuous maturity and development of Internet technology and big data technology, the status of campus information sharing is getting higher and higher in today's era. In the context of the development of the times, it is particularly important to improve data quality. The improvement of data processing capabilities and the mining of data value are the requirements of the times, and the standards need to be gradually improved. At the same time, the actual level of data sharing is also related to the ability of data processing, and the security and privacy issues involved in the improvement of the level of data sharing are gradually being exposed. [0003] However, in the current early stage of ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/22G06F16/26G06F16/28G06F16/36
CPCG06F16/2282G06F16/26G06F16/285G06F16/36
Inventor 柯越美曾琼海黄梦韩罗龙韩高强王晋民
Owner SANMENG TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap