Data import method and device

A data import and data technology, applied in the computer field, can solve the problems of real-time acquisition and utilization of various types of data

Active Publication Date: 2021-04-27
绿湾网络科技有限公司
View PDF11 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In an environment where a large amount of Internet data is generated every day, it is difficult to achieve real-time acquisition and utilization of various types of data by relying on manual configuration, and its efficiency is unacceptable

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data import method and device
  • Data import method and device
  • Data import method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0035] The method and device for importing data in the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0036] figure 1 It is a schematic flowchart of a data import method according to an embodiment of the present invention.

[0037] Such as figure 1 As shown, the data import method includes the following steps:

[0038] S101. Obtain a source data set to be imported into a database.

[0039]As an exemplary implementation manner, the source data set to be im...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention proposes a data import method and device, wherein the method includes: acquiring a source data set to be imported into a database, wherein the source data set includes multiple source data tables from multiple data sources; Identify the data structure of the table, and cluster multiple source data tables according to the identification results to obtain multiple sets of isomorphic data; obtain the cleaning and conversion rules corresponding to each set of isomorphic data; Cleaning and conversion rules process each group of isomorphic data, and import each group of processed data into the database. The method provided by the embodiment of the present invention reduces manual repeated operations, realizes the simultaneous import of multiple source data tables from multiple data sources, and improves the import efficiency of source data tables from multiple data sources.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data import method and device. Background technique [0002] In the construction of a big data platform, one of the most tedious tasks is to extract, clean and transform various heterogeneous data from various data sources. Among them, data sources may come from different organizations, different platforms, and different operating systems. This diversity has caused a large number of non-standard "dirty data". The main reasons for the non-standard are: 1) Data storage of different data sources There are different methods, including data in relational database systems (such as Mysql, oracle, etc.), unstructured file data (such as Excel, text files, etc.) and semi-structured data (such as data captured from web pages, etc.) , even if it is the same type of storage, the data type structure may also be different. 2) Different organizations adopt different data standards, and the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/215G06F16/25
CPCG06F16/215G06F16/254
Inventor 赵云志秦锋剑
Owner 绿湾网络科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap