Data extracting and loading method based on mass data migration

A technology of data extraction and massive data, which is applied in electronic digital data processing, structured data retrieval, special data processing applications, etc. It can solve the problems of long data loading time, data damage or loss, etc. Influence and improve the effect of migration efficiency

Active Publication Date: 2017-05-31
INSPUR SOFTWARE TECH CO LTD
View PDF18 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] 3. When large data is migrated, data loading takes too long, which may easily cause data damage or loss

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data extracting and loading method based on mass data migration
  • Data extracting and loading method based on mass data migration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0027] as attached figure 1 , figure 2 As shown, a data extraction and loading method based on massive data migration is applied to massive data migration to ensure the high-speed processing capability of the entire migration process. The invention separates the external table from the data source by creating a temporary read-only user during massive data migration, partitions the massive data in parallel to generate multiple DUMP files, analyzes and processes the data, converts and filters abnormal data, and finally adopts Import data in batches to migrate data to the target data source. The present invention can quickly extract and load massive data from various data sources, and has very little impact on the data source system, ensures the accuracy of data, and minimizes the impact on the business system.

[0028] This patent involves ma...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data extracting and loading method based on mass data migration. The realizing process of the method comprises the steps of extracting data firstly, leading an external table and a data source to be separated via constructing a casual user, and carrying out partitioning and parallel segmentation on the mass data to generate a plurality of dump files; then performing data loading, namely carrying out analysis and processing on the data, transforming and filtering abnormal data, and finally adopting a bulk import data mode to migrate the data to a target data source. Compared with the prior art, the data extracting and loading method based on the mass data migration can quickly extract and load the mass data from various data sources, has very little influence on the system of a data source, guarantees the accuracy of the data, reduces the influence on an operation system to the maximum extent, and is strong in practicability and wide in the range of application, thereby having very good popularization and application values.

Description

technical field [0001] The invention relates to the technical field of data migration, in particular to a data extraction and loading method based on mass data migration. Background technique [0002] When migrating massive amounts of data in a database, it generally has the following requirements: [0003] 1. Heterogeneous database version upgrade: such as upgrading from Oralce 8i to 9i, or even Oracle10g, etc.; [0004] 2. Operating system upgrade: such as AIX 4.3 upgrade to 5.2, even HP Unix environment, etc.; [0005] 3. Disk array expansion: upgrade from old disks to new ones, high-end disks from other manufacturers, and so on. [0006] However, in the actual migration process, there are also the following problems: [0007] 1. During big data migration, data extraction and loading consume a lot of physical resources and the time-consuming cannot be guaranteed. Once an error occurs in the loading, it needs to be restarted, which is laborious and time-consuming. [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/254G06F16/27
Inventor 董志鹏徐兵兵丁庆申郭凤
Owner INSPUR SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap