A cross-platform multi-engine supported mass data migration device

A massive data, multi-engine technology, applied in the direction of structured data retrieval, database design/maintenance, etc., can solve the problems of operation, inability to correlate and process data at the same time, cost increase, etc.

Pending Publication Date: 2019-04-02
南京数睿数据科技有限公司
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the context of big data with extremely large data volume, these technologies need to be modified according to the requirements of data scale, data storage physical medium, and data processing complexity. running in a computing environment, and may not be able to correlate and process data distributed on different storage media at the same time
Because the computing engine of traditional data transplantation device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A cross-platform multi-engine supported mass data migration device
  • A cross-platform multi-engine supported mass data migration device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0015] Today's traditional data migration technology cannot adapt to new data characteristics without coding modification when the data volume changes and the data timeliness changes. When the data volume becomes larger, it cannot automatically switch the centralized computing engine to The distributed computing engine cannot be processed or processed under the premise of low latency. When the data changes from offline to real-time, the offline processing engine cannot be automatically switched to a real-time stream processing engine, resulting in the inability to process real-time data in a timely manner. The present invention aims to provide a cross-platform massive data migration device that supports multiple computing engines, and provides a general-purpose non-customized data migration device for cross-platform heterogeneous data migration and adaptation of multiple data processing engines.

[0016] The present invention provides a cross-platform multi-engine massive data ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cross-platform multi-engine supported mass data migration device. The system comprises an original data storage system, a migration data storage system, a migration flow configuration module, a flow engine execution module and a flow engine scheduling module, wherein the original data storage system is used for storing an original data set to be migrated, and the migration data storage system is used for storing the migrated data set; the migration process configuration module is used for visually configuring the migrated business process logic; the process engine execution module is used for executing an actual migration process; and the flow engine scheduling module is used for regularly scheduling the migrated business flow. The device disclosed by the invention ensures that the data can be migrated by using an engine technology which is most suitable for data characteristics; the processing time delay is reduced to the maximum extent, the processing efficiency is improved, the migration cost is reduced, and the calculation engine can be quickly switched when the data timeliness is changed and the data volume is changed so as to adapt to new data characteristics without the secondary coding development.

Description

technical field [0001] The invention belongs to the technical field of data migration, and specifically relates to a method and system for massive data migration that can run across platforms and support multiple engines. Background technique [0002] The data transfer device refers to the logical and physical aggregation of data from different systems for centralized management, so that users can research and analyze the data and draw valuable conclusions. [0003] The volume of data and the type of data have both increased significantly in the past three decades. Data warehouse technology has grown from scratch. Whether it is the data warehouse methodology proposed by Kimball or Inmon, in order to meet the needs of data storage and data analysis, it needs a A kind of data migration device, this device has also revealed many problems and defectives that need to be improved urgently in the development process of many years. [0004] Traditional data migration devices mainly...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/21
Inventor 王任康俞亚君刘晓杰李鸿飞
Owner 南京数睿数据科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap