Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Big data storage management system considering data migration efficiency

A big data storage and management system technology, applied in the field of big data storage management, can solve problems such as ignoring the load balance of migration tasks between clusters and reducing the efficiency of big data migration, so as to achieve the effect of improving migration efficiency and increasing parallelism

Inactive Publication Date: 2019-07-09
广州明领基因科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Most of the current data migration technologies only consider the unilateral balance of the cluster, but ignore the load balance of migration jobs between clusters, which will also reduce the efficiency of big data migration

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data storage management system considering data migration efficiency
  • Big data storage management system considering data migration efficiency
  • Big data storage management system considering data migration efficiency

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0011] The present invention will be described in more detail and complete below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

[0012] refer to figure 1 , a big data storage management system considering data migration efficiency in the present invention, the system includes: a data source layer, a job scheduling layer, and a migration target layer; wherein, the data source layer horizontally expands different types of data sources into distributed The job scheduling layer uses the MapReduce framework as the basis of the distributed program, and uses YARM to precisely control the resources used for data migration; the migration target layer uses a distributed file storage system to virtualize the target cluster into an overall storage system.

[0013] The design basis of the present invention is...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a big data storage management system considering data migration efficiency. The system comprises a data source layer, an operation scheduling layer and a migration target layer, wherein the data source layer is used for providing operation data for the operation scheduling layer; and the operation scheduling layer is responsible for completing data migration according to requirements and migrating the data to the migration target layer.

Description

technical field [0001] The invention belongs to the technical field of big data storage management and relates to a big data storage management system considering data migration efficiency. Background technique [0002] Massive data analysis often relies on a distributed environment. However, due to the different types of business data, data migration and summary will be the primary task. Whether the source data can be efficiently and stably migrated to the target storage system largely determines the efficiency of data analysis. . [0003] The data migration technology for distributed storage mainly solves the problem of data migration between storage systems or between different instances of the same storage system: the migration between systems focuses on factors such as data storage format, transmission path, and network conditions; Migration between storage systems focuses on a series of factors such as the storage form and interface performance of the storage system. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06
CPCG06F3/0614G06F3/0638G06F3/0647G06F3/0664G06F3/067
Inventor 祁建明周峻松徐继峰陈墩金
Owner 广州明领基因科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com