Data migration method and device, equipment and medium

A technology of data and medium, applied in the field of data processing

Pending Publication Date: 2019-11-05
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF0 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But its enthusiasm for data only con

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data migration method and device, equipment and medium
  • Data migration method and device, equipment and medium
  • Data migration method and device, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] figure 1 It is a flowchart of a data migration method provided by Embodiment 1 of the present invention. This embodiment is applicable to the situation of accurately migrating and storing data. The method can be executed by a data migration device, and the device can be realized by software and / or hardware. see figure 1 , the data migration method provided in this embodiment includes:

[0034] S110. Determine a candidate storage medium from candidate medium categories corresponding to the medium category to which the current storage medium belongs according to the medium category to which the current storage medium of the data to be migrated belongs.

[0035] Specifically, if the current storage medium is a memory, the medium category to which the current storage medium belongs is the fast-read medium category; if the current storage medium is a disk, the medium category to which the current storage medium belongs is the delayed-read medium category.

[0036] Wherei...

Embodiment 2

[0054] figure 2 It is a flowchart of a data migration method provided in Embodiment 2 of the present invention. This embodiment is an optional solution proposed on the basis of the foregoing embodiments. see figure 2 , the data migration method provided in this embodiment includes:

[0055] S210. Determine a candidate storage medium from candidate medium categories corresponding to the medium category to which the current storage medium belongs according to the medium category to which the current storage medium of the data to be migrated belongs.

[0056] S220. Determine the migration benefit according to the change value of the migration storage cost from the current storage medium to the candidate storage medium, the data size of the data to be migrated, and the access mode of the data to be migrated.

[0057] Wherein, the migration storage cost change value from the current storage medium to the candidate storage medium refers to the remaining value of the storage cos...

Embodiment 3

[0079] image 3 It is a schematic diagram of execution of a data migration method provided in Embodiment 3 of the present invention. This embodiment is an optional solution proposed on the basis of the foregoing embodiments. see image 3 , the data migration method provided in this embodiment includes:

[0080] (1) Judgment of hot and cold data.

[0081] Through the analysis system other than the independent and stand-alone IO (reading and writing) engine, extract the IO information in units of blocks (data blocks), including: block size, access frequency, and access mode. Migration loss is mainly the increase of access delay. The benefits of migration come from the reduction of storage costs. Calculate the migration loss and migration gain by combining the access information of the block to reasonably judge whether the data needs to be migrated.

[0082] needMigrate=gain(size, accessFreq, accessPattern, storeCost)–lost(size, accessFreq, accessPattern, transCost)>thresho...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a data migration method and device, equipment and a medium, and relates to data processing technology. The method comprises the steps of determining a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of to-be-migrated data belongs according to the medium category to which the current storage medium belongs; respectively calculating migration income and migration loss of migration from the current storage medium to the candidate storage medium wherein the migration income andthe migration loss are used for quantifying changes of data reading performance before and after migration; determining a target storage medium from the candidate storage media according to the migration income and the migration loss; and migrating the to-be-migrated data to the target storage medium. The embodiment of the invention provides the data migration method and device, equipment and themedium. Automatic and accurate migration of data is achieved.

Description

technical field [0001] Embodiments of the present invention relate to data processing technologies, and in particular, to a data migration method, device, device, and medium. Background technique [0002] In a distributed storage system, there are great differences in the access patterns of different types of data. For hot data, it will be read frequently after writing, and the throughput and delay requirements for writing and reading are very high, so it is more suitable for storage in fast media such as SSD and NVM. For cold data, it is basically not read after writing, and the throughput and delay requirements for writing and reading are not very high, so it is more suitable for storage on cold standby HDD or even tape. By realizing the distinction between hot and cold data and using different media for storage, the cost of data storage can be greatly reduced without affecting user access. [0003] The automatic flow of hot and cold data is currently solved by the follo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/185G06F16/18G06F16/172
CPCG06F16/172G06F16/1805G06F16/1847G06F16/185
Inventor 崔嘉诚陈志鹏
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap