Data migration method and device and distributed data storage system

The data migration method in distributed storage systems addresses the issue of uneven data distribution during cluster expansion by strategically relocating files based on meta-data and node information, achieving balanced storage and improved performance.

CN119719065APending Publication Date: 2025-03-28MGI TECH CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202311224605.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-09-20
Publication Date
2025-03-28

AI Technical Summary

Technical Problem

In distributed data storage systems, the expansion of storage clusters often leads to uneven data distribution across nodes, resulting in reduced overall system performance due to unbalanced storage capacity, which existing methods fail to effectively address.

Method used

A method for data migration that involves determining target nodes for file relocation based on meta-data and node information, ensuring balanced storage capacity across nodes by migrating files from existing nodes to new nodes during expansion, utilizing meta-data and node performance metrics to optimize data distribution.

Benefits of technology

This approach ensures balanced storage capacity across all nodes, maximizing bandwidth utilization and enhancing overall system performance by evenly distributing data across the expanded cluster nodes.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a data migration method and device and a distributed data storage system. The method comprises the steps that node information of a plurality of first data nodes and a plurality of second data nodes in a storage cluster is obtained, the first data nodes are existing data nodes before capacity expansion of the storage cluster, and the second data nodes are newly added data nodes after capacity expansion of the storage cluster; obtaining metadata information of all stored files in the storage cluster, wherein the stored files are all stored in the first data node; according to the metadata information and the node information, determining a to-be-migrated file from stored files of each first data node, and determining a target data node corresponding to each to-be-migrated file from a plurality of second data nodes; and migrating each to-be-migrated file from the first data node to a target data node corresponding to the to-be-migrated file. The technical problem that the overall performance of the system is low due to the fact that the data storage capacity of each data node is unbalanced after the capacity expansion of the distributed storage cluster in the related technology is solved.
Need to check novelty before this filing date? Find Prior Art