Hash distribution method and system without data migration during system capacity expansion
By adopting a hash distribution method without data migration in a distributed system, the data migration problem during the expansion of the distributed system is solved through the allocation strategy of virtual hash slots, reducing bandwidth and resource consumption, and improving system efficiency and performance .
Patent Information
- Application Number
- PCT/CN2023/140281
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-12-20
- Publication Date
- 2024-12-05
AI Technical Summary
Distributed systems require data migration when expanding capacity, resulting in increased communication bandwidth usage and resource consumption, affecting system performance.
Using the hash distribution method without data migration, the virtual hash slot is divided into ordinary and to-enabled slots. Only when the capacity is expanded are expanded, the new nodes are allocated to the new nodes. New data is written into the new slot, and old data is not migrated to avoid bandwidth. and resource consumption.
It greatly reduces the system bandwidth usage and resource consumption during capacity expansion, improves system efficiency and IO read and write performance, and ensures data balance.
Smart Images

Figure CN2023140281_05122024_PF_FP_ABST
Abstract
Description
A hash distribution method and system without data migration during system expansion Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to an optimized design for a distributed system expansion link. Background Art
[0002] A data storage system refers to a collection of software and hardware components used to store and manage data. It typically includes storage media and storage management tools. Storage media refers to the media used to store data, such as mechanical hard drives, solid-state drives, and magnetic disks. Data storage systems are widely used in enterprises, the internet, cloud computing, and other fields, and are a vital component of modern information technology infrastructure. With the advancement of the times and the rapid development of internet technology, a special type of data storage system—distributed systems—has begun to find a wider range of applications.
[0003] For example, Chinese patent publication number CN201510967204.7 discloses a distributed system consisting of a cluster of autonomous computer nodes connected via a communication network. Data is distributed to different nodes for storage by the distributed system's management system, resulting in greater scalability, availability, fault tolerance, and stability.
[0004] However, for ordinary data storage systems, when new data needs to be stored and the original storage space is not enough, it is only necessary to add a new storage medium, such as adding a hard disk, and then write the new data. For distributed systems, when the original storage space is not enough, it is necessary to expand the capacity, that is, to increase the storage capacity of the system by adding nodes. Specifically, as disclosed in a Chinese patent document with publication number CN201610221781.6, a method for expanding the capacity of a distributed cache system is disclosed. When expanding the capacity, it is necessary to add new nodes to the distributed system. Due to the stability design of the distributed system, the data is evenly and dispersedly distributed in each node. Therefore, when a new node enters the system, not only the new data needs to be written to the new node, but some data in the original node also needs to be transferred to the new node for the uniformity of the entire system. This phenomenon is called data migration during distributed system expansion.
[0005] The data migration process consumes bandwidth and resources. Specifically, data migration requires transferring large amounts of data from the old node to the new node, which consumes the system's communication bandwidth and causes slower responses or delays in other services. Especially in the case of high-concurrency access and large datasets, data migration places even greater pressure on the system's communication bandwidth. Data migration consumes node computing resources, which can lead to performance degradation. In particular, data replication and synchronization between the old and new nodes consumes significant CPU and memory resources, increasing node load.
[0006] Summary of the Invention
[0007] The purpose of the present invention is to provide a hash distribution method and system without data migration during system expansion. During expansion, only nodes need to be added, and there is no phenomenon of migrating old data from old nodes to new nodes, thereby greatly reducing the system bandwidth occupancy and resource consumption during expansion, improving system efficiency, and improving IO read and write performance.
[0008] The present invention is implemented by the following technical solution: a hash distribution method without data migration during system expansion, comprising the following steps:
[0009] S01, virtual hash slot activation setting steps;
[0010] Divide a total of n virtual hash slots into m normal hash slots and nm to-be-enabled hash slots; set the nm to-be-enabled hash slots to a silent state;
[0011] S02, virtual hash slot allocation step;
[0012] Allocate m common hash slots to data nodes in a distributed storage system, and establish a corresponding mapping rule between the slot numbers of the common hash slots and the data nodes;
[0013] S03, data guiding and writing steps;
[0014] Obtaining a user's write data request and directing its write location to the corresponding common hash slot; the data is written to the corresponding common hash slot;
[0015] S04, step of allocating new slots for expansion;
[0016] The distributed storage system enters the expansion phase, and allocates a new hash slot to the new data node, where the new hash slot comes from the nm hash slots to be enabled;
[0017] S05, new data guidance and writing step;
[0018] The old data is located in the original ordinary hash slot and is not changed or transferred; a new write data request from the user is obtained, and the new data is written into the new hash slot, or the new data is written into the new hash slot and the original ordinary hash slot.
[0019] As a preferred embodiment of the present invention, in S02, the m common hash slots are evenly distributed to each of the data nodes for use.
[0020] As a preferred embodiment of the present invention, in S03, a metadata guidance method is adopted: the data identification ID of the data in the write data request is obtained, and the mapping relationship between the data identification ID and the slot number of the assigned ordinary hash slot is stored in the metadata directory.
[0021] As a preferred embodiment of the present invention, in S03, a valid sn number is generated, where the sn number is a natural number that increases in sequence, and the valid sn number is defined as: mod=sn%n, (mod∈), where mod is a modulo operation value, % is a modulus operator, n is the total number of the virtual hash slots, and m is the total number of the ordinary hash slots;
[0022] The valid sn number is assigned to the data to be written, and the valid sn number, the corresponding data identification ID and the corresponding common hash slot number are stored together in the metadata directory.
[0023] As a preferred embodiment of the present invention, in said S03, some of the generated sn numbers are selected and uniformly saved locally first, and then the sn numbers and the data identification IDs are allocated and corresponded.
[0024] As a preferred embodiment of the present invention, in S05, the data node to which the new data will be written is predicted by the valid sn number, and further, the flow direction of the new data is controlled by allocating / abandoning the valid sn number.
[0025] A hash distribution system that does not require data migration during system expansion, including:
[0026] Data nodes, computer nodes used to store data;
[0027] Central management component, a software component used to coordinate and manage distributed storage systems;
[0028] Client, an application used to use the services provided by the distributed storage system;
[0029] The central management component includes a virtual hash slot management unit and a writing unit,
[0030] The virtual hash slot management unit divides all the virtual hash slots into normal hash slots in normal use and unused hash slots to be enabled, and allocates the normal hash slots to each of the data nodes;
[0031] When capacity is expanded, a new data node is generated, and the hash slot management unit allocates a new hash slot to the new data node, and the new hash slot comes from the hash slot to be enabled.
[0032] The writing unit obtains a data writing request from the client, and writes data content into the common hash slot or the new hash slot.
[0033] As a preferred embodiment of the present invention, the virtual hash slot management unit evenly distributes the common hash slots to each of the data nodes for use.
[0034] A computer device comprising:
[0035] memory for storing computer programs;
[0036] A processor configured to implement the aforementioned hash distribution method without data migration during system expansion when executing the computer program. A computer-readable storage medium having a computer program stored thereon, configured to implement the aforementioned hash distribution method without data migration during system expansion when executed by the processor.
[0037] The technical effects of the present invention are:
[0038] 1. Existing data remains in the original hash slot and does not change. Since only new data is communicated, and old data is not migrated, bandwidth and resource consumption are avoided. This significantly improves the efficiency of the entire distributed storage system and enhances I / O read and write performance.
[0039] 2. Using metadata guidance, the process of guiding data to different common hash slots is more intuitive and controllable.
[0040] 3. The metadata includes the sn number, data identification ID and hash slot number, making data mapping, search, reading and writing more convenient.
[0041] 4. By using the weight adjustment module to use / discard SN, we can accurately control the flow of new data, which helps ensure the consistency of data balance in the distributed storage system. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] FIG1 is a schematic diagram of the process of the hash distribution method in Example 1; DETAILED DESCRIPTION
[0043] The present invention is described in further detail below.
[0044] This specific embodiment is merely an explanation of the present invention and is not intended to limit the present invention. After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed. However, as long as such modifications are within the scope of the claims of the present invention, they are protected by patent law.
[0045] Example 1, a hash distribution method without data migration during system expansion, is applied in a distributed storage system, which includes data nodes, a central management component, and a client.
[0046] A data node is a computer node that stores and manages data. It is usually composed of one or more computers, each of which can store a portion of data and communicate and collaborate with other nodes through the network.
[0047] A central management component is a software component responsible for coordinating and managing the entire distributed storage system. Typically consisting of one or more computers, it processes metadata, configuration information, and monitoring information within the distributed system to ensure its normal operation and efficient management.
[0048] Furthermore, the central management component includes a virtual hash slot management unit and a writing unit.
[0049] The virtual hash slot management unit effectively manages the virtual hash slot, while the writing unit obtains the write data request from the client and writes the data content into the hash slot. Both can be implemented in software.
[0050] A client is an application that uses the services or resources provided by a distributed system. A client sends a request to the distributed system, which then selects one or more nodes to process the request based on the request content and load balancing policy, and returns the results to the client.
[0051] In this embodiment, there are first three data nodes in the distributed storage system, which are denoted as node0, node1 and node2 below.
[0052] As shown in FIG1 , first enter S01 , the virtual hash slot enabling setting step.
[0053] In this step, the virtual hash slot management unit divides all virtual hash slots into two categories: normal hash slots and pending hash slots. Normal hash slots are usable hash slots, while pending hash slots are in a silent state and cannot be used. For example, in this embodiment, the total number of virtual hash slots n is 32, and the number of normal hash slots m is set to 12. The number of pending hash slots is nm, or 20. These 20 hash slots are initially not allowed to be used.
[0054] Then enter S02, the virtual hash slot allocation step.
[0055] The virtual hash slot management unit assigns regular hash slots to data nodes and establishes corresponding mapping rules for regular hash slots and data nodes. The allocation method is user-defined and should be selected based on actual conditions. In this embodiment, a uniform distribution method is used to distribute regular hash slots evenly to each data node. That is, 12 regular hash slots are evenly distributed across three data nodes, with each data node receiving four hash slots. In this embodiment, the specific allocation method is: (slot0, slot1, slot2, slot3) -> node0; (slot4, slot5, slot6, slot7) -> node1; (slot8, slot9, slot10, slot11) -> node2.
[0056] S03: Data guiding and writing steps.
[0057] In this step, the client receives a data write request from the user, and the central management component guides the write location of the data request to the corresponding common hash slot, thereby writing the data into the corresponding common hash slot.
[0058] This bootstrapping process can use static mapping and dynamic mapping.
[0059] Static mapping predefines the mapping relationship for each virtual hash slot during system initialization or configuration, directing different types of data to different virtual hash slots. This approach offers the advantages of simple implementation, stability, and reliability. Dynamic mapping utilizes dynamic algorithms, such as the consistent hashing algorithm, to dynamically allocate virtual hash slots. This approach offers advantages such as flexibility and scalability, allowing for dynamic adjustment of virtual hash slot allocation based on data or node changes to ensure balanced data distribution and efficient access. However, dynamic mapping is more complex to implement and requires consideration of algorithmic correctness and performance.
[0060] In this embodiment, a metadata-based approach is used. Metadata refers to data that describes data, typically including information such as its structure, attributes, relationships, type, format, and access permissions. Metadata is a description or explanation of data and is one of the key elements of data management and processing.
[0061] In this embodiment, metadata can be considered a directory and stored in a distributed storage system, either in the cloud or locally. The metadata directory stores the correspondence between data IDs and assigned common hash slots. For example, if data with ID A corresponds to slot 2, the data will be written to slot 2 of node 0 by the write unit. If data with ID B corresponds to slot 8, the data will be written to slot 8 of node 2.
[0062] S04: New slot allocation step for capacity expansion.
[0063] Steps S01-S03 above represent the normal state of a distributed storage system. As data volume increases, capacity expansion is required, meaning new data nodes are added to the distributed storage system. This step allocates new virtual hash slots to these newly added data nodes.
[0064] For example, in this embodiment, a new data node, node 3, is added.
[0065] At this point, a new hash slot needs to be allocated for node 3. Specifically, the central management component selects several hash slots to be enabled, sets them as normal hash slots, and allocates them to node 3.
[0066] In this embodiment, four hash slots are also allocated to the new data node, namely (slot12, slot13, slot14, slot15)->node3.
[0067] S05: New data guidance and writing steps.
[0068] The original data remains in the original hash slot and, naturally, in the original data node, without any changes.
[0069] When a client receives a new data request from a user, the central management component allocates hash slots for the new data request, directing more of the new data to hash slots 12, 13, 14, and 15. This allows more new data to be stored on new data nodes. This process avoids bandwidth and resource consumption by only communicating new data and not migrating old data. This significantly improves the efficiency of the entire distributed storage system and enhances I / O read and write performance.
[0070] A computer device comprising:
[0071] memory for storing computer programs;
[0072] A processor configured to implement the aforementioned hash distribution method without data migration during system expansion when executing the computer program. A computer-readable storage medium having a computer program stored thereon, which, when executed by the processor, implements the aforementioned hash distribution method without data migration during system expansion.
[0073] The difference between Example 2 and Example 1 is the specific implementation form of S03.
[0074] In this embodiment 2, S03 further adopts the sn modulus guidance method.
[0075] Before this, the three data nodes have been allocated ordinary hash slots, namely: (slot0, slot1, slot2, slot3)->node0; (slot4, slot5, slot6, slot7)->node1; (slot8, slot9, slot10, slot11)->node2.
[0076] Slot11 indicates that the slot number of the common hash slot is 11.
[0077] At this time, the client receives a write data request from the user. The central management component is responsible for generating a valid sn number. The sn number is a natural number that increases and cannot be reused. The maximum sn currently used is recorded as cur.sn and saved persistently so that the calculation can continue from the current maximum sn value after the system restarts.
[0078] In this embodiment, the effective sn is defined as: mod=sn%32, (mod∈).
[0079] Among them, mod is the modulo operation value, and % is the modulo operator, which refers to the remainder after dividing two numbers. For example, the result of "10%3" is "1" because the quotient of 10 divided by 3 is 3 and the remainder is 1.
[0080] Therefore, when sn is 0-11, it is determined to be a valid sn, and when it is 22-31, it is determined to be an invalid sn. Invalid sns are discarded and not used. When it is 32-43, it is determined to be a valid sn.
[0081] After the effective SN is calculated, it is originally stored in the cloud. In this embodiment, the client applies to the central management component for a batch of effective SNs to be cached locally for backup, so as to reduce the number of interactions between the client and the central management component, thereby further reducing the occupation of communication resources.
[0082] When a client receives a data write request, it selects one from the valid sn number set in the local cache and associates it with the data's external unique identifier as metadata and writes it into the distributed storage system.
[0083] Then, mod = sn % 32 is calculated. The mod value is the corresponding hash slot number, or index number. For example, if data with identifier F is assigned to sn 42, the central management component will recognize that mod = 10 and that data F should be assigned to hash slot 10, or data node 2.
[0084] The use of SN module guidance makes the mapping relationship clearer and more intuitive, and facilitates subsequent maintenance by software debuggers.
[0085] Example 3 adds a weight distribution design based on Example 2.
[0086] Specifically, in S05, the new data guidance and writing step, a weight distribution design is added.
[0087] When the client receives a new data request from the user, the central management component allocates hash slots for the new data request, directing more new data to hash slots such as slot12, slot13, slot14, and slot15, so that more new data can be stored in the new data node.
[0088] In this third embodiment, the central management component includes a weight adjustment module to ensure that more data is allocated to hash slots 12, 13, 14, and 15. The weight adjustment module also relies on sn. As described above, the modulo operation of sn is used to determine the index of the hash slot to be allocated. The weight adjustment module can distribute and invalidate sn to direct data to different data nodes.
[0089] For example, data numbered G needs to be written to the distributed storage system. Since the fourth node, node3, is assigned hash slots 12, 13, 14, and 15, the weight adjustment module can identify the next data item to be assigned. For example, if the data item to be assigned is 137, the weight adjustment module recognizes that the mod value of 138 / 32 is 9, and discards it. The same applies to data items 138 and 139. However, if the data item is 140, the weight adjustment module recognizes that its mod value is 12, and assigns it to data G, which is then written to slot 12 of node3.
[0090] Based on this, the weight adjustment module can control the precise flow of new data by using or discarding SNs. When the weight adjustment module finds that the data volume of each data node has reached a balance, it can no longer specify that new data only flow into the new data node, but instead write it to all data nodes in a balanced manner.
[0091] As for the adjustment of weights by the weight adjustment module and the monitoring of the data volume of each data node, these contents can be easily implemented by those skilled in the art through programming. The software is written as existing technical content and will not be described in detail in this article.
[0092] The design of the weight adjustment module helps to ensure the consistency of the final data balance in the distributed storage system.
Claims
1. A hash distribution method without data migration during system expansion, characterized in that: The method comprises the following steps: S01, a virtual hash slot activation setting step; dividing a total of n virtual hash slots into m ordinary hash slots and nm hash slots to be activated; setting the nm hash slots to be activated to a silent state; S02, a virtual hash slot allocation step; allocating the m ordinary hash slots to the data nodes in the distributed storage system for use, and establishing a corresponding mapping rule between the slot number of the ordinary hash slot and the data node; S03, a data guiding and writing step; obtaining a user's write data request, and guiding its write position to the corresponding ordinary hash slot; the data is written to the corresponding ordinary hash slot; S04, a new slot allocation step for capacity expansion; the distributed storage system enters the capacity expansion stage, and allocates a new hash slot to the new data node, and the new hash slot comes from the nm hash slots to be activated; S05, a new data guiding and writing step; the old data is located in the original ordinary hash slot and is not transferred without change; obtaining a new write data request from the user, and the new data is written to the new hash slot, or the new data is written to the new hash slot and the original ordinary hash slot.
2. A hash distribution method without data migration during system expansion according to claim 1, characterized in that: In S02, the m common hash slots are evenly allocated to each of the data nodes for use.
3. The hash distribution method without data migration during system expansion according to claim 1, characterized in that: In the S03, a metadata guidance method is adopted: the data identification ID of the data in the write data request is obtained, and the mapping relationship between the data identification ID and the slot number of the allocated ordinary hash slot is stored in the metadata directory.
4. A hash distribution method without data migration during system expansion according to claim 3, characterized in that: In the S03, a valid sn number is generated, where the sn number is a natural number that increases in sequence, and the valid sn number is defined as: mod∈[0,m-1], where mod=sn%n, % is a modulus operator, n is the total number of the virtual hash slots, and m is the total number of the ordinary hash slots; The valid sn number is assigned to the data to be written, and the valid sn number, the corresponding data identification ID and the corresponding ordinary hash slot number are stored together in the metadata directory.
5. A hash distribution method without data migration during system expansion according to claim 4, characterized in that: In the S03, some of the sn numbers that have been generated are selected and uniformly saved locally first, and then the sn numbers and the data identification IDs are allocated and corresponded.
6. A hash distribution method without data migration during system expansion according to claim 4 or 5, characterized in that: In S05, the data node to which the new data is to be written is predicted by the effective sn number, and then the flow direction of the new data is controlled by allocating / abandoning the effective sn number.
7. A hash distribution system without data migration during system expansion, characterized in that: It comprises: a data node, a computer node for storing data; a central management component, a software component for coordinating and managing a distributed storage system; a client, an application for using services provided by the distributed storage system; the central management component comprises a virtual hash slot management unit and a write unit, the virtual hash slot management unit divides all the virtual hash slots into ordinary hash slots for normal use and unused hash slots to be enabled, and allocates the ordinary hash slots to each of the data nodes; when capacity is expanded, a new data node is generated, the hash slot management unit allocates a new hash slot to the new data node, the new hash slot comes from the hash slot to be enabled, the write unit obtains a write data request in the client, and writes the data content to the ordinary hash slot or the new hash slot.
8. A hash distribution system without data migration during system expansion according to claim 7, characterized in that: The virtual hash slot management unit evenly distributes the common hash slots to each of the data nodes for use.
9. A computer device, characterized in that: Include: Memory for storing computer programs; A processor, used for implementing the hash distribution method without data migration during system expansion as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the hash distribution method without data migration during system expansion as described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Database dynamic capacity expansion method and storage medium
CN110222030A
Video CDN node instant capacity expansion method, scheduler and CND storage system
CN111669629A
Database processing method and device, computer equipment and storage medium
CN112541039A
Data storage method and device, electronic equipment and computer readable storage medium
CN114844911A
Hash distribution method and system without data migration during system capacity expansion
CN116361299A
Cited By
Method for carrying out load balancing on distributed storage system under capacity reduction scene
CN121397006A
