Method, electronic device and computer program product for backup data
By calculating the deduplication rate of the backup node and selecting the target backup node, the problem of low data backup space utilization in the existing technology is solved, and a more efficient data backup process is achieved.
Patent Information
- Application Number
- CN202310836638.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-07
- Publication Date
- 2025-01-07
AI Technical Summary
When selecting a backup node, the existing data backup method fails to achieve optimal space utilization, resulting in high space occupation and low efficiency during the backup process.
By obtaining the data block deduplication information of the data to be backed up and the deduplication information of the backup node, the deduplication rate of each backup node is calculated, so as to select the target backup node with the highest deduplication rate for backup, and optimize the space occupation and backup of the backup node. efficiency.
This reduces the space occupation of backup nodes, improves the efficiency of the backup process, and achieves better space utilization.
Smart Images
Figure CN119271451A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to the field of computer technology, and more particularly to a method, an electronic device, and a computer program product for backing up data. Background Art
[0002] Data backup is an important method to protect data integrity and recoverability. When the source data on the storage device in the storage system is very important, users can configure backup services for the important data to ensure that the data can be recovered even when the storage device is damaged. In addition, through data backup, the protected data can also be restored to a specified time point to view the corresponding data.
[0003] When backing up data, you often need to consider multiple factors, such as backup speed, backup space, and number of backups, among which backup space is the most critical. By selecting a suitable backup node for the source data to be backed up, you can save backup space and reduce storage costs. Summary of the invention
[0004] Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for backing up data.
[0005] Embodiments of the present disclosure provide a method, electronic device, and computer program product for backing up data. The method includes obtaining data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in a backup node set, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the corresponding backup node. The method also includes determining multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set based on the data block deduplication information of the source data and the multiple backup node deduplication information, wherein the deduplication rate in the multiple deduplication rates indicates the deduplication ratio of the source data when the source data is backed up to the corresponding backup node. In addition, the method also includes selecting a target backup node for backing up the source data from the backup node set based on the multiple deduplication rates for the multiple backup nodes.
[0006] In another aspect of the present disclosure, an electronic device is provided. The device includes a processing unit and a memory, wherein the memory is coupled to the processing unit and stores instructions. The instructions perform the following actions when executed by the processing unit: obtain data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in a backup node set, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the corresponding backup node; based on the data block deduplication information of the source data and the multiple backup node deduplication information, determine multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set, wherein the deduplication rate in the multiple deduplication rates indicates the deduplication ratio of the source data when the source data is backed up to the corresponding backup node; based on the multiple deduplication rates for the multiple backup nodes, select a target backup node for backing up the source data from the backup node set.
[0007] In another aspect of the present disclosure, a computer program product is provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which, when executed, cause a computer to perform a method or process according to an embodiment of the present disclosure.
[0008] The invention summary is provided to introduce relevant concepts in a simplified form, which will be further described in the following detailed description. The invention summary is not intended to identify key features or essential features of the present disclosure, nor is it intended to limit the scope of various embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like elements throughout the exemplary embodiments of the present disclosure.
[0010] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;
[0011] Figure 2 A flowchart of a method for backing up data according to some disclosed embodiments is shown;
[0012] Figure 3A is a flowchart of a process of obtaining data block deduplication information of source data according to some embodiments of the present disclosure;
[0013] Figure 3B is a schematic diagram of a process of maintaining a hash cache according to some embodiments of the present disclosure
[0014] Figure 4 is a schematic diagram of a process of calculating a deduplication rate according to some embodiments of the present disclosure;
[0015] Figure 5 is a schematic diagram of a backup node deduplication process for maintaining a backup node according to some embodiments of the present disclosure;
[0016] Figure 6 is a flowchart of a method for backing up data according to some embodiments of the present disclosure;
[0017] Figure 7 A schematic block diagram of a device that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0018] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some specific embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0019] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly indicated to be different.
[0020] Data backup is an important method for protecting data in a storage system. When performing data backup, it is necessary to select a target backup node from multiple backup nodes in a backup node set for the source data to be backed up, so as to back up the source data to be backed up. When selecting a backup node for the source data to be backed up, the current data backup method will select it based on the remaining space of the backup node, or simply determine the compression rate and deduplication rate of the source data, resulting in failure to achieve optimal space utilization. The reason is that the source data will be divided into multiple data blocks when being backed up, and some of these data blocks may already exist on the backup node. In this case, it is not necessary to actually store these data blocks during the backup, which can save space. Therefore, when the data blocks of the source data are not on the selected backup node, it will result in low space utilization.
[0021] In order to solve the above problems, an embodiment of the present disclosure provides a method for backing up data. The method selects a suitable backup node by utilizing the data characteristics of the source data to be backed up. Specifically, the data block deduplication information of the source data is obtained, and the data block deduplication information records the number of occurrences of each data block in the source data. The data block deduplication information of the source data is compared with the data block deduplication information of multiple backed up data on the backup node to determine the deduplication rate of the source data relative to each backup node, thereby selecting the target backup node through the multiple deduplication rates of multiple backup nodes. By analyzing the data characteristics of the source data to be backed up, the embodiment of the present disclosure can reduce the space occupancy of the backup node when backing up the source data, thereby achieving better space utilization.
[0022] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, the example environment 100 includes a storage service 102, a storage service 104, and a storage service 106. These storage services may be the same storage service or different storage services, such as different storage services from the same provider. It should be understood that only three storage services are listed in the example environment 100, but there may actually be fewer or more storage services.
[0023] Some source data are stored in each storage service, and each source data belongs to a specific tenant. For example, source data 112-1 and 112-2 are stored in storage service 102, source data 114-1 and 114-2 are stored in storage service 104, and source data 116-1, 116-2 and 116-3 are stored in storage service 106. It should be understood that the number of source data stored on each storage service here is only for illustrative purposes, and less or more source data can actually be stored. In some embodiments, each source data can be stored on different storage devices. For example, source data 112-1 can be stored in flash memory, hard disk, and file system.
[0024] In addition, the example environment 100 also includes a backup management system 120, through which a suitable backup node can be selected for any source data in the source data to save the space of the backup node to the greatest extent. Figure 1As shown, the backup management system 120 includes a backup node selection service 130, which uses a deduplication rate calculation service 132 to calculate the deduplication rate of the source data to be backed up relative to each backup node, so as to select the target backup node. For example, when the source data 112-1 is the source data to be backed up, the backup management system 120 will obtain the data block deduplication information of the source data 112-1, which includes the key-value pairs of the data blocks and the number of occurrences of the data blocks in the source data 112-1, and then multiple deduplication rates can be calculated through the deduplication rate calculation service 132 in the backup node selection service 130. The data block deduplication information of the source data 112-1 can be pre-calculated, or it can be calculated when the backup request is initiated, and the present disclosure does not limit this. In addition, when the data block deduplication information is pre-calculated, it can be stored on the storage device where the source data 112-1 is located, or it can be stored on the backup management system 120, and the present disclosure does not limit this.
[0025] In addition, the example environment 100 also includes a backup node set 150, which includes backup nodes 152 to 158. When calculating multiple deduplication rates, it is necessary to obtain backup node deduplication data 142-1 to 142-N in the backup node deduplication database 140. For example, the backup node deduplication data 142-1 is the deduplication data of the backup node 152, which includes the data block deduplication information of each backed up data in the backed up data 162-1 to 162-4. It should be understood that the data block deduplication information of the backed up data and the data block deduplication information of the source data are the same in data structure. In addition, backup node deduplication data 142-2 is the deduplication data of backup node 154, which includes the data block deduplication information of each backed up data 164-1 to 162-3; node deduplication data 142-3 is the deduplication data of backup node 156, which includes the data block deduplication information of each backed up data 166-1 to 166-4; backup node deduplication data 142-4 is the deduplication data of backup node 158, which includes the data block deduplication information of each backed up data 168-1 to 162-8. It should be understood that the number of backup nodes and the number of backed up nodes in each backup node are exemplary, and may include fewer or more numbers.
[0026] It should be understood that the architecture and functions in the example environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. Embodiments of the present disclosure may also be applied to other environments with different structures and / or functions.
[0027] Figure 2A flowchart of a method 200 for backing up data according to some disclosed embodiments is shown. At 202, data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in a backup node set are obtained, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the backup node. For example, when the source data is backed up to the backup node, it is divided into multiple data blocks and stored in the form of data blocks on the backup node. When the source data is divided into multiple data blocks, due to the duplication of data in the source data itself, duplication will occur between the data blocks. The data blocks are deduplicated and the number of occurrences of each deduplicated data block is counted, so that the data block deduplication information of the source data can be obtained. In addition, the deduplication information of the backup node can refer to Figure 1 For example, the backup node deduplication data 142-1 of the backup node 152 includes the data block deduplication information of each of the backed-up data 162-1 to 162-4. For example, when the backed-up data 162-4 is backed up to the backup node 152, its data block deduplication information is also recorded in the backup node deduplication data 142 of the backup node 152.
[0028] At 204, based on the data block deduplication information of the source data and the deduplication information of the plurality of backup nodes, a plurality of deduplication rates of the source data relative to the plurality of backup nodes in the backup node set are determined, wherein the deduplication rate in the plurality of deduplication rates indicates a deduplication ratio of the source data when the source data is backed up to the corresponding backup node. Figure 1 When the source data 112-1 is backed up, the deduplication rate of the source data 112-1 relative to the backup nodes 152 to 158 is calculated based on the data block deduplication information of the source data 112-1 and the backup node deduplication data 142-1 to 142-4. In other words, the deduplication ratio of the duplicate data blocks when the source data 112-1 is backed up to each backup node. Accordingly, the greater the deduplication ratio, the more storage space of the backup node is saved, and the efficiency of the backup process can be promoted.
[0029] At 206, based on multiple deduplication rates for multiple backup nodes, a target backup node for backing up the source data is selected from the backup node set. Figure 1 When backing up source data 112-1, the four deduplication rates of source data 112-1 relative to backup nodes 152 to 158 are calculated. Then, the backup node with the largest deduplication rate is selected as the target backup node for source data 112-1, which can save the storage space of the entire backup node set and optimize the backup service.
[0030] Therefore, according to the method 200 of some embodiments of the present disclosure, by using the data characteristics of the source data, the deduplication rate of the source data relative to each backup node is calculated, so as to select a suitable backup node for the source data. Therefore, the backup node with the highest deduplication rate can be selected for the source data, thereby optimizing the space occupation of the backup node and improving the efficiency of the backup process.
[0031] Figure 3A FIG. 3 is a flow chart of a process 300 for obtaining data block deduplication information of source data according to some embodiments of the present disclosure. At 302, the source data is divided into a plurality of data blocks, wherein the size of each data block may be fixed, such as 8 KB. For example, the plurality of data blocks of the source data may be b 1 , b 2 , …, b N At 304, a hash value of each data block in the plurality of data blocks is obtained. For example, when calculating the hash value of each data block, various commonly used hash algorithms, such as the SHA2 algorithm, can be used to obtain the hash value of the data block. After obtaining the hash value of the data block, it can be determined whether the corresponding data blocks are the same by comparing whether the hash values are the same, for example, directly comparing b 1 and b 2 It may be difficult to determine whether it is the same data block, but in getting b 1 The hash value h 1 and b 2 The hash value h 2 After that, the hash values h can be directly compared 1 and h 2 Whether they are the same to determine data block b 1 and b 2 Are they the same?
[0032] At 306, it is determined whether the hash value of each data block is in a predetermined hash cache. Here, the predetermined hash cache is a hash set with a high probability of duplication maintained in the storage system. Figure 3B Describe how to maintain a hash cache. When it is determined that the hash value is not in the predetermined hash cache, at 308, the count value of the hash value is not updated, that is, the count value corresponding to the hash value is always 0. When it is determined that the hash value is in the predetermined hash cache, at 310, the count value corresponding to the hash value is increased, for example, the count value is increased by 1. For example, data block b 1 The hash value h 1 If it is not in the predetermined hash cache, the hash value h is not updated. 1 The corresponding count value is always 0; data block b 2 The corresponding hash value h 2 In the predetermined hash cache, the hash value h 2The count value of is increased by 1. It should be understood that there may be other data blocks b m Its hash value is also h 2 (That is, data block b m and data block b 2 is the same data block), then the hash value h 2 The count value will be increased by 1. Finally, the data block deduplication information obtained for the source data is C = {(h 0 ,c 0 ),(h 1 ,c 1 ),…,(h n ,c n )}, for example h 0 is the hash value of a data block of the source data, and the hash value is in the predetermined hash cache, c 0 is the number of times the hash value h0 appears in the source data, that is, the hash value h 0 The number of times the corresponding data block appears in multiple data of the source data.
[0033] Therefore, according to process 300 of some embodiments of the present disclosure, when obtaining data block deduplication information of source data, a predetermined hash cache is utilized, and the predetermined hash cache is a hash set with a high probability of duplication maintained in the storage system. For example, the source data may have a large number of different hash values, so the data block deduplication information may be very redundant, and many of the hash values may not be repeated with the data blocks on the backup node in subsequent backups. Through the predetermined hash cache. Therefore, the obtained data block deduplication information only maintains the hash values and their count values that may have a high probability of duplication during backup, thereby optimizing the data block deduplication information.
[0034] Figure 3B350 is a schematic diagram of a process 350 for maintaining a hash cache according to some embodiments of the present disclosure. For a primary storage server, a deduplication service is usually enabled. Deduplication can reduce the amount of storage required for user data by retaining only a small number of copies (usually only one copy) of a data block with a given content. Deduplication can be completed in real time when data enters the storage device, which is called in-line deduplication. In-line deduplication can not only save space, but also provide additional performance improvements. On a storage system, there are a large number of write IOs, and in-line deduplication technology can be used to store the corresponding data blocks on the storage device. At 352, the host writes IO. At 354, the hash value of the data block in the IO is calculated, and a variety of hash algorithms can be used to calculate the hash value, such as the SHA512 (256 bits with the first 256 bits truncated) cryptographic hash function. At 356, the hash cache is updated using the hash value. For example, when a specific hash value does not exist in the hash cache, the specific hash value is written to the hash cache, or the specific hash value already exists in the hash cache, then the count value corresponding to the hash cache is increased by 1. The subsequent steps are conventional operations of writing IO, for example, determining whether the hash value is in the predetermined cache at 358, if it is a hit, only mapping the data block without actually writing it at 360, and if it is a miss, compressing the data block at 362 and writing the data block.
[0035] Thus, the hash values and count values of a large number of data blocks on the storage system are stored in the hash cache, and the repetition level of the hash value can be determined by the absolute size or relative order of the count value, which also indicates the possibility of the hash value being deduplicated. In some embodiments, the hash value with a larger absolute size or a higher relative order is selected as Figure 3A The predetermined hash cache used in the described process 300 is used to obtain the data block deduplication information of the source data. It should be understood that the absolute size being larger or the hash value being higher in relative order mentioned here is not limiting, and those skilled in the art can select more or less hash values as needed.
[0036] Figure 4 FIG. 4 is a schematic diagram of a process 400 for calculating a deduplication rate according to some embodiments of the present disclosure. Figure 4 As shown, block 402 is the data block deduplication information of the source data to be backed up, and the data block deduplication information includes the data block hash value of the source data and its corresponding count value. For example, the data block deduplication information 402 includes the data block hash values H0 to H6 of the source data, and the corresponding count values are C0 to C6 respectively.
[0037] also, Figure 4 Including backup nodes 412 to 416, when performing backup, it is necessary to calculate the deduplication rate of the source data relative to each backup node. To calculate the source data relative to the backup node 414 (that is, equivalent to Figure 1 For example, the deduplication rate 430 of the backup node 154 in the example, the deduplication node 414 includes multiple backed up data (ie, equivalent to Figure 1 In the example, the plurality of backed up data 164-1 to 161-3 in the backup node 414 are backed up, each of which has data block deduplication information, which is the same in form as the data block deduplication information 402 of the source data. Through the data block deduplication information of the backed up data of the backup node 414, the stored hash set of the backup node 414 can be obtained to determine whether each hash value in the data block deduplication information 402 is on the backup node 414.
[0038] like Figure 4 As shown, hash value H0 is in the stored hash set of backup node 414, so hash value H0 is added to hit set 422, and hash value H1 is not in the stored hash set of backup node 414, so hash value H1 is added to miss set 424. When calculating the deduplication rate 430, the deduplication value corresponding to the hash value in miss set 422 can be determined, that is, C0+C3+C4+…, and the deduplication value corresponding to the hash value in miss set 422 can be determined (C1-1)+(C2-1)+(C5-1)+(C6-1)…. Here, since the data block corresponding to hash value H1 is not on backup node 414, the data block needs to be written to backup node 414, and it only needs to be written once, so its corresponding deduplication value is (C1-1), and the other parts have the same meaning. More generally, the deduplication rate R i It can be calculated as shown in Formula 1:
[0039]
[0040] Among them, x j is the count value of the hash value in the hit set 422 (ie, the count value of the hash value in the data block deduplication information of the source data), y k is a count value of the hash values in the miss set 424 , Block_Size indicates the size of the data block (eg, 8 KB), and Total_Primary_Data_Size indicates the size of the source data.
[0041] Figure 5 FIG. 5 is a schematic diagram of a backup node deduplication process 500 for maintaining a backup node according to some embodiments of the present disclosure. Figure 5As shown, it includes a backup node deduplication database 510 and a backup node set 520. For the purpose of illustration, the backup node deduplication database 510 only shows the backup node deduplication data 512, and the backup node set 520 further shows the backup node 522. At the beginning, the backup node 522 includes the backed up data 522-1 to 522-4, and the data block deduplication information corresponding to each backed up data is stored in the backup node deduplication data 512, that is, the deduplication information 512-1 to 512-4. At a certain moment, the backed up data 522-2 in the backup node 522 is removed, and then the corresponding deduplication information 512-2 also needs to be removed in the backup node deduplication data 512. At another moment, new backup data is added to the backup node 522, that is, the backed up data 525-N, and then the corresponding deduplication information 512-N will also be added to the backup node deduplication data 512.
[0042] Figure 6 It is a flow chart of a method 600 for backing up data according to some embodiments of the present disclosure. At 602, a source data backup request is received. At 604, it is determined whether the source data is in the backup database. In addition, it can also be determined whether the data block deduplication information of the source data is in the backup data deduplication database. Because the source data may have been backed up to a backup node at a previous moment, when the source data does not change much between the current moment and the previous moment, it is a better choice to directly back up the source data to the previous backup node, so it is necessary to perform a judgment at 604 to determine whether the source data is in the database. When it is determined at 604 that the source data is in the backup node, that is, whether the source data is backed up for the first time, proceed to 606 to determine whether the change rate of the deduplication information of the source data is less than a predetermined threshold. The reason for performing this judgment is that when the source data is not backed up for the first time, but compared with the previous backup, it has changed a lot, then it is still necessary to select a suitable backup node for the source data, and when the change is not big, then directly backing up the source data to the original backup node can achieve the maximum deduplication rate. When it is determined at 606 that the change rate of the deduplication information of the source data is less than a predetermined threshold, the process proceeds to 608 to select an original backup node for this backup.
[0043] When it is determined at 604 that the source data is not in the backup node, that is, the source data is backed up for the first time, proceed to 610. When it is determined at 606 that the change rate of the deduplication information of the source data is greater than a predetermined threshold, that is, the source data has a larger change rate than the previous backup, also proceed to 610. At 610, determine whether all backup nodes are empty backup nodes. When it is determined at 610 that all backup nodes are empty backup nodes, proceed to 612, select a backup node based on the remaining storage space of the backup node, and when the remaining storage space of all backup nodes is the same, a backup node can be randomly selected as the target backup node. Since all backup nodes are empty backup nodes, it is easy to understand that the deduplication rate is the same when the source data is backed up to any backup node.
[0044] When it is determined at 610 that all backup nodes are not empty backup nodes, proceed to 614. At 614, the deduplication rate of the source data relative to each backup node is calculated. At 616, it is determined whether there is an empty backup node. When there is an empty backup node, the deduplication rate of the source data relative to the empty backup node will be less than the deduplication rate of other backup nodes, which will cause the empty node to be unable to be put into use. Therefore, it is determined at 616 whether there is an empty backup node to solve this problem. When it is determined at 616 that there is an empty backup node, proceed to 618. At 618, it is determined whether the deduplication rate is lower than a predetermined threshold. The purpose of this judgment is that when the deduplication rate is lower than the predetermined threshold, it can be considered that the deduplication rate of the source data relative to each backup node is low, then the empty backup node can be selected as the target backup node to help the empty backup node to perform a cold start. When it is determined at 618 that the deduplication rate is lower than the predetermined threshold, proceed to 620. At 620, an empty backup node is selected as the target backup node. When it is determined at 618 that the deduplication rate is not lower than the predetermined threshold, proceed to 622. At 622, a backup node with the highest deduplication rate is selected, and the backup node needs to have enough remaining space to back up the source data. Finally, at 624, the source data is backed up to the selected target backup node.
[0045] Figure 7 1 shows a schematic block diagram of a device 700 that can be used to implement an embodiment of the present disclosure, and the device 600 can be a device or apparatus described in an embodiment of the present disclosure. Figure 7As shown, the device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704. Although not shown in FIG. Figure 6 As shown in FIG. 7 , device 700 may further include a coprocessor.
[0046] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0047] The various methods or processes described above may be performed by the processing unit 701. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU 701, one or more steps or actions in the methods or processes described above may be performed.
[0048] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.
[0049] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0050] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0051] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.
[0052] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0053] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0054] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the equipment, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each block in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the blocks can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous blocks can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flow chart, and the combination of the blocks in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0055] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for backing up data, comprising: Acquire data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in the backup node set, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the corresponding backup node; Determine, based on the data block deduplication information of the source data and the deduplication information of the multiple backup nodes, multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set, wherein a deduplication rate in the multiple deduplication rates indicates a deduplication ratio of the source data when the source data is backed up to the corresponding backup node; as well as A target backup node for backing up the source data is selected from the set of backup nodes based on the multiple deduplication rates for the multiple backup nodes.
2. The method according to claim 1, wherein obtaining the data block deduplication information of the source data comprises: By dividing the source data into multiple data blocks, multiple hash values of the multiple data blocks are obtained; Based on the multiple hash values, data block deduplication information of the source data is determined.
3. The method according to claim 2, wherein determining the data block deduplication information of the source data based on the multiple hash values comprises: determining whether a hash value among the plurality of hash values exists in a predetermined hash cache; In response to determining that the hash value exists in the predetermined hash cache, increasing a count value of a data block corresponding to the hash value; as well as Based on the multiple count values of the multiple data blocks, data block deduplication information of the source data is determined.
4. The method according to claim 3, wherein determining the multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set comprises: Determine a plurality of hit data block sets and a plurality of miss data block sets based on the data block deduplication information and the plurality of backup node deduplication information, wherein the hit data block sets in the plurality of hit data block sets include data blocks in the data block deduplication information of the source data that exist in the backup node, and the miss data block sets in the plurality of miss data block sets include data blocks in the source data that do not exist in the backup node; as well as The plurality of deduplication rates are determined based on the plurality of hit data block sets and the plurality of miss data block sets.
5. The method according to claim 4, wherein determining the plurality of deduplication rates based on the plurality of hit data block sets and the plurality of miss data block sets comprises: In response to determining that a data block in the source data exists in the plurality of hit data block sets, determining a plurality of first deduplication values; In response to determining that a data block in the source data exists in the plurality of missed data block sets, determining a plurality of second deduplication counts; as well as The plurality of deduplication rates are determined based on the plurality of first deduplication values, the plurality of second deduplication values, the size of a data block in the source data, and the size of the source data.
6. The method according to claim 1, further comprising: In response to removing the backed-up data from a backup node among the plurality of backup nodes, data block deduplication information of the backed-up data is removed from corresponding backup node deduplication information among the plurality of backup node deduplication information.
7. The method according to claim 6, further comprising: In response to adding new backup data to a backup node among the plurality of backup nodes, data block deduplication information of the new backup data is added to corresponding backup node deduplication information among the plurality of backup node deduplication information.
8. The method according to claim 1, further comprising: Before obtaining the data block deduplication information of the source data and the plurality of backup node deduplication information of the plurality of backup nodes in the backup node set, in response to determining that the source data was previously backed up to a specific backup node in the backup node set, determining a rate of change of the data block deduplication information of the source data; as well as In response to determining that the change rate is greater than a first predetermined threshold, data block deduplication information of the source data and the plurality of backup node deduplication information of the plurality of backup nodes in the backup node set are obtained.
9. The method according to claim 8, further comprising: Before obtaining the data block deduplication information of the source data and the multiple backup node deduplication information of the multiple backup nodes in the backup node set, determining whether the multiple backup nodes in the backup node set have backed up data; as well as In response to determining that the plurality of backup nodes in the backup node set have backed up data, data block deduplication information of the source data and the plurality of backup node deduplication information of the plurality of backup nodes in the backup node set are obtained.
10. The method according to claim 9, further comprising: In response to determining that the backup node set includes an empty backup node, the target backup node for backing up the source data is determined based on the multiple deduplication rates.
11. The method according to claim 10, wherein determining the target backup node based on the multiple deduplication rates comprises: In response to determining that each of the plurality of deduplication rates is less than a second predetermined threshold, determining the empty backup node as the target backup node for backing up the source data.
12. An electronic device comprising: processor; as well as a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform actions, the actions comprising: Acquire data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in the backup node set, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the corresponding backup node; Determine, based on the data block deduplication information of the source data and the multiple backup node deduplication information, multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set, wherein a deduplication rate in the multiple deduplication rates indicates a deduplication ratio of the source data when the source data is backed up to the corresponding backup node; and A target backup node for backing up the source data is selected from the set of backup nodes based on the multiple deduplication rates for the multiple backup nodes.
13. The electronic device according to claim 12, wherein obtaining the data block deduplication information of the source data comprises: By dividing the source data into multiple data blocks, multiple hash values of the multiple data blocks are obtained; Based on the multiple hash values, data block deduplication information of the source data is determined.
14. The electronic device according to claim 13, wherein determining the data block deduplication information of the source data based on the multiple hash values comprises: determining whether a hash value among the plurality of hash values exists in a predetermined hash cache; In response to determining that the hash value exists in the predetermined hash cache, increasing a count value of a data block corresponding to the hash value; as well as Based on the multiple count values of the multiple data blocks, data block deduplication information of the source data is determined.
15. The electronic device of claim 14, wherein determining the plurality of deduplication rates of the source data relative to the plurality of backup nodes in the backup node set comprises: Determine a plurality of hit data block sets and a plurality of miss data block sets based on the data block deduplication information and the plurality of backup node deduplication information, wherein the hit data block sets in the plurality of hit data block sets include data blocks in the data block deduplication information of the source data that exist in the backup node, and the miss data block sets in the plurality of miss data block sets include data blocks in the source data that do not exist in the backup node; as well as The plurality of deduplication rates are determined based on the plurality of hit data block sets and the plurality of miss data block sets.
16. The electronic device according to claim 15, wherein determining the plurality of deduplication rates based on the plurality of hit data block sets and the plurality of miss data block sets comprises: In response to determining that a data block in the source data exists in the plurality of hit data block sets, determining a plurality of first deduplication values; In response to determining that a data block in the source data exists in the plurality of missed data block sets, determining a plurality of second deduplication counts; as well as The plurality of deduplication rates are determined based on the plurality of first deduplication values, the plurality of second deduplication values, the size of a data block in the source data, and the size of the source data.
17. The electronic device according to claim 12, wherein the actions further comprise: In response to removing the backed-up data from a backup node among the plurality of backup nodes, data block deduplication information of the backed-up data is removed from corresponding backup node deduplication information among the plurality of backup node deduplication information.
18. The electronic device according to claim 17, wherein the actions further comprise: In response to adding new backup data to a backup node among the plurality of backup nodes, data block deduplication information of the new backup data is added to corresponding backup node deduplication information among the plurality of backup node deduplication information.
19. The electronic device according to claim 12, wherein the actions further comprise: Before obtaining the data block deduplication information of the source data and the plurality of backup node deduplication information of the plurality of backup nodes in the backup node set, in response to determining that the source data was previously backed up to a specific backup node in the backup node set, determining a rate of change of the data block deduplication information of the source data; as well as In response to determining that the change rate is greater than a first predetermined threshold, data block deduplication information of the source data and the plurality of backup node deduplication information of the plurality of backup nodes in the backup node set are obtained.
20. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions which, when executed, cause a machine to perform the following actions: Acquire data block deduplication information of source data and multiple backup node deduplication information of multiple backup nodes in the backup node set, wherein the backup node deduplication information in the multiple backup node deduplication information includes data block deduplication information of backed up data in the corresponding backup node; Determine, based on the data block deduplication information of the source data and the deduplication information of the multiple backup nodes, multiple deduplication rates of the source data relative to the multiple backup nodes in the backup node set, wherein a deduplication rate in the multiple deduplication rates indicates a deduplication ratio of the source data when the source data is backed up to the corresponding backup node; as well as A target backup node for backing up the source data is selected from the set of backup nodes based on the multiple deduplication rates for the multiple backup nodes.