Data backup method and device, equipment and storage medium
By splitting data blocks during the data backup process and querying using hash index library, the problem of low data backup efficiency in the existing technology is solved, and the effect of saving storage space and improving backup efficiency is achieved.
Patent Information
- Application Number
- CN202411528087.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-18
AI Technical Summary
In the prior art, when backing up data, all data needs to be transmitted to the backup terminal for deduplication, resulting in a large number of network transmissions and low efficiency.
By dividing the target data stream into multiple data blocks, and building a hash index library on the target source and backup ends separately, querying and sending unique hash index values to avoid duplicate backups.
Saves storage space on the backup side, reduces network transmission volume, and improves data backup efficiency.
Smart Images
Figure CN119473719A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a data backup method, device, equipment and storage medium. Background Art
[0002] With the rapid development of information technology, more and more companies and individuals use computer systems to store data. Therefore, in order to ensure the security and stability of the data, it is usually necessary to back up the data stored in the computer system regularly.
[0003] In the prior art, in order to prevent data from being backed up repeatedly, after the target data stream of the target source end is transmitted to the backup end, the target data stream is usually deduplicated according to the data stored in the backup end, and the deduplicated target data stream is stored in the backup end.
[0004] However, according to the existing technology of performing data deduplication after data is transmitted to the backup end, all data needs to be transmitted to the backup end each time a backup is performed, resulting in a large number of network transmission times and low data backup efficiency. Summary of the invention
[0005] The present invention provides a data backup method, device, equipment and storage medium, which saves storage space on the backup end and improves data backup efficiency.
[0006] In a first aspect, an embodiment of the present invention provides a data backup method, which is applied to a data backup system, wherein the data backup system includes multiple source terminals and backup terminals, and the method includes:
[0007] In the process of backing up the target data stream of the target source end to the backup end, the target data stream is divided into a plurality of target data blocks, and a target hash index value corresponding to each target data block is determined;
[0008] In the target source end hash index library pre-built in the target source end and matching the target data stream, query each target hash index value. The source end hash index library stores the hash index values of the data blocks successfully sent from the source end to the backup end.
[0009] If at least one first-class target hash index value is stored in the target source-side hash index library, only the first-class target hash index values are sent to the backup side to avoid duplicate backup of the source-side data block;
[0010] If at least one second-category target hash index value is not stored in the target source-side hash index library, then each second-category target hash index value is queried in the backup-side hash index library pre-built in the backup-side, and the backup-side hash index library stores the hash index values of the data blocks that the backup-side has successfully backed up from all source ends;
[0011] According to the query results of each second-class target hash index value, it is selected to send only the second-class target hash index value, or the second-class target hash index value and the matching target data block to the standby end.
[0012] Optionally, each first-category target hash index value is sent to the backup end, including: whenever a first-category target hash index value is queried, the first-category hash index value is marked as a duplicate hash index value, and the number value of the current duplicate hash index value is updated; whenever it is detected that the number value reaches a preset upper limit of the number value, each duplicate hash index value is sent to the backup end at one time, and the number value is cleared to zero.
[0013] Optionally, based on the query results of each second-class target hash index value, choose to send only the second-class target hash index value, or the second-class target hash index value and the matching target data block to the standby end, including: if the current second-class target hash index value is not stored in the standby end hash index library, then the current second-class target hash index value and the target data block matching the current second-class target hash index value are sent together to the standby end; if the current second-class target hash index value is stored in the standby end hash index library, then only the second-class target hash index value is sent to the standby end.
[0014] Optionally, after sending the current second-category target hash index value and the target data block matching the current second-category target hash index value to the standby end, it also includes: storing the current second-category target hash index value in the target source end hash index library.
[0015] Optionally, after querying each target hash index value in the target source hash index library pre-built in the target source that matches the target data stream, it also includes: whenever a first-class target hash index value is successfully matched in the target source hash index library, the fingerprint count value of the matched first-class target hash index value in the target source hash index library is increased by one; the method also includes: when it is detected that the storage capacity of the target source hash index library is insufficient, after sorting each hash index value in the target source hash index library in ascending order of the fingerprint count value, each hash index value in the target source hash index library is deleted in the sorted order until the storage capacity of the target source hash index library meets the demand.
[0016] Optionally, the method also includes: obtaining at least one data root directory to which each data stored in each source end belongs, and creating at least one source end hash index library corresponding to each source end according to the name of the data root directory; after determining the target hash index value corresponding to each target data block, it also includes: determining the target name of the target data root directory to which each target data block belongs; according to the target name, obtaining the target source end hash index library matching the target data stream from each source end hash index library matching the target source end.
[0017] Optionally, the method further includes: after detecting prompt information that the backup end has deleted the data block, obtaining a target root directory corresponding to the data block deleted by the backup end, and deleting a source end hash index library corresponding to the target root directory.
[0018] In a second aspect, an embodiment of the present invention further provides a data backup device, which is applied to a data backup system, wherein the data backup system includes multiple source terminals and backup terminals, and the device includes:
[0019] A target index value determination module is used to divide the target data stream into multiple target data blocks and determine the target hash index value corresponding to each target data block during the process of backing up the target data stream of the target source end to the backup end;
[0020] A source-side query module is used to query each target hash index value in a target source-side hash index library pre-built in the target source and matching the target data stream. The source-side hash index library stores the hash index values of data blocks successfully sent from the source end to the backup end.
[0021] A first backup module, configured to send only the first-class target hash index values to the backup end if at least one first-class target hash index value is stored in the target source end hash index library, so as to avoid duplicate backup of the source end data block;
[0022] A standby query module, for querying each second-class target hash index value in a standby hash index library pre-built in the standby end if at least one second-class target hash index value is not stored in the target source end hash index library, and the standby hash index library stores the hash index values of the data blocks successfully backed up by the standby end from all the source ends;
[0023] The second backup module is used to select and send only the second-category target hash index value, or the second-category target hash index value and the matching target data block to the backup end according to the query result of each second-category target hash index value.
[0024] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0025] at least one processor; and
[0026] a memory communicatively connected to at least one processor; wherein,
[0027] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the data backup method provided by any embodiment of the present invention.
[0028] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the data backup method provided by any embodiment of the present invention when executed.
[0029] The technical solution of the embodiment of the present invention first determines whether the data blocks corresponding to each target hash index value have been stored in the backup end through the target source end hash index library and the backup end hash index library, which can avoid duplicate backup of data blocks and save storage space on the backup end. Secondly, by first performing a preliminary deduplication in the target source end hash index library, the transmission volume and backup time of the backup data can be reduced, thereby improving the backup efficiency.
[0030] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0032] Figure 1 is a flow chart of a data backup method provided according to Embodiment 1 of the present invention;
[0033] Figure 2 is a flow chart of another data backup method provided according to Embodiment 2 of the present invention;
[0034] Figure 3 is a structural diagram of a data backup device provided according to Embodiment 3 of the present invention;
[0035] Figure 4 It is a schematic diagram of the structure of an electronic device provided by Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0038] Embodiment 1
[0039] Figure 1 This is a flow chart of a data backup method provided according to Embodiment 1 of the present invention. This embodiment is applicable to the case of backing up data from a source end to a backup end. The method can be executed by a data backup device. The data backup device can be implemented in the form of hardware and / or software. The data backup device can be configured in an electronic device such as a computer.
[0040] like Figure 1 As shown, a data backup method disclosed in this embodiment is applied to a data backup system, wherein the data backup system includes multiple source ends and backup ends. The method includes:
[0041] S110. In the process of backing up the target data stream of the target source end to the backup end, the target data stream is divided into a plurality of target data blocks, and a target hash index value corresponding to each target data block is determined.
[0042] In this embodiment, the source end may be a party that needs to perform data backup, the backup end may be a party that needs to store backup data sent by at least one source end, and the target source end may be a source end that currently needs to perform data backup.
[0043] In this step, specifically, after obtaining each source disk in the target source, the target data stream can be obtained in the same root directory or different root directories of the same source disk. Then, the target data stream can be divided into multiple target data blocks according to a predefined block strategy. There can be multiple predefined block strategies, for example, the target data stream is divided into blocks according to a preset data block size or a preset number of blocks. Finally, the target hash index value corresponding to each target data block can be determined according to a predefined hash algorithm.
[0044] S120. In a target source end hash index library pre-built in the target source end and matching the target data stream, query each target hash index value.
[0045] In this embodiment, the source end hash index library stores the hash index values of the data blocks successfully sent from the source end to the backup end. The same source end hash index library can only be opened by one source end process, and the threads in the source end process share the source end hash index library.
[0046] In this step, specifically, the source deduplication thread in the source process can receive the data blocks sent by the hash value calculation thread and the target hash index values corresponding to each target data block, and query whether each target hash index value is stored in the target source hash index library.
[0047] S130. If at least one first-category target hash index value is stored in the target source-end hash index library, only each first-category target hash index value is sent to the backup end to avoid duplicate backup of the source-end data block.
[0048] In this embodiment, the first type of target hash index value may be an index value that is the same as any hash index value stored in the target source-end hash index library.
[0049] In this step, specifically, when the number of first-category target hash index values reaches a preset upper limit, each first-category target hash index value may be sent to the standby end. Alternatively, each first-category target hash index value may be sent to the standby end periodically.
[0050] The advantage of this setting is that if the first-class target hash index value is stored in the target source hash index library, it can be considered that the first-class target hash index value is duplicated, that is, the data block corresponding to the first-class target hash index value has been backed up to the backup end. Therefore, by only sending each first-class target hash index value to the backup end, the source end data block can be avoided from being backed up repeatedly.
[0051] S140: If at least one second-category target hash index value is not stored in the target source-end hash index library, query each second-category target hash index value in the standby-end hash index library pre-built in the standby-end.
[0052] In this embodiment, the backup end hash index stores the hash index values of the data blocks successfully backed up by the backup end from all the source ends. The second type of target hash index value may be an index value that is different from any hash index value stored in the target source end hash index library.
[0053] In this step, specifically, if at least one second-category target hash index value is not stored in the target source-side hash index library, the second-category target hash index value is sent to the standby-side query thread in the source-side process for processing, and the standby-side query thread is used to query whether the second-category target hash index value is stored in the standby-side hash index library.
[0054] S150. According to the query results of each second-category target hash index value, select to send only the second-category target hash index value, or the second-category target hash index value and the matching target data block to the standby end.
[0055] In this step, specifically, if the current second-class target hash index value is not stored in the standby-end hash index library, the current second-class target hash index value and the target data block matching the current second-class target hash index value are sent to the standby-end together. If the current second-class target hash index value is stored in the standby-end hash index library, only the second-class target hash index value is sent to the standby-end.
[0056] The technical solution of this embodiment is to divide the target data stream into multiple target data blocks and determine the target hash index value corresponding to each target data block in the process of backing up the target data stream of the target source end to the backup end; query each target hash index value in the target source end hash index library pre-built in the target source end and matching the target data stream, and the source end hash index library stores the hash index values of the data blocks successfully sent from the source end to the backup end; if at least one first-class target hash index value is stored in the target source end hash index library, only each first-class target hash index value is sent to the backup end to avoid duplicate backup of the source end data block; if at least one second-class target hash index value is stored in the target source end hash index library, then only each first-class target hash index value is sent to the backup end to avoid duplicate backup of the source end data block; If the value is not stored in the target source hash index library, then each second-category target hash index value is queried in the backup hash index library pre-built in the backup end. The backup hash index library stores the hash index values of the data blocks that the backup end has successfully backed up from all source ends. According to the query results of each second-category target hash index value, a technical means is selected to send only the second-category target hash index value, or the second-category target hash index value and the matching target data block to the backup end. This solves the problem of existing technology of deduplication of data after data is transmitted to the backup end, in which all data needs to be transmitted to the backup end each time a backup is performed, resulting in low data backup efficiency. This saves storage space on the backup end and improves data backup efficiency.
[0057] Embodiment 2
[0058] Figure 2 It is a flowchart of another data backup method provided according to the second embodiment of the present invention. This embodiment is a further optimization and expansion based on the above embodiments, and can be combined with various optional technical solutions in the above implementation methods.
[0059] like Figure 2 As shown, a data backup method disclosed in this embodiment is applied to a data backup system, wherein the data backup system includes multiple source ends and backup ends. The method includes:
[0060] S210. In the process of backing up the target data stream of the target source end to the backup end, the target data stream is divided into a plurality of target data blocks, and a target hash index value corresponding to each target data block is determined.
[0061] S220. In a target source end hash index library pre-built in the target source end and matching the target data stream, query each target hash index value.
[0062] The source-side hash index library stores hash index values of data blocks successfully sent from the source side to the backup side.
[0063] In this step, specifically, a target source-side hash index library matching the target data stream may be obtained, and a query may be made as to whether each target hash index value is stored in the target source-side hash index library.
[0064] Optionally, before obtaining the target source hash index library that matches the target data stream, it also includes: obtaining at least one source disk to which each data stored in each source belongs, and creating at least one source hash index library corresponding to each source according to the name of the source disk. Alternatively, obtaining at least one data root directory to which each data stored in each source belongs, and creating at least one source hash index library corresponding to each source according to the name of the data root directory.
[0065] Further, obtaining the target source-side hash index library matching the target data stream includes: determining the target source-side disk to which each target data block corresponding to the target data stream belongs, and obtaining the target source-side hash index library matching the target data stream from each source-side hash index library matching the target source according to the name of the target source-side disk. Alternatively, determining the target name of the target data root directory to which each target data block corresponding to the target data stream belongs, and obtaining the target source-side hash index library matching the target data stream from each source-side hash index library matching the target source according to the target name.
[0066] Furthermore, it is queried whether each target hash index value is stored in the target source hash index library.
[0067] The advantage of this setting is that by creating a source-side hash index library for each disk or each root directory, the efficiency of querying each target hash index value can be improved.
[0068] Optionally, after querying each target hash index value in the target source hash index library pre-built in the target source that matches the target data stream, it also includes: whenever a first-class target hash index value is successfully matched in the target source hash index library, the fingerprint count value of the matched first-class target hash index value in the target source hash index library is increased by one; the method also includes: when it is detected that the storage capacity of the target source hash index library is insufficient, after sorting each hash index value in the target source hash index library in ascending order of the fingerprint count value, each hash index value in the target source hash index library is deleted in the sorted order until the storage capacity of the target source hash index library meets the demand.
[0069] The fingerprint count value may be used to reflect the number of times the data block corresponding to the first type target hash index value is requested to be backed up.
[0070] In a specific embodiment, when it is detected that the storage capacity of the target source hash index library is insufficient, it can be determined whether the target source hash index library is being used. If the target source hash index library is being used, the target source hash index library can be made to refuse to respond to hash index value addition, deletion, and modification operations, but support hash index value query operations. If the target source hash index library is not being used, the hash index values in the target source hash index library that are in the front of the sorting order can be deleted according to the preset deletion number. Alternatively, only the hash index values in the target source hash index library whose fingerprint count value is less than or equal to 1 can be deleted.
[0071] The advantage of such a setting is that, when a target source is backed up regularly, there may be a request for repeated backup of the same data block. Therefore, the technical solution of this embodiment, by adjusting the fingerprint count value of the matched first-class target hash index value in the target source hash index library, can provide a basis for deleting the hash index value when the storage capacity of the target source hash index library is insufficient. Secondly, by preferentially deleting the hash index values with a higher sorting order in the target source hash index library, it is possible to avoid missing detection of data blocks with high duplicate backup requirements, thereby reducing the number of further data deduplication based on the backup hash index library and improving data backup efficiency.
[0072] S230. If at least one first-class target hash index value is stored in the target source-end hash index library, whenever a first-class target hash index value is queried, the first-class hash index value is marked as a duplicate hash index value, and the number of current duplicate hash index values is updated.
[0073] In this step, specifically, whenever a first-category target hash index value is queried, the first-category hash index value is marked as a repeated hash index value, and the number of current repeated hash index values is increased by one.
[0074] S240. Whenever it is detected that the quantity value reaches a preset upper limit of the quantity value, each repeated hash index value is sent to the standby end at one time, and the quantity value is cleared to zero.
[0075] In this step, specifically, after each update of the current number of repeated hash index values, it can be determined whether the number of the current repeated hash index values has reached the preset upper limit of the number. If so, each repeated hash index value is sent to the backup end at one time, and the number is cleared to zero. If not, it can be determined whether the instruction for the backup to be closed is received. If the instruction for the backup to be closed is received, all repeated hash index values can be sent to the backup end. If the instruction for the backup to be closed is not received, it returns to execute the operation of determining whether the number of the current repeated hash index values has reached the preset upper limit of the number, until the number of the current repeated hash index values reaches the preset upper limit of the number, or the instruction for the backup to be closed is received.
[0076] S250: If at least one second-category target hash index value is not stored in the target source-end hash index library, query each second-category target hash index value in the standby-end hash index library pre-built in the standby-end.
[0077] The backup-side hash index stores hash index values of data blocks that are successfully backed up by the backup side from all source sides.
[0078] In this step, specifically, each second-class target hash index value can be used as the current second-class target hash index value in turn, and a query can be made as to whether the current second-class target hash index value is stored in the standby-end hash index library pre-built in the standby end. It is determined whether the query processing of all second-class target hash index values corresponding to the target source end is completed. If not, it returns to execute the operation of using each second-class target hash index value as the current second-class target hash index value in turn until the query processing of all second-class target hash index values corresponding to the target source end is completed.
[0079] The advantage of this setting is that in the case of repeated backup of the same source, by further deduplication only on data blocks that are not found in the target source hash index library, the number of network transmissions can be reduced, data redundancy in the backup process can be reduced, and backup efficiency and storage utilization can be improved.
[0080] It should be noted that the technical solution of this embodiment supports repeated backup of each source end.
[0081] S260: If the current second-category target hash index value is not stored in the standby-end hash index library, the current second-category target hash index value and the target data block matching the current second-category target hash index value are sent to the standby-end together.
[0082] Optionally, after sending the current second-category target hash index value and the target data block matching the current second-category target hash index value to the standby end, it also includes: storing the current second-category target hash index value in the target source end hash index library.
[0083] In a specific implementation, in order to ensure that the target source hash index library and the backup hash index library correspond, the current second-class target hash index value can be stored in the target source hash index library only after the current second-class target hash index value and the target data block matching the current second-class target hash index value are stored together in the backup end.
[0084] The advantage of this setting is that by storing the current second-category target hash index value in the target source hash index library, preliminary deduplication can be performed in the target source hash index library, thereby reducing the backup time and the network bandwidth occupied.
[0085] S270: If the current second-category target hash index value is stored in the standby-end hash index library, only the second-category target hash index value is sent to the standby-end.
[0086] Optionally, each time a second-type target hash index value is sent to the standby end, the fingerprint count value of the second-type target hash index value in the standby end hash index library is incremented by one.
[0087] The advantage of this setting is that the fingerprint count value is set by the second type of target hash index value in the backup hash index library. Each time the user's instruction to delete the target data block is received, the fingerprint count value can be reduced by one until the indicator count value corresponding to the target data block is 0, and then the target data block is completely deleted.
[0088] Optionally, the method further includes: after detecting prompt information that the backup end has deleted the data block, obtaining a target root directory corresponding to the data block deleted by the backup end, and deleting a source end hash index library corresponding to the target root directory.
[0089] In a specific implementation, since the process of determining the hash index value corresponding to the data block deleted on the backup side is relatively complicated, the source-side hash index library corresponding to the target root directory can be directly deleted to avoid the situation where at least one data block has been deleted on the backup side, but the source-side hash index library still retains the hash index values corresponding to the at least one data block, thereby causing data backup failure.
[0090] The above technical solution can reduce the network bandwidth occupied by sending each repeated hash index value to the backup end at one time whenever the quantity value reaches the preset upper limit of the quantity value, thereby improving the efficiency of backing up the second type of target hash index value and the target data block matching the second type of target hash index value to the backup end. Secondly, by further deduplication operations only on data blocks that are not found in the target source end hash index library, the number of network transmissions can be reduced, data redundancy in the backup process can be reduced, and backup efficiency can be improved.
[0091] Embodiment 3
[0092] Figure 3 It is a structural diagram of a data backup device provided according to Example 3 of the present invention. This embodiment can be applied to the situation of backing up data from the source end to the backup end. The data backup device can be implemented in the form of hardware and / or software and can be configured in an electronic device such as a computer.
[0093] like Figure 3 As shown, the data backup device disclosed in this embodiment is applied to a data backup system, and the data backup system includes multiple source ends and backup ends. The device includes:
[0094] The target index value determination module 31 is used to divide the target data stream into a plurality of target data blocks and determine the target hash index value corresponding to each target data block in the process of backing up the target data stream of the target source end to the backup end;
[0095] A source end query module 32, used to query each target hash index value in a target source end hash index library pre-built in the target source end and matching the target data stream, the source end hash index library storing the hash index values of the data blocks successfully sent from the source end to the backup end;
[0096] A first backup module 33 is configured to send only the first-class target hash index values to the backup end if at least one first-class target hash index value is stored in the target source end hash index library, so as to avoid duplicate backup of the source end data block;
[0097] The standby query module 34 is used to query each second-class target hash index value in a standby hash index library pre-built in the standby end if at least one second-class target hash index value is not stored in the target source end hash index library, and the standby end hash index library stores the hash index values of the data blocks successfully backed up by the reserve end from all the source ends;
[0098] The second backup module 35 is used to select and send only the second type target hash index value, or the second type target hash index value and the matching target data block to the backup end according to the query result of each second type target hash index value.
[0099] The technical solution in this embodiment solves the problem of low data backup efficiency in the prior art of deduplication of data after data is transmitted to the backup end, through the mutual cooperation of the target index value determination module, the source end query module, the first backup module, the backup end query module and the second backup module. This saves storage space on the backup end and improves data backup efficiency.
[0100] Optionally, the first backup module 33 is specifically used to: whenever a first-category target hash index value is queried, mark the first-category hash index value as a duplicate hash index value, and update the number value of the current duplicate hash index value; whenever it is detected that the number value reaches a preset upper limit of the number value, send each duplicate hash index value to the backup end at one time, and clear the number value to zero.
[0101] Optionally, the second backup module 35 is specifically used to: if the current second-class target hash index value is not stored in the backup-end hash index library, then the current second-class target hash index value and the target data block matching the current second-class target hash index value are sent to the backup end together; if the current second-class target hash index value is stored in the backup-end hash index library, then only the second-class target hash index value is sent to the backup end.
[0102] Optionally, the device further includes a source index library update module, a count value update module, a source index value deletion module, a source index library creation module, a source index library selection module and a data block deletion module, wherein:
[0103] The source-side index library update module is used to store the current second-category target hash index value in the target source-side hash index library.
[0104] A count value updating module, used for increasing the fingerprint count value of the matched first-class target hash index value in the target source-end hash index library by one whenever a first-class target hash index value is successfully matched in the target source-end hash index library;
[0105] The source-side index value deletion module is used to sort the hash index values in the target source-side hash index library in ascending order of fingerprint count values when it is detected that the storage capacity of the target source-side hash index library is insufficient, and then delete the hash index values in the target source-side hash index library in the sorted order until the storage capacity of the target source-side hash index library meets the demand;
[0106] A source-end index library creation module, used to obtain at least one data root directory to which each data stored in each source end belongs, and create at least one source-end hash index library corresponding to each source end according to the name of the data root directory;
[0107] The source-end index library selection module is used to determine the target name of the target data root directory to which each target data block belongs; according to the target name, from each source-end hash index library matching the target source end, obtain the target source-end hash index library matching the target data stream;
[0108] The data block deletion module is used to obtain the target root directory corresponding to the data block deleted by the backup end after detecting the prompt information of data block deletion on the backup end, and delete the source end hash index library corresponding to the target root directory
[0109] The data backup device provided in the embodiment of the present invention can execute the data backup method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not described in detail in this embodiment, reference can be made to the description in any method embodiment of the present application.
[0110] Embodiment 4
[0111] Figure 4 FIG. 1 is a schematic diagram showing the structure of an electronic device 10 that can be used to implement an embodiment of the present invention. Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0112] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0113] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data backup method.
[0114] In some embodiments, the data backup method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data backup method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data backup method in any other appropriate manner (e.g., by means of firmware).
[0115] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0116] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0117] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0118] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0119] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0120] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0121] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0122] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A data backup method, characterized in that: Applied to a data backup system, the data backup system includes multiple source terminals and backup terminals, and the method includes: In the process of backing up the target data stream of the target source end to the backup end, the target data stream is divided into a plurality of target data blocks, and a target hash index value corresponding to each target data block is determined; In the target source end hash index library pre-built in the target source end and matching the target data stream, query each target hash index value. The source end hash index library stores the hash index values of the data blocks successfully sent from the source end to the backup end. If at least one first-class target hash index value is stored in the target source-side hash index library, only the first-class target hash index values are sent to the backup side to avoid duplicate backup of the source-side data block; If at least one second-category target hash index value is not stored in the target source-side hash index library, then each second-category target hash index value is queried in the backup-side hash index library pre-built in the backup-side, and the backup-side hash index library stores the hash index values of the data blocks that the backup-side has successfully backed up from all source ends; According to the query results of each second-class target hash index value, it is selected to send only the second-class target hash index value, or the second-class target hash index value and the matching target data block to the standby end.
2. The method according to claim 1, characterized in that Send each first-category target hash index value to the standby end, including: Whenever a first-category target hash index value is queried, the first-category hash index value is marked as a duplicate hash index value, and the number of current duplicate hash index values is updated; Whenever it is detected that the quantity value reaches a preset upper limit of the quantity value, each repeated hash index value is sent to the standby end at one time, and the quantity value is cleared to zero.
3. The method according to claim 1 or 2, characterized in that: According to the query result of each second-class target hash index value, selecting to send only the second-class target hash index value, or the second-class target hash index value and the matching target data block to the standby end includes: If the current second-category target hash index value is not stored in the standby end hash index library, the current second-category target hash index value and the target data block matching the current second-category target hash index value are sent to the standby end together; If the current second-category target hash index value is stored in the standby-end hash index library, only the second-category target hash index value is sent to the standby-end.
4. The method according to claim 3, characterized in that: After the current second-category target hash index value and the target data block matching the current second-category target hash index value are sent to the standby end, the method further includes: The current second-category target hash index value is stored in the target source-side hash index library.
5. The method according to any one of claims 1 to 4, characterized in that: After querying each target hash index value in the target source end hash index library pre-built in the target source end and matching the target data stream, the following steps are also included: Whenever a first-class target hash index value is successfully matched in the target source-side hash index library, the fingerprint count value of the matched first-class target hash index value in the target source-side hash index library is increased by one; The method further comprises: When it is detected that the storage capacity of the target source hash index library is insufficient, each hash index value in the target source hash index library is sorted in ascending order of the fingerprint count value, and then each hash index value in the target source hash index library is deleted in the sorted order until the storage capacity of the target source hash index library meets the demand.
6. The method according to claim 1, characterized in that The method further comprises: Obtain at least one data root directory to which each data stored in each source end belongs, and create at least one source end hash index library corresponding to each source end according to the name of the data root directory; After determining the target hash index value corresponding to each target data block, the method further includes: Determine the target name of the target data root directory to which each target data block belongs; According to the target name, obtain the target source hash index library that matches the target data stream from each source hash index library that matches the target source.
7. The method according to claim 6, characterized in that The method further comprises: After detecting the prompt information of data block deletion on the standby side, the target root directory corresponding to the data block deleted on the standby side is obtained, and the source-side hash index library corresponding to the target root directory is deleted.
8. A data backup device, characterized in that: Applied to a data backup system, the data backup system includes multiple source terminals and backup terminals, and the device includes: A target index value determination module is used to divide the target data stream into multiple target data blocks and determine the target hash index value corresponding to each target data block during the process of backing up the target data stream of the target source end to the backup end; A source-side query module is used to query each target hash index value in a target source-side hash index library pre-built in the target source and matching the target data stream. The source-side hash index library stores the hash index values of data blocks successfully sent from the source end to the backup end. A first backup module, configured to send only the first-class target hash index values to the backup end if at least one first-class target hash index value is stored in the target source end hash index library, so as to avoid duplicate backup of the source end data block; A standby query module, for querying each second-class target hash index value in a standby hash index library pre-built in the standby end if at least one second-class target hash index value is not stored in the target source end hash index library, and the standby hash index library stores the hash index values of the data blocks successfully backed up by the standby end from all the source ends; The second backup module is used to select and send only the second-category target hash index value, or the second-category target hash index value and the matching target data block to the backup end according to the query result of each second-category target hash index value.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data backup method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data backup method according to any one of claims 1 to 7 when executed.