Data backup method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data backup method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: in response to a data backup request initiated by a user, obtaining to-be-backed-up data sent by the user and partition sizes of a plurality of candidate backup devices expected to store the to-be-backed-up data; segmenting the to-be-backed-up data according to the partition size to obtain a plurality of data slices corresponding to the to-be-backed-up data; for each data slice, determining a target backup device matched with the data slice from the candidate backup devices; and in a plurality of device partitions contained in the target backup device, screening a target partition corresponding to the data slice, and storing the data slice in the target partition. By adopting the method, the controllability of data backup can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data management, and in particular to a data backup method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of data management technology, data backup technology has emerged, which can copy the data that users need to store and store it in other locations so that it can be restored when the original data is lost or damaged.

[0003] Traditional data backup methods generally use object storage to back up data. However, during the object storage process, data is stored in a dispersed manner on multiple servers, and the storage location of the data cannot be directly controlled, resulting in poor controllability of data backup. Summary of the invention

[0004] Based on this, it is necessary to provide a data backup method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the controllability of data backup in response to the above technical problems.

[0005] In a first aspect, the present application provides a data backup method, comprising:

[0006] In response to a data backup request initiated by a user, obtaining the to-be-backed-up data sent by the user and the partition sizes of a plurality of candidate backup devices expected to store the to-be-backed-up data;

[0007] Dividing the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up;

[0008] For each of the data slices, determining a target backup device that matches the data slice from the candidate backup devices;

[0009] Among the multiple device partitions included in the target backup device, a target partition corresponding to the data slice is selected, and the data slice is stored in the target partition.

[0010] In one embodiment, for each of the data slices, determining a target backup device matching the data slice from the candidate backup devices includes:

[0011] Sorting the candidate backup devices according to their respective device addresses to obtain a device sequence;

[0012] According to the position of each candidate backup device in the device sequence, taking out the first device ranked first from the device sequence;

[0013] If the device state of the first device is in a fully loaded state, returning to the step of taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence;

[0014] If the device state of the first device is not fully loaded, the first device is used as a target backup device matching the data slice.

[0015] In one of the embodiments, selecting a target partition corresponding to the data slice from among multiple device partitions included in the target backup device includes:

[0016] Determine a target partition from each device partition according to the locations of the multiple device partitions included in the target backup device in the target backup device;

[0017] All device partitions before the target partition are in an unavailable state, and the target partition is in an available state.

[0018] In one embodiment, the method further comprises:

[0019] For the used partitions storing historical data slices in each device partition of the target backup device, obtaining the data status of each historical data slice;

[0020] In the case where each of the data states includes a to-be-reclaimed state, deleting the historical data slices in the to-be-reclaimed state in the target backup device;

[0021] The partition state of the used partition corresponding to the historical data slice is updated from an unavailable state to an available state.

[0022] In one embodiment, the method further comprises:

[0023] For each of the candidate backup devices, determining the partition status of each of the plurality of device partitions included in the candidate backup device; the partition status includes an available status and an unavailable status;

[0024] Determine the number of available partitions and the number of unavailable partitions of each candidate backup device based on the partition status of each device partition;

[0025] If, among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, the data slices stored in the second candidate backup device are migrated to the first candidate backup device.

[0026] In one of the embodiments, each of the candidate backup devices corresponds to a plurality of partition sizes;

[0027] The step of determining, for each of the data slices, a target backup device that matches the data slice from the candidate backup devices includes:

[0028] For each of the data slices, the data volume of the data slice is matched with each of the partition sizes, and a selected backup device having a partition size matching the data slice is determined from the candidate backup devices;

[0029] According to the respective device addresses of the selected backup devices, a target backup device for storing the data slice is determined from the selected backup devices.

[0030] In a second aspect, the present application also provides a data backup device, including:

[0031] A data acquisition module, configured to respond to a data backup request initiated by a user and acquire the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up;

[0032] A data segmentation module, used to segment the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up;

[0033] A target backup device determination module, configured to determine, for each of the data slices, a target backup device that matches the data slice from among the candidate backup devices;

[0034] The target partition screening module is used to screen the target partition corresponding to the data slice from the multiple device partitions included in the target backup device, and store the data slice in the target partition.

[0035] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0037] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0038] The above-mentioned data backup method, apparatus, computer equipment, computer-readable storage medium and computer program product, in response to a data backup request initiated by a user, obtain the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up, can determine the specific situation of the data backup, divide the data to be backed up according to the partition size, and obtain multiple data slices corresponding to the data to be backed up, and can process the data to be backed up in a targeted manner to improve the efficiency of the subsequent data backup process. For each data slice, a target backup device matching the data slice is determined from each candidate backup device, and the target partition corresponding to the data slice is screened from the multiple device partitions contained in the target backup device, and the data slice is stored in the target partition, which can achieve the purpose of strictly controlling the data storage location and improve the controllability of data backup. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 An application environment diagram of a data backup method in an embodiment;

[0041] Figure 2 A schematic diagram of a data backup method in one embodiment;

[0042] Figure 3 A diagram of the overall cloud backup business architecture of a data backup method in an embodiment;

[0043] Figure 4 A block backend backup library functional module diagram of a data backup method in an embodiment;

[0044] Figure 5 A flowchart of reading and writing a block backend backup library of a data backup method in one embodiment;

[0045] Figure 6 An address mapping diagram of a data backup method in one embodiment;

[0046] Figure 7 A device partition state transition diagram of a data backup method in an embodiment;

[0047] Figure 8 is a flow chart of a data backup method in another embodiment;

[0048] Fig. 9is a structural block diagram of a data backup device in one embodiment;

[0049] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The data backup method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. Specifically, during the data backup process, the user can initiate a data backup request based on the terminal 102, and the server 104 communicates with the terminal 102 to obtain the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices that are expected to store the data to be backed up; the data to be backed up is divided according to the partition size to obtain multiple data slices corresponding to the data to be backed up; for each data slice, a target backup device matching the data slice is determined from each candidate backup device; among the multiple device partitions contained in the target backup device, the target partition corresponding to the data slice is screened, and the data slice is stored in the target partition.

[0052] In an exemplary embodiment, Figure 2 As shown, a data backup method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:

[0053] Step S202: in response to a data backup request initiated by a user, obtaining the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up.

[0054] Among them, a data backup request generally refers to a request or application initiated by a user, which is intended to create a copy of the data and retain it on a secure storage medium to prevent the original data from being lost or inaccessible due to various reasons (such as deletion, overwriting, failure or disaster). The data to be backed up is the data that needs to be copied and retained. Candidate backup devices refer to various storage devices or systems that may be selected for use during the data backup process. For example, the types of candidate backup devices can be cloud hard disks, physical hard disks, and other block media. Partition size refers to the data storage size of each device partition in the candidate backup device after the candidate backup device is divided into regions. Exemplarily, the candidate backup device can be partitioned with the same fixed size, for example, each candidate backup device is managed with a fixed size slice (20MB). Taking the backup library capacity of 20TB as an example, a total of 200 candidate backup devices are required, each with a capacity of 100GB and 5120 device partitions. It can be understood that the candidate backup device can also be partitioned with different fixed sizes, which can be determined according to actual conditions and are not limited here.

[0055] Specifically, when a user has a need for data backup, the user can initiate a data backup request based on the terminal. After receiving the data backup request, the server can obtain the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up, so as to determine the specific situation of the data backup. It can be understood that the action of obtaining the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up can be active acquisition or passive reception.

[0056] Step S204 , dividing the data to be backed up according to the partition size to obtain a plurality of data slices corresponding to the data to be backed up.

[0057] Data slicing refers to splitting a data set according to specific rules or conditions to obtain multiple more detailed data subsets.

[0058] Specifically, after obtaining the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices that are expected to store the data to be backed up, the server can divide the data to be backed up according to the above partition sizes, thereby obtaining multiple data slices corresponding to the divided data to be backed up. In some specific embodiments, each candidate backup device may correspond to the same partition size or to different partition sizes. It can be understood that the data slices obtained by dividing according to the same partition size have the same data volume, that is, multiple data slices with the same data volume are obtained. The data slices obtained by dividing according to different partition sizes have different data volumes, that is, multiple data slices with different data volumes are obtained.

[0059] Step S206: for each data slice, determine a target backup device that matches the data slice from the candidate backup devices.

[0060] The target backup device refers to a device or system selected to store backup data during the data backup process.

[0061] Specifically, after the data to be backed up is segmented into multiple data slices, the server can match each data slice with a target backup device corresponding to the data slice. It can be understood that the data slice will be subsequently stored in the target backup device.

[0062] In some specific embodiments, the candidate backup devices may correspond to the same partition size or different partition sizes. When the candidate backup devices correspond to the same partition size, the server may determine the target backup devices corresponding to the data slices in sequence according to a certain order. When the candidate backup devices correspond to different partition sizes, the server may determine the target backup devices corresponding to the data slices according to the matching degree between the partition sizes corresponding to the candidate backup devices and the data volume of the data slices.

[0063] Step S208, selecting a target partition corresponding to the data slice from among the multiple device partitions included in the target backup device, and storing the data slice in the target partition.

[0064] The target partition refers to the device partition selected to store the backup data during the data backup process.

[0065] Specifically, after determining the target backup device corresponding to the data slice, you can first determine which device partitions the target backup device contains, then filter out the target partition corresponding to the data slice from these device partitions, store the data slice in the target partition of the target backup device, and complete a data backup.

[0066] In a specific embodiment, the server can determine the target partition from each device partition according to the location of multiple device partitions contained in the target backup device in the target backup device, and each device partition before the target partition is in an unavailable state, and the target partition is in an available state. In another specific embodiment, the server can also select the target partition corresponding to the data slice according to the matching relationship between the data type of the data slice and the partition storage data type of each device partition.

[0067] The above-mentioned data backup method responds to the data backup request initiated by the user, obtains the data to be backed up sent by the user, and the partition sizes of multiple candidate backup devices expected to store the data to be backed up, and can determine the specific situation of the data backup, divide the data to be backed up according to the partition size, and obtain multiple data slices corresponding to the data to be backed up. The data to be backed up can be processed in a targeted manner to improve the efficiency of the subsequent data backup process. For each data slice, a target backup device matching the data slice is determined from each candidate backup device, and the target partition corresponding to the data slice is screened from the multiple device partitions contained in the target backup device, and the data slice is stored in the target partition, which can achieve the purpose of strictly controlling the data storage location and improve the controllability of data backup.

[0068] In an exemplary embodiment, for each data slice, a target backup device matching the data slice is determined from each candidate backup device, including: sorting each candidate backup device according to its respective device address to obtain a device sequence; taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence; if the device status of the first device is fully loaded, returning to the step of taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence; if the device status of the first device is not fully loaded, taking the first device as the target backup device matching the data slice.

[0069] The device sequence refers to a group of candidate backup devices with a certain arrangement order. The first device is the candidate backup device ranked first in the device sequence. The full load state means that the data stored in the candidate backup device completely occupies the data space that the candidate backup device can back up. The non-full load state means that the data stored in the candidate backup device does not completely occupy the data space that the candidate backup device can back up.

[0070] Specifically, after determining the multiple data slices corresponding to the data to be backed up, the server can obtain the device address of each candidate backup device, and sort the candidate backup devices according to the above device addresses to obtain a device sequence. After that, the first device ranked first can be taken out in turn according to the position of each candidate backup device in the device sequence. If the device status of the first device is fully loaded, the next first device ranked first is taken out again, and the device status of the first device is re-judged until the device status of the first device is not fully loaded. The first device is used as the target backup device matching the data slice, so that the candidate backup devices in the device sequence can always be guaranteed to perform data backup in the order of arrangement, thereby ensuring the orderliness of data backup.

[0071] In an exemplary embodiment, among multiple device partitions included in a target backup device, a target partition corresponding to a data slice is screened, including: determining the target partition from each device partition according to the locations of the multiple device partitions included in the target backup device in the target backup device; each device partition before the target partition is in an unavailable state, and the target partition is in an available state.

[0072] The unavailable state refers to a state in which data cannot be backed up for the device partition, and the available state refers to a state in which data can be backed up for the device partition.

[0073] Specifically, after determining the target backup device, the server can first determine the multiple device partitions contained in the target backup device and the position of each device partition in the target backup device, and then poll the status of each device partition in turn to ensure that the target partition is in an available state, and all device partitions before the target partition are in an unavailable state, thereby ensuring that the data backup process in the target backup device will not have a single device partition vacant, thereby ensuring the orderliness of data backup.

[0074] In an exemplary embodiment, the data backup method also includes: for used partitions storing historical data slices in each device partition of the target backup device, obtaining the data status of each historical data slice; when each data status includes a to-be-reclaimed state, deleting the historical data slices in the to-be-reclaimed state in the target backup device; and updating the partition status of the used partitions corresponding to the historical data slices from an unavailable state to an available state.

[0075] The historical data slice refers to the data slice that has been backed up in the previous historical data backup process. The data state of the historical data slice can be used to characterize the data situation corresponding to the historical data slice. For example, after the historical data slice has been backed up to the preset period, the historical data slice needs to be cleaned up. At this time, the data state corresponding to the historical data slice is the state to be recovered. It can be understood that the data state can also include other states, such as abnormal state, etc., which are not limited here.

[0076] Specifically, since the data capacity of the candidate backup device is certain, and data backup is always ongoing, in order to ensure the orderly progress of data backup, some historical data slices that have been backed up for a long time can be recycled. Therefore, the server can obtain the data status of each historical data slice for the used partitions storing historical data slices in each device partition of the target backup device. When each data status includes a to-be-recovered status, the historical data slices in the to-be-recovered status are deleted in the target backup device, and the partition status of the used partitions corresponding to the historical data slices is updated from an unavailable status to an available status, thereby ensuring the continuity of the data backup process.

[0077] In an exemplary embodiment, the data backup method also includes: for each candidate backup device, determining the partition status of each of the multiple device partitions contained in the candidate backup device; the partition status includes an available status and an unavailable status; based on the partition status of each device partition, determining the number of available partitions and the number of unavailable partitions of each candidate backup device; if among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, then migrating the data slices stored in the second candidate backup device to the first candidate backup device.

[0078] The number of available partitions refers to the number of device partitions that are in an available state, and similarly, the number of unavailable partitions refers to the number of device partitions that are in an unavailable state.

[0079] Specifically, in the process of data backup and recovery, there will be a situation where the device partitions corresponding to multiple candidate devices all have backed up data slices, but the candidate device is in a non-fully loaded state, which is not conducive to subsequent data backup. Therefore, for each candidate backup device, the server can determine the partition status of each of the multiple device partitions contained in the candidate backup device, and determine the number of available partitions and the number of unavailable partitions of each candidate backup device based on the partition status of each device partition. If, among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, it means that the partition usage of the first candidate backup device is complementary to the partition usage of the second candidate backup device. Therefore, the data slices stored in the second candidate backup device can be migrated to the first candidate backup device, so that the first candidate backup device changes from a non-fully loaded state to a fully loaded state, and the second candidate backup device changes from a non-fully loaded state to an idle state, thereby achieving effective data migration and facilitating the stable progress of subsequent data backup.

[0080] In an exemplary embodiment, each candidate backup device corresponds to a plurality of partition sizes; for each data slice, a target backup device matching the data slice is determined from each candidate backup device, including: for each data slice, the data volume of the data slice is matched with each partition size respectively, and a selected backup device having a partition size matching the data slice is determined from each candidate backup device; and a target backup device for storing the data slice is determined from each selected backup device according to the respective device addresses of each selected backup device.

[0081] The data volume of the data slice is used to characterize the memory size occupied by the data of the data slice. The selected backup device refers to a backup device selected from the candidate backup devices that can be used to store the data slice. The device address refers to a unique identifier corresponding to the selected backup device.

[0082] Specifically, in this embodiment, each candidate backup device corresponds to multiple partition sizes, that is, the partition sizes corresponding to each candidate backup device are not exactly the same. Therefore, the server needs to match the data volume of each data slice with the above-mentioned partition sizes respectively, and select the candidate backup device whose partition size corresponds to the data volume as the selected backup device whose partition size matches the data slice. Since there may be candidate backup devices with the same partition size, that is, there may be multiple selected backup devices, it is also necessary to obtain the device address of each selected backup device, and determine the target backup device for storing the data slice from each selected backup device according to each device address. Among them, the process of determining the target backup device for storing the data slice from each selected backup device according to each device address can be, for example, selecting the selected backup device seat arranged in the first place as the target backup device for storing the data slice according to the arrangement order of each device address.

[0083] In this embodiment, when each candidate backup device corresponds to multiple partition sizes, the data volume of the data slice is matched with each partition size according to the actual situation, and the final target backup device is determined based on the device address of each selected backup device, which can ensure the orderliness of the data slice storage.

[0084] In a specific embodiment, a data backup design method based on an actual application scenario is also provided. Specifically, with the continuous development of computer virtualization technology and the widespread application of distributed technology, cloud computing has gradually become a mature IT operation model known and accepted by the majority of government enterprises and small and medium-sized customers. Various traditional businesses are on the cloud and cloud-native applications are constantly emerging and occupying an important position in various industries, making cloud computing (cloud host, bare metal) and cloud storage (block storage, file storage, object storage) a red ocean. Competition in the cloud computing industry continues to intensify with the maturity of products and technologies.

[0085] As cloud computing and cloud storage are widely used, users have gradually put forward higher requirements for data security and business availability. This is not only reflected in the high availability and high reliability of cloud computing itself. The security of user data and user business across computer rooms and regions cannot be fully met by relying solely on the capabilities of distributed storage itself, especially in the fields of finance, security, etc. There are clear requirements for data protection capability RPO and business recovery capability RTO. In this context, cloud backup and cloud disaster recovery services designed based on cloud computing base came into being.

[0086] As the foundation and core component of cloud backup services, the selection and design of the backup library is particularly important. Through the business analysis of cloud backup, it is not difficult to find that its data storage mode has several significant characteristics, such as fixed-length size, sequential access, more writes and less reads, and expandable capacity.

[0087] Fixed-length size: For the data that users specify to be protected, the backup service usually does not directly write it into the backup library in its original format, but formats the data through fixed-length slicing, deduplication and compression to better meet the requirements of transmission and storage. Therefore, the data stored in the backup library is mostly processed data slices, and may belong to multiple data sources.

[0088] Sequential access: After the user data copies are processed by the backup service, they are usually written to the backup repository sequentially. In the recovery process, a complete data copy is restored, which usually also includes reading several consecutive data slices. The deduplication scenario may be an exception here, and a copy may consist of multiple historical slices. However, compared with the backup frequency, the recovery process runs much less frequently.

[0089] More writes and less reads: As mentioned in the previous article, the main purpose of using cloud backup is to regularly protect key data, so the backup process (write process) will be regularly executed, while the recovery process (read process) will occur more in extreme scenarios such as disaster switching, virus infection, and regular recovery. Therefore, in general, users' write operations on the backup library are far more than read operations.

[0090] Expandable capacity: Since the actual usage of the backup library is affected by many factors such as the amount of change in user business data, data deduplication rate, compression rate, and as the use of backup services continues to grow, the backup library needs to be able to cope with capacity expansion at any time without affecting the storage and indexing of existing backup data during the expansion process.

[0091] Based on the above analysis of the characteristics of cloud backup services, and combined with the advantages and disadvantages of various cloud storage products, we often use object storage as the backup library medium in standard cloud backup services to give full play to its sequential read and write, scalability, and high cost-effectiveness. However, in many smaller resource pools or cloud environments, there are no conditions for deploying object storage, or due to various factors such as data security and business reliability, users require that backup data be stored separately from other business data. In addition, backup / recovery performance is also a key indicator that affects the business in some scenarios, and traditional object storage cannot meet this high-performance scenario. In the above cases, other storage media need to be selected as the backup library media.

[0092] The purpose of this embodiment is to provide a data backup design method based on block storage based on the business characteristics of cloud backup and the product characteristics of block storage, and to use block storage to achieve the same backup library purpose as object storage without changing the basic business logic of user business and cloud backup. It can be understood that the block backup library includes multiple block devices, where the block devices correspond to candidate backup devices.

[0093] Figure 3 This is the overall business logic architecture of cloud backup. As can be seen from the figure: customer business does not need to be aware of the difference in backend backup library types; at the same time, the backup server provides an abstract adaptation layer to connect to different backup library backends.

[0094] The functions that need to be implemented in this solution include: backup library block storage mapping management, backup library block storage garbage collection, and hot and cold data separation migration. The overall process of this solution does not require modification of the existing cloud backup basic business process, but only requires the addition of a new backup library backend type. The internal logic of the above backup library is not perceived by users and backup services.

[0095] The main technical solutions of this solution are divided into three main functional layers, namely: realizing the logical / physical mapping of backup library block devices, realizing the backup library block device slice pool, and realizing data recovery and hot and cold data separation. Figure 4 As shown, by address mapping, the correspondence between data slices and candidate backup devices and backup partitions can be achieved, and the functions of slice allocation, slice recovery, and data stratification can be realized. Specifically, each device (candidate backup device) can be sliced ​​and managed to obtain multiple block storages (device partitions). The cloud backup service can achieve the following goals by using this solution: 1. Implement sequential writing of the block backend backup library; 2. Implement sequential recovery and release of data in the block backend backup library; 3. Implement separation of hot and cold data, providing a basis for subsequent backup library archiving; 4. Implement dynamic expansion of the block backend backup library.

[0096] Specifically, this embodiment mainly includes the following key contents:

[0097] Pooling management of fine-grained slices of block devices: The entire block backend backup library is composed of several block candidate backup devices (cloud hard disks, physical hard disks, and other block media), and each candidate backup device is managed with fixed-size slices (20MB). Taking a 20TB backup library capacity as an example, a total of 200 candidate backup devices are required, each with a capacity of 100GB, including 5120 device partitions. Among them, this embodiment is aimed at various scenarios of unconfigured object storage or even other cloud storage. The candidate backup devices here may be cloud hard disks or physical hard disks. There are various limitations such as the inability to expand the capacity of physical hard disks and the inability to guarantee various indicators after the cloud hard disk reaches a certain capacity. Based on the above reasons, this solution chooses the management method of block device pooling + fine-grained slices within the device.

[0098] Address mapping of logical addresses and physical slices: Compared with the solution where the backup server (server) directly writes to the block storage, the introduction of address mapping relationships can ensure that the underlying continuous sequential read and write recovery always provides a linear address space to the backup server. However, one problem this brings is that additional mapping relationships require memory space, and as the amount of data continues to increase, the mapping relationships need to be persisted regularly. Therefore, we have a two-level mapping relationship between logical / physical addresses: the mapping of logical addresses to block devices. The size of this part of the data is related to the number of block devices and is usually not very large; the other part is the mapping of logical addresses to the serial numbers of device partitions within the block device. This type of mapping depends on the size of a single block device and the granularity of the device partitions. For example Figure 5 As shown in the figure, data slices can be allocated to device partitions, data slices can be recycled, and data slices can be migrated and merged. Overall, the physical address corresponding to the logical address is composed of the serial number of the disk (candidate backup device) + the serial number of the slice in the disk (device partition), as shown in the figure. Figure 6 shown.

[0099] Slice merging and hot and cold data tiering: In this solution, device partitions will be in three and only three states: allocable (available state), data available (unavailable state), pending merging / migration. The first two states are set after the front-end write and data recycling requests are completed, and the third state is handled by separate lifecycle management and continuous scanning logic. Figure 7 As shown, to ensure the continuity of slices, data merging and migration are performed in units of large-grained slice groups.

[0100] In summary, this solution implements linear writing to the backup server by slicing block devices and performing pooling management; through two-level mapping of disk serial number and slice serial number, the mapping information of logical address and physical address is controlled within a controllable range; finally, through the background scanning of the slice life cycle and data continuity, the written data is regularly migrated and maintained vigorously to ensure continuous access to the slice, which also lays the foundation for subsequent data hierarchical archiving storage.

[0101] In a specific embodiment, Figure 8 As shown, a data backup method is also provided, including:

[0102] Step S801, in response to a data backup request initiated by a user, obtaining the to-be-backed-up data sent by the user and the partition sizes of multiple candidate backup devices expected to store the to-be-backed-up data;

[0103] Step S802, dividing the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up;

[0104] Step S803, when each candidate backup device corresponds to multiple partition sizes, for each data slice, the data volume of the data slice is matched with each partition size respectively, and a selected backup device whose partition size matches the data slice is determined from each candidate backup device;

[0105] Step S804, determining a target backup device for storing the data slice from the selected backup devices according to the device addresses of the selected backup devices;

[0106] Step S805, when each candidate backup device corresponds to a partition size, sort the candidate backup devices according to their respective device addresses to obtain a device sequence;

[0107] Step S806, taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence;

[0108] Step S807, determining whether the device status of the first device is in a fully loaded state;

[0109] If yes, return to step S806;

[0110] If not, executing step S808, taking the first device as the target backup device matching the data slice;

[0111] Step S809, for the used partitions storing historical data slices in each device partition of the target backup device, obtain the data status of each historical data slice;

[0112] Step S810: when each data state includes a state to be recycled, deleting the historical data slices in the state to be recycled in the target backup device;

[0113] Step S811, updating the partition state of the used partition corresponding to the historical data slice from an unavailable state to an available state;

[0114] Step S812, according to the locations of the multiple device partitions included in the target backup device in the target backup device, determine the target partition from the device partitions, and store the data slice in the target partition;

[0115] Among them, each device partition before the target partition is in an unavailable state, and the target partition is in an available state;

[0116] Step S813, for each candidate backup device, determining the partition status of each of the multiple device partitions included in the candidate backup device;

[0117] Among them, the partition status includes an available status and an unavailable status;

[0118] Step S814, determining the number of available partitions and the number of unavailable partitions of each candidate backup device based on the partition status of each device partition;

[0119] Step S815: If, among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, the data slices stored in the second candidate backup device are migrated to the first candidate backup device.

[0120] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides a data backup device for implementing the data backup method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data backup device embodiments provided below can refer to the limitations on the data backup method above, and will not be repeated here.

[0122] In an exemplary embodiment, Fig. 9 As shown, a data backup device 900 is provided, including: a data acquisition module 902, a data segmentation module 904, a target backup device determination module 906 and a target partition screening module 908, wherein:

[0123] The data acquisition module 902 is used to respond to a data backup request initiated by a user and acquire the to-be-backed-up data sent by the user and the partition sizes of multiple candidate backup devices expected to store the to-be-backed-up data;

[0124] A data segmentation module 904 is used to segment the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up;

[0125] A target backup device determination module 906 is used to determine, for each of the data slices, a target backup device that matches the data slice from the candidate backup devices;

[0126] The target partition screening module 908 is used to screen the target partition corresponding to the data slice from the multiple device partitions included in the target backup device, and store the data slice in the target partition.

[0127] In an exemplary embodiment, each of the candidate backup devices corresponds to a plurality of partition sizes. In the case of this embodiment, the target backup device determination module 906 is used to:

[0128] For each of the data slices, the data volume of the data slice is matched with each of the partition sizes, and a selected backup device having a partition size matching the data slice is determined from the candidate backup devices;

[0129] According to the respective device addresses of the selected backup devices, a target backup device for storing the data slice is determined from the selected backup devices.

[0130] In an exemplary embodiment, the target backup device determination module 906 is further configured to:

[0131] Sorting the candidate backup devices according to their respective device addresses to obtain a device sequence;

[0132] According to the position of each candidate backup device in the device sequence, taking out the first device ranked first from the device sequence;

[0133] If the device state of the first device is in a fully loaded state, returning to the step of taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence;

[0134] If the device state of the first device is not fully loaded, the first device is used as a target backup device matching the data slice.

[0135] In an exemplary embodiment, the target partition screening module 908 is specifically used to:

[0136] Determine a target partition from each device partition according to the locations of the multiple device partitions included in the target backup device in the target backup device;

[0137] All device partitions before the target partition are in an unavailable state, and the target partition is in an available state.

[0138] In an exemplary embodiment, the data backup device 900 further includes a data recovery module, which is specifically used to:

[0139] For the used partitions storing historical data slices in each device partition of the target backup device, obtaining the data status of each historical data slice;

[0140] In the case where each of the data states includes a to-be-reclaimed state, deleting the historical data slices in the to-be-reclaimed state in the target backup device;

[0141] The partition state of the used partition corresponding to the historical data slice is updated from an unavailable state to an available state.

[0142] In an exemplary embodiment, the data backup device 900 further includes a data migration module, which is specifically used to:

[0143] For each of the candidate backup devices, determining the partition status of each of the plurality of device partitions included in the candidate backup device; the partition status includes an available status and an unavailable status;

[0144] Determine the number of available partitions and the number of unavailable partitions of each candidate backup device based on the partition status of each device partition;

[0145] If, among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, the data slices stored in the second candidate backup device are migrated to the first candidate backup device.

[0146] Each module in the above data backup device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0147] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a data backup method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above method when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0151] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0153] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data backup method, characterized in that: The method comprises: In response to a data backup request initiated by a user, obtaining the to-be-backed-up data sent by the user and the partition sizes of a plurality of candidate backup devices expected to store the to-be-backed-up data; Dividing the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up; For each of the data slices, determining a target backup device that matches the data slice from the candidate backup devices; Among the multiple device partitions included in the target backup device, a target partition corresponding to the data slice is selected, and the data slice is stored in the target partition.

2. The method according to claim 1, characterized in that The step of determining, for each of the data slices, a target backup device that matches the data slice from the candidate backup devices includes: Sorting the candidate backup devices according to their respective device addresses to obtain a device sequence; According to the position of each candidate backup device in the device sequence, taking out the first device ranked first from the device sequence; If the device state of the first device is in a fully loaded state, returning to the step of taking out the first device ranked first from the device sequence according to the position of each candidate backup device in the device sequence; If the device state of the first device is not fully loaded, the first device is used as a target backup device matching the data slice.

3. The method according to claim 2, characterized in that The step of selecting a target partition corresponding to the data slice from among the multiple device partitions included in the target backup device includes: Determine a target partition from each device partition according to the locations of the multiple device partitions included in the target backup device in the target backup device; All device partitions before the target partition are in an unavailable state, and the target partition is in an available state.

4. The method according to claim 3, characterized in that The method further comprises: For the used partitions storing historical data slices in each device partition of the target backup device, obtaining the data status of each historical data slice; In the case where each of the data states includes a to-be-reclaimed state, deleting the historical data slices in the to-be-reclaimed state in the target backup device; The partition state of the used partition corresponding to the historical data slice is updated from an unavailable state to an available state.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: For each of the candidate backup devices, determining the partition status of each of the plurality of device partitions included in the candidate backup device; the partition status includes an available status and an unavailable status; Determine the number of available partitions and the number of unavailable partitions of each candidate backup device based on the partition status of each device partition; If, among the candidate backup devices, the number of available partitions of the first candidate backup device is the same as the number of unavailable partitions of the second candidate backup device, the data slices stored in the second candidate backup device are migrated to the first candidate backup device.

6. The method according to claim 1, characterized in that Each of the candidate backup devices corresponds to a plurality of partition sizes; The step of determining, for each of the data slices, a target backup device that matches the data slice from the candidate backup devices includes: For each of the data slices, the data volume of the data slice is matched with each of the partition sizes, and a selected backup device having a partition size matching the data slice is determined from the candidate backup devices; According to the respective device addresses of the selected backup devices, a target backup device for storing the data slice is determined from the selected backup devices.

7. A data backup device, characterized in that: The device comprises: A data acquisition module, configured to respond to a data backup request initiated by a user and acquire the data to be backed up sent by the user and the partition sizes of multiple candidate backup devices expected to store the data to be backed up; A data segmentation module, used to segment the data to be backed up according to the partition size to obtain multiple data slices corresponding to the data to be backed up; A target backup device determination module, configured to determine, for each of the data slices, a target backup device that matches the data slice from among the candidate backup devices; The target partition screening module is used to screen the target partition corresponding to the data slice from the multiple device partitions included in the target backup device, and store the data slice in the target partition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images