Data backup, recovery method and system

Abstract

This specification provides a data backup and recovery method and system. During data backup, a current file block is determined for writing the obtained current data block, and the current data block is written to this current file block to obtain at least one file of backup data. Each of the at least one file includes at least one file block, and the at least one file block is stored as a linked list structure. In the linked list structure: the previously written data block points to the previously written data block through its storage address; the last file block of the last written data block in the at least one file includes an entry block address, which is the storage address of the last written data block. This improves the flexibility, integration, and effectiveness of data backup. Especially in large-scale data backup scenarios, it enables ordered backup of large amounts of data. Furthermore, the linked list structure facilitates data recovery.

Description

Technical Field

[0001] This manual relates to the field of data processing, and in particular to a data backup and recovery method and system. Background Technology

[0002] Data backup is the foundation of disaster recovery. It can be understood as the process of copying all or part of the data set from the application host's hard drive or array to other storage media to prevent data loss due to operational errors or system failures. Therefore, reliable data backup methods can not only recover lost data but also ensure transaction continuity and reduce downtime caused by data unavailability.

[0003] In related technologies, taking Oracle RMAN (Recovery Manager) data backup as an example: Oracle database backup and recovery tools support multi-file backups. For instance, Oracle database backup and recovery tools can physically back up each data file to be backed up in the form of a "backup set," and use the database's System Change Number (SCN) and checkpoints to ensure data backup consistency.

[0004] However, although the above data backup methods support multi-file backups, they lack a mechanism for linking files; even within the same file, there is a lack of a mechanism for linking data within that file. This results in limitations in the flexibility, integration, and effectiveness of data backups.

[0005] It should be noted that the above-mentioned related technologies are only information known to the inventor personally, and do not mean that the above information had entered the public domain before the application date of this specification, nor do they mean that it can be considered prior art in this specification. Summary of the Invention

[0006] This specification provides a data backup and recovery method and system to avoid at least one of the above-mentioned technical problems.

[0007] Firstly, this specification provides a data backup method, including: Obtain the current data block of the data to be backed up; Determine the current file block for writing the current data block; and Write the current data block to the current file block to back up at least one file corresponding to the data to be backed up in the data storage system; Each of the at least one file includes at least one file block, and the at least one file block is stored as a linked list structure. In the linked list structure, the first data block and the second data block in any file block are data blocks that were written consecutively. The first data block and the second data block are associated by a storage address, and the second data block points to the first data block through the storage address of the first data block. The last file block written to the at least one file includes an entry block address, which is the storage address of the last data block written to the last file block.

[0008] Secondly, this specification provides a data recovery method for recovering data backed up to at least one file in a data storage system based on the method described in the first aspect. Each of the at least one file includes at least one file block, and the at least one file block is stored as a linked list structure. The method includes: The entry block address is obtained from the last file block of the last data block written to the at least one file, where the entry block address is the storage address of the last data block written; and The linked list structure is traversed in reverse order based on the storage address of the last written data block to recover the data blocks in the multiple files.

[0009] Thirdly, this specification provides a data backup system, including: At least one storage medium storing at least one instruction set for data backup; At least one processor is communicatively connected to the at least one storage medium, wherein when the at least one processor is running, it reads the at least one instruction set and executes the data backup method as described in the first aspect according to the instructions of the at least one instruction set.

[0010] Fourthly, this specification provides a data recovery system, including: At least one storage medium storing at least one instruction set for data recovery; At least one processor is communicatively connected to the at least one storage medium, wherein when the at least one processor is running, it reads the at least one instruction set and executes the data recovery method as described in the second aspect according to the instructions of the at least one instruction set.

[0011] Fifthly, this specification provides a computer-readable non-transitory storage medium, wherein the computer-readable non-transitory storage medium stores at least one instruction set, which is executed by at least one processor to implement the method as described in the first or second aspect.

[0012] As can be seen from the above technical solutions, the data backup and recovery method and system provided in this specification, in the data backup method, by obtaining the current data block of the data to be backed up, determining the current file block used to write the current data block, and writing the current data block to the current file block, at least one file corresponding to the data to be backed up is backed up in the data storage system. Furthermore, each file in the data storage system is a linked list structure. In the linked list structure, the first data block and the second data block are data blocks that have been written twice consecutively. The first data block and the second data block are associated through storage addresses, and the second data block is associated with the first data block through the storage address of the first data block. The last file block of the file containing the last written data block includes an entry block address, which is the storage address of the last written data block. This maintains the continuity of the linked list structure and realizes the association between two adjacent file blocks and between two adjacent data blocks. This improves the flexibility, integration, and effectiveness of data backup. Especially in large-scale data backup scenarios, it enables the orderly organization of large-scale data. In addition, the linked list structure also facilitates data recovery. For example, the backed-up data can be read quickly and reliably by reverse traversal.

[0013] The data backup and recovery methods and other system functions provided in this manual are partially listed in the following description. The inventive aspects of the data backup and recovery methods and systems provided in this manual can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A schematic diagram of the linked list structure provided in the embodiments of this specification; Figure 2 This diagram illustrates application scenarios of the data backup and recovery methods provided in the embodiments of this specification. Figure 3 This is a schematic diagram of the structure of the data backup system and data recovery system provided in the embodiments of this specification; Figure 4 A flowchart illustrating a data backup method provided in one embodiment of this specification; Figure 5 A flowchart illustrating a data backup method provided in another embodiment of this specification; Figure 6 A flowchart illustrating the entry writing process provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the data block writing process provided in the embodiments of this specification; Figure 8 This is a flowchart illustrating the data recovery method provided in the embodiments of this specification. Detailed Implementation

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0017] It should be understood that the terms “comprising” and “having”, and any variations thereof, in the embodiments of this specification are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0018] The term "and / or" in the embodiments of this specification describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0019] In the embodiments of this specification, the term "multiple" refers to two or more, and other quantifiers are similar.

[0020] The terms “first,” “second,” “third,” etc., used in this specification are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, in situations where implementation can proceed in an order other than those given in the embodiments illustrated or described in this specification.

[0021] As used in this specification, the term "unit / module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0022] To avoid at least one of the technical problems mentioned in the background section, this specification proposes a technical concept developed through inventive effort: For data to be backed up, the data is written to a data storage system in the form of data blocks. Furthermore, in the data storage system, the data to be backed up is written to one or more files, each file containing one or more file blocks, and the files are stored in the data storage system in the form of a linked list; that is, the structure of the one or more files containing the backed-up data is a linked list structure.

[0023] For example, in any file block of any file in a linked list structure: if the first data block and the second data block are data blocks that are written to consecutively, then the first data block and the second data block are associated through their storage addresses, and specifically, the second data block points to the first data block through the storage address of the first data block; in addition, the last file block in the file that is the last data block written to includes the entry block address, which is the storage address of the last data block written.

[0024] In other words, adjacent file blocks are associated with each other through memory address pointers, and adjacent file blocks within the same file block are associated with each other through memory address pointers. The first data block of the later file block in two adjacent file blocks is linked to the last data block of the earlier file block through the memory address of the last data block of the earlier file block. Similarly, the later data block in the same file block is linked to the earlier data block through the memory address of the earlier data block. The last file block of the last data block in multiple file blocks includes the entry block address, which is the memory address of the last data block written.

[0025] For example, such as Figure 1 As shown, a file contains three file blocks, meaning the data to be backed up is written into a linked list structure, and this file includes file block 1, file block 2, and file block 3. File block 1 includes data block 1, data block 2, ..., data block M (M is an integer greater than or equal to 1); file block 2 includes data block M+1, data block M+2, ..., data block N; file block 3 includes data block N+1, ..., data block K-1, and data block K. The storage address of data block 1 is storage address 1, the storage address of data block 2 is storage address 2, and so on, with the storage address of data block K being storage address K.

[0026] File block 2 and file block 3 are two adjacent file blocks that are written to in two consecutive transactions. For example, after writing data block N in file block 2, data block N+1 is written to file block 3. That is, data block N and data block N+1 are two adjacent data blocks that are written to in two consecutive transactions, but they are written to two different file blocks.

[0027] Therefore, file block 3 and file block 2 are associated through memory address pointers. For example... Figure 1 As shown, data block N+1 in file block 3 is stored at the address of data block N in file block 2 (e.g., ...). Figure 1 The storage address N shown points to data block N in file block 2. Thus, the association between two adjacent file blocks (file block 2 and file block 3) is achieved through this storage address.

[0028] The association between file block 2 and file block 1 via memory address pointers can be found in the examples above, and will not be listed here again.

[0029] Data block 1 and data block 2 are two data blocks that are written to file block 1 on two consecutive occasions. For example, if data block 1 is written to file block 1 and then data block 2 is written to file block 1, then data block 1 and data block 2 are two consecutive data blocks written to the same file block.

[0030] Therefore, data block 1 and data block 2 are associated through memory addresses. For example... Figure 1 As shown, data block 2 is stored at the address of data block 1 (e.g., ...). Figure 1 The storage address shown (1) points to data block 1. Thus, the association between two adjacent data blocks within the same file block is achieved through this storage address.

[0031] Similarly, regarding the memory address association between two adjacent data blocks written in the same file block, please refer to the description in the above example, which will not be listed here again.

[0032] In addition, such as Figure 1 As shown, file block 3 is the last file block to be written to. Therefore, file block 3 also includes the entry block address, which is the storage address of data block K (e.g., ...). Figure 1 The storage address K is shown.

[0033] The technical solution provided in this specification is based on the above-described technical concept. As described above, the technical solution provided in this specification achieves this by writing the data to be backed up into at least one file block of a linked list structure. Specifically, each subsequent data block is linked to the previous data block via its storage address, and the first data block in each subsequent file block is linked to the last data block in the previous file block via its storage address. Furthermore, the entry block address, i.e., the storage address of the last data block, is written into the last file block. This maintains the continuity of the linked list structure and establishes associations between adjacent file blocks and between adjacent data blocks. This improves the flexibility, integration, and effectiveness of data backup. Especially in large-scale data backup scenarios, it enables the orderly organization of large-scale data. Additionally, the linked list structure facilitates data recovery. For example, backed-up data can be read quickly and reliably using reverse traversal.

[0034] To facilitate readers' understanding of this manual, the application scenarios of this manual are introduced below.

[0035] The technical solutions provided in this manual are applicable to scenarios requiring data backup, such as log systems, real-time data stream processing, and databases. Furthermore, in database scenarios, the technical solutions provided in this manual can be applied to various types of databases, such as distributed databases, distributed relational databases, relational databases, and non-relational databases. For example, any enterprise or organization using a database can employ the data backup methods provided in this manual for data backup.

[0036] For example, consider the scenario described above using the database: The data backup method provided in this manual can be used to write the corresponding data from the database to the disk.

[0037] It is worth noting that the above examples are merely illustrative of the application scenarios to which the technical solutions in this specification can be applied, and should not be construed as limiting the application scenarios. Those skilled in the art should understand that the data backup method described in this specification, when applied to other use cases, is also within the scope of protection of this specification.

[0038] Figure 2 This is a schematic diagram illustrating an application scenario of the data backup method according to an embodiment of this specification. The data backup method of this specification can be applied to, for example... Figure 2 Scenario 200 is shown. (e.g.) Figure 2 As shown, scenario 200 may include target user 201, client 202, server 203, and network 204.

[0039] The target user 201 can be the user who triggers the data backup, such as the data producer. For example, the target user 201 can perform a target operation on the client 202 to trigger the data backup.

[0040] Client 102 may be an electronic device that provides interactive functionality to target user 201. For example, client 202 may provide an interactive interface to target user 201, allowing target user 201 to perform interactive operations. In some embodiments, client 202 executes the data backup method described herein in response to detecting a target operation triggered by target user 201. In this case, client 202 may store data or instructions for executing the data backup method described herein, and may execute or be used to execute the data or instructions. In some embodiments, client 102 may include a hardware device with data processing capabilities and the necessary programs required to drive the hardware device to execute the data backup method described herein.

[0041] In some embodiments, client 202 may include a mobile device, tablet, laptop, built-in device in a motor vehicle, or similar content, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or similar device, or any combination thereof. In some embodiments, smart home devices may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, smart mobile devices may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, built-in devices in a motor vehicle may include an in-vehicle computer, an in-vehicle television, etc.

[0042] In some embodiments, client 202 may have one or more applications (APPs) installed. APPs provide target user 201 with the ability and interface to interact with the outside world via network 204. APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc.

[0043] like Figure 2 As shown, client 202 can establish a communication connection with server 203. Server 203 can communicate with one client 202 or multiple clients 202. In some embodiments, client 202 can interact with server 203 via network 204 to receive or send messages, etc.

[0044] Server 203 can be a server that provides various services. For example, server 203 can be a cloud server or a local server. Server 203 can communicate with one client 202 and receive data sent by that client 202, or it can communicate with multiple clients 202 and receive data sent by each client 202.

[0045] In some embodiments, the data backup method described herein can be executed on server 203. In this case, server 203 may store data or instructions for executing the data backup method described herein, and may execute or be used to execute the data or instructions. Server 203 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices.

[0046] Network 204 is a medium used to provide a communication connection between client 202 and server 203. Network 204 can facilitate the exchange of information or data. Figure 2 As shown, client 202 and server 203 can connect to network 204 respectively, and transmit information or data to each other through network 204.

[0047] In some embodiments, network 204 can be any type of wired or wireless network, or a combination thereof. For example, network 204 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network™, a ZigBee™ short-range wireless network, a near field communication (NFC) network, or a similar network.

[0048] In some embodiments, network 204 may include one or more network access points. For example, network 204 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of client 202 and server 203 can connect to network 204 to exchange data or information.

[0049] It is worth noting that, Figure 2The number of clients 202, servers 203, and networks 204 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 202, servers 203, and networks 204. Furthermore, the data backup method provided in this specification can be executed entirely on client 202, entirely on server 203, or partially on client 202 and partially on server 203.

[0050] In other words, Figure 2 and targeting Figure 2 The above description is only used to illustrate the possible application scenarios for the data backup method in this specification, and should not be construed as limiting the application scenarios.

[0051] Furthermore, it should be understood that, to avoid tedious descriptions, identical or similar content will not be repeated in this specification. For example, regarding the application scenarios of the data recovery methods provided in this specification, please refer to the description of the application scenarios of the data backup methods in the above examples, which will not be repeated here.

[0052] Figure 3 A hardware structure diagram of a data backup system 300 provided according to an embodiment of this specification is shown. The data backup system 300 can execute the data backup method described in this specification. The data backup method is described in other parts of this specification. When the data backup method is executed on client 202, the data backup system 300 can be client 202. When the data backup method is executed on server 203, the data backup system 300 can be server 203. When the data backup method is executed partly on client 202 and partly on server 203, the data backup system 300 can be a system including client 202 and server 203.

[0053] like Figure 3 As shown, the data backup system 300 may include at least one storage medium 303 and at least one processor 302. In some embodiments, the data backup system 300 may also include a communication port 304 and an internal communication bus 301. The data backup system 300 may also include I / O components 305.

[0054] The internal communication bus 301 can connect to different system components. For example, the internal communication bus 301 can connect to storage medium 303, processor 302, communication port 304, and I / O component 305.

[0055] I / O component 305 supports input / output between data backup system 300 and other components.

[0056] Communication port 304 is used for data communication between the data backup system 300 and the outside world. For example, communication port 304 can be used for data communication between the data backup system 300 and network 304. Communication port 304 can be a wired communication port or a wireless communication port.

[0057] Storage medium 303 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 3031, a read-only storage medium (ROM) 3032, or a random access storage medium (RAM) 3033. Storage medium 303 also includes at least one instruction set stored in the data storage device. The instruction set includes computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the data backup methods provided in this specification.

[0058] At least one processor 302 may be communicatively connected to at least one storage medium 303. The at least one processor 302 is used to execute the at least one instruction set described above. When the data backup system 300 is running, the at least one processor 302 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the data backup method provided in this specification. The processor 302 may execute all steps included in the data backup method. The processor 302 may be in the form of one or more processors. In some embodiments, the processor 302 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0059] For illustrative purposes only, only one processor 302 is shown in the accompanying drawings of the data backup system 300. However, it should be noted that the data backup system 300 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor or by multiple processors in combination. For example, if the processor 302 of the data backup system 300 described in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 302 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0060] Furthermore, for a description of the application scenarios of the data recovery method, please refer to the description of the application scenarios of the data backup method in the above examples, which will not be repeated here. The data recovery system and the data backup system can be the same system or different systems. If they are different systems, for a description of the data recovery system provided in this manual, please refer to the description of the data backup system in the above examples, which will not be repeated here.

[0061] Please see Figure 4 , Figure 4 This is a flowchart illustrating a data backup method provided in one embodiment of this specification. Wherein, Figure 4 The entity executing the data backup method shown can be a data backup system. For a description of the data backup system, please refer to the example above; it will not be repeated here.

[0062] like Figure 4 As shown, the method includes the following steps S401 to S403: S401: Obtain the current data block of the data to be backed up.

[0063] The data to be backed up can be understood as the data that needs to be backed up. This data can be a data stream, a dataset, etc., and this embodiment does not impose any limitations.

[0064] For example, the data to be backed up can be a backup set. Such as the collection of all data corresponding to a complete backup operation. For example, in the scenario of automated backup of a log system, specifically performing a full backup of a database server every night at 2 AM, a backup set can be the collection of data corresponding to the backup operation on a certain night (i.e., the collection of data between 2 AM the previous night and 2 AM the current night).

[0065] A data block can be understood as a fixed- or variable-sized unit into which data is logically divided. Correspondingly, the current data block can be understood as the data block that needs to be backed up at the moment.

[0066] S402: Determine the current file block to be written to the current data block.

[0067] A data storage system can be understood as an object that stores data to be backed up; that is, the object on which the data to be backed up is stored. For example, a data storage system can be an actual physical or logical storage backend, such as a local disk, network file system, or object storage. In short, whatever object the data to be backed up needs to be backed up to, that object can be called a data storage system.

[0068] The current file block can be understood as the file block that needs to be written to the current data block in order to back up the current data block.

[0069] S403: Write the current data block to the current file block to back up at least one file corresponding to the data to be backed up in the data storage system.

[0070] Each of the at least one file includes at least one file block, and the at least one file has a linked list structure. In the linked list structure, the first data block and the second data block in any file block are data blocks that were written consecutively. The first data block and the second data block are linked by a storage address, and the second data block points to the first data block through the storage address of the first data block. The last file block written to in the at least one file includes the entry block address. That is, the last file block of the last written data block includes the entry block address, which is the storage address of the last written data block.

[0071] For example, if a backup file contains multiple file blocks, the linked list structure can represent two levels of linking relationships: one level represents the linking relationships within a file block, and the other level represents the linking relationships between file blocks. If the backup file contains only one file block, the linked list structure can represent the linking relationships between file blocks.

[0072] The internal linking relationships within a file block are as follows: when writing the nth data block (e.g., corresponding to the second data block), the storage address of the (n-1)th data block (e.g., corresponding to the first data block) is recorded at the beginning or end of that data block (as metadata). This ensures that the nth data block points to the (n-1)th data block through the storage address of the (n-1)th data block. In this way, all data blocks within a file block form a singly linked list from back to front. The nth data block and the (n-1)th data block are two different data blocks within the same file block.

[0073] For example, combining the above examples and Figure 1 Data block 2 is stored at the address of data block 1 (e.g., ...). Figure 1 The storage address shown (1) points to data block 1. Thus, through this storage address, a link is established between two adjacent data blocks within the same file block. Similarly, all data blocks within the same file block form a singly linked list from back to front.

[0074] The linking relationship between file blocks is as follows: the first data block of the later file block records the storage address of the last data block of the previous file block. This establishes a link between the two file blocks.

[0075] Continuing with the examples above and Figure 1 Data block N+1 in file block 3 is accessed through the storage address of data block N in file block 2 (e.g., Figure 1The storage address N shown points to data block N in file block 2. Thus, this storage address establishes a link between two adjacent file blocks. Similarly, all files corresponding to the data to be backed up form a singly linked list from back to front.

[0076] Additionally, after each successful write of a data block, the data backup system can update a global or local "entry block address" variable. Once all data blocks have been written, this "entry block address" (i.e., the address of the last data block) will be written to the corresponding position of the last file block (such as the end of the file) as the entry block address.

[0077] This results in multiple file blocks in a linked list structure.

[0078] Based on the analysis of S401 to S403 above, it can be seen that in this embodiment, the data to be backed up is written to multiple file blocks in a linked list structure. Specifically, each subsequent data block is linked to the storage address of the previous data block, and the entry block address, i.e., the storage address of the last data block, is written into the last file block. This maintains the continuity of the linked list structure and establishes associations between adjacent file blocks and between adjacent data blocks within the same file block. This improves the flexibility, integration, and effectiveness of data backup. Furthermore, the linked list structure facilitates data recovery. For example, backed-up data can be read quickly and reliably using reverse traversal.

[0079] To enable readers to gain a deeper understanding of the technical solutions provided in this manual, the following is combined with... Figure 5 This manual provides a more detailed explanation of the data backup methods. For example... Figure 5 As shown, the method includes: S501: Get the current entry to be written.

[0080] S502: Write the current entry to be written to the memory buffer.

[0081] S503: Obtain the amount of data for the entries stored in the memory buffer.

[0082] S504: If the amount of data of the entries stored in the memory buffer reaches the preset second threshold, the entries stored in the memory buffer are constructed into the current data block, and the current data block includes the storage address of the previous data block.

[0083] For example, data backup may include two parts of write logic: one part is the writing of entries, and the other part is the writing of data blocks. For instance, a data backup system may first construct data blocks based on entries (which can be understood as the writing logic of entries), and then write the data blocks to the data storage system (which can be understood as the writing logic of data blocks).

[0084] An item, also known as a data unit, can be understood as the smallest logical unit of data to be backed up in a data storage system. It is an abstract concept, as opposed to a data block.

[0085] For example, in a logging system, an entry might be a log message. Correspondingly, a data block might be a data unit that includes multiple log messages.

[0086] For example, in a database system, an entry might be the result of executing a Structured Query Language (SQL) statement, a redo log of a transaction, or a key-value pair. Correspondingly, a data block might be the result of executing multiple SQL statements.

[0087] A memory buffer can be understood as a temporary storage area located in a data storage system, specifically in the random access memory (RAM) of the data storage system, used to temporarily store data during the process of writing data from one place to another.

[0088] like Figure 6 As shown, when an entry is being written, and specifically when the data backup system obtains the current entry to be written, the data backup system can first write the entry to be written to the memory buffer; then check the data volume of all entries stored in the memory buffer; then compare the data volume of all entries in the memory buffer with a preset second threshold; if the data volume of all entries in the memory buffer reaches the preset second threshold, all entries in the memory buffer can be constructed into a data block for writing; if the data volume of all entries in the memory buffer does not reach the preset second threshold, it can continue to wait for the next entry to be written, and so on, until all entries to be written are completed, such as constructing the corresponding data block.

[0089] Alternatively, the data backup system can first determine the amount of data in the memory buffer after writing the entry to be written, and if the amount of data reaches a preset second threshold, construct the existing entries in the memory buffer into corresponding data blocks so as to construct new data blocks based on the entry to be written.

[0090] In some embodiments, a data backup system may include an item writer for writing entries and a block writer for writing data blocks. Accordingly, the item writer performs item writing, and the block writer performs data block writing, thereby completing the data backup.

[0091] When starting to write an entry, the entry writer can obtain the file path prefix corresponding to the data backup passed in by the user, and can initialize the write file path for the backup. The backup of the data to be backed up is then completed based on the corresponding write file path.

[0092] Additionally, when the entry writer receives an entry from the user, it can acquire a safety lock to enter the critical section, preventing multiple threads from modifying the memory buffer simultaneously, in order to add the user-supplied entry to the memory buffer in that case.

[0093] In other words, the number of entries is usually several, and writing several entries can be achieved through multithreading. This improves both the efficiency and accuracy of entry writing. Correspondingly, after completing the construction of the corresponding data block, the entry writer can clear the memory buffer and release the lock.

[0094] It is worth noting that this manual does not limit the size of the preset second threshold; it can be determined by the data backup system based on requirements, historical records, experiments, etc. For example, the preset second threshold can be 2MB. That is, the size of a data block is approximately 2MB. In other words, the data volume of an entry in a data block is approximately close to 2MB.

[0095] Based on the above analysis of S501 to S504, it can be seen that in this embodiment, data backup includes entry writing and data block writing, which can improve the fineness of data backup, thereby improving the accuracy and reliability of data backup.

[0096] Furthermore, batch processing can be achieved through entry writing, accumulating multiple small entry writes and merging them into a large I / O operation, which significantly improves I / O efficiency (reduces seek time and increases throughput), and makes it easier for users (such as data producers) to quickly "throw" data into the memory buffer and continue working.

[0097] In addition, by constructing entries of corresponding data volume into corresponding data blocks, memory buffer overflows can be prevented, and the buffer can be prevented from occupying too much memory, thus affecting the operation of other parts of the data storage system.

[0098] In some embodiments, "constructing the entries stored in the memory buffer into the current data block" in S504 includes: constructing a current link header that includes the storage address of the previous data block, and constructing a current data block that includes the current link header and the entries stored in the memory buffer.

[0099] For example, a data block may include two parts: a link header (also called a linked header, or simply a header) and the data body. The link header can be the metadata of the data block, containing the storage address of the previous data block.

[0100] like Figure 6 As shown, the constructed data block includes two parts: a link header and a data body. The link header includes the storage address of the previous data block. The data body includes all entries in the memory buffer used to construct the data block.

[0101] Furthermore, this manual does not impose any restrictions on the storage address of the data block. In principle, the storage address of the corresponding data block should be sufficient to locate it, so that the corresponding data block can be read during data recovery.

[0102] For example, in some embodiments, the storage address of the nth data block includes the identifier (file id) of the file block to which the nth data block belongs, and the offset of the nth data block within the file block.

[0103] For example, such as Figure 6 As shown, the link header of the data block includes the storage address of the previous data block, and specifically includes the identifier of the file block to which the previous data block belongs, as well as the offset of the previous data block within its file block.

[0104] Combining the above examples and Figure 1 If the nth data block is data block N+1, then the link header of data block N+1 includes the storage address of the (n-1)th data block, i.e., data block N. This storage address can specifically include the ID of the file block to which data block N belongs, such as the ID of file block 2, "file_id=2"; this storage address can also specifically include the offset of data block N within file block 2, such as "offset: 1987654", which is a byte offset and indicates that the starting position of data block N is located at the 1,987,654th byte of file block 2.

[0105] In some embodiments, if the first data block and the second data block are written to the same file block, and the first data block is a data block other than the first data block written in its file block, then the storage address of the first data block includes the offset of the first data block in the same file block.

[0106] If the first data block and the second data block are written to different file blocks, and the first data block is the first data block written to its own file block, then the storage address of the first data block includes the identifier of the file block to which the first data block belongs, and the offset of the first data block in its own file block.

[0107] For example, for the first data block written in a file block, the storage address of the previous data block included in the data block includes both the corresponding identifier and the corresponding offset.

[0108] Conversely, for a data block that is not the first one written in a file block, the storage address of the previous data block included in that data block includes the corresponding offset.

[0109] Continuing with the examples above and Figure 1 If the first data block is data block 2, then data block 2 includes the storage address of data block 1, and the storage address may include the offset of data block 1 in file block 1.

[0110] If the first data block is data block N+1, then data block N+1 includes the storage address of data block N, and this storage address may include not only the offset of data block N in file block 2, but also the identifier of file block 2.

[0111] It is worth noting that the link header may also include, in addition to the storage address mentioned above, its contents, such as timestamp, backup set identifier, data length, checksum, number of entries, etc.

[0112] In this context, data length can be understood as the length of a data block, allowing for the reading of data blocks of the appropriate length during data recovery. Checksums are calculated when data blocks are generated and stored / transmitted with them for recalculation and comparison during data recovery to check data block integrity (e.g., whether the data block was corrupted during backup). Entry count can be understood as the total number of entries included in a data block, used to determine data block integrity during data recovery (e.g., whether the number of entries read from the data block is accurate).

[0113] Based on the above analysis, it can be seen that the link header of each data block can be the corresponding metadata. In some embodiments, the data backup system can store the metadata of each data block in the last file block. For example, each data block may store a small amount of metadata (the specific amount can be determined based on requirements, historical records, experiments, etc.), or it may not store metadata at all. This achieves unified management of metadata, simplifies the complexity of metadata maintenance, and saves storage space and improves write efficiency. Especially when the technical solution in this specification is applied to a distributed system, it can avoid distributed metadata consistency problems.

[0114] S505: Obtain the total amount of data in the current data block and the previous file block used to write the previous data block in the data storage system; if the total amount of data reaches a preset first threshold, encapsulate the previous file block and switch to the new file block to determine the new file as the current file block; if the total amount of data is less than the preset first threshold, determine the previous file block as the current file block and write the current data block to the current file block.

[0115] S506: Continue in this manner until the last data block in the data to be backed up is written to the last file block. The file end information is written to the last file block, including the entry block address, so as to back up at least one file in the data storage system that corresponds to the linked list structure of the data to be backed up.

[0116] Therefore, at least one file is a linked list structure, and in the linked list structure: two data blocks written in consecutive orders are associated through storage address pointers, and the data block written later points to the data block written earlier through the storage address of the data block written earlier; the last file block of the last data block written includes the entry block address, which is the storage address of the last data block written.

[0117] Similarly, this embodiment does not limit the size of the preset first threshold, which can be determined by the data backup system based on needs, historical records, experiments, etc. For example, the preset first threshold can be 4GB. That is, the size of a file block is approximately 4GB. In other words, the data volume of a data block in a file block is approximately close to 4GB.

[0118] Based on the above analysis, it can be seen that a data backup system can determine the file block size based on the amount of data. In other embodiments, the data backup system can also determine the data block and file block size based on time windows, data content hash distribution, etc.

[0119] For example, taking the time window to determine the file block size as an example: the data backup system can write continuously generated data blocks into different file blocks according to a preset time interval (the specific time interval can be determined based on requirements, historical records, experiments, etc., and this embodiment does not limit it). That is, different file blocks contain data blocks within the corresponding time interval range.

[0120] Moreover, in this embodiment, the identifiers of different file blocks can be corresponding timestamps.

[0121] Based on the above analysis, data backup can include entry writing and data block writing. Furthermore, data block writing can be implemented using a data block writer.

[0122] For example, for the obtained current data block, the data block writer can first determine the data volume of all data blocks in the previous file block, and can also determine the data volume of the current data block. Therefore, the two can be added together to obtain the total data volume of all data blocks in the current file block after the current data block is written to the current file block.

[0123] It is important to clarify here that the preceding file block does not refer to the file block that has already been encapsulated into a chain in the linked list structure, but rather to the file block that has been written to the previous data block before it has been encapsulated.

[0124] Continuing with the example above, if the total data volume reaches a preset first threshold, the previous file block is encapsulated so that it becomes a complete file block in the linked list structure. Then, the system switches to a new file block (i.e., the next file block in the previous file) and writes the current data block to the new file block to continue acquiring new data blocks (as the next data block). If the total data volume does not reach the preset first threshold, the current data block is written to the previous file, and new data blocks are acquired (as the next data block).

[0125] This process continues until the last data block is written to the corresponding file block. After writing the last data block, the data block writer can write end-of-file information (also known as a filetailer) to the last file block. The end-of-file information includes the address of the entry block.

[0126] From this point on, through entry writing and data block writing, the data backup system writes the data to be backed up to at least one file in the data storage system, thus completing the backup of the data to be backed up.

[0127] For example, such as Figure 7 As shown, during data block writing, for the obtained nth data block, the data block writer can determine the size of the total data volume (e.g., the data volume of a file block after writing the nth data block to an existing file block) and a preset first threshold. If the total data volume does not reach the preset first threshold, the nth data block is written to that file block. If the total data volume reaches the preset first threshold, the file block is encapsulated, and the nth data block is written to a new file block.

[0128] Continue reading Figure 7 If a new data block is available (e.g., the (n+1)th data block), the process continues by writing the new data block to the corresponding file block based on the aforementioned logic. If no new data block is available, such as when the data to be backed up has been completed (i.e., the nth data block is the last data block to be backed up), the data block writer writes end-of-file information to the last file block to obtain the following result: Figure 1 Backup data in a chain structure as shown.

[0129] Based on the above analysis, it can be seen that in this embodiment, by combining the writing of data blocks with a preset first threshold, the data size of each file block can be kept within a relatively optimal range. This allows the data to be backed up to be automatically divided into multiple file blocks (shards) of no more than 4GB in size for storage, achieving efficient, reliable, and seamless data storage. Furthermore, it avoids the drawback of the entire backup failing due to file block corruption during backup based on a single file block. This is particularly suitable for efficiently handling large data volumes, such as terabytes / petabytes (TB / PB) level data backups.

[0130] In addition, by responding to the writing of the last data block in the data to be backed up to the last file block, and writing the end-of-file information in the last file block, the entry block address can be written in the end-of-file information. This allows for quick location of the entry block address during later data recovery, enabling the quick finding of the last written data block and thus achieving rapid recovery of the backup data.

[0131] Based on the above analysis, it can be seen that the above examples mainly illustrate the technical principles of data backup. Data recovery can be understood as the reverse process of data backup, such as reading data from at least one file in a linked list structure that has been written to the data storage system based on the data backup method described above. Combining the above analysis, it can be seen that the last file block includes the entry block address, that is, the storage address of the last written data block. Therefore, the data recovery system can traverse the linked list structure in reverse based on this storage address to recover the corresponding data block from at least one file, and ultimately recover the backed-up data.

[0132] For example, continuing to combine the above examples and Figure 1 First, the data recovery system obtains the entry block address, such as the storage address K of data block K. Then, the system reads data block K based on storage address K, obtains the storage address of data block K-1, and reads data block K-1 based on its storage address. This process continues until the storage address 1 of data block 1 is obtained from storage address 2 of data block 2, and data block 1 is read from storage address 1. This completes the recovery of the entire backed-up data.

[0133] Based on the above analysis, it can be seen that in this embodiment, the continuity of data recovery is achieved by traversing the linked list structure in reverse, which can improve the accuracy and reliability of data recovery.

[0134] To help readers better understand the technical principles behind the data recovery methods provided in this manual, the following explanation is provided in conjunction with... Figure 8This specification provides a detailed description of the data recovery methods. It is understood that, to avoid tedious explanations, details identical or similar to those in the data backup methods will not be repeated in this embodiment. For example, regarding the application scenarios and execution entities (such as data recovery systems) of the data recovery methods, please refer to the descriptions of the application scenarios and data backup systems for the data backup methods in the above examples; they will not be repeated here.

[0135] like Figure 8 As shown, data recovery methods include: S801: Detects the number of file blocks in at least one file.

[0136] For example, as can be seen from the above analysis, during data backup, the data backup system can initialize the write file path for the backup. Therefore, during data recovery, the data recovery system can perform a directory scan based on the write file path to detect the number of multiple file blocks corresponding to the backup data.

[0137] For example, corresponding to the data block writer and item writer included in a data backup system, a data recovery system may include a data block reader and an item reader. The data block reader can obtain the write file path for the backed-up data input by the user, and scan the directory of multiple file blocks of the backup data based on the write file path to obtain the number of at least one file block.

[0138] Alternatively, based on the above analysis, the data to be backed up can be a backup set, and the backup set can have a corresponding identifier. Therefore, in some embodiments, the data recovery system can scan the directory containing at least one file block of the corresponding data based on the backup set identifier to obtain the number of the corresponding at least one file block.

[0139] Additionally, in some embodiments, S801 is executed when the system does not know which file block to start reading from during data recovery. For example, the data recovery system may already know which file block to start reading from during data recovery; in this case, the data recovery system simply reads the corresponding file block. Conversely, if the data recovery system does not know which file block to start reading from, it can scan the directory of each file block in the backup data to obtain the number of file blocks present at the time of backup.

[0140] S802: Determine the file tail information of the last file block based on the number of multiple file blocks detected, and obtain the entry block address from the file tail information. The entry block address includes the identifier of the file block to which the last written data block belongs, its offset, and the storage address of the second-to-last written data block.

[0141] For example, after detecting the number of file blocks of at least one file, the data recovery system can determine which file block among the corresponding number of file blocks is the last file block to be written to the corresponding data block (i.e., the last file block), and can obtain the file tail information of the last file block in order to obtain the entry block address from the file tail information.

[0142] Based on the above analysis and Figure 1 It is known that the end-of-file information of the last file block stores the entry block address, that is, the storage address of the last written data block. Therefore, the data recovery system can obtain such... Figure 1 The storage address of data block K shown in the figure.

[0143] Based on the above analysis of S801 and S802, it can be seen that in this embodiment, the data recovery system (such as a data block reader) can improve the efficiency and reliability of obtaining the entry block address by detecting the number of file blocks corresponding to the data that has been divided.

[0144] Based on the above analysis, the storage address of the nth data block includes: the identifier of the file block to which the (n-1)th data block belongs, and the offset of the (n-1)th data block within its file block. The entry block address is the storage address of the last data block written. Therefore, the entry block address includes the identifier of the file block to which the last data block was written, its offset, and the storage address of the second-to-last data block written.

[0145] Continuing with the examples above and Figure 1 The storage address K of data block K includes: the address of the file block to which data block K belongs (e.g., the address of file block 3, file_id=3), the byte offset of data block K in file block 3 (e.g., which byte in file block 3 starts as data block K), and the storage address of data block K-1.

[0146] S803: Determine the last file block based on the identifier of the file block to which the last written data block belongs; read the last written data block from the last file block based on the offset of the last written data block; and read the second-to-last written data block from the corresponding file block based on the storage address of the second-to-last written data block, until the linked list structure is traversed in reverse to recover the data in at least one file block.

[0147] Continuing with the examples above and Figure 1The data recovery system can obtain the entry block address from the information at the end of the file. Specifically, it can obtain the address of the file block to which data block K belongs (file_id=3), the byte offset of data block K in file block 3, and the storage address of data block K-1 (such as the address of the file block to which data block K-1 belongs (file_id=3) and the byte offset of data block K-1 in file block 3).

[0148] Correspondingly, the data recovery system can read data block K based on the address of the file block to which data block K belongs (file_id=3) and the byte offset in file block 3 of data block K.

[0149] Furthermore, based on the above analysis, the entry block address can also include the data length of data block K. Therefore, the data recovery system can read data block K from file block 3 based on the data offset and data length of data block K.

[0150] In some embodiments, as can be seen from the above analysis, the data recovery system can also obtain the checksum of data block K in order to verify the integrity of data block K.

[0151] In addition, since data block K also stores the storage address of data block K-1, the data recovery system can also read data block K-1 based on the storage address of data block K-1 (such as the address of the file block to which data block K-1 belongs, file_id=3, and the byte offset of data block K-1 in file block 3).

[0152] Furthermore, the data recovery system can obtain the storage address of data block K-2 from data block K-1, and so on, until the linked list structure is traversed in reverse and data block 1 is read, so as to complete the recovery of the backed-up data.

[0153] In some embodiments, if the data block recovered by the m-th reverse traversal of the linked list structure (where m is an integer greater than or equal to 1) is the third data block, and the third and fourth data blocks were written to the storage system in two consecutive steps, then taking the m-th reverse traversal of the linked list structure as an example, the data recovery method includes the following two cases: Case 1: If the file block to which the third data block belongs is the first file block, and the file block to which the fourth data block belongs is the second file block, then the storage address of the third data block is obtained from the fourth data block. The storage address of the third data block includes the identifier of the first file block to which the third data block belongs, and the offset of the third data block in the first file block. The third data block is then read according to the identifier of the first file block and the offset of the third data block in the first file block.

[0154] Case 2: If the third data block and the fourth data block belong to the same file block, then obtain the storage address of the third data block from the fourth data block. The storage address of the third data block includes the offset of the third data block in the same file block, and read the third data block according to the offset of the third data block in the same file block.

[0155] Case 1 describes the reverse traversal of backup data in the linked list structure when reading data blocks across file blocks; Case 2 describes the reverse traversal of backup data in the linked list structure when reading two adjacent data blocks within the same file block.

[0156] The understanding of Case 1 is as follows: The data blocks in the previous reverse traversal of the linked list structure and the data blocks in the subsequent reverse traversal of the linked list structure belong to different file blocks.

[0157] Continuing with the examples above and Figure 1 File block 1 and file block 2 are two different file blocks that are written to data blocks in two consecutive steps.

[0158] The data block recovered by the (m-1)th reverse traversal of the linked list structure can be the last data block recovered from file block 2, such as data block M+1.

[0159] Therefore, the data recovery system can also obtain the storage address of data block M from data block M+1, such as the ID of the file block where data block M is stored—file block 1 (e.g., file_id=1), and the offset of data block M in file block 1.

[0160] Correspondingly, the data recovery system can generate a file path based on the address of the file block to which data block M belongs (file_id=1), and read data block M from file block 1 based on the offset of data block M in file block 1.

[0161] In other words, in this case, the data recovery system can achieve cross-file block data addressing by traversing the linked list structure in reverse. This allows it to read data blocks from file blocks where data blocks were written later, and then read data blocks from file blocks where data blocks were written earlier, and so on, until all data blocks in the linked list structure have been read.

[0162] The understanding of situation 2 is as follows: The data blocks from the previous reverse traversal of the linked list structure and the data blocks from the subsequent reverse traversal of the linked list structure belong to the same file block. For example, the data blocks written later and the data blocks written earlier are backed up in the same file block.

[0163] Continuing with the examples above and Figure 1 Data block 1 and data block 2 are two different data blocks that are written into the same file block in two consecutive instances.

[0164] The data block recovered by the (m-1)th reverse traversal of the linked list structure can be data block 2 read from file block 1.

[0165] Therefore, the data recovery system can also obtain the storage address of data block 1 from data block 2, such as the file block where data block 1 is stored—file block 1, and the offset of data block 1 in file block 1.

[0166] Therefore, the data recovery system can read data block 1 from file block 1 based on the offset of data block 1 in file block 1.

[0167] Furthermore, it's worth noting that, based on the above analysis, the later written data block includes the storage address of the previously written data block, and this storage address can include the address and offset of the file block it belongs to. For example, the nth data block includes the storage address of the (n-1)th data block, and this storage address includes: the identifier of the file block to which the (n-1)th data block belongs, and the offset of the (n-1)th data block within its file block.

[0168] Therefore, for the reverse traversal between two different file blocks in the above case 1, since the storage address of data block M can be obtained from data block M-1, such as the file block where data block M is stored—file block 1, and the offset of data block M in file block 1.

[0169] Therefore, during data recovery, after reading data block M, the data recovery system can generate a file path based on the address of the file block to which data block M-1 belongs, such as the address of file block 1 file_id=1, and read data block M from file 1 based on the file path.

[0170] Similarly, for the reverse traversal between the same file blocks in situation 2 above, the storage address of data block 1 can be obtained from data block 2, such as the file block where data block 1 is stored—file block 1, and the offset of data block 1 in file block 1.

[0171] Since data block 1 and data block 2 belong to the same file block, the data recovery system does not need to regenerate the corresponding file path in this case. Instead, it can directly read data block 1 based on its offset within file block 1.

[0172] Therefore, in some embodiments, when the data recovery system reads a previous data block from a previous file block, it can determine whether there are any unread data blocks in the previous file block. If so, it continues to read data blocks from the previous file block based on the corresponding offset. Otherwise, if not, it determines the file block to which the next data block to be read belongs, and reads the next data block from it. This process continues until all data blocks in the chained file block structure have been read in reverse order.

[0173] Alternatively, in other embodiments, during data backup, for the first data block written in each file block, the data block may include the storage address of the previously written data block, and the storage address includes the file block to which the previously written data block belongs and its offset. For data blocks that are not the first data blocks written in a file block, only the offset of the previously written data block may be included.

[0174] Additionally, in some embodiments, each file block may also include the total number of data blocks in that file block. For example, the total number of data blocks in that file block may be included at the end or beginning of the file block. When reading data blocks from that file block, this total number can be used to determine whether all data blocks in that file block have been read. If so, the process can proceed in reverse, reading data blocks from adjacent file blocks.

[0175] Continuing with the examples above and Figure 1 Between two consecutive writes of data blocks from two different file blocks, such as data block M+1 including the file block to which data block M belongs and its offset; and data block N+1 including the file block to which data block N belongs and its offset. Since file block 1 is the first file block, the file block to which the preceding data block belongs and its offset included in data block 1 can be empty. Between two consecutive writes of data blocks from the same file block, such as data block 2 including the offset of data block 1; and data block M+2 including the offset of data block M+1; and data block M+2 including the offset of data block M+1.

[0176] Furthermore, based on the above description of data backup methods, it is clear that data backup can include data block writing and entry writing. And considering the above description of data recovery methods, the above example primarily describes data recovery methods from the perspective of data block recovery.

[0177] Therefore, in some embodiments, after the data recovery system reads any data block, the data block can be stored in a memory buffer, and the arbitrary data block in the memory buffer can be parsed to obtain the corresponding entry.

[0178] In other words, data recovery can be divided into data block reading and entry reading. The logic of data block reading has been explained in detail in the example above. Correspondingly, entry reading can be implemented based on the entry reader in the recovery system. After the data block reader reads a data block, it can store the data block in a memory buffer. The entry reader can then parse the data block in the memory buffer to obtain the corresponding entry.

[0179] Continuing with the examples above and Figure 1After the data block reader reads data block K from file block 3, it can store data block K in the memory buffer. The entry reader can parse data block K in the memory buffer to obtain the corresponding entry.

[0180] Based on the above analysis, it can be seen that multithreading can be used during data backup, specifically when writing entries. Similarly, multithreading can also be used during data recovery, specifically when reading entries, to improve the efficiency of entry reading. Likewise, when using multithreading for entry reading, a locking mechanism can be employed to improve both the efficiency and accuracy and reliability of entry reading. Furthermore, based on the above analysis, each data block can include a link header, and the link header can also include the number of entries. Accordingly, when parsing the entries of a data block, the number of each parsed entry can be compared with the number of entries in the link header to determine if the data block is complete. For example, if the number of each parsed entry is the same as the number of entries in the link header, the data block is considered complete; otherwise, the data block is considered incomplete.

[0181] It is worth noting that the above examples are merely illustrative of possible implementations of the data backup method in this specification, and should not be construed as limiting the implementation of the data backup method in this specification. For example, based on the above technical concept, some of the technical features described above can be combined to obtain new embodiments; new technical features can be added to the above examples to obtain new embodiments; some technical features can be removed from the above examples to obtain new embodiments; some technical features in the above examples can be replaced with other technical features; some technical features and their order in the above examples can be adjusted to obtain new embodiments, and so on, which will not be listed here.

[0182] Based on the above technical concept, this specification also provides a computer-readable non-transitory storage medium storing at least one instruction set, which, when executed by a processor, implements the steps of the data backup method and data recovery method described in this specification.

[0183] In some possible implementations, various aspects of this specification can also be implemented as a program product comprising program code. Taking data backup system 300 as an example, when the program product is run on data backup system 300, the program code causes data backup system 300 to perform the steps of the data backup method described in this specification. The program product for implementing the above method can employ a portable compact disc read-only memory (CD-ROM) containing program code and can run on data backup system 300. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system. The program product can employ any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on data backup system 300, partially on data backup system 300, as a standalone software package, partially on data backup system 300 and partially on a remote data backup system, or entirely on the remote data backup system 300.

[0184] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user-related information (such as data to be backed up) involved in the technical solutions of this specification all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0185] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0186] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0187] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0188] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

[0189] Every patent, patent application, publication of a patent application, and other material cited herein, such as articles, books, specifications, publications, documents, and literature (excluding any related historical examination documents), is referenced for all purposes relevant to this document, including in the specification and claims herein. However, in the event of any inconsistency or conflict between the descriptions, definitions, and / or terms used in the foregoing and those used herein, the descriptions, definitions, and / or terms used herein shall prevail.

[0190] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A data backup method, comprising: Obtain the current data block of the data to be backed up; Determine the current file block to be written to the current data block; as well as Write the current data block to the current file block to back up at least one file corresponding to the data to be backed up in the data storage system; Each of the at least one file includes at least one file block, and the at least one file block is stored as a linked list structure. In the linked list structure, the first data block and the second data block in any file block are data blocks that were written consecutively. The first data block and the second data block are associated by a storage address, and the second data block points to the first data block through the storage address of the first data block. The last file block written to the at least one file includes an entry block address, which is the storage address of the last data block written to the last file block.

2. The method according to claim 1, wherein, Determining the current file block for writing the current data block includes: Obtain the total amount of data in the current data block and the previous file blocks in the data storage system used to write the previous data block; If the total amount of data reaches a preset first threshold, the preceding file block is encapsulated and the process switches to a new file block, so that the new file block is identified as the current file block. If the total amount of data is less than the preset first threshold, then the previous file block will be determined as the current file block.

3. The method according to claim 1, wherein, Obtain the current data block of the data to be backed up, including: Obtain the amount of data in the memory buffer storing the entries to be written; If the amount of data in the memory buffer reaches a preset second threshold, the entries to be written in the memory buffer are constructed into the current data block, which includes the storage address of the previous data block.

4. The method according to claim 3, wherein, Constructing the current data block from the entries to be written in the memory buffer includes: Construct the current link header including the storage address of the previous data block; and The current data block, including the current link header and the entries to be written in the memory buffer, is constructed.

5. The method according to claim 3, wherein, Obtain the amount of data in the memory buffer storing the entries to be written, including: Get the current entry to be written; Write the current entry to be written to the memory buffer; and Obtain the amount of data for the entries stored in the memory buffer.

6. The method according to claim 5, wherein, The data to be backed up includes several entries, including the current entry to be written; the method further includes: Multiple threads are started, and the entries are written to the memory buffer in parallel by the multiple threads.

7. The method according to claim 4, wherein, The storage address of the previous data block includes: the identifier of the file block to which the previous data block belongs, and the offset of the previous data block within its file block.

8. The method according to claim 7, wherein, The storage address of the previous data block also includes at least one of the following: the timestamp of the previous data block being written, the amount of data in the previous data block, the checksum, and the number of entries in the previous data block.

9. The method according to claim 1, wherein, The method further includes: In response to writing the last data block in the data to be backed up to the last written file block, file tail information is written to the last written file block, and the file tail information includes the entry block address.

10. The method according to claim 1, wherein, If the first data block and the second data block are written to the same file block, and the first data block is a data block other than the first data block written in its file block, then the storage address of the first data block includes the offset of the first data block in the same file block. If the first data block and the second data block are written to different file blocks, and the first data block is the first data block written to its own file block, then the storage address of the first data block includes the identifier of the file block to which the first data block belongs, and the offset of the first data block in its own file block.

11. A data recovery method, the method being used to recover data backed up to at least one file in a data storage system according to any one of claims 1 to 10, each of the at least one file comprising at least one file block, and the at least one file block being stored as a linked list structure, the method comprising: The entry block address is obtained from the last file block of the last data block written in the at least one file, and the entry block address is the storage address of the last data block written. as well as The linked list structure is traversed in reverse order based on the storage address of the last written data block to recover the data blocks in at least one file.

12. The method according to claim 11, wherein, Obtaining the entry block address from the last file block of the last data block written to the at least one file includes: Detect the number of file blocks in at least one file; and The last file block is determined based on the detected number, and the entry block address is obtained from the last file block.

13. The method according to claim 11, wherein, To recover data from at least one file, the linked list structure is traversed in reverse order based on the storage address of the last written data block, including: The last written data block is read from the last file block according to the storage address of the last written data block, wherein the last written data block includes the storage address of the penultimate written data block; and The penultimate data block is read from the storage address of the penultimate data block, and the process continues until the linked list structure is traversed in reverse to recover the data in at least one file.

14. The method according to claim 11, wherein, If the data block recovered by the m-th reverse traversal of the linked list structure is the third data block, and the third data block and the fourth data block were written to the storage system in two consecutive steps, then the m-th reverse traversal of the linked list structure includes: If the file block to which the third data block belongs is the first file block, and the file block to which the fourth data block belongs is the second file block, then the storage address of the third data block is obtained from the fourth data block, and the storage address of the third data block includes the identifier of the first file block to which the third data block belongs, and the offset of the third data block in the first file block, and the third data block is read according to the identifier of the first file block and the offset of the third data block in the first file block; If the third data block and the fourth data block belong to the same file block, the storage address of the third data block is obtained from the fourth data block, and the storage address of the third data block includes the offset of the third data block in the same file block, and the third data block is read according to the offset of the third data block in the same file block.

15. The method of claim 14, wherein the third data block is read based on the identifier of the first file block and the offset of the third data block within the first file block, comprising: Generate a file path based on the identifier of the first file block; as well as The third data block is read from the first file block according to the file path, corresponding to the offset of the third data block in the first file block.

16. The method according to claim 11, wherein, The method further includes: Data blocks read from any file block are stored in a memory buffer; and The data blocks in the memory buffer are parsed to obtain the corresponding entries.

17. The method according to claim 16, wherein, The storage address of any data block in the storage system includes: the number of entries in the arbitrary data block; the method further includes: Obtain the number of entries in the parsed arbitrary data block; and The number of entries obtained is compared with the number of entries in the arbitrary data block to verify the integrity of the read arbitrary data block.

18. A data backup system, comprising: At least one storage medium storing at least one instruction set for data backup; At least one processor is communicatively connected to the at least one storage medium, wherein when the at least one processor is running, it reads the at least one instruction set and executes the data backup method as described in any one of claims 1 to 10 according to the instructions of the at least one instruction set.

19. A data recovery system, comprising: At least one storage medium storing at least one instruction set for data recovery; At least one processor is communicatively connected to the at least one storage medium, wherein when the at least one processor is running, it reads the at least one instruction set and executes the data recovery method as described in any one of claims 11 to 17 according to the instructions of the at least one instruction set.

Images