Disk data management method, data recovery method, device, equipment and medium
By creating bitmap files on offline disks and using the file allocation table to synchronize incremental modifications, the problem of high IO load during the offline disk recovery process in the cluster file system is solved, achieving fast online and low-load data recovery.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN INST OF COMPUTING SCI
- Filing Date
- 2025-12-29
- Publication Date
- 2026-05-29
AI Technical Summary
In existing cluster file systems, the offline disk recovery process suffers from high IO load and long duration. How can we optimize the offline disk recovery process to shorten the recovery time and reduce IO load?
When the target disk is set to offline status, a bitmap file is created to record the mapping relationship between the storage address and data status of data blocks. The data blocks to be updated are identified through the file allocation table, and incremental modifications are synchronized during online recovery, using the replica data for updates.
It enables rapid recovery of offline disks, reduces IO load, shortens uptime, and ensures data consistency.
Smart Images
Figure CN122111322A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a disk data management method, recovery method, apparatus, device, and medium. Background Technology
[0002] A clustered file system (CFS) is a redundant file system that allows multiple nodes to concurrently access shared storage resources. It achieves high availability and fault tolerance through data replication or erasure coding techniques, ensuring data integrity and system continuity even in the event of node or disk failure. In high-availability file systems involving data replication, data exists in multiple copies or mirrors. If some copies are corrupted, the remaining complete copies can still support normal data access. Typically, a clustered file system manages multiple disks, which can be simply understood as the disks forming a mirror group, redundantly storing data copies.
[0003] When such a cluster file system detects a disk failure, the failed disk must be set to offline status. Setting a disk to offline status means that, due to the failure, the disk no longer participates in reading and writing redundant data. For existing redundant data, a usable copy is lost, and all subsequent read and write operations are only effective on the online disk. Over time, discrepancies arise between the original data copies. The offline disk stops updating at a certain point in time, and compared to the online disk, the data version on it is older and unusable. When dealing with recoverable disk failures (such as temporary failures caused by network connectivity issues in shared storage), after the offline disk failure is resolved, data exists on the offline disk, but it is outdated. To bring the offline disk back online, it needs to synchronize with the status of the online disk. Current methods involve copying all data from the online disk to the offline disk and then setting the offline disk back online, but this introduces a large amount of I / O and takes a long time.
[0004] Therefore, optimizing the offline disk recovery process to shorten its online time and reduce the IO load during the online process has become an urgent problem to be solved. Summary of the Invention
[0005] This invention provides a disk data management method, recovery method, apparatus, device, and medium to address the issue of optimizing the online recovery process of offline disks, thereby shortening their online time and reducing IO load during the online process.
[0006] A disk data management method, applied in a clustered file system, includes: When the target disk is set to offline status, a bitmap file corresponding to the target disk is created, wherein the bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block; When performing an update operation on a target file, the file allocation table of the target file is obtained. The file allocation table is used to record the mapping relationship between the corresponding data block and its copy of the target file and the disk where they are stored and the storage address within the corresponding disk. Each data block and its copy are stored on different disks. The data block corresponding to the update operation is determined, and the data block corresponding to the update operation and its copy are designated as data blocks to be updated. Based on the file allocation table of the target file, the disk where each data block to be updated is stored and its storage address on the corresponding disk are determined. If a target data block to be updated exists in the target disk among all the data blocks to be updated, then the data status of the target data block to be updated in the bitmap file of the target disk is marked as pending update, according to the storage address of the target data block to be updated in the target disk.
[0007] A disk data recovery method, based on a disk data management method, includes: When performing online recovery on the target disk, obtain the file allocation table of all files managed by the cluster file system, and determine the target data block stored in the target disk and the storage address of the corresponding target data block in the target disk based on the file allocation table. Obtain the bitmap file of the target disk; for any target data block, determine the data status of the target data block based on its storage address on the target disk and the bitmap file of the target disk. If the data status of the target data block is the pending update status, then a target copy is determined from the copies of the target data block, and the target data block is updated according to the target copy to obtain the updated target data block; Clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to the steps for any target data block, and continue until all markers for the pending update status in the bitmap file of the target disk are cleared, then set the target disk to online status.
[0008] A disk data management device, used in a clustered file system, includes: A creation module is used to create a bitmap file corresponding to the target disk when the target disk is set to offline state. The bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block. The first acquisition module is used to acquire the file allocation table of the target file when performing an update operation on the target file. The file allocation table is used to record the mapping relationship between the corresponding data block and its copy of the target file and the disk where they are stored and the storage address in the corresponding disk. Each data block and its copy are stored in different disks. The first determining module is used to determine the data block corresponding to the update operation, designate the data block corresponding to the update operation and its copy as the data block to be updated, and determine the disk where each data block to be updated is stored and its storage address on the corresponding disk according to the file allocation table of the target file. The first marking module is used to mark the data status of the target data block to be updated in the bitmap file of the target disk as pending update status if there is a target data block to be updated stored in the target disk among all the data blocks to be updated.
[0009] A disk data recovery device, based on a disk data management method, includes: The second acquisition module is used to acquire the file allocation table of all files managed by the cluster file system when performing online recovery on the target disk, and determine the target data block stored in the target disk and the storage address of the corresponding target data block in the target disk based on the file allocation table. The second determining module is used to obtain the bitmap file of the target disk, and for any target data block, determine the data status of the target data block based on the storage address of the target data block in the target disk and the bitmap file of the target disk. The first update module is used to determine a target copy from the copies of the target data block if the data state of the target data block is the state to be updated, and update the target data block according to the target copy to obtain an updated target data block; The recovery module is used to clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to execute the steps for any target data block, and continue until all markers for the pending update status in the bitmap file of the target disk are cleared, and then set the target disk to online status.
[0010] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the aforementioned disk data management method or disk data recovery method.
[0011] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned disk data management method or disk data recovery method.
[0012] The aforementioned disk data management method, applied in a clustered file system, involves creating a bitmap file corresponding to the target disk when it is set to offline status. When performing an update operation on the target file, the file allocation table of the target file is obtained to determine the data block corresponding to the update operation. The data block corresponding to the update operation and its copy are designated as data blocks to be updated. Based on the file allocation table of the target file, the disk where each data block to be updated is stored and its storage address within the corresponding disk are determined. If a target data block to be updated exists on the target disk among all the data blocks to be updated, the data status of the target data block to be updated in the bitmap file of the target disk is marked as pending update based on the storage address of the target data block to be updated within the target disk.
[0013] The disk data recovery method described above, based on the disk data management method, involves obtaining the file allocation table of all files managed by the cluster file system when performing online recovery on the target disk. Based on this table, the target data blocks stored on the target disk and their corresponding storage addresses are determined. The bitmap file of the target disk is then obtained. For any target data block, its data status is determined based on its storage address and the bitmap file. If the target data block is in a pending update state, a target copy is determined from the target data block's copies. The target data block is then updated based on the target copy to obtain the updated target data block. The pending update status markers for the target data block in the target disk's bitmap file are cleared. The process returns to executing the steps for any target data block until all pending update status markers in the target disk's bitmap file are cleared. Finally, the target disk is set to an online state.
[0014] This invention creates a bitmap file for the target disk when it is set to offline to record incremental updates of file data during the disk's offline process and stores it in the cluster file system. Failure of any node or client in the cluster will not cause loss of incremental update information. When restoring the target disk to online status, the data of the target disk is synchronized based on the file allocation table of all files and the bitmap file of the target disk. Compared to disk-level mirroring, this invention uses a reliable mechanism to identify incremental modifications after the disk goes offline. Upon going online, only incremental modifications need to be synchronized to bring the offline disk data to the latest state, achieving rapid disk recovery and significantly shortening the offline disk's online time and reducing the IO load during offline disk online restoration. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a schematic diagram of an application environment for a disk data management method and a disk data recovery method according to an embodiment of the present invention; Figure 2 This is a flowchart of a disk data management method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a file allocation table in one embodiment of the present invention; Figure 4 This is an overall schematic diagram of disk data management in one embodiment of the present invention; Figure 5 This is a flowchart of a disk data recovery method according to an embodiment of the present invention; Figure 6 This is an overall schematic diagram of disk data recovery in one embodiment of the present invention; Figure 7 This is an overall schematic diagram of disk data management and disk data recovery performed by a cluster file system in one embodiment of the present invention; Figure 8 This is a schematic diagram of a disk data management device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of a disk data recovery device according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] The disk data management method and disk data recovery method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment shown includes, for example: Figure 1The diagram illustrates a client and server that communicate over a network to optimize the offline disk recovery process, reducing uptime and I / O load. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be a standalone server or a server cluster.
[0019] In one embodiment, such as Figure 2 As shown, a disk data management method is provided, applied in a cluster file system, including the following steps: Step S201: When the target disk is set to offline status, create a bitmap file corresponding to the target disk.
[0020] In this embodiment, the target disk can refer to any disk managed by the cluster file system. The bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block. The data status is used to characterize whether the corresponding data block has been updated.
[0021] Specifically, when the cluster file system detects a target disk failure, it sets the target disk to an offline state. When the target disk is offline, it no longer participates in reading and writing redundant data due to the failure. For existing redundant data, it loses a usable copy. After that, all read and write operations are only valid for online disks. A bitmap file corresponding to the target disk is created and saved in the cluster file system. Failure of any node or client in the cluster will not cause the loss of incremental update information.
[0022] Step S202: When performing an update operation on the target file, obtain the file allocation table of the target file.
[0023] In this embodiment, the target file can refer to the file to be updated. The file allocation table is used to record the mapping relationship between the target file's corresponding data block and its copy, the disk it is stored on, and the storage address within the corresponding disk. Each data block and its copy are stored on different disks.
[0024] For example, such as Figure 3 The diagram shown is a schematic of a file allocation table provided in an embodiment of the present invention. The target data block is an array, and each item in the array records the position of the file's data block on the disk. In the cluster file system of this embodiment, several adjacent positions in the file allocation table store copies of the same data block of the file on other disks.
[0025] See Figure 3 The target file is a file with three data blocks, containing characters A, B, and C. Each character is stored in a separate data block. Each entry in the File Allocation Table (FAT) represents a data block on a specific disk. Adjacent entries (like the copy in Example 2) point to data blocks that contain the same data; they are copies or mirror images of each other. For example, for data block A, the FAT records that it is stored in diskA,0 and diskB,1, meaning it's stored at index 0 on diskA and index 1 on diskB, respectively. When the client updates, all copies of the file data blocks should be updated synchronously. During reading, any readable copy can be selected. It's important to note that while the copies of the file data blocks are stored on different disks, their locations on each disk are not necessarily the same. For example, diskA and diskB are not disk-level mirror images.
[0026] Step S203: Determine the data block corresponding to the update operation, and designate the data block corresponding to the update operation and its copy as the data block to be updated. Based on the file allocation table of the target file, determine the disk where each data block to be updated is stored and its storage address on the corresponding disk.
[0027] In this embodiment, the data block to be updated can refer to the data block corresponding to the target file that the update operation needs to update.
[0028] Specifically, for any data block corresponding to the target file, if an update operation is performed on the data block, the copy of the data block also needs to be updated accordingly to maintain data synchronization. Therefore, the data block corresponding to the update operation and its copy are the data blocks to be updated. According to the file allocation table of the target file, the disk where each data block to be updated is stored and its storage address on the corresponding disk are determined.
[0029] Step S204: If there is a target data block to be updated stored on the target disk among all the data blocks to be updated, then mark the data status of the target data block to be updated in the bitmap file of the target disk as pending update status according to the storage address of the target data block to be updated in the target disk.
[0030] In this embodiment, the target data block to be updated can refer to the data block to be updated stored in the target disk, and the pending update status can be used to indicate that the data block needs to be updated.
[0031] Specifically, if there is a target data block to be updated stored on the target disk among all the data blocks to be updated, then the data status of the target data block to be updated in the bitmap file of the target disk is marked as pending update based on the storage address of the target data block to be updated in the target disk.
[0032] For example, if the bitmap file is an array, and each item in the array records the data status of the data block at the corresponding storage address in the target disk, when updating the data block at the storage address with index 0 in the target disk, the data status of the data item at index 0 in the bitmap file is changed from the default value of 0 to 1. That is, it indicates that the data block at the storage address with index 0 corresponding to this data item needs to be updated. Even if multiple clients concurrently update a certain position in the bitmap file and repeatedly set it to 1, the result is still correct. Therefore, this solution has no concurrency limit in cluster scenarios and has excellent performance.
[0033] Optionally, after finding that the target data block to be updated exists among all the data blocks to be updated and is stored on the target disk, the method further includes: For any data block to be updated other than the target data block to be updated, obtain the disk status where the data block to be updated is stored; If the disk containing the data block to be updated is in an online state, then the update operation is performed on the data block to be updated according to its storage address on the disk, and the updated data block is obtained.
[0034] Among them, "online status" can refer to the state in which the disk can be read and written normally and participate in system data operations, and "updated data block" can refer to the data block after the update is performed on the data block to be updated.
[0035] Optionally, after obtaining the status of the disk where the data block to be updated is stored, the method further includes: If the disk where the data block to be updated is stored is in an offline state, then obtain the bitmap file of the disk where the data block to be updated is stored; Based on the storage address of the data block to be updated on the disk, mark the data status of the data block to be updated in the bitmap file of the disk where the data block to be updated is stored as pending update.
[0036] That is, for any data block to be updated other than the target data block to be updated, if the disk where the data block to be updated is stored is online, the update operation can be directly performed on the data block to be updated at the storage address of the corresponding disk to obtain the updated data block. If the disk where the data block to be updated is stored is offline, the bitmap file of the corresponding disk needs to be obtained, and the data status of the data block to be updated in the bitmap file is marked as pending update status.
[0037] For example, such as Figure 4 The diagram shown is an overall schematic diagram of disk data management provided in this embodiment.
[0038] See Figure 4The target disk is diskA, the target disk's bitmap file is bitmap fileA, and the target file is a file with a size of 3 data blocks, containing the characters A, B, and C. Each character is stored on a separate data block, and every two entries in its File Allocation Table (FAT) are copies of each other, indicating that the same data is stored at specific locations on different disks. For example, for the character B, the corresponding two copies in the FAT are diskA,3 and diskC,2, indicating that the character B is stored at both the storage address at index 3 on diskA and the storage address at index 2 on diskC.
[0039] When the client performs an update operation on character B in the target file, and needs to update character A to character E, according to the FAT instructions, diskA,3 and diskC,2 need to be updated. Since the target disk diskA is offline, only the data block on diskC can be updated to the latest character E. The data block on diskA is actually still the old version of character B. Therefore, the corresponding position (i.e., 3) in the bitmap file of diskA is marked as needing to be updated to indicate that the data at this position is outdated and needs to be updated.
[0040] In this embodiment, a bitmap file is created for the target disk when it is set to offline state to record incremental updates of file data during the disk's offline process. This bitmap file is then stored in the cluster file system. Failure of any node or client in the cluster will not cause loss of incremental update information. When the target disk is brought back online, the data of the target disk is synchronized according to the file allocation table of all files and the bitmap file of the target disk. Compared with disk-level mirroring, this invention uses a reliable mechanism to identify incremental modifications after the disk goes offline. When the disk comes back online, only incremental modifications need to be synchronized to bring the offline disk data to the latest state, achieving rapid recovery of the disk and greatly shortening the time for the offline disk to come back online, as well as reducing the IO load of the offline disk coming back online.
[0041] In one embodiment, such as Figure 5 As shown, a disk data recovery method is provided, based on a disk data management method, including the following steps: Step S501: When performing online recovery on the target disk, obtain the file allocation table of all files managed by the cluster file system, and determine the target data block stored in the target disk and the corresponding storage address of the target data block in the target disk based on the file allocation table.
[0042] Step S502: Obtain the bitmap file of the target disk. For any target data block, determine the data status of the target data block based on its storage address on the target disk and the bitmap file of the target disk.
[0043] In this embodiment, the target data block can refer to a data block stored in the target disk.
[0044] Specifically, when performing online recovery on the target disk, the file allocation table of all files managed by the cluster file system is obtained, all file allocation tables are scanned to determine the target data block stored on the target disk and the corresponding storage address of the target data block on the target disk, the bitmap file of the target disk is obtained, and the data status of the target data block is determined from the bitmap file of the target disk based on the storage address of the target data block on the target disk.
[0045] Step S503: If the data status of the target data block is pending update, then determine the target copy from the copies of the target data block, update the target data block according to the target copy, and obtain the updated target data block.
[0046] Step S504: Clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to the execution of the steps for any target data block, until all markers for the pending update status in the bitmap file of the target disk are cleared, and then set the target disk to online status.
[0047] In this embodiment, the target copy can refer to a copy of the data that is in the latest state and can be used to update the target data block, and the updated data block can refer to the target data block after the update.
[0048] Specifically, if the data status of the target data block is not in the "pending update" state, it indicates that the target data block has not been updated by a copy during the offline phase of the target disk, and its data is in the latest state, so data recovery is not required. If the data status of the target data block is in the "pending update" state, the target copy that can be used to update the target data block is determined from the copies of the target data block, and the data of the target copy is copied to the target data block to obtain the updated target data block. After the target data block is updated, the mark of the "pending update" state of the target data block in the bitmap file of the target disk is cleared, and the process returns to step S502, which determines the data status of the target data block based on its storage address in the target disk and the bitmap file of the target disk. This process continues until all target data blocks are traversed and all the marks of the "pending update" state in the bitmap file of the target disk are cleared. At this point, the data in the target disk is in the latest state, and the target disk is set to the online state.
[0049] Optionally, a target copy is determined from the copies of the target data block, and the target data block is updated based on the target copy to obtain an updated target data block, including: Based on the file allocation table corresponding to the target data block, determine the disk where each copy of the target data block is stored and its storage address on the corresponding disk; Get the disk status where each copy of the target data block is stored. If there is a copy of the target data block whose disk status is online, then the copy whose disk status is online will be the target copy. The target copy is obtained from the storage address on the disk where the target copy is stored. The target data block is then updated based on the target copy to obtain the updated target data block.
[0050] Optionally, after obtaining the state of the disk where each copy of the target data block is stored, the method further includes: If all replicas of the target data block are stored on disks that are in an offline state, then obtain the offline time for each replica of the target data block on the disks that are stored on the disks that are in an offline state. The target data block is obtained by taking a copy of the target data block stored on the disk with the latest offline time as the target copy, retrieving the target copy from the storage address on the disk with the latest offline time, and updating the target data block based on the target copy to obtain the updated target data block.
[0051] That is, when determining the target copy from the copies of the target data block, the disk where each copy of the target data block is stored and its storage address on the disk are determined according to the file allocation table of the file corresponding to the target data block. The status of the disk where each copy is stored is obtained. If there is a copy whose disk status is online, the data in that copy can be determined to be the latest. That copy is the target copy. The target copy is obtained from the storage address of the corresponding disk. The data in the target copy is copied to the target data block to obtain the updated target data block.
[0052] If, after obtaining the status of the disks where each replica is stored, all disks where replicas are stored are offline, then obtain the offline time of each disk where replicas are stored, determine the disk that went offline last, and the data in the replicas stored on the disk that went offline last is the latest status. This replica is designated as the target replica. The target replica is obtained from the storage address of the disk that went offline last, and the data in the target replica is copied to the target data block to obtain the updated target data block.
[0053] For example, such as Figure 6 The diagram shown is an overall schematic diagram of a disk data recovery method provided in this embodiment.
[0054] See Figure 6 During the online recovery of the target disk diskA, the file allocation table of all files in the cluster file system is scanned. Figure 6At position ① in the FAT, data block diskA,0 is found. Comparing the position information with the corresponding bitmap fileA, it can be seen that the data at index 0 of diskA was not updated during offline processing, therefore no processing is required. Continuing to scan the file FAT to position ②, comparing the position information with the corresponding bitmap fileA, it can be seen that data block 3 of diskA was updated during offline processing and needs to be repaired. Since every two items in the FAT are a pair of copies of file data blocks, diskC,2 (online) is copied to data block diskA,3. At this point... Figure 6 As shown on the right, all copies of character E in the second data block of the file are the latest data and have been updated. Therefore, the cluster file system can clear the mark of this data block at the corresponding position in bitmap fileA.
[0055] Once all FATs for all files in the file system have been scanned and all data blocks in bitmap fileA that are yet to be updated have been updated, diskA can be set back to online status. You can then see that each of the file's three data blocks has the same number of copies (two copies each).
[0056] For example, such as Figure 7 The diagram shown is an overall schematic diagram of disk data management and disk data recovery using a cluster file system provided in this embodiment.
[0057] See Figure 7 For a two-replica cluster file system managing three disks, diskA, diskB, and diskC, after the file system has been running for a period of time, diskA goes offline due to some recoverable failure. From the moment it goes offline, all updates that should have been written to diskA will fail. However, the file system will record the location information of these data blocks that should have been updated in a bitmap file, let's call it bitmap fileA, which corresponds to the offline disk A. Meanwhile, client writes to diskB and diskC will continue normally. For a two-replica file system, there must be copies of the data on diskA on diskB and diskC, and these copies must be updated to the latest state.
[0058] All clients of the cluster file system will update the bitmap file in parallel. Since the default value of the bitmap is 0, even if multiple clients concurrently update a certain position of the bitmap and repeatedly set it to 1, the result will still be correct. Therefore, this solution has no concurrency limit in cluster scenarios and has excellent performance.
[0059] After diskA recovers from its failure, the file system scans the FAT records of all files before bringing diskA back online. Using bitmap fileA as a reference, it processes the data blocks on diskA that need updating, copying data from other available copies of the data blocks and updating the corresponding positions on diskA. For each data block updated on diskA, the corresponding record on bitmap fileA is cleared. Once all data blocks on diskA are updated to the latest state, diskA can be set to an online state.
[0060] In this embodiment, a bitmap file is created for the target disk when it is set to offline state to record incremental updates of file data during the disk's offline process. This bitmap file is then stored in the cluster file system. Failure of any node or client in the cluster will not cause loss of incremental update information. When the target disk is brought back online, the data of the target disk is synchronized according to the file allocation table of all files and the bitmap file of the target disk. Compared with disk-level mirroring, this invention uses a reliable mechanism to identify incremental modifications after the disk goes offline. When the disk comes back online, only incremental modifications need to be synchronized to bring the offline disk data to the latest state, achieving rapid recovery of the disk and greatly shortening the time for the offline disk to come back online, as well as reducing the IO load of the offline disk coming back online.
[0061] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0062] In one embodiment, a disk data management device is provided, applied in a clustered file system, and this disk data management device corresponds one-to-one with the disk data management methods in the above embodiments. For example... Figure 8 As shown, the disk data management device includes a creation module 81, a first acquisition module 82, a first determination module 83, and a first marking module 84. Detailed descriptions of each functional module are as follows: Module 81 is used to create a bitmap file corresponding to the target disk when the target disk is set to offline state. The bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block. The first acquisition module 82 is used to acquire the file allocation table of the target file when performing an update operation on the target file. The file allocation table is used to record the mapping relationship between the corresponding data block and its copy of the target file and the disk where they are stored and the storage address in the corresponding disk. Each data block and its copy are stored in a different disk. The first determining module 83 is used to determine the data block corresponding to the update operation, and to take the data block corresponding to the update operation and its copy as the data block to be updated. According to the file allocation table of the target file, the disk where each data block to be updated is stored and the storage address in the corresponding disk are determined. The first marking module 84 is used to mark the data status of the target data block to be updated in the bitmap file of the target disk as pending update status if there is a target data block to be updated stored in the target disk among all the data blocks to be updated.
[0063] Optionally, the disk data management device further includes: The third acquisition module is used to acquire the disk status of any data block to be updated, other than the target data block to be updated; The second update module is used to perform the update operation on the data block to be updated according to the storage address of the data block to be updated in the disk where it is stored, if the disk where the data block to be updated is stored is in an online state, so as to obtain the updated data block.
[0064] Optionally, the disk data management device further includes: The fourth acquisition module is used to acquire the bitmap file of the disk where the data block to be updated is stored if the disk is in an offline state. The second marking module is used to mark the data status of the data block to be updated in the bitmap file of the disk where the data block to be updated is stored as pending update status, based on the storage address of the data block to be updated in the disk where it is stored.
[0065] For specific limitations regarding the disk data management device, please refer to the limitations on the disk data management method above, which will not be repeated here. Each module in the aforementioned disk data management device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0066] In one embodiment, a disk data recovery device is provided, based on a disk data management method. This disk data recovery device corresponds one-to-one with the disk data recovery methods described in the above embodiments. For example... Figure 9 As shown, the disk data recovery device includes a second acquisition module 91, a second determination module 92, a first update module 93, and a recovery module 94. Detailed descriptions of each functional module are as follows: The second acquisition module 91 is used to acquire the file allocation table of all files managed by the cluster file system when performing online recovery on the target disk, and determine the target data block stored in the target disk and the storage address of the corresponding target data block in the target disk based on the file allocation table. The second determining module 92 is used to obtain the bitmap file of the target disk and, for any target data block, determine the data status of the target data block based on the storage address of the target data block in the target disk and the bitmap file of the target disk. The first update module 93 is used to determine the target copy from the copies of the target data block if the data status of the target data block is pending update, and update the target data block according to the target copy to obtain the updated target data block; Recovery module 94 is used to clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to execute the steps for any target data block, until all markers for the pending update status in the bitmap file of the target disk are cleared, and then set the target disk to online status.
[0067] Optionally, the aforementioned first update module 93 includes: The second determining unit is used to determine the disk where each copy of the target data block is stored and its storage address on the corresponding disk, based on the file allocation table corresponding to the target data block. The third determining unit is used to obtain the disk status of each copy of the target data block. If there is a copy of the target data block whose disk status is online, then the copy whose disk status is online is the target copy. The third update unit is used to obtain the target copy from the storage address on the disk where the target copy is stored, and update the target data block according to the target copy to obtain the updated target data block.
[0068] Optionally, the first update module 93 mentioned above also includes: The fourth acquisition unit is used to acquire the offline time when the disk where each copy of the target data block is stored is set to offline if the disk where all copies of the target data block are stored is in an offline state. The fourth update unit is used to take the target data block stored on the disk with the latest offline time as the target copy, obtain the target copy from the storage address in the disk with the latest offline time where the target copy is stored, and update the target data block according to the target copy to obtain the updated target data block.
[0069] For specific limitations regarding disk data recovery devices, please refer to the limitations on disk data recovery methods described above, which will not be repeated here. Each module in the aforementioned disk data recovery device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0070] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores bitmap files and file allocation tables. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a disk data management method or a disk data recovery method.
[0071] In one embodiment, a computer device is provided, including a memory, a first processor, a second processor, and a computer program stored in the memory and executable on the processor. When the first processor executes the computer program, it implements the disk data management method described in the above embodiment, for example... Figure 2 As shown in S201-S204, or Figures 3 to 4 As shown, to avoid repetition, it will not be described again here. Alternatively, when the first processor executes the computer program, it implements the functions of each module / unit in this embodiment of the disk data management device, for example, Figure 8 The functions of the creation module 81, the first acquisition module 82, the first determination module 83, and the first marking module 84 shown are not described again here to avoid repetition. When the second processor executes the computer program, it implements the disk data recovery method described in the above embodiments, for example... Figure 5 As shown in S501-S504, or Figures 6 to 7 As shown, to avoid repetition, it will not be described again here. Alternatively, when the second processor executes the computer program, it implements the functions of each module / unit in this embodiment of the disk data recovery device, for example, Figure 9 The functions of the second acquisition module 91, the second determination module 92, the first update module 93, and the recovery module 94 shown are not described again here to avoid repetition.
[0072] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a first processor, the computer program implements the disk data management method described in the above embodiment, for example... Figure 2 As shown in S201-S204, or Figures 3 to 4 As shown, to avoid repetition, it will not be described again here. Alternatively, when the first processor executes the computer program, it implements the functions of each module / unit in this embodiment of the disk data management device, for example, Figure 8 The functions of the creation module 81, the first acquisition module 82, the first determination module 83, and the first marking module 84 shown are not described again here to avoid repetition. When this computer program is executed by the second processor, it implements the disk data recovery method described in the above embodiments, for example... Figure 5 As shown in S501-S504, or Figures 6 to 7 As shown, to avoid repetition, it will not be described again here. Alternatively, when the second processor executes the computer program, it implements the functions of each module / unit in this embodiment of the disk data recovery device, for example, Figure 9 The functions of the second acquisition module 91, the second determination module 92, the first update module 93, and the recovery module 94 shown are not described again here to avoid repetition.
[0073] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0074] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0075] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A disk data management method, characterized in that, Applied in cluster file systems, including: When the target disk is set to offline status, a bitmap file corresponding to the target disk is created, wherein the bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block; When performing an update operation on a target file, the file allocation table of the target file is obtained. The file allocation table is used to record the mapping relationship between the corresponding data block and its copy of the target file and the disk where they are stored and the storage address within the corresponding disk. Each data block and its copy are stored on different disks. The data block corresponding to the update operation is determined, and the data block corresponding to the update operation and its copy are designated as data blocks to be updated. Based on the file allocation table of the target file, the disk where each data block to be updated is stored and its storage address on the corresponding disk are determined. If a target data block to be updated exists in the target disk among all the data blocks to be updated, then the data status of the target data block to be updated in the bitmap file of the target disk is marked as pending update, according to the storage address of the target data block to be updated in the target disk.
2. The disk data management method according to claim 1, characterized in that, After the statement "If a target data block to be updated exists in the target disk among all data blocks to be updated", the method further includes: For any data block to be updated other than the target data block to be updated, obtain the state of the disk where the data block to be updated is stored; If the disk where the data block to be updated is stored is in an online state, then the update operation is performed on the data block to be updated according to its storage address on the disk, and the updated data block is obtained.
3. The disk data management method according to claim 2, characterized in that, After obtaining the status of the disk where the data block to be updated is stored, the process further includes: If the disk where the data block to be updated is stored is in an offline state, then obtain the bitmap file of the disk where the data block to be updated is stored; Based on the storage address of the data block to be updated on the disk in which it is stored, the data status of the data block to be updated in the bitmap file of the disk in which it is stored is marked as the pending update status.
4. A disk data recovery method, characterized in that, The disk data management method according to any one of claims 1 to 3 includes: When performing online recovery on the target disk, obtain the file allocation table of all files managed by the cluster file system, and determine the target data block stored in the target disk and the storage address of the corresponding target data block in the target disk based on the file allocation table. Obtain the bitmap file of the target disk; for any target data block, determine the data status of the target data block based on its storage address on the target disk and the bitmap file of the target disk. If the data status of the target data block is the pending update status, then a target copy is determined from the copies of the target data block, and the target data block is updated according to the target copy to obtain the updated target data block; Clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to the steps for any target data block, and continue until all markers for the pending update status in the bitmap file of the target disk are cleared, then set the target disk to online status.
5. The disk data recovery method according to claim 4, characterized in that, The step of determining a target copy from the copies of the target data block, and updating the target data block according to the target copy to obtain an updated target data block, includes: Based on the file allocation table corresponding to the target data block, determine the disk where each copy of the target data block is stored and its storage address within the corresponding disk; Obtain the disk status of each replica of the target data block. If there is a replica among all replicas of the target data block whose disk status is online, then the replica whose disk status is online is the target replica. The target copy is obtained from the storage address on the disk where the target copy is stored, and the target data block is updated according to the target copy to obtain the updated target data block.
6. The disk data recovery method according to claim 5, characterized in that, After obtaining the state of the disk where each copy of the target data block is stored, the method further includes: If the disks on which all copies of the target data block are stored are in the offline state, then obtain the offline time when the disks on which each copy of the target data block is stored are set to the offline state. The target data block is obtained by taking a copy of the target data block stored on the disk with the latest offline time as the target copy, retrieving the target copy from the storage address on the disk with the latest offline time, and updating the target data block according to the target copy to obtain the updated target data block.
7. A disk data management device, characterized in that, Applied in cluster file systems, including: A creation module is used to create a bitmap file corresponding to the target disk when the target disk is set to offline state. The bitmap file is used to record the mapping relationship between the storage address of each data block in the target disk and the data status of the corresponding data block. The first acquisition module is used to acquire the file allocation table of the target file when performing an update operation on the target file. The file allocation table is used to record the mapping relationship between the corresponding data block and its copy of the target file and the disk where they are stored and the storage address in the corresponding disk. Each data block and its copy are stored in different disks. The first determining module is used to determine the data block corresponding to the update operation, designate the data block corresponding to the update operation and its copy as the data block to be updated, and determine the disk where each data block to be updated is stored and its storage address on the corresponding disk according to the file allocation table of the target file. The first marking module is used to mark the data status of the target data block to be updated in the bitmap file of the target disk as pending update status if there is a target data block to be updated stored in the target disk among all the data blocks to be updated.
8. A disk data recovery device, characterized in that, The disk data management method according to any one of claims 1 to 3 includes: The second acquisition module is used to acquire the file allocation table of all files managed by the cluster file system when performing online recovery on the target disk, and determine the target data block stored in the target disk and the storage address of the corresponding target data block in the target disk based on the file allocation table. The second determining module is used to obtain the bitmap file of the target disk, and for any target data block, determine the data status of the target data block based on the storage address of the target data block in the target disk and the bitmap file of the target disk. The first update module is used to determine a target copy from the copies of the target data block if the data state of the target data block is the state to be updated, and update the target data block according to the target copy to obtain an updated target data block; The recovery module is used to clear the markers for the pending update status of the target data block in the bitmap file of the target disk, return to execute the steps for any target data block, and continue until all markers for the pending update status in the bitmap file of the target disk are cleared, and then set the target disk to online status.
9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the disk data management method as described in any one of claims 1 to 3, or the disk data recovery method as described in any one of claims 4 to 6.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the disk data management method as described in any one of claims 1 to 3, or the disk data recovery method as described in any one of claims 4 to 6.