Data processing method, block server, storage medium, and program product

Abstract

This application provides a data processing method, a block server, a storage medium, and a program product. The method relates to the field of distributed storage and includes: in response to a hard link request for a first data block, obtaining the identification information of the first data block and the identification information of a first linked data block to be linked to the first data block; if the first data block is determined to be a source data block, then, based on the identification information of the first linked data block and the first data block itself, adding a link mapping relationship from the first linked data block to the first data block to a link mapping relationship table; updating the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks linked to the first data block, thereby implementing a hard link from the first linked data block to the first data block, and thus realizing block-level hardlink functionality.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a data processing method, a block server, a storage medium, and a program product. Background Technology

[0002] In a distributed file system, a chunk server is responsible for storing the individual data blocks of a file. Each file is divided into multiple fixed-size data blocks (e.g., 64MB or 128MB), which are replicated and stored on different chunk servers to improve data reliability and availability.

[0003] To meet the horizontal scaling requirements of namespaces in Elastic Block Storage (EBS), distributed file systems typically enable federation to improve scalability, performance, and isolation. Enabling federation allows data from one volume in a ChunkServer to be dynamically and evenly distributed across new volumes. EBS requires zero-copy migration across volumes, transferring only the metadata while the data itself is handled via hard links to avoid the overhead of copying the data to the new volume. Therefore, ChunkServers must support hardlink functionality at the chunk level. Summary of the Invention

[0004] This application provides a data processing method, a block server, a storage medium, and a program product for implementing Hardlink functionality at the chunk level.

[0005] Firstly, this application provides a data processing method, including:

[0006] In response to a hard link request for the first data block, the identification information of the first data block and the identification information of the first linked data block to be linked to the first data block are obtained.

[0007] Based on the identification information of the first data block, if the first data block is determined to be a source data block, then based on the identification information of the first linked data block and the identification information of the first data block, add the link mapping relationship from the first linked data block to the first data block to the link mapping relationship table.

[0008] Update the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks that are linked to the first data block.

[0009] In a second aspect, this application provides a block server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the block server to perform the method provided in the first aspect above.

[0010] Thirdly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method provided in the first aspect above.

[0011] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in the first aspect above.

[0012] The data processing method, block server, storage medium, and program product provided in this application, in response to a hard link request for a first data block, obtain the identification information of the first data block and the identification information of the first linked data block to be linked to the first data block; if the first data block is determined to be the source data block, then according to the identification information of the first linked data block and the first data block, add the link mapping relationship from the first linked data block to the first data block to the link mapping relationship table; update the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks linked to the first data block, thereby realizing a hard link from the first linked data block to the first data block, and thus realizing the hard link function at the data block level. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] Figure 1 A schematic diagram of a distributed file system architecture provided for an embodiment of this application;

[0015] Figure 2 An architecture diagram of a Chunkserver memory metadata system provided in this application embodiment;

[0016] Figure 3 A schematic diagram of the data structure of the metadata table and the link mapping table, and the global lock provided for an exemplary embodiment of this application;

[0017] Figure 4 A flowchart illustrating a data processing method provided in an exemplary embodiment of this application;

[0018] Figure 5 An example diagram of creating a Hardlink to a source data block is provided as an exemplary embodiment of this application;

[0019] Figure 6 A flowchart illustrating a data processing method provided in an exemplary embodiment of this application;

[0020] Figure 7 Example diagram of creating a recursive hardlink provided for an exemplary embodiment of this application;

[0021] Figure 8 A flowchart for deleting a data block provided as an exemplary embodiment of this application;

[0022] Figure 9 This is a schematic diagram of the structure of a block server provided in an embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0026] First, let me explain the terms used in this application:

[0027] A data block (Chunk) is an independent storage unit for data defined in a distributed storage system.

[0028] ChunkHandle: The identifier for data blocks within the Chunkserver, primarily used for data block management within the Chunkserver.

[0029] ChunkId: A globally unique identifier for a data block, used for data access and location between the client and the master server.

[0030] Source data block (SrcChunk): The original data chunk in the hard link mechanism of a distributed storage system.

[0031] LinkChunk: In the hard link mechanism of a distributed storage system, a hard link chunk that links to (points to) SrcChunk.

[0032] A ChunkBlob instance is an object in Chunkserver used to store and represent a data block. A ChunkBlob instance is the actual storage form of a data block on Chunkserver. When a data block is written to Chunkserver, it is encapsulated into a ChunkBlob instance and stored on disk.

[0033] MetaTable: A table in a distributed storage system used to store the metadata of all source data blocks.

[0034] Link Table: A table in a distributed storage system used to store the link mapping relationships between linked data blocks and source data blocks.

[0035] This application provides a data processing method that, in response to a hard link request for a first source data block, obtains the identification information of the first source data block and the identification information of a first link data block to be linked to the first source data block; based on the identification information of the first link data block and the identification information of the first source data block, adds a link mapping relationship from the first link data block to the first source data block to a link mapping relationship table, and updates the number of links in the metadata of the first source data block, where the number of links represents the number of link data blocks linked to the first source data block, so as to realize a hard link from the first link data block to the first source data block.

[0036] This embodiment involves two types of data blocks: source data blocks (SrcChunk) and link data blocks (LinkChunk). Source data blocks refer to the data blocks stored in the storage devices (such as cloud disks, volumes, etc.) of the distributed storage system. Link data blocks are used to implement the links created by Hardlink to source data blocks; they are logical data blocks. The distributed storage system registers the identifier (ChunkId) of the link data block, stores the link mapping relationship between the link data block and the source data block, but does not independently store the metadata of the link data block. Link data blocks are used to index the source data blocks they are linked to.

[0037] The method of this application stores the metadata of the source data block through a metadata table (MetaTable) and adds the number of links to the metadata of the source data block to record the number of linked data blocks linked to the source data block; it stores the link mapping relationship from the linked data block to the source data block through a link mapping relationship table (LinkTable); and it manages the source data block and the linked data block in a unified manner through the metadata table (MetaTable) and the link mapping relationship table (LinkTable), and records and maintains the link mapping relationship from the linked data block to the source data block, thereby realizing efficient and accurate indexing from the linked data block to the source data block, and realizing the hardlink function at the data block level.

[0038] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0039] Figure 1 This is a schematic diagram of a distributed file system architecture provided in an embodiment of this application. Figure 1 As shown, the distributed file system includes: Client 10, Master 11, and Chunkserver 12. Client 10 is responsible for receiving file read / write requests from callers, and then interacting with Master 11 and Chunkserver 12 to complete data read / write operations.

[0040] In a distributed file system, files are abstracted as streams. A stream can be considered a large file that can be appended to indefinitely. Files are divided into multiple fixed-size (e.g., 64MB or 128MB) data chunks. A stream consists of a series of chunks, which are stored on a chunkserver.

[0041] Master server 11 is primarily responsible for managing metadata and maintaining its consistency. Master server 11 provides namespace services and Stream metadata services. The namespace service includes maintaining the file directory tree and namespaces. The Stream metadata service maintains the index of chunks and provides file-to-data-block mappings, etc.

[0042] Chunkserver 12 is responsible for storing chunks. Each file is divided into multiple fixed-size data chunks (e.g., 64MB or 128MB), which are replicated and stored on different ChunkServers to improve data reliability and availability.

[0043] In practical applications, one or more physical storage devices (such as cloud disks, volumes, etc.) can be deployed on a Chunkserver. To meet the current EBS's requirement for horizontal scaling of namespaces, distributed file systems typically enable federation to improve scalability, performance, and isolation. Based on this application's Chunk-level Hardlink solution, after enabling federation, when zero-copy migration of target data from one volume in a ChunkServer to another new volume, the ChunkId list of the source data blocks corresponding to the target data to be migrated is determined. Based on the ChunkId list of the source data blocks corresponding to the target data, the metadata of the source data blocks corresponding to the target data is copied to the new volume. However, the source data blocks corresponding to the target data are not copied to the new volume; instead, linked data blocks connecting to the source data blocks corresponding to the target data are created on the new volume. These linked data blocks on the new volume can access the linked source data blocks, avoiding the additional overhead of copying the target data to the new volume.

[0044] Specifically, based on the list of ChunkIds for the source data blocks corresponding to the target data, the Stream metadata service corresponding to the new volume can generate the ChunkIds for the second linked data blocks corresponding to each source data block of the target data, where there is a one-to-one correspondence between the source data blocks and the second linked data blocks. Furthermore, by using the Chunk-level Hardlink implementation scheme provided in this embodiment, hard links from the second linked data blocks to the source data blocks can be created, thus enabling the creation of linked data blocks on the new volume that link to the source data blocks corresponding to the target data.

[0045] To support chunk-level Hardlink functionality, this application provides a novel ChunkServer in-memory metadata indexing system. This system manages source data blocks and linked data blocks in a unified manner by maintaining a metadata table MetaTable and a link mapping table LinkTable in memory. It records and maintains the link mapping relationship between linked data blocks and source data blocks, and supports one or more linked data blocks linking to the same source data block. This enables efficient and accurate indexing of the same chunk data (source data block) based on LinkChunk.

[0046] The MetaTable stores metadata about the source data blocks, including the size and status of the data blocks (such as whether they are being migrated), as well as other information related to data block management, such as the creation time, modification time, and access frequency of the data blocks.

[0047] In this embodiment, a new field, the number of links to the source data block, is added to the source data block's metadata. This field records the number of linked data blocks that link to the source data block. Additionally, another marker is added to the source data block's metadata to indicate whether the source data block is publicly visible.

[0048] In this embodiment, a link table (Linktable) is introduced to store the link mapping relationship between linked data blocks and source data blocks. This ensures that the metadata of the source data blocks linked to by a linked data block can be indexed, thereby enabling access to the linked source data blocks and achieving chunk-level hardlink functionality. Linked data blocks and their linked source data blocks share the same metadata, eliminating the need to store the metadata of linked data blocks in memory, thus reducing memory usage.

[0049] This embodiment implements a chunk-level hardlink functionality scheme. Adding the number of links and their visibility status to the metadata of the source data block requires only a small amount of memory space (e.g., 4 bytes). For any source data block, regardless of the number of linked data blocks connected to it, only the link count in the source data block's metadata needs to be updated. All linked data blocks share the same metadata and use the same ChunkHandle. Each added linked data block only requires adding a link mapping record to the Linktable, and a single link mapping record also requires only a small amount of memory space (e.g., 48 bytes).

[0050] As the number of linked data blocks increases, although the number of records in the Linktable continues to increase, the number of records in the Metatable, which is used to record metadata, does not increase. Therefore, the solution for implementing the chunk-level Hardlink function in this embodiment does not increase the query time of the Metatable and can improve the efficiency of data block access.

[0051] In addition, this embodiment implements a chunk-level hardlink function, where all linked data blocks that link to the source data block share the same metadata, use the same chunkHandle, and jointly reference the same chunkBlob instance. This decouples the implementation from the underlying file system and can be applied to various file systems such as EXT4 (Fourth extended filesystem), Userspace File System (USSFS), and recordstore. It effectively shields the differences between the underlying file systems, improves scalability, and reduces implementation complexity and system maintenance costs.

[0052] In an optional embodiment, the Linktable can be stored using the same data structure as the Metatable. For example, the Linktable can be stored using a LogStructured Merge Tree (LSM-Tree) structure to support efficient write-to-read operations and improve data block query efficiency.

[0053] For example, an LSM-Tree structure typically includes two layers: the first layer comprises a memory table (MemTable) and an immutable memory table (ImmTable), and the second layer comprises a disk table (BaseTable). MemTable supports read and write operations, while ImmTable and BaseTable only support read operations. MemTable and ImmTable manage their elements using skip lists, while BaseTable manages its elements using B-trees. By storing the link mapping table (LinkTable) and the metadata table (MetaTable) using the LSM-Tree structure, write-to-read operations can be supported for both LinkTable and MetaTable, improving the query efficiency of these tables.

[0054] In one optional embodiment, the Linktable can be periodically (at preset checkpoints) written in full to a checkpoint file; and update operations on the Linktable can be persistently stored by writing to the operation log. During Chunkserver startup, the number of links in the source data blocks and the Linktable can be recovered and reconstructed using the persistently stored checkpoint file and operation log, thereby improving system stability and data reliability.

[0055] Furthermore, to ensure transactional consistency between the metadata table MetaTable and the link mapping table LinkTable during the persistence process, data from both tables needs to be written simultaneously in a continuous byte stream during the checkpoint writing process. In other words, the metadata table MetaTable and the link mapping table LinkTable are persisted to the same checkpoint file; and the operation log (oplog) is written synchronously and both tables are updated in the same thread.

[0056] Optionally, to ensure data consistency between the metadata table MetaTable and the link mapping table LinkTable in memory, a global lock is set. During the write checkpointing process of the metadata table MetaTable and the link mapping table LinkTable, both tables need to be locked simultaneously to ensure data consistency between the two tables.

[0057] In addition, since the data from the MetaTable and the LinkTable need to be written to the same checkpoint file at the same time, the format of the checkpoint file can be adaptively adjusted so that the new format of the checkpoint file can be compatible with existing checkpoint file versions, thus improving system compatibility.

[0058] For example, Figure 2 This is an architecture diagram of a Chunkserver in-memory metadata system provided in an embodiment of this application. Figure 2 As shown, the Chunkserver memory metadata system can be logically divided into the following 5 layers: service interface layer, routing table, storage engine, Blob layer, and device layer.

[0059] The Service layer (also known as the Services layer) is the interface layer through which ChunkServer provides services. It handles requests from clients or other servers and forwards these requests to the appropriate components for processing. The Services layer may include various service interfaces, such as those for deleting, reading, and copying data blocks. In this example, an interface for creating linked data blocks has been added to the Services layer. Upper-layer components (such as clients) send a hard link request to the ChunkServer for the specified source data block by calling this interface, enabling ChunkServer to establish a hard link pointing to the source data block.

[0060] The RouteTable in the ChunkServer stores path information pointing to specific data blocks (Chunks). This path information guides read and write operations on the data blocks. The RouteTable may also include other relevant information, such as the replica distribution of the data blocks and access permissions. Through the RouteTable, the ChunkServer can achieve fast location and access to data blocks. In this embodiment, the RouteTable registers the ChunkId of the data blocks (including source data blocks and linked data blocks) and maintains the path information of the data blocks, such as the mapping information from the ChunkId to the corresponding storage engine.

[0061] The storage engine is one of the core components of a ChunkServer, responsible for storing and managing data blocks, including writing, reading, and deleting them. The storage engine may employ various storage technologies, such as distributed file systems and databases. Within a ChunkServer, the storage engine must ensure high reliability, high availability, and high performance. In practical applications, multiple clients may simultaneously send read and write requests to the ChunkServer. Without an effective concurrency control mechanism, these requests can lead to data inconsistency, conflicts, or performance degradation. The storage engine in a ChunkServer provides concurrency control functionality to ensure data consistency and improve system throughput. In this embodiment, the storage engine is also responsible for managing and maintaining the Linktable (link mapping table) and the Metatable (metadata table).

[0062] The Blob (Binary Large Object) layer in ChunkServer is used to store large binary object data, such as images, audio, or multimedia files. The Blob layer provides read and write interfaces for large object data and supports operations such as uploading and downloading large object data in chunks. Through the Blob layer, ChunkServer can efficiently store and manage large object data.

[0063] The storage device layer is the lowest layer of the ChunkServer and includes physical storage devices such as cloud disks and solid-state drives (SSDs). The device layer is responsible for the physical storage and retrieval of data, including writing and reading. The device layer may also include data block verification and recovery mechanisms to ensure data integrity and reliability. In this embodiment, for example... Figure 2 As shown, the Linktable mapping table can be persistently stored on a storage device by writing checkpoint files and operation logs.

[0064] For example, Figure 3 This diagram illustrates the data structure of the metadata table and link mapping table, as well as the global lock, provided for an exemplary embodiment of this application. Figure 3 As shown, both the Linktable (link mapping table) and Metatable (metadata table) are stored using an LSM-Tree structure, divided into two layers: the first layer includes a memory table (MemTable) and an immutable memory table (ImmTable), and the second layer includes a disk table (BaseTable). Figure 3 As shown, a global lock is set up to simultaneously lock the MetaTable and LinkTable during the write checkpoint process, thereby ensuring data consistency between the two tables.

[0065] Figure 4 This is a flowchart illustrating a data processing method provided in an exemplary embodiment of this application. The execution entity in this embodiment is the block server ChunkServer in the aforementioned distributed file system, and the method in this embodiment aims to create a hard link pointing to a first data block. Figure 4 As shown, the specific steps of this method are as follows:

[0066] Step S401: In response to the hard link request for the first data block, obtain the identification information of the first data block and the identification information of the first linked data block to be linked to the first data block.

[0067] The first data block can be any data block, including source data blocks and linked data blocks.

[0068] In this embodiment, the Services layer of ChunkServer provides service interfaces to external components, including but not limited to: interfaces for creating linked data blocks (CreateLinkChunk), deleting data blocks (DeleteChunk), reading data blocks (ReadChunk), and replicating data blocks (ReplicateChunk). External components can send a hard link request to a specified first data block to ChunkServer (i.e., a call request to the CreateLinkChunk interface) by calling the CreateLinkChunk interface. This hard link request includes the identifier information (ChunkId) of the first data block to be linked to, and the identifier information (ChunkId) of the first linked data block to be linked to.

[0069] Before creating a hard link, the Stream metadata service corresponding to the first linked data block generates identification information for the first linked data block and registers this identification information, along with its mapping to the corresponding storage engine, in the RouteTable. Through the routing table, the ChunkServer can quickly locate and access the first linked data block.

[0070] Step S402: Determine the first data block as the source data block based on the identification information of the first data block.

[0071] In this embodiment, when creating a Hardlink, the first data block to be linked can be either a source data block or a link data block.

[0072] The ChunkServer first identifies whether the first data block is a source data block based on its identifier (ChunkId). Specifically, the ChunkServer checks the MetaTable to see if any source data block's metadata contains the first data block's ChunkId. If the MetaTable contains a source data block's metadata containing the first data block's ChunkId, then the first data block is determined to be a source data block. If the MetaTable does not contain a source data block's metadata containing the first data block's ChunkId, then the first data block is determined not to be a source data block.

[0073] In an optional embodiment, the ChunkServer may also cache the ChunkHandle of the source data block. In this step, the ChunkServer may first check in the cache whether there is a ChunkHandle corresponding to the ChunkId of the first data block. If there is a ChunkHandle corresponding to the ChunkId of the first data block in the cache, then the first data block is determined to be the source data block.

[0074] If the cache does not contain a ChunkHandle corresponding to the ChunkId of the first data block, then the MetaTable is searched to see if any source data block's metadata contains the ChunkId of the first data block. If the MetaTable contains a source data block's metadata containing the ChunkId of the first data block, then the first data block is determined to be a source data block. If the MetaTable does not contain a source data block's metadata containing the ChunkId of the first data block, then the first data block is determined not to be a source data block.

[0075] By first searching the cached ChunkHandle, and only searching the MetaTable if no ChunkHandle corresponding to the ChunkId of the first data block exists in the cached ChunkHandle, the query efficiency can be improved.

[0076] In this step, if the first data block is determined to be the source data block, subsequent steps S403-S404 are executed to create a Hardlink from the first linked data block to the first data block, thereby realizing the function of creating a Hardlink to the source data block.

[0077] Step S403: Based on the identification information of the first linked data block and the identification information of the first data block, add the link mapping relationship from the first linked data block to the first data block to the link mapping relationship table.

[0078] In this step, ChunkServer adds a new mapping entry to the link mapping table. This mapping entry includes the identifier information (ChunkId) of the first linked data block and the identifier information (ChunkId) of the first data block to which the first linked data block is linked, storing the link mapping relationship from the first linked data block to the first data block.

[0079] Step S404: Update the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks that are linked to the first data block.

[0080] In this embodiment, a link count is added to the metadata of the source data block to record the number of linked data blocks that link to the source data block. Initially, no linked data blocks link to the source data block, so the initial link count of the source data block is set to 0.

[0081] After adding the link mapping relationship from the first link data block to the first data block to the link mapping relationship table, the number of link data blocks linked to the first data block increases by one, so the link count in the metadata of the first data block is incremented by 1.

[0082] For example, such as Figure 5 As shown, taking the first data block as the source data block SrcChunkA and the first link data block as LinkChunkB as an example, assuming that the current number of links in SrcChunkA is 0. When performing a Hardlink operation on SrcChunkA to create LinkChunkB that links to SrcChunkA, the information of LinkChunkB is first registered in the RouteTable, and then a new mapping entry is added to the LinkTable.<LinkChunkId B,SrcChunkId A> Here, LinkChunkId B is the ChunkId of the first linked data block LinkChunkB, and SrcChunkId A is the ChunkId of the first data block SrcChunkA. This mapping indicates that LinkChunkB is linked to SrcChunkA. Further, the metadata of the first data block SrcChunkA is updated by incrementing the link count of SrcChunkA by 1, resulting in a link count of 1 for the updated SrcChunkA.

[0083] Optionally, before adding the link mapping relationship from the first link data block to the first data block in the link mapping relationship table and updating the number of links in the metadata of the first data block, ChunkServer can write operation logs for adding the link mapping relationship from the first link data block to the first data block in the link mapping relationship table and updating the number of links in the metadata of the first data block, so as to persistently store the operation logs of the link mapping relationship table and the metadata table.

[0084] The recorded operation logs may include operation time, operation content, etc., and may also include other information. The specific configuration and adjustment can be made according to actual application needs, and no specific limitations are made here.

[0085] In this embodiment, based on the aforementioned Chunkserver memory metadata system, Chunkserver can create linked data blocks that are linked to any source data block, thereby implementing the Chunk-level Hardlink function.

[0086] Figure 6 This is a flowchart illustrating a data processing method provided in an exemplary embodiment of this application. In an optional embodiment, the ChunkServer can also create a Hardlink pointing to any linked data block, implementing a recursive Hardlink function. Figure 6 As shown, the specific steps of this method are as follows:

[0087] Step S601: In response to the hard link request for the first data block, obtain the identification information of the first data block and the identification information of the first linked data block to be linked to the first data block.

[0088] The first data block can be any data block, specifically a source data block or a linked data block.

[0089] In this embodiment, the Services layer of ChunkServer provides service interfaces to external components, including but not limited to interfaces for creating linked data blocks, deleting data blocks, reading data blocks, and copying data blocks. External components can send a hard link request to a specified first data block (i.e., a call request to the Create Linked Data Block interface) to ChunkServer by calling the Create Linked Data Block (e.g., named CreateLinkChunk) interface. This hard link request includes the identification information (ChunkId) of the first data block to be linked to, and the identification information (ChunkId) of the first linked data block to be linked to.

[0090] Before creating a hard link, the Stream metadata service corresponding to the first linked data block generates identification information for the first linked data block and registers this identification information, along with its mapping to the corresponding storage engine, in the RouteTable. Through the routing table, the ChunkServer can quickly locate and access the first linked data block.

[0091] Step S602: Determine whether the first data block is a source data block or a linked data block.

[0092] In this embodiment, when creating a Hardlink, the first data block to be linked can be either a source data block or a link data block.

[0093] The ChunkServer first determines whether the first data block is a source data block or a linked data block based on its identifier (ChunkId). Specifically, the ChunkServer checks the MetaTable to see if any source data block's metadata contains the first data block's ChunkId. If the MetaTable contains a source data block's metadata containing the first data block's ChunkId, then the first data block is determined to be a source data block. If the MetaTable does not contain a source data block's metadata containing the first data block's ChunkId, then the first data block is determined not to be a source data block.

[0094] In an optional embodiment, the ChunkServer may also cache the ChunkHandle of the source data block. In this step, the ChunkServer may first check in the cache whether there is a ChunkHandle corresponding to the ChunkId of the first data block. If there is a ChunkHandle corresponding to the ChunkId of the first data block in the cache, then the first data block is determined to be the source data block.

[0095] If the cache does not contain a ChunkHandle corresponding to the ChunkId of the first data block, then the MetaTable is searched to see if any source data block's metadata contains the ChunkId of the first data block. If the MetaTable contains a source data block's metadata containing the ChunkId of the first data block, then the first data block is determined to be a source data block. If the MetaTable does not contain a source data block's metadata containing the ChunkId of the first data block, then the first data block is determined not to be a source data block.

[0096] By first searching the cached ChunkHandle, and only searching the MetaTable if no ChunkHandle corresponding to the ChunkId of the first data block exists in the cached ChunkHandle, the query efficiency can be improved.

[0097] If it's determined that the first data block is not the source data block, ChunkServer can check the LinkTable to see if a link mapping relationship (i.e., a mapping entry in the table) exists that contains the ChunkId of the first data block. This will help determine if the first data block is an existing linked data block. If a link mapping relationship containing the ChunkId of the first data block exists in the LinkTable, then the first data block is determined to be a linked data block. If no link mapping relationship containing the ChunkId of the first data block exists in the LinkTable, then the ChunkId of the first data block does not exist, and ChunkServer returns an error message to indicate to the external component that the ChunkId of the first data block does not exist.

[0098] In this step, if the first data block is determined to be the source data block, then the Hardlink from the first linked data block to the first data block is created through steps S603-S604, thereby realizing the function of creating a Hardlink to the source data block.

[0099] If the first data block is determined to be a linked data block, then a Hardlink from the first linked data block to the first data block is created through steps S605-S607, thereby realizing the function of creating a Hardlink to the linked data block (i.e., a recursive Hardlink).

[0100] Step S603: If the first data block is determined to be the source data block, add the link mapping relationship from the first linked data block to the first data block to the link mapping relationship table according to the identification information of the first linked data block and the identification information of the first data block.

[0101] In this embodiment, the specific implementation principle of step S603 is the same as that of step S403. For details, please refer to the relevant content of the aforementioned embodiment, which will not be repeated here.

[0102] Step S604: Update the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks that are linked to the first data block.

[0103] In this embodiment, the specific implementation principle of step S604 is the same as that of step S404. For details, please refer to the relevant content of the aforementioned embodiment. The steps are not repeated here.

[0104] Step S605: If the first data block is determined to be a linked data block, then determine the identification information of the first source data block to which the first data block is linked.

[0105] In this embodiment, when the first data block is determined to be a linked data block, ChunkServer first determines the identification information (ChunkId) of the first source data block to which the first data block is linked, and then directly links the first linked data block to the first source data block.

[0106] Specifically, ChunkServer can look up the mapping entry containing the ChunkId of the first data block in the LinkTable based on the ChunkId of the first data block, and obtain the ChunkId of the first source data block to which the first data block is linked from the mapping entry containing the ChunkId of the first data block.

[0107] Step S606: Based on the identification information of the first linked data block and the identification information of the first source data block, add the link mapping relationship from the first linked data block to the first source data block to the link mapping relationship table.

[0108] After determining the ChunkId of the first source data block to which the first data block is linked, in this step, ChunkServer adds a new mapping entry to the link mapping table. This mapping entry includes the identification information (ChunkId) of the first linked data block and the identification information (ChunkId) of the first source data block to which the first linked data block is linked, storing the link mapping relationship from the first linked data block to the first source data block.

[0109] Step S607: Update the number of links in the metadata of the first source data block.

[0110] After adding the link mapping relationship from the first link data block to the first source data block to the link mapping relationship table, the number of link data blocks linked to the first source data block increases by one, so the number of links in the metadata of the first source data block is incremented by 1.

[0111] For example, such as Figure 7 As shown, in Figure 5 Based on the example, let's take creating LinkChunkC by performing a Hardlink operation on LinkChunkB as an example. In this case, the first data block is the linked data block LinkChunkB, and the first linked data block is LinkChunkC. First, register the information of LinkChunkC in the RouteTable. Since the first data block to be linked is a linked data block LinkChunkB, determine the ChunkId of the first source data block SrcChunkA to which the first data block is linked. At this point, the number of links for the first source data block SrcChunkA is 1. Then, add a new mapping entry to the LinkTable.<LinkChunkId C,SrcChunkId A> Here, LinkChunkId C is the ChunkId of the first linked data block LinkChunkC, and SrcChunkId A is the ChunkId of the first source data block SrcChunkA, to which the first data block is linked. This mapping directly links LinkChunkC to SrcChunkA. Further, the metadata of the first data block SrcChunkA is updated by incrementing the link count of SrcChunkA by 1, resulting in a link count of 2 for the first data block SrcChunkA after the update.

[0112] Optionally, before adding the link mapping relationship from the first link data block to the first source data block to the link mapping relationship table, and updating the number of links in the metadata of the first source data block, ChunkServer can write operation logs for adding the link mapping relationship from the first link data block to the first source data block to the link mapping relationship table, and updating the number of links in the metadata of the first source data block, to persistently store the operation logs of the link mapping relationship table and the metadata table.

[0113] The recorded operation logs may include operation time, operation content, etc., and may also include other information. The specific configuration and adjustment can be made according to actual application needs, and no specific limitations are made here.

[0114] In this embodiment, when creating the first link data block that links to the link data block (first data block), the first link data block is directly linked to the first source data block that the first data block links to, and the number of links in the first source data block is updated. This achieves the recursive creation of hardlinks pointing to the link data block, which can ensure the consistency between the link data block and the source data block, and effectively ensure the accuracy of the number of links in the source data block.

[0115] Figure 8 A flowchart illustrating the deletion of a data block as provided in an exemplary embodiment of this application. Figure 8 As shown, the specific steps of this method are as follows:

[0116] Step S801: In response to the deletion request for the second data block, obtain the identification information of the second data block.

[0117] The second data block can be any data block, specifically a source data block or a linked data block.

[0118] In this embodiment, the Services layer of ChunkServer provides service interfaces to external components, including but not limited to interfaces for creating linked data blocks, deleting data blocks, reading data blocks, and copying data blocks. External components can send a deletion request for a specified second data block to ChunkServer by calling the delete data block interface (e.g., named DeleteChunk). This deletion request contains the identification information (ChunkId) of the second data block to be deleted.

[0119] Step S802: Determine whether the second data block is a source data block or a linked data block.

[0120] Before deleting the second data block, ChunkServer first determines whether the second data block is a source data block or a linked data block based on the identification information (ChunkId) of the first data block.

[0121] In this step, it is determined whether the second data block is a source data block or a linked data block. The specific implementation principle is the same as that of determining whether the first data block is a source data block or a linked data block in the aforementioned step S602. For details, please refer to the relevant content of the aforementioned embodiment. The steps are not repeated here.

[0122] In this step, if the second data block is determined to be the source data block, the second data block is deleted through steps S803-S805, thereby deleting the source data block.

[0123] If the second data block is determined to be a linked data block, then the second data block is deleted through steps S806-S807, thereby deleting the linked data block.

[0124] Step S803: If it is determined that the second data block is the source data block, then determine the number of links for the second data block.

[0125] In this embodiment, when deleting any source data block, the following two rules must be followed: First, the data of the source data block can only be actually deleted after all linked data blocks connected to the source data block have been deleted; second, if there are still linked data blocks connected to the source data block, the source data block is marked as invisible so that it will not be detected (sniffed) by the main server, but the data of the source data block can still be accessed through the linked data blocks connected to the source data block.

[0126] In this step, if it is determined that the second data block to be deleted is the source data block, ChunkServer first determines the number of links to the second data block before deleting it, and determines whether there are any linked data blocks that link to the source data block based on the number of links to the second data block.

[0127] Specifically, ChunkServer can look up the metadata of the second data block from the MetaTable based on the identification information (ChunkId) of the second data block, and extract the number of links of the second data block from the metadata of the second data block.

[0128] Step S804: If the number of links in the second data block is greater than 0, then mark the second data block as invisible.

[0129] If the number of links to the second data block is greater than 0, it means that there is a linked data block that links to the source data block. According to the two rules mentioned above that must be followed to delete the source data block, the second data block cannot be deleted directly. ChunkServer marks the second data block as invisible by modifying the visibility status information in the metadata of the second data block.

[0130] Step S805: If the number of links in the second data block is equal to 0, then delete the metadata of the second data block and the second data block.

[0131] If the number of links to the second data block is 0, it means that there are no linked data blocks linking to the source data block. ChunkServer then deletes the metadata of the second data block and the second data block.

[0132] When deleting the second data block, the ChunkServer must delete the second data block stored in the physical storage device. Furthermore, the ChunkServer must clear the registration information of the second data block, that is, delete the ChunkId and other related information of the second data block from the RouteTable.

[0133] Step S806: If it is determined that the second data block is a linked data block, then determine the second source data block to which the second data block is linked.

[0134] If the second data block is determined to be a linked data block, ChunkServer determines the ChunkId of the second source data block to which the second data block is linked by looking up the LinkTable.

[0135] Step S807: Decrease the number of links in the metadata of the second source data block by 1, and delete the link mapping relationship from the second data block to the second source data block in the link mapping relationship table.

[0136] In this embodiment, when deleting a linked data block (second data block), the number of links to the source data block (second source data block) linked to by the second data block needs to be reduced, and the link mapping relationship between the second data block and the second source data block needs to be removed.

[0137] Specifically, ChunkServer can update the metadata of the second source data block in the MetaTable based on the ChunkId of the second source data block to which the second data block is linked, and decrement the number of links to the second source data block by 1. Furthermore, ChunkServer updates the LinkTable, deleting any link mappings from the second data block to the second source data block that were present in the LinkTable.

[0138] In addition, if the second data block to be deleted is a linked data block, ChunkServer also needs to clear the registration information of the second data block, that is, delete the ChunkId and other related information of the second data block from the RouteTable.

[0139] Step S808: If the number of links in the metadata of the second source data block minus 1 equals 0, and the second source data block is in an invisible state, then mark the second source data block as visible and delete the metadata of the second source data block and the second source data block.

[0140] If the number of links to the second source data block is reduced to 0 after decrementing by 1 in step S807, and the second source data block is determined to be in an invisible state based on its metadata, then the process of deleting the source data block needs to be triggered.

[0141] In this step, when deleting the last linked data block that links to the second source data block, if the second source data block is in an invisible state, it is first marked as visible before the physical deletion process is performed. This ensures that if the deletion of the second source data block fails, the visible second source data block can be detected (sniffed) by the master server. The master will then send another request to the chunk server to delete the second source data block, causing the chunk server to delete the second source data block, thus avoiding the problem of data residue in the second source data block.

[0142] In an optional embodiment, if the number of links in the metadata of the second source data block minus 1 equals 0, and the second source data block is in an invisible state, then ChunkServer can delete the metadata of the second source data block and the second source data block.

[0143] Optionally, before performing any update operations on the link mapping table and the metadata table, ChunkServer can persist the operation logs for the link mapping table and the metadata table. The recorded operation logs may include the operation time, operation content, and other information. These can be configured and adjusted according to actual application needs, and are not specifically limited here.

[0144] In addition, since the deletion operation involves updating the metadata of the source data block and requires persistent storage through the write operation log, which involves disk write operations, the deletion logic can be handled after concurrency control in ChunkServer.

[0145] In this embodiment, when deleting a source data block, the system ensures that all linked data blocks connected to the source data block have been deleted before the data in the source data block can be actually deleted. If there are still linked data blocks connected to the source data block, the source data block is marked as invisible so that it cannot be detected (sniffed) by the main server, but the data in the source data block can still be accessed through the linked data blocks connected to it. When deleting a linked data block, the number of links to the source data blocks linked to by the linked data block needs to be decremented, the link mapping relationship between the linked data block and the source data block needs to be removed, and the link mapping table LinkTable in memory needs to be updated. If the number of links to the source data block is reduced to 0 and the source data block is in an invisible state, the process of deleting the linked source data block needs to be triggered. First, the linked source data block is marked as visible, and then the physical deletion of the source data block is performed. In the event that the deletion of the source data block fails, it can be ensured that the visible source data block can be detected by the Master. The Master will send a request to ChunkServer to delete the source data block again, thereby clearing the data residue of the source data block. This can ensure the correctness of the deletion operation of the source data block and the linked data block and the data consistency.

[0146] Based on any of the foregoing embodiments, a preset checkpoint can be configured on the ChunkServer. When a checkpoint arrives, the ChunkServer writes all data from the MetaTable and LinkTable to the same checkpoint file, thereby achieving persistent storage of the MetaTable and LinkTable. The preset checkpoint can be set periodically or at regular intervals, and can be configured and adjusted according to actual application requirements; no specific limitations are made here.

[0147] To ensure data consistency between the metadata table MetaTable and the link mapping table LinkTable in memory, a global lock is used to lock both MetaTable and LinkTable simultaneously before persisting them to the same checkpoint file.

[0148] In addition, before performing update operations (such as adding, modifying, or deleting) on ​​the metadata table MetaTable and the link table LinkTable, ChunkServer persistently stores the operation logs for both the link table LinkTable and the metadata table MetaTable. Optionally, ChunkServer can complete the writing of operation logs and the updating of the metadata table MetaTable and the link table LinkTable in the same thread to ensure transactional consistency of the metadata table MetaTable and the link table LinkTable during the persistence process.

[0149] Optionally, after persistently storing the metadata table MetaTable and the link mapping table LinkTable in the same checkpoint file, ChunkServer can delete the operation logs of the metadata table and the link mapping table stored before the checkpoint to save storage space.

[0150] Based on persistent storage of checkpoint files and operation logs, the metadata table MetaTable and the link mapping table LinkTable can be recovered and rebuilt.

[0151] In response to a data recovery command, ChunkServer can rebuild the metadata table and the link mapping table based on the checkpoint file stored at the latest checkpoint, as well as the operation logs of the metadata table and the link mapping table stored after the latest checkpoint.

[0152] Specifically, in response to a data recovery command, ChunkServer can obtain the checkpoint file stored at the latest checkpoint, as well as the operation logs of the metadata table and the link mapping table. Based on the checkpoint file stored at the latest checkpoint, ChunkServer can recover the metadata table MetaTable and the link mapping table LinkTable at the latest checkpoint. Furthermore, based on the operation logs of the metadata table and the link mapping table stored after the latest checkpoint, the same operations are performed on the recovered metadata table MetaTable and link mapping table LinkTable at the latest checkpoint to reconstruct the latest metadata table MetaTable and link mapping table LinkTable.

[0153] In this embodiment, ChunkServer persistently stores the metadata table and the link mapping table in the same checkpoint file at preset checkpoints, and also persistently stores the operation logs of the link mapping table and the metadata table. Based on the persistently stored checkpoint file and operation logs, the metadata table MetaTable and the link mapping table LinkTable can be restored and rebuilt, improving the stability and reliability of the system.

[0154] In an example scenario, the Chunk-level Hardlink implementation scheme provided in this application enables file cloning. In practical applications, one or more physical storage devices (such as cloud disks, volumes, etc.) can be deployed on the Chunkserver. When it is necessary to clone a target file stored on the first cloud disk to the second cloud disk, the metadata of the source data block corresponding to the target file can be copied to the second cloud disk. It is not necessary to copy the source data block corresponding to the target file to the second cloud disk. Instead, a link data block is created on the second cloud disk that links to the source data block corresponding to the target file. Through the link data block on the second cloud disk, the linked source data block can be accessed, thereby enabling access to the target file. This avoids the additional overhead of copying all source data blocks of the target file to the second cloud disk.

[0155] Specifically, the Master can determine the list of ChunkIds for the source data blocks corresponding to the target file to be migrated. Based on this list, it generates ChunkIds for the second linked data blocks corresponding to each source data block of the target file, ensuring a one-to-one correspondence between the source data blocks and the second linked data blocks. Further, the Master calls the Chunkserver's interface for creating linked data blocks. The Chunkserver implements a Chunk-level Hardlink scheme to create hard links from the second linked data blocks to the source data blocks, thus enabling the creation of second linked data blocks on the second cloud disk that link to the various source data blocks corresponding to the target file. The source data blocks linked to by these second linked data blocks can then be accessed through the second linked data blocks on the second cloud disk.

[0156] Figure 9 This is a schematic diagram of the structure of a block server provided in an embodiment of this application. Figure 9As shown, the block server includes a memory 901 and a processor 902. The memory 901 stores computer-executable instructions and can be configured to store various other data to support operations on the block server. The processor 902 is communicatively connected to the memory 901 and executes the computer-executable instructions stored in the memory 901 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they achieve are similar and will not be repeated here.

[0157] Optional, such as Figure 9 As shown, this server also includes other components such as a firewall 903, a load balancer 904, a communication component 905, and a power supply component 906. Figure 9 The diagram only shows some components and does not mean that the block server only includes... Figure 9 The components shown.

[0158] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method of any of the foregoing embodiments. The specific functions and technical effects to be achieved are not described here.

[0159] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments. The computer program is stored in a readable storage medium, and at least one processor of the server can read the computer program from the readable storage medium. The execution of the computer program by the at least one processor causes the server to perform the technical solution provided in any of the above method embodiments. The specific functions and the technical effects that can be achieved are not described here.

[0160] This application provides a chip, including a processing module and a communication interface. The processing module is capable of executing the technical solution of the server in the aforementioned method embodiments. Optionally, the chip further includes a storage module (e.g., a memory), which stores instructions. The processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solution provided in any of the aforementioned method embodiments.

[0161] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0162] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), a graphics processing unit (GPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in at least one processor.

[0163] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0164] The aforementioned storage device can be object storage service (OSS).

[0165] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0166] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G) / Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0167] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0168] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0169] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside within an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within an electronic device or host device.

[0170] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0171] The order of the embodiments described above is merely for illustrative purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. "Multiple" means two or more, unless otherwise explicitly specified.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0173] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0174] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data processing method, characterized in that, include: In response to a hard link request for the first data block, the identification information of the first data block and the identification information of the first linked data block to be linked to the first data block are obtained. If the first data block is determined to be the source data block, then according to the identification information of the first linked data block and the identification information of the first data block, the link mapping relationship from the first linked data block to the first data block is added to the link mapping relationship table; Update the number of links in the metadata of the first data block, where the number of links represents the number of linked data blocks that are linked to the first data block.

2. The method according to claim 1, characterized in that, Also includes: If the first data block is determined to be a linked data block, then the identification information of the first source data block to which the first data block is linked is determined; Based on the identification information of the first linked data block and the identification information of the first source data block, add the link mapping relationship from the first linked data block to the first source data block to the link mapping relationship table; Update the number of links in the metadata of the first source data block.

3. The method according to claim 1, characterized in that, Also includes: In response to a deletion request for the second data block, obtain the identification information of the second data block; If it is determined that the second data block is the source data block, then determine the number of links to the second data block; If the number of links to the second data block is greater than 0, then the second data block is marked as invisible. If the number of links to the second data block is 0, then delete the second data block and its metadata.

4. The method according to claim 3, characterized in that, Also includes: If, based on the identification information of the second data block, it is determined that the second data block is a linked data block, then the second source data block to which the second data block is linked is determined. Decrease the number of links in the metadata of the second source data block by 1, and delete the link mapping relationship from the second data block to the second source data block in the link mapping relationship table.

5. The method according to claim 4, characterized in that, Also includes: If the number of links in the metadata of the second source data block minus 1 equals 0, and the second source data block is in an invisible state, then the second source data block is marked as visible, and the second source data block and its metadata are deleted.

6. The method according to any one of claims 1-5, characterized in that, Also includes: The operation logs of the link mapping table and the metadata table are persistently stored, wherein the metadata table is used to store the metadata of the source data blocks.

7. The method according to claim 6, characterized in that, Also includes: At a preset checkpoint, the metadata table and the link mapping table are persistently stored in the same checkpoint file.

8. The method according to claim 7, characterized in that, Before persisting the metadata table and the link mapping table to the same checkpoint file, the method further includes: Lock the metadata table and the link mapping table.

9. The method according to claim 7, characterized in that, After persistently storing the metadata table and the link mapping table in the same checkpoint file at a preset checkpoint, the method further includes: Delete the operation logs of the metadata table and the link mapping table stored before the checkpoint.

10. The method according to claim 7, characterized in that, Also includes: In response to a data recovery command, the metadata table and the link mapping table are reconstructed based on the checkpoint file stored at the latest checkpoint, and the operation logs of the metadata table and the link mapping table stored after the latest checkpoint.

11. A block server, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the block server to perform the method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-10.

Images