A distributed block storage method, device and electronic equipment

Abstract

The application provides a storage method and device of distributed blocks and an electronic device. By aggregating data belonging to the same volume and having continuous or close LBA in the cache, and then writing the data into a target stripe group in which the last write volume identifier and LBA information are recorded, the data of the same backup task is physically stored in the same stripe group. When a deletion instruction is received, whether the stripe group still contains valid data can be quickly checked. If not, the whole stripe group is directly released, so that the cumbersome operation of moving valid data when deleting due to mixed writing of data of different tasks can be avoided, the data moving amount of space recycling in the mechanical disk scene can be reduced, and the system overhead and the influence on the front-end IO performance are reduced.

Description

Technical Field

[0001] This application relates to the field of distributed block storage technology, and in particular to a distributed block storage method, apparatus and electronic device. Background Technology

[0002] Distributed block storage systems typically consist of multiple server nodes, each configured with storage media such as hard disk drives (HDDs) and solid-state drives (SSDs). Data services are provided and interconnected with each node via a front-end and back-end network. To fully utilize storage resources, the system distributes data across nodes and employs multiple replicas or erasure coding to ensure reliability. During writes, SSDs are used as caches, employing a redirected write strategy. Data is first written to the SSDs and then aggregated and flushed to the HDDs. Simultaneously, a mapping table records the correspondence between logical and physical addresses.

[0003] When the cached data volume reaches the stripe size, the I / O (Input / Output) processing service of each node requests or selects a stripe group and writes the cached data to the mechanical disk as a whole stripe. When data is deleted or overwritten, the remaining valid data in the stripe group must be moved to a new stripe before the original stripe group's reclaimable space can be released. However, in backup scenarios, multiple backup tasks run simultaneously, and all written data is mixed indiscriminately in the same stripe group, resulting in physical interleaving of data from different tasks and different lifecycles. When deleting old backup data, because it is mixed with valid data from other tasks in the same stripe group, the entire stripe group cannot be released directly. A time-consuming data migration must be triggered to move other valid data out of the stripe group before space can be reclaimed. This migration process is slow and costly on the mechanical disk and interferes with front-end I / O processing, affecting system performance. Summary of the Invention

[0004] This application provides a distributed block storage method, apparatus, and electronic device to solve the problems of slow mechanical disk speed, high overhead, interference with front-end IO processing, and significant impact on system performance caused by data migration during data deletion scenarios.

[0005] The first aspect of this application provides a method for storing distributed blocks, the method comprising: When a data write stripe event is detected, the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement are aggregated to obtain the current aggregated data. Find the target stripe group whose volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and whose LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement; Write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA currently being written and the volume identifier; When a delete command indicating the deletion of the correspondence between LBA and stripe is detected, check whether the stripe group containing the stripe contains valid data. If not, release the entire stripe group directly.

[0006] In some embodiments, The difference in LBA between any two adjacent logical blocks in the current aggregated data is less than a preset first threshold; or, The difference between the LBA of the last logical block and the LBA of the first logical block in the current aggregated data is less than the preset second threshold. And / or, The second proximity requirement is that the difference between the LBA of the last write and the LBA of the current aggregated data is less than a preset third threshold.

[0007] In some embodiments, the detected data write stripe event includes: Upon receiving a write command, the data is added to the memory cache and then appended to the end of the SSD cache partition. Monitor the data already stored in the SSD cache partition. If the data already stored in the SSD cache partition is greater than a set data amount, then it is determined that the data write stripe event has been detected.

[0008] In some embodiments, the write information further includes: the last write timestamp of the target stripe group; The method further includes: checking the stripe group list; if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clearing the write information of that stripe group.

[0009] In some embodiments, before finding a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy a second proximity requirement, the method further includes: If no target strip group that meets the conditions is found, it is further determined whether there is a strip group whose time difference between the last write timestamp and the current timestamp exceeds a preset time; if so, the current aggregated data is written to the strip group and the write information of the strip group is updated.

[0010] In some embodiments, the method further includes: When the SSD cache partition is not enough space, data belonging to different volumes or whose LBA does not meet the first proximity requirement will be aggregated in the cache. Find the specific stripe group used to store out-of-order data, and write the aggregated out-of-order data into that specific stripe group; Upon detecting a cache release event, the corresponding cache is released sequentially and the corresponding data in the SSD cache partition is deleted.

[0011] In some embodiments, checking whether the band group to which the band belongs contains valid data includes: Obtain the bitmap information of the stripe group, the bitmap information being used to identify the valid state of each physical block within the stripe group; Traverse the bitmap information. If the state of all physical blocks in the bitmap information is invalid, then determine that the stripe group does not contain valid data and trigger the release process.

[0012] A second aspect of this application provides a distributed block storage device, the device comprising: The first aggregation module is configured to, when a data write stripe event is detected, first aggregate the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement, to obtain the current aggregated data. The first lookup module is configured to find a target stripe group in which the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy the second proximity requirement. The write module is configured to write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA currently being written and the volume identifier. The first inspection module is configured to check whether the stripe group containing the stripe contains valid data when it detects a deletion instruction that indicates the deletion of the correspondence between LBA and stripe. If it does not contain valid data, the entire stripe group is released directly.

[0013] In some embodiments, the difference in LBA between any two adjacent logical blocks in the current aggregated data is less than a preset first threshold; or, The difference between the LBA of the last logical block and the LBA of the first logical block in the current aggregated data is less than the preset second threshold. And / or, The second proximity requirement is that the difference between the LBA of the last written data and the LBA of the current aggregated data is less than a preset third threshold. And / or, The monitored data write-to-strip events include: Upon receiving a write command, the data is added to the memory cache and then appended to the end of the SSD cache partition. Monitor the data already stored in the SSD cache partition. If the data already stored in the SSD cache partition is greater than a set data amount, then it is determined that the data write stripe event has been detected. And / or, The write information also includes: the last write timestamp of the target stripe group; The device further includes: a second inspection module, configured to inspect the stripe group list, and if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clear the write information of that stripe group; And / or, Before finding a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy the second proximity requirement, the apparatus further includes: The determination module is configured to, if no target stripe group that meets the conditions is found, further determine whether there is a stripe group whose time difference between the last write timestamp and the current timestamp exceeds a preset time; if so, write the current aggregated data to the stripe group and update the write information of the stripe group. And / or, The device further includes: The second aggregation module is configured to aggregate data in the cache that belong to different volumes or whose LBA does not meet the first proximity requirement when the SSD cache partition is insufficient. The second search module is configured to search for a specific stripe group for storing out-of-order data and write the aggregated out-of-order data into that specific stripe group. The release module is configured to release the corresponding cache and delete the corresponding data in the SSD cache partition in sequence when a cache release event is detected; And / or, The inspection module checks whether the strip group to which the strip belongs contains valid data, including: Obtain the bitmap information of the stripe group, the bitmap information being used to identify the valid state of each physical block within the stripe group; Traverse the bitmap information. If the state of all physical blocks in the bitmap information is invalid, then determine that the stripe group does not contain valid data and trigger the release process.

[0014] A third aspect of this application provides an electronic device, which includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method steps of any of the first aspects.

[0015] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application embodiment aggregates data belonging to the same volume in the cache that has consecutive or close LBAs, and then writes it to a target stripe group that records the last written volume identifier and LBA information. This ensures that data from the same backup task is physically stored in the same stripe group. When a delete command is received, it can quickly check whether the stripe group still contains valid data. If not, the entire stripe group is released directly. This avoids the cumbersome operation of moving valid data during deletion due to mixed writing of data from different tasks. It can reduce the amount of data moving during space reclamation in mechanical disk scenarios, reduce system overhead, and reduce the impact on front-end IO performance.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this application, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 This is a flowchart illustrating a distributed block storage method using some exemplary embodiments; Figure 2 This is a flowchart illustrating a distributed block storage method according to some other exemplary embodiments; Figure 3 This is a flowchart illustrating a distributed block storage method with some exemplary embodiments; Figure 4 These are structural schematic diagrams of a distributed block storage device illustrating some exemplary embodiments; Figure 5 These are schematic diagrams illustrating the structure of an electronic device through some exemplary embodiments. Detailed Implementation

[0019] Distributed block storage systems typically consist of multiple server nodes, each configured with a certain number of storage media, such as hard disk drives (HDDs) and SSDs. Interconnection between nodes is achieved through a backend network, while a frontend network provides data services. To maximize storage space and performance, the system distributes write data across the storage media of each node and employs techniques such as multiple replicas or erasure coding to improve data reliability. During data writing, to improve the write performance of HDDs, SSDs are typically used as a cache. Data is first written to the SSD cache via redirected writes, and then aggregated and flushed to the HDDs. Simultaneously, to avoid the readback overhead in erasure coding scenarios caused by directly overwriting write data, the system typically uses a redirected write strategy to write to the HDDs. New space is allocated for each write, and a mapping table records the correspondence between logical and physical addresses.

[0020] When running the IO processing service on each node, when the cached data volume reaches the stripe size, the IO processing service requests or selects a stripe group and writes the cached data to the mechanical disk in whole stripes. When data is deleted or overwritten, a small amount of still valid data in the stripe group needs to be moved to a new stripe before the entire stripe group can be released to reclaim space. However, in backup scenarios, multiple backup tasks run simultaneously, mixing all written data indiscriminately in the same stripe group, causing data from different backup tasks and at different lifecycles to be physically intertwined. When it is necessary to delete a backup that has already been backed up, because this data is mixed with valid data from other backup tasks in the same stripe group, the entire stripe group cannot be released directly. A time-consuming data migration operation must be triggered to move the still valid data in the stripe group to a new stripe before space can be reclaimed. This migration process is slow and costly for the mechanical disk and interferes with the normal processing of front-end IO, affecting system performance.

[0021] This application provides a method, apparatus, and electronic device for storing distributed blocks.

[0022] To make the method provided in this application easier to understand, the method provided in this application will be described in detail below with reference to the accompanying drawings and embodiments: Please see Figure 1 , Figure 1 This is a flowchart illustrating a distributed block storage method with some exemplary embodiments. The method may include the following steps: Step S110: When a data write stripe event is detected, first aggregate the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement to obtain the current aggregated data.

[0023] In this step, the IO processing service running on each storage node continuously monitors data write striping events. The trigger condition for this data write striping event can be: after receiving a write command from the front end, the IO processing service simultaneously adds the data to be written to the memory cache and appends it to the end of the SSD cache partition, then returns a write success message; the background process monitors the total amount of data stored in the SSD cache partition in real time. If the total amount of stored data exceeds a set data size, a data write striping event is detected, where the preset data size can be the stripe's data size. At this point, the IO processing service retrieves the data blocks to be flushed from the memory cache and groups and aggregates the data blocks according to the logical volume identifier to which each data block belongs, such as the volume ID (Identifier) ​​and its LBA (Logical Block Address).

[0024] Specifically, data blocks belonging to the same volume can be filtered out, and then the LBAs of each data block within the same volume can be checked to see if they are consecutive or meet a first proximity requirement. For example, it can be determined whether the difference between the LBAs of any two adjacent logical blocks is less than a preset first threshold, such as 4KB, or whether the difference between the largest and smallest LBAs in the entire data block set is less than a preset second threshold, such as 1GB. If the above conditions are met, these data blocks are clustered into the current aggregated data; other data blocks that do not meet the conditions are either not processed temporarily or enter other aggregation processes.

[0025] Step S120: Find the target stripe group whose volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and whose LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement.

[0026] In this step, the IO processing service maintains a list of stripe groups. Each stripe group in this list corresponds to an allocated physical storage unit, which consists of multiple stripes spanning multiple disks. Metadata information for the most recently written data is associated with each stripe group, including the volume identifier, the starting LBA, and the last write timestamp. After obtaining the volume identifier and LBA range of the current aggregated data, the IO processing service traverses the stripe group list to find a target stripe group that meets the following two conditions: comparing the volume identifier of the current aggregated data with the volume identifier of the last written data recorded in the stripe group; and calculating the LBA of the current aggregated data, for example, by taking the absolute value of the difference between the starting or median LBA of the aggregated data and the LBA of the last write recorded in the stripe group, and determining whether this difference is less than a preset third threshold, such as 1GB. If a stripe group that simultaneously meets the conditions of the same volume identifier and close LBA is found, this stripe group is identified as the target stripe group. In one specific implementation, the current aggregated data comes from volume A and its LBA range is 2GB-2GB+256KB. If a stripe group in the stripe group list records that its last written volume is A and its last written LBA is 1.9GB, and the difference between the two is approximately 100MB, which is less than the 1GB threshold, then the stripe group is matched. If the volume identifier is different or the LBA difference exceeds the threshold, then the condition is not met. This ensures that consecutive or adjacent data belonging to the same backup task are assigned to the same stripe group, creating conditions for subsequent overall release.

[0027] Step S130: Write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA and volume identifier currently being written.

[0028] In this step, the IO processing service selects one or more free stripes from the target stripe group. In a specific embodiment, the stripe group consists of 1024 stripes, each stripe spanning a fixed-size block across multiple disks, such as 256KB. The current aggregated data is then written to the selected stripe as a whole stripe. During writing, the size of the current aggregated data may occupy one or more consecutive stripes. The IO processing service ensures that the writing process is sequential large-block writing, avoiding random small writes to the mechanical disk. After the data is successfully written to the stripe, the IO processing service immediately updates the write information of the target stripe group, that is, updating the volume identifier in the write information to the volume identifier to which the current aggregated data belongs, and updating the LBA of the previous write to the logical block address corresponding to the current aggregated data. Specifically, the start LBA or end LBA of the aggregated data can be retrieved. For a write consisting of multiple stripes, only the LBA range information of that write can be recorded. For example, the start LBA and length, or the end LBA can be recorded as a reference for the next match.

[0029] In one feasible embodiment, the write information may further include the last write timestamp of the target stripe group; the method may further include: checking the stripe group list, and if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clearing the write information of that stripe group.

[0030] For example, the IO processing service adds a last write timestamp to each stripe group in the maintained stripe group list. Whenever data is successfully written to a stripe group, the IO processing service updates the last write timestamp of that stripe group to the current system time. Simultaneously, the IO processing service starts a periodically executing background thread, for example, every 60 seconds. This thread traverses the entire stripe group list, reads the last write timestamp of each stripe group, and compares it with the current system time. If the interval between the last write timestamp and the current timestamp of a stripe group exceeds a preset time, such as 30 minutes, it is determined that the backup data belonging to that stripe group has been written completely, and no new continuous data will be written in the short term. At this time, the IO processing service clears the write information recorded in the stripe group, including clearing the volume identifier of the last write, the LBA of the last write, and the last write timestamp itself. After clearing the write information, the stripe group becomes a stripe group with no recorded information. When subsequent write requests cannot find a matching target stripe group, the current stripe group can be reused as a candidate stripe group. The new write data will overwrite its remaining blank space, and the new write information will be re-recorded. This avoids long-term inactive stripe groups occupying matching resources, and also allows the remaining space in the stripe group corresponding to the written backup data to be reused by other subsequent tasks, improving storage space utilization.

[0031] Step S140: When a deletion instruction indicating the deletion of the correspondence between LBA and stripe is detected, check whether the stripe group to which the stripe belongs contains valid data. If not, release the entire stripe group directly.

[0032] In this step, the IO processing service continuously monitors deletion commands from the front end, which can be UNMAP (unmap) commands. When a deletion command is received for a continuous range of LBAs, the IO processing service first searches the mapping table based on the LBA range to be deleted, finds one or more stripes and offsets corresponding to that LBA range, and then removes the mapping relationship between these LBAs and stripes. After unmapping, the IO processing service further determines the stripe group to which each unmapped stripe belongs and checks whether the stripe group still contains any valid data. If not, the entire stripe group is released directly.

[0033] This application embodiment aggregates data belonging to the same volume in the cache that has consecutive or close LBAs, and then writes it to a target stripe group that records the last written volume identifier and LBA information. This ensures that data from the same backup task is physically stored in the same stripe group. When a delete command is received, it can quickly check whether the stripe group still contains valid data. If not, the entire stripe group is released directly. This avoids the cumbersome operation of moving valid data during deletion due to mixed writing of data from different tasks. It can reduce the amount of data moving during space reclamation in mechanical disk scenarios, reduce system overhead, and reduce the impact on front-end IO performance.

[0034] In some embodiments, before finding a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy the second proximity requirement, the method may further include: if no target stripe group that meets the conditions is found, further determining whether there is a stripe group where the time difference between the last write timestamp and the current timestamp exceeds a preset duration; if so, writing the current aggregated data into the stripe group and updating the write information of the stripe group.

[0035] For example, before finding a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement, the IO processing service first traverses the stripe group list for matching based on the condition that the volume identifiers are the same and the LBAs meet the second proximity requirement. If no stripe group meeting the above conditions is found after the traversal, the stripe group list is traversed again, and the last write timestamp of each stripe group record is checked. The difference between the timestamp and the current system time is calculated. If the difference exceeds a preset time, such as 30 minutes, it indicates that the backup data corresponding to the stripe group has been written and no new data has been added for a long time. At this time, even if the volume identifier of the stripe group record is different from the volume identifier of the current aggregated data, or the LBA of the record is significantly different from the current LBA, the system still regards it as a reusable target stripe group. The IO processing service selects one from these options, such as the first found stripe group or the one with the lowest utilization. It writes the current aggregated data to the free stripe in that stripe group and updates the write information of that stripe group, namely, updating the volume identifier to the volume identifier of the current aggregated data, updating the LBA of the last write to the LBA of the current aggregated data, and updating the last write timestamp to the current time.

[0036] This application embodiment determines the last write timestamp of a stripe group by analyzing it, providing effective alternative write paths for the current aggregated data. This allows stripe groups that have not been written to for a long time to be reused, utilizing the remaining storage space of idle stripe groups and improving the overall utilization of storage resources. Since the original backup data corresponding to these timed-out stripe groups has already been written and has not been accessed for a long time, even if newly written data is physically mixed with it, it will not increase the data migration overhead during future space reclamation.

[0037] In some embodiments, see Figure 2 , Figure 2 This is a flowchart illustrating a distributed block storage method according to some other exemplary embodiments. The method may further include the following steps: Step S210: When the SSD cache partition is not enough space, aggregate the data in the cache that belong to different volumes or whose LBA does not meet the first proximity requirement.

[0038] In this step, the IO processing service monitors the available space of the SSD cache partition in real time. When it detects that the available space of the SSD cache partition is lower than a preset threshold, such as 10% of the total capacity or less than 1GB of absolute remaining space, it determines that the cache space is insufficient and some data needs to be flushed to release the cache space. At this time, the IO processing service no longer follows the aggregation rules in step S110, that is, only aggregating data from the same volume with consecutive or close LBAs. Instead, it retrieves data blocks to be flushed in batches from the memory cache and forcibly aggregates data blocks that belong to different volumes or belong to the same volume but whose LBAs do not meet the first proximity requirement, such as data blocks whose adjacent LBA differences exceed the preset threshold or whose overall LBA span exceeds the second threshold, to form a set of aggregated data. For example, the current cache contains data at LBAs 0-8KB of volume A, data at LBA 100GB of volume B, and data at LBA 10TB of volume A. These three are neither from different volumes nor meet the LBA proximity requirement, but when the SSD space is insufficient, they will be aggregated and enter as a whole into the subsequent aggregation process.

[0039] Step S220: Locate the specific stripe group for storing out-of-order data, and write the aggregated out-of-order data into that specific stripe group.

[0040] In this step, the IO processing service maintains one or more specific stripe groups dedicated to storing out-of-order data. These specific stripe groups can be pre-allocated and marked during system initialization or dynamically specified during runtime. When data from different volumes or with incompatible LBAs is forcibly aggregated, the IO processing service searches the stripe group list for specific stripe groups marked for storing out-of-order data. During the search, if no marked specific stripe groups exist, it prioritizes stripe groups with available free stripes and whose write information has timed out or been cleared to fully utilize existing space. Alternatively, a new stripe group can be temporarily requested, marked as a specific stripe group, and added to the stripe group list. After finding the target specific stripe group, the IO processing service writes the aggregated out-of-order data to the free stripes within that specific stripe group as a whole stripe and updates the write information of that stripe group. By directing out-of-order data to independent specific stripe groups, the mixing of out-of-order data with normal continuous backup data is avoided, thereby protecting the lifecycle consistency of data in ordinary stripe groups and ensuring that ordinary stripe groups still have a high probability of being released as a whole when backup data is deleted.

[0041] Step S230: When a cache release event is detected, release the corresponding cache in sequence and delete the corresponding data in the SSD cache partition.

[0042] In this step, after the IO processing service completes writing the aggregated data to the stripe group and updating the mapping table, it triggers a cache release event. Specifically, the IO processing service marks the cached data blocks that have been successfully written to the stripe as pending release in the memory cache and records the order identifier of each data block when it was written to the SSD cache partition, such as a sequence number sorted by write time. Subsequently, the IO processing service initiates the cache release process, which involves traversing each data block in the memory cache in the order it was written to the cache. Whenever a data block marked as pending release is encountered, the memory cache space occupied by that data block is immediately released, and a deletion command is sent to the SSD cache management module. Based on the storage location of the data block in the SSD cache partition, the area is marked as invalid or directly deleted from the mapping, thereby realizing the reclamation of the corresponding data in the SSD cache space.

[0043] In this embodiment, when the SSD cache partition space is insufficient, data from different volumes or with incompatible LBAs is aggregated and written to a specific stripe group dedicated to storing out-of-order data. This effectively alleviates cache pressure, prevents front-end write requests from being blocked due to a full cache, and ensures continuous system availability under high load or sudden write scenarios. Simultaneously, physically isolating out-of-order data from normal continuous backup data avoids interference with the data lifecycle consistency in ordinary stripe groups, thus protecting ordinary stripe groups and maintaining a high probability of complete group release during backup deletion, reducing data migration overhead caused by mixed writes. Furthermore, releasing the corresponding data in the memory cache and SSD cache according to the order of writes to the cache ensures the orderly reclamation of flushed data, preventing data loss risks caused by premature cache space release, achieving efficient cyclical utilization of cache space, and further improving the overall system's write performance and stability.

[0044] In some embodiments, see Figure 3 , Figure 3 This is a flowchart illustrating a distributed block storage method with some exemplary embodiments. Checking whether the stripe group to which the stripe belongs contains valid data may include the following steps: Step S310: Obtain the bitmap information of the stripe group. The bitmap information is used to identify the valid state of each physical block in the stripe group.

[0045] In this step, the IO processing service maintains an independent bitmap data structure for each stripe group, where each bit in the bitmap corresponds to a physical block within the stripe group. When a stripe group is created or initialized, the IO processing service allocates a bitmap of appropriate length for that stripe group and initializes all bits to their valid or invalid default states. The bitmap information is stored in memory for fast access and can be periodically persisted to disk to prevent loss in the event of a system crash. When it is necessary to check whether a stripe group contains valid data, such as in response to a UNMAP command, the IO processing service first retrieves the corresponding bitmap pointer from the stripe group management structure based on the stripe group's identifier, thereby reading the entire bitmap information. A bit value of 1 in the bitmap indicates that the corresponding physical block currently stores valid data, meaning that the logical address mapping relationship of that physical block has not yet been released; a bit value of 0 indicates that the physical block is invalid or idle. By acquiring and parsing this bitmap information, the IO processing service can quickly determine whether any valid physical blocks exist within the stripe group, providing a basis for subsequent decisions regarding the release of the entire stripe group.

[0046] Step S320: Traverse the bitmap information. If the state of all physical blocks in the bitmap information is invalid, it is determined that the stripe group does not contain valid data, and the release process is triggered.

[0047] In this step, after obtaining the bitmap information of the stripe group, the IO processing service begins to traverse the bitmap information bit by bit. For each bit checked, if a bit value of 1 is encountered, it indicates that the corresponding physical block is valid, and the traversal immediately stops, determining that the stripe group contains valid data, and the release process is not triggered. If all bits of the entire bitmap have been traversed and the value of each bit is 0, meaning all physical blocks are in an invalid state, it is determined that the stripe group no longer contains any valid data. At this point, the IO processing service triggers the release process, marking all physical spaces associated with the stripe group as free, removing the stripe group from the allocatable area of ​​the stripe group list, and simultaneously releasing the memory space occupied by the bitmap.

[0048] This application embodiment maintains independent bitmap information for each stripe group and determines the existence of valid data within the stripe group by traversing the bitmap information. This allows for rapid and accurate determination of stripe group validity with minimal computational overhead and memory access cost, avoiding inefficient operations such as querying mapping tables or scanning physical disks one by one. Once the traversal finds that all physical blocks are invalid, the entire group release process can be triggered immediately, reclaiming the physical space associated with the stripe group as free resources, thereby improving the response speed of space reclamation and the overall system efficiency.

[0049] Please see Figure 4 , Figure 4 This is a structural schematic diagram of a distributed block storage device illustrated by some exemplary embodiments. The device includes: The first aggregation module 410 is configured to, when a data write stripe event is detected, first aggregate the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement, to obtain the current aggregated data. The first lookup module 420 is configured to find a target stripe group in which the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy the second proximity requirement. The write module 430 is configured to write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA and volume identifier of the current write. The first inspection module 440 is configured to check whether the stripe group containing the stripe contains valid data when a deletion instruction indicating the deletion of the correspondence between LBA and stripe is detected. If it does not contain valid data, the entire stripe group is released directly.

[0050] In some embodiments, the difference in LBA between any two adjacent logical blocks in the current aggregated data is less than a preset first threshold; or, The difference between the LBA of the last logical block and the LBA of the first logical block in the current aggregated data is less than the preset second threshold. In some embodiments, the second proximity requirement is met when the LBA of the last written data and the LBA of the current aggregated data are less than a preset third threshold. In some embodiments, detecting a data write stripe event includes: Upon receiving a write command, the data is added to the memory cache and then appended to the end of the SSD cache partition. Monitor the data already stored in the SSD cache partition. If the data already stored in the SSD cache partition is greater than the set data amount, then it is determined that a data write stripe event has been detected. In some embodiments, the write information further includes: the last write timestamp of the target stripe group; The device also includes: a second inspection module, configured to inspect the stripe group list, and if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clear the write information of that stripe group; In some embodiments, before finding a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data satisfy a second proximity requirement, the apparatus further includes: The determination module is configured to, if no target stripe group that meets the conditions is found, further determine whether there is a stripe group whose time difference between the last write timestamp and the current timestamp exceeds a preset duration; if so, write the current aggregated data to the stripe group and update the write information of the stripe group. In some embodiments, the apparatus further includes: The second aggregation module is configured to aggregate data in the cache that belong to different volumes or whose LBA does not meet the first proximity requirement when the SSD cache partition is insufficient. The second search module is configured to search for a specific stripe group for storing out-of-order data and write the aggregated out-of-order data into that specific stripe group. The release module is configured to release the corresponding cache and delete the corresponding data in the SSD cache partition in sequence when a cache release event is detected; In some embodiments, the inspection module checks whether the strip group to which the strip belongs contains valid data, including: Obtain the bitmap information of the stripe group. The bitmap information is used to identify the valid status of each physical block in the stripe group. Traverse the bitmap information. If the state of all physical blocks in the bitmap information is invalid, it is determined that the stripe group does not contain valid data, and the release process is triggered.

[0051] Based on the same application concept as the method described above, embodiments of this application also provide an electronic device, such as... Figure 5As shown, the electronic device includes: a processor and a machine-readable storage medium; the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example.

[0052] Based on the same concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of machine-executable instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0053] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

Claims

1. A method for storing distributed blocks, characterized in that, The method includes: When a data write stripe event is detected, the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement are first aggregated to obtain the current aggregated data; wherein, the difference between the LBAs of any two adjacent logical blocks in the current aggregated data is less than a preset first threshold; or, the difference between the LBA of the last logical block and the LBA of the first logical block in the current aggregated data is less than a preset second threshold. If a target stripe group is found where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement; the second proximity requirement means that the difference between the LBA of the last written data and the LBA of the current aggregated data is less than a preset third threshold. Write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA currently written, the volume identifier, and the last write timestamp of the target stripe group. If no target stripe group is found where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement, then it is further determined whether there is a stripe group where the time difference between the last write timestamp and the current timestamp exceeds a preset duration; if so, the current aggregated data is written to the stripe group, and the write information of the stripe group is updated. When a delete command indicating the deletion of the correspondence between LBA and stripe is detected, check whether the stripe group containing the stripe contains valid data. If not, release the entire stripe group directly.

2. The method according to claim 1, characterized in that, The monitored data write-to-strip events include: Upon receiving a write command, the data is added to the memory cache and then appended to the end of the SSD cache partition. Monitor the data already stored in the SSD cache partition. If the data already stored in the SSD cache partition is greater than a set data amount, then it is determined that the data write stripe event has been detected.

3. The method according to claim 1, characterized in that, The method further includes: checking the stripe group list; if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clearing the write information of that stripe group.

4. The method according to claim 1, characterized in that, The method further includes: When the SSD cache partition is insufficient, aggregate data belonging to different volumes or whose LBA does not meet the first proximity requirement in the cache; find a specific stripe group for storing out-of-order data, and write the aggregated out-of-order data into that specific stripe group; when a cache release event is detected, release the corresponding cache in sequence and delete the corresponding data in the SSD cache partition.

5. The method according to claim 1, characterized in that, The step of checking whether the band group to which the band belongs contains valid data includes: Obtain the bitmap information of the stripe group, which is used to identify the valid state of each physical block in the stripe group; traverse the bitmap information, and if the state of all physical blocks in the bitmap information is invalid, determine that the stripe group does not contain valid data and trigger the release process.

6. A distributed block storage device, characterized in that, The device includes: The first aggregation module is configured to, when a data write stripe event is detected, first aggregate the data in the cache that belong to the same volume and whose logical block addresses (LBAs) are consecutive or whose LBAs meet the first proximity requirement, to obtain the current aggregated data; wherein, the difference between the LBAs of any two adjacent logical blocks in the current aggregated data is less than a preset first threshold; or, the difference between the LBA of the last logical block and the LBA of the first logical block in the current aggregated data is less than a preset second threshold. The first search module is configured to find a target stripe group where the volume identifier of the last written data is the same as the volume identifier of the current aggregated data, and the LBA of the last written data and the LBA of the current aggregated data meet the second proximity requirement; the second proximity requirement means that the difference between the LBA of the last written data and the LBA of the current aggregated data is less than a preset third threshold. The write module is configured to write the current aggregated data to the target stripe group and update the write information of the target stripe group; the updated write information includes at least the LBA currently being written, the volume identifier, and the last write timestamp of the target stripe group. The determination module is configured to, if no target stripe group that meets the conditions is found, further determine whether there is a stripe group whose time difference between the last write timestamp and the current timestamp exceeds a preset time; if so, write the current aggregated data to the stripe group and update the write information of the stripe group. The first inspection module is configured to check whether the stripe group containing the stripe contains valid data when it detects a deletion instruction that indicates the deletion of the correspondence between LBA and stripe. If it does not contain valid data, the entire stripe group is released directly.

7. The apparatus according to claim 6, characterized in that, The monitored data write-to-strip events include: Upon receiving a write command, the data is added to the memory cache and then appended to the end of the SSD cache partition. Monitor the data already stored in the SSD cache partition. If the data already stored in the SSD cache partition is greater than a set data amount, then it is determined that the data write stripe event has been detected. And / or, The device further includes: a second inspection module, configured to inspect the stripe group list, and if the interval between the last write timestamp and the current timestamp of a certain stripe group exceeds a preset time, then clear the write information of that stripe group; And / or, The device further includes: The second aggregation module is configured to aggregate data in the cache that belong to different volumes or whose LBA does not meet the first proximity requirement when the SSD cache partition is insufficient. The second search module is configured to search for a specific stripe group for storing out-of-order data and write the aggregated out-of-order data into that specific stripe group. The release module is configured to release the corresponding cache and delete the corresponding data in the SSD cache partition in sequence when a cache release event is detected; And / or, The inspection module checks whether the strip group to which the strip belongs contains valid data, including: Obtain the bitmap information of the stripe group, the bitmap information being used to identify the valid state of each physical block within the stripe group; Traverse the bitmap information. If the state of all physical blocks in the bitmap information is invalid, then determine that the stripe group does not contain valid data and trigger the release process.

8. An electronic device, characterized in that, The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method steps of any one of claims 1-5.

Images