I / O hang detection method, storage engine, device, program product, and storage medium

Abstract

Provided in the present disclosure are an I / O hang detection method, a storage engine, a device, a program product, and a storage medium. The method is applied to a main thread of a storage engine, wherein the storage engine further comprises an auxiliary thread, the storage engine is configured to access a zoned-namespace-based memory, the memory comprises at least one zone, and the auxiliary thread is used for executing a zone reclamation task related to a garbage collection zone and submitted by the main thread and an I / O hang detection task for the garbage collection zone. The method comprises: if it is determined that I / O hang detection needs to be performed currently, accessing an I / O request queue of a common zone in a memory, wherein the common zone is another zone in the memory that is different from a garbage collection zone; detecting whether a hung I / O request is present in the I / O request queue of the common zone, so as to obtain an I / O hang detection result of the common zone; and acquiring from an auxiliary thread an I / O hang detection result for the garbage collection zone.

Description

I / O hang detection methods, storage engines, devices, software products, and storage media

[0001] This disclosure claims priority to Chinese Patent Application No. 202510121582.7, filed with the China National Intellectual Property Administration on January 24, 2025, entitled “I / O Suspension Detection Method, Storage Engine, Device, Program Product and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of storage technology, and in particular to I / O suspend detection methods, storage engines, devices, software products, and storage media. Background Technology

[0003] SSD (Solid State Disk) is a type of computer storage that primarily uses NAND flash memory as its permanent storage. Currently, some SSDs employ an interface technology specification based on ZNS (Zoned Namespace). In ZNS technology, the memory's address space is abstracted into contiguous logical partitions (Zones), and this partition information is exposed to upper-layer applications, allowing applications to partition storage according to their specific projects.

[0004] The user-space storage engine built on ZNS SSDs provides foreground read / write functionality for user data. Data is written to storage partitions, and a partition reclamation process is needed to promptly release these partitions for user use. Furthermore, the storage engine also needs to implement I / O (Input / Output) suspension detection. I / O suspension refers to the phenomenon where I / O requests do not complete as expected. Therefore, reducing the overhead of I / O suspension detection is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides an I / O suspend detection method, a storage engine, a device, a program product, and a storage medium.

[0006] According to a first aspect of the present disclosure, an I / O suspension detection method is provided. The method is applied to the main thread of a storage engine, the storage engine further comprising an auxiliary thread. The storage engine is used to access memory based on a partition namespace, the memory including at least one partition. The auxiliary thread is used to execute partition reclamation tasks related to a garbage collection partition submitted by the main thread, and I / O suspension detection tasks for the garbage collection partition. The method includes:

[0007] If it is determined that I / O suspend detection is required, access the I / O request queue of the normal partition in the memory; the normal partition is another partition in the memory that is different from the garbage collection partition.

[0008] Detect whether there are any pending I / O requests in the I / O request queue of the normal partition, and obtain the I / O pending detection result of the normal partition;

[0009] Obtain the I / O suspension detection result of the auxiliary thread on the garbage collection partition.

[0010] According to a second aspect of the present disclosure, an I / O suspend detection method is provided, the method being applied to an auxiliary thread of a storage engine, the storage engine further comprising a main thread, the storage engine being used to access memory based on a partition namespace, the memory comprising at least one partition; the auxiliary thread being used to execute partition reclamation tasks related to garbage collection partitions submitted by the main thread; the method comprising:

[0011] If it is determined that I / O suspend detection is required, access the I / O request queue corresponding to the garbage collection partition;

[0012] Detect whether there are any pending I / O requests in the I / O request queue to obtain the I / O pending detection result of the garbage collection partition;

[0013] The I / O suspension detection results of the garbage collection partition are provided to the main thread.

[0014] According to a third aspect of the present disclosure, a storage engine is provided, the storage engine including a main thread and an auxiliary thread, the storage engine being used to access a partition-based namespace memory, the memory including at least one partition; the main thread being used to perform the steps of the method described in the first aspect, and the auxiliary thread being used to perform the steps of the method described in the second aspect.

[0015] According to a fourth aspect of the present disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first or second aspect above.

[0016] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0017] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.

[0018] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0019] In this embodiment of the disclosure, when the storage engine includes a main thread and an auxiliary thread, and the auxiliary thread executes the partition reclamation tasks related to the garbage collection partition submitted by the main thread, for ordinary zones that do not participate in Recycle, the main thread performs I / O suspension detection; for garbage collection zones that participate in Recycle, the I / O suspension detection is removed from the main thread's I / O suspension detection and performed on the auxiliary thread. In this way, the I / O request queue corresponding to the garbage collection zone will not be accessed by the main thread and the auxiliary thread at the same time, but only by the auxiliary thread. Therefore, there is no need to set an additional thread concurrency synchronization strategy, which can reduce synchronization overhead and thus reduce the overhead of I / O suspension detection.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] Figure 1A is a schematic diagram of an in-memory partition according to an exemplary embodiment of the present disclosure.

[0022] Figure 1B is a state transition flowchart of a Zone according to an exemplary embodiment of this disclosure.

[0023] Figure 2A is a flowchart illustrating an I / O hang-up detection method according to an exemplary embodiment of this disclosure.

[0024] Figure 2B is a schematic diagram illustrating the processing of a waste recycling task according to an exemplary embodiment of this disclosure.

[0025] Figure 3A is a flowchart illustrating another I / O hang-up detection method according to an exemplary embodiment of this disclosure.

[0026] Figure 3B is a flowchart illustrating another I / O hang-up detection method according to an exemplary embodiment of this disclosure.

[0027] Figure 4 is a hardware structure diagram of a computer device containing an I / O suspend detection device according to an exemplary embodiment of the present disclosure.

[0028] Figure 5 is a block diagram of an I / O hang-up detection device according to an exemplary embodiment of the present disclosure.

[0029] Figure 6 is a block diagram of an I / O hang-up detection device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0034] Solid-state drives (SSDs) consist of a control unit and storage units. The storage units contain flash memory chips or DRAM chips (Dynamic Random Access Memory), and they are widely used in many fields. Compared to hard disk drives (HDDs), SSDs offer higher read and write speeds, but are also more expensive. Currently, in some scenarios, service providers also use user-space storage engines based on ZNS SSDs to provide storage services to users. A user-space storage engine is a storage system that runs in user space rather than kernel space. It can directly access hardware resources without the intervention of the traditional operating system kernel layer.

[0035] The ZNS SSD-based storage engine is suitable for applications requiring a large number of sequential writes and a small number of random writes, enabling high-speed data access. In providing storage services to users, the ZNS SSD-based storage engine needs to provide data read and write functionality to the user in the foreground.

[0036] The storage engine allows external users to read and write data. The granularity of data read and write operations that the storage engine passes to users can be called a data block (Chunk), while the granularity of the storage engine managing ZNS SSDs is a partition (Zone). The size of a Chunk and the size of a Zone can be different. The foreground can write to the storage engine and can also read the contents of a written Chunk. In addition to providing Chunk read and write functionality, the storage engine also needs to support Chunk deletion functionality.

[0037] Because ZNS SSDs only support append writes, the storage engine needs to introduce a Recycle function to support functions such as releasing space after data deletion and Zone space resource management while providing foreground read and write capabilities. By properly managing the storage Zones, Zones can be released in a timely manner for users to use.

[0038] The ZNS SSD interface to the storage engine abstracts the storage device address space into a series of contiguous logical partitions (Zones). Its operational characteristics are as follows: a partition needs to be opened before it can be written to, becoming an open / active partition; logical pages within an open partition can only be written sequentially and cannot be updated in place; if data needs to be updated, the entire partition must be reset before writing can begin again from the beginning.

[0039] For example, as shown in Figure 1A, in ZNS SSDs, the Logical Block Addressing (LBA) is divided into multiple zone granularities according to the configuration. Figure 1A shows X partitions, from Zone 1 to Zone X. In ZNS SSDs, the state of a zone can include various types:

[0040] This indicates the free state of a Zone where no data has been written.

[0041] This indicates that the Zone is in an open state, meaning it can be written to.

[0042] This indicates that the Zone is full and can only support a read-only Closed state.

[0043] This indicates that the data in the Zone is in a garbage state, meaning it contains invalid data.

[0044] As shown in Figure 1B, this disclosure illustrates a Zone state transition process according to an exemplary embodiment, the process of which can be:

[0045] A Zone in the Free state can be transformed into the Open state through the OpenZone operation;

[0046] Within an Open Zone, data can only be appended to and written, with random reads supported; that is, data can only be written sequentially to the Zone and cannot be updated in-place. Within each Zone, data writing must follow a sequential appending method; existing data cannot be randomly overwritten. After each sequential write operation, a "Write Pointer" is set to record the LBA address of the last written data within the Zone.

[0047] Once the write operation is complete in the Open Zone, the Zone is converted to the Closed Zone state by closing the Closed Zone operation. The Closed Zone is read-only.

[0048] After the Recycle operation, a Zone in the Closed state is transformed into a Garbage state after the valid data within the Zone is migrated out.

[0049] A Zone in Garbage state can be converted to Free state by resetting the ResetWritePtr operation.

[0050] The storage engine needs to support data reading and writing by external users. Foreground write requests generate data chunks, while foreground read requests read the contents of the written data chunks.

[0051] A ZNS-based storage engine, in addition to providing read and write access to data blocks (Chunks), must also support Chunk deletion. Due to the hardware constraint that data within a Zone cannot be deleted / modified in situ, the space occupied by deleted stale data needs to be released through garbage collection (GC). Specifically, this can be achieved through:

[0052] Read the remaining valid data on the source partition (Source Zone) and write it to the destination partition (Dest Zone). The source partition no longer contains valid user data and is converted to a garbage state.

[0053] After the Garbage Zone is erased, it can continue to accept write requests (starting from the initial LBA of the Zone).

[0054] Storage engines built on high-speed ZNS SSD media typically employ a RunToCompletion (RTC) thread model to provide high-speed data read and write capabilities, and because ZNS SSDs require data to be written sequentially within each zone, they generally use this model. The core idea is that each thread is responsible for one I / O (Input / Output) task, and this thread executes the task continuously until it is completed—that is, "run to completion"—without being interrupted by other tasks. This means that even if other tasks are waiting in the queue, the current task will continue to occupy CPU resources until it completes. Only after the current task is completed will the next task be executed. This model avoids the context switching overhead during task execution because all operations of the same task are executed sequentially by the same thread, thus reducing lock contention and context switching problems common in multithreading.

[0055] In some scenarios, user-space storage engines using the RTC thread model handle both foreground I / O tasks and background Recycle tasks within a single I / O main thread. Since switching between foreground and background tasks within a single I / O thread incurs some time overhead, this process is problematic.

[0056] In other scenarios, the storage engine can introduce auxiliary threads on top of the main thread to split some subtasks from the partition reclamation task process and offload these subtasks from the main thread to the auxiliary threads. This can significantly reduce the performance impact of the partition reclamation task on data read and write tasks and improve the storage engine's processing efficiency for data read and write tasks.

[0057] In addition, the storage engine needs to handle the following situations during the SSD read / write process:

[0058] (1) Error message returned by the SSD due to hardware failure;

[0059] In this situation, the storage engine can receive a fault message from the SSD for a certain zone, and therefore can take appropriate action based on the error message. For example, it can stop reading and writing to the faulty zone, or in a distributed file system scenario, the storage engine can notify the upper management module of the message to process read and write requests to the faulty zone based on the replica of the faulty zone, or trigger the upper management module to handle the faulty hardware, etc.

[0060] (2) Hardware failure caused I / O requests to take longer than expected, resulting in I / O request hang.

[0061] In practical applications, there are also situations where hardware failures occur but the SSD does not actively return error messages. Therefore, additional logic is needed in the storage engine to detect I / O request hangups.

[0062] In the aforementioned storage engine that uses a single I / O main thread to handle foreground I / O tasks and background Recycle tasks, if I / O request suspension detection is to be implemented within the storage engine, the newly added I / O request suspension detection task would, predictably, also be handled by the main thread. For example, the main thread maintains the storage's metadata, which can be refined to the metadata of each Zone within the storage. The main thread can iterate through the metadata of each Zone, access the Zone's read / write request queue based on the Zone's metadata, and determine whether any I / O request suspensions exist within the Zone's read / write request queue.

[0063] For storage engines that offload some tasks from the main thread's partition reclamation task to auxiliary threads, implementing I / O request suspension detection presents a significant challenge. For example, if the traditional approach of having the main thread handle the I / O request suspension tasks for each partition of the memory is followed, it would introduce additional inter-thread synchronization overhead.

[0064] For example, the read / write request queues of Zones participating in the Recycle process can be accessed simultaneously by both the main thread and auxiliary threads. This is because, with the introduction of auxiliary threads, the main thread can construct Recycle tasks and submit them to the auxiliary threads. When executing these tasks, the auxiliary threads perform the actual Recycle I / O, such as reading data from the source Zone and writing the read data to the destination Zone and the source Zone. It can be seen that the auxiliary threads have read and write permissions to the I / O request queues of both the source and destination Zones. However, if an I / O request suspension detection task is added to the main thread, the main thread will also access the I / O request queues of both the source and destination Zones. Therefore, in this case, a thread concurrency synchronization strategy for the main thread and auxiliary threads needs to be implemented, increasing the overhead of thread synchronization. For high-frequency data structures like read / write request queues in the I / O process, thread synchronization is unacceptable in high-performance scenarios under the RTC thread model.

[0065] Based on this, embodiments of this disclosure provide an I / O suspension detection scheme. This scheme eliminates the need for inter-thread concurrency synchronization strategies during I / O suspension detection, thus avoiding additional synchronization overhead and reducing the overhead of I / O suspension detection. The embodiments of this disclosure will now be described in detail.

[0066] As shown in Figure 2A, which is a flowchart illustrating an I / O suspension detection method according to an exemplary embodiment of this disclosure, the method can be applied to the main thread of a storage engine. The storage engine further includes an auxiliary thread. The storage engine is used to access memory based on a partition namespace, and the memory includes at least one partition. The auxiliary thread is used to execute garbage collection tasks related to the garbage collection partition submitted by the main thread, as well as I / O suspension detection tasks for the garbage collection partition. The method may include the following steps:

[0067] In step 202, if it is determined that I / O suspend detection is required, the I / O request queue of the ordinary partition in the memory is accessed; the ordinary partition is another partition in the memory that is different from the garbage collection partition.

[0068] In step 204, the system detects whether there are any pending I / O requests in the I / O request queue of the normal partition, and obtains the I / O pending detection result of the normal partition.

[0069] In step 206, obtain the I / O suspension detection result of the auxiliary thread on the garbage collection partition.

[0070] The method in this embodiment can be applied to the main thread of a storage engine based on ZNS SSD, which can be used to access memory based on partition namespaces.

[0071] As an example, the storage engine in this embodiment can be a user-space storage engine, such as a user-space standalone storage engine. A standalone storage engine refers to a functional node in a distributed storage system responsible for managing the storage of user data on the storage medium. Generally, a single computer device (such as a storage server) can run one standalone storage engine process, responsible for managing one or more storage devices on the computer device.

[0072] In some examples, a storage engine process may run in a computer device, and the storage engine process may include one or more threads; for example, a computer device may include multiple memories, and each memory may correspond to a main thread, with each main thread responsible for handling the data read and write tasks of the corresponding memory.

[0073] In some examples, the storage can be one or more ZNS-based solid-state drives (SSDs), each divided into multiple partitions (Zones). As an example, partitions can support append writes, random reads, and other operations, but data within a partition must be completely erased; partial erasure and overwrite are not allowed.

[0074] In some examples, the storage engine in this embodiment includes a main thread and auxiliary threads. The main thread, in response to a received data read / write request, executes a data read / write task based on the metadata related to the partition in the memory. During the execution of the data read / write task, if a partition reclamation task is triggered, the main thread processes the first type of subtasks related to metadata within the partition reclamation task and submits the second type of subtasks unrelated to metadata to the auxiliary thread for processing. That is, the aforementioned "auxiliary thread for executing reclamation tasks related to garbage collection partitions submitted by the main thread" can refer to the second type of subtasks unrelated to metadata within the partition reclamation task submitted by the main thread.

[0075] For example, when the storage engine receives a data read / write request for a partition in the memory, the request is first processed by the main thread. The storage engine can manage individual zones through the metadata of the zones in the memory. The metadata associated with a zone can include the zone's own metadata, including but not limited to a combination of one or more of the following: zone status information, zone address, write pointer, used space or erase count, etc.

[0076] Metadata related to a Zone can also include other types of metadata, such as metadata related to the data stored in the Zone. For example, when a storage engine provides data storage services to a caller, it can provide services at a different management granularity than a Zone, referred to in this embodiment as a Chunk. The size of a Chunk can be larger than the size of a Zone. Therefore, data read / write requests can be requests for one or more Chunks; the storage engine can also combine the metadata of a Chunk to perform data read / write tasks. The metadata of a Chunk can include, but is not limited to, a combination of one or more of the following information: the Chunk's address, timestamp, owner or permissions, etc.

[0077] The main thread needs to rely on metadata to perform data read and write tasks. For example, when processing data read and write requests, the main thread needs to query relevant metadata and then perform specific read and write operations based on this information. These operations may include partition location, write pointer management, partition status tracking, data integrity checks, or chunk permission checks. After completing the read and write operations, the metadata may also need to be updated.

[0078] Since the main thread needs to rely on metadata to perform data read and write tasks, it needs to manage the metadata. This can be achieved through locking mechanisms or other means to ensure the consistency of the metadata and prevent concurrent operations.

[0079] In some examples, the main thread can determine the SourceZone, the partition containing valid data to be reclaimed, and a free partition, DstZone. The main thread can construct a second type of subtask containing the identifiers of the partition to be reclaimed and the free partition, and submit it to the auxiliary thread so that the auxiliary thread can receive and process the second type of subtask. Based on this, the garbage collection partitions in this embodiment can be the partition to be reclaimed and the free partition required in the partition reclamation task. The partition to be reclaimed can also be called the source partition SourceZone, and the free partition can be called the destination partition DstZone, because the operation of the partition reclamation task is performed by the auxiliary thread to perform the actual Recycle I / O, read data from SourceZone, write data to DstZone, and erase SourceZone.

[0080] For example, Figure 2B is a schematic diagram illustrating how the main thread and auxiliary thread jointly complete a partition reclamation task according to an exemplary embodiment of this disclosure, wherein:

[0081] 1. The main thread can query the partitions that need garbage collection and construct a Recycle task to submit to the auxiliary thread;

[0082] 2. The auxiliary thread, based on the Recycle task, performs either a Close Zone operation on the source partition or an Open Zone operation on the destination partition as needed. That is, it updates the SourceZone to the closed state and updates the DstZone (which was in the Free state) to the Open state. Alternatively, it closes the Old DstZone (used for writing data in the previous Recycle task) and opens the New DstZone (a free partition needed for writing data in the current Recycle task).

[0083] 3. The auxiliary thread prepares to recycle I / O tasks according to the configured I / OSize;

[0084] 4. The auxiliary thread performs the actual recycling task, Recycle I / O, which involves reading data from SourceZone and writing it to DstZone; erasing SourceZone.

[0085] 5. When a ChunkSegment Recycle in SourceZone ends, the auxiliary thread updates its metadata;

[0086] 6. Once the recycling of valid data in SourceZone is complete, the auxiliary thread notifies the main thread that this round of recycling has ended;

[0087] 7. The main thread updates Zone / Chunk metadata based on the results of the Recycle task;

[0088] Of the steps above, only steps 1 and 7 are executed in the main I / O thread. While the auxiliary thread is processing the Recycle task, the main thread can handle the foreground I / O process normally.

[0089] In this embodiment, the subtasks offloaded to the auxiliary thread may include: logic for interacting with the hardware, such as closing / opening DstZone and actual I / O; CPU-intensive tasks such as preparing Recycle I / O Tasks;

[0090] Before the Recycle task ends, the ChunkSegment on DstZone can be exclusively accessed by the Recycle thread, and its corresponding metadata updates can be offloaded to auxiliary threads. The subtasks that remain on the main thread are mainly the subprocesses of selecting SourceZone and updating data block metadata ChunkMeta / partition metadata ZoneMeta after the Recycle task ends. This is done mainly to avoid thread safety issues introduced by multi-threaded access.

[0091] In some examples, the storage engine runs on a computer device, which may include a multi-core processor; the main thread and the auxiliary thread run on different processor cores.

[0092] In some examples, a computer device may have multiple memories, and the storage engine may include a main thread corresponding to each memory. The number of processor cores in the computer device is greater than the number of main threads. The main thread corresponding to each memory runs on a different processor core. The main thread is associated with one or more auxiliary threads. The main thread may submit second-type subtasks in the partition reclamation task that are not related to the metadata to the associated auxiliary threads for processing.

[0093] In some cases, the auxiliary thread can have a task queue, and the main thread can submit the second type of subtask to the task queue corresponding to the associated auxiliary thread.

[0094] In this embodiment, the main thread also performs I / O suspension detection tasks, but these tasks target ordinary partitions in memory that are different from garbage-collected partitions. In other words, the I / O suspension detection of garbage-collected partitions in memory is separated from the main thread and performed by an auxiliary thread.

[0095] In step 202, there are several ways for the main thread to determine whether I / O suspension detection is needed. For example, it could be triggered by receiving an instruction to perform I / O suspension detection. This instruction could be initiated by other functional modules within the storage engine or received by the storage engine from external sources, such as user instructions. Alternatively, it could be triggered based on other preset conditions, such as if the number of I / O requests received within a preset time period is greater than or equal to a preset threshold. In practical applications, these conditions can be set as needed. For instance, a timed detection rule can be configured in the main thread. This rule represents the time rule for triggering I / O suspension detection, allowing the main thread to determine whether I / O suspension detection is needed at any given moment. If so, the main thread can begin performing I / O suspension detection on the I / O request queues of each ordinary partition.

[0096] As an example, the storage engine maintains metadata for each Zone in the storage. This metadata includes the corresponding I / O request queue, which stores the I / O requests for that Zone. The main thread can distinguish which Zones are currently normal partitions and which are garbage collection partitions. For instance, in the storage engine, triggering a partition reclamation task can be determined by the storage engine through a set reclamation strategy. For example, it could be based on the proportion of used partitions to the total number of partitions in the storage to determine whether to trigger the reclamation of certain garbage partitions. In some examples, the process of determining whether to trigger a partition reclamation task can be executed by certain threads within the storage engine. For example, one thread could determine whether to trigger a partition reclamation task for each storage device, and then notify the main thread for that storage device after determining that a partition reclamation task needs to be triggered. In other examples, it can be executed by the main thread itself, with each main thread independently determining whether a corresponding storage device needs to trigger a partition reclamation task. Therefore, the main thread can know which garbage collection Zones are involved in the partition reclamation task.

[0097] Based on this, as an example, the main thread can traverse the I / O request queue of each normal partition to obtain the I / O suspension detection result of the normal partition by checking whether there are any suspended I / O requests in the I / O request queue.

[0098] There are several ways to detect whether there are pending I / O requests in the I / O request queue. For example, a preset threshold representing a pending request can be set, or the queuing time of each I / O request in the queue can be checked. The presence of I / O pending requests is determined based on the queuing time and the preset threshold. For instance, if there is one pending I / O request in the queue, it is considered that there is an I / O pending issue for that ordinary partition. Alternatively, if there are multiple (the number can be set as needed) pending I / O requests in the queue, it is considered that there is an I / O pending issue for that ordinary partition. Or, if the proportion of pending I / O requests in the queue is greater than or equal to a preset proportion threshold, then the ordinary partition is considered to have an I / O pending issue. In practical applications, these settings can be configured as needed; this embodiment does not impose any limitations on them.

[0099] Whether an I / O request in the queue is suspended can be determined not only based on the queuing time and a preset threshold, but also through other implementation methods. For example, the I / O request carries an enqueue time indicating that the I / O request was enqueued. This enqueue time can be added to a preset time threshold to obtain the expected latest completion time. This expected latest completion time is then compared with the current time. If the expected latest completion time is equal to or before the current time, the I / O request is suspended; if the latest completion time is after the current time, the I / O request is not suspended.

[0100] Thus, the main thread can obtain the I / O suspension detection results for each ordinary partition. The partition I / O suspension detection result indicates whether I / O suspension exists in the partition, and the existence of I / O suspension can be determined through the various implementations described above. The I / O suspension detection for the garbage collection partition is performed by an auxiliary thread. The process of the auxiliary thread performing I / O suspension detection for the garbage collection partition is essentially the same as the process of the main thread performing I / O suspension detection for ordinary partitions. Therefore, the auxiliary thread can obtain the I / O suspension detection result for the garbage collection partition. Thus, the main thread can obtain the I / O suspension detection result for the garbage collection partition through the auxiliary thread.

[0101] There are several ways for the main thread to obtain the I / O suspension detection results of the garbage collection partition. For example, the main thread and the auxiliary thread can agree on a shared storage location, with the auxiliary thread writing the I / O suspension detection results of the garbage collection partition to this shared storage location, and the main thread accessing this storage location to read them. Alternatively, in practical applications, the auxiliary thread can call a pre-registered callback function after completing the detection to pass the detection results to the main thread, or other methods.

[0102] As can be seen from the above embodiments, in this embodiment, for ordinary Zones that do not participate in Recycle, the main thread performs I / O suspension detection; for garbage collection Zones that participate in Recycle, the I / O suspension detection is removed from the main thread and performed on an auxiliary thread. In this way, the I / O request queue corresponding to the garbage collection Zone will not be accessed by the main thread and the auxiliary thread at the same time, but only by the auxiliary thread. Therefore, there is no need to set an additional thread concurrency synchronization strategy, which can reduce synchronization overhead.

[0103] Figure 3A is a flowchart illustrating another I / O hang detection method according to an exemplary embodiment of this disclosure. The method is applied to an auxiliary thread of a storage engine, which also includes a main thread. The storage engine is used to access memory based on a partition namespace, and the memory includes at least one partition. The auxiliary thread is used to execute partition reclamation tasks related to garbage collection partitions submitted by the main thread. The method may include the following steps:

[0104] In step 302, if it is determined that I / O suspend detection is required, the I / O request queue corresponding to the garbage collection partition is accessed;

[0105] In step 304, it is detected whether there are any pending I / O requests in the I / O request queue, and the I / O pending detection result of the garbage collection partition is obtained;

[0106] In step 306, the I / O suspension detection result of the garbage collection partition is provided to the main thread.

[0107] Referring to Figure 3B, in this embodiment, the auxiliary thread performs I / O suspension detection on the garbage collection partitions. In practical applications, the auxiliary thread can determine which partitions are garbage collection partitions; for example, with the introduction of an auxiliary thread, the main thread can construct a Recycle task and submit it to the auxiliary thread. This Recycle task contains the identifiers of the source zone and the destination zone. Therefore, the auxiliary thread can determine from the received Recycle task that the source zone and the destination zone are garbage collection partitions, and can access their I / O request queues based on the identifiers of the source zone and the destination zone.

[0108] In some examples, for a secondary thread, providing the I / O suspension detection result of the garbage collection partition to the main thread may include:

[0109] Access a preset memory region shared with the main thread, and write the I / O suspension detection result into the preset memory region so that the main thread can obtain the I / O suspension detection result of the garbage collection partition from the preset memory region.

[0110] In some examples, for the main thread, obtaining the I / O suspension detection result of the auxiliary thread on the garbage collection partition may include:

[0111] Access a preset memory region shared with the auxiliary thread, and read the I / O suspension detection result of the garbage collection partition from the preset memory region; the I / O suspension detection result of the garbage collection partition is written by the auxiliary thread after performing I / O suspension detection on the I / O request queue corresponding to the garbage collection partition.

[0112] In this embodiment, a region for storing the detection results of the garbage collection partition can be created in memory. The creation of this preset memory region can be done by the main thread, and the address of the preset memory region can be passed to the auxiliary thread by the main thread.

[0113] As an example, the auxiliary thread obtains the identifier of each garbage collection zone it detects. It can configure a corresponding identifier for each garbage collection zone to indicate whether there is an I / O hang. The detection results of each garbage collection zone can be written to the shared preset memory area in a format such as key-value.

[0114] Optionally, each garbage collection zone can have its own atomic variable. This atomic variable can store information representing the I / O suspension detection result of that garbage collection zone, such as whether I / O suspension exists in that zone. Optionally, this atomic variable can be a Boolean-valued atomic variable. An atomic operation is an indivisible operation, meaning that its execution cannot be interrupted or interfered with by other operations. In other words, in a multi-threaded scenario, when one thread executes an atomic operation, other threads cannot insert their own operations in the middle of this operation. An atomic operation either executes completely or not at all. Therefore, atomic variables can be safely updated and read in a multi-threaded environment without the need for additional locking mechanisms. Of course, using other types of information is also optional in practical applications, and this embodiment does not limit this.

[0115] As an example, we design ZoneWriter as the object responsible for updating (writing) the detection results in a shared preset memory region, and ZoneReader as the object for reading the detection results from the shared preset memory region; these two can be different instances or different methods of the same class; each garbage-collected partition has a unique identifier (such as Zone ID) and a corresponding std::atomic <bool>Properties. ZoneWriter and ZoneReader can be designed to contain a mapping table (such as a hash table or array). The main thread can create ZoneWriter objects and initialize their internal mapping table, allocating a std::atomic pointer to each garbage-collected partition. <bool>The `ZoneWriter` property can be initialized to `false`. The main thread can start an auxiliary thread and pass the address of the `ZoneWriter` / `ZoneReader` object to the auxiliary thread. The auxiliary thread updates the detection results through the `ZoneWriter` object. For example, if an I / O hang is found in a garbage collection partition, the auxiliary thread uses the `ZoneWriter` object to update the corresponding atomic boolean property to `true`. The main thread can periodically or as needed read the I / O hang status of each garbage collection partition through the `ZoneReader` object, for example, reading the `Zone ID` whose property is `true`.

[0116] Based on this, in this embodiment, the auxiliary thread writes the I / O suspension detection result to a preset memory area shared with the main thread. This allows for the transfer of the I / O suspension detection result of the garbage collection partition between threads with reduced overhead. The main thread can also quickly obtain the I / O suspension detection result of the garbage collection partition by accessing this preset memory area.

[0117] In some examples, the method may also include:

[0118] After reading the I / O suspend detection result of the garbage collection partition from the preset memory area, the I / O suspend detection result of the garbage collection partition is cleared from the preset memory area.

[0119] In this embodiment, after the main thread reads the I / O suspension detection result of the garbage collection partition from the preset memory area, it can promptly clear the I / O suspension detection result of the garbage collection partition stored in the preset memory area. This avoids occupying memory space and ensures that when the I / O suspension detection is performed again in the next round, the preset memory area has enough space for the auxiliary thread to write the I / O suspension detection result of the garbage collection partition.

[0120] In some examples, detecting whether there are pending I / O requests in the I / O request queue of the normal partition may include:

[0121] Read the first I / O request in the I / O request queue of the normal partition and obtain the queuing time of the first I / O request;

[0122] The queuing time is compared with a preset suspension threshold to determine whether there are any suspended I / O requests in the I / O request queue of the normal partition.

[0123] In this embodiment, the I / O request queue of the partition can be a first-in, first-out (FIFO) queue. The first I / O request in the queue, i.e., the oldest active request, is the earliest submitted but not yet completed request in the queue. Each request in the queue can have a start timestamp, which records the time when the request was submitted. The queuing time for the first I / O request can be determined based on the current time. If the queuing time is greater than a preset suspension threshold, the first I / O request can be considered to have been suspended. The preset suspension threshold can be configured with a specific value according to actual needs, and this embodiment does not limit it.

[0124] In this embodiment, only the oldest I / O request is examined. Whether this oldest I / O request is suspended is used to determine if there is an I / O suspension issue in the partition. Therefore, this embodiment can quickly and accurately determine if there is an I / O suspension issue in a partition.

[0125] Similarly, for the auxiliary thread, detecting whether there are any pending I / O requests in the I / O request queue can include:

[0126] Read the first I / O request in the I / O request queue and obtain the queuing time of the first I / O request;

[0127] By comparing the queuing time with a preset suspension threshold, it is determined whether there are any suspended I / O requests in the I / O request queue of the garbage collection partition.

[0128] Similar to the process described above where the main thread checks the I / O request queue of a normal partition for pending I / O requests, the auxiliary thread can simply check the oldest I / O request. Based on whether this oldest I / O request is pending, it can determine whether there is I / O pending in the garbage collection partition. Therefore, this embodiment can quickly and accurately determine whether there is I / O pending in a garbage collection partition.

[0129] In some examples, the auxiliary thread may include a task scheduler, a task executor, and a task queue, and the method may further include:

[0130] The timed detection rules are registered to the task scheduler of the auxiliary thread, so that the task scheduler adds the I / O suspension detection task to the task queue based on the timed detection rules. After the task executor retrieves the I / O suspension detection task from the task queue, it performs I / O suspension detection on the I / O request queue of the garbage collection partition.

[0131] In this embodiment, when the main thread starts, the timed detection rule can be registered with the auxiliary thread. For example, the registration can occur after the auxiliary thread is started. For instance, the auxiliary thread can be started by the main thread, and the timed detection rule can be registered with the auxiliary thread's task scheduler after the main thread starts the auxiliary thread. In this way, the auxiliary thread can periodically drive the execution of the I / O suspension detection task based on this rule. Optionally, the timed detection rule can be a rule that characterizes when to start I / O suspension detection. Specifically, it can be a rule describing the time to start I / O suspension detection, such as a rule describing that I / O suspension detection is performed once every fixed period (e.g., every 10 seconds or every minute).

[0132] As an example, the auxiliary thread has a corresponding task queue that stores the tasks that the auxiliary thread's task executor needs to execute. I / O suspension detection tasks can be placed in this task queue, allowing the task executor to retrieve the I / O suspension detection task from the queue and then perform I / O suspension detection on the I / O request queue of the garbage collection partition. Optionally, this task queue is the aforementioned task queue used to prevent garbage collection tasks from the auxiliary thread, i.e., the task queue where the main thread can place the second type of sub-tasks of the auxiliary thread.

[0133] Based on this, in this embodiment, considering that the main thread needs to perform I / O suspension detection for the normal partition and the auxiliary thread also needs the I / O suspension detection result for the garbage collection partition, the main thread is configured to set the timed detection rules for the auxiliary thread to perform I / O suspension detection, so that the I / O suspension detection task of the auxiliary thread can cooperate with the main thread to complete.

[0134] In some examples, considering that each time the main thread performs I / O suspension detection, it needs to obtain the I / O suspension detection result for the ordinary partition and also needs to obtain the I / O suspension detection result for the garbage collection partition from the auxiliary thread, this embodiment designs two threads to use the same timed detection rule. As an example, if it is determined that I / O suspension detection is needed, accessing the I / O request queue of the ordinary partition in the memory includes:

[0135] If it is determined based on the timing detection rules that I / O suspend detection is required, access the I / O request queue of the normal partition in the memory;

[0136] The access to the preset memory region shared with the auxiliary thread includes:

[0137] Compare the time difference between the current time and the start time of I / O suspend detection to see if it is greater than or equal to a preset duration threshold.

[0138] If yes, then access the preset memory area shared with the auxiliary thread; if no, wait until the time difference between the current time and the start time is greater than or equal to a preset duration threshold before accessing the preset memory area shared with the auxiliary thread.

[0139] As an example, in the previous example, the timing detection rules of the auxiliary thread were configured by the main thread; while in this embodiment, the main thread determines whether I / O suspension detection is needed based on the timing detection rules. Therefore, it can be seen that in this embodiment, by using the same timing detection rules for both threads, the main thread and the auxiliary thread can simultaneously start I / O suspension detection on the partition. In this way, after the main thread obtains the I / O suspension detection results of the ordinary partition, it can also promptly obtain the I / O suspension detection results of the auxiliary thread on the garbage collection partition, thereby completing the I / O suspension detection of each partition of the memory in this round.

[0140] In practical applications, considering that the number of ordinary partitions in memory is usually greater than the number of garbage-collected partitions, both threads start I / O suspension detection on the partitions simultaneously. Normally, the auxiliary thread will complete the I / O suspension detection earlier than the main thread. This ensures that when the main thread accesses the preset memory area after obtaining the I / O suspension detection results for all ordinary partitions, the garbage-collected partition's I / O suspension detection results are already stored in that area. However, there may be cases where the auxiliary thread completes the garbage-collected partition's I / O suspension detection later. Therefore, in this embodiment, when the main thread accesses the preset memory area shared with the auxiliary thread, it can also compare the time difference between the current time and the start time of the I / O suspension detection to see if it is greater than or equal to a preset duration threshold. The preset duration threshold can be configured according to actual needs, and this embodiment does not limit it. Therefore, if the main thread finishes the I / O suspension detection of the normal partition and the time difference between the start detection time and the end time exceeds a preset duration threshold, it can directly access the preset memory area shared with the auxiliary thread. Otherwise, it can wait until the time difference between the current time and the start time is greater than or equal to the preset duration threshold before accessing the preset memory area shared with the auxiliary thread. Thus, this embodiment ensures that the main thread can promptly and reliably obtain the I / O suspension detection results of the garbage collection partition from the auxiliary thread after receiving the I / O suspension detection results of the normal partition.

[0141] In some examples, the method may also include:

[0142] From the I / O pending detection results of the normal partition and the I / O pending detection results of the garbage collection partition, obtain the target partition with the pending I / O request.

[0143] In the target metadata of the target partition, mark information indicating that the target partition has a fault is set.

[0144] In this embodiment, considering that the metadata of each partition in the memory is maintained by the main thread, after the main thread collects the I / O hang detection results of each partition (normal partition + garbage collection partition) in the memory, it can obtain the target partition with the suspended I / O request, that is, the partition where the I / O hang phenomenon occurs. The main thread can access the metadata of these target partitions, set a flag to represent the fault, and then perform subsequent response processing, such as stopping the read and write service of the faulty partition, notifying the upper management module, etc.

[0145] As can be seen from the above embodiments, in this embodiment, the I / O hang detection method maintains the basic logic of detecting I / O hangs based on the queuing time of read and write requests on the Zone; by separating the I / O hang detection logic of the Recycle Zone to the Recycle thread, the I / O hang detection framework on the main I / O thread asynchronously obtains the I / O hang detection results of the Recycle Zone. This also avoids the additional synchronization overhead required for multiple threads (Recycle / IO threads) to access the Zone request queue on the I / O path.

[0146] Corresponding to the embodiments of the aforementioned I / O suspend detection method, this disclosure also provides embodiments of an I / O suspend detection device and the computer equipment on which it is applied.

[0147] Embodiments of the I / O suspend detection device disclosed herein can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, as shown in Figure 4, which is a hardware structure diagram of the computer device where the I / O suspend detection device of this disclosure is located, in addition to the processor 410, network interface 420, memory 430, and non-volatile memory 440 shown in Figure 4, the computer device where the I / O suspend detection device is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be elaborated further.

[0148] As shown in Figure 5, Figure 5 is a block diagram of an I / O suspension detection device according to an exemplary embodiment of this disclosure. The device is applied to the main thread of a storage engine, the storage engine further includes an auxiliary thread, the storage engine is used to access memory based on a partition namespace, the memory includes at least one partition; the auxiliary thread is used to execute partition reclamation tasks related to the garbage collection partition submitted by the main thread and I / O suspension detection tasks of the garbage collection partition; the method includes:

[0149] Access module 51 is configured to: if it is determined that I / O suspend detection is required, access the I / O request queue of a normal partition in the memory; the normal partition is another partition in the memory that is different from the garbage collection partition;

[0150] The detection module 52 is used to: detect whether there are any suspended I / O requests in the I / O request queue of the normal partition, and obtain the I / O suspension detection result of the normal partition;

[0151] The acquisition module 53 is used to: acquire the I / O suspension detection result of the auxiliary thread on the garbage collection partition.

[0152] In some examples, the acquisition module 53 acquires the I / O hang detection results of the auxiliary thread on the garbage collection partition, including:

[0153] Access a preset memory region shared with the auxiliary thread, and read the I / O suspension detection result of the garbage collection partition from the preset memory region; the I / O suspension detection result of the garbage collection partition is written by the auxiliary thread after performing I / O suspension detection on the I / O request queue corresponding to the garbage collection partition.

[0154] In some examples, the acquisition module 53 is also used for:

[0155] After reading the I / O suspend detection result of the garbage collection partition from the preset memory area, the I / O suspend detection result of the garbage collection partition is cleared from the preset memory area.

[0156] In some examples, the detection module 52 detects whether there are any pending I / O requests in the I / O request queue of the normal partition, including:

[0157] Read the first I / O request in the I / O request queue of the normal partition and obtain the queuing time of the first I / O request;

[0158] The queuing time is compared with a preset threshold to determine whether there are any pending I / O requests in the I / O request queue of the normal partition.

[0159] In some examples, the auxiliary thread includes a task scheduler, a task executor, and a task queue, and the device further includes a registration module for:

[0160] The timed detection rules are registered to the task scheduler of the auxiliary thread, so that the task scheduler adds the I / O suspension detection task to the task queue based on the timed detection rules. After the task executor retrieves the I / O suspension detection task from the task queue, it performs I / O suspension detection on the I / O request queue of the garbage collection partition.

[0161] In some examples, the access module 51 is further configured to: if it is determined based on the timing detection rule that I / O suspend detection is currently required, access the I / O request queue of the normal partition in the memory;

[0162] The acquisition module 53 is further configured to:

[0163] Compare the time difference between the current time and the start time of I / O suspend detection to see if it is greater than or equal to a preset duration threshold.

[0164] If yes, then access the preset memory region shared with the auxiliary thread; if no, wait until the time difference between the current time and the start time is greater than or equal to a preset duration threshold before accessing the preset memory region shared with the auxiliary thread.

[0165] In some examples, the device further includes a setting module for:

[0166] From the I / O pending detection results of the normal partition and the I / O pending detection results of the garbage collection partition, obtain the target partition with the pending I / O request; in the target metadata of the target partition, set the marker information indicating that the target partition has a fault.

[0167] As shown in Figure 6, Figure 6 is a block diagram of another I / O hang detection device according to an exemplary embodiment of the present disclosure. The device is applied to an auxiliary thread of a storage engine, the storage engine further including a main thread. The storage engine is used to access memory based on a partition namespace, the memory including at least one partition. The auxiliary thread is used to execute partition reclamation tasks related to garbage collection partitions submitted by the main thread. The device includes:

[0168] Access module 61 is used to: access the I / O request queue corresponding to the garbage collection partition if it is determined that I / O suspend detection is required at present;

[0169] Detection module 62 is used to: detect whether there are any suspended I / O requests in the I / O request queue, and obtain the I / O suspension detection result of the garbage collection partition;

[0170] Module 63 is provided for: providing the I / O suspension detection results of the garbage collection partition to the main thread.

[0171] In some examples, the access module 61 provides the I / O suspend detection result of the garbage collection partition to the main thread, including:

[0172] Access a preset memory region shared with the main thread, and write the I / O suspension detection result into the preset memory region so that the main thread can obtain the I / O suspension detection result of the garbage collection partition from the preset memory region.

[0173] In some examples, the detection module 62 detects whether there are any pending I / O requests in the I / O request queue, including:

[0174] Read the first I / O request in the I / O request queue and obtain the queuing time of the first I / O request;

[0175] The queuing time is compared with a preset suspension threshold to determine whether there are any suspended I / O requests in the I / O request queue of the garbage collection partition.

[0176] The specific implementation process of the functions and roles of each module in the above-mentioned I / O suspend detection device can be found in the implementation process of the corresponding steps in the above-mentioned I / O suspend detection method, and will not be repeated here.

[0177] Accordingly, this disclosure also provides a storage engine, which includes a main thread and an auxiliary thread. The storage engine is used to access a memory based on a partition namespace, the memory including at least one partition. The main thread is used to execute the steps of the foregoing method embodiments, and the auxiliary thread is used to execute the steps of the foregoing method embodiments.

[0178] Accordingly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned I / O suspend detection method embodiment.

[0179] Accordingly, this disclosure also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the I / O suspend detection method embodiment.

[0180] Accordingly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the I / O hang-up detection method embodiment.

[0181] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0182] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0183] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0184] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0185] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0186] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0187] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0188] While this disclosure contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the multiple embodiments of this disclosure may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0189] The terms "specific example" or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the embodiments or examples, which are included in at least one embodiment or example of this disclosure. In this disclosure, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0190] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0191] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0192] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / bool> < / bool>

Claims

1. An I / O suspension detection method, the method being applied to the main thread of a storage engine, the storage engine further comprising an auxiliary thread, the storage engine being used to access memory based on a partition namespace, the memory comprising at least one partition; the auxiliary thread being used to execute partition reclamation tasks related to a garbage collection partition submitted by the main thread and I / O suspension detection tasks for the garbage collection partition; the method comprising: If it is determined that I / O suspend detection is required, access the I / O request queue of the normal partition in the memory; The ordinary partition is another partition in the memory that is different from the garbage collection partition; Detect whether there are any pending I / O requests in the I / O request queue of the normal partition, and obtain the I / O pending detection result of the normal partition; Obtain the I / O suspension detection result of the auxiliary thread on the garbage collection partition.

2. The method according to claim 1, wherein obtaining the I / O suspension detection result of the auxiliary thread on the garbage collection partition includes: Access a preset memory region shared with the auxiliary thread, and read the I / O suspension detection result of the garbage collection partition from the preset memory region; The I / O suspension detection result of the garbage collection partition is written by the auxiliary thread after performing I / O suspension detection on the I / O request queue corresponding to the garbage collection partition.

3. The method according to claim 2, further comprising: After reading the I / O suspend detection result of the garbage collection partition from the preset memory area, the I / O suspend detection result of the garbage collection partition is cleared from the preset memory area.

4. The method according to any one of claims 1-3, wherein detecting whether there are any pending I / O requests in the I / O request queue of the normal partition includes: Read the first I / O request in the I / O request queue of the normal partition and obtain the queuing time of the first I / O request; The queuing time is compared with a preset suspension threshold to determine whether there are any suspended I / O requests in the I / O request queue of the normal partition.

5. The method according to any one of claims 1-4, wherein the auxiliary thread comprises a task scheduler, a task executor, and a task queue, and the method further comprises: The timed detection rules are registered to the task scheduler of the auxiliary thread, so that the task scheduler adds the I / O suspension detection task to the task queue based on the timed detection rules. After the task executor retrieves the I / O suspension detection task from the task queue, it performs I / O suspension detection on the I / O request queue of the garbage collection partition.

6. The method according to claim 5, wherein accessing the I / O request queue of the ordinary partition in the memory if it is determined that I / O suspend detection is currently required includes: If it is determined based on the timing detection rules that I / O suspend detection is required, access the I / O request queue of the normal partition in the memory; The access to the preset memory region shared with the auxiliary thread includes: Compare the time difference between the current time and the start time of I / O suspend detection to see if it is greater than or equal to a preset duration threshold. If yes, then access the preset memory area shared with the auxiliary thread; if no, wait until the time difference between the current time and the start time is greater than or equal to a preset duration threshold before accessing the preset memory area shared with the auxiliary thread.

7. The method according to any one of claims 1-6, further comprising: From the I / O pending detection results of the normal partition and the I / O pending detection results of the garbage collection partition, obtain the target partition with the pending I / O request. In the target metadata of the target partition, mark information indicating that the target partition has a fault is set.

8. The method according to any one of claims 1-7, wherein determining that I / O pending detection is currently required includes: In response to receiving an external trigger command, or based on a preset timed detection rule, or when the number of I / O requests received per unit time exceeds a preset threshold, the I / O suspension detection process is triggered.

9. The method according to claim 8, wherein the preset memory region maintains an atomic Boolean variable for each garbage collection partition to characterize whether there is an I / O pending phenomenon in the partition; the auxiliary thread updates the variable through an atomic write operation, and the main thread obtains the value of the variable through an atomic read operation.

10. The method according to claim 7, after setting the fault characterization marker information in the target metadata of the target partition, further comprising: Stop submitting new I / O requests to the target partition and update the status of the target partition to a fault state; If the storage engine is running in a distributed system, it reports the fault information of the target partition to the upper-layer management module to trigger the replica switch or data reconstruction process.

11. The method according to claim 5, wherein the timing detection rule is configured by the main thread when the auxiliary thread is started, and is passed to the task scheduler of the auxiliary thread through a read-only configuration structure shared between threads, and the timing detection rule includes a detection period or an absolute trigger time point.

12. The method according to any one of claims 1-11, further comprising, upon detecting a pending I / O request: Record the identifier, partition, enqueue time, and current system time of the suspended I / O request, and generate a diagnostic log; The diagnostic logs are used for subsequent analysis of hardware health status or optimization of I / O scheduling strategies.

13. An I / O suspend detection method, the method being applied to an auxiliary thread of a storage engine, the storage engine further comprising a main thread, the storage engine being used to access memory based on a partition namespace, the memory comprising at least one partition; The auxiliary thread is used to execute the garbage collection tasks related to the garbage collection partition submitted by the main thread; the method includes: If it is determined that I / O suspend detection is required, access the I / O request queue corresponding to the garbage collection partition; Detect whether there are any pending I / O requests in the I / O request queue to obtain the I / O pending detection result of the garbage collection partition; The I / O suspension detection results of the garbage collection partition are provided to the main thread.

14. The method according to claim 13, wherein providing the I / O suspension detection result of the garbage collection partition to the main thread comprises: Access a preset memory region shared with the main thread, and write the I / O suspension detection result into the preset memory region so that the main thread can obtain the I / O suspension detection result of the garbage collection partition from the preset memory region.

15. The method according to claim 13 or 14, wherein detecting whether there are any pending I / O requests in the I / O request queue comprises: Read the first I / O request in the I / O request queue of the garbage collection partition and obtain the queuing time of the first I / O request; The queuing time is compared with a preset suspension threshold to determine whether there are any suspended I / O requests in the I / O request queue of the garbage collection partition.

16. The method according to any one of claims 13-15, wherein the garbage collection partition includes a source partition and a destination partition; the step of accessing the I / O request queue corresponding to the garbage collection partition includes: The I / O request queues of the source partition and the destination partition are accessed respectively, and it is checked whether there are any pending I / O requests.

17. A storage engine comprising a main thread and auxiliary threads, the storage engine being used to access a partition-based namespace memory, the memory including at least one partition; the main thread being used to perform the steps of the method of any one of claims 1 to 12, and the auxiliary threads being used to perform the steps of the method of any one of claims 13 to 16.

18. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 16.

19. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

20. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

Images