Host request processing method, storage medium and electronic equipment

Abstract

The invention discloses a host request processing method, a storage medium and electronic equipment, and the method comprises the steps: starting a log writing mode under the condition of detecting that the cache protection of an all-flash memory array fails, namely writing object data in a host request into a cache space and a log space at the same time, and after the full flash memory array has a complete fault and the fault is repaired, the data which is not completely refreshed in the cache is refreshed to the memory pool through the log space. In other words, through the data backup and processing process of the log space, the problem of write amplification in order to fill up the stripe in a single control state is avoided, and meanwhile, the data security under the condition that the array is completely powered down is also ensured; the technical problem that the processing efficiency of the host request is low under the condition that the all-flash memory array controller fails or the standby power supply fails in the prior art is solved, and the technical effect of improving the processing efficiency of the host request under the cache protection failure state is achieved.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a method for processing host requests, a storage medium, and an electronic device. Background Technology

[0002] In all-flash storage arrays, to cope with sudden cache protection failures, a write-through mode is typically used to directly write write requests from the host to the backend storage device. For example, when an all-flash storage array experiences a controller failure or a backup power failure, the write cache switches to write-through mode.

[0003] While the above methods can ensure data security when the controller is redundant or the backup power supply is normal, the write-through mode is usually a non-full RAID (Redundant Array of Independent Disks) stripe write. It requires reading the data of the data block to be updated and the data of the parity block in the stripe, and then writing the data and parity again. This increases the number of read and write operations and the processing latency of host requests, resulting in a technical problem of low processing efficiency for host requests. Summary of the Invention

[0004] This application provides a method for processing host requests, a storage medium, and an electronic device to at least solve the problem of low processing efficiency of host requests in the context of cache protection failure in all-flash storage arrays in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for processing host requests is provided, comprising: in response to a cache protection failure of an all-flash storage array, writing object data in a host request into a cache space in the all-flash storage array and writing the object data into a log space, wherein the log space is a storage space in the all-flash storage array independent of the cache space; during the process of writing the object data into the cache space and the log space and performing a flush operation on the object data in the cache space, in response to fault repair of the all-flash storage array, determining the remaining data in the cache space that has not been completely flushed based on the flush progress indicated by a target identifier in the log space, wherein the object data includes the remaining data, and the value of the target identifier is updated each time the data in the cache space is flushed, and the flush progress represents the ratio between the cumulative number of data that has been flushed and the total number of data in the object data; and flushing the remaining data to a storage pool by replaying the log entries in the log space.

[0006] According to another aspect of the embodiments of this application, a host request processing apparatus is also provided, comprising: a first writing unit, configured to, in response to a cache protection failure of an all-flash storage array, write object data in a host request into a cache space in the all-flash storage array and write the object data into a log space, wherein the log space is a storage space in the all-flash storage array independent of the cache space; a first processing unit, configured to, in response to fault repair of the all-flash storage array, determine the remaining data in the cache space that has not been completely flushed based on the flushing progress indicated by a target identifier in the log space, during the process of writing the object data into the cache space and the log space and performing a flushing operation on the object data in the cache space, wherein the object data includes the remaining data, and the value of the target identifier is updated each time the data in the cache space is flushed, and the flushing progress represents the ratio between the cumulative number of data that has been flushed and the total number of data in the object data; and a second processing unit, configured to flush the remaining data to a storage pool by replaying log entries in the log space.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of the processing method for any of the above-described host requests through the computer program.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the steps of any of the above-described host request processing methods when running.

[0009] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the host request processing methods described above.

[0010] By employing the embodiments provided in this application, a write-log mode is enabled when the cache protection of the all-flash storage array fails. Object data from host requests is simultaneously written to both the cache space and the log space. This ensures that in the event of a complete power outage or backup power failure of the array, the remaining data in the object data that has not been completely flushed is replayed in the log space to continue the flushing operation, guaranteeing that the object data can be written completely and efficiently to the storage pool. In other words, through the collaborative work of the cache space and the log space, the write amplification problem that occurs when flushing object data to the storage pool in a pass-through mode under single-controller conditions is avoided. Simultaneously, it ensures that even after a complete array failure and recovery, the remaining data that has not been completely flushed can be written to the storage pool. This solves the problem of excessive latency caused by non-full-strip writing in the prior art, achieving the technical effect of improving the processing efficiency of host requests. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of a host request processing method according to an embodiment of this application.

[0013] Figure 2 This is a flowchart of an optional host request processing method according to an embodiment of this application.

[0014] Figure 3 This is a flowchart illustrating the data writing method when the all-flash storage array controller is redundant or the BBU backup battery is healthy.

[0015] Figure 4 This is a flowchart of an optional method for enabling log writing according to an embodiment of this application.

[0016] Figure 5 This is a flowchart of an optional log inspection and replay process according to an embodiment of this application.

[0017] Figure 6 This is a structural block diagram of an optional host request processing apparatus according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] According to one aspect of the embodiments of this application, a method for processing host requests is provided. Optionally, in this embodiment, the above-described method for processing host requests may be applied to, but is not limited to, methods such as... Figure 1 In the hardware scenario shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0022] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the host request processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0024] The embodiments of this application can be applied to data writing and fault recovery scenarios under controller failure or backup power failure conditions of all-flash storage arrays. For ease of understanding, specific examples of several application scenarios are given below.

[0025] (1) Data protection for critical applications in data centers: In large data center environments, server clusters run various services with extremely high data security requirements, such as financial transactions and medical record storage. When encountering cache redundancy loss or BBU (Battery Backup Unit) backup battery failure, the technical solution of this application records write operations to an independent log space in real time, ensuring that even after the system suffers an unexpected interruption, the incomplete write operations can be quickly recovered, maintaining data integrity and consistency, and significantly enhancing the continuity and reliability of critical services.

[0026] (2) Elastic storage management of cloud computing platforms: Cloud computing platforms typically need to provide stable storage performance under varying workloads. When cache protection mechanisms are challenged, traditional write-through strategies can lead to a significant drop in storage performance. However, the technical solution in this application achieves efficient data aggregation and writing even in cache write-through mode by dynamically adjusting RAID stripe size and utilizing log replay technology. This reduces read / write amplification and ensures that cloud storage services maintain high throughput and low latency even when facing various hardware failures, meeting elasticity requirements while improving resource utilization and user satisfaction.

[0027] (3) High-performance write optimization for enterprise-level storage: Enterprise-level all-flash arrays have extremely stringent requirements for write performance when handling a large number of concurrent write operations, such as database transaction log recording and big data analysis writes. The technical solution of this application logs the data written to the cache. Even under the adverse condition of cache protection failure, it can still achieve efficient and orderly writing to the storage pool through subsequent log replay and data integrity verification. This avoids the additional overhead caused by non-full stripe writing, ensures the stability of storage performance in high-concurrency write scenarios, and provides a solid data storage foundation for enterprise-level applications.

[0028] By applying the technical solution of this application to the above-mentioned scenarios, the processing capability of the all-flash storage array in the event of a failure and the optimization of write performance are improved, especially in terms of data security, performance stability and efficient resource utilization.

[0029] The host request processing method in this embodiment can be executed by a server device, or by a server device in combination with at least one of a terminal device (which can also be understood as input / output device 108). Alternatively, the terminal device can execute the host request processing method of this embodiment by a client installed on it.

[0030] Taking the method of processing host requests in this embodiment, executed by the server, as an example, Figure 2 This is a flowchart illustrating an optional host request processing method according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include steps S202 to S206.

[0031] Step S202: In response to the failure of the cache protection of the all-flash storage array, the object data in the host request is written to the cache space in the all-flash storage array, and the object data is written to the log space, wherein the log space is a storage space in the all-flash storage array that is independent of the cache space.

[0032] Step S204: During the process of writing the object data to the cache space and the log space and performing a flush operation on the object data in the cache space, in response to the fault repair of the all-flash storage array, based on the flush progress indicated by the target identifier in the log space, the remaining data in the cache space that has not been completely flushed is determined. The object data includes the remaining data. The value of the target identifier is updated every time the data in the cache space is flushed. The flush progress represents the ratio between the cumulative number of data that has been flushed and the total number of data in the object data.

[0033] During the process of flushing object data through the cache space, as long as a flush operation is performed once, the value of the target identifier in the log space will be updated synchronously. The purpose is to synchronize the data flushing progress in the cache space to the log space in real time, so as to accurately locate the first piece of data in the object data that has not been flushed during fault recovery.

[0034] For example, suppose 100 data entries are written to the cache space and log space. When the first 21 data entries are flushed to the storage pool through the cache space, the value of the target identifier in the log space is updated to... If the all-flash storage array experiences a complete power failure at this point, the flush operation will continue from the 22nd record in the log space after the failure is repaired, based on the value of the target identifier.

[0035] Step S206: By replaying the log entries in the log space, the remaining data is flushed to the storage pool.

[0036] To facilitate understanding, let's first combine... Figure 3 A brief description is given of the data writing method when the all-flash storage array (hereinafter referred to as the array) is redundant or the BBU backup battery is healthy.

[0037] S302, the host sends a write I / O request to the array.

[0038] S304 writes data to the array cache space.

[0039] The S306 uses an internal link to mirror data to another controller cache, achieving redundancy protection.

[0040] S308, check the BBU battery level, and if the battery level is sufficient, proceed to step S310.

[0041] S310 sends an ACK "Write complete" to the host.

[0042] At this point, the data is stored only in the cache space protected by the BBU battery.

[0043] S312, asynchronous disk flushing in the background.

[0044] That is, the controller sorts and merges multiple random lowercase letters in the cache space according to LBA (Logical Block Address, the smallest granular block address of a logical volume or logical block device) to form a full RAID stripe, and then writes the data sequentially into the storage device, such as each SSD (Solid-State Drive).

[0045] At this point, the writing is performed in full strips, eliminating the problem of read-modify-write amplification.

[0046] S314, Update metadata.

[0047] This involves updating the mapping relationship between LBA and PBA (PBA Pool Block Address, the lower-level address of the storage pool, which can be used to address lower-level devices, such as RAID addresses) to complete the data write-to-disk process.

[0048] It's important to note that an all-flash storage array is an array where all storage drives are solid-state drives (SSDs). These drives are grouped into a RAID array, and space is managed at a specific granularity above the RAID. Metadata is used to map logical addresses (LBAs) of logical volumes to logical addresses (PBAs) at the storage pool or RAID layer, as well as data fingerprints to PBAs. During data writes, multiple random, discrete LBAs are mapped to a contiguous segment of PBAs, achieving a random-to-sequential write operation, thereby reducing write amplification at the RAID and disk layers. This contiguous, sequentially written space is called a BLOCK (data block), and space is allocated, used, and reclaimed at the BLOCK granularity.

[0049] Among them, BLOCK is a data block in the all-flash storage pool that is allocated and reclaimed. After the data block is allocated, it is written sequentially until it is full. During garbage collection, the valid data in the data block is migrated and reclaimed for reuse.

[0050] To achieve higher space utilization, all-flash storage arrays typically use RAID 5 or RAID 6 algorithms. Under conditions of array controller redundancy and BBU health, host writes data are first written to the cache and mirrored to other controllers before returning a success message. The cache is then asynchronously flushed to the backend storage pool. Because the cache size is large enough, it ensures that the RAID 6 stripe can be fully utilized for writing. However, in the event of controller failure or BBU backup battery failure, to ensure data reliability, the write cache mode will switch to pass-through mode. This can lead to insufficient host write concurrency, preventing the all-flash storage pool layer from fully utilizing the RAID 5 or RAID 6 stripe. Consequently, writes must be performed in a non-full-strip manner. This requires reading the data block to be updated and the parity block data of the stripe, performing parity calculations with the new data to obtain new parity data, and then writing the data and parity again. This increases write amplification, the number of read / write operations, latency, and significantly degrades performance.

[0051] This application's technical solution is an improvement addressing the aforementioned problems. Specifically, when the redundancy caching mechanism of the all-flash storage array fails (e.g., controller redundancy is lost or the health of the BBU backup battery declines), the system switches to cache pass-through mode. At this time, for each host write request, the array not only writes the data to the cache but also writes this data and its metadata (including logical unit ID, data starting sector number, data length, checksum, etc.) to the log space. This process ensures that even in subsequent system failures or power outages, critical write operation records are preserved, providing a basis for data recovery.

[0052] For example, suppose a host requests to write 10MB of object data, and the system is currently in cache pass-through mode. The array writes this 10MB of data to the cache space and simultaneously generates a log entry containing information such as the logical location of the data, the data content, and the checksum. This log entry is then stored in the log space.

[0053] If a complete power outage or failure of the all-flash storage array is detected, and the failure has been repaired, the system will automatically check the log space to find data entries that have not yet been flushed. Based on the target identifier in each entry, the storage array can determine which cached data has been successfully flushed to the storage pool and which data is still in the cache and needs to be flushed. Then, by replaying the log entries, the remaining data (i.e., the data that has not yet been flushed) will be flushed to the storage pool.

[0054] To continue explaining using the example above, if the system checks the logs after the fault is repaired and finds that only a portion of the previous 10MB of data was flushed to the storage pool, then based on the target identifier in the log entry, it will identify the portion of the object data in the cache space that was not flushed (let's say 5MB), and then write this portion of data to the storage pool.

[0055] Replaying log entries in the log space can include, but is not limited to, performing data integrity and consistency checks on the entries. Each log entry is associated with a checksum upon creation to verify the validity and integrity of the data. The storage array verifies each checksum during log replay; only log entries that pass the integrity check are processed, and the data is then flushed back to the storage pool. Furthermore, the storage array must handle redo conflicts, preventing multiple writes to the same data block simultaneously to ensure data consistency and security.

[0056] For example, when replaying log entries, the checksum of the log entry containing the remaining 5MB of data will be checked to ensure that the data is not corrupted before it is flushed to the storage pool. If multiple redo requests target the same data block simultaneously, the system will ensure that only one correct data version is written according to conflict handling rules. The replay process for log entries will be described in detail below with specific examples.

[0057] It should be noted that during the process of flushing data to the storage pool through the storage space, the input in the cache space will be written to the storage pool in the manner of array redundancy or when the BBU backup battery is healthy. Since it is not limited by the host write concurrency, the storage pool can fill the RAID5 or RAID6 stripe write at a sufficient speed.

[0058] However, if the array experiences a complete power outage or battery failure during the process of writing cached data to the storage pool (i.e., flushing cached data), the following will occur: Figure 5 The log checks and redo processes are shown.

[0059] By employing the above approach, even under adverse conditions where cache protection mechanisms fail, the synergy between the cache space and the log space ensures the continuity of data writing and data integrity. Specifically, data is backed up to the log space in real time when the host writes data. Even in the event of system failure or power outage, the residual data in the cache can be accurately resumed based on the log replay mechanism. This avoids read / write amplification issues caused by non-full stripe writing, reduces data writing latency, improves the performance recovery speed and data processing efficiency of the storage array after fault recovery, and ensures data consistency and security.

[0060] In an exemplary embodiment, determining the remaining data in the cache space that has not been completely flushed based on the flushing progress indicated by the target identifier in the log space includes: obtaining the target identifier from the log space, the target identifier being used to indicate the ratio between the number of partially flushed cached data in the object data before the power failure of the all-flash storage array and the total number of the object data; and determining the data in the object data other than the partially flushed cached data as the remaining data.

[0061] When an all-flash storage array experiences a complete controller power failure or BBU battery failure, upon resuming operation and preparing to handle incomplete write operations in the cache, it first queries the log space for the target identifier of the log entry. This is crucial information used by the storage array to mark the data portions that were successfully flushed before the failure. The target identifier is not only associated with a specific write request but also carries metadata about the data flushing progress, including the range and location of data that has been flushed. By parsing this information, the storage array can accurately identify which data in the cache has been successfully persisted to the storage pool and which data still needs to be flushed.

[0062] For example, suppose that before a power outage, a 10MB write request had already undergone partial flushing, with 6MB of data written to the storage pool. However, the remaining 4MB of data, which hadn't been written to the cache due to a BBU battery failure, was also lost. After recovery, the storage array will read the target identifier associated with the write request from the log space. This identifier will clearly indicate that the first 6MB of data has been flushed, while the remaining data has not. Therefore, the storage array can accurately perform subsequent flushing processing on the remaining data that was not flushed.

[0063] As can be seen, the target identifier is used to instruct the storage array to locate the data status in the cache space, specifically distinguishing which data has been successfully flushed and which data has not yet been flushed. Subsequently, the storage array uses this data status to lock the remaining data that has not been flushed, and performs a series of data integrity checks, recovery, and flushing operations on it to ensure that this data can be safely and completely written to the storage pool.

[0064] The log space not only ensures secure data writing when cache protection fails, but it also forms the basis for data recovery and continued flushing operations after a failure. By recording write operation log entries and their checksums in the log space, the storage array can accurately replay these operations during the fault recovery phase, avoiding duplicate or erroneous data writes and solving the "write hole" problem, thus ensuring data continuity and consistency.

[0065] The above methods improve the fault recovery efficiency and data write robustness of the all-flash storage array in the event of cache protection failure. Through log replay and data integrity checks, unnecessary read / write amplification is avoided, storage media wear is reduced, fault recovery time is shortened, and system availability and data processing performance are improved.

[0066] In an exemplary embodiment, the method further includes: querying the header field of at least one log entry in the log space used to store the object data, wherein the header field is used to indicate status information of whether the at least one log entry has been flushed; determining the partial log entries that have not been flushed based on the header field of the at least one log entry; and determining the data in the partial log entries as the remaining data.

[0067] Once the all-flash storage array recovers from a failure, the system needs to check whether the data in the cache has been fully flushed to the storage pool. To efficiently locate any data that hasn't been flushed, the storage array first queries the header fields of log entries related to the object data in the log space. These header fields contain information about the current state of the log entry, such as a flag indicating whether it has been flushed. Using this flag, the storage array can quickly determine which data has been safely persisted to the storage pool and which data needs to be flushed during the recovery phase.

[0068] In this embodiment, before the array fails completely, the log space is written once every time the data in the cache is flushed, that is, the status of the header field of the log entry corresponding to the data of each flush operation is changed to "flushed".

[0069] After querying the header fields and determining the flush status, the storage array will filter out all log entries containing data that has not been flushed, i.e., log entries marked as not flushed in the header field. This step is implemented based on the data organization and marking mechanism in the log space, ensuring that the system can centrally process data that failed to complete the flush operation before the failure occurred, avoiding duplicate processing of flushed data, thereby improving the efficiency of the log replay process.

[0070] Finally, the storage array identifies the remaining data from the previously filtered log entries that were not completely flushed, determining that this data needs to be further flushed to the storage pool. This process involves not only data identification but also data integrity verification and potential conflict handling to ensure data consistency and correctness. After identifying the remaining data, the storage array will sequentially flush the data according to the system recovery strategy until all data from the unflushed log entries has been successfully written to the storage pool.

[0071] Through the above methods, all-flash storage arrays can quickly restore data processing capabilities when faced with abnormal situations such as cache protection failure, ensuring data security and storage system stability. They are particularly suitable for application scenarios with strict requirements for data real-time performance and fault recovery capabilities, such as high-performance computing and data-intensive business systems, providing users with more reliable and efficient data storage services.

[0072] In an exemplary embodiment, the above-described method of replaying log entries in the log space to flush the remaining data to the storage pool includes: determining a first group of log entries in the log space corresponding to the remaining data; sequentially reading each log entry in the first group of log entries into the controller memory of the all-flash storage array based on the log number, wherein the controller is used for read / write management, cache synchronization, and fault recovery of the object data; checking the integrity of each log entry in the controller memory based on the checksum contained in each log entry in the first group of log entries, wherein each log entry contains the checksum, log number, logical unit identifier, data starting sector number, data length, and data content; determining at least some log entries with incomplete content based on the check result, and removing the at least some log entries in the first group of log entries to obtain a second group of log entries; and flushing the remaining data to the storage pool by replaying the second group of log entries.

[0073] When an array experiences a power outage or failure and is subsequently repaired—that is, after an all-flash storage array has undergone a failure or restart—log checks and redo processes are necessary before providing host read / write services to ensure data integrity and consistency. For details, please refer to [link to relevant documentation]. Figure 5 The flowchart shown.

[0074] S502, read log entries.

[0075] After a system restart or fault recovery, the storage array will read each log entry in the log space starting from the lowest log number (in ascending order of number). Each log entry contains key data such as the logical unit ID at the time of writing, the starting sector number of the data, the data length, the data content, and the checksum.

[0076] S504 determines whether all logs have been read and processed.

[0077] If not, proceed to step S506; otherwise, proceed to step S520.

[0078] S506, Check log integrity.

[0079] In the controller's memory, the system performs a data integrity check on the first set of log entries read. This process relies on a checksum contained in each log entry. By recalculating and comparing the checksum with the log content, it can be verified whether the log entry was corrupted during the fault. Log entries with incomplete data or failed checksums are marked to prevent subsequent erroneous data from being written.

[0080] Simultaneously, based on the integrity check results, log entries with incomplete content will be removed. This is to ensure the accuracy and reliability of data recovery, preventing data damaged by failures from being incorrectly rewritten to the storage pool, thus causing data inconsistency or loss. The second set of log entries retained by the system after removal consists of complete log entries that can be safely replayed.

[0081] S508 determines whether the integrity check of the current log entry has passed.

[0082] If successful, proceed to step S510; otherwise, proceed to step S502.

[0083] This is because if the result of the judgment is that the content of the current log or log entry is incomplete, then the current log is considered invalid and discarded directly.

[0084] Specifically, after reading each log entry, the system verifies the integrity and validity of the log entry using a checksum. The checksum is typically a hash value, used to compare whether the log content has been damaged or tampered with during storage.

[0085] S510, check log redo conflicts.

[0086] S512 determines whether the current log entry conflicts with other log entries being reworked, according to the log conflict checking rules.

[0087] If yes, suspend the log; otherwise, proceed to step S514.

[0088] It's important to note that for logs that pass the integrity check, the storage array will continue to check for redo concurrency conflicts. This means checking whether the log currently being redone and other logs being redone involve the same logical unit ID and data sector range. If a conflict is found, the current log is suspended, waiting for the potential conflict to be resolved; if no conflict is found, the redo process continues.

[0089] S514, Execute log redo.

[0090] Logs without concurrent conflicts will be selected for redo, which means writing the data in the log back to the corresponding location in the storage pool. The redo process ensures that the data is written with the correct RAID stripe size and format, avoiding data corruption and read / write amplification. In other words, the data for the current log entry is rewritten.

[0091] When writing data from log entries to the storage pool, it is not emphasized whether it is written in full stripes.

[0092] S516, check for conflicts in the suspended log.

[0093] Throughout the redo process, the system will continuously check for any previously suspended logs that can be processed, ensuring that all potentially affected data is correctly redone.

[0094] S518 checks if there are any previously suspended logs.

[0095] If yes, proceed to step S512; otherwise, proceed to step S502.

[0096] S520, waiting for all redo logs to complete.

[0097] Once all logs have been read and processed, and all redo operations have been completed, the system will consider the log recovery phase to be over. It is then safe to enable host read / write services and resume normal business operations, i.e., to process the latest host requests, such as write or read requests.

[0098] S522, complete the log recovery process.

[0099] After log recovery and redo processing are completed, the storage array will continuously monitor the system status, including host write concurrency and data block size, to dynamically adjust the striping parameters of the storage pool. This ensures that it can effectively cope with various abnormal situations in future operation, guaranteeing data processing efficiency and system reliability.

[0100] It should be noted that when the array experiences a power outage or battery failure and recovers, the integrity of the log entries must be checked and redoed before host read and write services can be resumed.

[0101] The above describes how a storage array ensures data integrity and consistency by checking and redoing logs after a restart or failure recovery, thereby providing users with reliable and secure storage services. Through log integrity checks and conflict checks, the storage array can recover quickly, reducing service interruption time and the risk of data loss.

[0102] This embodiment describes how, during fault recovery, the all-flash storage array safely flushes the remaining data that was not completely flushed from the cache to the storage pool by replaying log entries in the log space. This process includes identifying log entries, reading them into the controller memory, integrity checks, removing incomplete entries, and replaying the data. By using log numbers and checksums, the storage array ensures the correctness and consistency of data recovery, avoids data duplication and "write holes," effectively reduces read / write amplification, and improves the data processing efficiency and reliability of the storage system.

[0103] In an exemplary embodiment, the above-described method of replaying the second group of log entries to flush the remaining data to the storage pool includes: based on the log number, sequentially obtaining each log entry from the second group of log entries as the current log entry, wherein the current log entry contains the current portion of the remaining data; obtaining the current logical unit identifier and the current data sector range of the current log entry; obtaining the other logical unit identifiers and other data sector ranges of other log entries that are currently being replayed and have not yet been completed; comparing the current logical unit identifier with the other logical unit identifiers, and comparing the current data sector range with the other logical unit identifiers. The data sector range is determined to obtain the target comparison result. If the target comparison result indicates that the current logical unit identifier is inconsistent with other logical unit identifiers, and the current data sector range does not overlap with other data sector ranges, the current portion of data is written into the current stripe in the current logical storage unit. The current stripe includes multiple stripe blocks, each corresponding to a continuous physical address segment of a storage hard disk. Based on the mapping relationship between the multiple stripe blocks and the physical addresses of the multiple storage hard disks in the all-flash storage array, the current portion of data in the current stripe is written into each continuous storage space of the multiple storage hard disks.

[0104] Based on the log number order, the system reads and processes each log entry sequentially from the second set of log entries. For example, each log entry acquired at the current moment is designated as the current log entry. This process ensures the sequentiality and continuity of data replay, avoids out-of-order data writing, and thus improves the accuracy of data recovery.

[0105] From the current log entry, the logical cell identifier and data sector range associated with that entry will be extracted. The logical cell identifier is used to locate the logical storage unit to which the data belongs, while the data sector range specifically specifies the location of the data within that logical storage unit. This information is crucial for subsequent data writing and stripe allocation.

[0106] Before processing the current log entry, it is necessary to check if there are other log entries being replayed but not yet completed. This check involves obtaining the logical unit identifier and data sector range of these entries to determine whether a data write conflict will occur between the current log entry and them.

[0107] For example, a system check might reveal that, in addition to the current log entry 1001, there is also a log entry 1002 being processed, with a logical unit ID of 20, a data sector range of starting sector 2000, and a length of 32KB. This information will be used to determine whether writing data to log entry 1001 will conflict with writing data to the data sector of log entry 1002.

[0108] The system will compare the logical unit identifier and data sector range of the current log entry with all other log entries that are being replayed and have not yet been completed. If the logical unit identifier of the current log entry is different, or the current data sector range does not overlap with the data sectors of other entries, it will be determined that the current log entry can be flushed independently without causing data conflicts or overwriting.

[0109] After confirming that writing data to the current log entry will not cause conflicts, the system will write the current portion of data to the current stripe in the current logical storage unit according to the pre-set RAID strategy. A stripe consists of multiple stripe blocks, each corresponding to a contiguous physical address range on a storage hard drive in the storage pool. This ensures that data is stored in a distributed manner, improving data read / write performance and reliability.

[0110] For example, based on the RAID6 strategy, the system distributes the data in log entry 1001 to the current stripe constructed from multiple hard disk segments according to the rule of full stripe aggregation writing. Each stripe block corresponds to a 32KB contiguous physical address in a storage device or storage hard disk, ensuring efficient and orderly data writing.

[0111] Finally, the system utilizes the address mapping relationship between LBA and PBA to write the data in the current stripe to contiguous storage space across multiple storage disks in the all-flash storage array. This process achieves the final flushing of data from the cache to the storage pool and updates the storage pool's metadata, ensuring the accuracy and consistency of the LBA to PBA mapping.

[0112] For example, based on the address mapping relationship mentioned above, the data in the starting sector 1000 and with a length of 64KB in log entry 1001 is written to the contiguous storage space of multiple hard drives. In this way, the data is not only securely persisted, but the efficient operation of the storage pool is also maintained.

[0113] It should be noted that each data block or parity block in a stripe block is mapped to a physical address on the hard drive participating in the RAID. For example, if the stripe block size is 64KB, then each stripe block will be allocated a contiguous 64KB physical space on the hard drive.

[0114] This embodiment describes the implementation details of how an all-flash storage array, during the fault recovery phase, securely and efficiently flushes data that has not yet been flushed from the cache to the storage pool based on log entry information. The entire process includes sequentially reading log entries, obtaining key information and checking for conflicts, determining the data write location, and performing the flush operation.

[0115] In another optional example, the specific methods for checking logs and redoing processes before providing host read / write services after the array loses power (or the single controller also loses power) or the battery fails and is repaired include the following procedures.

[0116] (1) Read one or more logs into memory from low to high according to the log number.

[0117] (2) Check the log integrity based on the log checksum. If it is incomplete, do not redo the log and read the next log. Otherwise, if the log is complete, check the log redo concurrency conflict. If there is no conflict, write the log to the storage pool according to the logical unit ID, data starting sector number, data length (number of sectors), and data content recorded in the log. For logs that do not conflict, redo can be performed in parallel, so that the storage pool can be filled with RAID stripes for writing.

[0118] The log conflict checking rule is as follows: if the logical unit ID and data sector range of the log JN-A to be replayed do not overlap with the logical unit ID and data sector range of all incomplete logs currently being replayed, then the JN-A log can be replayed; otherwise, it is judged as a conflict, JN-A cannot be replayed, and is suspended and waited for the next log to be read and checked according to the concurrency and stripe completion requirements.

[0119] (3) After the log part is redone or all is redone, the log status can be modified. This can be done through a fixed location in the log space (e.g., the superblock). Similarly, the logs that need to be redone are identified based on the information in the superblock when the system starts up.

[0120] By comparing the aforementioned log entries and performing ordered data replay, data write conflicts are effectively avoided, ensuring the continuity and accuracy of data flushing. This mechanism improves the data processing efficiency of the storage array during fault recovery, reduces performance loss caused by overlapping writes, ensures data consistency, avoids "write holes," and enhances the stability and data recovery capabilities of the storage system. It also enhances the fault response capabilities and data processing robustness of the storage array.

[0121] In an exemplary embodiment, the method further includes: in response to the received host request, writing the object data into the cache space; writing the object data into the log space and returning a write completion instruction; and before the all-flash storage array experiences a power failure, in response to performing a flush operation on a portion of the cached data in the object data, marking the log status of the target log entry corresponding to the portion of cached data in the log space as flushed.

[0122] When the all-flash storage array receives a write request from the host, the system first writes the object data to the cache space. The purpose of the cache space is to temporarily store data so that subsequent aggregate write operations can be performed more efficiently.

[0123] Before the array is completely powered off or completely faulty, because the current state controller is still in a single-control state, it is easy to have a non-full stripe pass-through mode, which will cause multiple read and write operations and cause write amplification problems.

[0124] To avoid the aforementioned problems, this embodiment requires initiating a write log mode to handle host write request processes when the array controller loses redundancy or BBU protection. For details, please refer to... Figure 4 As shown.

[0125] S402, Initialize log space.

[0126] Before entering write-log mode, the storage array pre-reserves log space in the storage pool. This space is the size of the write cache for all controllers and has a certain degree of redundancy (between 0 and 2) to ensure high log reliability. This step is completed when the system is running normally and redundancy exists, preparing for subsequent write-log operations.

[0127] S404 receives write requests and data.

[0128] The host sends a write request to the storage array, carrying the data to be written. The storage array receives these requests and temporarily stores the data in its cache.

[0129] S406 writes the data to the cache space.

[0130] Data is temporarily stored in the cache, which quickly responds to write operations from the host and provides low-latency write services.

[0131] S408 records write requests and the data itself to the log space.

[0132] The content written to the log space includes, but is not limited to, logical unit ID, data starting sector number, data length (number of sectors), data content, checksum, time, and log number. The checksum is a hash value, or digest, calculated from the log content using a hash algorithm, used to verify the integrity and validity of the log content.

[0133] S410, the response host.

[0134] After completing the log write, the storage array acknowledges to the host that the I / O request has ended. Once the cache is written, the storage array immediately sends a confirmation to the host that the write is complete, ensuring data reliability without affecting the host's continued operations.

[0135] S412 writes the cache to the storage pool.

[0136] This also involves flushing the data in the cache space, and subsequently, the storage array flushes the data in the cache to the storage pool. During the flushing process, step S414 is executed.

[0137] S414: Data will be written according to the block allocation principle, trying to fill the RAID stripe as much as possible.

[0138] The purpose of striving to fill all RAID stripes is to reduce write amplification and optimize storage performance.

[0139] S416, change the log status to "flashed".

[0140] After the data is flushed, the storage array will update the status of the corresponding entry in the log space to "flushed". This helps to quickly identify which data has been safely written to the storage pool and which data still needs to be redone when the system restarts or recovers from a failure.

[0141] It should be noted that after the system restarts due to a failure, it will check the logs and redo any data that was not flushed or was incompletely flushed, i.e., rewrite it to the storage pool to ensure data integrity and consistency. During the redo process, the system will check the integrity and concurrency conflicts of the logs to avoid duplicate writes or data corruption, ensuring reliable data recovery.

[0142] Log space is used in chronological order under normal circumstances. That is, after the log number is confirmed and flushed, the space of the log can be overwritten by the new log content, thus achieving efficient space management and utilization.

[0143] By implementing the above process and enabling log writing, persistent data storage, minimized read / write amplification, and optimized system performance are ensured in cache-write-through mode. This also provides a reliable mechanism for data recovery after a failure. It not only improves the overall data processing capabilities of the system but also guarantees data security and storage efficiency, demonstrating significant advantages in storage applications with high concurrency and high reliability requirements.

[0144] In this embodiment, in order to improve the write performance of the storage system, another optional example is proposed, which is mainly used to dynamically adjust the RAID stripe size in single controller operation or BBU backup battery failure state, so that only a smaller write concurrency is needed to fill the stripe for writing.

[0145] Specifically, this includes: in response to a cache protection failure of the all-flash storage array, monitoring and statistically analyzing the average write concurrency and the data block size during each write operation when issuing host requests; and dynamically adjusting the striping parameters of at least one stripe contained in the data block of the all-flash storage array based on the average write concurrency and the data block size, wherein the striping parameters include stripe width and stripe block granularity, the stripe width indicating the number of hard drives constituting a stripe, and the stripe block granularity representing the space occupied by each data block in the stripe when written to a single storage hard drive.

[0146] The implementation of this embodiment is based on the fact that the storage space of the all-flash storage pool is allocated, used, and reclaimed according to BLOCK. After the sequential writing of a BLOCK is completed, it will not be written again before it is reclaimed. Each time a BLOCK is written, the RAID stripe size of the BLOCK space is determined first. According to the principle of full stripe aggregation writing, in the single controller or when the BBU protection is lost, in the cache pass-through mode, small RAID stripes are used to achieve most or all full stripe aggregation writing, thereby reducing or eliminating the read and write amplification problem caused by non-full stripe writing.

[0147] To facilitate understanding, we will first briefly introduce the relationship between the all-flash storage pool block distribution, RAID stripes, and hard drives.

[0148] (1) Divide the hard disk into fixed sizes and call each part a hard disk segment.

[0149] (2) A BLOCK contains hard disk segments of multiple hard disks, and each of the multiple hard disks that make up a BLOCK contains the same number of hard disk segments.

[0150] (3) BLOCK uses the hard disk segments it contains to build RAID. Each stripe is aligned with the boundary of the hard disk segment. The RAID type (RAID5 / RAID6), stripe width and stripe size are determined by BLOCK. Different BLOCKs may have different RAID types, stripe widths and stripe sizes.

[0151] (4) The pool address PBA of the storage pool consists of two parts: the BLOCK number and the address within the BLOCK. The RAID stripe number and address corresponding to the PBA are determined based on the address within the BLOCK, the RAID type of the BLOCK, the stripe width, and the stripe size.

[0152] When the cache protection features of the all-flash storage array (such as BBU backup battery protection) fail, the system automatically switches to a more conservative cache-through mode. The system monitors write requests issued by the host in real time, statistically analyzing the average write concurrency (host write qdepth, hwqd) and the data block size (host_write_block_size, hwbs) for each write operation. This monitoring process provides necessary data for subsequent striping parameter adjustments, ensuring that the storage pool can still process data efficiently in cache-through mode.

[0153] Based on the monitored average write concurrency and data block size, the storage array dynamically adjusts the striping parameters of the stripes contained in the data blocks within its storage pool. These striping parameters include stripe width and stripe block granularity. Stripe width reflects the number of hard drives that make up a stripe, determining the degree of data dispersion and read / write performance. Stripe block granularity, on the other hand, indicates the space occupied by each data block in the stripe when written to a single storage hard drive, directly affecting the ability to aggregate data writes and the degree of read / write amplification.

[0154] It should be noted that in cache non-write-through mode (such as controller redundancy or BBU health status), BLOCK uses larger stripes, which is achieved by using larger stripe widths or stripe blocks. For example, in RAID6, the stripe width is 10 and the stripe block is 128KB, corresponding to a stripe size of 1MB.

[0155] However, in cache write-through mode (such as when the controller lacks redundancy or the BBU fails), it is expected that the BLOCK will use a smaller stripe, which can be achieved by using a smaller stripe width or stripe block, such as RAID6, stripe width 10, stripe block 4KB, corresponding to a stripe size of 32KB; the specific stripe width or block size can be determined based on the write concurrency and block size issued by the host.

[0156] While keeping the stripe width constant, by reducing the stripe block granularity, each stripe now requires less write concurrency to fill, reducing read / write amplification, lowering write latency in the storage pool, and improving performance.

[0157] By reducing the stripe block granularity (i.e., the stripe block size), the storage array can still perform aggregated writes by filling the stripes even when host concurrent writes are insufficient, effectively avoiding performance bottlenecks caused by incomplete stripe writes, such as read / write amplification issues. Simultaneously, because the stripe width remains constant, the available space ratio of the storage pool is maintained, and the overall system redundancy and data security are not affected. This parameterized adjustment not only improves the storage array's ability to handle sudden failures but also ensures data write efficiency and storage device lifespan management in cache pass-through mode.

[0158] In an exemplary embodiment, the above-described method of dynamically adjusting the striping parameter of at least one stripe contained in the data block of the all-flash storage array based on the average write concurrency and the data block size includes: determining the product between the average write concurrency and the data block size as the adjusted striping parameter, wherein the adjusted striping parameter is used to reduce read / write amplification caused by non-full stripe data writing.

[0159] When the cache protection of the all-flash storage array fails, the system automatically enters cache pass-through mode. In this mode, the average write concurrency of host write requests and the size of each data block written become key factors in adjusting the storage pool striping parameters. The system determines an adjusted striping parameter value by calculating the product of these two parameters. This parameter value directly guides the dynamic adjustment of the stripe width and stripe block granularity contained in the data blocks within the storage pool, ensuring that data writing is performed in a full-striped manner, thereby avoiding read / write amplification issues caused by non-full-striped writes.

[0160] The average write concurrency issued by the host can, but is not limited to, represent the average number of "incomplete" write commands issued by the host to the storage array simultaneously within a unit of time, which is the average write I / O queue depth seen by the host.

[0161] It should be noted that setting the RAID stripe size corresponding to BLOCK to be less than or equal to the product of the two parameters mentioned above (this product is denoted as the average concurrent write data volume) can, in principle, maintain the stripe width and the available space ratio of the storage pool by only modifying the stripe block size.

[0162] The core objective of this embodiment is to reduce read / write amplification caused by incomplete striping writes, while ensuring data write continuity and efficient utilization of the storage pool. By calculating the product of the average write concurrency of host requests and the data block size, and using this as a reference for determining the adjusted striping parameters, the all-flash storage array effectively optimizes the storage pool stripe width and stripe block granularity after cache protection failure. This dynamic adjustment mechanism reduces the occurrence of incomplete striping writes, directly reducing read / write amplification, while maintaining data write continuity and system performance stability. This ensures that the storage array can still provide low-latency, high-efficiency data processing services even in cache pass-through mode.

[0163] As can be seen from the description of the above embodiments, the technical problem to be solved by the technical solution of this application is that when the storage array experiences controller failure or BBU backup battery failure, after the write cache is switched to pass-through, the traditional RAID5 or RAID6 non-full stripe pass-through method is avoided, and a method with smaller write amplification is adopted to obtain lower latency and higher performance, while the data is still in pass-through mode to ensure reliability.

[0164] To address the aforementioned issues, this application proposes two solutions. The first is to add a cache write-log function. When handling host write I / O, data is saved to the cache and simultaneously written to the log space allocated on the storage pool. After completing the log write, the host is acknowledged to end the I / O request. Then, the data in the cache is written to the storage pool according to the array redundancy or BBU backup battery health status. Since it is not limited by host write concurrency, the storage pool can quickly fill the RAID5 or RAID6 stripe for writing. The second solution is for the storage pool to support dynamic adjustment of RAID stripe size at the block level. In single-controller operation or BBU backup battery failure state, the RAID stripe size is adjusted to a smaller granularity (e.g., 4KB~32KB), thus requiring less write concurrency to fill the stripe for writing.

[0165] It should be noted that during non-full stripe writes, read / write amplification increases dramatically. For example, when writing a block of data in RAID6, it is necessary to read 3 copies of the data and write 3 copies of the data, namely 1 data block and 2 parity blocks. The number of read / write operations is 6 times that of the upper layer. When writing a block of data in RAID5, it is necessary to read 2 copies of the data and write 2 copies of the data. Since it is a synchronous write, the latency is large, the storage performance is greatly affected, and the write amplification of the storage device is also large.

[0166] Furthermore, non-full stripe writing also has the problem of "write holes," which means that if the writing of the same stripe is interrupted during the writing process of multiple stripe blocks, such as due to power failure or restart, the writing will be incomplete. In particular, if the parity block is not written completely, it cannot be recovered after the system restarts, which can easily lead to data corruption.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0168] According to another aspect of the embodiments of this application, a host request processing apparatus is also provided, the structural schematic diagram of which is shown below. Figure 6 As shown, it includes the following modules:

[0169] The first writing unit 602 is configured to, in response to a cache protection failure of the all-flash storage array, write the object data in the host request into the cache space of the all-flash storage array and write the object data into the log space, wherein the log space is a storage space in the all-flash storage array independent of the cache space; the first processing unit 604 is configured to, in response to fault repair of the all-flash storage array, determine the remaining data in the cache space that has not been completely flushed based on the flushing progress indicated by the target identifier in the log space, wherein the object data includes the remaining data, and the value of the target identifier is updated each time the data in the cache space is flushed, and the flushing progress represents the ratio between the cumulative number of data that has been flushed and the total number of data in the object data; the second processing unit 606 is configured to flush the remaining data to the storage pool by replaying the log entries in the log space.

[0170] The specific execution steps involved in the various calculation processes, service configurations, and anomaly warnings in the above modules can be referred to the descriptions in the above embodiments, and will not be repeated here.

[0171] Obviously, the aforementioned host request processing apparatus can be used to implement the host request processing method provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0172] It should be noted that the first writing unit 602 in this embodiment can be used to execute the above step S202, the first processing unit 604 in this embodiment can be used to execute the above step S204, and the second processing unit 606 in this embodiment can be used to execute the above step S206.

[0173] In an exemplary embodiment, the first processing unit 604 includes: a first acquisition module, configured to acquire the target identifier from the log space, wherein the target identifier is used to indicate the ratio between the amount of partially cached data that has been flushed in the object data before the power failure of the all-flash storage array and the total amount of the object data; and a first processing module, configured to determine the data in the object data other than the partially cached data as the remaining data.

[0174] In one exemplary embodiment, the above apparatus further includes: a query module, configured to query the header field of at least one log entry in the log space used to store the object data, wherein the header field is used to indicate status information of whether the at least one log entry has been flushed; a second processing module, configured to determine a portion of log entries that have not been flushed based on the header field of the at least one log entry; and a third processing module, configured to determine the data in the portion of log entries as the remaining data.

[0175] In an exemplary embodiment, the second processing unit 606 includes: a fourth processing module, configured to determine a first group of log entries in the log space corresponding to the remaining data; a first reading module, configured to sequentially read each log entry in the first group of log entries into the controller memory of the all-flash storage array based on the log number, wherein the controller is used for read / write management, cache synchronization, and fault recovery of the object data; a first checking module, configured to check the integrity of each log entry in the controller memory based on the checksum contained in each log entry in the first group of log entries, wherein each log entry contains the checksum, log number, logical unit identifier, data starting sector number, data length, and data content; a removal module, configured to determine at least some log entries with incomplete content based on the check result, and remove the at least some log entries in the first group of log entries to obtain a second group of log entries; and a replay module, configured to flush the remaining data to the storage pool by replaying the second group of log entries.

[0176] In an exemplary embodiment, the replay module includes: a first acquisition submodule, configured to sequentially acquire each log entry from the second group of log entries as the current log entry based on the log number, wherein the current log entry includes the current portion of the remaining data; a second acquisition submodule, configured to acquire the current logical unit identifier and the current data sector range of the current log entry; a third acquisition submodule, configured to acquire other logical unit identifiers and other data sector ranges of other log entries currently being replayed and not yet completed; a comparison submodule, configured to compare the current logical unit identifier with the other logical unit identifiers, and compare the current data sector range with the other data sector ranges to obtain a target comparison result; and a first write submodule. A submodule is used to write the current portion of data into the current stripe in the current logical storage unit when the target comparison result indicates that the current logical unit identifier is inconsistent with the other logical unit identifiers and the current data sector range does not overlap with the other data sector ranges. The current stripe includes multiple stripe blocks, each of which corresponds to a continuous physical address segment of a storage hard disk. A second write submodule is used to write the current portion of data in the current stripe into each continuous storage space of the multiple storage hard disks based on the mapping relationship between the multiple stripe blocks and the physical addresses of the multiple storage hard disks in the all-flash storage array. The remaining data includes the current portion of data.

[0177] In one exemplary embodiment, the apparatus further includes: a second writing unit, configured to write the object data into the cache space in response to a received host request; a return unit, configured to write the object data into the log space and return a write completion instruction; and a third processing unit, configured to, before a power failure occurs in the all-flash storage array, in response to performing a flush operation on a portion of the cached data in the object data, mark the log status of the target log entry corresponding to the portion of cached data in the log space as flushed.

[0178] In one exemplary embodiment, the above apparatus further includes: a monitoring unit, configured to monitor and statistically analyze the average write concurrency and the data block size during each write operation in response to a cache protection failure of the all-flash storage array; and an adjustment unit, configured to dynamically adjust the striping parameters of at least one stripe contained in a data block in the storage pool of the all-flash storage array based on the average write concurrency and the data block size, wherein the striping parameters include a stripe width and a stripe block granularity, the stripe width indicating the number of hard disks constituting a stripe, and the stripe block granularity representing the space occupied by each data block in the stripe when written to a single storage hard disk.

[0179] In an exemplary embodiment, the above-mentioned adjustment unit includes: a fifth processing module, configured to determine the product between the average write concurrency and the data block size as the adjusted striping parameter, wherein the adjusted striping parameter is used to reduce read / write amplification caused by non-full stripe data writing.

[0180] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0181] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described host request processing method embodiments.

[0182] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to execute the steps in any of the above-described host request processing method embodiments when it is run.

[0183] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0184] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a computer program, which, when executed by a processor, implements the steps in any of the above-described host request processing method embodiments.

[0185] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described host request processing method embodiments.

[0186] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] The above provides a detailed description of a host request processing method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for processing host requests, characterized in that, include: In response to the failure of cache protection of the all-flash storage array, the object data in the host request is written to the cache space in the all-flash storage array, and the object data is written to the log space, wherein the log space is a storage space in the all-flash storage array that is independent of the cache space; During the process of writing the object data to the cache space and the log space and performing a flush operation on the object data in the cache space, in response to the fault repair of the all-flash storage array, based on the flush progress indicated by the target identifier in the log space, the remaining data in the cache space that has not been flushed is determined, wherein the object data includes the remaining data, and the value of the target identifier is updated every time the data in the cache space is flushed, and the flush progress represents the ratio between the cumulative number of data that has been flushed and the total number of data in the object data; By replaying the log entries in the log space, the remaining data is flushed to the storage pool.

2. The method according to claim 1, characterized in that, The step of determining the remaining data in the cache space that has not been completely flushed, based on the flushing progress indicated by the target identifier in the log space, includes: The target identifier is obtained from the log space, wherein the target identifier is used to indicate the ratio between the amount of cached data that has been flushed in the object data before the power failure of the all-flash storage array and the total amount of the object data; The data in the object data other than the cached data is determined as the remaining data.

3. The method according to claim 1, characterized in that, The method further includes: Query the header field of at least one log entry in the log space used to store the object data, wherein the header field is used to indicate whether the at least one log entry has been flushed; Based on the header field of the at least one log entry, determine the log entries that have not been completely flushed; The data in the aforementioned log entries is identified as the remaining data.

4. The method according to claim 1, characterized in that, The step of replaying log entries in the log space and flushing the remaining data to the storage pool includes: Determine the first group of log entries in the log space that correspond to the remaining data; Based on the log number, each log entry in the first group of log entries is sequentially read into the controller memory of the all-flash storage array, wherein the controller is used to perform read and write management, cache synchronization and fault recovery of the object data; In the controller memory, the integrity of each log entry is checked based on the checksum contained in each log entry in the first group of log entries, wherein each log entry contains the checksum, log number, logical unit identifier, data start sector number, data length, and data content; Based on the inspection results, at least some log entries with incomplete content are identified, and the at least some log entries in the first group of log entries are removed to obtain the second group of log entries; By replaying the second group of log entries, the remaining data is flushed to the storage pool.

5. The method according to claim 4, characterized in that, The step of replaying the second group of log entries and flushing the remaining data to the storage pool includes: Based on the log number, each log entry is sequentially obtained from the second group of log entries as the current log entry, wherein the current log entry contains the current part of the remaining data; Obtain the current logical unit identifier and current data sector range of the current log entry; Retrieve the other logical cell identifiers and other data sector ranges for other log entries that are currently being replayed and are not yet complete; The target comparison result is obtained by comparing the current logical unit identifier with the other logical unit identifiers and comparing the current data sector range with the other data sector ranges. When the target comparison result indicates that the current logical unit identifier is inconsistent with the other logical unit identifiers and the current data sector range does not overlap with the other data sector ranges, the current part of the data is written into the current stripe in the current logical storage unit. The current stripe includes multiple stripe blocks, and each stripe block in the multiple stripe blocks corresponds to a continuous physical address of a storage hard disk. Based on the mapping relationship between the multiple stripe blocks and the physical addresses of the multiple storage hard drives in the all-flash storage array, the current part of the data in the current stripe is written into each contiguous storage space of the multiple storage hard drives, wherein the remaining part of the data includes the current part of the data.

6. The method according to claim 1, characterized in that, The method further includes: In response to the received host request, the object data is written to the cache space; Write the object data to the log space and return a write completion command; Before the all-flash storage array experiences a power failure, in response to performing a flush operation on a portion of the cached data in the object data, the log status of the target log entry corresponding to the portion of cached data in the log space is marked as flushed.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: In response to the failure of the cache protection of the all-flash storage array, the average write concurrency and the data block size during each write operation of the host request are monitored and statistically analyzed. Based on the average write concurrency and the data block size, the striping parameters of at least one stripe contained in the data block of the all-flash storage array are dynamically adjusted. The striping parameters include stripe width and stripe block granularity. The stripe width is used to indicate the number of hard drives that make up a stripe, and the stripe block granularity is used to indicate the space occupied by each data block in the stripe when it is written to a single storage hard drive.

8. The method according to claim 7, characterized in that, The step of dynamically adjusting the striping parameters of at least one stripe contained in a data block in the storage pool of the all-flash storage array based on the average write concurrency and the data block size includes: The product between the average write concurrency and the data block size is determined as the adjusted striping parameter, wherein the adjusted striping parameter is used to reduce read / write amplification caused by non-full stripe data writes.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the host request processing method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the host request processing method as described in any one of claims 1 to 8 when executing the computer program.

Images