Processing method of host request, storage medium and electronic device

Abstract

The application discloses a host request processing method, a storage medium and an electronic device. The method comprises the following steps: in the case that it is detected that the cache protection of a full flash storage array is invalid, a write log mode is enabled, that is, object data in a host request is written into a cache space and a log space at the same time, so that after the full flash storage array completely fails and is repaired, data in the cache which is not flushed to a storage pool is flushed to the storage pool through the log space. In other words, through the data backup and processing process of the log space, the problem of write amplification caused by full bands in a single control state is avoided, and the data security in the case of complete power failure of the array is ensured, the technical problem of low processing efficiency of the host request under the condition of full flash storage array controller failure or backup power failure in the related art is solved, and the technical effect of improving the processing efficiency of the host request under the cache protection invalid state is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and in particular to a host request processing method, a storage medium and an electronic device. BACKGROUND

[0002] In a full flash storage array, in order to cope with sudden cache protection failure, a transparent write mode is usually adopted to directly write host write request data to a backend storage device. For example, when a controller failure or a backup power failure occurs in the full flash storage array, the write cache is switched to the transparent write mode.

[0003] The above-mentioned mode can ensure the safety of data when the controller is redundant or the backup power is normal. However, since the transparent write mode is usually non-full RAID (Redundant Array of Independent Disks) strip writing, the data of the strip data block to be updated and the data of the check block need to be read out, and then the data and the check are written again, which increases the read-write times and the processing delay time of the host request, thereby causing the technical problem of low processing efficiency of the host request. SUMMARY

[0004] The present application provides a host request processing method, a storage medium and an electronic device to at least solve the problem of low processing efficiency of the host request in the related art when a cache protection failure occurs in a full flash storage array.

[0005] According to an aspect of an embodiment of the present application, a host request processing method is provided, including: in response to cache protection failure of a full flash storage array, writing object data in a host request into a cache space in the full flash storage array, and writing the object data into a log space, wherein the log space is a storage space in the full flash storage array independent of the cache space; in a process of writing the object data into the cache space and the log space and performing a flushing operation on the object data in the cache space, in response to failure repair of the full flash storage array, determining remaining part data in the cache space which is not flushed completely based on a target identifier in the log space indicating a flushing progress, wherein the object data includes the remaining part data, the value of the target identifier is updated every time the flushing of the data in the cache space is completed once, and the flushing progress represents a proportion between a cumulative number of data which has been flushed and a total number of data in the object data; and flushing the remaining part data into a storage pool by replaying log entries in the log space.

[0006] According to another aspect of the embodiments of the present application, a host request processing apparatus is further provided, comprising: a first writing unit configured to write object data in a host request into a cache space in a full flash storage array and into a log space in response to cache protection invalidation of the full flash storage array, wherein the log space is a storage space in the full flash storage array independent of the cache space; a first processing unit configured to determine, in a process of writing the object data into the cache space and the log space and performing a flushing operation on the object data in the cache space, remaining partial data in the cache space that is not flushed in response to failure recovery of the full flash storage array based on a flushing progress indicated by a target identifier in the log space, wherein the object data comprises the remaining partial data, a value of the target identifier is updated each time data in the cache space is flushed, and the flushing progress represents a ratio between a cumulative amount of data that has been flushed and a total amount of data in the object data; and a second processing unit configured to flush the remaining partial data into a storage pool by replaying log entries in the log space.

[0007] According to still another aspect of the embodiments of the present application, an electronic device is further provided, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to execute steps of any of the host request processing methods by using the computer program.

[0008] According to still another aspect of the embodiments of the present application, a computer readable storage medium is further provided, which stores a computer program, and the computer program is configured to execute steps of any of the host request processing methods when running.

[0009] According to still another aspect of the embodiments of the present application, a computer program product or a computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute steps of any of the host request processing methods.

[0010] By using the above embodiments provided in the application, when the cache protection failure occurs in the all-flash storage array, the write log mode is enabled, the object data in the host request is written into the cache space and the log space at the same time, so that when the array completely loses power or the backup power supply fails, the remaining part of the object data which is not completed in the flushing is processed through the replay of the log space, and the flushing operation is continued, so as to ensure that the object data can be completely and efficiently written into the storage pool. In other words, through the cooperative work of the cache space and the log space, the write amplification problem in the process of flushing the object data to the storage pool in the single control state using the transparent write mode is avoided, and at the same time, it is ensured that the remaining part of the data which is not flushed can be written into the storage pool after the array completely fails and recovers from the failure, the problem of too long time delay caused by the non-full stripe writing in the prior art is solved, and the technical effect of improving the processing efficiency of the host request is realized. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0012] Figure 1 It is an application scenario diagram of a processing method of a host request according to an embodiment of the application.

[0013] Figure 2 It is a flow diagram of an optional processing method of a host request according to an embodiment of the application.

[0014] Figure 3 It is a flow chart of the data writing mode when the all-flash storage array controller is redundant or the BBU backup battery is healthy.

[0015] Figure 4 It is a flow chart of an optional write log mode according to an embodiment of the application.

[0016] Figure 5 It is a flow chart of an optional log checking and replay processing according to an embodiment of the application.

[0017] Figure 6 It is a structure block diagram of an optional processing device of a host request according to an embodiment of the application. DETAILED DESCRIPTION

[0018] With reference to the drawings and the specific embodiments described below, a better understanding of the present application can be obtained.

[0019] It should be noted that, in the description of the present application, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements not only includes those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article, or apparatus. The terms "first", "second", etc. in the present application are used to distinguish similar objects, and are not intended to describe a specific order or sequence.

[0020] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0021] According to an aspect of the embodiments of the present application, a host request processing method is provided. Optionally, in the present embodiment, the above-mentioned host request processing method can be applied in, but is not limited to, the hardware scenario as shown in Figure 1 , wherein the server device can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device can further include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned server device. For example, the server device can further include more or less components than those shown in Figure 1 , or have a different configuration from that shown in Figure 1 .

[0022] The memory 104 can be used to store computer programs, such as software programs of application software and modules, for example, a computer program corresponding to the processing method of the host request in the embodiments of the present application. The processor 102 performs various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to a server device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0023] The transmission device 106 is used to receive or send data via a network. The specific examples of the above network can include a wireless network provided by a communication provider of a server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module which is used to communicate with the Internet in a wireless manner.

[0024] The embodiments of the present application can be but are not limited to applied to the data write and failure recovery scenarios under the controller failure or backup power failure conditions of the all-flash storage array. In order to facilitate understanding, specific examples of several application scenarios are given below.

[0025] (1) Data protection of key applications in data centers: In a large data center environment, a server cluster runs various businesses with extremely high requirements for data security, such as financial transactions, medical record storage, etc. When encountering cache redundancy loss or BBU (Battery Backup Unit, battery backup unit) backup battery failure, the technical solution of the present application records the write operation to an independent log space in real time, which ensures that even after the system suffers from unexpected interruption, the unfinished write operation can be quickly recovered, the integrity and consistency of the data are maintained, and the continuity and reliability of the key business are significantly enhanced.

[0026] (2) Elastic storage management of cloud computing platform: Cloud computing platforms usually need to provide stable storage performance under variable workloads. When the cache protection mechanism is challenged, the traditional transparent strategy will cause a significant decline in storage performance. However, the technical solution of the present application can achieve efficient data aggregation writing, reduce read-write amplification, and ensure high throughput and low latency of cloud storage services when facing various hardware failures, while meeting the demand for flexibility and improving resource utilization and user satisfaction, even in the cache transparent mode.

[0027] (3) High-performance write optimization of enterprise-level storage: Enterprise-level all-flash storage arrays have extremely high requirements for write performance when handling a large number of concurrent write operations, such as database transaction log recording and big data analysis writing. The technical solution of the present application can achieve efficient and orderly writing to the storage pool through subsequent log replay and data integrity verification, even in the case of cache protection failure, avoiding the additional overhead caused by non-full stripe writing, and ensuring the stability of storage performance in high-concurrency write scenarios, providing a solid data storage foundation for enterprise-level applications.

[0028] By applying the technical solution of the present application to the above scenarios, the processing capacity and write performance optimization of all-flash storage arrays in the face of failures are improved, especially in terms of data security, performance stability, and efficient resource utilization.

[0029] The processing method of the host request in the embodiment of the present application can be executed by the server device, or by the server device in combination with at least one of the terminal device (which can also be understood as the input / output device 108). Among them, the terminal device executing the processing method of the host request in the embodiment of the present application can also be executed by the client installed thereon.

[0030] Taking the server as an example to execute the processing method of the host request in the present embodiment, Figure 2 is a flowchart of an optional processing method of a host request according to an embodiment of the present application, as Figure 2 shown, the flow of the method can include steps S202 to S206.

[0031] Step S202, in response to the cache protection failure of the all-flash storage array, writing the object data in the host request to the cache space in the all-flash storage array, and writing the object data to the log space, wherein the log space is a storage space in the all-flash storage array independent of the cache space.

[0032] Step S204, in the process of writing the object data into the cache space and the log space and performing a flushing operation on the object data in the cache space, in response to a failure recovery of the all-flash storage array, determining remaining partial data in the cache space that is not flushed, based on a flushing progress indicated by the target identifier in the log space, wherein the object data includes the remaining partial data, and each time the flushing of the data in the cache space is completed, the value of the target identifier is updated, and the flushing progress represents a ratio between a cumulative amount of data that has been flushed and a total amount of data in the object data.

[0033] In the process of flushing the object data through the cache space, the value of the target identifier in the log space is updated synchronously as long as a flushing operation is performed, and the purpose is to synchronize the flushing progress of the data in the cache space to the log space in real time, so as to ensure that the first data that is not flushed in the object data is accurately located in the failure recovery.

[0034] For example, it is assumed that a total of 100 pieces of data are written into the cache space and the log space, and when the first 21 pieces of data are flushed to the storage pool through the cache space, the value of the target identifier in the log space is updated to 21. At this time, if a complete power failure fault occurs in the all-flash storage array, the flushing operation will be continued from the 22nd piece of data in the log space according to the value of the target identifier after the failure recovery.

[0035] Step S206, flushing the remaining partial data to the storage pool by replaying the log entries in the log space.

[0036] In order to facilitate understanding, first, the data writing method of the all-flash storage array (hereinafter referred to as array) in the case of redundancy or BBU backup battery health is described briefly. Figure 3

[0037] S302, the host issues a write I / O request to the array.

[0038] S304, write data to the array cache space.

[0039] S306, mirror the data to another controller cache through an internal link to achieve redundancy protection.

[0040] S308, query the BBU power, and perform step S310 under the condition of ensuring sufficient power.

[0041] S310, reply "write complete" ACK to the host.

[0042] At this time, the data is only saved in the cache space protected by the BBU battery. ​

[0043] S312, background asynchronous disk brushing.

[0044] That is, by the controller, the plurality of random small write in the cache space is sorted, merged, and rounded into a full RAID strip according to LBA (Logical Block Address, the minimum granularity block address of logical volume or logical block device), and then the data is sequentially written into the storage device, for example, each SSD (Solid-State Drive, solid-state drive or solid-state hard disk).

[0045] At this time, it is written according to the full strip, and there is no read-modify-write amplification problem.

[0046] S314, updating metadata.

[0047] That is, the mapping relationship between LBA and PBA (PBA Pool Block Address, the address of the lower layer of the storage pool, which can be used to address the lower layer device such as RAID) is updated, and the data is completed.

[0048] It should be noted that the all-flash storage array is an array in which all storage hard disks use solid-state hard disks. The hard disks are combined into an array RAID, and the space is managed according to a certain granularity on the RAID. The logical address (LBA) of the logical volume is mapped to the address PBA of the storage pool layer or the RAID layer through metadata, and the mapping of the data fingerprint value to the PBA. When writing data, a plurality of random discrete LBAs are mapped to a continuous PBA to realize random-to-sequential writing, thereby reducing the write amplification of the RAID layer and the hard disk layer. The space of this continuous sequential write is called BLOCK (data block). The space is allocated and recycled in the granularity of BLOCK.

[0049] Among them, BLOCK is a data block for space allocation and recycling in the all-flash storage pool. After the data block is allocated, it is sequentially written full. When garbage collection is performed, the valid data in the data block is migrated and then recycled for reuse.

[0050] For higher space utilization, RAID5 or RAID6 algorithm is generally used in all-flash storage array. In the case of array controller redundancy and BBU health, the host writes data first to the cache and mirrors to other controllers, and returns success. The cache is then asynchronously brushed to the back-end storage pool. Since the cache is large enough, it can guarantee full RAID6 strip writing. However, in the case of controller failure or BBU backup battery failure, in order to ensure data reliability, the write cache mode will be changed to transparent write mode, which will cause the host write concurrency to be insufficient, resulting in the full-flash storage pool layer not being able to fill the RAID5 or RAID6 strip, and thus the data and the check block data of the strip to be updated need to be read out, and the new data is calculated to obtain new check data, and then the data and the check are written, thereby increasing the write amplification, increasing the read-write times, increasing the delay, and greatly reducing the performance.

[0051] The technical solution of the present application is an improved solution for the above problems. Specifically, when the redundancy cache mechanism of the all-flash storage array fails (such as controller redundancy loss or BBU backup battery health decline), the system switches to cache transparent write mode. At this time, for each host write request, the array not only writes data to the cache, but also writes these data and their metadata (including logical unit ID, data start sector number, data length, check code, etc.) to the log space. This process ensures that even in subsequent system failure or power failure events, the key write operation record is retained, providing a basis for data recovery.

[0052] For example, assume that the host requests to write a 10MB object data, and the system is currently in cache transparent write mode. The array writes the 10MB data to the cache space and simultaneously generates a log entry containing the logical location information, data content and check code of the data, and then stores the log entry in the log space.

[0053] In the case of detecting that the all-flash storage array is completely powered off or completely failed, and the failure has been repaired, the system will automatically check the log space to find data entries that have not completed the flushing operation. According to the target identifier in each entry, the storage array can determine which cache data has been successfully brushed to the storage pool and which data still remains in the cache and needs to be flushed. Then the remaining data (i.e. the data that has not completed flushing) is brushed to the storage pool by replaying the log entry.

[0054] Still taking the above example for explanation, if the system checks the log after the failure is repaired and finds that only part of the previous 10MB data has been brushed to the storage pool, then according to the target identifier in the log entry, the unflushed part of the object data in the cache space (assuming 5MB) will be identified, and then this part of data is written to the storage pool.

[0055] The replaying of the log entries in the log space can include, but is not limited to, data integrity and consistency check of the log entries. Each log entry is accompanied by a check code when created, which is used to verify the validity and integrity of the data. The storage array will verify the check code one by one when replaying the log, and only the log entries that pass the integrity check will be processed, and the data will be re-flushed to the storage pool. In addition, the storage array also needs to handle redo conflicts, that is, prevent multiple writes to the same data block at the same time, to ensure the consistency and security of the data.

[0056] For example, when replaying the log entries, the check code of the log entry where the remaining 5MB data is located will be checked to confirm that the data is not damaged before it is flushed to the storage pool. If there are multiple redo requests pointing to the same data block at the same time, the system will ensure that only one correct version of the data is written according to the conflict handling rules. The replaying of the log entries will be described in detail below in conjunction with specific embodiments.

[0057] It should be noted that in the process of flushing the data to the storage pool through the storage space, the input in the cache space will be written to the storage pool according to the array redundancy or the health of the BBU backup battery at the time, and since it is not limited by the host write concurrency, the storage pool can write to the RAID5 or RAID6 stripe at a sufficient speed.

[0058] However, if the array is completely powered off or the battery is completely failed during the process of writing the cache data to the storage pool (i.e., flushing the cache data), the log check and redo processing as shown in Figure 5 will be started.

[0059] In this way, the continuity of data writing and the guarantee of data integrity can be achieved through the cooperation of the cache space and the log space under the adverse conditions of the failure of the cache protection mechanism. Specifically, the data written by the host is backed up to the log space in real time, and even if a system failure or power failure occurs, the residual data in the cache can be accurately written based on the log replay mechanism, avoiding the read-write amplification problem caused by non-full stripe writing, reducing the delay of data writing, improving the performance recovery speed and data processing efficiency of the storage array after fault recovery, and at the same time ensuring the consistency and security of the data.

[0060] In an example embodiment, determining the remaining portion data in the cache space which is not completed by the destaging, based on the destaging progress indicated by the target identifier in the log space, comprises: obtaining the target identifier from the log space, the target identifier being used to indicate a proportion between a number of the partial cache data in the object data which is completed by the destaging before the power failure of the all-flash storage array and a total number of the object data; determining data in the object data other than the partial cache data as the remaining portion data.

[0061] When the all-flash storage array experiences a complete controller power failure or BBU battery failure, after the recovery and preparation for processing the uncompleted write operation in the cache, the target identifier of the log entry is first queried from the log space, which is the key information used by the storage array to mark the data portion which is successfully destaged before the failure occurs. The target identifier is not only associated with a specific write request, but also carries metadata about the data destaging progress, including the data range and location which is completed by the destaging. By analyzing these information, the storage array can accurately identify which data in the cache has been successfully persisted to the storage pool and which data still needs to be destaged.

[0062] For example, assuming that a 10MB write request has been partially destaged before a power failure, 6MB of data is written to the storage pool, and the remaining 4MB of data which is not written to the cache space is also lost due to the BBU battery failure. After the failure recovery, the storage array will read the target identifier related to the write request from the log space, which will explicitly indicate that the first 6MB of data has completed destaging, and the remaining portion data has not completed. Therefore, the storage array can accurately perform subsequent destaging processing on the remaining portion data which is not destaged.

[0063] As can be seen, the target identifier is used to indicate the data state of the storage array to find the cache space, especially to distinguish which data has been successfully destaged and which data has not been destaged. Subsequently, the storage array uses the data state to lock the remaining portion data which is not destaged, and performs a series of data integrity checks, recovery and destaging operations to ensure that the data can be safely and completely written to the storage pool.

[0064] The log space not only provides protection for the safe writing of data when the cache protection fails, but also is the basis for data recovery and continued destaging operation after the failure. By recording the log entry and its check code of the write operation in the log space, the storage array can accurately replay these operations during the failure recovery phase, avoiding repeated or erroneous writing of data, and also solving the "write hole" problem, ensuring the continuity and consistency of the data.

[0065] By the above manner, the fault recovery efficiency and the robustness of data writing of the all-flash storage array in the cache protection failure case are improved. Through the log replay and the data integrity check, unnecessary read-write amplification is avoided, the wear of the storage medium is reduced, the fault recovery time is shortened, and the availability and data processing performance of the system are improved.

[0066] In an exemplary embodiment, the above method further comprises: querying a header field of at least one log item in the log space for storing the object data, wherein the header field is used to represent state information of whether the at least one log item has been flushed; determining a partial log item with uncompleted flushing according to the header field of the at least one log item; and determining data in the partial log item as the remaining part data.

[0067] When the all-flash storage array recovers from the failure state, the system needs to check whether the data in the cache has been completely flushed to the storage pool. In order to efficiently locate the partial data with uncompleted flushing, the storage array will first query the header field of the log item related to the object data in the log space. The header field contains information about the current state of the log item, such as a flag indicating whether it has been flushed. Through this flag, the storage array can quickly determine which data has been safely persisted to the storage pool and which data needs to be supplemented during the recovery phase.

[0068] In this embodiment, before the array completely fails, the log space is written once for each time the data in the cache is flushed, that is, the state of the header field of the log item corresponding to the data of each flushing operation is modified to "flushed".

[0069] After querying the header field and judging the flushing state, the storage array will screen out the log items where the data with uncompleted flushing is located, that is, the log items whose header fields are marked as not flushed. This step is realized based on the data organization and marking mechanism in the log space, ensuring that the system can centrally process the data that failed to complete the flushing operation before the failure occurred, avoiding repeated processing of the flushed data, and thus improving the efficiency of the log replay process.

[0070] Finally, the storage array will determine the data in the above screened log items with uncompleted flushing as the remaining part data, that is, determine these data as the objects that need to be continuously flushed to the storage pool. This process not only includes data identification, but also involves data integrity verification and potential conflict processing to ensure data consistency and correctness. After determining the remaining part data, the storage array will orderly perform data flushing according to the system recovery strategy until the data in all unflushed log items are successfully written to the storage pool.

[0071] Through the above methods, all-flash storage arrays can quickly restore data processing capabilities when faced with abnormal situations such as cache protection failure, ensuring data security and storage system stability. They are particularly suitable for application scenarios with strict requirements for data real-time performance and fault recovery capabilities, such as high-performance computing and data-intensive business systems, providing users with more reliable and efficient data storage services.

[0072] In an exemplary embodiment, the above-described method of replaying log entries in the log space to flush the remaining data to the storage pool includes: determining a first group of log entries in the log space corresponding to the remaining data; sequentially reading each log entry in the first group of log entries into the controller memory of the all-flash storage array based on the log number, wherein the controller is used for read / write management, cache synchronization, and fault recovery of the object data; checking the integrity of each log entry in the controller memory based on the checksum contained in each log entry in the first group of log entries, wherein each log entry contains the checksum, log number, logical unit identifier, data starting sector number, data length, and data content; determining at least some log entries with incomplete content based on the check result, and removing the at least some log entries in the first group of log entries to obtain a second group of log entries; and flushing the remaining data to the storage pool by replaying the second group of log entries.

[0073] When an array experiences a power outage or failure and is subsequently repaired—that is, after an all-flash storage array has undergone a failure or restart—log checks and redo processes are necessary before providing host read / write services to ensure data integrity and consistency. For details, please refer to [link to relevant documentation]. Figure 5 The flowchart shown.

[0074] S502, read log entries.

[0075] After a system restart or fault recovery, the storage array will read each log entry in the log space starting from the lowest log number (in ascending order of number). Each log entry contains key data such as the logical unit ID at the time of writing, the starting sector number of the data, the data length, the data content, and the checksum.

[0076] S504 determines whether all logs have been read and processed.

[0077] If not, proceed to step S506; otherwise, proceed to step S520.

[0078] S506, Check log integrity.

[0079] In the controller memory, the system will perform a data integrity check on the first set of read log entries. This process relies on the check code included in each log entry, which can verify whether the log entry was damaged during the failure by comparing the check code with the re-computed log content. Log entries with data incompleteness or check failure will be marked to prevent subsequent erroneous data writing.

[0080] Meanwhile, according to the above integrity check result, log entries with incomplete content will be discarded. This is to ensure the accuracy and reliability of data recovery, and to avoid damaged data due to failure being erroneously rewritten into the storage pool, thus causing data inconsistency or loss. The second set of log entries retained by the system after discarding are log entries with complete content and safe for replay processing.

[0081] S508, determine whether the integrity check of the current log entry passes.

[0082] If yes, proceed to step S510; otherwise, jump to step S502.

[0083] This is because when the result of the determination is that the content of the current log entry is incomplete, the current log entry is considered invalid and discarded directly.

[0084] Specifically, after reading each log entry, the system verifies the integrity and validity of the log entry through the check code. The check code is usually a hash value, which is used to compare whether the log content is damaged or tampered during storage.

[0085] S510, check log redo conflict.

[0086] S512, according to the log conflict checking rule, determine whether the current log entry conflicts with other log entries being redone.

[0087] If yes, suspend the log entry; otherwise, proceed to step S514.

[0088] It should be noted that for log entries that pass the integrity check, the storage array will continue to detect whether there is a redo concurrency conflict. This means checking whether the current log entry to be redone and other log entries being redone involve the same logical unit ID and data sector range. If there is a conflict, the current log entry is suspended, waiting for possible conflict resolution; if there is no conflict, the redo is continued.

[0089] S514, perform log redo.

[0090] Logs without concurrent conflicts will be selected for redo, i.e. the data in the logs will be written back to the corresponding location in the storage pool. The redo process ensures that data is written in the correct RAID stripe size and format, avoiding data corruption and read-write amplification. That is, the current log is rewritten with the data.

[0091] In the process of writing data in the log entry to the storage pool, it is not emphasized whether to write in full stripe.

[0092] S516, check the suspension of log conflict.

[0093] During the entire redo process, the system will continue to check whether there are previously suspended logs that can continue to be processed, ensuring that all possible affected data is correctly redone.

[0094] S518, determine whether there are previously suspended logs.

[0095] If so, perform step S512; otherwise, jump to step S502.

[0096] S520, wait for all redo logs to complete.

[0097] If all logs have been read and processed, and all redo operations have been completed, the system will consider that the log recovery phase is complete, and the host read-write service can be safely started, and normal business operations can be resumed, i.e. normally processing the latest host requests such as write requests or read requests.

[0098] S522, complete the log recovery processing flow.

[0099] After the log recovery and redo processing is completed, the storage array will continue to monitor the system state, including the host write concurrency and data block size, to dynamically adjust the striping parameters of the storage pool, ensuring that various abnormal situations can be effectively handled in future operation, and the efficiency of data processing and the reliability of the system are guaranteed.

[0100] It should be noted that when the array experiences power failure or battery failure and recovers, the integrity of the log entry and the redo processing need to be checked first, and then the host read-write service can be restarted.

[0101] Through the above method, it is described how the storage array checks and redoes logs after restart or failure recovery to ensure data integrity and consistency, thereby providing reliable and secure storage services for users. Through log integrity checking and conflict checking, the storage array can quickly recover, reducing service interruption time and the risk of data loss.

[0102] In the embodiment, the full flash storage array is described to realize the safe flushing of the remaining part data in the cache to the storage pool by replaying the log entries in the log space during the fault recovery process. The process includes the determination of the log entries, reading to the controller memory, integrity check, rejection of incomplete entries, and replaying. By using the log number and the check code, the storage array can ensure the correctness and continuity of data recovery, avoid data repeated writing and "write hole" problem, effectively reduce the read-write amplification, and improve the data processing efficiency and reliability of the storage system.

[0103] In an exemplary embodiment, the above flushing the remaining part data to the storage pool by replaying the second group of log entries includes: sequentially obtaining each log entry from the second group of log entries as a current log entry based on the log number, wherein the current log entry contains a current part of data in the remaining part data; obtaining a current logical unit identifier and a current data sector range of the current log entry; obtaining other logical unit identifiers and other data sector ranges of other log entries that are currently being replayed and not completed; comparing the current logical unit identifier with the other logical unit identifiers, and comparing the current data sector range with the other data sector ranges to obtain a target comparison result; in the case that the target comparison result indicates that the current logical unit identifier is inconsistent with the other logical unit identifiers, and the current data sector range does not overlap with the other data sector ranges, writing the current part of data into a current stripe in a current logical storage unit, wherein the current stripe includes a plurality of stripe blocks, and each stripe block in the plurality of stripes corresponds to a segment of continuous physical addresses of a storage hard disk; based on the mapping relationship between the plurality of stripe blocks and the physical addresses of the plurality of storage hard disks in the full flash storage array, writing the current part of data in the current stripe to each continuous storage space of the plurality of storage hard disks.

[0104] Based on the order of the log number, the system will read and process each log entry from the second group of log entries one by one, for example, determining each log entry obtained at the current time as a current log entry. This process ensures the order and continuity of data replay, avoiding out-of-order writing of data, thereby improving the accuracy of data recovery.

[0105] From the current log entry, the logical unit identifier and the data sector range associated with the log entry will be extracted. The logical unit identifier is used to locate the logical storage unit to which the data belongs, and the data sector range specifies the location of the data in the logical storage unit. This information is crucial for subsequent data writing and stripe allocation.

[0106] Before processing the current log entry, it is also necessary to check whether there are other log entries that are being replayed and have not yet been completed. This check involves obtaining the logical unit identification and data sector range of these entries, which are used to determine whether the data write of the current log entry will conflict with them.

[0107] For example, the system checks and finds that, in addition to the current log entry 1001, there is a log entry 1002 being processed, with a logical unit ID of 20 and a data sector range of start sector 2000, length 32KB. These pieces of information will be used to determine whether the data write of the log entry 1001 will conflict with the data sectors of 1002.

[0108] The system will compare the logical unit identification and data sector range of the current log entry with all other log entries that are being replayed and have not yet been completed. If it is found that the logical unit identification of the current log entry is different, or the current data sector range does not overlap with the data sectors of other entries, it is determined that the current log entry can be independently performed the next flush operation and will not cause data conflict or overwrite.

[0109] After confirming that the data write of the current log entry will not cause a conflict, the system will write the current partial data to the current stripe in the current logical storage unit according to the pre-set RAID strategy. The stripe is composed of multiple stripe blocks, each of which corresponds to a segment of continuous physical addresses of a storage hard disk in the storage pool, ensuring that data can be stored in a dispersed manner, improving the read / write performance and reliability of the data.

[0110] For example, based on the RAID6 strategy, the system will allocate the data in the log entry 1001 to the current stripe composed of multiple hard disk segments according to the full stripe aggregation write rule, and each stripe block corresponds to a 32KB continuous physical address in a storage device or storage hard disk, ensuring efficient and orderly data write.

[0111] Finally, the system will use the address mapping relationship between LBA and PBA to write the data in the current stripe to the continuous storage space of multiple storage hard disks in the all-flash storage array. This process realizes the final flush of data from the cache to the storage pool and updates the metadata of the storage pool, ensuring the accuracy and consistency of the LBA to PBA mapping.

[0112] For example, according to the above address mapping relationship, the data with a start sector of 1000 and a length of 64KB in the log entry 1001 is written to the continuous storage space of multiple hard disks. In this way, the data is not only safely and persistently stored, but also maintains the efficient operation of the storage pool.

[0113] It should be noted that each data block or check block in the strip block is mapped to the physical address of the hard disk participating in the RAID, for example, if the strip block size is 64KB, each strip block is allocated to a continuous 64KB physical space on the hard disk.

[0114] In the present embodiment, the implementation details of how the all-flash storage array safely and efficiently flushes the data in the cache that has not been completed to the storage pool based on log entry information during the fault recovery phase are described. The entire process includes sequentially reading log entries, obtaining key information and checking conflicts, determining data write locations, and performing flushing operations.

[0115] In another optional example, when the array experiences a power failure (single controller also powers off) or battery failure and is repaired, the specific way of checking logs and redoing processing before providing host read-write services also includes the following process.

[0116] (1) Read one or more logs from low to high according to the log number into memory.

[0117] (2) Check the integrity of the log according to the log check code, if it is not complete, do not redo the log, read the next log, otherwise if the log is complete, check the log redo and conflict, if there is no conflict, write the data starting sector number, data length (sector number), and data content to the storage pool according to the logical unit ID, data starting sector number, data length (sector number), and data content recorded in the log. For non-conflicting logs, redo can be performed in parallel, so that the storage pool can be filled with RAID strip writing.

[0118] The log conflict checking rule is: if the logical unit ID and data sector range corresponding to the log to be redone JN-A do not overlap with the logical unit ID and data sector range of all logs being redone, JN-A can start redoing, otherwise it is judged as a conflict, JN-A cannot be redone, and is suspended, the next log is read and checked according to the concurrency and full strip requirement.

[0119] (3) Modify the log state after partial or complete log redo, which can be modified by a fixed position (e.g., superblock) in the log space. Similarly, the information in the superblock is used to identify the logs that need to be redone when the system starts.

[0120] Through the above log entry comparison and sequential data replay, data write conflicts are effectively avoided, ensuring the coherence and accuracy of data flushing. This mechanism improves the data processing efficiency of the storage array during fault recovery, reduces the performance loss caused by overlapping writing, ensures data consistency, avoids "write hole" problems, and enhances the stability and data recovery ability of the storage system. The fault handling capability and data processing robustness of the storage array are enhanced.

[0121] In an exemplary embodiment, the method further comprises: in response to the received host request, writing the object data into the cache space; writing the object data into the log space and returning a write completion instruction; before the all-flash storage array fails due to power failure, in response to performing a flushing operation on part of the cached data in the object data, marking a log state of a target log entry corresponding to the part of the cached data in the log space as flushed.

[0122] When the all-flash storage array receives a write request from the host, the system first writes the object data into the cache space. The cache space temporarily stores the data so that subsequent aggregate write operations can be performed more efficiently.

[0123] Before the array fails completely or fails completely, because the current state controller is still in single control state, non-full stripe transparent write mode is prone to occur, resulting in multiple read and write operations and write amplification problems.

[0124] To avoid the above problems, in the present embodiment, the write log mode needs to be started to process the host write request flow when the array controller loses redundancy or loses BBU protection. For details, please refer to the flowchart as shown in Figure 4 .

[0125] S402, initialize the log space.

[0126] Before entering the write log mode, the storage array reserves a log space in the storage pool in advance. The size of the space matches the write cache of all controllers and has a certain degree of redundancy (between 0 and 2) to ensure the high reliability of the log. This step is completed when the system is running normally and redundancy exists, and prepares for subsequent write log operations.

[0127] S404, receive the write request and data.

[0128] The host sends a write request to the storage array, carrying the data to be written. The storage array receives these requests and temporarily stores the data in the cache.

[0129] S406, write the data into the cache space.

[0130] The data is temporarily stored in the cache to quickly respond to the write operation of the host and provide low-latency write services.

[0131] S408, record the write request and data itself to the log space.

[0132] The content written in the log space includes, but is not limited to, a logical unit ID, a data start sector number, a data length (a number of sectors), data content, a check code, a time, a log number, and the like. The check code is a hash value or a digest calculated by a hash algorithm for the log content, and is used to check the integrity and validity of the log content.

[0133] S410, the host is replied.

[0134] After the log is written, the host is replied to end the IO request. After the cache is written, the storage array immediately sends a write completion confirmation to the host, ensuring data reliability while not affecting the continuous operation of the host.

[0135] S412, the cache is written to the storage pool.

[0136] That is, the data in the cache space is flushed, and then the storage array flushes the data in the cache to the storage pool. In the flushing process, step S414 is performed.

[0137] S414, the data is written according to the BLOCK allocation principle, and tries to fill the RAID stripe.

[0138] The purpose of trying to fill the RAID stripe is to reduce write amplification and optimize storage performance.

[0139] S416, the log state is modified to be flushed.

[0140] After the flushing is completed, the storage array updates the state of the corresponding entry in the log space to "flushed", which helps to quickly identify which data has been safely written to the storage pool and which data needs to be redone during system restart or fault recovery.

[0141] It should be noted that after the system is restarted due to a fault, the log is checked, and the data that has not been flushed or has not been completely flushed is redone, that is, it is written to the storage pool again to ensure the integrity and consistency of the data. In the redone process, the system checks the integrity and concurrency of the log to avoid repeated writing or data confusion and ensure reliable data recovery.

[0142] The log space is used in a time sequence under normal circumstances, that is, the log with a smaller log number can be overwritten by new log content after the flushing is confirmed, realizing efficient space management and utilization.

[0143] Through the above process, the write log mode is enabled, ensuring that in the cache write-through mode, the data is persistently stored, the read-write amplification is minimized, and the system performance is optimized, and a reliable mechanism is provided for data recovery after a fault. Not only does it improve the overall data processing capacity of the system, but also ensures data security and storage efficiency, and has obvious advantages in high-concurrency and high-reliability storage applications.

[0144] In this embodiment, in order to improve the write performance of the storage system, another optional example is proposed, which is mainly used to dynamically adjust the RAID strip size in the single control running state or the BBU backup battery failure state, so that only smaller write concurrency is needed to fill the strip write.

[0145] Specifically, in response to cache protection failure of the all-flash storage array, the average write concurrency and the data block size of each write operation of the host request are monitored and counted; and based on the average write concurrency and the data block size, the striping parameters of at least one strip contained in the data block of the storage pool of the all-flash storage array are dynamically adjusted, wherein the striping parameters include strip width and strip block granularity, the strip width is used to indicate the number of hard disks constituting a strip, and the strip block granularity is used to represent the space size occupied by each data block in the strip when written to a single storage hard disk.

[0146] The implementation basis of this embodiment is that the storage space of the all-flash storage pool is allocated, used and recycled according to BLOCK, BLOCK will not be written again before recycling after sequential writing is completed, the RAID strip size of BLOCK space is determined when writing BLOCK each time, and according to the principle of full strip aggregation write, when the single controller or loses BBU protection, the small RAID strip is used in the cache transparent write mode to realize most or all full strip aggregation write, thereby reducing or eliminating the read-write amplification problem caused by non-full strip write.

[0147] In order to facilitate understanding, first, the relationship between the BLOCK distribution of the all-flash storage pool and the RAID strip and the hard disk is simply introduced.

[0148] (1) The hard disk is divided according to a fixed size, and each part is recorded as a hard disk segment.

[0149] (2) The BLOCK contains the hard disk segments of multiple hard disks, and each hard disk of the multiple hard disks constituting each BLOCK contains the same number of hard disk segments.

[0150] (3) The BLOCK uses the contained hard disk segments to build RAID, each strip is aligned with the boundary of the hard disk segment, and the type (RAID5 / RAID6) of the RAID, the strip width and the strip size are determined by the BLOCK. The RAID type, strip width and strip size can be different between different BLOCKs.

[0151] (4) The pool address PBA of the storage pool is composed of two parts: the BLOCK number and the address in the BLOCK. According to the address in the BLOCK, the RAID type, the strip width and the strip size of the BLOCK, the RAID strip number and the address corresponding to the PBA are determined.

[0152] When the cache protection function (e.g. BBU backup battery protection) of the all-flash storage array fails, the system automatically switches to a more conservative cache-through mode. The system monitors the write requests issued by the host in real time, and counts and analyzes the average write concurrency (host write qdepth, hwqd) and the data block size (host_write_block_size, hwbs) of each write operation issued by the host. This monitoring process provides the necessary data basis for subsequent striping parameter adjustment, to ensure that the storage pool can still efficiently process data in the cache-through mode.

[0153] According to the monitored average write concurrency and data block size, the storage array dynamically adjusts the striping parameters of the striping contained in the data blocks (BLOCK) in the storage pool. The striping parameters mentioned here include striping width and striping block granularity. The striping width reflects the number of hard disks that make up a striping, and determines the degree of data dispersion and read-write performance. The striping block granularity represents the size of the space occupied by each data block in a striping when written to a single storage hard disk, and directly affects the data aggregation write capability and the degree of read-write amplification.

[0154] It should be noted that in the cache non-write-through mode (e.g. controller redundancy or BBU health status), the BLOCK uses a larger striping, which is achieved by using a larger striping width or striping block, for example, RAID6, striping width 10, striping block 128KB, corresponding to a striping size of 1MB.

[0155] But in the cache-through mode (e.g. no controller redundancy or BBU failure), it is expected that the BLOCK uses a smaller striping, which is achieved by using a smaller striping width or striping block, for example, RAID6, striping width 10, striping block 4KB, corresponding to a striping size of 32KB; the specific striping width or striping block can be determined according to the write concurrency and block size issued by the host as described above.

[0156] In the case of keeping the striping width unchanged, by reducing the striping block granularity, each striping now only needs a smaller write concurrency to be full, reducing read-write amplification, reducing the write delay of the storage pool, and improving performance.

[0157] By reducing the strip block granularity (i.e. the strip block size), the storage array can also aggregate the write in full strip under the condition of insufficient host concurrent write, effectively avoiding the performance bottleneck caused by non-full strip write, such as read-write amplification problem. At the same time, since the strip width remains unchanged, the available space ratio of the storage pool is maintained, and the overall redundancy and data security of the system are not affected. Through this parameter adjustment, not only the ability of the storage array to handle sudden failures is improved, but also the data write efficiency and life management of the storage device under the cache transparent write mode are guaranteed.

[0158] In an exemplary embodiment, the above-mentioned dynamic adjustment of the striping parameters of at least one strip contained in the data block in the storage pool of the all-flash storage array based on the average write concurrency and the data block size includes: determining the product of the average write concurrency and the data block size as the adjusted striping parameter, wherein the adjusted striping parameter is used to reduce the read-write amplification caused by non-full strip data write.

[0159] When the cache protection of the all-flash storage array is invalid, the system automatically enters the cache transparent write mode, at this time the average write concurrency of the host write request and the data block size of each write become the key basis for adjusting the striping parameters of the storage pool. The system determines an adjusted striping parameter value by calculating the product of the two parameters. This parameter value will be directly used to guide the dynamic adjustment of the strip width and strip block granularity of the strip contained in the data block (BLOCK) in the storage pool, ensuring that the data write process can be carried out in full strip mode, thereby avoiding the read-write amplification problem caused by non-full strip write.

[0160] The above-mentioned average write concurrency issued by the host can but not limited to represent the average value of the number of "not completed" write commands issued by the host to the storage array at the same time within a unit time, that is, the average write I / O queue depth seen from the host side.

[0161] It should be noted that the RAID strip size corresponding to the BLOCK is set to be less than or equal to the product of the above two parameters (the product is called average write concurrency data amount), in principle, by only modifying the strip block size, the strip width is kept the same as in the non-transparent write mode, thereby maintaining the available space ratio of the storage pool.

[0162] The core target of the embodiment is to reduce the read-write amplification phenomenon caused by non-full stripe writing, while ensuring the continuity of data writing and efficient utilization of the storage pool. By calculating the product of the average write concurrency requested by the host and the data block size, and taking it as the reference basis for determining the adjusted striping parameter, the effective optimization of the storage pool stripe width and stripe block granularity is realized after the cache protection failure of the all-flash storage array. The dynamic adjustment mechanism reduces the occurrence of non-full stripe writing, directly reduces the read-write amplification, while maintaining the continuity of data writing and the stability of system performance, ensuring that the storage array can still provide low-latency and high-efficiency data processing services in the cache transparent write mode.

[0163] It can be known from the above description of each embodiment that the technical problem to be solved by the technical scheme of the application is to avoid using the traditional RAID5 or RAID6 non-full stripe transparent write mode after the write cache is converted to transparent write when the controller of the storage array fails or the BBU backup battery fails, to use a smaller write amplification mode, to obtain lower latency and higher performance, and at the same time, the data is still in transparent write mode to ensure reliability.

[0164] In order to solve the above problems, two solutions are proposed in the embodiments of the application. The first solution is to increase the cache write log function, save the data to the cache when processing the host write IO, and at the same time, write the data to the log space allocated on the storage pool. After completing the write log, the host is answered to end the IO request. Then the data in the cache is written to the storage pool according to the array redundancy or BBU backup battery health mode. Since it is not limited by the host write concurrency, the storage pool can be full of RAID5 or RAID6 stripe writing at a sufficient speed. The second solution is that the storage pool supports the function of dynamically adjusting the RAID stripe size in BLOCK granularity. In the single control running state or the BBU backup battery failure state, the RAID stripe size is adjusted to a small granularity (such as 4KB~32KB), so that only a smaller write concurrency is needed to fill the stripe writing.

[0165] It should be noted that when non-full stripe writing occurs, read-write amplification will increase sharply. For example, for RAID6 writing a block of data, 3 pieces of data need to be read and 3 pieces of data need to be written, which are 1 number block and 2 check blocks respectively. The read-write times are 6 times the upper write times. For RAID5 writing a block of data, 2 pieces of data need to be read and 2 pieces of data need to be written. Since it is synchronous writing, the delay is large, the storage performance is greatly affected, and the storage device write amplification is also large.

[0166] Moreover, there is a "write hole" problem in non-full stripe writing. That is, the same stripe is interrupted in the process of writing multiple stripe blocks, such as power failure, restart, etc., and the writing is incomplete. Especially if the check block is not completely written, the system cannot be recovered after restarting, which is easy to cause data damage.

[0167] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a necessary general hardware platform, and of course can also be realized by hardware, but in many cases the former is a better embodiment.

[0168] According to another aspect of the embodiments of the present application, a host request processing apparatus is also provided, a structural schematic diagram of the system is shown in Figure 6 The host request processing apparatus comprises the following modules:

[0169] The first write unit 602 is configured to write object data in a host request into a cache space in a full flash storage array and write the object data into a log space in response to cache protection invalidation of the full flash storage array, wherein the log space is a storage space in the full flash storage array independent of the cache space; the first processing unit 604 is configured to, in a process of writing the object data into the cache space and the log space and performing a destage operation on the object data in the cache space, determine remaining part data in the cache space which is not completed by the destage operation in response to fault repair of the full flash storage array based on a destage progress indicated by a target identifier in the log space, wherein the object data comprises the remaining part data, a value of the target identifier is updated each time the data in the cache space is completed by the destage operation, and the destage progress represents a ratio between a cumulative amount of data which is completed by the destage operation and a total amount of data in the object data; and the second processing unit 606 is configured to destage the remaining part data into a storage pool by replay processing on log entries in the log space.

[0170] The specific execution steps involved in various computing processes in the above modules and service configuration and abnormality early warning can refer to the description in the above embodiments, and will not be described here.

[0171] Obviously, the host request processing apparatus can be used to implement the host request processing method provided in the above embodiments, and the description has been made and will not be repeated. As used in the following, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0172] It should be noted that the first write unit 602 in this embodiment can be used to perform the above step S202, the first processing unit 604 in this embodiment can be used to perform the above step S204, and the second processing unit 606 in this embodiment can be used to perform the above step S206.

[0173] In an example embodiment, the first processing unit 604 comprises: a first obtaining module, configured to obtain the target identifier from the log space, wherein the target identifier is used to indicate a proportion between a quantity of the partial cache data completed with destaging in the object data and a total quantity of the object data before a power failure occurs in the all-flash storage array; and a first processing module, configured to determine data other than the partial cache data in the object data as the remaining part data.

[0174] In an example embodiment, the apparatus further comprises: a querying module, configured to query a header field of at least one log entry for storing the object data in the log space, wherein the header field is used to represent state information of whether the at least one log entry has been flushed; a second processing module, configured to determine a partial log entry with uncompleted flushing according to the header field of the at least one log entry; and a third processing module, configured to determine data in the partial log entry as the remaining part data.

[0175] In an example embodiment, the second processing unit 606 comprises: a fourth processing module, configured to determine a first group of log entries corresponding to the remaining part data in the log space; a first reading module, configured to sequentially read each log entry in the first group of log entries into a controller memory of the all-flash storage array based on a log number, wherein the controller is used to perform read-write management, cache synchronization and fault recovery on the object data; a first checking module, configured to check integrity of each log entry in the first group of log entries based on a check code contained in each log entry in the first group of log entries in the controller memory, wherein each log entry comprises the check code, the log number, a logical unit identifier, a data start sector number, a data length and data content; a rejection module, configured to determine at least part of the log entries with incomplete content according to a checking result, and reject the at least part of the log entries in the first group of log entries to obtain a second group of log entries; and a replay module, configured to destage the remaining part data to the storage pool by performing replay processing on the second group of log entries.

[0176] In an example embodiment, the replay module comprises: a first obtaining sub-module, configured to sequentially obtain each log entry from the second group of log entries as a current log entry based on the log number, wherein the current log entry contains current partial data in the remaining partial data; a second obtaining sub-module, configured to obtain a current logical unit identifier and a current data sector range of the current log entry; a third obtaining sub-module, configured to obtain other logical unit identifiers and other data sector ranges of other log entries that are currently being processed and not completed; a comparison sub-module, configured to compare the current logical unit identifier with the other logical unit identifiers, and compare the current data sector range with the other data sector ranges, to obtain a target comparison result; a first writing sub-module, configured to write the current partial data into a current stripe in a current logical storage unit in a case where the target comparison result indicates that the current logical unit identifier is inconsistent with the other logical unit identifiers, and the current data sector range does not overlap with the other data sector ranges; wherein the current stripe comprises a plurality of stripe blocks, and each of the plurality of stripe blocks corresponds to a continuous physical address of a storage hard disk; and a second writing sub-module, configured to write the current partial data in the current stripe into each continuous storage space of a plurality of storage hard disks based on a mapping relationship between the plurality of stripe blocks and physical addresses of the plurality of storage hard disks in the all-flash storage array, wherein the remaining partial data comprises the current partial data.

[0177] In an example embodiment, the device further comprises: a second writing unit, configured to write the object data into the cache space in response to the received host request; a returning unit, configured to write the object data into the log space and return a write completion instruction; and a third processing unit, configured to, before a power failure occurs in the all-flash storage array, in response to performing a flushing operation on part of the cached data in the object data, mark a log state of a target log entry corresponding to the part of the cached data in the log space as flushed.

[0178] In an example embodiment, the device further comprises: a monitoring unit, configured to, in response to cache protection failure of the all-flash storage array, monitor and count an average write concurrency degree of the host request and a data block size at each write operation; and an adjusting unit, configured to dynamically adjust a striping parameter of at least one stripe contained by a data block in a storage pool of the all-flash storage array based on the average write concurrency degree and the data block size, wherein the striping parameter comprises a stripe width and a stripe block granularity, the stripe width is used to indicate a number of hard disks constituting a stripe, and the stripe block granularity is used to represent a space size occupied by each data block in a stripe when written into a single storage hard disk.

[0179] In an example embodiment, the adjusting unit comprises a fifth processing module configured to determine a product of the average write concurrency and the data block size as an adjusted striping parameter, wherein the adjusted striping parameter is used to reduce read-write amplification caused by non-full striping data writing.

[0180] It should be noted that the above-mentioned modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: all the modules are located in the same processor; or the modules are located in different processors in any combination.

[0181] According to another aspect of the embodiments of the present application, an electronic device is provided, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned processing method embodiments.

[0182] According to another aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned processing method embodiments when running.

[0183] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0184] According to another aspect of the embodiments of the present application, a computer program product is provided, which comprises a computer program, and the computer program is configured to execute the steps in any of the above-mentioned processing method embodiments when running by a processor.

[0185] The embodiments of the present application further provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is configured to execute the steps in any of the above-mentioned processing method embodiments when running by a processor.

[0186] Those skilled in the art will further realize that the mere concepts, teachings, and embodiments described herein are merely meant to provide an enabling description of the claimed application. Accordingly, modifications and / or additions, other than those explicitly described herein, can be obvious to those skilled in the art in the light of this disclosure. The claimed application is intended to embrace all such modifications and / or additions.

[0187] The above has carried out the detailed introduction to the processing method of the host request provided by the application. The principle and implementation mode of the application are described by applying specific examples in the present article. The above description of the embodiments is only applicable to help understanding the method and its core idea of the application. It should be pointed out that, for the ordinary skilled in the art, some improvements and modifications can be made to the application without departing from the principle of the application. These improvements and modifications also fall within the protection scope of the claims of the application.

Claims

1. A method for processing a host request, comprising: in response to cache protection of an all-flash storage array being disabled, writing object data in the host request to a cache space in the all-flash storage array and writing the object data to a log space, wherein the log space is a storage space in the all-flash storage array independent of the cache space; in a process of writing the object data to the cache space and the log space being completed and performing a flushing operation on the object data in the cache space, in response to a failure recovery of the all-flash storage array, determining remaining part data in the cache space that is not flushed completely based on a flushing progress indicated by a target identifier in the log space, wherein the object data comprises the remaining part data, a value of the target identifier is updated each time data in the cache space is flushed completely, and the flushing progress represents a ratio between a cumulative amount of data that has been flushed completely and a total amount of data in the object data; and flushing the remaining part data to a storage pool by replaying log entries in the log space.

2. The method of claim 1, wherein: the determining the remaining part data in the cache space that is not flushed completely based on the flushing progress indicated by the target identifier in the log space comprises: obtaining the target identifier from the log space, wherein the target identifier is used to indicate a ratio between an amount of part cache data in the object data that has been flushed completely before a power failure of the all-flash storage array occurs and a total amount of the object data; and determining data in the object data other than the part cache data as the remaining part data.

3. The method of claim 1, wherein: the method further comprises: querying a header field of at least one log entry in the log space that stores the object data, wherein the header field is used to represent state information indicating whether the at least one log entry has been flushed; determining part log entries that have not been flushed based on the header field of the at least one log entry; and determining data in the part log entries as the remaining part data.

4. The method of claim 1, wherein: the flushing the remaining part data to the storage pool by replaying the log entries in the log space comprises: determining a first group of log entries in the log space corresponding to the remaining part data; reading each log entry in the first group of log entries to a controller memory of the all-flash storage array in sequence based on a log number, wherein the controller is used to perform read-write management, cache synchronization, and failure recovery on the object data; and checking integrity of each log entry in the first group of log entries based on a check code contained in each log entry in the first group of log entries in the controller memory, wherein each log entry contains the check code, the log number, a logical unit identifier, a data start sector number, a data length, and data content. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ determine at least part of the log entries with incomplete content according to the checking result, and eliminate the at least part of the log entries in the first group of log entries to obtain a second group of log entries; flush the remaining part of the data to the storage pool by replaying the second group of log entries.

5. The method of claim 4, wherein the flushing the remaining part of the data to the storage pool by replaying the second group of log entries comprises: obtaining each log entry from the second group of log entries as a current log entry in sequence based on the log number, wherein the current log entry contains a current part of the remaining part of the data; obtaining a current logical unit identifier and a current data sector range of the current log entry; obtaining other logical unit identifiers and other data sector ranges of other log entries that are currently being replayed and not completed; comparing the current logical unit identifier with the other logical unit identifiers and comparing the current data sector range with the other data sector ranges to obtain a target comparison result; in a case where the target comparison result indicates that the current logical unit identifier is inconsistent with the other logical unit identifiers and the current data sector range does not overlap with the other data sector ranges, writing the current part of the data into a current stripe in a current logical storage unit, wherein the current stripe includes a plurality of stripe blocks, and each of the plurality of stripe blocks corresponds to a continuous physical address of a storage hard disk. writing the current part of the data in the current stripe into each continuous storage space of a plurality of storage hard disks based on a mapping relationship between the plurality of stripe blocks and the physical addresses of the plurality of storage hard disks, wherein the remaining part of the data includes the current part of the data.

6. The method of claim 1, wherein the method further comprises: writing the object data into the cache space in response to the received host request; writing the object data into the log space and returning a write completion instruction; before a power failure occurs in the all-flash storage array, in response to performing a flushing operation on part of the cached data in the object data, marking a log state of a target log entry corresponding to the part of the cached data in the log space as flushed.

7. The method of any one of claims 1 to 6, wherein the method further comprises: in response to cache protection failure of the all-flash storage array, monitoring and counting an average write concurrency and a data block size at each write operation of the host request; based on the average write concurrency and the data block size, dynamically adjusting a striping parameter of at least one stripe contained in a data block in the storage pool of the all-flash storage array, wherein the striping parameter includes a stripe width and a stripe block granularity, the stripe width is used to indicate a number of hard disks constituting a stripe, and the stripe block granularity is used to represent a space size occupied by each data block in a stripe when written into a single storage hard disk. ​ ​ ​ 8. The method of claim 7, wherein dynamically adjusting the striping parameters of at least one stripe containing data blocks in the storage pool of the all-flash storage array based on the average write concurrency and the data block size comprises: determining a product between the average write concurrency and the data block size as an adjusted striping parameter, wherein the adjusted striping parameter is used to reduce read-write amplification caused by non-full stripe data writes. The computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the host request processing method according to any one of claims 1 to 8.

9. A computer-readable storage medium, characterized in that, comprises:

10. An electronic device, comprising: a memory for storing a computer program; a processor for implementing the steps of the host request processing method according to any one of claims 1 to 8 when executing the computer program. ​

Images