A log processing method, a log replay method, and a storage device

Abstract

The application discloses a log processing method, a log replay method and a storage device, relates to the technical field of storage systems, and comprises the following steps: first, logically splitting logical block addresses into regular address segments according to address ranges, and matching target logs of corresponding types for each segment, so that the address interval and data length corresponding to a single log are more controllable and regular, the probability of non-32-bit alignment of logs is reduced from the source, and the amount of address boundary calculation required in the power-on initialization stage is reduced; meanwhile, the target logs corresponding to each address segment are stored in the memory and then subjected to deduplication processing, the number of logs that actually need to be stored and loaded is reduced, repeated DDR read-write operations caused by too many logs are avoided, the occupation of the processing resources of a storage controller and the bandwidth of DDR is reduced, and finally, the power-on initialization time is reduced and the starting speed of the storage device is improved.

Description

Technical Field

[0001] This application relates to the field of storage system technology, and in particular to a log processing method, a log replay method, and a storage device. Background Technology

[0002] In existing storage protocol specifications, ATA8 The ACS2 specification defines the Trim command, the SCSI specification defines the UNMAP command, and the NVMe specification defines the Deallocate command. Essentially, all these commands are de-allocation operations that release storage space and mark invalid data. For consistency, the term "de-allocation commands" is commonly used in this field to refer to these functionally identical or similar protocol commands.

[0003] In existing storage systems, after performing a deallocation operation, the storage system generates a large number of allocation / deallocation logs. These logs are used to record key information such as logical block physical address updates and deallocation table state restoration, and are an important basis for storage system fault recovery and state traceability.

[0004] However, due to the storage format of allocation / deallocation logs, the starting address of the log is often not guaranteed to be 32-bit aligned, or the data length of the starting address may not be an integer multiple of 32 bits. Therefore, processing a single log entry during the storage device's power-on initialization requires numerous complex address boundary calculations and multiple DDR operations. This directly consumes the storage controller's processing resources and DDR bandwidth, increasing the power-on initialization time. Since DDR bandwidth resources are limited, the time spent processing a single log entry is long. When the number of log entries accumulates to a large amount, it directly extends the storage device's power-on initialization time, affecting the storage device's startup speed and user experience. Summary of the Invention

[0005] This application provides a log processing method, a log replay method, and a storage device to at least solve the problem in the prior art where, due to the non-alignment or non-integer multiple of the length of the allocated log address, a large number of DDR read / write operations and address boundary calculations are required during power-on initialization, which consumes DDR bandwidth and causes excessively long power-on time.

[0006] This application provides a log processing method, including: in response to an allocation command or a deallocation command, obtaining the logical block address of the allocation command or deallocation command; splitting the logical block address into different address segments according to the address range, and matching the target log of the corresponding type to the different address segments; storing the target log corresponding to each address segment into memory, and performing deduplication processing.

[0007] This application also provides a log replay method, which includes: in response to the storage device being powered on again after an abnormal power outage, splitting the logical block address of the allocation command or the deallocation command into different address segments according to the address range; and performing a log replay operation on the target log corresponding to each address segment according to the log type; wherein the log type includes a first type log and a second type log, the first type log is a log of the update allocation table type, and the second type log is a log of the allocation / deallocation type.

[0008] This application also provides a storage device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described log processing methods or log replay methods when executing the computer program.

[0009] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described log processing methods or log replay methods.

[0010] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described log processing methods or log replay methods.

[0011] This application avoids excessively large and irregular address spaces for single log entries by segmenting the logical block addresses corresponding to allocation / deallocation commands according to address ranges, thereby reducing misalignment issues caused by messy address spans. By matching different address segments with target logs of corresponding types, the address range and data length of single log entries become more regular, making it easier to meet alignment requirements. By storing the segmented logs in memory and performing deduplication, the total amount of logs that actually need to be processed is reduced, avoiding excessive log accumulation, thereby reducing the number of DDR operations, alleviating the occupation of controller processing resources and DDR bandwidth, and ultimately shortening the power-on initialization time. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a log processing method provided in an embodiment of this application; Figure 2 A schematic diagram illustrating address range segmentation and log matching provided in an embodiment of this application; Figure 3A schematic diagram illustrating a log storage process provided in an embodiment of this application; Figure 4 A schematic diagram illustrating another log storage process provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a log processing device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0015] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0016] To more clearly illustrate the embodiments of this application, the technical terms used in the embodiments will be briefly introduced below: In relevant embodiments, the invalid logical block table is a core data structure in a storage system (especially non-volatile storage devices) used to record the invalid status of logical blocks. Through the invalid logical block table, the storage system can accurately record key information such as the identifier, invalidation reason, and invalidation time of all logical blocks that cannot normally carry valid data due to data updates, deletions, or logical errors. It typically resides in memory or non-volatile storage media and works in conjunction with bad block management mechanisms to achieve precise control over invalid logical blocks. Its core function is to provide accurate invalid logical block location data for core operations such as garbage collection, logical address mapping updates, and wear leveling, preventing invalid logical blocks from being misallocated or misread / written, preventing data corruption caused by invalid data residue, reducing system performance losses caused by invalid block scanning, ensuring the accuracy, reliability, and operational efficiency of data storage in the storage system, and providing fundamental data support for the stable operation of the storage system.

[0017] The deallocate operation is a process by which a storage system, in response to instructions from the host or internal management module, releases a specified logical address range or storage resource. This operation severes the association between the logical address and the physical storage resource, updates the relevant allocation management table entries, marks the corresponding storage resource as idle, and ensures that the deallocated storage resource can be subsequently reallocated and used without resulting in the loss of unbacked-up valid data. Its core function is to achieve dynamic reclamation and efficient reuse of storage resources, reducing storage resource waste.

[0018] The Trim operation is an invalid data notification instruction sent by the host operating system to the non-volatile storage device. Upon receiving the instruction, the storage device marks data within the specified logical address range as invalid and performs batch cleanup of invalid data within this range when the system is idle, synchronously updating the status flags of the corresponding logical addresses. This operation does not affect the file system records on the host side. Its core function is to solve the performance degradation problem caused by the "erase before write" characteristic of non-volatile storage devices, enabling the storage device to accurately distinguish between valid and invalid data. During garbage collection, it only moves valid data, reducing unnecessary data rewrite operations and lowering write amplification. This improves garbage collection efficiency and the read / write performance of the storage device, reduces wear on the storage medium, and extends the lifespan of the storage device. Furthermore, by setting a bitmap to record the Trim operation status, it ensures that relevant information can be recovered after a power outage, improving system security.

[0019] The standard definition of Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) operations is as follows: Following the series of standards established by JEDEC (e.g., JESD79-4 for DDR4), it encompasses a series of standardized operations performed on DDR memory in a storage system, including addressing, data reading / writing, timing control, calibration, and status management. Its core characteristic is the ability to transmit data on both the rising and falling edges of the system clock, offering high bandwidth, low latency, and high stability. Its primary function is to act as a high-speed data cache and temporary storage medium for the storage system, providing high-speed data reading / writing support for core modules such as the storage controller and Flash Translation Layer (FTL). It caches I / O commands, FTL mapping tables, and temporary data, reducing the frequency of data interaction between the storage system and external storage media. This addresses the bottleneck of mismatch between storage media read / write speeds and system processing speeds, significantly improving the overall response speed and data processing efficiency of the storage system, making it one of the core supports for efficient storage system operation.

[0020] The standardized log records generated during the operation of the Flash Translation Layer (FTL) are defined as follows: When the FTL performs various operations such as address mapping updates, data read / write, garbage collection, wear leveling, allocation removal, and Trim operations, it synchronously records key data such as operation type, operation time, involved logical and physical addresses, operation results, and exception information. These log files are typically stored in a dedicated area of ​​non-volatile storage media. Their core function is to provide reliable evidence for FTL fault recovery, operation tracing, and status verification. When the storage system experiences power outages, abnormal restarts, or other failures, the system can quickly reconstruct the FTL mapping table by parsing the FTL logs, restoring the storage state before the failure and preventing data loss and address mapping errors. Simultaneously, the logs can be used to trace the execution process of various operations, facilitating the investigation of operational anomalies, optimizing FTL management strategies, ensuring the stability and reliability of FTL operation, and ultimately improving the fault tolerance and maintainability of the entire storage system, providing crucial protection for data security.

[0021] In the relevant implementation storage protocol specifications, ATA8 The ACS2 specification defines the Trim command, the SCSI specification defines the UNMAP command, and the NVMe specification defines the Deallocate command. Essentially, all these commands are de-allocation operations that release storage space and mark invalid data. For consistency, the term "de-allocation commands" is commonly used in this field to refer to these functionally identical or similar protocol commands.

[0022] In the storage system of the relevant embodiments, after the Trim operation and its equivalent deallocation operation are performed, subsequent write operations will frequently generate a large number of restore allocation logs used to update the physical address of logical blocks and synchronously modify the corresponding deallocation table status.

[0023] In the system power-on initialization process, each allocation / deallocation log requires three DDR operations, including one write operation and one DDR read-modify-write operation. These operations directly consume the storage controller's processing resources and DDR bandwidth, increasing the power-on initialization time. In related embodiments, due to the storage format of the allocation / deallocation logs, the starting address of the log is usually not guaranteed to be 32-bit aligned, and the data length is not necessarily an integer multiple of 32 bits. This necessitates splitting the non-aligned addresses during power-on recovery. Specifically, processing a single log requires the following steps: a DDR read-modify-write operation for the non-32-bit aligned portion of the starting address, a DDR batch write operation for the 32-bit aligned portion of the starting address with a length that is an integer multiple of 32 bits, and a DDR read-modify-write operation for the 32-bit aligned portion of the starting address but with a length less than 32 bits. Processing a single log requires multiple DDR interactions.

[0024] Due to the inherent bottleneck of DDR bandwidth, the processing time for large amounts of logs increases significantly, directly leading to excessively long power-on initialization times for storage devices. This prolonged power-on wait not only reduces the operational efficiency of the storage system but may also affect business continuity in scenarios with extremely high responsiveness requirements, such as data centers. Furthermore, frequent DDR operations increase system power consumption and exacerbate hardware wear and tear, becoming a key technical bottleneck restricting the performance optimization and efficient operation of non-volatile storage devices.

[0025] To address all or part of the aforementioned technical problems, this application provides a log processing method. This method indirectly alleviates the power-on initialization efficiency problem caused by irregular log formats by standardizing and simplifying the allocation / deallocation log generation process. During the log generation stage, the method first splits the logical block address into regular address segments according to address ranges and matches each segment with a corresponding type of target log. This makes the address range and data length of a single log entry more controllable and regular, reducing the probability of non-32-bit alignment in logs from the source and reducing the amount of address boundary calculations required during power-on initialization. Simultaneously, the method performs deduplication after storing the target logs corresponding to each address segment in memory, effectively reducing the actual number of logs that need to be stored and loaded. This avoids repeated DDR read / write operations due to excessive logs, thereby reducing the consumption of storage controller processing resources and DDR bandwidth, ultimately reducing power-on initialization time and improving the startup speed of the storage device.

[0026] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, an embodiment of this application provides a log processing method, which includes the following: S100: In response to an allocation command or a deallocation command, obtain the logical block address of the allocation command or the deallocation command.

[0028] The logical block address of the allocation command or deallocation command is the overall address range of the logical block to be operated on in this log when the allocation / deallocation log needs to be added during the operation of the storage device. It is the address range of the operation object corresponding to the allocation / deallocation log.

[0029] It is understandable that when a storage device performs a partitioning operation, these operations will be applied to a continuous logical block. The address range from the beginning to the end of this logical block being operated on is the address range of the logical block that needs to be split.

[0030] For example, if a storage device performs a Trim allocation operation, which requires processing 4KB logical blocks with addresses from 0x1004 to 0x1040, then the range of 0x1004 to 0x1040 is the address range of the 4KB logical blocks corresponding to the log to be allocated in this operation.

[0031] S101. Split the logical block address into different segments according to the address range, and match the corresponding type of target log for different address segments.

[0032] During the operation of the storage device, when the system needs to add allocation / deallocation logs, the logical block address of the allocation command or deallocation command is first split into different segments according to the address range, and the corresponding log type is matched for each segment.

[0033] In some embodiments, for address segments whose starting address is not aligned with the allocation table and whose length is less than the allocation table, a first type of log is matched; for address segments whose starting address is aligned with the allocation table and whose length is an integer multiple of the allocation table, a second type of log is matched; and for address segments whose starting address is aligned with the allocation table but whose length is less than the allocation table, a first type of log is matched.

[0034] In this embodiment of the application, the allocation table is 32-bit aligned as an example. Therefore, the first type of log can be a log of the 32-bit allocation table update type, and the second type of log can be a log of the allocation / deallocation type.

[0035] like Figure 2 As shown, for address segments whose starting address is not 32-bit aligned and whose length is less than 32 bits, add a 32-bit allocation table update type log; for address segments whose starting address is 32-bit aligned and whose length is an integer multiple of 32 bits, retain the original processing method and add a regular allocation / deallocation type log; for address segments whose starting address is 32-bit aligned but whose length is less than 32 bits, also add a 32-bit allocation table update type log.

[0036] By segmenting address ranges and matching logs in this way, the operation range of regular allocation / deallocation logs strictly meets the conditions of 32-bit alignment and a length that is a multiple of 32 bits. This segmented processing method makes the second type of log naturally adaptable to DDR batch operations. Smaller, non-standard address ranges that do not meet these conditions are handled by the newly added first type of log. This type of log is specifically designed for small-range, non-standard address operations and can be processed with only a single DDR write operation, replacing the multiple DDR read-modify-write operations performed on non-standard address ranges in the original technology. This directly reduces the number of DDR operations for processing a single address range and lowers the DDR bandwidth usage. All segments are matched with the corresponding log type processing logic according to rules, giving each address operation a dedicated and efficient log processing method. This avoids the complex multi-step DDR operations and address calculations performed on the entire mixed address range in the original technology, simplifies the overall logic of adding allocation / deallocation logs at runtime, and reduces the computational overhead in the log generation process.

[0037] In addition, the standardized design of segmented processing allows the storage device to replay allocation / deallocation logs during the power-on recovery phase without performing any address range segmentation or boundary calculations. It can directly perform DDR batch assignment operations on regular allocation / deallocation logs and perform a single DDR write operation on the 32-bit allocation table update logs. This reduces the amount of computation for replaying logs during power-on and the number of DDR operations, directly shortening the processing time of a single log entry and thus significantly reducing the power-on initialization time of the SSD.

[0038] The storage format of the Flash Translation Layer (FTL) log is shown in Table 1. In Table 1, logs with log type 5 are the first type of logs, and logs with log types 2 and 3 are the second type of logs.

[0039] Table 1

[0040] The FTL log is stored in a byte-based structure, divided into a byte offset 0 area and a byte offset 4 area. The byte offset 0 area is further divided into two bit fields: bits 31-29 are 3-bit log type identifiers used to distinguish different log operation types, and bits 28-0 are 29-bit FTL log offset bits used to identify the offset of the logical block to which each log belongs in the FTL stream log. The byte offset 4 area contains a fixed 4 bytes of business data, the specific content of which is determined by the value of the log type identifier bits; different log types correspond to different business data meanings.

[0041] It should be noted that the log storage format of the relevant embodiments is shown in Table 2: Table 2

[0042] The related embodiments define five log types from 0 to 4. Compared with the FTL log storage format in the related embodiments shown in Table 2, the log storage format shown in Table 1 of this application has deleted the log of type 1 (updating the physical address of the logical block and its corresponding de-allocation table status). This type originally stored the updated physical block address in the byte offset 4 area. After deletion, this type no longer participates in the recording and processing of FTL logs. The operations that originally required this type will be replaced by combinations of other log types.

[0043] For example, the original composite operation of "updating the logical block physical address and its corresponding allocation table status (restoring allocation)" which required Type 1 logs to be completed is split into two separate operations performed by Type 0 and Type 5 logs, which work together to complete the composite operation. Specifically, Type 0 logs are responsible for implementing the "updating the logical block physical address" operation of the original Type 1 logs, while Type 5 logs are responsible for implementing the "updating the corresponding allocation table status (restoring allocation)" operation of the original Type 1 logs. Together, they can fully cover the business functions of the original Type 1 logs. It is important to emphasize that processing a single Type 1 log entry originally required three DDR operations, while the split approach only requires one Type 0 and one Type 5 log entry, requiring only two DDR operations. This directly saves one DDR operation, reduces DDR bandwidth usage, and shortens the processing time per log entry.

[0044] Meanwhile, in the reserved value of the log type identifier bit, type 5 is started, defined as the 32-bit allocation table (UPDATE_ONE_TRIM_DW) log. The basic field framework of this type is consistent with the original type. Bit 28:0 at byte offset 0 is still the offset of the FTL stream log to which the logical block belongs. The 4 bytes of business data bits at byte offset 4 are specially stored to store the 32-bit allocation table content value containing the allocation table bit corresponding to the starting logical block, which is used to realize the direct update operation of the 32-bit allocation table.

[0045] The log storage format defined in this application does not change the original core framework. By adding or deleting log types and redefining the corresponding business data content, a special log type adapted to the 32-bit allocation table update has been added. At the same time, the original low-efficiency restore allocation type log has been removed, laying the format foundation for the standardization of allocation / deallocation log addresses.

[0046] S102. Store the target logs corresponding to each address segment into memory and perform deduplication.

[0047] In some embodiments, when performing step S102, the log is not directly written to the buffer. Instead, it is first determined whether there is a duplicate log in memory that is the same as the target log. If there is, deduplication is performed according to the logical block address range corresponding to the target log and the duplicate log. After deduplication, the target log is stored in memory.

[0048] Optionally, when the target log corresponding to the address segment is a first type log, it is determined whether there is a log to be added in memory; if there is no log to be added in memory, the first type log is used as the log to be added. Here, the log to be added refers to the log that is temporarily stored in memory and has not yet been written to the buffer between memory and non-volatile storage medium.

[0049] If a log to be added already exists in memory, it is determined whether the address range of the logical block to be operated on by the first type of log is the same as the address range of the logical block to be added log. If they are the same, the first type of log is discarded; if they are different, the first type of log is temporarily stored in memory as a log to be added.

[0050] Then, the logs to be added are processed. If they are the same, they are discarded; if they are different, the logs to be added are temporarily stored in memory.

[0051] It's understandable that when adding a new 32-bit allocation table type log, the process first checks if a temporary 32-bit allocation table type log exists in memory. If it doesn't, the new type log is added as a pending log and temporarily stored in memory. If it does exist, the logical address range of the new log is compared with that of the existing log. If their logical block address ranges are exactly the same, the existing log is discarded, and the new log is temporarily stored in memory. If their logical block address ranges are different, the existing log is written to the buffer, and the new log is temporarily stored in memory.

[0052] like Figure 3 As shown, when the system needs to add a log entry of type "Update 32-bit Allocation Table", it first enters a judgment node to check if a duplicate log of the same type already exists in memory. If no duplicate log exists, the new "Update 32-bit Allocation Table" log entry is directly added and temporarily stored in memory. If a duplicate log exists, the next judgment step is taken to compare the logical block address range to be operated on by the new log entry with the logical block address range of the existing duplicate log entry in memory. If the address ranges are exactly the same, the duplicate log entry in memory is deleted first, and then the new "Update 32-bit Allocation Table" log entry is temporarily stored in memory; if the address ranges are different, the new "Update 32-bit Allocation Table" log entry is directly added and temporarily stored in memory. Through precise duplicate detection and address range comparison, redundant logs with exactly the same address range are eliminated, avoiding the generation and temporary storage of duplicate logs, thereby reducing the number of subsequent log write buffers and flash memory, optimizing memory resource usage and log processing efficiency.

[0053] The aforementioned optional implementation uses a dual-determination approach: first determining whether a temporary log of the same type exists in memory, and then comparing whether the address ranges are consistent. This accurately identifies duplicate 32-bit allocation table update logs with identical addresses. By directly deleting the original duplicate logs and only temporarily storing the new logs, completely redundant log data is eliminated at the source, avoiding the generation, writing, and subsequent power-on replay processing of duplicate logs, reducing the total number of logs, and saving memory, buffer, and flash storage resources occupied by redundant logs. Secondly, for logs of the same type with different address ranges, the original log is written to the buffer first, and then the new log is temporarily stored. This ensures the reliable storage of the original log data while maintaining the state of temporarily storing one log of the same type in memory, avoiding the problem of excessive resource consumption caused by temporarily storing the same type of log in memory. Placing the duplicate determination and removal operations of updating the 32-bit allocation table log in the memory temporary storage stage avoids invalid DDR operations on duplicate logs during the power-on recovery stage, reducing the number of subsequent hardware operations and data calculations.

[0054] Optionally, if the log to be added already exists in memory, it is determined whether the address range of the logical block to be operated on by the second type of log includes the address range of the logical block to be added. If it does, the log to be added is discarded, and the second type of log is added to memory; if it does not, the second type of log is added to memory directly.

[0055] It is understandable that when adding a new allocation / deallocation type log, the system first checks whether a temporary allocation / deallocation type log exists in memory. If it exists, and the logical block address range of the new log to be added includes the logical block address range of the original log, then the original log is discarded and the new log is temporarily stored in memory. If it does not exist, or the logical block address range of the new log does not include the logical block address range of the original log, then the new log is added, and the original log continues to be temporarily stored in memory.

[0056] like Figure 4As shown, when the system needs to add a "allocate or deallocate" log entry, it first enters a judgment node to check if a duplicate log of the same type already exists in memory. If no duplicate log exists, the new "allocate or deallocate" log entry is directly added and temporarily stored in memory. If a duplicate log exists, the system proceeds to the next judgment step, comparing the address range of the logical block to be operated on by the new log entry to see if it completely includes the address range of the existing duplicate log entry in memory. If the address range of the new log entry completely includes the address range of the original duplicate log entry, the duplicate log entry in memory is deleted first, and then the new "allocate or deallocate" log entry is temporarily stored in memory; if the address range of the new log entry does not include the address range of the original duplicate log entry, the new "allocate or deallocate" log entry is directly added and temporarily stored in memory. Through precise duplicate detection and address range inclusion comparison, redundant logs with overlapping address ranges are eliminated, avoiding duplicate log recordings in the same address range. This reduces the number of subsequent log write buffers and flash memory, optimizes memory resource usage and log processing efficiency, and ensures the integrity of log data in different address ranges.

[0057] In the above optional implementation, when adding a new allocation / deallocation log, if a temporary log of the same type exists in memory and the new log address range can completely encompass the original log, the original temporary log is discarded, and only the new log with a wider coverage is retained and processed. This design can eliminate redundant logs with overlapping address ranges, avoid duplicate log recording, writing, and power-on replay processing for the same address range, and eliminate the need for subsequent DDR operations and addressing stages for redundant logs, reducing the ineffective occupation of DDR bandwidth, buffers, and flash memory storage resources. If the new log address range cannot encompass the original temporary log, the original log is retained and the new log is added directly. This ensures the integrity of log data in different address ranges, avoids log loss due to non-overlapping address ranges, and ensures accurate log recording for allocation / deallocation operations in different address ranges. Furthermore, this decision-making logic places the deduplication operation of allocation / deallocation type logs in the memory temporary storage stage, eliminating the need for secondary processing after the logs are written to the buffer or flash memory. This simplifies the runtime log management process. Simultaneously, it writes valid, non-redundant logs to the buffer and finally flushes them to flash memory, improving the efficiency of buffer writing and flash memory flushing. The above embodiment designs memory temporary storage and dynamic deduplication rules for newly added 32-bit allocation table update type logs. By temporarily storing this type of log and dynamically processing it at runtime based on the type and address range of the new log, this type of log is not directly written to the buffer but is temporarily stored in memory. The core purpose of this temporary storage is to wait for the generation of subsequent new logs. If the operation address range of the subsequently generated new logs overlaps with that of the temporarily stored logs, deduplication can be performed to eliminate redundant logs, avoiding the generation and writing of redundant logs and further optimizing log processing efficiency.

[0058] After all log addition operations are completed, if there are still unprocessed 32-bit allocation table type logs in memory, the logs are first written to the buffer. The buffer follows the rule of full write or power failure trigger. When the number of logs in the buffer reaches the capacity limit, or when the storage device loses power, all logs stored in the buffer will be written to flash memory to ensure the reliable storage of temporary logs.

[0059] Compared to related technologies, in the scenarios of sequential write on an empty disk and sequential write after deallocation of the entire disk, the total amount of logs actually added in this application does not increase significantly, so it will not affect the sequential write performance of the storage device. In the scenario where a small-scale deallocation operation and write operation are mixed, the number of logs will increase to a certain extent, but the increase is small and will not have a significant negative impact on write performance.

[0060] In some embodiments, in response to a power outage, it is first determined whether there is a log to be added in memory. If there is a log to be added, and the log to be added is a second type of log, then the second type of log is not directly written to the non-volatile storage medium, but is still temporarily stored in memory. When a new log is subsequently generated in response to an allocation command or a deallocation command, the logical block address range of the new log is compared with that of the second type of log temporarily stored in memory. If there is an overlap, deduplication is performed.

[0061] Specifically, when a power outage occurs in the storage device, the status of the logs in memory is first assessed to identify logs that are temporarily stored but not yet written to the buffer between memory and non-volatile storage media. If such logs exist and belong to the second type of allocation / deallocation logs, they are not directly written to the non-volatile storage media; instead, they are temporarily stored in memory. After power outage recovery, when a new allocation or deallocation command is received and a corresponding new log is generated, the logical block address range of the new log is compared with the address range of the original second-type logs temporarily stored in memory. If the address ranges overlap, deduplication is performed on the overlapping logs, thereby merging and optimizing the logs. This embodiment avoids frequently writing unbuffered allocation / deallocation logs to non-volatile storage media during power outages, reducing I / O operations and storage write overhead during power outages, lowering the risk of data write failures or media damage during power outages, and improving the reliability of power outage protection. Secondly, by temporarily storing the second type of log in memory and performing address range overlap judgment and deduplication on newly generated logs of the same type, the number of redundant logs can be effectively reduced, making the log structure more concise and regular. This alleviates the problems of complex address boundary calculations and frequent DDR operations during power-on initialization caused by messy and excessive logs, reducing the processing pressure on the controller and the DDR bandwidth usage. At the same time, the overall log volume decreases after deduplication, which also reduces the storage space occupied by non-volatile storage media. Furthermore, the regularized logs are more likely to meet address and length alignment requirements, further shortening the power-on initialization time and improving the startup efficiency and operational stability of the storage device.

[0062] In some embodiments, the target log in the buffer is written to flash memory. To perform a power-off and power-on operation, first disconnect the device's main power supply, wait at least 30 seconds to ensure residual charge is released, then reconnect the power and complete the system startup.

[0063] like Figure 5 As shown in the figure, this application provides a log replay method, which includes the following steps: S501. In response to the storage device being powered on again after an abnormal power outage, the logical block address of the allocation command or deallocation command is split into different address segments according to the address range, and the corresponding target log is matched for each address segment.

[0064] S502. Perform log replay operation on the target logs corresponding to each address segment according to the log type.

[0065] The log types include a first type of log and a second type of log. The first type of log is a log for updating the allocation table, and the second type of log is a log for allocating / deallocating.

[0066] This log replay method optimizes log recovery efficiency after abnormal power outages by segmenting allocation / deallocation logs and replaying them differently according to their type. When the device is powered on again, the logical block addresses corresponding to allocation or deallocation commands are split into regular address segments according to address ranges. This makes the address range and length of a single log entry more standardized, reducing 32-bit alignment issues caused by address irregularities and decreasing the amount of address boundary calculations and DDR operations during replay. Simultaneously, replay operations are performed separately for the first type of logs updating the allocation table and the second type of logs (allocation / deallocation), achieving categorized and orderly log processing. This avoids logical confusion and redundant verification caused by replaying different types of logs together, improving the clarity and execution efficiency of the replay process. Overall, this reduces the resource consumption of the storage controller and DDR bandwidth during replay, accelerates log replay and system state recovery, shortens power-on initialization time, and improves the startup speed and operational stability of the storage device after power failure recovery.

[0067] After the storage device is powered on, it is necessary to restore the normal data state of the FTL and invalid logical tables by replaying the previously recorded FTL stream logs. This application has made corresponding modifications to the replay operation for different log types, with different simplified replay methods for different types of logs.

[0068] Ensure that the 4k logical block address range operated on by the allocation / deallocation log meets the following conditions: the starting address is 32-bit aligned and the length is an integer multiple of 32, simplifying the logic of power-on processing of allocation / deallocation logs.

[0069] In some embodiments, for the first type of log, a single write operation is performed on the double-rate memory, overwriting the current allocation table.

[0070] For Type 1 logs (updating 32-bit allocation tables), a single DDR write operation is performed to directly overwrite the current allocation table. This approach adapts to the characteristics of this type of log, which involves small-scale, non-standard address operations. It replaces the multiple DDR read-modify-write operations for the same address type found in previous technologies, reducing the number of DDR operations from 2-4 to 1, thus reducing DDR bandwidth usage. It also eliminates complex address splitting and calculation steps, improving the processing efficiency of a single Type 1 log entry. Furthermore, the single-address overwrite operation for Type 1 logs ensures accurate updates to the allocation table data.

[0071] For the second type of log in the buffer, if the second type of log is an allocation log, a batch zeroing operation is performed on the double-speed memory, and the Inverse Lookup Block Table (ILBT) is updated, setting the invalid logical table area corresponding to the second type of log in the ILBT to 0. If the second type of log is a deallocation log, a batch 1-assignment operation is performed on the double-speed memory, and the ILBT is updated, setting the invalid logical table area corresponding to the second type of log in the ILBT to 1. The ILBT is used to check whether the logical block address markers recorded in the FTL are valid. A value of 0 indicates that the logical block address marker is valid, meaning that the logical block address has been allocated a physical block address, and the physical block address is in the allocated state; a value of 1 indicates that the logical block address marker is invalid, meaning that the logical block address has not been allocated a physical block address, and the physical block address is in the unallocated state.

[0072] For the allocation log, there is no need to perform address alignment and splitting operations. Instead, perform a batch DDR assignment of 0 and set the invalid logical table area corresponding to the log to 0. For the deallocation log, perform a batch DDR assignment of 1 and set the invalid logical table area corresponding to the log to 1.

[0073] For the second type of logs (allocation / deallocation type logs), differentiated DDR batch operations are performed based on log attributes. Allocation logs undergo batch zeroing, and deallocation logs undergo batch assignment of 1s. This eliminates the need for any address boundary range segmentation calculations, leveraging the efficiency of DDR batch operations. It avoids the multi-step DDR operations and numerous address calculations caused by address misalignment and non-integer length multiples in related embodiments when processing the second type of logs. This reduces 2-4 DDR operations and eliminates time-consuming boundary calculations, almost halving the processing time for a single second type of log. The batch assignment operation for the second type of logs also ensures the rapid setting of invalid logical block table regions.

[0074] The above embodiments, together with the 32-bit alignment and deduplication of logs at runtime, form a complete technical closed loop. The standardized processing at runtime eliminates the need for additional adaptation calculations during the DDR operation at power-on, allowing for direct execution of single or batch operations. This makes the hardware operation process for log replay simpler and more efficient, avoiding unnecessary hardware scheduling and data processing overhead.

[0075] Referring to Table 1, after a power-on reboot, log entries of type 0 are directly overwritten in the current flash translation layer table. For log entries of type 2 (allocation logs), a batch DDR assignment of 1 is performed, setting the corresponding invalid logical block table area to 1. For log entries of type 3 (allocation logs), a batch DDR assignment of 0 is performed, setting the corresponding invalid logical table area to 0. For log entries of type 4, the corresponding FTL is cleared. For log entries of type 5 (allocation table update logs), a single DDR write operation is performed, overwriting the current allocation table.

[0076] The replay operations for Type 0 and Type 4 logs retain their original logic. Type 0 logs directly overwrite the current FTL table entries, while Type 4 logs directly clear their corresponding FTL regions. For Type 2 and Type 3 allocation / deallocation logs, there is no need to perform cumbersome address boundary and range segmentation calculations as in related embodiments. DDR batch assignment operations can be performed directly. Type 2 logs will set the corresponding invalid logical block table region to "1", and Type 3 logs will set the corresponding region to "0". The newly added Type 5 (updating the 32-bit allocation table) log only requires one DDR write operation to directly overwrite the current allocation table and complete the replay processing of this log.

[0077] The reason why power-on replay can save a lot of address boundary calculations is that during the operation of the storage device, the allocation / deallocation log has already been 32-bit address aligned and standardized, so that the operation range of this type of log meets the requirements of 32-bit alignment and length being an integer multiple of 32 bits. There is no need to do related calculations again during the power-on phase. Because these time-consuming calculation steps are eliminated, the overall log replay time is shortened, thereby improving the efficiency of power-on recovery of the storage device.

[0078] In some embodiments, for the second type of log, the address range length of the second type of log is identified. If the address range length is less than a preset value, an operation is performed on the double-rate memory. If the address range length is greater than or equal to the preset value, the address boundary calculation is removed, and only the double-rate memory operation is performed.

[0079] Since DDR batch operations require the amount of data to be an integer multiple of 32 bits, and the DDR address to be operated on must also meet the 32-bit alignment condition, the length of the allocation / deallocation log to be operated on is directly used as the basis for dividing small-range and large-range logs.

[0080] When processing the second type of log, first identify its corresponding address range length and compare it with a 32-bit preset value. If the address range length of the log is less than the preset value, then perform an operation on the double-rate memory directly. If the address range length of the log is greater than or equal to the preset value, then no additional address boundary calculation is performed, and the corresponding double-rate memory operation is directly executed.

[0081] Compared to related technologies, this application simplifies DDR operations for processing allocation / deallocation logs of different sizes. For small-scale allocation / deallocation logs with an operation length of less than 32 bits, the number of DDR operations required to process a single log entry is reduced from 2-4 to 1. For large-scale allocation / deallocation logs with an operation length greater than 32 bits, processing a single log entry not only reduces the number of DDR operations by 2-4, but also eliminates a large amount of tedious address boundary calculation work in related technologies. Because of the reduction in the number of DDR operations and the omission of address boundary calculations, the processing time for a single allocation / deallocation log entry in this application is almost halved. Furthermore, when the number of log entries is large, the overall power-on initialization time is also shortened.

[0082] Test data shows that the 4TB disk lost power abnormally after 10 minutes of trimwrite operation on 128k base. The power-on initialization time of existing technology is about 30 seconds, while this application only requires about 10 seconds. At the same time, the processing time of a single log is almost halved, the number of DDR operations is reduced, and the address boundary calculation step is eliminated.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0084] like Figure 6 As shown, embodiments of this application also provide a storage device, including a memory 601 and a processor 602. The memory 601 stores a computer program, and the processor 602 is configured to run the computer program to perform the steps in any of the above-described log processing method embodiments.

[0085] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described log processing method embodiments at runtime.

[0086] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0087] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described log processing method embodiments.

[0088] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described log processing method embodiments.

[0089] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] The log processing method, log replay method, and storage device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A log processing method characterized by, include: In response to an allocation command or a deallocation command, obtain the logical block address of the allocation command or the deallocation command; The logical block address is split into different address segments according to the address range, and the corresponding target log is matched for each address segment; the target log corresponding to each address segment is stored in memory and deduplicated.

2. The method of claim 1, wherein, The step of splitting the logical block address into different address segments according to the address range and matching the corresponding type of target log to the different address segments includes: For address segments within the address range whose starting address is not aligned with the allocated table bit and whose length is insufficient for the allocated table bit, match them with the first type of log. For address segments within the address range whose starting address is aligned with the allocation table bit and whose length is an integer multiple of the allocation table bit, a second type of log is matched for them; For address segments within the address range whose starting address is aligned with the allocation table bit but whose length is insufficient for the allocation table bit, match them with the first type of log. The first type of log is the log for updating the allocation table, and the second type of log is the log for allocating / deallocating.

3. The method of claim 2, wherein, The step of storing the target logs corresponding to each address segment into memory includes: Determine if a duplicate log identical to the target log exists in memory; If duplicates exist, deduplication is performed based on the target log and the logical block address range corresponding to the duplicate log. After deduplication, the log is stored in memory.

4. The method according to claim 3, characterized in that, If such a log exists, then deduplication is performed based on the target log and the logical block address range corresponding to the duplicate log. After deduplication, the log is stored in memory, including: When the target log corresponding to the address segment is the first type of log, determine whether there is a log to be added in the memory; If the log to be added does not exist in memory, then the first type of log will be used as the log to be added. If the log to be added already exists in memory, determine whether the address range of the logical block to be operated on by the first type of log is the same as the address range of the logical block to be added. If they are the same, discard the first type of log; If they are different, the first type of log will be temporarily stored in memory as the log to be added; The logs to be added are processed; if they are the same, they are discarded; if they are different, the logs to be added are temporarily stored in memory. The log to be added is a log that is temporarily stored in memory and has not yet been written to a buffer between memory and non-volatile storage medium.

5. The method of claim 3, wherein, If such a log exists, then deduplication is performed based on the logical block address range of the target log and the duplicate log. After deduplication, the log is stored in memory, including: If the log to be added already exists in memory, then determine whether the address range of the logical block to be operated on for the second type of log includes the address range of the logical block to be added. If it is included, discard the log to be added and add the second type of log to memory; If not included, the second type of log is added directly to memory.

6. The method according to claim 4 or 5, characterized in that, The method further includes: In response to a power outage, determine whether there are logs to be added in the memory; If there is a log to be added, and the log to be added is a second type of log, then the second type of log will not be directly written to the non-volatile storage medium, but will still be temporarily stored in memory; When a new log is generated in response to an allocation command or a deallocation command, the logical block address range of the new log is compared with that of the second type of log temporarily stored in memory. If there is an overlap, deduplication is performed.

7. A log replay method characterized by, include: In response to a storage device power failure and subsequent power restoration, the logical block address of the allocation command or deallocation command is split into different address segments according to the address range, and the corresponding target log is matched for different address segments. For the target logs corresponding to each address segment, perform log replay operations according to the log type; The log types include a first type of log and a second type of log. The first type of log is a log for updating the allocation table, and the second type of log is a log for allocating / deallocating.

8. The method of claim 7, wherein, The step of performing log replay operations on the target logs corresponding to each address segment according to the log type includes: For the first type of log, a single write operation is performed on the double-rate memory; For the second type of log, if the second type of log is an allocation log, then perform a batch zeroing operation on the double-rate memory and update the reverse lookup block table, setting the invalid logical table area corresponding to the second type of log in the reverse lookup block table to 0; If the second type of log is a de-allocation log, then perform a batch assignment operation of 1 on the dual-speed multiplier memory, update the reverse lookup block table, and set the invalid logical table area corresponding to the second type of log in the reverse lookup block table to 1; The reverse lookup block table is used to mark whether each logical block address recorded in the flash translation layer (FTL) table is valid. A value of 0 represents valid, indicating that the logical block address has been allocated a physical block address and the physical block address is in the allocated state. A value of 1 represents invalid, indicating that the logical block address has not been allocated a physical block address and the physical block address is in the unallocated state.

9. The method of claim 8, wherein, For the second type of log, the method further includes: Identify the address range length of the second type of log; if the address range length is less than a preset value, perform an operation on the double-rate memory. If the length of the address range is greater than or equal to the preset value, then address boundary calculations are removed, and only double-rate memory operations are performed.

10. A storage device, characterized in that, include: The control unit and the non-volatile memory chip, the control unit performing the steps of the log processing method according to any one of claims 1 to 6, or the steps of the log replay method according to any one of claims 7 to 9.

Images