A physical fragment processing method and related device

Abstract

The application discloses a physical fragment processing method to solve the problems of the existing technology, such as the data physical distribution discretization leading to the decrease of sequential reading throughput bandwidth, the amplification of IO time delay jitter in the scene with high fragmentation, and the increase of tail delay of reading requests. The method comprises the following steps: determining at least one fragment obtained by dividing a logical block address (LBA) interval; if the discrete condition of a physical block address (PBA) mapped by the fragment meets a preset discrete condition, performing the following steps on the fragment: reading data from the PBA mapped by the fragment; and sequentially writing the data into a target PBA according to the order of the LBA corresponding to the data.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a physical fragmentation processing method and related equipment. Background Technology

[0002] During long-term service, three types of underlying firmware operating mechanisms can cause physical fragmentation (discretionary physical distribution) of user data in the flash memory physical storage space. The specific causes are broken down as follows: 1. Off-site update: When the host initiates a data overwrite request for a specified logical block address (LBA), the SSD will not overwrite the data at the original physical block address (PBA). Instead, it will write the new data to the new PBA and then synchronously refresh the logical to physical address mapping table (Logical to Physical Mapping, or L2P for short, which is used to record the correspondence between LBA and PBA). This results in data belonging to the same continuous LBA range being scattered and stored in multiple independent physical flash memory blocks, creating basic physical fragmentation.

[0003] 2. Garbage Collection (GC): GC operations move data to new blocks in the order of valid data in the physical block, rather than in LBA order, resulting in data still being distributed out of order on the physical address after GC.

[0004] 3. Wear leveling: To balance the number of erases and writes across physical blocks, the Flash Translation Layer (FTL, the software layer in SSD firmware responsible for LBA to PBA mapping management) actively breaks up the physical locations of cold and hot data, and rotates frequently rewritten data to idle physical blocks with lower erase and write cycles, thus leveling out the wear difference across all flash blocks. This scheduling strategy further breaks the physical space continuity corresponding to consecutive logical addresses, amplifying the degree of physical fragmentation.

[0005] The aforementioned discretization of physical data distribution leads to the following: When data fragmentation at the physical level intensifies, when the host issues a sequential read IO request to a continuous LBA range, the SSD controller cannot complete the data retrieval through a single continuous flash array addressing. Instead, it needs to issue independent NAND read commands to multiple physically separated flash blocks (resulting in read amplification). This results in a decrease in sequential read throughput bandwidth, and in scenarios with high fragmentation, it also amplifies IO latency jitter and increases the tail latency of read requests. Summary of the Invention

[0006] This application provides a physical fragmentation processing method and related equipment to solve the problems of decreased sequential read throughput bandwidth caused by the discretization of data physical distribution, amplified IO latency jitter in scenarios with high fragmentation, and increased tail latency of read requests.

[0007] The embodiments of this application adopt the following technical solutions: A physical fragmentation processing method includes: determining at least one fragment obtained by dividing a logical block address (LBA) interval; if the discreteness of the physical block addresses (PBAs) mapped by the fragment meets a preset discrete condition, then performing the following for the fragment: reading data from the PBA mapped by the fragment; and sorting and writing the data into a target PBA according to the order of the LBAs corresponding to the data.

[0008] A storage controller for performing the physical fragmentation processing method described above.

[0009] A storage device includes: a memory and a storage controller, wherein the memory is used to store a computer program; and the storage controller is coupled to the memory and is used to execute the computer program stored in the memory to perform the method described above.

[0010] A physical fragmentation processing method includes: acquiring address information and fragmentation specification information of the Logical Block Address (LBA) range of the physical fragment to be reorganized; sending the address information and fragmentation specification information to a storage device to trigger the storage device to perform the following operations: determining at least one fragment obtained by dividing the LBA range; if the discreteness of the PBA mapped by the fragment meets a preset discrete condition, then performing the following operations for the fragment: reading data from the PBA mapped by the fragment; and sorting and writing the data into a target PBA according to the order of the LBAs corresponding to the data.

[0011] A physical fragmentation processing apparatus includes: an acquisition unit for acquiring address information and fragmentation specification information of the Logical Block Address (LBA) interval of the physical fragment to be processed; and a sending unit for sending the address information and fragmentation specification information to a storage device to trigger the storage device to perform the following operations: determining at least one fragment obtained by dividing the LBA interval; if the discreteness of the PBA mapped by the fragment meets a preset discrete condition, then performing the following operations for the fragment: reading data from the PBA mapped by the fragment; and sorting and writing the data into a target PBA according to the order of the LBAs corresponding to the data.

[0012] A computing device includes: a memory and a processor, wherein the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program stored in the memory to perform the method described above.

[0013] A computer-readable storage medium storing a computer program that, when executed by a computer, enables the implementation of the above-described method.

[0014] A computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described above.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: By first dividing the LBA interval into fragments and selecting fragments whose PBA dispersion meets the preset conditions, data on the scattered PBAs is only collected in these fragments, and then written to the target PBA in LBA order. This makes logically continuous data within the fragment correspond to continuous physical addresses. When the host initiates a sequential read, it does not need to access multiple non-contiguous physical blocks, reducing additional NAND read commands and read amplification. This solves the problems of decreased sequential read throughput bandwidth, amplified IO latency jitter, and increased read request tail latency caused by the discretization of physical data distribution. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating the specific implementation of a physical debris processing method provided in this application embodiment; Figure 2a This is a schematic diagram of a system architecture for applying the physical fragmentation processing method provided in the embodiments of this application; Figure 2b This is a schematic diagram illustrating the principle of discreteness calculation used in the embodiments of this application; Figure 2c This is a flowchart illustrating the process of organizing physical debris in an embodiment of this application; Figure 2d This is a schematic diagram illustrating the data migration and organization process performed by the data migration execution module in this embodiment of the application. Figure 3 This is a schematic diagram of the specific structure of a storage controller provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0019] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0020] Example 1 Embodiment 1 of this application provides a physical fragmentation processing method to solve the problems of decreased sequential read throughput bandwidth caused by the discretization of data physical distribution, amplified IO latency jitter in scenarios with high fragmentation, and increased tail latency of read requests.

[0021] Different steps of this method can be implemented by the same execution entity or by different execution entities. This application does not limit which execution entity is used to implement the method.

[0022] Furthermore, the embodiments of this application do not limit the execution order of different steps. When using the method provided in the embodiments of this application, the execution order of different steps can be adjusted according to actual needs.

[0023] The following describes the method provided in this application embodiment in detail, taking the storage controller in the storage device as the execution subject of this method.

[0024] In this embodiment of the application, the storage device may be an SSD (such as an enterprise-grade SSD, a consumer-grade SSD, a RAID or array-based SSD, etc.), a client solid-state drive (cSSD), an embedded solid-state drive (eSSD), or universal flash storage (UFS). This embodiment of the application does not limit the specific type of storage device.

[0025] like Figure 1 The diagram shown is a flowchart illustrating a physical debris processing method provided in this application, comprising the following steps: Step 11: Determine at least one segment obtained by partitioning the LBA interval; In one alternative implementation, the LBA interval can refer to a continuous LBA range, which is the global processing boundary of this physical debris straightening task and is the highest-level processing unit.

[0026] In one optional implementation, an LBA interval can be defined by a starting LBA and a total interval length, containing a continuous, ordered logical storage unit. In a specific example, the LBA interval starts at address 0x100000 and has a length of 0x100000 LBAs, corresponding to the complete LBA set: 0x100000~0x1FFFFF. For this LBA interval, in a specific example, assuming a fragment size of 0x20000 LBAs, a single fragment of this LBA interval corresponds to 64MB of data.

[0027] In this embodiment of the application, the “slicing” refers to the smallest independent regularized unit obtained by equally dividing (or unequally dividing) the LBA interval.

[0028] In an optional implementation, the method provided in this application embodiment allows the storage controller to determine the at least one fragment in the following manner: Receive the parameter submission command sent by the host; By parsing the parameter submission command, the address information of the LBA range (e.g., LBA range starting address 0x1000, range length 8 LBAs) and fragmentation specification information contained in the parameter submission command can be obtained. Based on the address information of the LBA interval and the information of the fragmentation rules, the LBA interval is divided to obtain at least one fragment.

[0029] In a specific example, the information regarding the sharding specification may include, for example, the number of LBAs contained in a single shard. Based on this information, the same LBA interval will be uniformly divided into several shards. A shard is the smallest execution unit for performing discreteness calculations and data migration in this embodiment of the application.

[0030] In another specific example, the fragmentation specification information may include, for example, fragment size = 0x20000 LBAs (corresponding to 64MB of data). Based on this information, for an LBA interval starting at address 0x100000 and with a total length of 0x100000 LBAs (corresponding to the complete LBA set 0x100000~0x1FFFFF), it can be evenly divided into 5 fragments: Fragment 0: 0x100000~0x11FFFF; Fragment 1: 0x120000~0x13FFFF; Fragment 2: 0x140000~0x15FFFF; Fragment 3: 0x160000~0x17FFFF; Fragment 4: 0x180000~0x19FFFF; Fragment 5: 0x1A0000~0x1FFFFF.

[0031] In one alternative implementation, all shards will be traversed sequentially for discreteness calculation and data migration; in another alternative implementation, at least one, but not all, of the resulting shards will be subjected to discreteness calculation and data migration.

[0032] In one alternative implementation, besides partitioning based on parameter submission commands, partitioning can also be achieved using one of the following methods: Method 1: When the preset physical fragmentation period is detected, determine at least one fragment obtained by dividing the logical block address LBA range.

[0033] In a specific example, assuming the physical defragmentation cycle is 24 hours, the storage controller's background timing module triggers the defragmentation process (i.e., executes step 11) when the 24-hour cycle is reached. For example, the starting address of the LBA range to be processed can be selected as 0x100000, the total length of the range can be 0x100000 LBAs, and the fragment size can be configured as 0x20000 LBAs, so that the entire LBA range is evenly divided into 5 independent fragments.

[0034] Method 2: When the sequential read delay of the monitored data exceeds the preset delay threshold, determine at least one fragment obtained by dividing the logical block address LBA range.

[0035] In a specific example, assume the preset sequential read latency threshold is 80. The storage controller monitors the average latency of sequential read requests within a certain LBA range for a specific service in real time. When it detects a service range with a starting address of 0x100000 and a length of 0x100000 LBAs, the average latency of 10 consecutive sequential reads reaches 95ms. If the delay exceeds the preset threshold, the fragmentation straightening process is immediately triggered (i.e., step 11 is executed).

[0036] In an alternative implementation, the LBA interval can be divided into at least one fragment according to the physical block capacity boundary of the storage medium, so that the data of each fragment can be completely written into one or more complete physical blocks.

[0037] In a specific example, assuming that a single physical block of the SSD storage medium corresponds to 0x10000 LBAs, based on the physical block boundary alignment rules, the LBA interval to be normalized, starting at address 0x100000 and having a total length of 0x100000 LBAs, is divided into fragments: using the LBA capacity of a single physical block as the alignment unit, the LBA interval is divided into 10 fragments aligned with the physical block boundaries, each fragment containing 0x10000 LBAs.

[0038] After partitioning, the first and last LBAs of all fragments are strictly aligned with the physical block address boundaries of the SSD. The data corresponding to each fragment can be completely written into an independent physical block, effectively avoiding the situation where fragment data is stored across multiple physical blocks, reducing intra-block fragmentation generated during the defragmentation process, and improving defragmentation efficiency.

[0039] In another optional implementation, the LBA interval can be divided into at least one segment based on the dispersion of the PBA corresponding to the LBA interval, according to a preset adjustment rule. The preset adjustment rule includes the following: the dispersion of the PBA is negatively correlated with the size of the segment.

[0040] In a specific example, for an LBA interval to be normalized with a starting address of 0x100000 and a total length of 0x100000 LBAs, the preset dispersion threshold interval corresponds to the adaptive fragmentation size rule: high dispersion regions with a dispersion greater than 0.3 are adapted to small fragments of 0x10000 LBAs, and low dispersion regions with a dispersion less than or equal to 0.3 are adapted to large fragments of 0x20000 LBAs.

[0041] After thorough testing, the first segment of the LBA interval, from 0x100000 to 0x15FFFF, has a dispersion of 0.38, belonging to a high dispersion region. The partition size is adaptively adjusted to 0x10000 LBAs to finely divide the densely fragmented area. The latter segment of the LBA interval, from 0x160000 to 0x1FFFFF, has a dispersion of 0.15, belonging to a low dispersion region. The partition size is adaptively adjusted to 0x20000 LBAs to quickly traverse and regularize the low-fragmentation region. This method strictly adheres to the rule that dispersion is negatively correlated with partition size, ensuring both the accuracy of processing highly fragmented regions and improving the efficiency of processing low-fragmentation regions, thus balancing regularization effect and processing throughput.

[0042] Step 12: If the discreteness of the PBA mapped by the partitioned segment meets the preset discrete conditions, then for the segment, perform the following: read data from the PBA mapped by the segment; sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

[0043] PBA, or Physical Block Address, is the number of the actual physical storage page on the NAND flash memory medium side. It is a storage addressing unit at the hardware level of the storage controller.

[0044] PBAs are bound one-to-one with LBAs (often individual LBAs) through an L2P mapping table, representing the actual physical location of the corresponding logical data in the flash memory. In real-world scenarios, multiple consecutive LBAs within the same slice may have their bound PBAs distributed discretely across different physical blocks of flash memory, which is the root cause of physical fragmentation.

[0045] In this embodiment, the LBA interval is the global logical addressing range of the physical fragmentation straightening task. Based on the fragment size configured by the host, the complete LBA interval can be divided into several non-overlapping fragments equally (or unequally). The fragment is the smallest execution unit for discreteness assessment and data migration straightening. Each fragment contains a set of continuous LBAs. Each LBA uniquely corresponds to a PBA through the L2P mapping relationship. If the PBAs corresponding to the LBAs in the fragment are discretely distributed in the flash memory, physical fragmentation will occur, and straightening operations need to be performed.

[0046] The target PBA refers to a contiguous free physical block address used to store fragmented data ordered by LBA.

[0047] In one alternative implementation, a free physical block can be selected as the target PBA. In another alternative implementation, contiguous PBA space can be reserved as the target PBA during the power-on initialization phase of the storage device.

[0048] In one optional implementation, the method provided in this application embodiment can determine the discrete cases in any of the following ways: 1. Determine the discrete case based on the total number of discrete iterations of all PBAs mapped by the partition and the total number of all PBAs.

[0049] The total number of PBA discretizations refers to traversing all ordered valid PBA sequences within a fragment. If two adjacent valid PBAs do not belong to the same physical page's continuous valid address space and a physical page jump occurs, it is counted as one discretization. This is the cumulative total number of physical page jump breakpoints.

[0050] The total number of PBAs refers to the total number of PBAs corresponding to the one-to-one mapping of valid LBAs within the current shard, which is the total number of samples in the shard regularization statistics.

[0051] In one alternative implementation, the ratio of the total number of discrete iterations to the total number of elements can be calculated as the discreteness of the partition. This discreteness characterizes the discreteness.

[0052] In a specific example, consider a target fragment containing 0x20000 PBAs. By iterating through and verifying each adjacent PBA, the continuity of each PBA is checked, resulting in a total of D = 3000 PBA discrete occurrences for this fragment. According to the dispersion calculation rule, dispersion = total discrete occurrences / total number of PBAs = 3000 / 0x20000 = 3000 / 65536 ≈ 0.0458. This dispersion value directly reflects the overall density of physical address breaks in this fragment.

[0053] The higher the calculated dispersion value, the more prominent the physical fragmentation problem of the data carried by this shard, and the higher the business necessity to initiate physical fragmentation straightening operations. Generally, dispersion = 0: all PBAs are completely continuous (ideal state); dispersion = 1: each adjacent PBA is discontinuous (completely discrete); dispersion = 0.5: about half of the PBA jumps are discontinuous.

[0054] 2. Determine the discrete case based on the total number of cross-physical block times of all PBAs mapped by the shard and the total number of all PBAs.

[0055] The total number of cross-physical block jumps refers to the cumulative total of all cross-physical block jumps. If two consecutive valid PBAs belong to different physical blocks of the SSD, regardless of whether their values ​​are adjacent, it is counted as one cross-block jump.

[0056] In one alternative implementation, the ratio of the total number of times across physical blocks to the total number can be calculated as the discreteness of the fragmentation.

[0057] This approach focuses on the discrete characteristics of the physical block level. When two adjacent PBAs belong to different physical blocks, it is counted as one cross-block. The dispersion is obtained by the ratio of the total number of cross-blocks to the total number of fragmented PBAs, which emphasizes the hardware access overhead caused by the distribution of data across physical blocks.

[0058] In a specific example, assuming a single SSD physical block holds 0x10000 PBAs, and a target fragment containing 0x20000 PBAs is traversed and detected, the total number of times the PBAs in this fragment cross physical blocks is found to be 1200. According to the cross-block dispersion calculation rule, dispersion = total number of cross-physical-block instances / total number of PBAs = 1200 / 65536 ≈ 0.0183. This metric can accurately reflect the severity of cross-block storage of fragmented data.

[0059] 3. Determine the discrete case based on the physical address interval distance generated by each discrete step of each PBA mapped by the shard.

[0060] The single discrete physical address interval distance refers to the address hole size between effective PBAs for each discrete breakpoint of a physical page jump, expressed by the formula... The calculation shows that the larger the value, the greater the jump in physical address and the more serious the fragmentation problem.

[0061] In one alternative implementation, the weighted sum of the physical address interval distances generated by each PBA in each discrete step can be calculated as the discreteness of the fragment.

[0062] This method introduces a physical discrete distance weighting mechanism, which not only counts the number of discrete breakpoints, but also quantifies the address interval range of each discrete step. The discreteness is obtained by weighted summation and normalization of all discrete interval distances, which can distinguish fragmentation scenarios with different degrees of discreteness under the same number of discrete steps.

[0063] The calculation formula is:

[0064] Where Max_Page_Size is the maximum physical page addressing size fixed at the factory for the flash memory hardware of the storage device (such as SSD), which is an inherent fixed parameter of the device; N is the total number of PBAs mapped by the fragment.

[0065] In a specific example, assuming the total number of PBAs mapped by the partition is N=65536 and Max_Page_Size=16, then, traversing all discrete breakpoints of this partition, summing the absolute values ​​of all discrete interval distances... Substituting into the above formula, we can obtain: .

[0066] This method takes into account both the number of discrete breakpoints and the magnitude of discrete offsets, and compared with the simple counting method, it can more comprehensively and precisely characterize the severity of fragmented physical debris.

[0067] In one optional implementation, for any of the above methods of determining discreteness, if the discreteness calculated by this method is greater than a preset discreteness threshold, it can be determined that the discreteness of the PBA mapped by the segment meets the preset discreteness condition.

[0068] If the dispersion is not greater than a preset dispersion threshold, it is determined that the dispersion of the PBA mapped by the fragment does not meet the preset dispersion condition, so that physical fragmentation of the PBA mapped by the fragment is not required.

[0069] In this embodiment, a conditional data migration strategy based on a dispersion threshold is adopted, which can perform data migration and reorganization only on fragments with dispersion exceeding the threshold, avoiding unnecessary relocation of fragments with low dispersion and reducing write amplification.

[0070] In one optional implementation, step 12, "reading data from the PBA mapped by the shard; sorting and writing the data into the target PBA according to the order of the LBAs corresponding to the data," can be implemented in at least two of the following ways: Method 1: After reading data (referred to as first data for easy distinction and description) from the PBA mapped by the shard, cache the first data in the first buffer according to the order of the corresponding LBAs; write the first data in the first buffer into the target PBA.

[0071] Method 2: Multi-level buffer pipeline migration, which uses multiple buffers to implement pipeline migration. Specifically, when a slice writes a new block in Buffer A, the next slice simultaneously reads data from Buffer B, improving overall throughput.

[0072] In terms of specific technical implementation, Method 2, based on the technical features of Method 1, may further include: while writing the first data in the first buffer to the target PBA, simultaneously reading the second data in the PBA mapped to the next shard of the shard (current shard), and caching the second data in the second buffer according to the corresponding LBA order; and writing the second data in the second buffer to the target PBA.

[0073] Using method two, while writing the current fragment ordered data to the target PBA, the next fragment data can be read and the LBA ordered cache can be completed simultaneously, making full use of the storage device's read / write channel resources and eliminating the time consumption of single-buffer read / write serial waiting.

[0074] In an optional implementation, the method provided in this application embodiment may further include: responding to a status query command sent by the host, and feeding back the status of physical fragmentation to the host.

[0075] In one alternative implementation, the defragmentation status may include, but is not limited to, not started, in progress, and completed. The storage controller may fill in the defragmentation status in a specified field (such as the Status field) of the completion response sent back to the host, and then send the completion response back to the host.

[0076] In one alternative implementation, the storage controller may obtain the sorting status by reading an internal status register (an on-chip hardware register set integrated within the storage controller's main control chip).

[0077] In one optional implementation, the storage controller can move all data corresponding to LBAs within a fragment that meets the preset discrete conditions sequentially to a pre-reserved contiguous physical block space (i.e., write to the target PBA) in LBA order. After the move is completed, the LBA-PBA mapping relationship in the FTL is updated, the old data stored in the original physical block is erased, and the current compaction progress is updated to the on-chip register (the storage controller's internal status register).

[0078] By employing the method provided in the embodiments of this application, since the LBA interval is first divided into fragments and fragments whose PBA dispersion meets the preset conditions are selected, data on the scattered PBAs are only collected in such fragments, and then written to the target PBA in LBA order, so that logically continuous data in the fragments correspond to continuous physical addresses. When the host initiates a sequential read, it does not need to access multiple non-contiguous physical blocks, reducing additional NAND read commands and read amplification. This solves the problems of decreased sequential read throughput bandwidth, amplified IO latency jitter, and increased read request tail latency caused by the discretization of data physical distribution.

[0079] From the host's perspective, this application also provides another method for physical fragmentation processing. This method includes the following steps: Step a: Obtain the address information and fragmentation specification information of the LBA range of the physical fragments to be reorganized; Step b: Send the address information and the fragmentation specification information to the storage controller to trigger the storage controller to perform the following operation: Determine at least one segment obtained by dividing the LBA interval; if the discreteness of the PBA mapped by the segment meets the preset discreteness conditions, then for the segment, perform the following: read data from the PBA mapped by the segment; sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

[0080] In an alternative implementation, the method may further include the step of sending a status query command to the storage controller to obtain the defragmentation status of the physical fragments as reported by the storage controller.

[0081] In one optional implementation, after sorting and writing the data into the target PBA according to the order of the LBAs corresponding to the data, the L2P mapping table can be updated according to the target PBA and the LBA of the shard, and the original PBA where the data was located before migration can be marked as invalid.

[0082] Example 2 Example 2 mainly introduces a specific implementation method for applying the physical fragmentation processing method provided in Example 1 of this application to a real-world scenario.

[0083] In practical application scenarios, the method provided in Embodiment 1 of this application can be used in, for example... Figure 2a The system architecture shown is implemented in this way.

[0084] like Figure 2a As shown, the system architecture includes the following core modules: (1) Host: Primarily consists of a defraction management driver module, host application / operating system, and NVMe interface layer. Among them: The defragmentation management driver module is a driver program running in the host operating system, responsible for issuing NVMe Vendor Commands (which can be denoted as Vendor Command 1) to the SSD. This NVMe Vendor Command 1 contains parameters for the LBA range to be defragmented (equivalent to the LBA range information mentioned in Embodiment 1 of this application, including but not limited to: LBA range, fragment size, and dispersion threshold). In this embodiment, the defragmentation management driver module can also poll the SSD's defragmentation status via another NVMe Vendor Command (which can be denoted as Vendor Command 2).

[0085] The full name of the NVMe Vendor Command standard is NVMe Vendor-Specific Command, which is a custom command defined by the NVMe vendor.

[0086] In this embodiment, two NVMe Vendor Specific Commands can be defined, one for submitting defragmentation parameters to the SSD and the other for querying the SSD's defragmentation status. These two NVMe Vendor Specific Commands can be denoted as NVMe Vendor Specific Command 1 and NVMe Vendor Specific Command 2.

[0087] The NVMe Vendor Specific Command 1, also known as the "Parameter Submission Command," has the following format as shown in Table 1: Table 1:

[0088] In Embodiment 2 of this application, the host can send all defragmentation parameters to the SSD at once through a parameter submission command. After receiving the parameters, the SSD can start the defragmentation process autonomously.

[0089] The NVMe Vendor Specific Command 2, as a "status query command," has the following format as shown in Table 2: Table 2:

[0090] The NVMe interface layer contained in the host mainly includes the standard NVMe protocol communication interface between the host and the SSD. This interface includes a commit queue (SQ) and a completion queue (CQ) for issuing commands and receiving completion responses.

[0091] In one optional embodiment, the host can periodically send status query commands to the solid-state drive (SSD) to obtain the execution progress of the physical fragmentation consolidation task in a polling manner. This command is a lightweight status query NVMe vendor command. The SSD only needs to read its internal status registers (on-chip hardware registers integrated within the SSD controller chip) to complete the response. The processing flow has no flash memory access overhead and will not block the front-end regular input / output (IO) service path.

[0092] In this embodiment, the host actively polls the compaction status through the second Vendor Command, rather than relying on asynchronous push from NVMe AER, mainly based on the following considerations: 1. The defragmentation status changes infrequently (usually in units of fragments or percentage progress), so real-time push notifications are not required; 2. The host actively queries according to its own scheduling rhythm, avoiding the overhead of asynchronous interruption; 3. The status query command is a lightweight register read operation, with extremely low SSD response latency and no blocking of normal I / O; 4. It avoids the resource consumption problem of AER commands (AER commands need to be submitted in advance and continuously occupy the submission queue entries).

[0093] In this embodiment of the application, the following state machine definition, as shown in Table 3, can be adopted: Table 3:

[0094] In this embodiment of the application, after the host sends the parameter submission command, it begins to periodically send status query commands.

[0095] If the SSD returns Status=0x00, it means that the SSD has not yet started and the host continues to wait; If Status=0x01 is returned, the host reads fields such as Progress and Completed Segments to perceive the consolidation progress—the host can adjust IO scheduling according to the progress (such as reducing the priority of IO requests in the consolidation interval). If Status=0x02 is returned, the cleanup is complete, and the host reads fields such as Avg Dispersion Before / After to obtain the cleanup result.

[0096] In this embodiment, the host can adaptively adjust the polling interval according to the total number of shards. When there are many shards (and the sorting takes a long time), a longer polling interval is used, and when there are few shards, a shorter polling interval is used to balance the timeliness of state awareness and command overhead.

[0097] In this embodiment of the application, the above-mentioned "parameter submission command" and "status query command" can both be submitted to the submission queue (SQ) by the defraction management driver module, and then sent to the SSD based on the standard NVMe protocol.

[0098] For SSDs, a "completion response" can be sent to the host in response to a "parameter submission command" or a "status query command". This completion response can be submitted to the standard NVMe protocol communication interface and added to the completion queue (CQ).

[0099] In one alternative implementation, the SSD may return the following status information as shown in Table 4 below in the completion response: Table 4:

[0100] For the fields in Table 4, in a specific example, SSD can determine the specific values ​​of the fields in the following way.

[0101] In this example, assume the host sends a command with the following parameters: a total LBA interval length of 0x100,000 LBAs to be normalized, and a shard size of 0x20,000 LBAs. Then, a total of 5 shards can be obtained. Furthermore, assume the SSD has a preset dispersion threshold of 0.03; normalization is only performed when the shard dispersion is greater than 0.03, and processing is skipped if it is less than or equal to the threshold.

[0102] Based on the above assumptions, the field values ​​are determined as follows: 1. Status (offset 0, length 1 byte) The methods for determining the value of this field include: SSD controller real-time monitoring of background defragmentation task execution markers: If the host issues the organization parameters but does not start the fragment traversal calculation Assign the value 0x00; If we are calculating the dispersion and performing data migration regularization piece by piece... Assign the value 0x01; If all segments have completed discreteness determination and normalization / skip processing Assign the value 0x02.

[0103] Specifically, after the host issues the defragmentation command, the SSD initializes the shard list, at which point Status=0x00; the SSD begins to traverse the first shard to calculate the dispersion, and the field is refreshed to 0x01; after all 5 shards have been processed, Status is updated to 0x02.

[0104] 2. Progress (offset 1, length 1 byte) The methods for determining the value of this field include: According to the calculation formula: Calculate the Progress value, round it down, and refresh it in real time.

[0105] For example, if Total Segments=5, 2 normalized segments have been completed and 1 segment has been skipped, for a total of 3 segments processed. The field is filled with the value 60.

[0106] 3. Total Segments (offset 2, length 2 bytes) The method for determining the value of this field includes: SSD calculates the total number of fragments based on the issued LBA range and fragmentation rules (fixed size / adaptive / physical block alignment), calculates and fills the fragments once when the task starts, and keeps them unchanged throughout the process.

[0107] For example, assuming the total length of the LBA interval is 0x100000 LBAs, and the fixed segment size is 0x20000 LBAs, the total number of segments = 0x100000 ÷ 0x20000 = 5, therefore Total Segments = 5.

[0108] 4. Completed Segments (offset 4, length 2 bytes) The methods for determining the value of this field include: This field increments by 1 for each fragment migration and rewrite completed by the SSD; only fragments with a dispersion greater than a preset threshold are included in this count.

[0109] For example, suppose the dispersion of the 5 segments is 0.045, 0.038, 0.022, 0.015, and 0.010 respectively; the threshold is 0.03. The first 2 segments with dispersion exceeding the threshold are normalized, and the last 3 segments are skipped. During task execution, Completed Segments = 1 when the first segment is completed; after all segments are processed, this field = 2.

[0110] 5. Skipped Segments (offset 6, length 2 bytes) This field means: the total number of fragments whose dispersion does not exceed the dispersion threshold and therefore do not require data normalization and can be skipped directly.

[0111] The methods for determining the value of this field include: If the fragment dispersion is less than or equal to the preset dispersion threshold, data migration will not be initiated, and the value of this field will increment by 1.

[0112] Continuing from the previous example, the dispersion of the last three segments (0.022, 0.015, and 0.010) is less than the threshold of 0.03, so they are all skipped; after the task is completed, Skipped Segments = 3.

[0113] 6. Avg Dispersion Before (offset 8, length 4 bytes) The meaning of this field is: the average dispersion of all pieces to be normalized before the normalization process.

[0114] The methods for determining the value of this field include: During the task startup phase, the SSD traverses all shards, calculates the dispersion of each shard, and then calculates the arithmetic mean, which is written to this field all at once and remains unchanged throughout the task.

[0115] For example, the original dispersion of the 5 fragments is 0.045, 0.038, 0.022, 0.015, and 0.010. The average dispersion is (0.045 + 0.038 + 0.022 + 0.015 + 0.010) ÷ 5 = 0.026. The SSD encodes 0.026 according to the preset 4-byte fixed-point encoding rule and stores it in this field.

[0116] 7. Avg Dispersion After (offset 12, length 4 bytes) The meaning of this field is: the average dispersion of all out-of-range fragments after normalization. The value is only valid when Status=0x02 (task completed); if the task is not completed, this field is filled with invalid placeholder values.

[0117] The methods for determining the value of this field include: Only after all shards have been processed and Status is set to 0x02, the SSD reads the new PBA sequence after all shards have been rewritten, recalculates the dispersion and calculates the average value, and stores the encoded data in this field; it is not updated during task execution.

[0118] For example, assuming the dispersion of the first two out-of-range fragments is reduced to 0.002 and 0.001 respectively after normalization, the average dispersion = (0.002 + 0.001) / 2 = 0.0015; the task completion status = 0x02, and 0.0015 is encoded and filled into this field; if the task is in progress (Status = 0x01), this field has no valid data.

[0119] In this embodiment of the application, in addition to obtaining the defragmentation running status through Vendor Command polling, the following two status acquisition methods can also be used: 1. Obtain the defragmentation running status by polling the NVMe log page command.

[0120] Specifically, a vendor-specific log page can be predefined. A vendor-specific log page is a private extended log page in the NVMe log page system. It is different from the general standard log page specified by the protocol. It stores the background task execution data customized by the vendor and is only adapted to the private function parsing of this device.

[0121] Based on Vendor Specific Log Page, the host uses the NVMe GetLog Page command to periodically read the runtime status information stored in the log page, such as fragmentation progress, dispersion determination results, and fragmentation processing completion markers.

[0122] This implementation method is suitable for application scenarios with lower call costs and better compatibility in the standard NVMe driver stack log page read link.

[0123] The NVMe Get Log Page command is a public management command defined by the NVMe standard protocol, which provides a standardized interface for the host to read information such as the internal operating logs, hardware status, and background task progress of the solid-state drive.

[0124] 2. Obtain the defragmentation running status based on the status information carried by the Admin Completion Queue.

[0125] The Admin Completion Queue is a message queue in the NVMe architecture used to receive the execution results of management commands issued by the host. After the SSD completes the processing of the management commands, it returns a completion response carrying the status and parameters to the host through this queue.

[0126] Under the NVMe protocol architecture, the execution results and response data of all management commands issued by the host are uniformly returned to the host by the SSD through the management completion queue.

[0127] Based on this mechanism, after sending the parameter submission command, the host can listen to and read the command completion response messages in the management completion queue, and directly parse the initial start status of the defragmentation task from them. As for the dynamic update status during the continuous operation of defragmentation, the host cannot continuously obtain it through a single queue response. Therefore, it can periodically resend the parameter submission command to refresh the queue response content, or combine it with the aforementioned Get Log Page polling method to continuously obtain the real-time execution progress and running status of defragmentation.

[0128] (2) SSD side module: like Figure 2a As shown, some of the main modules included in an SSD are as follows: 1. NVMe Command Parsing Module: Primarily used to parse NVMe vendor commands issued by the host, distinguishing between parameter submission commands and status query commands, and extracting corresponding fields from the commands. The specific method for distinguishing commands is as follows: The value of the Opcode field of the NVMe vendor command is used to determine the command type. If it is the Opcode corresponding to a parameter submission command, the defragmentation parameters carried in the command are extracted, stored in the on-chip memory of the SSD controller, marked as pending defragmentation, and a completion response is returned to the host. If it is a status query command, the defragmentation progress and status information stored in the on-chip hardware registers are directly read, encapsulated into a completion response, and returned to the host without accessing the backend NAND flash memory.

[0129] 2. Status Maintenance Module: Relying on the on-chip hardware registers of the SSD controller, the module persistently maintains the fragmentation and regularization status throughout its entire lifecycle (including three status indicators: not started, running, and completed). It synchronously records the fragmentation processing progress, skipped fragment statistics, and task execution result summaries. The host uses the NVMe vendor's custom status query command to poll and read the data in the registers, achieving non-intrusive progress observation.

[0130] 3. Defragmentation and defragmentation scheduling module: This module is responsible for the overall scheduling of the entire physical defragmentation process. Specifically, it schedules and initiates defragmentation tasks based on the defragmentation parameters. According to the order of the fragments marked as needing defragmentation in the defragmentation waiting queue, it serially issues three types of sub-tasks: L2P mapping traversal, discreteness assessment, and data migration. During the task progress, it works in conjunction with the status maintenance module to refresh the progress indicators in the on-chip registers, providing a real-time data source for host-side polling, and controlling the entire process of fragment loop processing, task start-up, and task termination.

[0131] 4. The L2P mapping table traversal module is used to first divide the entire LBA range to be processed into several independent fragments according to the fragment size based on the LBA range and fragment size given by the parameter submission command (a single fragment is a group of continuous LBA ranges, which is the smallest unit of regularization processing); it traverses all LBA-PBA mapping entries in the L2P mapping table fragment by fragment, and collects the physical block address (PBA) bound to each logical block address; after a single fragment completes data migration, it rewrites the L2P mapping entries according to the new physical storage location of the data, establishes the updated LBA and PBA binding relationship, and ensures that upper layer IO requests can complete normal data addressing by relying on the new version of the mapping entries.

[0132] 5. The dispersion calculation module is used to receive the single-shard PBA distribution data output by the L2P mapping table traversal module, count the physical address dispersion characteristics within the shard, and calculate the shard dispersion. The calculation results are compared with the preset dispersion threshold: if the shard dispersion exceeds the limit, the data migration process of this shard is triggered; if the dispersion does not meet the standard, the shard is skipped, the skipped shard count is accumulated, and the next shard analysis process is carried out in sequence.

[0133] In this embodiment of the application, the fragmentation dispersion can be quantified and calculated according to the formula "dispersion = D / N".

[0134] Wherein, variable D refers to the number of physical address discretes obtained from the statistics within the fragment; variable N is the total number of LBAs contained in this fragment (fragmentation specifications are measured by the number of LBAs; in this embodiment of the application, for example, it can be preset that a single LBA uniquely corresponds to 1 PBA).

[0135] In a specific example, assuming a certain fragment size is configured with 256 LBAs, traversing the mapping entries yields 256 corresponding PBAs. Statistically, a total of 80 physical address discretes occur within the fragment. Substituting into the formula, we can obtain the discreteness = 80 / 256 ≈ 0.3125 (accounting for 31.25%).

[0136] The larger the value obtained from the dispersion calculation, the more prominent the physical fragmentation problem of the data carried by this shard is, and the higher the business necessity to initiate physical fragmentation straightening operation.

[0137] In one alternative implementation, the formula for calculating the dispersion is: Dispersion = Number of discrete iterations within a slice / Total number of PBAs within a slice. Normalization property: Since the dispersion is the ratio of the number of discrete iterations to the total number of PBAs, it is a dimensionless value between 0 and 1, independent of the partition size. This makes the dispersion measure comparable for partitions of different sizes. Discreteness = 0: All PBAs are completely continuous (ideal state) Discreteness = 1: Each adjacent PBA is discontinuous (completely discrete). Dispersion = 0.5: Approximately half of the PBA jumps are discontinuous. Compared with absolute value measures: The traditional approach uses the number of "discontinuous fragments (DF)" as a measure. Thus, a fragment containing 500 PBAs with 50 discontinuous fragments (absolute value 50) and a fragment containing 100 PBAs with 50 discontinuous fragments (absolute value 50) give the same fragmentation level value. However, the former has a dispersion of 50 / 500 = 0.1 (milder) and the latter has a dispersion of 50 / 100 = 0.5 (more severe). The severity of the two is completely different.

[0138] The comparison shows that the normalized dispersion calculation method provided in the embodiments of this application can more accurately reflect the severity of fragmentation.

[0139] In one optional implementation, the L2P mapping table traversal module reads the PBA sequence corresponding to each LBA in the current slice in ascending order of LBA size: P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, P1, P1, P2, P1, P2, P2, P2, P3, P4, P5, P6, P7, P8, P9, P1, P1, P2, P2, P2, 2...... P N-1 ; Then, compare the PBAs corresponding to two consecutive adjacent LBAs in turn: Let the physical address P corresponding to the current i-th LBA be... i The next adjacent LBA (LBA i+1) corresponds to the physical address P. i+1 If satisfied If a physical address discretization occurs once, the discretization count D is incremented by 1; if the condition is met... , which means that two logically consecutive data are also consecutive on the physical medium, no fragmentation occurs, and the D value remains unchanged. In short, in an optional implementation of the embodiments of the present application: the PBAs corresponding to adjacent consecutive LBAs are non-consecutive, which is recorded as one fragmentation.

[0140] In a specific example, the fragment size is set to N=256 (a fragment contains 256 consecutive LBAs, with LBA numbers L0 to L 255 , each LBA uniquely corresponds to one PBA).

[0141] Further assume that a segment of PBA sequence corresponding to a consecutive LBA segment is obtained by traversing the L2P mapping: L0 corresponds to P 100 ——that is, the binding relationship in the L2P mapping table is: the service data corresponding to the logical address L0 is currently actually stored at the physical address P 100 , and similar expressions in the following text have the same meaning as here; L1 corresponds to P 101 ——that is, the data of the logically adjacent No. 1 LBA is stored in the No. 101 physical page of the flash memory); L2 corresponds to P 200 ——that is, the data of the logically adjacent No. 1 LBA is stored in the No. 101 physical page of the flash memory; L3 corresponds to P 201 ——that is, the data of the logical No. 3 LBA follows the previous one and is stored in the No. 201 physical page; L4 corresponds to P 50 ——that is, the data of the logically adjacent No. 4 LBA no longer continues in the No. 202 physical page, and jumps to the No. 50 physical page of the flash memory.

[0142] Here, Px represents a PBA, and x is the number of the real physical page on the flash storage medium.

[0143] Then, the corresponding PBAs are read strictly in ascending order of the logical addresses L0, L1, L2, L3, L4 according to the traversal sequence, and adjacent PBAs are compared pairwise to check whether they are consecutive: L0(P 100 ) and L1(P 101 ): 101=100+1, physically consecutive, no fragmentation is counted; L1(P 101 ) and L2(P 200 ): 200=101+1, physically non-consecutive, counted as one fragmentation; L2(P 200 ) and L3(P 201 ): 201=200+1, physically consecutive, no fragmentation is counted; L3(P 201 ) and L4(P 50): 50 = 201 + 1, physical discontinuity, let’s record it as discrete 1 more time.

[0144] Therefore, it can be determined that within these 5 consecutive LBA segments alone, a total of 2 physical address discrepancies were generated.

[0145] In this embodiment, it is assumed that after traversing and statistically analyzing all 256 adjacent LBA-PBA pairs in the fragment, the total number of discrete occurrences is D=80. Substituting D=80 into the dispersion calculation formula, we can calculate that: Dispersion = D / N = 80 / 256 ≈ 0.3125, corresponding to a fragmentation rate of 31.25%.

[0146] 6. The data migration execution module is used to sequentially retrieve the fragments to be processed from the processing waiting queue, move the data corresponding to all logical addresses in the fragment to the pre-reserved contiguous physical block space in logical address order, update the LBA-PBA mapping relationship in FTL after the transfer is completed, erase the old data stored in the original physical block, update the current processing progress to the on-chip register, and then process the next fragment to be processed until all fragments to be processed are processed and the processing status is updated to complete.

[0147] In addition, SSDs may also include Figure 2a The idle management reservation module, not shown, is used to reserve a continuous physical block space of sufficient size to meet defragmentation requirements during the SSD power-on initialization phase and normal idle block maintenance process. It is specifically used for temporary data migration during the defragmentation process to avoid insufficient idle physical blocks during defragmentation and ensure that the defragmentation process can be executed smoothly.

[0148] about Figure 2a Explanation of data flow in the middle: In this embodiment, the host sends out the configuration parameters via the NVMe vendor command (parameter submission command). SSD command parsing module extracts parameters The defragmentation scheduling module initiates the defragmentation process. The L2P mapping table traversal module traverses the mapping table. The discreteness calculation module calculates the discreteness. The data migration execution module performs the migration. Status maintenance module updates status The host polls for the status using the NVMe vendor command (status query command). Loop through all fragments Completed.

[0149] Based on such Figure 2a The system architecture shown in Embodiment 2 of this application can be adopted as follows: Figure 2cThe process shown enables the organization of physical debris.

[0150] like Figure 2c The flowchart shown includes the following steps: Step S1: The host issues the NVMe Vendor Command; The host-side operating system calls the defraction management driver module to construct an NVMe vendor-defined command (NVMeVendor Command), and fills the command payload with all the configuration parameters required for physical fragmentation. These parameters include: 1. The starting address of the address range of the logic block to be normalized (example value: LBA0x100000); 2. Total length of the address range of the logic block to be normalized (example value: 0x10000, corresponding to 65536 consecutive LBAs). 3. Fragment size (Example value: 0x100, corresponding to 256 LBAs as a single independent regularized fragment unit). 4. Dispersion determination threshold (example value: 0.3, i.e. 30%).

[0151] The defragmentation management driver module sends the parameter submission class NVMe vendor-defined command to the solid-state drive through the NVMe interface layer's submission queue (SQ). After receiving the complete command, the solid-state drive automatically starts a physical defragmentation background task, and the host does not need to continuously block and wait for the task to be executed.

[0152] Step S2: SSD parsing command parameters; The NVMe command parsing module inside the SSD receives and parses the NVMe vendor-defined commands issued by the host, extracts all normalization configuration parameters from the command payload, and passes the parameters to the defragmentation scheduling module. The defragmentation scheduling module completes the normalization context initialization operation based on the received parameters. The initialization content includes: Calculate the total number of fragments: Total number of fragments = Total length of LBA intervals to be normalized ÷ Fragment size; Example calculation: 65536 ÷ 256 = 256 fragments; Initialize the shard processing index: The current shard index i=0, which means that the shards will be processed sequentially starting from the 0th shard; Refresh task running status: Call the status maintenance module to mark the regularized task status in the on-chip hardware register as Status=0x01 (executing).

[0153] Step S3: The host polls and sorts the status; In step S3, the host-side defragmentation management driver module issues status query-type NVMe vendor-defined commands according to a preset cycle; After the solid-state drive NVMe command parsing module recognizes the query command, it directly reads the regular task information stored in the hardware registers on the SSD controller chip, and the status maintenance module encapsulates the status message and sends it back to the host. The entire process does not access the backend NAND flash memory medium, has no media access overhead, and does not block the front-end regular IO business. Furthermore, after the host parses the returned message, it obtains the current sharding processing progress through the Progress field in the message. The operating system can dynamically adjust its own business IO scheduling strategy based on the real-time sharding progress to avoid the sharding task competing with the front-end business IO for hardware resources.

[0154] Step S4: FTL traverses the L2P mapping table of the current fragment; For the current shard (the i-th shard, where i starts from 0), the L2P mapping table traversal module performs the following operations: 1. Calculate the LBA range corresponding to this fragment: Fragment start LBA = Start LBA of the interval to be normalized + i × fragment size; Fragment end LBA = Fragment start LBA + fragment size 1.

[0155] If the starting point of the interval to be normalized is denoted as LBA Start, and the segment size is denoted as Segment Size (that is, the total number of LBAs contained in this segment), then the starting LBA of the segment = LBA Start + i × Segment Size, and the ending LBA = starting LBA + Segment Size - 1.

[0156] 2. Traverse the L2P mapping table entries of each LBA within the LBA range to obtain the corresponding PBA sequence: PBA[0], PBA[1], ..., PBA[N-1] (N=Segment Size) Step S5: Check physical continuity and count the number of discrete iterations D; The dispersion calculation module performs a continuity check on the PBA sequence: Traverse index j from 0 to N 2. Compare PBA[j] and PBA[j+1] sequentially—specifically, starting from PBA[0], determine whether PBA[j+1] is equal to PBA[j]+1 (physical addresses are contiguous); If PBA[j+1]≠PBA[j]+1, then it is counted as one discretization, and the total number of discretizations D is incremented by 1. If PBA[j+1]=PBA[j]+1, it means that logically continuous data are also physically stored in continuous locations, without producing discreteness, and the value of D remains unchanged. After the traversal is complete, count the total number of discrete iterations within the current partition, denoted as D.

[0157] Step S6: Calculate the dispersion; The dispersion of the current partition is calculated using the formula "dispersion = D / N". Here, D is the number of discrete iterations, and N is the number of PBAs in the current partition (i.e., the partition size, measured in terms of LBAs, assuming each LBA corresponds to one PBA).

[0158] For example, if the partition size is 256 LBAs, and traversal yields 256 PBAs, with 80 discrete iterations, then the dispersion = 80 / 256 ≈ 0.3125 (approximately 31.25%).

[0159] Step S7: Determine whether the dispersion is greater than the threshold; In step S7, if the dispersion is determined to be greater than the dispersion threshold (e.g., 0.3125 > 0.3), then proceed to step S8 to perform data migration and sorting; if the dispersion is determined to be less than or equal to the dispersion threshold, then skip the current shard and proceed to step S10 to process the next shard.

[0160] Step S8: Data migration and organization; In step S8, the data migration execution module performs the following operations (such as...). Figure 2d (as shown) S8a: Read data into the buffer. Read all valid data of the current fragment from its original physical location (PBA[0]~PBA[N-1] are scattered in each physical block) into the SSD internal buffer, and arrange them in LBA order.

[0161] S8b: Select a free physical block. Select one or more free physical blocks from the free block pool as the target physical blocks for data writing.

[0162] S8c: Write in physical order. Write the data in the buffer to the target physical block in LBA order, so that after writing, PBA[0]~PBA[N-1] are arranged consecutively in the physical address (PBA[j+1]=PBA[j]+1).

[0163] S8d: Update the L2P mapping table. Update the PBA corresponding to each LBA of the current fragment to the new physical address.

[0164] S8e: Mark the original physical page as invalid. The data page at the original physical location is marked as invalid, awaiting subsequent GC reclamation.

[0165] Combination Figure 2c Assuming a single fragment has a fragment size of N=8 (including contiguous LBAs: 0x1000), the data before normalization is scattered across three isolated original physical blocks A, B, and C. The binding relationships of the L2P mapping table before normalization are as follows: LBA 0x1000 corresponds to PBA 0x2000 (physical block A). LBA 0x1001 corresponds to PBA 0x2001 (physical block A). LBA 0x1002 corresponds to PBA 0x2002 (physical block A). LBA 0x1003 corresponds to PBA 0x2050 (physical block B). LBA 0x1004 corresponds to PBA 0x2051 (physical block B). LBA 0x1005 corresponds to PBA 0x2003 (physical block C). LBA 0x1006 corresponds to PBA 0x2004 (physical block A). LBA 0x1007 corresponds to PBA 0x2005 (physical block A). After dispersion calculation, the dispersion of this fragment exceeds the preset threshold, triggering the data migration and normalization process in step S8. Accordingly, the specific implementation process of step S8 is shown in the following example: S8a: Read data into the Buffer (matching the blue, orange, and green read arrows in the diagram); The data migration execution module reads all eight valid service data entries from three dispersed raw physical addresses in the NAND flash memory in ascending LBA order: PBAs 0x2000~0x2004 and 0x2050~0x2051 are used to retrieve the corresponding payloads, and the eight read data entries are stored in the SSD's internal buffer, strictly following LBA 0x1000. The 0x1007 ascending order is used for structured cache arrangement without changing the business payload bound to a single LBA.

[0166] S8b: Select an idle physical block (match the green target block in the diagram); The SSD free block management module selects a continuous set of unoccupied physical pages with a length of 8 pages and an address range of PBA 0x7000~0x7007 from the global free physical block resource pool. This set is written to the storage space as the dedicated target after the fragment is normalized. The total number of pages in the storage space is perfectly aligned with the total number of fragment LBAs N=8, which can fully accommodate the fragment data cached in the buffer.

[0167] S8c: Write the new physical block to the target in physical order; The data in the buffer, sorted in ascending order by LBA, is sequentially written into the continuous target physical space, with each address corresponding to the next: LBA 0x1000 data is written to PBA 0x7000, LBA 0x1001 data is written to PBA 0x7001, and so on, with the last LBA 0x1007 data being written to PBA 0x7007.

[0168] After the write operation is completed, adjacent physical addresses satisfy the continuity constraint: PBA[j+1]=PBA[j]+1 (j takes values ​​in the range 0≤j≤6). The data that was originally scattered in 3 independent physical blocks is arranged continuously in the flash physical layer, completely eliminating the physical fragments corresponding to the local block.

[0169] S8d: Update the L2P mapping table (match the blue arrow pointing left to the L2P mapping table); The flash translation layer refreshes the 8 mapping entries for this segment in the L2P mapping table, replacing all the old PBA binding relationships with the normalized contiguous physical addresses. The updated mapping result is as follows: LBA 0x1000 corresponds to PBA 0x7000 LBA 0x1001 corresponds to PBA 0x7001 LBA 0x1002 corresponds to PBA 0x7002 LBA 0x1003 corresponds to PBA 0x7003 LBA 0x1004 corresponds to PBA 0x7004 LBA 0x1005 corresponds to PBA 0x7005 LBA 0x1006 corresponds to PBA 0x7006 When the host subsequently issues read / write I / O requests for this range, corresponding to LBA 0x1007 and PBA 0x7007, the controller will rely on the updated mapping entries to address the regularized contiguous physical blocks.

[0170] S8e: Marks the original physical page as invalid (matches invalid pages marked with a red cross at the bottom). The original eight source physical pages (PBA: 0x2000, 0x2001, 0x2002, 0x2003, 0x2004, 0x2005, 0x2050, 0x2051) are uniformly marked as invalid data pages in the FTL page status bitmap; the SSD background garbage collection (GC) process scans and identifies invalid pages during subsequent idle periods, performs flash erase operations on the corresponding physical blocks, and reclaims reusable flash storage space.

[0171] Step S9: Update the organization progress; After the data migration execution module completes the sharding operation, it calls the status maintenance module to update the task progress indicators. The progress value is calculated as: the number of shards completed / the total number of shards. The updated progress value is written to the hardware registers on the SSD controller chip, and the host can read the progress data in real time through the status query NVMe vendor-defined command.

[0172] Step S10: Process the next fragment in a loop; Increment the current shard index i by 1, then compare the incremented shard index i with the total number of shards to determine if all shards have been processed, and execute the branch logic based on the result: If i < total number of fragments, return to step S4 to process the next fragment; if i = total number of fragments, proceed to step S11.

[0173] Step S11: The process is complete, and the status can be checked.

[0174] The status maintenance module sets the scrambling status to Status=0x02 (completed) and writes a summary of the scrambling results (number of scrambled fragments, number of skipped fragments, average dispersion of all fragments before scrambling, average dispersion of all fragments after scrambling, etc.).

[0175] After the host polls for Status=0x02 using the status query command, it obtains the final processing result.

[0176] Another complete example is as follows: Assuming the LBA interval is [0x1000, 0x10FF], the slice size is 8 LBAs, and the dispersion threshold is 0.3, the PBA sequence of the first slice (LBA 0x1000~0x1007) is obtained by traversing the L2P mapping table, as shown in Table 5 below: Table 5:

[0177] Since the number of discrete iterations D=2 and the number of PBAs N=8, according to the aforementioned method for calculating the discreteness, the discreteness is calculated to be 2 / 8=0.25.

[0178] Since 0.25 < 0.3 (threshold), the dispersion of this fragment does not exceed the threshold, so it is skipped and not sorted.

[0179] Assume the PBA sequence of the second slice (LBA0x1008~0x100F) is: 0x3000,0x3001,0x4000,0x4001,0x5000,0x5001,0x6000,0x6001.

[0180] Discrete order D=3 (0x3001) 0x4000, 0x4001 0x5000, 0x5001 (0x6000), dispersion = 3 / 8 = 0.375.

[0181] Since 0.375 > 0.3 (threshold), this fragment needs to be cleaned up.

[0182] Next, the data from the 8 LBAs is read into the buffer, and a free physical block is selected to write the following data consecutively starting from PBA0x7000: 0x7000, 0x7001, 0x7002, 0x7003, 0x7004, 0x7005, 0x7006, 0x7007. The L2P mapping table is then updated, marking the original physical page as invalid.

[0183] It should be noted that the core design concept of this application's embodiments is "decoupled from the Host but still aware of it." Decoupling is manifested in: 1. The host does not specify a specific compaction execution strategy (such as data migration path, physical block selection strategy), these are decided autonomously by the SSD; 2. The host computer does not participate in the data migration during the cleanup process; all data migration is handled internally by the SSD. 3. The host does not specify the sorting order of individual LBAs, but only the LBA range and fragmentation parameters.

[0184] The perceptible aspect is reflected in: 1. The host submits the defragmentation parameters via a parameter submission command, and knows that the SSD is about to be defragmented; 2. The SSD maintains and organizes its status registers, and the host actively polls to obtain the status through a status query command; 3. The host can adjust the IO scheduling strategy based on the polled status (such as avoiding a large number of IOs to the cleanup area).

[0185] This interactive mode ensures that the SSD has autonomous decision-making power over physical layer information (the SSD knows its own physical distribution best), while also enabling the host to perceive and control the defragmentation process, thus avoiding the problem of the host being an uncontrollable "black box" in a fully autonomous mode.

[0186] Furthermore, it should be noted that the method provided in this application embodiment can follow the following rules during specific implementation: 1. If a host I / O request is made to access the shard being scrambled during the defragmentation process, the SSD can prioritize handling the host I / O, pause defragmentation, save the current state, and resume defragmentation after handling the I / O.

[0187] 2. The write amplification caused by the defragmentation can be controlled within a reasonable range. It is recommended to perform the defragmentation during the SSD's idle period (when the IO pressure is low).

[0188] 3. The frequency of defragmentation should not be too high to avoid excessively depleting the NAND flash memory. It is recommended to reasonably control the defragmentation trigger frequency by using a dispersion threshold.

[0189] 4. Once the original physical page is marked as invalid, it will be reclaimed by subsequent GC. It is not recommended to erase the original physical block immediately during the compaction process to avoid affecting normal I / O.

[0190] Example 3 To address the issues of decreased sequential read throughput bandwidth due to the discretization of physical data distribution, amplified IO latency jitter in scenarios with high fragmentation, and increased tail latency of read requests, based on the same inventive concept as the above embodiments of this application, Embodiment 3 of this application provides a storage controller for implementing the method described in Embodiment 1 of this application.

[0191] In one alternative implementation, the specific structural diagram of the storage controller in terms of its logical structure can be as follows: Figure 3 As shown, it includes the following functional units: Fragmentation determination unit 31 is used to determine at least one fragment obtained by dividing the logical block address LBA range; The data processing unit 32 is configured to, if the discreteness of the physical block address (PBA) mapped by the fragment meets a preset discrete condition, perform the following for the fragment: read data from the PBA mapped by the fragment; and sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

[0192] In one alternative implementation, the fragmentation determination unit 31 may specifically be used for: Upon receiving the parameter submission command from the host, in response to the parameter submission command, determine at least one fragment obtained by dividing the logical block address LBA range; or, When the preset physical fragmentation period is detected, determine at least one fragment obtained by dividing the logical block address LBA range; or, When the sequential read delay of the monitored data exceeds a preset delay threshold, at least one fragment is determined by dividing the logical block address LBA range.

[0193] In one optional implementation, the fragmentation determination unit 31 can be specifically used to: obtain the address information and fragmentation specification information of the LBA interval contained in the parameter submission command by parsing the parameter submission command; and divide the LBA interval into at least one fragment according to the address information of the LBA interval and the fragmentation rule information.

[0194] In one optional implementation, the fragmentation determination unit 31 can be specifically used to: divide the LBA interval according to the physical block capacity boundary of the storage medium to obtain at least one fragment, so that the data of each fragment can be completely written into one or more complete physical blocks; based on the dispersion of the PBA corresponding to the LBA interval, adjust the fragment size according to a preset adjustment rule, and then divide the LBA interval according to the adjusted fragment size to obtain at least one fragment; the preset adjustment rule includes: the magnitude of the PBA dispersion is negatively correlated with the size of the fragment.

[0195] In one optional implementation, the data processing unit 32 may be specifically used to: read first data from the PBA mapped by the shard, cache the first data in a first buffer according to the order of the corresponding LBAs, and write the first data in the first buffer into the target PBA.

[0196] In an optional implementation, the data processing unit 32 may also be used to: write the first data in the first buffer to the target PBA while simultaneously reading the second data in the PBA mapped to the next fragment of the fragment, and cache the second data in the second buffer according to the corresponding LBA order; and write the second data in the second buffer to the target PBA.

[0197] In one alternative implementation, to enable the host to know the physical defragmentation status, the device may further include: The feedback unit is used to respond to the status query command sent by the host and provide feedback to the host on the sorting status of physical fragments; the sorting status includes: sorting progress percentage, number of completed fragments, number of skipped fragments, average dispersion before and after sorting, etc.

[0198] In one alternative implementation, the discrete cases can be determined in any of the following ways: The discrete case is determined based on the total number of discrete occurrences of all PBAs mapped by the partition and the total number of all PBAs. The discrete case is determined based on the total number of cross-physical block times of all PBAs mapped by the partition and the total number of all PBAs. The discrete case is determined based on the physical address interval distance generated by each discrete PBA mapped by the fragment.

[0199] In an optional implementation, the above-mentioned discreteness can be characterized by a degree of discreteness, and the method for determining the degree of discreteness may specifically include: The ratio of the total number of cross-physical block occurrences to the total number of cross-physical blocks is calculated as the discreteness of the partition; The ratio of the total number of cross-physical block occurrences to the total number of cross-physical blocks is calculated as the discreteness of the partition; The weighted sum of the physical address interval distances generated by each PBA in each discrete step is calculated as the discreteness of the fragment.

[0200] Furthermore, this application also provides a physical debris processing device. This device specifically includes the following functional units: The acquisition unit is used to acquire the address information and fragmentation specification information of the logical block address LBA range of the physical fragments to be reorganized. The sending unit is configured to send the address information and the fragmentation specification information to the storage device to trigger the storage device to perform the following operations: Determine at least one segment obtained by dividing the LBA interval; if the discreteness of the PBA mapped by the segment meets the preset discreteness conditions, then for the segment, perform the following: read data from the PBA mapped by the segment; sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

[0201] The apparatus provided in this application first divides the LBA interval into fragments and filters out fragments whose PBA dispersion meets the preset conditions. It then collects data on the scattered PBAs only for such fragments and writes them into the target PBA in LBA order. This makes logically continuous data in the fragments correspond to continuous physical addresses. When the host initiates a sequential read, it does not need to access multiple non-contiguous physical blocks, reducing additional NAND read commands and read amplification. This solves the problems of decreased sequential read throughput bandwidth, amplified IO latency jitter, and increased read request tail latency caused by the discretization of data physical distribution.

[0202] Example 4 Based on the same inventive concept as the foregoing embodiments of this application, Embodiment 4 of this application provides a storage device to solve the problems of decreased sequential read throughput bandwidth caused by the discretization of physical data distribution, amplified IO latency jitter in scenarios with high fragmentation, and increased tail latency of read requests.

[0203] The computing device includes a memory and a memory controller. The memory can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0204] A storage controller, coupled to a memory, is used to execute a computer program stored in the memory for performing a physical fragmentation processing method as described in Embodiment 1 of this application.

[0205] In addition to the functions described above, the storage controller can also perform other functions when executing computer programs in the memory, as detailed in the descriptions of the preceding embodiments.

[0206] In one alternative implementation, the storage device may be a computing device. The computing device may further include other components such as a display, communication components, power supply components, and audio components.

[0207] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the methods provided in the above embodiments.

[0208] Accordingly, this application also provides a computer program product, which stores instructions that, when executed by a computer, cause the computer to implement the methods provided in the above embodiments.

[0209] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for physical debris processing, characterized in that, The method includes: Determine at least one fragment obtained by dividing the logical block address LBA range; If the discreteness of the physical block address (PBA) mapped by the fragment meets the preset discrete conditions, then for the fragment, the following steps are performed: read data from the PBA mapped by the fragment; and sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

2. The method as described in claim 1, characterized in that, Determine at least one fragment obtained by dividing the logical block address LBA range, including: Upon receiving the parameter submission command from the host, in response to the parameter submission command, determine at least one fragment obtained by dividing the logical block address LBA range; or, When the preset physical fragmentation period is detected, determine at least one fragment obtained by dividing the logical block address LBA range; or, When the sequential read delay of the monitored data exceeds a preset delay threshold, at least one fragment is determined by dividing the logical block address LBA range.

3. The method as described in claim 1, characterized in that, In response to the parameter submission command, determine at least one fragment obtained by dividing the logical block address LBA range, including: By parsing the parameter submission command, the address information and fragmentation specification information of the LBA range contained in the parameter submission command can be obtained; Based on the address information of the LBA interval and the information of the fragmentation rules, the LBA interval is divided to obtain at least one fragment.

4. The method as described in claim 1, characterized in that, Determine at least one fragment obtained by dividing the logical block address LBA range, including: The LBA interval is divided into at least one fragment according to the physical block capacity boundary of the storage medium, so that the data of each fragment can be completely written into one or more complete physical blocks. Based on the dispersion of the PBA corresponding to the LBA interval, after adjusting the segment size according to the preset adjustment rule, the LBA interval is divided into at least one segment according to the adjusted segment size; the preset adjustment rule includes: the magnitude of the PBA dispersion is negatively correlated with the size of the segment size.

5. The method as described in claim 1, characterized in that, Reading data from the PBA mapped by the shard, and sorting and writing the data into the target PBA according to the order of the LBAs corresponding to the data, includes: After reading the first data from the PBA mapped by the shard, the first data is cached in the first buffer according to the order of the corresponding LBAs; Write the first data in the first buffer into the target PBA.

6. The method as described in claim 5, characterized in that, The method further includes: While writing the first data in the first buffer to the target PBA, the second data in the PBA mapped to the next fragment of the fragment is read in parallel, and the second data is cached in the second buffer according to the corresponding LBA order. Write the second data in the second buffer into the target PBA.

7. The method as described in claim 1 or 2, characterized in that, The method further includes: In response to a status query command sent by the host, the system returns feedback to the host regarding the status of physical fragmentation. The sorting status includes: sorting progress percentage, number of completed fragments, number of skipped fragments, and average dispersion before and after sorting.

8. The method as described in claim 1, characterized in that, The discrete cases are determined using any of the following methods: The discrete case is determined based on the total number of discrete occurrences of all PBAs mapped by the partition and the total number of all PBAs. The discrete case is determined based on the total number of cross-physical block times of all PBAs mapped by the partition and the total number of all PBAs. The discrete case is determined based on the physical address interval distance generated by each discrete PBA mapped by the fragment.

9. The method as described in claim 8, characterized in that: Based on the total number of discrete occurrences of all PBAs mapped by the partition and the total number of all PBAs, the discreteness of the partition is determined, including: calculating the ratio of the total number of discrete occurrences to the total number of PBAs as the discreteness of the partition; The discreteness is determined based on the total number of cross-physical blocks of all PBAs mapped by the partition and the total number of all PBAs, including: calculating the ratio of the total number of cross-physical blocks to the total number of PBAs as the discreteness of the partition; Determining the discreteness based on the physical address interval distance generated by each discrete PBA mapped by the fragment includes: calculating the weighted sum of the physical address interval distances generated by each discrete PBA, as the discreteness of the fragment.

10. A method for physical debris processing, characterized in that, The method includes: Obtain the address information and fragmentation specification information of the logical block address LBA range of the physical fragments to be reorganized; Send the address information and the fragmentation specification information to the storage device to trigger the storage device to perform the following operations: Determine at least one segment obtained by dividing the LBA interval; if the discreteness of the PBA mapped by the segment meets the preset discreteness conditions, then for the segment, perform the following: read data from the PBA mapped by the segment; sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

11. A storage controller, characterized in that, The storage controller is used to perform the method described in any one of claims 1 to 9.

12. A physical debris processing device, characterized in that, The device includes: The acquisition unit is used to acquire the address information and fragmentation specification information of the logical block address LBA range of the physical fragments to be reorganized. The sending unit is configured to send the address information and the fragmentation specification information to the storage device to trigger the storage device to perform the following operations: Determine at least one segment obtained by dividing the LBA interval; if the discreteness of the PBA mapped by the segment meets the preset discreteness conditions, then for the segment, perform the following: read data from the PBA mapped by the segment; sort and write the data into the target PBA according to the order of the LBAs corresponding to the data.

13. A storage device, characterized in that, include: Memory and memory controller, wherein, The memory is used to store computer programs; The storage controller, coupled to the memory, is configured to execute the computer program stored in the memory for performing the method according to any one of claims 1 to 9.

14. A computing device, characterized in that, include: Memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program stored in the memory for performing the method of claim 10.

15. A computer-readable storage medium storing a computer program, which, when executed by a computer, enables the implementation of the method described in any one of claims 1 to 10.

16. A computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 10.

Images