Flash translation layer optimization method based on hybrid mapping and metadata compression and computer system

Abstract

The invention discloses a flash translation layer optimization method based on hybrid mapping and metadata compression and a computer system. According to the flash translation layer optimization method, the advantages of block mapping and page mapping are combined, an innovative logic block description (LBD) structure is adopted, memory occupation is reduced, and the storage density and performance are improved; in the block mapping mode, storage is optimized through compression mapping and availability mapping, and memory consumption is reduced; in the page mapping mode, the storage efficiency is improved through sequential mapping and compression mapping strategies. According to the method, an efficient address conversion algorithm is further designed, rapid conversion between a logic page and a physical page is achieved, and high I / O performance is guaranteed. Experimental results show that the method is superior to a traditional mapping method in the aspects of memory efficiency and storage performance, is suitable for storage optimization of the high-capacity SSD, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of solid state drives (SSDs), and particularly to an optimization method for a flash translation layer (FTL) based on hybrid mapping and metadata compression and a computer system. Background Art

[0002] With the rapid development of computer technology, especially the wide application of technologies such as big data, cloud computing, and artificial intelligence, the requirements for the performance and capacity of storage systems are increasing day by day. Although traditional hard disk drives (HDDs) have certain advantages in terms of cost, their access speed and data transfer rate have become inadequate when dealing with large-scale data processing. As an alternative, solid state drives (SSDs) have gradually become the mainstream storage devices due to their lack of mechanical components, fast read and write speeds, and low power consumption, and are widely used in personal computers, servers, big data storage, and embedded systems.

[0003] Although SSDs have significant performance advantages, one of the major challenges they face is how to reduce memory consumption and storage overhead while maintaining high performance. The main working principle of an SSD is to complete data read and write operations by mapping logical addresses to physical addresses, and the management of the mapping relationship is crucial for the performance of the SSD. The flash translation layer (FTL) is a key component in an SSD and is responsible for this mapping process, and its efficiency directly affects the performance and memory usage of the SSD.

[0004] In an SSD, the mapping of logical addresses to physical addresses is the responsibility of the FTL. Traditional mapping methods mainly include page mapping and block mapping. The page mapping method maps each logical page to a physical page, which can provide high mapping accuracy but requires a large amount of memory resources. Since the storage units of an SSD are small, using pure page mapping will cause the memory requirement to increase rapidly on a large-capacity SSD, which has become a bottleneck for high-capacity storage environments (such as cloud storage).

[0005] On the other hand, block mapping maps multiple logical pages to a physical block, which has a small memory footprint but low accuracy, and may cause certain performance losses. To find a balance between performance and memory footprint, some studies have proposed a hybrid mapping scheme, that is, combining the advantages of page mapping and block mapping, and dynamically selecting different mapping methods according to the frequency of data access or other conditions.

[0006] However, existing hybrid mapping methods mostly focus on static partitioning or mapping scheduling based on simple rules, and cannot fully utilize the changing characteristics of data access. Especially when facing large-capacity and high-throughput storage systems, there are still high memory overheads and performance bottlenecks. In addition, with the gradual increase in the capacity of SSDs, traditional mapping management methods are also facing increasing challenges. How to efficiently manage this mapping information, optimize memory usage, and reduce overheads has become a key issue in improving SSD performance.

[0007] In addition, the storage and compression of metadata are also another research hotspot in current SSD technology. Since the mapping tables and related metadata in SSDs need to occupy a large amount of storage space, especially in high-density storage devices, how to effectively compress this metadata and further reduce memory occupancy has become an important topic for optimizing SSD design. Although existing metadata compression methods have achieved certain effects to some extent, most methods only target static mapping scenarios and lack support for dynamic mapping processes. Summary of the Invention

[0008] The purpose of the present invention is to overcome the deficiencies and defects of the prior art, and provide a flash translation layer optimization method and computer system based on hybrid mapping and metadata compression, aiming to optimize memory usage and mapping efficiency in a solid-state drive (SSD). The flash translation layer optimization method based on hybrid mapping combines block mapping and page mapping, introduces a flexible logical block description (LBD) structure, and an innovative address conversion algorithm. The present invention not only optimizes the memory occupancy of the SSD, but also significantly improves the storage density and scalability while ensuring high performance.

[0009] An object of the present invention is to provide a flash translation layer optimization method based on hybrid mapping and metadata compression for improving storage performance and memory efficiency in a solid-state drive, comprising the following steps:

[0010] a. Provide a logical block description, where each logical block description includes a logical block number, a flag bit, a timestamp, and a pointer to a physical block, for mapping a logical page address to a corresponding physical page;

[0011] b. In block mapping mode, allocate a compression mapping bit for each logical page to indicate whether the logical page starts from a new physical page boundary; at the same time, use an availability mapping to indicate whether each logical page is stored in block mapping mode;

[0012] c. In page mapping mode, adopt sequential mapping and compression mapping to map sequentially written logical pages, and further optimize the storage space of sequentially written pages through compression mapping;

[0013] d. Design and implement an address translation algorithm. Based on the logical page address and logical block description, by calculating the trailing bits and sequential mapping bits in the compression mapping, accurately translate the logical page address into a physical page address, thereby achieving efficient read and write operations.

[0014] Among them, in the block mapping mode, the compression mapping is used to store the physical offset of the logical page and is dynamically adjusted according to the page compression situation within the logical block: The compression mapping reduces the demand for physical space by recording the position of the logical page in the physical block according to the requirements of the page compression algorithm; when the logical page is stored at the boundary of a new physical page, the compression mapping indicates the position of the page, thereby optimizing the storage structure and avoiding memory waste.

[0015] Among them, the availability mapping is used to indicate whether each logical page uses the block mapping mode or the page mapping mode and dynamically adjusts the mapping mode according to the access frequency of the logical page; the availability mapping classifies the logical pages according to the access frequency, preferentially maps the frequently accessed data to the page mapping mode, and maps the cold data or infrequently accessed data to the block mapping mode.

[0016] Among them, the sequential mapping is used to indicate whether sequential writing is performed in the page mapping mode, and optimizes the storage of sequentially written logical pages through the compression mapping, reducing memory occupancy; when multiple logical pages are written sequentially, the system marks the pages through the sequential mapping and uses the compression mapping to compress the sequentially written pages when necessary.

[0017] Among them, the address translation algorithm includes calculating the trailing bits and sequential mapping bits of the compression mapping, and judging the page storage mode according to the availability mapping, and selecting the block mapping mode or the page mapping mode, so as to quickly locate the physical page address.

[0018] Among them, the address translation algorithm finds the position of the logical page on the physical device in the following way by checking the mapping mode of the logical page:

[0019] First, according to the logical page address, the address translation algorithm calculates the position of the logical page in the logical block and checks whether the logical page uses the block mapping mode or the page mapping mode through the availability mapping; if the block mapping mode is used, the address translation algorithm finds the physical address of the page according to the trailing bits in the compression mapping; if the page mapping mode is used, the address translation algorithm quickly locates the physical position of the sequentially written logical page through the sequential mapping and the compression mapping.

[0020] Among them, the method rationally selects the block mapping mode and the page mapping mode according to different data access patterns through a hybrid mapping strategy, and realizes the optimal allocation of memory and storage space in different scenarios.

[0021] Among them, the hybrid mapping strategy is dynamically adjusted according to the heat of the data, storing hot data in the page mapping mode and cold data in the block mapping mode.

[0022] Another object of the present invention is to provide a computer system, which optimizes the storage management and performance of a solid-state drive according to the flash translation layer optimization method based on hybrid mapping and metadata compression.

[0023] Among them, the computer system includes a hardware device for processing data storage and access requests. This hardware device implements the hybrid mapping strategy included in the flash translation layer optimization method based on hybrid mapping and metadata compression, and uses this hybrid mapping strategy to efficiently map and access logical pages in the storage device; this computer system can automatically select an appropriate mapping mode according to the access frequency of the data, and dynamically adjust the mapping strategy when needed, optimizing the system's memory usage, storage space utilization rate, and read and write performance, and realizing large-scale data storage and processing tasks.

[0024] The flash translation layer optimization method of the present invention based on hybrid mapping and metadata compression proposes a hybrid mapping strategy, which combines the advantages of block mapping and page mapping. By using block mapping for infrequently accessed data and page mapping for frequently accessed data, the effective utilization of memory and the optimization of SSD performance are achieved. The hybrid mapping strategy according to the present invention can dynamically adjust the mapping mode according to the data access frequency, reduce memory occupancy, and maintain high I / O performance.

[0025] In addition, the present invention also introduces a logical block description (LBD) data structure and designs a corresponding algorithm to optimize the conversion of logical page addresses to physical page addresses. Each logical block contains a 32-bit header information indicating its logical block number and status bits. Each logical block description also includes a pointer to the physical page and a compression mapping for processing page compression and mapping conversion.

[0026] The technical solution of the present invention effectively improves the performance of the SSD in a large-capacity storage environment and reduces the memory overhead at the same time.

[0027] The present invention has the following beneficial effects:

[0028] 1. Significantly improve memory efficiency: The present invention combines the advantages of block mapping and page mapping, dynamically selects the mapping method, thus significantly reducing the memory occupancy in the SSD. Compared with the traditional pure page mapping scheme, the memory occupancy is greatly reduced, meeting the high requirements of large-capacity SSDs for memory efficiency.

[0029] 2. Optimize Performance and Storage Density: By adopting a hybrid mapping strategy, the present invention can maintain a high I / O performance while ensuring a high storage density. Especially when using page mapping for frequently accessed data, it can keep a low access latency, meeting the requirements of high-performance computing and big data storage scenarios.

[0030] 3. Flexibly Respond to Different Data Access Patterns: By dynamically adjusting the mapping method, for data with different access frequencies, it can adaptively select block mapping or page mapping, ensuring the best performance of the storage system in various usage scenarios, especially outstanding in application scenarios such as cloud storage and mass storage.

[0031] 4. Simplify Mapping Management: By introducing a Logical Block Description (LBD) data structure, the present invention can effectively manage and organize mapping relationships, reducing the complexity of the mapping table, making the address conversion process more efficient and concise, and further improving the overall performance of the system.

[0032] 5. Be Compatible with Existing Storage Architectures: The present invention can be compatible with existing SSD architectures and can be implemented without large-scale hardware modifications, having good promotion and application value. It is especially suitable for mainstream SSD products and mass storage requirements in the current market.

[0033] 6. Enhance the Scalability of Large-capacity SSDs: The hybrid mapping strategy of the present invention not only improves storage efficiency but also enhances the scalability of large-capacity SSDs in high-performance storage applications. As the storage capacity increases, the increase in memory consumption will be effectively suppressed, ensuring the continuous and stable operation of the system under high load conditions.

[0034] 7. Support Multiple Data Storage Modes: The present invention is not only applicable to conventional data storage modes but also can play an important role in environments with high density, low latency, and large throughput, adapting to the diverse characteristics of different types of storage requirements. Brief Description of the Drawings

[0035] Figure 1 is a flowchart of an optimized method for a flash translation layer based on hybrid mapping and metadata compression according to an embodiment of the present invention.

[0036] Figure 2 shows the mapping relationship between the Logical Block Description (LBD) and the physical layer according to an embodiment of the present invention; wherein, the LBD data structure contains pointers to physical pages and stores the position information of each logical page through compressed mapping. Detailed Description of the Embodiment

[0037] The following further describes the present invention in detail with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] In an exemplary embodiment of the present application, the flash translation layer optimization method based on hybrid mapping and metadata compression is used to improve the storage performance and memory efficiency in a solid-state drive (SSD), and includes the following steps:

[0039] a. Provide a logical block description (LBD), where each logical block description includes a logical block number, a flag bit, a timestamp, and a pointer to a physical block, for mapping a logical page address to a corresponding physical page;

[0040] b. In the block mapping mode, assign a compression mapping (cm1) bit to each logical page to indicate whether the logical page starts from a new physical page boundary; at the same time, use an availability mapping (am) to indicate whether each logical page is stored in the block mapping mode;

[0041] c. In the page mapping mode, use sequential mapping (sm) and compression mapping (cm2) to map sequentially written logical pages to reduce memory occupancy, and further optimize the storage space of sequentially written pages through compression mapping;

[0042] d. Design and implement an address translation algorithm. According to the logical page address and the logical block description, by calculating the trailing bits and sequential mapping bits in the compression mapping, accurately convert the logical page address to a physical page address, so as to achieve efficient read and write operations.

[0043] In an exemplary embodiment of the present application, the provided hybrid mapping strategy includes two mapping methods: block mapping and page mapping. Among them, block mapping can be used for infrequently accessed data, and page mapping can be used for frequently accessed data. Each logical block is represented by a logical block description (LBD), and the logical block description (LBD) includes information such as a pointer to a physical page, a compression mapping, and an availability mapping. Each logical block has a logical block description (LBD), and this description structure is used to store the mapping information of the logical block. The LBD includes the following parts:

[0044] 1. Logical block number (32 bits), used to identify the logical block;

[0045] 2. 4-bit flag, indicating the status of the logical block;

[0046] 3. Physical block pointer, pointing to the physical block storing the data of the logical block;

[0047] 4. Compression mapping (cm1), used to indicate whether the logical page starts at a new physical page boundary.

[0048] In an embodiment of the present invention, the pointer in the logical block description points to the corresponding physical page according to the storage mode of the logical block. If the data adopts the block mapping method, the block pointer is used to locate the physical page; if the page mapping mode is adopted, the physical page address is located through the page mapping array (pma).

[0049] For the compression mapping (cm1), the mapping of each logical page stores additional bit information to indicate whether the page starts at a new physical page boundary. In the block mapping mode, the physical offset of the page is calculated by the number of bits in the compression mapping.

[0050] In an embodiment of the present invention, an address conversion technology is designed to convert the logical page address into a physical page address. The input of the algorithm is the logical page address (addr) and the corresponding logical block description (LBD). The algorithm first determines the offset of the logical page within the logical block through modulo operation, and judges whether the page is stored in the block mapping mode or the page mapping mode according to the availability mapping (am).

[0051] If the page is in the block mapping mode, the algorithm will calculate the physical offset of the page according to the compression mapping (cm1), and further determine the physical address of the page through the block pointer (bp1 or bp2). If the page is in the page mapping mode, the algorithm calculates the physical address through the sequential mapping (sm) and the compression mapping (cm2).

[0052] In an embodiment of the present application, when a write operation is performed, if the page is written sequentially, the corresponding sequential mapping (sm) and compression mapping (cm2) need to be updated. If the page is not written sequentially, the availability mapping (am) needs to be adjusted and the data needs to be converted from the block mapping mode to the page mapping mode.

[0053] In an exemplary embodiment of the present application, in the block mapping mode, the compression mapping is used to store the physical offset of the logical page and is dynamically adjusted according to the page compression situation within the logical block. The compression mapping records the position of the logical page in the physical block and reduces the demand for physical space according to the requirements of the page compression algorithm; when the logical page is stored at the new physical page boundary, the compression mapping will indicate the position of the page, thereby optimizing the storage structure and avoiding memory waste. In actual operation, the physical offset of the logical page may change due to page compression or data relocation. Therefore, this mapping needs to be dynamically adjusted to adapt to the changes in the storage structure and data access mode, so as to save memory without affecting the read and write performance of the system.

[0054] In an exemplary embodiment of the present application, the Availability Mapping (AM) is used to indicate whether each logical page uses the block mapping mode or the page mapping mode, and dynamically adjusts the mapping mode according to the access frequency of the logical page. Specifically, the Availability Mapping classifies logical pages according to the access frequency, preferentially maps frequently accessed data to the page mapping mode, and maps cold data or infrequently accessed data to the block mapping mode. This dynamic adjustment can effectively balance memory consumption and storage performance. By storing hot data in the page mapping mode, memory occupancy is reduced; at the same time, cold data is stored in the block mapping mode, improving storage density and write efficiency, ultimately achieving optimization of memory usage.

[0055] In an exemplary embodiment of the present application, the Sequential Mapping (SM) is used to indicate whether to write sequentially in the page mapping mode, and optimizes the storage of logically sequential pages through Compression Mapping (CM2) to reduce memory occupancy. The purpose of the Sequential Mapping is to perform efficient physical address mapping on logically sequential pages, thereby reducing computational and memory overhead. When multiple logical pages are written sequentially, the system marks these pages through the Sequential Mapping and, if necessary, uses Compression Mapping (CM2) to compress the sequentially written pages to reduce the demand for physical storage space. This method is particularly effective in large-capacity storage devices, can improve the utilization rate of storage space, reduce memory consumption, and maintain a high access efficiency at the same time.

[0056] In an exemplary embodiment of the present application, the address translation algorithm includes calculating the trailing bits of the compression mapping, the sequential mapping bits, and judging the page storage mode according to the availability mapping, and selecting the block mapping mode or the page mapping mode to quickly locate the physical page address. Specifically, the address translation algorithm determines how to quickly find the location of the page on the physical device by checking the mapping mode of the logical page. First, according to the logical page address, the algorithm calculates the position of the page in the logical block and checks whether the page uses the block mapping mode or the page mapping mode through the availability mapping. If the block mapping mode is used, the algorithm finds the physical address of the page according to the trailing bits in the compression mapping; if the page mapping mode is used, the algorithm quickly locates the physical position of the logically sequential page through the sequential mapping and the compression mapping (CM2). This conversion process can significantly reduce the amount of computation and improve the response speed of the storage system.

[0057] In the exemplary embodiments of the present application, the method has significant memory efficiency and I / O performance advantages in the large-capacity storage applications of SSDs. It can significantly reduce memory consumption while ensuring relatively high read and write performance. Specifically, the method can reasonably select block mapping mode and page mapping mode according to different data access patterns through a hybrid mapping strategy, so as to achieve optimal allocation of memory and storage space in different scenarios. Especially in high-capacity SSDs, the read and write frequencies of data vary greatly. Through this method, frequently accessed data can be stored and accessed more efficiently, while infrequently accessed data can reduce memory occupancy through the block mapping mode without affecting storage performance. Ultimately, this method can effectively reduce memory consumption while ensuring system response time and storage performance, and is applicable to large-capacity and low-cost SSD storage solutions.

[0058] In the exemplary embodiments of the present application, the hybrid mapping strategy can be dynamically adjusted according to the data heat, selectively storing hot data in the page mapping mode and cold data in the block mapping mode to improve storage density and reduce memory consumption. Specifically, the hybrid mapping strategy is adjusted according to the heat of data access. For frequently accessed data (hot data), the system selects the page mapping mode to provide more efficient access performance; while for data with lower access frequencies (cold data), the block mapping mode is selected to reduce memory occupancy and improve storage density. This strategy is dynamically adjusted according to the characteristics of different data, which can effectively reduce the memory occupancy of SSDs and improve the overall efficiency of the storage system, especially applicable to large-scale storage applications such as cloud storage and archiving that are sensitive to storage capacity and cost.

[0059] The exemplary embodiments of the present application also provide a computer system, which optimizes the storage management and performance of a solid-state drive according to the flash translation layer optimization method based on hybrid mapping and metadata compression.

[0060] In the exemplary embodiments of the present application, the computer system includes a hardware device for processing data storage and access requests. This hardware device implements the hybrid mapping strategy included in the flash translation layer optimization method based on hybrid mapping and metadata compression, and uses this hybrid mapping strategy to efficiently map and access logical pages in the storage device; this computer system can automatically select an appropriate mapping mode according to the access frequency of data and dynamically adjust the mapping strategy when needed, optimizing the memory usage, storage space utilization rate, and read and write performance of the system, and realizing large-scale data storage and processing tasks.

[0061] The foregoing has shown and described the basic principles, main features and advantages of the present invention. For a person skilled in the art, it is obvious that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and without departing from the spirit or basic features of the present invention, the present invention can be implemented in other specific forms;

[0062] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced by the present invention.

[0063] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A flash translation layer optimization method based on hybrid mapping and metadata compression, which is used to improve the storage performance and memory efficiency in a solid-state drive, and is characterized in that, The steps include the following: a. Provide logical block descriptions, where each logical block description includes a logical block number, a flag bit, a timestamp, and a pointer to a physical block, which is used to map a logical page address to the corresponding physical page; b. In the block mapping mode, assign a compression mapping bit to each logical page to indicate whether the logical page starts from a new physical page boundary; meanwhile, use an availability mapping to indicate whether each logical page is stored in the block mapping mode; c. In the page mapping mode, adopt sequential mapping and compression mapping to map sequentially written logical pages, and further optimize the storage space of sequentially written pages through compression mapping; d. Design and implement an address conversion algorithm. Based on the logical page address and the logical block description, accurately convert the logical page address to a physical page address by calculating the trailing bits and sequential mapping bits in the compression mapping, so as to achieve efficient read and write operations.

2. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 1, wherein In the block mapping mode, the compression mapping is used to store the physical offset of the logical page and is dynamically adjusted according to the page compression situation within the logical block: The compression mapping records the position of the logical page in the physical block and reduces the demand for physical space according to the requirements of the page compression algorithm; when the logical page is stored at a new physical page boundary, the compression mapping indicates the position of the page, thus optimizing the storage structure and avoiding memory waste.

3. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 1, characterized in that The availability mapping is used to indicate whether each logical page uses the block mapping mode or the page mapping mode and dynamically adjusts the mapping mode according to the access frequency of the logical page; the availability mapping classifies the logical pages according to the access frequency, maps the frequently accessed data to the page mapping mode preferentially, and maps the cold data or infrequently accessed data to the block mapping mode.

4. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 1, characterized in that The sequential mapping is used to indicate whether sequential writing is performed in the page mapping mode, and optimizes the storage of sequentially written logical pages through compression mapping, reducing memory occupancy; when multiple logical pages are written sequentially, the system marks the pages through sequential mapping and uses compression mapping to compress the sequentially written pages when necessary.

5. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 1, wherein The address conversion algorithm includes calculating the trailing bits and sequential mapping bits of the compression mapping, and judging the page storage mode according to the availability mapping, and selecting the block mapping mode or the page mapping mode to quickly locate the physical page address.

6. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 5, wherein The address conversion algorithm finds the position of the logical page on the physical device in the following way by checking the mapping mode of the logical page: First, according to the logical page address, the address conversion algorithm calculates the position of the logical page in the logical block and checks whether the logical page uses the block mapping mode or the page mapping mode through the availability mapping; if the block mapping mode is adopted, the address conversion algorithm finds the physical address of the page according to the trailing bits in the compression mapping; if the page mapping mode is adopted, the address conversion algorithm quickly locates the physical position of the sequentially written logical page through sequential mapping and compression mapping.

7. The method for optimizing a flash translation layer based on hybrid mapping and metadata compression according to claim 1, wherein The method adopts a hybrid mapping strategy, reasonably selects the block mapping mode and the page mapping mode according to different data access patterns, and realizes the optimal allocation of memory and storage space in different scenarios.

8. The flash translation layer optimization method based on hybrid mapping and metadata compression according to claim 7, characterized in that The hybrid mapping strategy is dynamically adjusted according to the data heat, storing hot data in the page mapping mode and cold data in the block mapping mode.

9. A computer system, characterized in that, The flash translation layer optimization method based on hybrid mapping and metadata compression according to any one of claims 1 to 8 optimizes the storage management and performance of the solid-state drive.

10. The computer system according to claim 9, characterized in that The computer system includes hardware devices for processing data storage and access requests. The hardware devices implement the hybrid mapping strategy included in the flash translation layer optimization method based on hybrid mapping and metadata compression, and use the hybrid mapping strategy to efficiently map and access logical pages in the storage device; The computer system can automatically select an appropriate mapping mode according to the access frequency of the data, and dynamically adjust the mapping strategy when needed, optimizing the memory usage, storage space utilization, and read / write performance of the system, and implementing large-scale data storage and processing tasks.

Images