Management of metadata in block storage system
By dividing SSDs into metadata and user data areas, migrating metadata to RAM, and forming cumulative segments for bulk writing, the solution addresses inefficiencies in metadata write operations, enhancing performance and flexibility in block storage systems.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2026-03-26
AI Technical Summary
Existing solutions for block storage systems face inefficiencies in metadata write operations, leading to reduced write performance and lack of flexibility due to the inability to group and reduce metadata writes effectively.
The solution involves dividing an SSD into metadata and user data areas, migrating valid metadata tuples to RAM, forming a cumulative metadata segment in RAM, and writing it in bulk to the SSD, thereby reducing metadata write operations by grouping them.
This approach significantly reduces metadata write operations by over 127 times, optimizing performance by minimizing redundant SSD access and metadata processing within RAM, while maintaining data integrity.
Smart Images

Figure RU2024000302_26032026_PF_FP_ABST
Abstract
Description
METHOD AND DEVICE FOR MANAGING METADATA IN A BLOCK DATA STORAGE SYSTEM AREA OF TECHNOLOGY [OOO 1] The claimed solution relates to data storage systems (DSS), in particular, to the management of metadata in a block data storage system. LEVEL OF TECHNOLOGY
[0002] When storing user data on a solid-state drive (SSD), service metadata also appears and must be preserved. For file systems, metadata includes the time of the file's last update, and for block storage systems, it includes the Cyclic Redundancy Check (CRC) of the data block. A consistent representation of the data on the SSD is formed only after both user data and metadata have been written. Although metadata can be much smaller than the minimum write block (512 or 4096 bytes), a simple implementation requires an additional write operation to update the metadata. This results in two writes to the SSD when writing a 4-kibibyte (KiB) block—one for the block itself and one for its metadata.
[0003] It's important to note that the problem of doubling or more I / O operations due to metadata is a common problem across all tiers: databases contain metadata; databases use a file system, which also contains metadata; and the file system is stored on a storage system, which also contains metadata. Collectively, metadata reduces write performance by several to tens of times. Experiments have shown that the Postgress + file system combination required up to 100 times more I / O operations than necessary, excluding storage system losses.
[0004] The proposed solution is designed for implementation in block storage systems. Block storage systems increase the reliability of user data storage, including by replacing local block devices. Block storage systems do not impose restrictions on the file system used and are used for high-performance computing systems, high-performance databases, and virtualization systems. To ensure reliability and backups, block storage systems feature checksums for user data blocks, the ability to take snapshots, and allocate disk space on demand. These capabilities necessitate metadata for user data blocks in block storage systems.
[0005] Known solutions are typically file system-oriented and therefore require adaptation for block storage systems. The first difference between file-based and block-based solutions is that a file requires a lot of metadata, while the proposed solution uses an 8-byte tuple of metadata per user data block. The second difference is the sheer number of blocks. For example, a single 4 TB SSD can hold almost a billion 4 KiB blocks. The third difference is the higher number of block operations—modern drives can perform over 200,000 block operations per second.
[0006] A prior art approach [1] is known that addresses the problem of reducing disk write operations using soft updates, when for some file system operations, one write to the SSD is sufficient to restore the integrity of the metadata.
[0007] In addition, the prior art includes a distributed replicated block device (DRBD) [2], which provides separate storage of data and metadata and work with metadata in RAM.
[0008] However, known solutions do not allow for a sufficient reduction in the number of metadata write operations for block storage systems, in particular due to the lack of the ability to increase the volume of service data and group metadata writes.
[0009] Thus, the disadvantage of the prior art is the lack of flexibility of the approach and the lack of adaptation for block storage systems, as well as an excessive number of metadata write operations. ESSENCE OF THE INVENTION
[0010] The claimed invention is aimed at eliminating the disadvantages inherent in existing solutions known from the prior art. [OOP] The technical result is a reduction in the number of metadata write operations for block storage systems.
[0012] The claimed technical result is achieved by implementing a method for managing metadata in a block data storage system consisting of RAM and a solid-state drive (SSD), comprising the following stages: • divide the SSD into consecutive volumes, each of which consists of a metadata area and a user data area, where the metadata area consists of segments with metadata and backup copies of segments with metadata, • divide the RAM into dedicated memory for loading segments from the SSD, volume arrays, and service information, with each volume containing an array of metadata tuples for all blocks in the user data area, an array of free space counters for each segment, memory for storing the highest segment version number for the current volume, and memory for a cumulative metadata segment, • migrate valid metadata tuples from each volume-specific metadata segment from SSD to RAM, where valid metadata tuples are those that do not have a copy in another segment with a higher version number, • form a cumulative segment of metadata in RAM, • update the metadata tuple in the accumulative metadata segment when writing a user block to the SSD, • write the cumulative metadata segment and its backup copy to the SSD.
[0013] In one particular example of implementation, the metadata area consists of 32 segments with metadata and 32 backup copies of segments with metadata, where each segment with metadata consists of a segment version number, a cyclic redundancy check (CRC), a signature, and 8-byte tuples with metadata, where an 8-byte tuple with metadata consists of 4 bytes of the CRC of the block with user data, 2 bytes of the block number in the user data area, 1 byte of flags, and 1 byte of block usage counters.
[0014] In another particular implementation example, the user data area consists of 32768 blocks of 4 kibibytes (KiB) each.
[0015] In another particular implementation example, after the valid metadata tuples from all segments have been migrated, the allocated memory for loading the segments is freed or reused for the next volume.
[0016] In another particular implementation example, the step of transferring valid metadata tuples from each metadata segment additionally contains steps in which: • check the CRC and signature of each segment with metadata, • if the segment with metadata does not pass the check, then the backup copy of the segment with metadata is read, • If a backup copy of a segment with metadata fails verification, the data on the volume is considered lost.
[0017] In another particular implementation example, the step of transferring valid metadata tuples from each metadata segment additionally includes a check of each of the metadata tuples for each of the 32 metadata segments, which includes the steps of: • if a free space marker is written in place of the metadata tuple, then the free space counter for the current segment is increased, • if the version number of the segment in the SSD is greater than the version number of the segment of the metadata tuple in RAM, then the data of the metadata tuple in RAM is updated, the counter of free spaces of the previously read segment is increased, • if the previously read segment has a higher version number, then the metadata tuple in the current segment is considered invalid, a free space marker is set, and the free space counter for the current segment is increased.
[0018] In another particular example of implementation, the stage of forming the cumulative metadata segment contains the stages in which: • select a segment with the number of free places in the free places counter exceeding the specified threshold, • load the selected segment into memory for the metadata storage segment, • increment the highest segment version number in memory to store the highest segment version number for the current volume, • write the incremented highest segment version number to the segment version numbers, • compile a list of free spaces in the cumulative metadata segment.
[0019] In another particular implementation example, the step of updating a metadata tuple in a cumulative metadata segment contains steps in which: • receive a request to write a block to the user data area on the SSD, • check the cumulative metadata segment for the presence of a metadata tuple corresponding to a block from the user data area that needs to be written to the SSD, • update the metadata tuple if it already exists in the metadata accumulation segment.
[0020] In another particular implementation example, if the accumulative metadata segment does not contain a tuple corresponding to a block from the user data area that needs to be written to the SSD, then: • decrease the counter of free spaces in the metadata storage segment, • write the current tuple into the cumulative metadata segment in a free place according to the list of free places in the cumulative metadata segment, • remove the recorded space from the list of free spaces in the cumulative metadata segment, • increase the counter of free spaces in the segment in which the current tuple was previously written.
[0021] In another particular implementation example, the step of writing the cumulative metadata segment to the SSD contains the steps of: • check the list of free spaces in the cumulative metadata segment, • if the list of free spaces is empty, then the cumulative metadata segment and its backup copy are written to the SSD.
[0022] The claimed technical result is also achieved by implementing a device for managing metadata in a block data storage system, containing at least one processor, at least one memory associated with the processor and containing machine-readable instructions, which, when executed by at least one processor, ensure the execution of a method for managing metadata in a block data storage system. BRIEF DESCRIPTION OF DRAWINGS
[0023] Fig. 1 shows a block diagram of a method for managing metadata in a block data storage system.
[0024] Fig. 2 shows the partitioning diagram of a solid state drive (SSD).
[0025] Fig. 3 shows the diagram of the division of RAM.
[0026] Fig. 4 shows the general diagram of the computing device. IMPLEMENTATION OF THE INVENTION
[0027] Below, concepts and terms necessary for understanding the present invention will be described.
[0028] A data storage system (DSS) is a computing system designed to store data and implement the following main functions: • providing access to data using data transfer protocols depending on the method of organizing the storage (structure) of data and the type of connection; • representation of data in the form of logical objects; • delimitation of access rights to logical objects; • maintaining access to data in the event of failure of some of the physical data storage devices (drives) used; • management of all functions of the computing system related to the organization of storage and access to data through the built-in user interface [3].
[0029] Block storage is a data storage system that provides access to stored data in the form of blocks of a certain size through external computing networks using Fibre Channel, iSCSI, or protocols with similar functionality [3].
[0030] File storage is a data storage system that provides access to stored data in the form of files through external computer networks using NFS, CIFS, SMB data transfer protocols or protocols with similar functionality [3].
[0031] Data is a reusable representation of information suitable for transmission, interpretation, or processing in a formalized manner [4].
[0032] Metadata is data that defines and describes other data [4].
[0033] A data block is a block of stored data, typically in block storage, a file system, or a database.
[0034] A file system is a description of the method of storing, distributing, naming and providing access to information stored on a machine-readable storage medium [5].
[0035] Data integrity is the property of ensuring that data has not been altered or destroyed in an unauthorized manner [6].
[0036] A kibibyte (Russian abbreviation: KiB; international: KiB) is a unit of measurement of the amount of information equal to 2 10 (1024) bytes [7].
[0037] Mebibyte (Russian designation: МиБ; international: MiB) is a unit of measurement of the amount of information equal to 220 (10242) bytes [8].
[0038] The proposed solution allows us to reduce the number of metadata write operations for block storage systems (but not writes of user data itself) by more than 127 times.
[0039] Reducing metadata write operations is achieved, in part, through a special organization of metadata storage on the SSD (a 2-byte index of the user data block is added to the metadata) and storing the entire metadata in RAM. Furthermore, twice the amount of space is allocated for metadata on the SSD, creating half-empty metadata segments that can accommodate metadata tuples and be written to the SSD in bulk. Thus, specific metadata tuples are not written to a single location, but to free spaces in the accumulative metadata segment. Once full, the accumulative segment is written to the SSD. Also, at the beginning of reading all metadata segments, it is necessary to determine which metadata version for a specific user data block is correct using the metadata segment version number.
[0040] The implementation of the claimed invention results in complete reading of metadata from the SSD at the start of operation and storage of metadata in RAM during operation. However, for high-speed block storage systems, such a solution is acceptable, since additional savings are achieved on metadata reading operations during operation, since metadata processing is carried out within RAM, thereby eliminating redundant access to the SSD.
[0041] During the implementation of the declared solution, the volume of service data also doubles, but from the total volume, taking into account user data, this is less than 1%, which is justified when the system performance is limited by the volume of input-output operations.
[0042] Fig. 1 shows a computer-implemented method (100) for managing metadata in a block data storage system consisting of random access memory and a solid-state drive (SSD). The method (100) is executed using at least one processor. In a particular example of implementation, the method (100) is executed using a device for managing metadata in the block data storage system, which can be implemented on the basis of a computing device modified in its hardware and software so as to perform the functions of a device for managing metadata in the block data storage system. A more detailed description of the computing device is disclosed below with reference to Fig. 4.
[0043] In the first stage (101), the SSD is divided into consecutive volumes, each of which consists of a metadata area and a user data area, and The metadata area consists of segments with metadata and backup copies of segments with metadata.
[0044] In a particular embodiment of the claimed solution, the metadata area consists of 32 segments with metadata and 32 backup copies of segments with metadata, where each segment with metadata consists of a segment version number, a cyclic redundancy code (CRC), a signature, and 8-byte tuples with metadata, where an 8-byte tuple with metadata consists of 4 bytes of the CRC of a block with user data, 2 bytes of the block number in the user data area, 1 byte of flags, and 1 byte of block usage counters.
[0045] Each metadata segment has a version number that determines the "freshness" of the metadata in that segment. Metadata from a segment with a more recent version, i.e., one with a higher version number, is considered valid.
[0046] In one implementation, the user data area consists of 32,768 blocks of 4 kibibytes (KiB) each. Because SSDs are divided into volumes of 32,768 blocks, each containing user data, information about which user block a particular metadata tuple belongs to (i.e., the block number in the user data area) fits in 2 bytes.
[0047] Fig. 2 shows a detailed diagram of the SSD partitioning. The disk is divided into consecutive volumes of 129 MiB, where the metadata area occupies 1 MiB and the user data area occupies 128 MiB. The 1 MiB metadata area consists of 32 segments with 16 KiB metadata and another 32 backup copies of segments with 16 KiB metadata. The 16 KiB metadata segment consists of a segment header and 2046 8-byte metadata tuples. The segment header includes 8-byte segment version numbers, a 4-byte CRC, and a 4-byte signature. Each of the 8-byte metadata tuples consists of a 4-byte CRC of the block with user data, a 2-byte block number in the user data area, a 1-byte flag, and a 1-byte block usage counters. The 128 MiB user data area consists of 32,768 4 KiB blocks. Metadata occupies 1 / 129 ~= 0.8% of the total disk space.
[0048] Due to excess space for metadata records (allocating backup copies of segments with metadata in the metadata area), there is always a segment with free space for metadata. For example, if there are two segments with three records each, and there are four records in total, then at least one segment has at least one free space. The calculated number of minimum existing free spaces in at least one segment is called the "suitable" number of free records. In the preferred The estimated number of free spaces in at least one segment for 8 bytes of metadata is 1022.
[0049] In the preferred embodiment of the claimed solution, the block storage system consists of RAM and a solid-state drive (SSD), where the SSD is chosen due to its high read / write performance. However, the claimed solution is not limited to this storage system configuration. In an alternative embodiment, the block storage system consists of RAM and a hard disk drive (HDD), or a combination of RAM, SSD, and HDD.
[0050] In the second stage (102), the RAM is divided into dedicated memory for loading segments from the SSD, volume arrays, and service information, with each volume containing an array with metadata tuples for all blocks in the user data area, an array of free space counters for each segment, memory for storing the highest segment version number for the current volume, and memory for a cumulative metadata segment.
[0051] Fig. 3 shows a detailed diagram of the RAM partitioning. RAM is divided into dedicated memory for loading segments from the SSD (1 MiB), volume arrays (~272 KiB per volume), and service information (~1 KiB). Each volume contains an array with 8-byte metadata tuples for all 32,768 blocks (256 KiB), an array of 32 2-byte free counters for each segment (64 bytes), memory for storing the highest segment version number for the current volume (8 bytes), and 16 KiB of memory for the accumulative metadata segment. Service information includes, but is not limited to, a back reference to the disk structure.
[0052] In the third stage (103), valid metadata tuples from each segment with metadata for each volume are transferred from SSD to RAM, where valid metadata tuples are tuples that do not have a copy in another segment with a higher version number.
[0053] Reading (unloading / transferring) valid metadata tuples from SSD to RAM helps reduce the number of metadata write operations.
[0054] In a particular implementation variant, after the migration (step 103) of valid metadata tuples from all segments, the allocated memory for loading segments is freed or reused for the next volume.
[0055] In a preferred embodiment of the claimed solution, the step (103) of transferring valid metadata tuples from each metadata segment additionally comprises the steps of: • check the CRC and signature of each segment with metadata, • if the segment with metadata does not pass the check, then the backup copy of the segment with metadata is read, • If a backup copy of a segment with metadata fails verification, the data on the volume is considered lost.
[0056] If the data on the volume is recognized as lost, the code "read error" is returned.
[0057] In a preferred embodiment of the claimed invention, the step (103) of transferring valid metadata tuples from each metadata segment further comprises a validation check of each of the metadata tuples for each of the 32 metadata segments, which includes the steps of: • if a free space marker is written in place of the metadata tuple, then the free space counter for the current segment is increased, • if the version number of the segment in the SSD is greater than the version number of the segment of the metadata tuple in RAM, then the data of the metadata tuple in RAM is updated, the counter of free spaces of the previously read segment is increased, • if the previously read segment has a higher version number, then the metadata tuple in the current segment is considered invalid, a free space marker is set, and the free space counter for the current segment is increased.
[0058] The integrity of the metadata segment is determined by the signature and CRC. The maximum 64-bit unsigned number 0xffffffffffffffffi^ in tuple locations is used as a free space marker. Therefore, the free space marker is assigned a value that cannot be assigned to a valid tuple, for example, 0xffffffffffffffffL, for further identification of the free space marker, for example, when checking for the presence of a free space marker in a metadata tuple location.
[0059] The same metadata segment contains both valid metadata tuples and invalid tuples. Invalid tuples arise when a metadata tuple for the same user block is written to another segment with a higher version number. The proposed solution does not erase the invalid tuple each time it is written, resulting in a reduction in redundant metadata write operations. The metadata tuple write is (semantically) invalidated in the old segment. segment (the segment with the lower version number) because a new one appeared in the segment with the higher version number.
[0060] At step (104), a cumulative metadata segment is formed in RAM. In the preferred embodiment, step (104) of forming the cumulative metadata segment comprises the following steps: • select a segment with the number of free places in the free places counter exceeding the specified threshold, • load the selected segment into memory for the metadata storage segment (i.e., designate the selected segment with free space as the metadata storage segment), • increment the highest segment version number in memory to store the highest segment version number for the current volume, • write the incremented highest segment version number to the segment version numbers, • compile a list of free spaces in the cumulative metadata segment.
[0061] Metadata segments contain both valid and invalid metadata tuples (i.e., tuples written to other segments with higher version numbers). Thus, forming a cumulative metadata segment essentially boils down to compiling a list of available slots from invalid metadata tuples.
[0062] To save RAM resources, a list of free spaces is created only for the cumulative metadata segment, while free space counters are used for the remaining metadata segments.
[0063] Savings on metadata write operations are achieved through metadata write grouping, which is implemented by creating a cumulative metadata segment and writing it to the SSD. Unlike prior art, where metadata is written to the SSD in a chaotic manner, the proposed solution groups writes by creating a cumulative metadata segment with valid metadata tuples and writing this cumulative metadata segment to the SSD.
[0064] For example, in prior art solutions, writing five metadata fragments (tuples) to five different locations on an SSD equates to five write operations. In contrast, in the proposed solution, five metadata tuples are grouped into a cumulative metadata segment and written to the SSD as a whole as a cumulative metadata segment, thereby performing only one write operation. This example clearly demonstrates the reduction in the number of operations. recording metadata for block storage systems when implementing the claimed invention, in particular, by grouping the recording.
[0065] For example, for 32,768 metadata tuples, there are 65,472 metadata segment slots, so there is always a metadata segment with 1,022 free slots that can be loaded as a cumulative metadata segment. In the worst case, writing 1,022 4KiB user data blocks results in writing a primary 16KiB metadata segment and a backup 16KiB metadata segment, i.e., 2 * 16 * 4096 / (1022 * 4096) ~= 0.8% of metadata writes per user data block write.
[0066] At step (105), the metadata tuple in the accumulative metadata segment is updated when a user block is written to the SSD. All metadata tuples in the accumulative metadata segment are valid, as the accumulative metadata segment has the highest version number. Therefore, as a result of implementing the proposed solution, only current metadata, i.e., data defining and describing the corresponding user data blocks, is written to the SSD.
[0067] In a preferred embodiment of the claimed solution, the step (105) of updating the metadata tuple in the cumulative metadata segment comprises the steps of: • receive a request to write a block to the user data area on the SSD, • check the cumulative metadata segment for the presence of a metadata tuple corresponding to a block from the user data area that needs to be written to the SSD, • update the metadata tuple if it already exists in the metadata accumulation segment.
[0068] In a particular implementation example, updating a metadata tuple, if it already exists in the storage metadata segment, is performed by copying a new metadata tuple to the same location, corresponding to a block from the user data area that needs to be written to the SSD.
[0069] In an alternative embodiment of the claimed solution, if the accumulative metadata segment does not contain a tuple corresponding to a block from the user data area that needs to be written to the SSD, then: • decrease the counter of free spaces in the metadata storage segment, • write the current tuple into the cumulative metadata segment in a free place according to the list of free places in the cumulative metadata segment, remove the recorded space from the list of free spaces in the cumulative metadata segment, • increase the counter of free spaces in the segment in which the current tuple was previously written.
[0070] At step (106), the cumulative metadata segment and its backup copy are written to the SSD.
[0071] In one embodiment of the claimed solution, the stage (106) of recording the cumulative metadata segment on the SSD contains stages in which: • check the list of free spaces in the cumulative metadata segment, • if the list of free spaces is empty, then the cumulative metadata segment and its backup copy are written to the SSD.
[0072] The absence of entries in the free space list indicates that the storage metadata segment is completely filled with valid metadata tuples and is ready to be written to the SSD.
[0073] Fig. 4 shows a general view of a computing device (200), on the basis of which a device for managing metadata in a block data storage system can be implemented, ensuring the implementation of a method for managing metadata in a block data storage system consisting of RAM and SSD.
[0074] In general, the computing device (200) comprises one or more processors (201) connected by a common information exchange bus, memory means such as RAM (202) and ROM (203), input / output interfaces (204), input / output devices (205), and means for network interaction (206).
[0075] The processor (201) (or several processors, a multi-core processor) can be selected from a range of devices that are widely used today, such as those from Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, etc. A graphics processor can also be used as the processor (201), such as those from Nvidia, AMD, Graphcore, etc. The processor (201) has a hierarchy of several cache levels (LI, L2, L3).
[0076] RAM (202) is random access memory (RAM) and is designed to store machine-readable instructions executed by the processor (201) to perform the necessary logical data processing operations. RAM (202) typically contains executable instructions from the operating system and corresponding software components (applications, software modules, etc.).
[0077] ROM (203) represents one or more permanent storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), a flash drive memory (EEPROM, NAND, etc.), optical storage media (CD-R / RW, DVD-R / RW, Blu-Ray Disc, MD), etc.
[0078] To organize the operation of the device components (200) and to organize the operation of external connected devices, various types of I / O interfaces (204) are used. The choice of the appropriate interfaces depends on the specific design of the computing device, which may include, but are not limited to: PCI, AGP, PS / 2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS / Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.
[0079] To ensure user interaction with the computing device (200), various I / O information devices (205) are used, for example, a keyboard, a display (monitor), a touch display, a touchpad, a joystick, a mouse, a light pen, a stylus, a touch panel, a trackball, speakers, a microphone, augmented reality tools, optical sensors, a tablet, light indicators, a projector, a camera, biometric identification tools (a retinal scanner, a fingerprint scanner, a voice recognition module), etc.
[0080] The network interaction means (206) ensures the transmission of data by the device (200) via an internal or external computer network, for example, the Intranet, the Internet, a LAN, etc. One or more means (206) may be, but are not limited to: an Ethernet card, a GSM modem, a GPRS modem, an LTE modem, a 5G modem, a satellite communication module, an NFC module, a Bluetooth and / or BLE module, a Wi-Fi module, etc.
[0081] Additionally, the device (200) may also use satellite navigation tools, such as GPS, GLONASS, BeiDou, Galileo.
[0082] The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology.
[0083] Sources of information: [1] Marshall Kirk McKusick et al. "Soft Updates: A Technique for Avoiding Most Synchronous Writes in a Fast File System," Open Systems. DBMS, No. 07-08 (1999). https: / / www.osp.ru / os / 1999 / 07-08 / 179904 [2] Internal structure of DRBD: algorithms for operating fault-tolerant storage / Habr. https: / / habr.com / ru / companies / flant / articles / 733770 / [3] PNST 807-2022 Data storage systems. Terms and definitions [4] GOST R ISO / IEC 11179-1-2010 Information technology (IT). Metadata registries (MDR). Part 1. Basic provisions [5] GOST R 57429-2017 Forensic computer-technical examination. Terms and definitions [6] GOST 33707-2016 Information technology. Dictionary [7] Kibibyte - Wikipedia, https: / / ru.wikipedia.org / wiki / KH6H6afiT [8] Mebibyte - Wikipedia. https: / / ru.wikipedia.org / wiki / Me6H6aftT
Claims
PC17RU2024 / 000302 FORMULA 1. A method for managing metadata in a block storage system consisting of RAM and a solid-state drive (SSD), comprising the steps of: • divide the SSD into consecutive volumes, each of which consists of a metadata area and a user data area, where the metadata area consists of segments with metadata and backup copies of segments with metadata, • divide the RAM into dedicated memory for loading segments from the SSD, volume arrays, and service information, with each volume containing an array of metadata tuples for all blocks in the user data area, an array of free space counters for each segment, memory for storing the highest segment version number for the current volume, and memory for a cumulative metadata segment, • migrate valid metadata tuples from each volume-specific metadata segment from SSD to RAM, where valid metadata tuples are those that do not have a copy in another segment with a higher version number, • form a cumulative segment of metadata in RAM, • update the metadata tuple in the accumulative metadata segment when writing a user block to the SSD, • write the cumulative metadata segment and its backup copy to the SSD.
2. The method according to paragraph 1, characterized in that the metadata area consists of 32 segments with metadata and 32 backup copies of segments with metadata, wherein each segment with metadata consists of a segment version number, a cyclic redundancy code (CRC), a signature and 8-byte tuples with metadata, wherein an 8-byte tuple with metadata consists of 4 bytes of the CRC of a block with user data, 2 bytes of the block number in the user data area, 1 byte of flags and 1 byte of block usage counters.
3. The method according to paragraph 1, characterized in that the user data area consists of 32768 blocks of 4 kibibytes (KiB) each.
4. The method according to any one of paragraphs 1-3, characterized in that after transferring valid metadata tuples from all segments, the allocated memory for loading segments is freed or reused for the next volume.
5. The method according to paragraph 4, characterized in that the stage of transferring valid metadata tuples from each segment with metadata additionally comprises stages in which: • check the CRC and signature of each segment with metadata, • if the segment with metadata does not pass the check, then the backup copy of the segment with metadata is read, • If a backup copy of a segment with metadata fails verification, the data on the volume is considered lost.
6. The method according to paragraph 4, characterized in that the stage of transferring valid metadata tuples from each segment with metadata additionally comprises checking each of the metadata tuples for each of the 32 metadata segments, which includes the stages in which: • if a free space marker is written in place of the metadata tuple, then the free space counter for the current segment is increased, • if the version number of the segment in the SSD is greater than the version number of the segment of the metadata tuple in RAM, then the data of the metadata tuple in RAM is updated, the counter of free spaces of the previously read segment is increased, • if the previously read segment has a higher version number, then the metadata tuple in the current segment is considered invalid, a free space marker is set, and the free space counter for the current segment is increased.
7. The method according to paragraph 4, characterized in that the stage of forming the cumulative segment of metadata contains stages in which: • select a segment with the number of free places in the free places counter exceeding the specified threshold, • load the selected segment into memory for the cumulative metadata segment, • increment the highest segment version number in memory to store the highest segment version number for the current volume, • write the incremented highest segment version number to the segment version numbers, • compile a list of free spaces in the cumulative metadata segment.
8. The method according to paragraph 7, characterized in that the step of updating the metadata tuple in the cumulative metadata segment comprises the steps of: • receive a request to write a block to the user data area on the SSD, • check the cumulative metadata segment for the presence of a metadata tuple corresponding to a block from the user data area that needs to be written to the SSD, • update the metadata tuple if it already exists in the metadata accumulation segment.
9. The method according to paragraph 8, characterized in that if the accumulative metadata segment does not contain a tuple corresponding to a block from the user data area that needs to be written to the SSD, then: • decrease the counter of free spaces in the metadata storage segment, • write the current tuple into the cumulative metadata segment in a free place according to the list of free places in the cumulative metadata segment, • remove the recorded space from the list of free spaces in the cumulative metadata segment, • increase the counter of free spaces in the segment in which the current tuple was previously written.
10. The method according to paragraph 7, characterized in that the step of recording the cumulative metadata segment on the SSD comprises the steps of: • check the list of free spaces in the cumulative metadata segment, • if the list of free spaces is empty, then the cumulative metadata segment and its backup copy are written to the SSD.
11. A device for managing metadata in a block data storage system, comprising at least one processor, at least one memory associated with the processor and containing machine-readable instructions that, when executed by at least one processor, ensure the execution of the method according to any one of paragraphs 1-10.
Citation Information
Patent Citations
Data storage system
RU2824327C1
Method and system for storage device metadata management and processing
US10445231B2
Managing metadata for data blocks used in a deduplication system
US20090327625A1
Metadata for data storage array
US20100280998A1
Managing metadata
US8095577B1