A data storage method, a distributed storage system, an apparatus, a medium and a product

Abstract

The application discloses a data storage method, a distributed storage system, a device, a medium and a product, and relates to the technical field of data processing. A hard disk persistent memory area is bound to a remote access network card in advance, so that neither receiving data nor flushing needs to use host memory, and consumption and occupation of host memory bandwidth are reduced. A target data block allocated to the hard disk persistent memory area is determined based on the relationship between the data length of target data and a preset data block size, small IO aggregation is performed in the hard disk persistent memory area, bandwidth consumption caused by data copy storage and transmission is not needed, bandwidth consumption is saved, the storage space of aggregated small IO data is increased, and storage cost is reduced. When the target data block reaches an aggregation condition, the aggregated target data block is written into a hard disk, the aggregated target data block is written into the hard disk after the aggregation meets the EC aggregation condition, write amplification of the hard disk is reduced, and the performance of writing into the hard disk is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data storage method, a distributed storage system, an apparatus, a medium, and a product. Background Technology

[0002] Conventional storage devices, during input / output (IO) operations, first aggregate a large number of random, small read / write requests in memory or cache into larger read / write IOs before writing them to disk all at once. Data persistence is primarily achieved through host-side cache pools in the form of data copies. However, host-side caches have limited storage space and increase storage costs while also increasing bandwidth consumption, making it difficult to meet the demands of aggregating numerous small IOs. Furthermore, buffering introduces additional data latency, leading to reduced performance when writing to disk.

[0003] Therefore, how to meet the high demand for small IO aggregation while improving write performance is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a data storage method, distributed storage system, device, medium, and product to solve the problems of high data bandwidth consumption, difficulty in meeting demand, and low write performance during conventional small IO aggregation.

[0005] To address the aforementioned technical problems, the present invention provides a data storage method, comprising:

[0006] The persistent memory area of ​​the hard disk is pre-bound to the remote access network card to respond to write requests and obtain the target data corresponding to the write request;

[0007] The target data block to be allocated to the persistent memory area of ​​the hard disk is determined based on the relationship between the data length of the target data and the preset data block size.

[0008] When the target data block meets the aggregation condition, the target data block is aggregated to obtain an aggregated target data block, and the aggregated target data block is written to the hard disk.

[0009] On the one hand, binding the hard drive's persistent memory area to the remote access network card includes:

[0010] The first data block is pre-divided within the persistent memory area of ​​the hard disk according to a preset data block size;

[0011] Bind the memory region of the first data block to the remote access network card to allocate the acquired target data into the first data block.

[0012] On the other hand, determining the target data blocks allocated to the persistent memory area of ​​the hard disk based on the relationship between the data length of the target data and the preset data block size includes:

[0013] Obtain the offset of the current free data block in the persistent memory region of the hard disk;

[0014] The target data block to accommodate the target data is determined based on the relationship between the data length of the target data, the offset of the current free data block, and the preset data block size.

[0015] On the other hand, determining the target data block to accommodate the target data based on the relationship between the data length of the target data, the offset of the current free data block, and the preset data block size includes:

[0016] When the offset of the current free data block is zero, the first target data block is determined according to the relationship between the data length of the target data and the preset data block size;

[0017] When the offset of the current free data block is non-zero, determine the first data length of the current free data block to accommodate the target data;

[0018] The second target data block is determined based on the relationship between the first data length of the target data and the preset data block size.

[0019] On the other hand, determining the second target data block based on the relationship between the first data length of the target data and the preset data block size includes:

[0020] The second data length of the target data is determined based on the first data length of the target data;

[0021] The second target data block is determined based on the relationship between the second data length and the preset data block size.

[0022] On the other hand, determining the first target data block based on the relationship between the data length of the target data and the preset data block size includes:

[0023] Divide the length of the target data by the preset data block size;

[0024] If the data obtained by division is an integer, then the integer is used as the number of the first target data blocks;

[0025] If the data obtained by division is a decimal, then the integer obtained by rounding the decimal is used as the number of the first target data blocks.

[0026] On the other hand, after determining the first target data block, the following is also included:

[0027] Based on the target data, query the persistent memory area of ​​the hard disk corresponding to the erasure code stripe;

[0028] If no persistent memory area on the hard disk corresponding to the erasure code stripe is found, it is determined that the erasure code stripe corresponding to the target data is written to the first target data block for the first time.

[0029] Allocate memory addresses for persistent memory regions on the hard disk for erasure codes corresponding to the target data.

[0030] On the other hand, the method also includes:

[0031] Retrieve the verification data block corresponding to the erasure code segment;

[0032] When the target data block reaches the aggregation condition, the control unit determines the verification data of the target data and writes the verification data to the hard disk.

[0033] On the other hand, when the target data block experiences an abnormal restart, the method further includes:

[0034] When an abnormal restart occurs in the target data block, the first target data block corresponding to the abnormal restart is determined;

[0035] Scan the target data in the persistent memory region of the hard disk;

[0036] Recover the target erasure code segments that were not previously downloaded to the hard drive;

[0037] Send data to the remaining target data blocks other than the first target data block to determine the missing data in the first target data block;

[0038] Perform data recovery processing on the missing data.

[0039] On the other hand, sending data to the remaining target data blocks other than the first target data block to determine the missing data in the first target data block includes:

[0040] Control the remaining target data blocks to query the persistent memory region of the hard disk to see if there are target error correction code stripes;

[0041] If it exists, then obtain the data corresponding to the target error correction code strips in the remaining target data blocks as the missing data;

[0042] If it does not exist, the data corresponding to the target error correction code stripe is read from the hard disk as the missing data.

[0043] On the other hand, data recovery processing is performed on the missing data, including:

[0044] If the data returned by the verification data block contains the data of the first target data block, then respond and read the data of the first target data block and write it to the hard disk;

[0045] If the data returned by the verification data block is verification data, then the data of the first target data block is determined by striping the target erasure code and written to the hard disk.

[0046] To address the aforementioned technical problems, the present invention also provides a distributed storage system, including a master node and slave nodes; the master node and slave nodes are connected, and the master node is used to execute the steps of the data storage method described above, so as to store the target data corresponding to the write request of the master node to the hard disk.

[0047] To address the aforementioned technical problems, the present invention also provides a data storage device, comprising:

[0048] Memory, used to store computer programs;

[0049] A processor for implementing the data storage method as described above when executing the computer program.

[0050] To address the aforementioned technical problems, the present invention also provides a non-volatile storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the data storage method described above.

[0051] To address the aforementioned technical problems, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the data storage method.

[0052] The beneficial effects of this invention are as follows: First, by pre-binding the persistent memory area of ​​the hard disk to the remote access network card, the data corresponding to the write request is received in the persistent memory area of ​​the hard disk. When flushing data, the write request sent to the hard disk carries the memory address of the persistent memory area, so that neither receiving data nor flushing data requires the use of host memory, reducing host memory bandwidth consumption and occupation. Second, the target data block allocated to the persistent memory area of ​​the hard disk is determined based on the relationship between the data length of the target data and the preset data block size. Compared with conventional technical solutions that separately configure a cache pool for data copy storage, increasing the cost of distributed storage, this invention performs small IO aggregation within the persistent memory area of ​​the hard disk, eliminating the need for data copy storage and bandwidth consumption from transmission, saving bandwidth consumption while increasing the storage space for aggregated small IO data and reducing storage costs. Finally, when the target data block meets the aggregation condition, the target data block is aggregated to obtain the aggregated target data block, and the aggregated target data block is written to the hard disk. This achieves writing to the hard disk after the aggregation meets the EC aggregation condition, reducing hard disk write amplification and improving the performance of writing to the hard disk.

[0053] The process of binding the persistent memory area of ​​the hard disk to the remote access network card eliminates the need for host memory in receiving data and writing it to the hard disk. This reduces host memory bandwidth usage, saves bandwidth consumption, and lowers memory costs. After a data block is accommodated by the previous write request, there is still remaining memory space within that block. This block is designated as the current free data block. After obtaining the offset of this free data block, the final target data block is determined based on the target data length, offset, and preset block size. This saves memory space in the persistent memory area of ​​the hard disk while fully utilizing the pre-set memory space of the data blocks, improving the memory utilization rate of the persistent memory area. The process of determining the target data block and free data block occupied by the target data, whether the offset of the current free data block is zero or non-zero, ensures reasonable space occupancy in the persistent memory area of ​​the hard disk while improving memory utilization and saving memory resources. In the process of determining the number of target data blocks, excluding cases where the offset is occupied by a free data block, a new data block is created for its use. This means that the target data needs to be written to multiple data blocks, ensuring that the entire target data writing process maximizes memory utilization.

[0054] In addition, the present invention also provides a distributed storage system, a data storage device, a non-volatile storage medium, and a computer program product, which have the same beneficial effects as the data storage method described above. Attached Figure Description

[0055] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of a typical small I / O aggregation scheme;

[0057] Figure 2 A flowchart illustrating a data storage method provided in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of a verification data block calculation process provided in an embodiment of the present invention;

[0059] Figure 4 A data storage method provided in this embodiment of the invention corresponds to a processing flowchart of a data block and a verification data block;

[0060] Figure 5 A schematic diagram of a process for writing to multiple data blocks is provided in an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram illustrating data recovery processing when a data block process experiences an abnormal restart, as provided in an embodiment of the present invention.

[0062] Figure 7 A structural diagram of a data storage device provided in an embodiment of the present invention;

[0063] Figure 8 This is a structural diagram of a data storage device provided in an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0065] The core of this invention is to provide a data storage method, distributed storage system, device, medium, and product to solve the problems of high data bandwidth consumption, difficulty in meeting demand, and low write performance during conventional small IO aggregation.

[0066] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Quad-Level Cell Solid-State Drive (QLC SSD) is a relatively new type of storage device in the current storage technology field. It increases storage density and reduces costs by storing four bits of data in a single storage cell.

[0068] As the capacity of Solid-State Drive NAND Flash Memory (SSD NAND) increases, the Indirection Units (IU) table also increases. For example, 4KB IU requires approximately 1GB of table space per TB. To reduce table space requirements and lower the cost of Dynamic Random Access Memory (DRAM), the IU size can be increased, such as to 8KB and 16KB. However, writes smaller than IU result in greater write amplification factor (WAF). Write amplification is the ratio of the actual amount of data written to the NAND to the amount written to the host, and is used to measure the degree of write amplification in SSDs.

[0069] QLC SSDs typically use 16K IU (Indirect Addressing Units) to manage storage space. Writing 4KB of data results in a 4x write amplification, meaning the actual amount of data written is four times the requested amount. This not only reduces write performance but also increases SSD wear. To address this issue, a method is needed to aggregate small I / O operations, merging them into larger data blocks before writing to the QLC SSD.

[0070] Figure 1 This is a schematic diagram of a typical small I / O aggregation scheme, such as... Figure 1As shown, Single-Level Cell (SLC) / Triple-Level Cell (TLC) SSDs are used as the cache pool. Small I / O requests are first written to the cache pool, and then aggregated into large I / O requests before being written to the data pool composed of QLC SSDs, reducing write amplification of QLC. The distributed storage protocol layer sends small I / O write requests to the primary Object Storage Daemon (OSD) with 3 replicas in the cache pool. The cache pool uses SLC / TLC SSDs with an IU (Indirect Addressing Unit) size of 4KB, resulting in relatively small write amplification. Small I / O requests generally use 3 replicas for data persistence instead of erasure coding (EC) to avoid data not filling the EC stripe size, which would lead to write amplification. The primary OSD forwards the write request to the two backup OSDs. The primary OSD receives the responses from the two backup OSDs and returns a response to the protocol layer. The primary OSD determines that the small I / O aggregation fills the EC stripe size, and in the background, it uses 4+2 EC redundancy to write to the data pool composed of QLC SSDs. For example, if the EC unit size is 32KB, the EC stripe size is 4*32=128KB. Once the total size reaches 128KB, it can be written to the data pool. After the data is written to the data pool, the main OSD deletes the flushed data from the cache pool and releases the cache pool space.

[0071] exist Figure 1 In the previous method, a separate cache pool was required for small I / O aggregation, which increased the cost of the distributed storage system. Data was first written to three replicas, tripling the network bandwidth, and then written to EC 4+2 redundancy, tripling the network bandwidth again, for a total network bandwidth amplification of six times, resulting in significant network bandwidth consumption. The data storage method provided by this invention can solve the above-mentioned technical problems.

[0072] Figure 2 A flowchart of a data storage method provided in an embodiment of the present invention is shown below. Figure 2 As shown, the method includes:

[0073] S11: Pre-bind the hard disk persistent memory area to the remote access network card to respond to write requests and obtain the target data corresponding to the write request;

[0074] S12: Determine the target data block to be allocated to the persistent memory area of ​​the hard disk based on the relationship between the data length of the target data and the preset data block size;

[0075] S13: When the target data block meets the aggregation condition, the target data block is aggregated to obtain the aggregated target data block, and the aggregated target data block is written to the hard disk.

[0076] Specifically, the persistent memory region of the hard disk is an independent non-volatile storage area located inside the SSD controller. It is addressed by bytes and data can still be retained even when power is lost. The Remote Direct Memory Access Network Interface Card (RDMA) allows the network card to directly read and write remote memory. Traditionally, the host Dynamic Random Access Memory (DRAM) is registered as a Mode Register (MR) and then copied to the SSD, resulting in DRAM bandwidth consumption and CPU intervention. This embodiment exposes the persistent memory region of the Perpendicular Magnetic Recording Hard Disk (PMR) as a contiguous physical address space through the controller. The driver creates a dedicated MR for this PMR in the kernel and then registers it with the RDMA network card. In this way, the data corresponding to the received write request can be directly sent to the PMR memory. When flushing data, the write request sent to the hard disk can carry the RMR memory address, without using host memory, thus reducing host memory bandwidth consumption.

[0077] In step S12, the target data blocks allocated to the persistent memory area of ​​the hard disk are determined based on the relationship between the target data length and the preset data block size. It should be noted that the number of preset data blocks to be used can be directly determined based on the target data length, and this number is taken as the number of target data blocks. Additionally, it can be considered that each preset data block will have some free memory space during the data writing process. Ideally, all preset data blocks should be unused, but in reality, some memory may be used, leaving the remaining memory space for the target data. Therefore, the number of target data blocks can be set according to the situation and is not limited here.

[0078] If it is the latter case, then it is necessary to check how much memory space is left inside the data block that can currently hold the target data, so as to split the target data into the free data blocks.

[0079] In step S13, when the target data block meets the aggregation condition, the target data block is aggregated. The aggregation condition here corresponds to the EC stripes formed by the target data blocks satisfying the storage requirements of EC units. Therefore, multiple target data blocks need to be aggregated to obtain the aggregated target data block, which is then flushed to the hard drive. It should be noted that the aggregation process in this embodiment can be the same as the conventional small IO aggregation process, or a new aggregation method can be used. No limitation is made here; it can be set according to the actual situation. Regarding the hard drive, it can be a four-layer solid-state drive, or a five-layer or six-layer drive, etc., without limitation.

[0080] The beneficial effects of this invention are as follows: First, by pre-binding the persistent memory area of ​​the hard disk to the remote access network card, the data corresponding to the write request is received in the persistent memory area of ​​the hard disk. When flushing data, the write request sent to the hard disk carries the memory address of the persistent memory area, so that neither receiving data nor flushing requires the use of host memory, reducing host memory bandwidth consumption and occupation. Second, the target data block allocated to the persistent memory area of ​​the hard disk is determined based on the relationship between the data length of the target data and the preset data block size. Compared with the conventional technical solution of configuring a separate cache pool for data copy storage, which increases the cost of distributed storage, this invention performs small IO aggregation within the persistent memory area of ​​the hard disk, eliminating the need for data copy storage and bandwidth consumption caused by transmission. This saves bandwidth consumption while increasing the storage space for aggregated small IO data, reducing storage costs. Finally, when the target data block meets the aggregation condition, the target data block is aggregated to obtain the aggregated target data block, and then written to the hard disk. This achieves writing to the hard disk after the aggregation meets the EC aggregation condition, reducing hard disk write amplification and improving the performance of writing to the hard disk.

[0081] In some embodiments, binding a persistent memory region of the hard disk to a remote access network interface card includes:

[0082] The first data block is pre-divided within the persistent memory area of ​​the hard disk according to a preset data block size;

[0083] Bind the memory region of the first data block to the remote access network card so that the acquired target data can be allocated into the first data block.

[0084] Specifically, the nodes in the distributed storage cluster directly register the persistent memory area of ​​the hard drive exposed on the hard drive to the RDMA network card, so that the entire process of network reception-PMR-in-disk DMA down-fetching bypasses the host DRAM, realizing a data path with zero copy and zero host memory bandwidth.

[0085] The first data block, which is pre-divided in the persistent memory area of ​​the hard disk according to the preset data block size, is bound to the RDMA network card. In other words, the received data (the target data in response to the write request) is allocated to the first data block, so as to realize direct access to PMR memory.

[0086] The process of binding the hard disk persistent memory area to the remote access network card provided in this embodiment makes it possible to receive data and refresh the hard disk without using the host memory, thereby reducing the bandwidth usage of the host memory, saving bandwidth consumption, and reducing memory costs.

[0087] In some embodiments, determining the target data block to be allocated to the persistent memory region of the hard disk based on the relationship between the data length of the target data and a preset data block size includes:

[0088] Obtain the offset of the current free data block in the persistent memory region of the hard disk;

[0089] The target data block to accommodate the target data is determined based on the relationship between the target data length, the offset of the current free data block, and the preset data block size.

[0090] Specifically, to conserve memory space in the hard disk's persistent memory area and prevent wasted memory space within data blocks, the target data block is determined based on the relationship between the target data length and the preset data block size. In other words, the target data block is located in the hard disk's persistent memory area, avoiding the use of memory space within the cache pool.

[0091] It should be noted that the current free data block is obtained from the free data blocks in the hard disk's persistent memory area. The order of obtaining the free data blocks can be sorted from largest to smallest offset, or according to the order in which the free data blocks were used, etc. There is no limitation here.

[0092] If the offset of the current free data block is large, it means that the remaining memory space of the current free data block is small; conversely, if the offset is small, the remaining memory space is large.

[0093] Based on the target data length, the offset of the current free data block, and the preset data block size, the target data block to accommodate the target data is determined. Here, the target data is first stored in the current free data block corresponding to the offset of the current free data block. If there are multiple current free data blocks, the memory space of the new data block is opened after the current free data blocks are filled.

[0094] This embodiment takes into account that after the data block is accommodated by the previous write request, there is still a remaining memory space in the data block. This data block is taken as the current free data block. After obtaining the offset of the free data block, the final target data block is determined based on the data length of the target data, the offset, and the preset data block size. This saves memory space in the hard disk persistent memory area while making full use of the memory space of the preset data block, thereby improving the memory space utilization rate of the hard disk persistent memory area.

[0095] In some embodiments, determining the target data block to accommodate the target data based on the relationship between the data length of the target data, the offset of the current free data block, and the preset data block size includes:

[0096] When the offset of the current free data block is zero, the first target data block is determined based on the relationship between the data length of the target data and the preset data block size;

[0097] When the offset of the current free data block is non-zero, determine the first data length of the current free data block to accommodate the target data;

[0098] The second target data block is determined based on the relationship between the first data length of the target data and the preset data block size.

[0099] Specifically, when the offset of the current free data block is zero, it means that the current free data block is a brand new data block that has not been added with any data. This means that the data corresponding to the previous write request has completely filled the memory space of the data block, and a new data block needs to be opened, namely the current free data block. Therefore, the first target data block can be determined directly based on the relationship between the data length of the target data and the preset data block size.

[0100] When the offset of the current free data block is non-zero, it means that the data corresponding to the previous write request did not fill the entire data block. It is necessary to determine the first data length that the current free data block can accommodate the target data. It should be noted that the number of current free data blocks can be one or more. In the case of multiple free data blocks, the offsets of all free data blocks are included. At this time, the remaining data length is determined by the total length of the target data and the first data length.

[0101] The second target data block is determined based on the relationship between the first data length of the target data and the preset data block size. Here, the second target data block is the remaining data block excluding the free data block.

[0102] This embodiment provides a specific process for determining the target data block and free data block occupied by the target data when the offset of the current free data block is zero or non-zero. This ensures reasonable space occupancy of the hard disk's persistent memory area while improving memory space utilization and saving memory resources.

[0103] In some embodiments, determining a second target data block based on the relationship between a first data length of the target data and a preset data block size includes:

[0104] Determine the second data length of the target data based on the first data length of the target data;

[0105] The second target data block is determined based on the relationship between the second data length and the preset data block size.

[0106] The second data length of the target data is obtained by subtracting the first data length from the total data length of the target data. The number of second target data blocks is determined by the relationship between the second data length and the preset data block size.

[0107] In the process of determining the number of second target data blocks provided in this embodiment, if the offset occupied by the idle data block is excluded, a new data block is opened for occupation. That is, the target data needs to be written into multiple data blocks to ensure that the memory space utilization rate is improved throughout the target data writing process.

[0108] In some embodiments, determining a first target data block based on the relationship between the data length of the target data and a preset data block size includes:

[0109] Divide the length of the target data by the preset data block size;

[0110] If the data obtained by division is an integer, then the integer is used as the number of the first target data block;

[0111] If the data obtained by division is a decimal, then the integer obtained by rounding the decimal is used as the number of the first target data blocks.

[0112] Specifically, in this embodiment, the offset of the current free data block is 0, indicating that a new data block needs to be opened to accommodate the target data. The length of the target data needs to be divided by the preset data block size. If the result is an integer, it means that the target data corresponding to the current write request can be perfectly accommodated in multiple data blocks. If the result is a decimal, it means that there will be an offset during the last data block's accommodation process. In this case, the number of first target data blocks corresponding to the target data needs to be rounded up. For example, if the decimal is 4.6, it is rounded up to 5, meaning 5 first target data blocks. The offset of the last first target data block needs to be recorded so that it can be used as the offset of the current free data block when allocating target data for the next write request, thus accommodating the target data of the next write request in the remaining memory space.

[0113] In this embodiment, the offset of the current free data block is 0. The process of determining the first target data block ensures that all target data of the target data block is written during the writing process, thereby improving the reliability of data writing. No additional transmission path and data are required, thus improving writing performance.

[0114] In some embodiments, after determining the first target data block, the method further includes:

[0115] Based on the target data, query the persistent memory area of ​​the hard disk corresponding to the erasure code stripes;

[0116] If the disk persistent memory area corresponding to the erasure code stripe is not found, then the erasure code stripe corresponding to the target data is written to the first target data block for the first time.

[0117] Allocate memory addresses for persistent memory regions on the hard disk for erasure codes corresponding to the target data.

[0118] Specifically, the disk persistent memory region corresponding to the erasure code stripe corresponding to the target data is queried based on the offset of the erasure code stripe. If it is not found, it means that the erasure code stripe corresponding to the target data is being written to the first target data block for the first time. At this time, it is necessary to allocate the memory address of the disk persistent memory region for the erasure code stripe corresponding to the target data.

[0119] OSD needs to manage the storage and recovery of data blocks. It queries the memory address corresponding to the EC stripe based on the internal object identification (ID) and the offset within the stripe. If it cannot find the address, it needs to allocate memory for the EC stripe.

[0120] In this embodiment, if the memory region corresponding to the erasure code stripe of the target data is not found, the corresponding memory region needs to be allocated for the erasure code stripe. By allocating this memory, the memory region where the target data is located can be quickly located without performing a complex search operation each time. This also reduces the context switching of the operating system during memory allocation and release, thereby improving overall performance.

[0121] In some embodiments, the method further includes:

[0122] Retrieve the verification data block corresponding to the erasure code segment;

[0123] When the target data block meets the aggregation condition, the control check data block determines the check data of the target data and writes the check data to the hard disk.

[0124] In this invention, the OSD (On-Site Data Process) determines that the target data block meets the aggregation condition, performs aggregation processing on the target data block, and writes it to the hard disk. Simultaneously, the checksum process also needs to be considered. If the target data block meets the aggregation condition, checksum data is determined. This checksum data can be calculated using an erasure coding algorithm. The specific calculation formula and method depend on the erasure coding algorithm used; for example, the Reed-Solomon algorithm generates checksum data blocks through matrix operations. After the checksum data is determined, it is written to the hard disk.

[0125] For example, the verification data block receives the full data of the EC stripe, based on the fact that it is sent to the verification data block with every write request, so that the EC verification data can be calculated when the verification data block is flushed. It does not need to read data from the data block OSD. Figure 3 This is a schematic diagram of a verification data block calculation process provided in an embodiment of the present invention, as shown below. Figure 3 As shown, when the EC cell size is 32KB and 128KB is written, it is determined to be a full EC stripe.

[0126] In this embodiment, all data within the EC stripe is sent to the verification data block. When the verification data block refreshes the verification data, it does not need to read data from other data blocks, thereby reducing network bandwidth consumption between the data block and the verification data block and improving data transmission efficiency.

[0127] In some embodiments, when the target data block experiences an abnormal restart, the method further includes:

[0128] When an abnormal restart occurs in the target data block, the first target data block corresponding to the abnormal restart is determined.

[0129] Scan the target data in the persistent memory area of ​​the hard drive;

[0130] Recover the target erasure code segments that were not previously downloaded to the hard drive;

[0131] Send data to the remaining target data blocks other than the first target data block to determine the missing data in the first target data block;

[0132] Perform data recovery processing on the missing data.

[0133] Specifically, when an abnormal restart occurs in the target data block, since the target data block can be a single block or multiple blocks, it is necessary to determine which data blocks experienced the abnormal restart and perform localization. The localization process is not limited; it can be done using a heartbeat query or by directly reporting problems encountered by individual processes. At this point, it is necessary to scan the target data in the persistent memory area of ​​the hard drive to recover the target erasure code segments that were not flushed to the hard drive. Data is sent to the processes of the remaining target data blocks (excluding the first target data block) to identify the truly missing data in the first target data block and to promptly perform data recovery processing on the missing data.

[0134] This embodiment provides a mechanism to improve the integrity, performance optimization, and fault tolerance of target data blocks in the event of an abnormal restart of the process corresponding to the target data block, by scanning memory and recovering incomplete EC stripes when the process of the first target data block restarts abnormally, communicating with other target data block processes, and determining whether data recovery is needed, thereby avoiding data loss.

[0135] In some embodiments, sending data to the remaining target data blocks other than the first target data block to determine missing data in the first target data block includes:

[0136] Control the remaining target data blocks to check if the target erasure code stripe exists in the disk persistent memory region;

[0137] If it exists, then retrieve the data corresponding to the target erasure code strips in the remaining target data blocks as the missing data;

[0138] If it does not exist, the data corresponding to the target erasure code segment is read from the hard drive as the missing data.

[0139] Specifically, in the process controlling the remaining target data blocks, the data block queries the memory space corresponding to the persistent memory region on the hard disk to see if the target erasure code stripe exists. If it exists, the corresponding data is directly determined from the target erasure code stripe and treated as the missing data. If it does not exist, the data corresponding to the target erasure code stripe needs to be read from the hard disk and treated as the missing data.

[0140] It's important to note that this query can employ either memory mapping or metadata checking. Memory mapping involves the OSD process maintaining a memory-mapped table that records the location of each EC stripe in the disk's persistent memory area. The address can be quickly located using the object ID and stripe offset. Metadata checking, on the other hand, involves the OSD process examining the metadata in the disk's persistent memory area to determine if the EC stripe exists. This metadata typically includes the stripe's identification information, status information, etc.

[0141] Missing data can be read from the hard drive. Because the data storage in the hard drive's flash memory is persistent, even if the data in the hard drive's persistent memory area is lost, it can be recovered from the hard drive.

[0142] This embodiment provides a method for recovering data from the hard drive when it is not present in the persistent memory region. Prioritizing the query of the persistent memory region reduces hard drive read operations, lowers I / O overhead, and enables rapid data recovery. This can reduce system downtime and improve system throughput. Furthermore, querying both the persistent memory region and the hard drive ensures data integrity.

[0143] In some embodiments, the method further includes:

[0144] Control the check data blocks in the remaining target data blocks to query whether the data in the persistent memory area of ​​the hard disk meets the aggregation conditions;

[0145] If satisfied, then determine the verification data for the hard disk's persistent memory area;

[0146] If the conditions are not met, then the full data corresponding to the target erasure code segment will be returned.

[0147] Specifically, it is necessary to control the processes of the checksum data blocks corresponding to the remaining target data blocks, query whether the data in the persistent memory area of ​​the hard disk meets the aggregation condition. Here, the aggregation condition is whether it meets the data storage requirements of the EC erasure coding stripe. If it does, the checksum data is calculated, and then a response is returned. If it does not meet the requirements, the full data corresponding to the target erasure coding stripe needs to be returned.

[0148] The verification data block process provided in this embodiment queries whether the hard disk persistent memory area has already stored the complete EC stripe. If so, it needs to calculate the verification data based on the data corresponding to the full EC stripe and return the calculated verification data to the requester. If not, it needs to return the full data in the received EC stripe and continue the data recovery operation. By querying the data in the hard disk persistent memory area, the complete EC stripe can be quickly determined, reducing unnecessary calculations and data transfers, as well as the time spent reading data from the hard disk, thus speeding up the data recovery process.

[0149] In some embodiments, data recovery processing for missing data includes:

[0150] If the data returned by the verification data block contains the data of the first target data block, then respond, read the data of the first target data block, and write it to the hard disk;

[0151] If the data returned by the verification data block is verification data, the data of the first target data block is determined by striping the target erasure code and written to the hard disk.

[0152] Specifically, if the data returned during the data block verification process includes data corresponding to the process of the first target data block, then the data is directly read from the response and then flushed to the hard disk from the persistent memory area. If the data returned during the data block verification process is verification data, then the data corresponding to the process of the first target data block is recovered by striping the target erasure coding, and then flushed to the hard disk.

[0153] This embodiment determines data integrity based on data responses collected from processes of other data blocks. If the data returned by the OSD verification includes data from this OSD, it can be directly read and flushed without additional calculations, significantly reducing recovery time. If the OSD verification returns verification data, recovery data is calculated via EC. Although calculations are required, they are necessary and efficient, avoiding unnecessary data transfer and I / O operations.

[0154] Furthermore, the present invention also provides a distributed storage system, including a master node and slave nodes; the master node and slave nodes are connected, and the master node is used to execute the steps of the data storage method to store the target data corresponding to the write request of the master node to the hard disk.

[0155] For an introduction to the distributed storage system provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above data storage method.

[0156] Figure 4 A data storage method provided in this embodiment of the invention corresponds to a processing flowchart of data blocks and verification data blocks, as shown in the flowchart. Figure 4 As shown, Figure 4 Only data block 1, data block 2, data block 3, check data block 5, and check data block 6 are shown. For simplicity, data block 4 is not shown. The steps are as follows:

[0157] The protocol layer sends small I / O write requests to the OSD containing the data block and the checksum block. The write request carries the object ID to be written, the offset within the object, and the data length. If the written data can fit into a single data block, corresponding to EC 4+2 redundancy, only one message needs to be sent to one data block and two checksum blocks. For example, if the data block size is 32KB, the written data length is 8KB, and the current EC stripe offset is 0, then only data block 1, checksum block 5, and checksum block 6 need to be sent.

[0158] When the OSD receives a write request, it writes the data to the persistent memory area of ​​the hard disk. The OSD queries the PMR memory corresponding to the EC stripe based on the internal object ID and the offset within the stripe. If it is not found, that is, the first time the stripe is written to this OSD, then PMR memory is allocated for the EC stripe.

[0159] For RDMA networks, each OSD binds the memory of the hard disk persistent memory area to the RDMA network card, so that the data is received in the PMR memory. When flushing data, the write request sent to the SSD carries the PMR memory address. In this way, neither receiving data nor flushing data needs to use the host memory, reducing the bandwidth usage of the host memory.

[0160] For the data block process, if the EC unit is full, the data is written from the hard disk persistent memory area to the hard disk, and then the hard disk persistent memory area is released. For example, if the EC unit size is 32KB, the data block OSD will perform a flush to the SSD flash memory if it determines that the received data is 32KB.

[0161] If writing data requires writing multiple data blocks, messages need to be sent to the OSDs of multiple data blocks and two check data blocks. For example, if the data block size is 32KB, the current EC stripe write offset is 20KB, and the write data length is 30KB, messages need to be sent to data block 1, data block 2, check data block 5, and check data block 6. Figure 5 This is a schematic diagram of a process for writing to multiple data blocks, provided as an embodiment of the present invention. Figure 5 As shown, when writing to multiple data blocks, a checksum is also sent to the data block.

[0162] Each write request must be sent to two verification data blocks. The verification data block contains the full data in the EC stripe and is used to calculate the EC verification data when the verification data block is flushed.

[0163] The OSD verifies that the data block is full when the EC stripe is filled, calculates the verification data, and then writes the verification data to the flash memory.

[0164] Figure 6This is a schematic diagram illustrating data recovery processing when a data block process experiences an abnormal restart, as provided in an embodiment of the present invention. Figure 6 As shown, Figure 6 Only data block 1, data block 2, data block 3, check data block 5, and check data block 6 are shown. For simplicity, data block 4 is not shown. The steps are as follows:

[0165] 1. OSD scans PMR memory to recover EC stripes that have not yet been refreshed;

[0166] 2. Send a GetData message to the relevant OSD for the EC stripe to retrieve the data of the EC stripe from other OSDs;

[0167] 3. Each OSD processes the GetData message;

[0168] For data OSD, first check if the EC stripe exists in PMR; if not, read the data from SSD flash memory.

[0169] For OSD verification, first check if the data in PMR is full of EC stripes. If it is full, calculate the verification data and then return the response; if it is not full, return the full data in the received EC stripes.

[0170] 4. The OSD receives all GetData responses and performs different processing based on the OSD's response:

[0171] If the data returned by the OSD verification contains data from this OSD, then the data is read directly from the response and then the SSD flash memory is flushed from the PMR.

[0172] If the OSD returns verification data, then the data for this OSD is calculated using EC, and then the data is processed for refresh.

[0173] The various embodiments corresponding to the data storage method have been described in detail above. Based on this, the present invention also discloses a data storage device corresponding to the above method. Figure 7 This is a structural diagram of a data storage device provided in an embodiment of the present invention. Figure 7 As shown, the data storage device includes:

[0174] The acquisition module 11 is used to pre-bind the persistent memory area of ​​the hard disk to the remote access network card in order to respond to the write request and acquire the target data corresponding to the write request;

[0175] The determination module 12 is used to determine the target data block to be allocated to the persistent memory area of ​​the hard disk based on the relationship between the data length of the target data and the preset data block size;

[0176] The write module 13 is used to aggregate the target data block when the target data block meets the aggregation conditions, so as to obtain the aggregated target data block and write the aggregated target data block to the hard disk.

[0177] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments of the method part for the description of the device part, and will not be repeated here.

[0178] For an introduction to the data storage device provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above data storage method.

[0179] Figure 8 A structural diagram of a data storage device provided in an embodiment of the present invention is shown below. Figure 8 As shown, the device includes:

[0180] Memory 21 is used to store computer programs;

[0181] Processor 22 is used to implement the steps of a data storage method when executing a computer program.

[0182] The data storage device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0183] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array. The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0184] The memory 21 may include one or more non-volatile storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the data storage method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the data storage method, etc.

[0185] In some embodiments, the data storage device may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0186] Those skilled in the field can understand, Figure 8 The structure shown does not constitute a limitation on the data storage device and may include more or fewer components than illustrated.

[0187] The processor 22 implements the data storage method provided in any of the above embodiments by calling instructions stored in the memory 21.

[0188] For an introduction to the data storage device provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above data storage method.

[0189] Furthermore, the present invention also provides a non-volatile storage medium on which a computer program is stored, and when the computer program is executed by the processor 22, it implements the steps of the data storage method described above.

[0190] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] For an introduction to the non-volatile storage medium provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above data storage method.

[0192] Furthermore, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of a data storage method.

[0193] For an introduction to the computer program product provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above data storage method.

[0194] The present invention has provided a detailed description of a data storage method, a distributed storage system, an apparatus, a medium, and a product. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

[0195] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A data storage method, characterized in that, include: The persistent memory region of the hard disk is pre-bound to a remote access network card to respond to write requests and obtain the target data corresponding to the write request. Specifically, the data corresponding to the write request is received in the persistent memory region of the hard disk. When flushing data, the write request sent to the hard disk carries the memory address of the persistent memory region, allowing data reception and flushing to bypass the host memory. This includes: exposing the persistent memory region of the hard disk as a contiguous physical address space through the controller; the driver kernel creating a dedicated mode register for the persistent memory region of the hard disk and registering it to the network card; binding the persistent memory region of the hard disk to the remote access network card includes: pre-dividing the persistent memory region of the hard disk into corresponding first data blocks according to a preset data block size; binding the memory region of the first data block to the remote access network card to allocate the obtained target data into the first data block. The target data block to be allocated to the persistent memory area of ​​the hard disk is determined based on the relationship between the data length of the target data and the preset data block size. When the target data block meets the aggregation condition, the target data block is aggregated to obtain an aggregated target data block, and the aggregated target data block is written to the hard disk. Correspondingly, determining the target data block allocated to the persistent memory area of ​​the hard disk based on the relationship between the data length of the target data and the preset data block size includes: Obtain the offset of the current free data block in the persistent memory region of the hard disk; The target data block to accommodate the target data is determined based on the relationship between the data length of the target data, the offset of the current free data block, and the preset data block size. Specifically, this includes: when the offset of the current free data block is zero, determining a first target data block based on the relationship between the data length of the target data and the preset data block size; when the offset of the current free data block is non-zero, determining a first data length for the current free data block to accommodate the target data; and determining a second target data block based on the relationship between the first data length of the target data and the preset data block size. Correspondingly, when the target data block experiences an abnormal restart, the method further includes: When an abnormal restart occurs in the target data block, the first target data block corresponding to the abnormal restart is determined; Scan the target data in the persistent memory region of the hard disk; Recover the target erasure code segments that were not previously downloaded to the hard drive; Send data to the remaining target data blocks other than the first target data block to determine the missing data in the first target data block; Perform data recovery processing on the missing data; Correspondingly, the method further includes: Control the check data blocks in the remaining target data blocks to query whether the data in the persistent memory area of ​​the hard disk meets the aggregation conditions; If satisfied, then determine the verification data for the hard disk's persistent memory area; If the conditions are not met, then the full data corresponding to the target erasure code segment will be returned.

2. The data storage method according to claim 1, characterized in that, Determining the second target data block based on the relationship between the first data length of the target data and the preset data block size includes: The second data length of the target data is determined based on the first data length of the target data; The second target data block is determined based on the relationship between the second data length and the preset data block size.

3. The data storage method according to claim 1, characterized in that, The first target data block is determined based on the relationship between the data length of the target data and the preset data block size, including: Divide the length of the target data by the preset data block size; If the data obtained by division is an integer, then the integer is used as the number of the first target data blocks; If the data obtained by division is a decimal, then the integer obtained by rounding the decimal is used as the number of the first target data blocks.

4. The data storage method according to claim 2 or 3, characterized in that, After determining the first target data block, the following steps are also included: Based on the target data, query the persistent memory area of ​​the hard disk corresponding to the erasure code stripe; If no persistent memory area on the hard disk corresponding to the erasure code stripe is found, it is determined that the erasure code stripe corresponding to the target data is written to the first target data block for the first time. Allocate memory addresses for persistent memory regions on the hard disk for erasure codes corresponding to the target data.

5. The data storage method according to claim 4, characterized in that, The method further includes: Retrieve the verification data block corresponding to the erasure code segment; When the target data block reaches the aggregation condition, the control unit determines the verification data of the target data and writes the verification data to the hard disk.

6. The data storage method according to claim 1, characterized in that, Sending data to the remaining target data blocks other than the first target data block to determine the missing data in the first target data block includes: Control the remaining target data blocks to query the persistent memory region of the hard disk to see if there are target error correction code stripes; If it exists, then obtain the data corresponding to the target error correction code strips in the remaining target data blocks as the missing data; If it does not exist, the data corresponding to the target error correction code stripe is read from the hard disk as the missing data.

7. The data storage method according to claim 1, characterized in that, Data recovery processing for missing data includes: If the data returned by the verification data block contains the data of the first target data block, then respond and read the data of the first target data block and write it to the hard disk; If the data returned by the verification data block is verification data, then the data of the first target data block is determined by striping the target erasure code and written to the hard disk.

8. A distributed storage system, characterized in that, It includes a master node and a slave node; the master node and the slave node are connected, and the master node is used to perform the steps of the data storage method according to any one of claims 1 to 7, so as to store the target data corresponding to the write request of the master node to the hard disk.

9. A data storage device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data storage method as described in any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a computer program, which, when executed by a processor, implements the steps of the data storage method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the data storage method according to any one of claims 1 to 7.

Images