Data storage method, device, computing equipment, storage system and storage medium
By improving the writing order of data blocks, the storage device can restore data through redundant algorithms when power is lost, solving the problem of data loss in the existing technology, realizing a high-reliability storage solution, and avoiding the use of backup power supplies , saving space and cost.
Patent Information
- Application Number
- CN202110857597.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-07-30
- Publication Date
- 2025-01-14
- Estimated Expiration
- 2039-07-30
AI Technical Summary
The existing storage technology cannot avoid data loss when all storage devices lose power at the same time, and backup power is required to prevent data loss, resulting in increased space usage and cost of storage devices.
By improving the writing order of data blocks, the number of blocks written in each band is less than or equal to the number of verification blocks N, ensuring that even if all storage devices lose power, data can be recovered through redundant algorithms to avoid data loss. Rely on backup power.
It realizes the avoidance of data loss when all storage devices are powered down at the same time, saves the internal space and costs of storage devices, and improves the reliability of data storage.
Smart Images

Figure CN113760186B_ABST
Abstract
Description
[0001] This case is a divisional application of patent application number 201910697302.1, application date July 30, 2019, and invention name “Data storage method, device, computing equipment, storage system and storage medium”. Technical Field
[0002] The present application relates to the field of storage technology, and in particular to a data storage method, apparatus, computing device, storage system and storage medium. Background Art
[0003] With the development of storage technology, in order to improve the reliability of data storage, data is usually stored redundantly using redundant arrays of independent drives (RAID) or erasure code (EC) technology, so that lost data can be restored through redundant data.
[0004] In the related art, the data storage process generally includes: a computing device obtains data to be stored and divides the data into one or more stripes; for each stripe in the one or more stripes, the computing device divides the stripe into M data blocks; the computing device uses a redundancy algorithm to encode the M data blocks to obtain N check blocks; the computing device determines the M+N blocks in each stripe as a batch of blocks to be written, and the computing device writes the (M+N) blocks in the same stripe to M+N storage devices at the same time point in the order from the first stripe to the last stripe; each of the M+N storage devices receives a block in the same stripe at the same time point; each storage device stores the received block in a cache; and each storage device performs persistent storage on the cached block. Taking M as 7 and N as 1 as an example, since M+N is 8, a batch of blocks to be written includes 8 blocks. The computing device will first determine the 8 blocks in the first stripe as the blocks to be written in the first batch; the 8 blocks in the first stripe will be written to 8 storage devices at the same time, and the 8 blocks in the first stripe will be cached in the 8 storage devices at the same time. The computing device will first determine the 8 blocks in the second stripe as the blocks to be written in the second batch, and write the 8 blocks in the second stripe to 8 storage devices at the same time, and the 8 blocks in the second stripe will be cached in the 8 storage devices at the same time, and so on. Each time a batch of writes is executed, all blocks in the same stripe will be written until the last stripe is written to 8 storage devices.
[0005] For the same stripe, if blocks are lost in the stripe, when the number of lost blocks is less than or equal to N, the lost blocks can be recovered based on the redundancy algorithm through the remaining blocks in the stripe; when the number of lost blocks is greater than N, the lost blocks cannot be recovered. M is a positive integer greater than or equal to 1, and N is a positive integer greater than or equal to 1.
[0006] When using the above method to store data, if M+N storage devices lose power at the same time, the data cached by the M+N storage devices will be lost. For the same stripe, since each block of the stripe is written to each storage device at the same or approximately the same time point, each block of the stripe will be cached to each storage device at the same or approximately the same time point. If the cached time point happens to be during the power failure, all blocks of the stripe will be lost. Once all blocks of the stripe are lost, the stripe cannot be restored by the redundancy algorithm. In order to avoid data loss caused by this situation, a backup power supply needs to be installed in the storage device. If the storage device loses power, the backup power supply will provide power to the storage device. The storage device will use the power of the backup power supply to store the cached data in a non-volatile storage medium. The backup power supply will occupy the internal space of the storage device, affecting the storage capacity of the storage device and increasing the cost of the storage device. Summary of the invention
[0007] The present application provides a data storage method, apparatus, computing device, storage system and storage medium, which can be independent of the backup power supply of the storage device when all storage devices lose power at the same time, and can also avoid the loss of stripes cached by the storage device, thereby breaking the limitation that the storage device must be installed with a backup power supply.
[0008] In a first aspect, a data storage method is provided, the method comprising:
[0009] Determine a batch of blocks to be written from S stripes, wherein the number of blocks belonging to the same stripe in the batch of blocks to be written is less than or equal to N; wherein each stripe in the S stripes includes (M+N) blocks, and the (M+N) blocks include M data blocks and N check blocks, and S, M, and N are positive integers; and write the blocks to be written in the batch into M+N storage devices respectively.
[0010] This embodiment provides a solution that can avoid the loss of stripes cached by storage devices without relying on the backup power supply of storage devices when all storage devices lose power at the same time. By improving the order in which each block in each stripe is written to the storage device, from writing M+N blocks in a batch to writing less than or equal to N blocks in a batch, for each stripe, since the number of blocks in the stripe that are simultaneously written to M+N storage devices is less than or equal to N, the number of blocks in the stripe that are simultaneously cached to M+N storage devices will also be less than or equal to N. Therefore, even if all M+N storage devices lose power at any point in time, causing the data cached by the M+N storage devices to be completely lost, the number of lost blocks in the stripe will be less than or equal to N, so the stripe can recover the lost blocks through a redundancy algorithm, thereby avoiding the loss of data in the stripe. Since this solution can avoid data loss and ensure the reliability of data storage even if the storage device does not have a backup power supply, the storage device does not need to install a backup power supply, so the internal space occupied by the backup power supply in the storage device can be saved, allowing the storage device to free up more internal space for storage media, thereby increasing the storage capacity of the storage device. In addition, the cost of manufacturing the backup power supply can be saved, thereby saving the cost of the storage device, and in addition, the backup power supply can be prevented from limiting the life of the storage device.
[0011] Optionally, the number of blocks belonging to the same stripe in the batch of blocks written is less than or equal to N, including: when N is equal to 1, each block in the batch of blocks written belongs to a different stripe.
[0012] This optional method takes into account the situation of whether there is one or more check blocks. If there is one check block in the stripe, for any of the S stripes, since each block in a batch of blocks written belongs to a different stripe, after executing any batch write, there is at most one block belonging to the stripe among the blocks cached by M+N storage devices at the same time or approximately at the same time. Then, after the M+N storage devices lose power at any time, there is at most one block belonging to the stripe among the blocks cached by the M+N storage devices. Therefore, there is at most one block of the stripe lost by the M+N storage devices, so the stripe can be recovered through the remaining M+N-1 blocks. If there are multiple check blocks in a stripe, for any stripe among the S stripes, since the number of blocks belonging to the same stripe in each batch of blocks written is less than or equal to N, after any batch of blocks is written, the number of blocks belonging to the stripe in the blocks cached simultaneously or approximately simultaneously by M+N storage devices is 1 to N. Then, after the M+N storage devices lose power at the same time at any point in time, there are at most N blocks belonging to the stripe in the blocks cached by the M+N storage devices. Therefore, there are at most N blocks of the stripe lost by the M+N storage devices, so the stripe can be recovered through the remaining blocks.
[0013] Optionally, for the same storage device among the M+N storage devices, the blocks written to the storage device in the batch of blocks written and the blocks written to the storage device in the previous batch of blocks written belong to different stripes.
[0014] Through this optional method, since for the same storage device, the blocks written to the storage device in one batch and the blocks written to the storage device in the previous batch belong to different stripes, then if the previous batch wrote blocks of stripe j to storage device i, this batch will write blocks of other stripes except stripe j to storage device i, and write blocks of stripe j to other storage devices except storage device i. Therefore, it can be ensured that different blocks of the same stripe are stored in different storage devices respectively. For example, 8 blocks of a stripe with a 7+1 structure can be stored in 8 different SSDs respectively. For redundant algorithms such as EC algorithm or RAID algorithm, it is usually required that different blocks of the stripe are stored in different storage devices for data recovery. Therefore, this method can meet the requirements of redundant algorithms.
[0015] Optionally, S is obtained according to M, N and K, where K is the maximum number of blocks cached by each storage device.
[0016] Optionally, S is obtained by the following formula:
[0017]
[0018] Among them, ceil represents the rounding up operation.
[0019] Through this optional method, S stripes are grouped together, and different stripes in the same group can be written together during each write process, thereby improving the overall storage efficiency of this group of stripes.
[0020] Optionally, after writing the blocks written in the batch into the M+N storage devices respectively, the method further includes: if a power failure event is detected, obtaining the first block and the second block in the S stripes, the first block being a block that has not yet been written to the M+N storage devices, and the second block being a block that has been persistently stored in the M+N storage devices; and recovering data of the lost blocks in the S stripes based on the first block and the second block.
[0021] Through this optional method, if the storage device loses cached blocks due to power outage, the blocks that have not been written to the storage device and the blocks that have been persistently stored on the storage device can be found, and the lost blocks can be restored using these remaining blocks that have not been lost, thereby avoiding stripe loss and improving storage reliability.
[0022] In a second aspect, a data storage device is provided, the device having the function of implementing the data storage in the first aspect or any optional manner of the first aspect. The device includes at least one module, the at least one module is used to implement the data storage method provided by the first aspect or any optional manner of the first aspect.
[0023] According to a third aspect, a computing device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the instruction is executed by the processor to implement the method described in the first aspect or any optional manner of the first aspect.
[0024] In a fourth aspect, a storage system is provided, wherein the storage system includes a computing device as described in the third aspect or any optional manner of the third aspect and M+N storage devices.
[0025] In a fifth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the instruction is executed by a processor to implement the method described in the first aspect or any optional manner of the first aspect.
[0026] In a sixth aspect, a computer program is provided, wherein the computer program is used to execute the method described in the first aspect or any optional manner of the first aspect.
[0027] In the seventh aspect, a chip is provided, comprising a processing circuit and an output interface connected to and communicating with the internal part of the processing circuit, wherein the processing circuit is used to execute the method described in the first aspect or any optional method of the first aspect, and the output interface is used to write the blocks written in a batch into M+N storage devices respectively. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 It is an architectural diagram of an implementation environment of a data storage method provided in an embodiment of the present application;
[0029] Figure 2 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;
[0030] Figure 3 is a structural schematic diagram of a storage device provided in an embodiment of the present application;
[0031] Figure 4 is a schematic diagram of the structure of an SSD provided in an embodiment of the present application;
[0032] Figure 5 It is an architecture diagram of a storage system provided in an embodiment of the present application;
[0033] Figure 6 is a flow chart of a data storage method provided by an embodiment of the present application;
[0034] Figure 7 is a schematic diagram of a strip and a block provided in an embodiment of the present application;
[0035] Figure 8 is a schematic diagram of a strip and a block provided in an embodiment of the present application;
[0036] Fig. 9 This is a flow chart of obtaining a stripe provided in an embodiment of the present application;
[0037] Fig.10 is a schematic diagram of a writing sequence provided in an embodiment of the present application;
[0038] Fig.11 is a schematic diagram of a writing sequence provided in an embodiment of the present application;
[0039] Fig.12 It is a schematic diagram of recovering lost blocks after power failure provided by an embodiment of the present application;
[0040] Fig.13 This is a flow chart of data recovery provided by an embodiment of the present application;
[0041] Fig.14 It is a structural schematic diagram of a data storage device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0043] The following is an illustrative introduction to the terms involved in this application.
[0044] Redundant arrays of independent drives (RAID): A data protection method that combines multiple physical disk drives into one or more logical units, thereby enabling redundant storage of data.
[0045] Erasure code (EC): A data protection method that divides data into blocks, uses a redundancy algorithm, expands and encodes the blocks, and stores them in a set of different locations or storage media.
[0046] Stripe (also called data stripe or data stripe): In RAID or EC technology, a continuous piece of data is formed after logical sequential data is segmented. Generally speaking, a stripe is further divided into data blocks so that the check blocks can be calculated for the data blocks according to the RAID or EC algorithm, and different blocks can be written to different storage devices.
[0047] Chunk (also called strides, stripe unit or stripe unit): In RAID or EC technology, a piece of sequential data written to or read from a storage device. Chunks are generally divided by stripes.
[0048] Solid state drives (SSDs, also known as solid-state devices, solid-state disks or flash drives) are storage devices that primarily use NAND flash memory (NAND stands for not AND) integrated circuit components as persistent memory.
[0049] The following is an exemplary introduction to the implementation environment provided by this application.
[0050] Figure 1 It is an architectural diagram of an implementation environment of a data transmission method provided in an embodiment of the present application, and the implementation environment includes a computing device 101, a storage device 102 and a client 103.
[0051] For example, the computing device 101 may be a server or a storage array controller, etc. The storage device 102 may be an SSD, a hard disk drive (HDD), etc. The client 103 may be any data access device, such as an application server, a host, or a terminal.
[0052] The computing device 101 and the storage device 102 can provide data storage services for the client 103. Specifically, the client 103 can provide the data to be stored to the computing device 101, and the computing device 101 can obtain the data from the client 103 and store the data in the storage device 102. Exemplarily, the client 103 can send an input / output (I / O) request to the computing device 101, and the I / O request carries the data to be stored, and the computing device 101 can receive the I / O request from the client 103, obtain the data from the I / O request, and store the data.
[0053] In some possible embodiments, the storage device 102 can be provided to the user as a cloud storage service. Specifically, the storage device 102 can run in a cloud environment, for example, it can run on a public cloud, a private cloud, or a hybrid cloud. The user can use the terminal 103 to apply for a certain capacity of storage space in the cloud storage service. The computing device 101 can allocate storage space of corresponding capacity to the user, for example, allocate one or more storage devices 102 to the user, so as to store data in the allocated storage space. As an example, the storage device 102 can be provided as an object storage service (English: object storage service, referred to as: OBS), a cloud hard disk, a cloud database, etc.
[0054] Those skilled in the art will appreciate that the number of computing devices 101, storage devices 102, and clients 103 in the above implementation environment may be more or less. For example, there may be only one computing device 101, or there may be dozens, hundreds, or more computing devices 101, and the above implementation environment may also include other computing devices 101.
[0055] The number of storage devices 102 connected to each computing device 101 may be greater than or equal to M+N, where M represents the number of data blocks and N represents the number of check blocks. For example, if a stripe includes 7 data blocks and 1 check block, the number of storage devices 102 may be greater than or equal to 8. Optionally, multiple computing devices 101 and multiple storage devices 102 may form a storage cluster to jointly provide storage services through collaborative operation.
[0056] The following is an exemplary introduction to the hardware structure provided by this application.
[0057] The present application provides a computing device, which may be the computing device 101 in the above implementation environment.
[0058] like Figure 2 As shown, the computing device includes one or more processors 201 and one or more memories 202 .
[0059] Processor 201 can be one or more central processing units (CPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, or any other suitable circuits.
[0060] The memory 202 stores at least one instruction, and the processor 201 is used to execute the instruction to implement step 601, step 602, step 603, step 604 and step 605 provided in the following method embodiment.
[0061] The memory 202 may be a data buffer. The memory 202 may be a volatile memory, such as a dynamic random access memory (DRAM), or a non-volatile memory, such as a persistent memory.
[0062] In some possible embodiments, the computing device may include a bus 203 , and the processor 201 and the memory 202 may communicate with each other via an internal connection of the bus 203 .
[0063] In some possible embodiments, the computing device may include a backup power supply, which can store electrical energy. When the computing device is running, the backup power supply may be in a charging state. When the computing device loses power, the backup power supply may provide electrical energy, thereby protecting the data cached in the memory 202. For example, if the memory 202 is a volatile memory, when the computing device loses power, the backup power supply may supply electrical energy, and the data cached in the memory 202 may be stored in other storage spaces through the electrical energy provided by the backup power supply, thereby preventing the data cached in the memory 202 from being lost due to power failure. The backup power supply may be a battery, a capacitor, or an uninterruptible power system (uninterruptible powersupply, UPS).
[0064] The present application provides a storage device, which may be the storage device 102 in the above implementation environment.
[0065] like Figure 3 As shown, the storage device may include one or more processors 301 , one or more cache memories (cache) 302 , and one or more persistent memories 303 .
[0066] Processor 301 may be one or more CPUs, FPGAs, ASICs, PLDs, controllers, state machines, gate logic, discrete hardware components, or any other suitable circuits.
[0067] The cache memory 302 may be, but is not limited to, a DRAM. The cache memory 302 is a volatile memory. If the storage device loses power, such as if the device where the storage device is located loses power, the storage device is suddenly unplugged, etc., the data in the cache memory 302 will be lost. In the following method embodiment, the cache memory 302 may be used to cache blocks written by the computing device. In one possible implementation, if the cache memory 302 caches a maximum of K blocks, the cache memory 302 may store the cached blocks in the persistent memory 303 after caching K blocks, thereby persistently storing the blocks.
[0068] The persistent memory 303 may be a non-volatile memory, for example, a flash memory, specifically a NAND flash memory. In the following method embodiment, the persistent memory 303 may be used to receive blocks from the cache memory 302 and perform persistent storage on the blocks.
[0069] In some possible embodiments, the storage device may further include a bus 304 , and the processor 301 , the cache memory, and the persistent memory 303 may communicate with each other via an internal connection of the bus 304 .
[0070] Take the storage device as SSD as an example, see Figure 4 , Figure 4 The schematic diagram of the structure of an SSD provided in this embodiment. The SSD may include an SSD interface, an SSD controller, a cache memory (cache), and a NAND flash memory. The SSD interface may be connected to a computing device. Figure 4 As shown, if the computing device sends a block to the SSD, the block will be input to the SSD from the SSD interface, the SSD controller will cache the block to the cache memory, and the cache memory can store the block in the NAND flash memory. In addition, the computing device can also provide power to the SSD through the SSD interface.
[0071] The storage system provided in this embodiment is exemplarily introduced below.
[0072] This embodiment provides a storage system, which may include Figure 2 The computing device and M+N storage devices shown in the embodiment can be connected to the computing device and the M+N storage devices. Figure 5 , if M is 7 and N is 1, the computing device can be connected to 8 SSDs. These 8 SSDs are Figure 5 SSD1 to SSD8 in the.
[0073] Among them, the storage device can be set inside the computing device, for example, the storage device can be connected to the bus of the computing device, and the computing device can communicate with the storage device through the bus. Of course, the storage device can also be set outside the computing device, for example, the computing device can be connected to the storage device 102 through a switching device or a network. Taking SSD as a storage device as an example, the computing device may include an input / output (IO) port, a chip or an adapter, which is connected to the storage device through a switching device or a network. In a possible scenario, if the computing device loses power, all storage devices connected to the computing device may lose power, and by executing the method provided in the following embodiment, the loss of data cached by all storage devices can be avoided.
[0074] The following is an illustrative introduction to the method flow provided by this application.
[0075] Figure 6 601 to 605:
[0076] 601. The computing device obtains S stripes to be stored.
[0077] S represents the number of stripes, each of the S stripes includes (M+N) blocks, the (M+N) blocks include M data blocks and N check blocks, M represents the number of data blocks, N represents the number of check blocks, and S, M and N are positive integers. N can be 1, or can be greater than or equal to 2. For example, a stripe can include 7 data blocks and 1 check block; for another example, a stripe can include 6 data blocks and 2 check blocks; for another example, a stripe can include 6 data blocks and 3 check blocks. This embodiment does not limit the specific number of check blocks and data blocks in a stripe.
[0078] Schematically, see Figure 7 and Figure 8 , Figure 7 8 stripes are shown, each stripe includes 7 data blocks and 1 check block. Figure 812 stripes are shown, each stripe includes 6 data blocks and 2 check blocks. Figure 7 as well as Figure 8 Each row in corresponds to a stripe, Figure 7 as well as Figure 8 Each column in corresponds to a block at the same position, Figure 7 as well as Figure 8 Each square in represents a block in a stripe. Figure 7 as well as Figure 8 In the above, Dij represents the data block at position j of stripe i, Pi represents the first check block of stripe i, Qi represents the second check block of stripe i, i is the identifier of the stripe, and i is a positive integer greater than or equal to 0. j is the identifier of the block in the stripe, and j is a positive integer greater than or equal to 0.
[0079] For the strip acquisition process, see Fig. 9 , which may include the following steps 1 to 3:
[0080] Step 1: The computing device divides the data in the data cache area into S stripes.
[0081] The computing device may obtain the data to be stored and cache the data in the data cache area. For example, the computing device may receive the data to be stored from the terminal through an I / O interface.
[0082] Step 2: For each of the S stripes, the computing device divides the strip into M data blocks.
[0083] Step 3: The computing device can calculate N check blocks based on M data blocks.
[0084] The computing device may use a redundancy algorithm to encode the M data blocks to obtain N check blocks. The redundancy algorithm may be, but is not limited to, a RAID or EC algorithm.
[0085] In a possible implementation, the computing device may determine S according to M, N, and K, where K is the maximum number of blocks cached by the computing device and K is a positive integer. S may be positively correlated with M, N, and K, and may be negatively correlated with N.
[0086] It should be noted that obtaining S according to M, N and K is only an optional method, not a mandatory method. In other possible embodiments, S may also be determined in other ways, and this embodiment does not limit the method for determining S.
[0087] In a possible implementation, the computing device may obtain S by the following formula:
[0088]
[0089] Among them, ceil means rounding up, for example, ceil(x) means getting the smallest integer not less than x. * means multiplication, and - means division.
[0090] The following is an exemplary description of the technical effect of this calculation method of S:
[0091] Take a stripe including 7 data blocks and 1 check block, and a storage device can cache 1 block at most as an example. If 1 block is lost in the stripe, after the block is lost, the lost block can be restored through the remaining blocks based on the redundancy algorithm; if 2 or more blocks are lost in the stripe, after the block is lost, the lost block cannot be restored through the remaining blocks. Therefore, at most 1 block is lost in a stripe. In this embodiment, in order to ensure that when all storage devices lose power, no more than 1 block is lost in the same stripe, only 1 block in the same stripe will be written to the storage device during each write process. Then, since different blocks in the same stripe are written to the storage device at different time points, different blocks in the same stripe are cached to the storage device at different time points, that is, at any time point, all storage devices cache at most 1 block of the same stripe. Then, even if all storage devices lose power at that time point, the stripe loses at most 1 block, and the lost block can be restored through the remaining 7 blocks.
[0092] In this way, a stripe needs to go through 8 writing processes before each block in the stripe can be stored: in the first writing process, the first data block in the stripe is written, in the second writing process, the second data block in the stripe is written, in the third writing process, the third data block in the stripe is written, and so on. In the eighth writing process, the eighth data block in the stripe is written. If each stripe is written to each storage device in this serial manner, the storage efficiency will be low. In view of this, the above calculation method can be used to group S stripes. The number of stripes required for each group is (7+1)*1 / 1=8. By writing different blocks in the 8 stripes in the same group together, the overall storage efficiency of this group of stripes is improved. For example, see Fig.10 , you can get 8 stripes and write the blocks in these 8 stripes respectively each time.
[0093] For example, a stripe includes 6 data blocks and 2 check blocks, and a storage device can cache 3 blocks at most. If 1 or 2 blocks are lost in the stripe, the lost blocks can be restored through the remaining blocks based on the redundancy algorithm; if 3 or more blocks are lost in the stripe, the lost blocks cannot be restored through the remaining blocks. Therefore, at most 2 blocks are lost in a stripe. In this embodiment, in order to ensure that when all storage devices lose power, no more than 2 blocks are lost in the same stripe, at most 2 blocks in the same stripe will be written to the storage device during every 3 writes. Then, since the number of blocks written to all storage devices by the same stripe at three adjacent time points does not exceed 2, the number of blocks cached to all storage devices by the same stripe at three adjacent time points does not exceed 2. In other words, at any three time points, all storage devices cache at most 2 blocks of the same stripe. Therefore, even if all storage devices lose power at that time point, the stripe will lose at most 2 blocks, and the lost blocks can be recovered from the remaining 6 blocks.
[0094] In this way, a stripe needs to go through 8*3 / 2=12 write processes before each block in the stripe can be stored: in the first write process to the third write process, the first data block and the second data block in the stripe are written, in the fourth write process to the sixth write process, the third data block and the fourth data block in the stripe are written, in the seventh write process to the ninth write process, the fifth data block and the sixth data block in the stripe are written, and in the tenth write process to the twelfth write process, the seventh data block to the eighth data block in the stripe are written. If each stripe is written to each storage device in this serial manner, the storage efficiency will be low. In view of this, in order to avoid affecting the storage efficiency, the blocks of multiple stripes can be grouped together for storage, and the number of stripes required for each group is (6+2)*3 / 2=12. For example, see Figure 8 , 12 stripes can be obtained, and the blocks in these 12 stripes are subsequently written in sequence.
[0095] It should be noted that the above formula is only an optional method for calculating S, not a mandatory method. In other possible embodiments, S may be calculated by other methods other than the above formula, and this embodiment does not limit the calculation method of S.
[0096] 602. The computing device determines a batch of blocks to be written from the S stripes, wherein the number of blocks in the batch of blocks that belong to the same stripe is less than or equal to N.
[0097] The blocks written in a batch include blocks written to M+N storage devices at the same time. Specifically, if each storage device caches at most 1 block, the blocks written in a batch may be M+N blocks, and each block in the blocks written in a batch corresponds to a storage device. In the subsequent step 603, each block in the blocks written in a batch will be written to a corresponding storage device. Among them, the blocks written in the same batch correspond to the same writing time point. In other words, the time points at which the blocks written in the same batch are written to the M+N storage devices will be the same or approximately the same.
[0098] If each storage device caches at most K blocks, and K is greater than 1, then the blocks written in a batch may be (M+N)*K blocks, and the K blocks in the blocks written in a batch correspond to one storage device. In the subsequent step 603, the K blocks in the blocks written in a batch will be written to a storage device corresponding to the K blocks. The blocks written in the same batch correspond to K writing time points. Specifically, at the first writing time point, the M+N blocks in the blocks written in a batch will be written to the M+N storage devices at the same time, and at the second writing time point, the other M+N blocks in the blocks written in a batch will be written to the M+N storage devices at the same time. And so on, at the Kth writing time point, the last M+N blocks in the blocks written in a batch will be written to the M+N storage devices at the same time. At this time, the writing of the blocks written in the same batch is completed.
[0099] For ease of description, in the following, the M+N blocks written to M+N storage devices at the same time are defined as blocks in one write process. If each storage device caches at most 1 block, then the blocks written in a batch are blocks in one write process. If each storage device caches at most K blocks, then the blocks written in a batch are blocks in K consecutive write processes.
[0100] In one possible implementation, a write process may be performed in the following manner: a computing device runs M+N threads, each thread writes a cached block to a storage device, and the M+N threads write at the same time point. In this implementation, the M+N blocks of the same write process are written at the same time point. In another possible implementation, a write process may be performed in the following manner: a computing device runs 1 thread, and the thread first writes the first cached block to the storage device, and then writes the second cached block to the next storage device, and so on. The thread writes the M+N cached blocks to the corresponding storage devices in sequence. After the thread writes any block to the storage device, it can directly write the next block to the next storage device without waiting for the storage device to return a confirmation message. In this implementation, the M+N blocks of the same write process are written at approximately the same time point.
[0101] For example, if K is 1, see Fig.10 , the blocks written in the first batch may be blocks written to SSD1 to SSD8 at the same time at T0, the blocks written in the second batch may be blocks written to SSD1 to SSD8 at the same time at T1, the blocks written in the third batch may be blocks written to SSD1 to SSD8 at the same time at T2, and the blocks written in the last batch may be blocks written to SSD1 to SSD8 at the same time at T7. Among them, the blocks written to SSD1 to SSD8 at the same time are 8 different blocks. For example, the 8 blocks written in the first batch are block D00 written to SDD1, block D10 written to SDD2, block D20 written to SDD3, block D30 written to SDD4, block D40 written to SDD5, block D50 written to SDD6, block D60 written to SDD7, and block D70 written to SDD8.
[0102] For example, if K is 3, see Fig.11 , the blocks written in the first batch may include blocks written in three times, the blocks in the first writing process in the first batch are blocks written to SSD1 to SSD8 simultaneously at time point T0, the blocks in the second writing process in the first batch are blocks written to SSD1 to SSD8 simultaneously at time point T1, and the blocks in the third writing process in the first batch are blocks written to SSD1 to SSD8 simultaneously at time point T2. The blocks written in the second batch include blocks written in three times, the blocks in the first writing process in the second batch are blocks written to SSD1 to SSD8 simultaneously at time point T3, the blocks in the second writing process in the second batch are blocks written to SSD1 to SSD8 simultaneously at time point T4, and the blocks in the third writing process in the second batch are blocks written to SSD1 to SSD8 simultaneously at time point T5. By analogy, the blocks written in the last batch include blocks of three writing processes. The blocks in the first writing process in the last batch are the blocks written to SSD1 to SSD8 simultaneously at time point T9, the blocks in the second writing process in the last batch are the blocks written to SSD1 to SSD8 simultaneously at time point T10, and the blocks in the third writing process in the last batch are the blocks written to SSD1 to SSD8 simultaneously at time point T11.
[0103] In this embodiment, since the number of blocks belonging to the same stripe in each batch of written blocks is less than or equal to N, for any stripe in the S stripes, after executing any write process in any batch, the number of blocks belonging to the stripe in the blocks cached simultaneously in the M+N storage devices is less than or equal to N. Then, if the M+N storage devices lose power at any time, resulting in the loss of all data cached by the M+N storage devices, since the number of blocks belonging to the stripe in the blocks cached by the M+N storage devices is less than or equal to N, the stripe will lose at most N blocks, so even if the stripe loses blocks, it can be restored by the remaining blocks, thereby avoiding the loss of the stripe when the M+N storage devices lose power at the same time.
[0104] Specifically, after the computing device executes any write process, each storage device in the M+N storage devices will receive a block and cache a block, so the M+N storage devices will cache a total of M+N blocks. Among them, since the time points of block writing in the same write process are the same or approximately the same, the time point at which each storage device in the M+N storage devices receives the block will be the same or approximately the same, and the time point at which each storage device in the M+N storage devices caches the block will be the same or approximately the same, so there will be M+N blocks cached in the M+N storage devices at the same time.
[0105] For example, if K is 1 and N is 1, see Fig.10At each time point, a row of blocks will be cached in 8 SSDs, and each block in this row of blocks belongs to a different stripe. At time point T0, the computing device writes the first batch of 8 blocks: block D00, block D10, block D20 to block D70, to SSD1 to SSD8 respectively. Then, each SSD in SSD1 to SSD8 will receive one of the 8 blocks D00, block D10, block D20 to block D70 respectively, and cache one block respectively. Since the time points of the 8 writes to the 8 SSDs are the same or approximately the same, the time point at which each of the 8 SSDs receives the block will be the same or approximately the same. Therefore, the time point at which each of the 8 SSDs caches the block will be the same or approximately the same, therefore, the cache time points of block D00, block D10, block D20 to block D70 will be the same or approximately the same. Since block D00, block D10, block D20 to block D70 belong to different stripes respectively, even if 8 SSDs lose power at the same time, the data cached by the 8 SSDs will be all lost, resulting in the loss of block D00, block D10, block D20 to block D70. Each of the 8 stripes, stripe 0, stripe 1 to stripe 7, has only lost 1 block. Therefore, each of the 8 stripes can be restored, thereby ensuring that none of the 8 stripes is lost. Similarly, the computing device writes the second batch of 8 blocks: block D71, block D01, block D11 to block D61, to SSD1 to SSD8 at time point T1. Then, each SSD in SSD1 to SSD8 will receive one of the 8 blocks D71, block D01, block D11 to block D61, and cache one block respectively. Since the 8 writes to the 8 SSDs are performed at the same or approximately the same time points, blocks D71, block D01, block D11 to block D61 are cached. The cache time point of block D61 will be the same or approximately the same. Since block D71, block D01, block D11 to block D61 belong to different stripes, even if the 8 SSDs lose power at the same time, all the data cached by the 8 SSDs will be lost, resulting in the loss of blocks D71, block D01, block D11 to block D61. Each of the 8 stripes, stripe 0, stripe 1 to stripe 7, has only lost 1 block. Therefore, each of the 8 stripes can be restored, thereby ensuring that none of the 8 stripes is lost.
[0106] For example, if K is 3 and N is 2, see Fig.10, at every 3 consecutive time points, there will be 2 rows of blocks cached in 8 SSDs, and the number of blocks in the same stripe in these 2 rows of blocks is less than or equal to 2. Schematically, from time point T0 to time point T2, the computing device writes the first batch of 24 blocks: block D00, block D10, block D20, block D30, block D40, block D50, block D60, block D70, block D80, block D90, block DA0, block DB0, block D01, block D11, block D21, block D31, block D41, block D51, block D61, block D71, block D81, block D91, block DA1 and block DB1, respectively, wherein the first process of the first batch will write blocks D00, block D10 to block D70, the second writing process of the first batch will write blocks D80, block D90 to block D31, and the third writing process of the first batch will write blocks D41, block D51 to block DB1. Since the write process is performed three times to the eight SSDs at three consecutive time points, each of the eight SSDs will receive blocks at three consecutive time points, and each of the eight SSDs will cache blocks at three consecutive time points. Therefore, the 24 blocks of blocks D00, D10, D20 to DB1 will be cached in the eight SSDs at three consecutive time points. Since each stripe in blocks D00, D10, D20 to DB1 has at most two blocks, even if the eight SSDs lose power at the same time, all the data cached by the eight SSDs will be lost, resulting in the loss of all blocks D00, D10, D20 to DB1, and each of the eight stripes of stripe 0, stripe 1 to stripe 7 will lose at most two blocks, so each of the eight stripes can be restored, thereby ensuring that none of the eight stripes is lost. Similarly, after the second batch of blocks is written from time point T3 to time point T5, it can also be guaranteed that during the writing to persistent storage, the 8 stripes will not be lost due to the simultaneous power failure of the 8 SSDs. After the third batch of blocks is written from time point T6 to time point T8, it can also be guaranteed that the 8 stripes will not be lost due to the simultaneous power failure of the 8 SSDs. After the third batch of blocks is written from time point T9 to time point T12, it can also be guaranteed that the 8 stripes will not be lost due to the simultaneous power failure of the 8 SSDs. In summary, after the blocks of any batch are written at any time point in any writing process, the 8 stripes can be guaranteed not to be lost.
[0107] In an exemplary scenario, see Fig.11, if each of the 8 SSDs can cache up to 3 blocks, and the stripe includes 6 data blocks and 2 check blocks, the computing device performs the first write process at time point T0, the second write process at time point T1, the third write process at time point T2, and so on, and performs the twelfth write process at time point T11. Let the current time point be T7, then for the three time points T4, T5 and T6 before T7, the blocks written at these three time points will be cached in each of the 8 SSDs, and for the time points T0, T1, T2 and T3 before T4, the blocks written at these four time points have been persistently stored by each of the 8 SSDs. If a power outage occurs at this time, the eight SSDs will lose the cached blocks at time points T4, T5, and T6. However, for any stripe, no more than two blocks of the stripe are lost. Since the 6+2 EC algorithm can tolerate the loss of two blocks in the same stripe, after the computing device is started, the EC algorithm can be used to recover the blocks at time points T4, T5, and T6 based on the blocks at time points T7 to T11 that are still protected by the computing device and the blocks at time points T0, T1, T2, and T3 that have been persistently stored in the storage device.
[0108] If N is equal to 1, each block in each batch of written blocks belongs to a different stripe. For example, if a stripe includes 7 data blocks and 1 parity block, each block in each batch of 8 written blocks belongs to 8 different stripes. For example, see Fig.10 , the first batch of blocks written include block D00, block D10, block D20, block D30, block D40, block D50, block D60 and block D70. Block D00 belongs to stripe 0, block D10 belongs to stripe 1, block D20 belongs to stripe 2, block D30 belongs to stripe 3, block D40 belongs to stripe 4, block D50 belongs to stripe 5, block D60 belongs to stripe 6, and block D70 belongs to stripe 7. The second batch of blocks written include block D71, block D01, block D11, block D21, block D31, block D41, block D51 and block D61. Block D71 belongs to stripe 7, block D01 belongs to stripe 0, block D11 belongs to stripe 1, block D21 belongs to stripe 2, block D31 belongs to stripe 3, block D41 belongs to stripe 4, block D51 belongs to stripe 5, and block D61 belongs to stripe 6.
[0109] In addition, if N is greater than 1, the number of blocks belonging to the same stripe in each batch of blocks written is less than or equal to N. For example, if the stripe includes 6 data blocks and 2 check blocks, the number of blocks belonging to the same stripe in the 8 blocks corresponding to each batch of blocks written may be less than or equal to 2. That is, each of the 8 blocks corresponding to each batch of blocks written may belong to 8 different stripes, or 2 of the 8 blocks corresponding to each batch of blocks written belong to the same stripe. For another example, if the stripe includes 6 data blocks and 3 check blocks, the number of blocks belonging to the same stripe in the 9 blocks corresponding to each batch of blocks written may be less than or equal to 3. That is, each of the 9 blocks corresponding to each batch of blocks written belongs to 9 different stripes, or 2 of the 9 blocks corresponding to each batch of blocks written belong to the same stripe, or 3 of the 9 blocks corresponding to each batch of blocks written belong to the same stripe.
[0110] It should be noted that step 602 is described by taking a batch of blocks written as an example. Similarly, the computing device can determine blocks written in multiple batches. The computing device can determine blocks written in multiple batches at the same time, such as determining blocks written in all batches at the same time. The computing device can also first determine a batch of blocks written, write the blocks written in this batch, and then determine the next batch of blocks written. This embodiment does not limit the timing of determination.
[0111] In some possible embodiments, for the same storage device among the M+N storage devices, the blocks written to the storage device in a batch of blocks written and the blocks written to the storage device in the previous batch of blocks written may belong to different stripes. Fig.10 For SSD1, in the first batch of blocks written, block D00 written to SSD1 belongs to stripe 0, in the second batch of blocks written, block D71 written to SSD1 belongs to stripe 7, in the third batch of blocks written, block D62 written to SSD1 belongs to stripe 6, in the fourth batch of blocks written, block D53 written to SSD1 belongs to stripe 5, in the fifth batch of blocks written, block D44 written to SSD1 belongs to stripe 4, in the sixth batch of blocks written, block D35 written to SSD1 belongs to stripe 3, in the seventh batch of blocks written, block D26 written to SSD1 belongs to stripe 2, in the eighth batch of blocks written, block P1 written to SSD1 belongs to stripe 8. In this order, in the eight batches of blocks written, the eight blocks written to SSD1 belong to eight different stripes. For example, see Fig.11For SSD1, among the blocks written in the first batch, block D00 written to SSD1 belongs to stripe 0, among the blocks written in the second batch, block D72 written to SSD1 belongs to stripe 7, among the blocks written in the third batch, block D64 written to SSD1 belongs to stripe 6, and among the blocks written in the fourth batch, block P5 written to SSD1 belongs to stripe 5. In this order, among the blocks written in 4 batches, the 8 blocks written to SSD1 belong to 4 different stripes respectively.
[0112] If a batch of blocks written includes blocks written in multiple processes, the blocks written to the storage device in one write process in the same batch and the blocks written to the storage device in the previous write process in the same batch may belong to different stripes. Fig.11 In the second write process of the first batch, block D00 written to SSD1 belongs to stripe 8, and in the third write process of the first batch, block D00 written to SSD1 belongs to stripe 4. Therefore, in this order, in the three write processes of the first batch, the three blocks written to SSD1 also belong to three different stripes. In the second write process of the second batch, block D33 written to SSD1 belongs to stripe 3, and in the third write process of the second batch, block DB3 written to SSD1 belongs to stripe 11. In this order, in the three write processes of the second batch, the three blocks written to SSD1 also belong to three different stripes.
[0113] In this way, since the blocks written to the same storage device belong to different stripes, different blocks of the same stripe will be stored in different storage devices respectively. For example, the 8 blocks of stripe 1 can be stored in 8 different SSDs respectively. For redundant algorithms such as EC algorithm or RAID algorithm, it is usually required that different blocks of the stripe are stored in different storage devices to perform data recovery. Therefore, this method can meet the requirements of the redundant algorithm.
[0114] 603. The computing device writes a batch of written blocks into M+N storage devices respectively.
[0115] The computing device can send K blocks in a batch of written blocks to M+N storage devices respectively, and the M+N storage devices can receive K blocks in a batch of written blocks respectively, cache K blocks in a batch of written blocks respectively, and perform persistent storage on the cached K blocks respectively. Among them, if K is greater than 1, for any storage device, the computing device can first write 1 block to the storage device in chronological order, and after the writing is completed, write the next block to the storage device until K blocks are written to the storage device.
[0116] If the storage device caches at most one block, then each time the storage device receives a block, the storage device can persistently store the last block written in the cache and update the last block written in the cache to the block written this time. Fig.10 , if SSD1 caches at most 1 block, when SSD1 receives D71, it can persistently store D00 in the cache and update D00 in the cache to D71. If the storage device caches at most K blocks, and K is greater than 1, then every time the storage device receives a block, the storage device can persistently store one or more cached blocks and update the blocks in the cache to the 1 block written this time. Among them, the older blocks with a longer cache time point can be persistently stored first. For example, see Fig.10 If SSD1 caches up to 3 blocks, when SSD1 receives D53, SSD1's cache includes D00, D71, and D62. SSD1 can persistently store the cached D00 and delete the cached D00. Of course, it can also persistently store the cached D00 and D71 and delete the cached D00 and D71. Of course, it can also persistently store the cached D00, D80, and D41 and delete the cached D00, D80, and D41.
[0117] It should be noted that step 603 is described using a batch of blocks written as an example. Similarly, the computing device can write multiple batches of blocks written into M+N storage devices respectively until all blocks of S stripes are written. Specifically, the computing device can write each batch of blocks written into M+N storage devices in sequence according to the chronological order. Accordingly, the M+N storage devices can receive each batch of blocks written in sequence according to the chronological order, cache each batch of blocks written in sequence, and perform persistent storage on each batch of blocks written in sequence. For example, if the computing device determines P batches of blocks written, the computing device will first write the first batch of blocks written into M+N storage devices, and then write the second batch of blocks written into M+N storage devices, and so on, until the Pth batch of blocks written is written into M+N storage devices. Accordingly, the M+N storage devices will first receive the blocks written in the first batch, cache the blocks written in the first batch, then receive the blocks written in the second batch, cache the blocks written in the second batch, and so on, until they receive the blocks written in the Pth batch and cache the blocks written in the Pth batch.
[0118] For example, see Fig.10, the computing device can complete the storage of 8 stripes by determining 8 batches of blocks to be written, and writing the 8 batches of blocks to 8 SSDs. Specifically, the computing device will write the first batch of 8 blocks to 8 SSDs. That is, write block D00 to SDD1, write block D10 to SDD2, write block D20 to SDD3, write block D30 to SDD4, write block D40 to SDD5, write block D50 to SDD6, write block D60 to SDD7, and write block D70 to SDD8; then write the second batch of 8 blocks to 8 SSDs, that is, write block D71 to SDD1, write block D01 to SDD2, write block D11 to SDD3, write block D21 to SDD4, and write block D30 to SDD5. 1 is written to SDD5, block D41 is written to SDD6, block D51 is written to SDD7, block D61 is written to SDD8, and so on, the 8 blocks of the 8th batch are written to 8 SSDs, that is, block P1 is written to SDD1, block P2 is written to SDD2, block P3 is written to SDD3, block P4 is written to SDD4, block P5 is written to SDD5, block P6 is written to SDD6, block P7 is written to SDD7, and block P0 is written to SDD8. In this way, the storage of stripes 0 to stripe 7 is completed.
[0119] One thing that needs to be explained is that Fig.10 The writing order shown is only an optional implementation. For a stripe including 7 data blocks and 1 check block, and a storage device caches at most 1 block, in some possible embodiments, the writing order can also be Fig.10 In other writing orders except 8 stripes are written to 8 storage devices. It is only necessary to ensure that the 8 blocks of each stripe are written to 8 different storage devices, and the total number of blocks written to the 8 storage devices by each stripe in each batch of writing does not exceed 1 block. In this way, the data protection function can be implemented, and any stripe of the 8 stripes can be prevented from being lost when all 8 storage devices lose power.
[0120] For example, see Fig.11, the computing device can complete the storage of 12 stripes by determining 4 batches of blocks to be written, and writing the 4 batches of blocks to 8 SSDs. Specifically, the computing device will divide the 24 blocks of the first batch into 3 writing processes and write them to 8 SSDs. That is, in the first writing process of the first batch, block D00 is written to SDD1, block D10 is written to SDD2, block D20 is written to SDD3, block D30 is written to SDD4, block D40 is written to SDD5, block D50 is written to SDD6, block D60 is written to SDD7, and block D70 is written to SDD8; in the second writing process of the first batch, block D80 is written to SDD1, block D90 is written to SDD2, block DA0 is written to SDD3, and block DB0 is written to SDD 4. Write block D01 to SDD5, write block D11 to SDD6, write block D21 to SDD7, write block D31 to SDD8, and so on. In the third writing process of the fourth batch, write block Q9 to SDD1, write block QA to SDD2, write block QB to SDD3, write block Q4 to SDD4, write block Q5 to SDD5, write block Q6 to SDD6, write block Q7 to SDD7, and write block Q8 to SDD8. In this way, the storage of stripes 0 to stripes 11 is completed.
[0121] One thing that needs to be explained is that Fig.11 The writing order shown is only an optional implementation. For a case where a stripe includes 6 data blocks and 2 check blocks, and the storage device caches a maximum of 3 blocks, in some possible embodiments, the writing order can also be Fig.11 In other writing orders other than the writing order shown, 12 stripes are written to 8 storage devices. It is only necessary to ensure that the 8 blocks of each stripe are written to 8 different storage devices, and the total number of blocks written to the 8 storage devices by each stripe in each 3 writing process does not exceed 2 blocks. In this way, the data protection function can be implemented, and any stripe of the 12 stripes can be prevented from being lost when all 8 storage devices lose power.
[0122] In a possible implementation, each block of each stripe obtained in step 601 can be represented by an element in an array, and accordingly, step 602 can be implemented by determining an element from the array.
[0123] Specifically, the S stripes can be represented as an array, which includes S rows and M+N columns. Each row in the S rows corresponds to the same stripe. For example, the first row can correspond to the first stripe, and the second row can correspond to the second stripe. Each column in the M+N columns corresponds to the same position in the S stripes. For example, the first column can correspond to the first position of each stripe, and the second column can correspond to the second position of each stripe. Each element of the array can represent a block in a stripe. For example, the element of the i-th row and the j-th column represents the j-th block of the i-th stripe, where i is a positive integer less than or equal to S, and j is a positive integer less than or equal to M+N. Exemplarily, if the S stripes are represented as the array dbuf[S][M+N], then dbuf[S][M+N] can be as follows, where each element in dbuf[S][M+N] can be Fig.10 or Fig.11 One Dij in .
[0124] dbuf[0][0], dbuf[0][1], ..., dbuf[0][M+N-2], dbuf[0][M+N-1]
[0125] dbuf[1][0], dbuf[1][1],..., dbuf[1][M+N-2], dbuf[1][M+N-1]
[0126] dbuf[2][0], dbuf[2][1],..., dbuf[2][M+N-2], dbuf[2][M+N-1]
[0127] …
[0128] dbuf[s-2][0], dbuf[s-2][1], ..., buf[s-2][M+N-2], dbuf[s-2][M+N-1]
[0129] dbuf[s-1][0], dbuf[s-1][1], ..., buf[s-1][M+N-2], dbuf[s-1][M+N-1]
[0130] When executing step 602, if the storage device caches at most 1 block, it can start from the first column of the array and traverse each column of the array. Whenever M+N elements are determined, the blocks corresponding to the determined M+N elements are used as blocks to be written in a batch. Among them, if S is equal to M+N, the elements of the M+Nth row happen to be the elements of the last row in the array. Then, the blocks corresponding to the elements of the first column can be determined as the blocks to be written in the first batch, the blocks corresponding to the elements of the second column can be determined as the blocks to be written in the second batch, and so on, the blocks corresponding to the elements of the M+Nth column can be determined as the blocks to be written in the M+Nth batch. If S is greater than M+N, there are elements of other rows after the elements of the M+Nth row in the array. Then, the blocks corresponding to the elements in the first row and first column to the elements in the first column of the M+Nth row can be determined as the blocks to be written in the first batch; starting from the elements in the first column of the M+N+1th row, the elements in the next row are determined. If the determined elements have reached M+N, the blocks corresponding to the determined M+N elements are used as the blocks to be written in the second batch. If the elements in the first column of the Sth row are determined, and the number of determined elements has not reached M+N, then the elements in the next row are determined starting from the elements in the second column of the first row until M+N elements are determined. The blocks corresponding to the determined M+N elements are used as the blocks to be written in the second batch, and so on. In addition, if the storage device can cache K blocks, and K is greater than 1, the method of determining blocks is similar to this, except that whenever (M+N)*K elements are determined, the blocks corresponding to the determined (M+N)*K elements are used as the blocks to be written in a batch.
[0131] Among them, in order to avoid writing multiple blocks of the same stripe to the same storage device, in the process of determining elements from the array, (M+N)*K elements can be used as a cycle, and each time (M+N)*K elements are determined, the storage device to which the first block to be written in the previous cycle can be used as the storage device to which the first element to be written in this cycle is written. That is, if the i-th storage device was used as the storage device to which the block corresponding to the first element determined last time was written when (M+N)*K elements were determined last time, then when (M+N)*K elements are determined this time, the i+1-th storage device can be used as the storage device to which the block corresponding to the first element determined this time is written. For any element from the second element to the last element of this cycle, the next storage device of the storage device to which the block corresponding to the previous element is written can be used as the storage device to which the block corresponding to the current element is written.
[0132] For example, see Fig.10, if M is 7, N is 1, and K is 1, every 8 elements can be 1 loop. The elements corresponding to the first loop are dbuf[0][0], dbuf[1][0] to dbuf[7][0]. The SSD to which dbuf[0][0] is to be written is SSD1. Then, starting from dbuf[1][0], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Then, the SSD to which dbuf[1][0] is to be written is SSD2, and the SSD to which dbuf[2][0] is to be written is SSD3. By analogy, the SSD to which dbuf[7][0] is to be written is SSD8.
[0133] The elements corresponding to the second loop are dbuf[0][1], dbuf[1][1] to dbuf[7][1]. Since dbuf[0][1] and dbuf[0][0] both belong to stripe 0, in order to ensure that different blocks of stripe 0 are written to different SSDs, it is necessary to avoid dbuf[0][1] and dbuf[0][0] being written to SSD1 at the same time. Also, since dbuf[1][1] and dbuf[1][0] both belong to stripe 1, in order to ensure that different blocks of stripe 1 are written to different SSDs, it is necessary to avoid dbuf[1][1] and dbuf[1][0] being written to SSD1 at the same time. D2, so SSD2 can be used as the SSD to be written to dbuf[0][1]. Starting from dbuf[1][1], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. The SSD to be written to dbuf[1][1] is SSD3, and the SSD to be written to dbuf[2][1] is SSD4. And so on, the SSD to be written to dbuf[6][1] is SSD8. Starting from dbuf[7][1], the first SSD is used as the SSD to be written again, and the SSD to be written to dbuf[7][1] is SSD1.
[0134] The elements corresponding to the third loop are dbuf[0][2], dbuf[1][2] to dbuf[7][2]. Since dbuf[0][2] and dbuf[0][1] both belong to stripe 0, in order to ensure that different blocks of stripe 0 are written to different SSDs, it is necessary to avoid dbuf[0][2] and dbuf[0][1] being written to SSD2 at the same time. Moreover, since dbuf[1][2] and dbuf[1][1] both belong to stripe 1, in order to ensure that different blocks of stripe 1 are written to different SSDs, it is necessary to avoid dbuf[1][2] and dbuf[1][1] being written to SSD3 at the same time. Therefore, SSD 3 is used as the SSD to be written to dbuf[0][2]. Starting from dbuf[1][2], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Then, the SSD to be written to dbuf[1][2] is SSD4, and the SSD to be written to dbuf[2][2] is SSD5. Similarly, the SSD to be written to dbuf[5][2] is SSD8. Starting from dbuf[5][2], the first SSD is used as the SSD to be written again. Then, the SSD to be written to dbuf[6][2] is SSD1, and the SSD to be written to dbuf[7][2] is SSD2.
[0135] For example, see Fig.11 , if M is 6, N is 2, and K is 3, every 24 elements can be a loop. The elements corresponding to the first loop are dbuf[0][0], dbuf[1][0]...dbuf[7][0], dbuf[0][1], dbuf[1][1]...dbuf[B][1]. The SSD to be written to dbuf[0][0] is SSD1. Starting from dbuf[1][0], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Therefore, the SSD to be written to dbuf[1][0] is SSD2, and the SSD to be written to dbuf[2][0] is SSD 3. By analogy, the SSD to be written to dbuf[7][0] is SSD8. Starting from dbuf[8][0], the first SSD is used as the SSD to be written. Then, the SSD to be written to dbuf[8][0] is SSD1. The SSD to be written to dbuf[9][0] is SSD2. By analogy, the SSD to be written to dbuf[3][1] is SSD8. Starting from dbuf[4][1], the first SSD is used as the SSD to be written. Then, the SSD to be written to dbuf[9][0] is SSD2. By analogy, the SSD to be written to dbuf[B][1] is SSD8.
[0136] The elements corresponding to the second loop are dbuf[0][2], dbuf[1][2], ... dbuf[B][2], dbuf[0][3], dbuf[1][3], ... to dbuf[B][3]. Since dbuf[0][2] and dbuf[0][0] both belong to stripe 0, in order to ensure that different blocks of stripe 0 are written to different SSDs, it is necessary to avoid dbuf[0][2] and dbuf[0][0] being written to the first SSD at the same time. In addition, since dbuf[1][2] and dbuf[1][0] both belong to stripe 1, in order to ensure that different blocks of stripe 1 are written to different SSDs, For SSD, we want to avoid dbuf[1][2] and dbuf[1][0] being written to the second SSD. Therefore, SSD2 can be used as the SSD to which dbuf[0][2] is to be written. Starting from dbuf[1][2], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Therefore, the SSD to which dbuf[1][2] is to be written is SSD3, and the SSD to which dbuf[2][2] is to be written is SSD4. Similarly, the SSD to which dbuf[6][2] is to be written is SSD8, and the SSD to which dbuf[7][2] is to be written is SSD1. As for the next element dbuf[7][2], dbuf[8][2], similarly, since SSD2 is the first disk to be written in the second loop, SSD2 is used as the SSD to be written to dbuf[8][2], just like dbuf[0][2]. Then, the SSD to be written to dbuf[9][2] is SSD3. Starting from dbuf[9][2], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Therefore, the SSD to be written to dbuf[A][2] is SSD4, and the SSD to be written to dbuf[B][2] is SSD5. Similarly, the SSD to be written to dbuf[2][3] is SSD8. Starting from dbuf[3][3], the first SSD is used as the SSD to be written again, and the SSD to be written to dbuf[3][3] is SSD1. For the next element dbuf[4][3] of dbuf[3][3], similarly, since SSD2 is the first disk to be written in the second loop, SSD2 is used as the SSD to be written to dbuf[4][3], just like dbuf[0][2], and so on.
[0137] The elements corresponding to the third loop are dbuf[0][4], dbuf[1][4], ... dbuf[B][4], dbuf[0][5], dbuf[1][5], ... to dbuf[B][5]. Since dbuf[0][4] and dbuf[0][2] both belong to stripe 0, in order to ensure that different blocks of stripe 0 are written to different SSDs, it is necessary to avoid dbuf[0][4] and dbuf[0][2] being written to SSD2 at the same time. In addition, since dbuf[1][4] and dbuf[1][2] both belong to stripe 0, Belongs to stripe 1. To ensure that different blocks of stripe 1 are written to different SSDs, it is necessary to avoid dbuf[1][4] and dbuf[1][2] being written to SSD3. Therefore, SSD3 can be used as the SSD to which dbuf[0][4] is to be written. Then, starting from dbuf[1][4], the previous element is to be written to the i-th SSD, and the next element is to be written to the i+1-th SSD. Then, the SSD to which dbuf[1][4] is to be written is SSD4, and the SSD to which dbuf[2][4] is to be written is SSD5, and so on.
[0138] If the array is dbuf[s][M+N] as shown above, the column number is y. In the process of determining the element, if the remainder of the column number y*S / (M+N) of the current element is 0, it can be determined that the current element is the first element of this loop. If the remainder of the column number y*S / (M+N) of the current element is not 0, it can be determined that the current element is not the first element of this loop. Therefore, in the process of determining the element, for dbuf[0][y], before encountering the dbuf[0][y] element whose remainder y*s / (m+n) is 0, each storage device can be used as a storage device to which the block corresponding to the corresponding element is to be written in the order from the first storage device to the M+Nth storage device. For example, if the stripe includes 6 data blocks and 2 check blocks, and each storage device can cache 3 blocks, the values of y for which the remainder of y*S / (M+N) is 0 are 0, 2, 4, and 6, respectively. If the first storage device is used as the storage device to which dbuf[0][0] is to be written, before encountering dbuf[0][2], the next storage device can be used as the storage device to which the next element is to be written. When encountering dbuf[0][2], the second storage device can be used as the storage device to which dbuf[0][2] is to be written, and then, before encountering dbuf[0][4], the next storage device can be used as the storage device to which the next element is to be written. When encountering dbuf[0][4], the third storage device can be used as the storage device to which dbuf[0][4] is to be written, and then the next storage device can be used as the storage device to which the next element is to be written, and so on.
[0139] By this sorting method, when executing step 603, if the writing process starts from the xth storage device, the x+1th storage device, the x+2th storage device, and the x+3th storage device will be written in sequence until the M+Nth storage device is written, and the writing will continue from the first storage device until the x-1th storage device is written. Then, the next writing process is executed, and the writing continues from the xth storage device, and the x+1th storage device, the x+2th storage device, and the x+3th storage device will be written in sequence until the M+Nth storage device is written, and the writing will continue from the first storage device until the x-1th storage device is written. If the next cycle is entered, that is, (M+N)*K blocks are written, the writing starts from the x+1th storage device, and the x+2th storage device, the x+3th storage device, and the x+Nth storage device are written in sequence, and the writing will continue from the first storage device until the xth storage device is written.
[0140] For example, if the array is dbuf[s][M+N] as shown above, S=M+N, then the write order may be as follows:
[0141] dbuf[0][0], dbuf[1][0], dbuf[2][0], ..., dbuf[s-2][0], dbuf[s-1][0]
[0142] dbuf[s-1][1], dbuf[0][1], dbuf[1][1],…, dbuf[s-3][1], dbuf[s-2][1]
[0143] …
[0144] dbuf[2][s-2], dbuf[3][s-2],…, dbuf[0][s-2], dbuf[1][s-2]
[0145] dbuf[1][s-1], dbuf[2][s-1],…, dbuf[s-2][s-1], dbuf[0][s-1]
[0146] Each row represents a batch of blocks written, each column corresponds to a storage device, and each element represents a block to be written to the corresponding storage device in the corresponding batch. For example, dbuf[0][0], dbuf[1][0], ..., dbuf[M+N-2][0], dbuf[s-1][0] represent the blocks written in the first batch, dbuf[0][0], dbuf[0][1], ...dbuf[0][M+N-1] correspond to the first storage device, and dbuf[1][1] represents the blocks written in the second batch to the first storage device. By determining such a writing order, in step 603, when the blocks written in the first batch are written into the M+N storage devices respectively, the block corresponding to dbuf[0][0], that is, the first block of the first stripe, is written to the first storage device in the order shown in the first row, and the block corresponding to dbuf[1][0], that is, the first block of the second stripe, is written to the second storage device, and so on, the block corresponding to dbuf[s-1][0], that is, the first block of the Sth stripe, is written to the M+Nth storage device. When writing the blocks written in the second batch to the M+N storage devices respectively, the block corresponding to dbuf[0][1], that is, the second block of the first stripe, is written to the first storage device in the order shown in the second row, and the block corresponding to dbuf[1][1], that is, the second block of the second stripe, is written to the second storage device. Similarly, the block corresponding to dbuf[s-2][1], that is, the second block of the S-1th stripe, is written to the M+Nth storage device, and the block corresponding to dbuf[s-1][1], that is, the second block of the Sth stripe, is written to the first storage device.
[0147] It should be noted that the above is only an exemplary description of the method for determining the writing order of blocks, and does not limit the method for determining the writing order of blocks to be in this way. In this embodiment, it is only necessary to ensure that after determining the blocks to be written in each batch, the number of blocks belonging to the same stripe in each batch of blocks written is less than or equal to N. This embodiment does not specifically limit how to determine the writing order of each block.
[0148] 604. If a power failure event is detected, the computing device obtains the first block and the second block in the S stripes.
[0149] A power-off event is an event in which the power supply of the storage device stops providing power to the storage device, such as an event in which the storage device is unplugged, a device where the storage device is located loses power, etc. If the storage device loses power, if the storage device is not equipped with a backup power supply, the data cached by the storage device may be lost, and the computing device can recover the lost data by executing steps 604 and 605. Optionally, the power-off event can be an event in which all M+N storage devices lose power. In an exemplary scenario, the computing device can be a storage server, and the M+N storage devices can be M+N SSDs inserted into a storage server. If the storage server loses power, all M+N SSDs will lose power, and the storage server can execute this step after powering on again.
[0150] The first block is a block that has not been written to the M+N storage devices. Specifically, the first block may be a block of each write process that has not been executed. For example, if S stripes need to be written through W write processes in total, and the first write process to the Pth batch write process have been executed when the power is off, the first block may be a block of the P+1th write process to the Wth write process. For example, see Fig.12 , 12 stripes are written through 12 writing processes. If the power-off time point is time point T6, and T6 corresponds to the 6th writing process (i.e., the 1st writing process of the 3rd batch), then when the power is off, the computing device has executed the 7th writing process, and has not yet executed the 8th writing process to the 12th writing process. Therefore, the first block can be the block of the 8th writing process to the 12th writing process, that is, the block corresponding to the time point T7 to the time point T11.
[0151] Regarding the specific process of obtaining the first block, the first block can be pre-cached in a data buffer of the computing device, and the computing device can obtain the first block from the data buffer. If the data buffer is a volatile memory, the data buffer can be protected by a backup power supply. When a power outage occurs, the first block in the data buffer can be stored in a certain storage space by the power provided by the backup power supply. After power is turned on, the computing device can re-store the first block from the storage space into the data buffer, and then read the first block from the data buffer. If the data buffer is a non-volatile memory, the first block in the data buffer will not be lost during the power outage, so after power is turned on, the computing device can directly read the first block from the data buffer.
[0152] The second block is a block that has been persistently stored in M+N storage devices. The second block may not be lost when power is off.
[0153] Among them, if S stripes need to be completed through W batches of writing in total, each storage device caches blocks written at most K times, and when the power is off, the computing device has executed the first write process to the Pth write process, then the blocks cached by the M+N cache devices are blocks from the (PK-1)th write process to the (P-1)th batch write process. These blocks may not have been persistently stored, and the blocks written before these blocks, that is, the blocks from the first write process to the (PK)th process have been persistently stored. Therefore, the second block can be the block from the first write process to the (PK)th write process.
[0154] For example, see Fig.12 , 12 stripes need to be written through 12 write processes in total. If the storage device caches blocks written 3 times at most, and the power-off time point is time point T6, then when the power is off, the blocks cached by M+N storage devices can be the blocks corresponding to time point T4 to the blocks corresponding to time point T6, that is, the blocks of the 5th write process to the blocks written in the 7th process, and the blocks written before time point T4 have been persistently stored, so the second block can be the blocks corresponding to time point T0 to time point T3, that is, the blocks of the 1st write process to the 4th write process.
[0155] Regarding the specific process of obtaining the second block, the second block may be stored in a persistent storage medium of a storage device, and the computing device may read the second block from the persistent storage medium of the storage device. Taking an SSD as an example, the computing device may read the second block from a NAND flash of the SSD.
[0156] 605. The computing device recovers data of the lost blocks in the S stripes according to the first block and the second block.
[0157] The computing device can detect whether each of the S stripes is complete based on the first block and the second block; if a block is lost in any stripe, the computing device can obtain the remaining blocks of the stripe from the first block and the second block; the computing device can recover the lost blocks in the stripe through the RAID algorithm or the EC algorithm based on the remaining blocks of the stripe. Among them, since the number of blocks belonging to the same stripe in each batch of blocks written is less than or equal to N, when power is lost at any time point, the number of blocks cached in the M+N storage devices of the same stripe will be less than or equal to N, and the number of lost blocks in the same stripe will also be less than or equal to N. Therefore, even if each stripe loses a block, the lost block can be recovered through the remaining blocks and the redundancy algorithm, so each stripe will not be truly lost and can be protected.
[0158] For example, see Fig.13 , the data recovery process can include the following steps:
[0159] Step 1: Restore the data in the data buffer when the power is off.
[0160] Step 2: Read data from the storage device
[0161] Step 3: Check whether the data is complete based on the data in the data buffer and the data read out from the storage device. If the data is incomplete, execute step 4. If the data is complete, the process ends.
[0162] Step 4: Recover data according to the data in the data buffer and the data read out from the storage device in accordance with the RAID algorithm or the EC algorithm.
[0163] Step 5: Determine whether the lost data in all stripes has been recovered. If the data in any stripe has not been recovered, continue to step 4. If the data in all stripes has been recovered, the process ends.
[0164] This embodiment provides a solution that can avoid the loss of stripes cached by storage devices without relying on the backup power supply of storage devices when all storage devices lose power at the same time. By improving the order in which each block in each stripe is written to the storage device, from writing M+N blocks in a batch to writing less than or equal to N blocks in a batch, for each stripe, since the number of blocks in the stripe that are simultaneously written to M+N storage devices is less than or equal to N, the number of blocks in the stripe that are simultaneously cached to M+N storage devices will also be less than or equal to N. Therefore, even if all M+N storage devices lose power at any point in time, causing the data cached by the M+N storage devices to be completely lost, the number of lost blocks in the stripe will be less than or equal to N, so the stripe can recover the lost blocks through a redundancy algorithm, thereby avoiding the loss of data in the stripe. Since this solution can avoid data loss and ensure the reliability of data storage even if the storage device does not have a backup power supply, the storage device does not need to install a backup power supply, so the internal space occupied by the backup power supply in the storage device can be saved, allowing the storage device to free up more internal space for storage media, thereby increasing the storage capacity of the storage device. In addition, the cost of manufacturing the backup power supply can be saved, thereby saving the cost of the storage device, and in addition, the backup power supply can be prevented from limiting the life of the storage device.
[0165] Fig.14 is a schematic diagram of a data storage device provided in an embodiment of the present application. Fig.14 As shown, the device comprises:
[0166] The determination module 1401 is used to execute step 602 ; the writing module 1402 is used to execute step 603 .
[0167] Optionally, the number of blocks belonging to the same stripe in the batch of blocks written is less than or equal to N, including: when N is equal to 1, each block in the batch of blocks written belongs to a different stripe.
[0168] Optionally, for each storage device in the M+N storage devices, the blocks written to the storage device in the batch of blocks written and the blocks written to the storage device in the last batch of blocks written belong to different stripes.
[0169] Optionally, S is obtained according to M, N, and K, where K is the maximum number of blocks cached by each storage device.
[0170] Optionally, S is obtained by the following formula:
[0171]
[0172] Wherein, ceil represents a round-up operation, and K represents the maximum number of blocks cached by each storage device.
[0173] Optionally, the device further includes: an acquisition module, used to execute step 604; and a recovery module, used to execute step 605.
[0174] One thing that needs to be explained is that Fig.14 The data storage device provided in the embodiment only uses the division of the above-mentioned functional modules as an example to illustrate when storing data. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the computing device is divided into different functional modules to complete all or part of the functions described above. In addition, the data storage device provided in the above embodiment and the data storage method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0175] As a possible product form, the computing device in the embodiment of the present application can be implemented by a chip. The chip includes a processing circuit and an output interface connected to and communicating with the processing circuit; the processing circuit is used to execute steps 601, 602, 604, and 605; the output interface is used to execute step 603. Optionally, the chip may also include a storage medium, which is used to store instructions executed by the processing circuit.
[0176] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0177] As a possible product form, the computing device described in the embodiment of the present application can also be implemented using the following: one or more field programmable gate arrays (Full name: field-programmable gate array, English abbreviation: FPGA), programmable logic devices (Full name: programmable logic device, English abbreviation: PLD), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout the present application.
[0178] Those of ordinary skill in the art will appreciate that the various method steps and modules described in the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0179] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0180] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, or it can be an electrical, mechanical or other form of connection.
[0181] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0182] In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.
[0183] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or partly contributed to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0184] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A data storage method, applied to a computing device, characterized in that: The method comprises: The computing device determines blocks that need to be written to the storage system in the same batch from a plurality of stripes; each stripe includes M data blocks and N check blocks; the number of blocks from the same stripe in the blocks written to the storage device in the same batch is less than or equal to N; M and N are both positive integers; The computing device writes the blocks that need to be written to the storage system in the same batch into corresponding storage devices in the storage system respectively, wherein, after determining the blocks that need to be written to the storage system in the same batch, the next storage device of the storage device to which the first block in the previous same batch is to be written is used as the storage device to which the first block to be written this time is determined, wherein the blocks that need to be written to the storage system in the same batch include (M+N)*K, K is the number of blocks that can be cached by the storage devices in the storage system, and K is an integer greater than or equal to 1.
2. The method according to claim 1, characterized in that The number of blocks written to the storage system in the same batch is M+N.
3. The method according to claim 1 or 2, characterized in that: For a storage device that receives a written block, two adjacent written blocks come from different stripes.
4. The method according to claim 1 or 2, characterized in that: After writing the blocks that need to be written to the storage system in the same batch into corresponding storage devices in the storage system respectively, the method further includes: If a power failure event is detected, the computing device obtains blocks in the plurality of stripes that have not been written to the storage system and blocks that have been persistently stored in the storage system; The computing device performs data recovery on lost blocks in the multiple stripes according to the blocks that have not been written to the storage system and the blocks that have been persistently stored.
5. A data storage device, characterized in that: include: A determination module, used to determine blocks that need to be written to the storage system in the same batch from multiple stripes; each stripe includes M data blocks and N check blocks; the number of blocks from the same stripe in the blocks written to the storage device in the same batch is less than or equal to N; M and N are both positive integers; A writing module is used to write the blocks that need to be written to the storage system in the same batch into corresponding storage devices in the storage system respectively, wherein, when the blocks that need to be written to the storage system in the same batch are determined, the next storage device of the storage device to which the first block in the same batch is to be written is used as the storage device to which the first block to be written this time is determined, wherein the blocks that need to be written to the storage system in the same batch include (M+N)*K, K is the number of blocks that can be cached by the storage devices in the storage system, and K is an integer greater than or equal to 1.
6. The device according to claim 5, characterized in that The number of blocks written to the storage system in the same batch is M+N.
7. The device according to claim 5 or 6, characterized in that For a storage device that receives a written block, two adjacent written blocks come from different stripes.
8. The device according to claim 5 or 6, characterized in that Also includes: an acquisition module, configured to, after writing the blocks that need to be written to the storage system in the same batch to the corresponding storage devices in the storage system respectively, acquire the blocks in the multiple stripes that have not been written to the storage system and the blocks that have been persistently stored in the storage system if a power failure event is detected; A recovery module is used to recover data of lost blocks in the multiple stripes according to the blocks that have not been written to the storage system and the blocks that have been persistently stored.
9. An adapter, characterized in that: The adapter comprises an interface and a processor; the interface communicates with the processor; and the processor is used to execute the method described in any one of claims 1-4.
Citation Information
Patent Citations
Method, device and system for transmitting video data
CN101166273A