Method for data storage, electronic device, and computer program product
By allocating storage units in the multi-copy data storage and generating verification information for only part, the problem of CPU and memory consumption for generation verification information is solved, and an efficient data storage solution is realized.
Patent Information
- Application Number
- CN202110014482.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-06
- Publication Date
- 2025-07-04
- Estimated Expiration
- 2041-01-06
AI Technical Summary
In multi-copy data storage scenarios, the CPU and memory resources required to generate verification information are consumed too much, especially when a large number of storage units need to generate verification information, resulting in inefficiency.
By allocating multiple storage units based on the target data and the number of replicas, and generating verification information for only some storage units, it is directly applied to other storage units, reducing the number of verification information generation tasks, and using parallel processing methods to improve efficiency and reduce memory consumption.
It significantly reduces the CPU resource and time consumption of multi-copy data storage, reduces memory requirements, and improves data storage efficiency.
Smart Images

Figure CN114721587B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure generally relate to the field of data storage, and more particularly, to methods, electronic devices, and computer program products for data storage. Background Art
[0002] In a storage system, data can be stored in one or more pre-partitioned storage units (chunks). To ensure data is not lost, each storage unit may include a plurality of data segments configured to store data and a plurality of coding segments configured to store parity information (e.g., erasure code). The parity information stored in the plurality of coding segments can be generated based on the data stored in the plurality of data segments. The data segments and coding segments of each storage unit can be stored in a plurality of storage devices. In this way, when a certain storage device fails, the data stored in that storage device can be recovered through the data and / or parity information stored in other storage devices.
[0003] In some application scenarios, a user may need to store multiple copies of the same data. Due to the large amount of data, a large amount of parity information may need to be generated for a large number of storage units, which will consume a large amount of CPU and memory resources. Summary of the Invention
[0004] Embodiments of the present disclosure provide methods, electronic devices, and computer program products for data storage.
[0005] In a first aspect of the present disclosure, a method for data storage is provided. The method includes: receiving a request to store target data, where the request includes the target data and indicates the number of copies of the target data to be stored; allocating a plurality of storage units for storing copies of the target data based on the target data and the number of copies, where each storage unit includes a data segment configured to store data and a coding segment configured to store parity information, and a set of storage units among the plurality of storage units are respectively configured to store a first number of copies of the target data; writing the first number of copies to the data segments of each storage unit in the set of storage units; generating first parity information for verifying the data stored in the data segment of one storage unit in the set of storage units; and writing the first parity information to the coding segments of each storage unit in the set of storage units.
[0006] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform actions, the actions including: receiving a request to store target data, where the request includes the target data and indicates the number of copies of the target data to be stored; based on the target data and the number of copies, allocating a plurality of storage units for storing the copies of the target data, where each storage unit includes a data segment configured to store data and an encoding segment configured to store check information, and a set of the plurality of storage units are respectively configured to store a first number of copies of the target data; writing the first number of copies to the data segments of each storage unit in the set of storage units respectively; for one storage unit in the set of storage units, generating first check information for verifying the data stored in the data segment of the one storage unit; and writing the first check information to the encoding segments of each storage unit in the set of storage units respectively.
[0007] In a third aspect of the present disclosure, a computer-readable storage medium is provided, on which machine-executable instructions are stored. When executed by a device, the machine-executable instructions cause the device to perform any steps of the method described in the above first aspect.
[0008] In a fourth aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any steps of the method described in the first aspect of the present disclosure.
[0009] The summary of the invention is provided to introduce a selection of concepts in a simplified form, which will be further described in the detailed implementation below. The summary of the invention is not intended to identify the key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. Brief Description of the Drawings
[0010] By describing the exemplary embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become more apparent, where, in the exemplary embodiments of the present disclosure, the same reference numerals generally represent the same components.
[0011] Figure 1 A schematic diagram showing an example environment in which the embodiments of the present disclosure can be implemented;
[0012] Figure 2 A schematic diagram showing an example storage unit of the embodiments of the present disclosure;
[0013] Figure 3 The flowchart of an example method for data storage according to an embodiment of the present disclosure is shown;
[0014] Figure 4 The schematic diagram of storage units allocated for the total effective data volume according to an embodiment of the present disclosure is shown;
[0015] Figure 5 The schematic diagram of an example data block according to an embodiment of the present disclosure is shown; and
[0016] Figure 6 The schematic block diagram of an example device that can be used to implement the embodiments of the present disclosure is shown.
[0017] In the respective drawings, the same or corresponding reference numerals denote the same or corresponding parts. Detailed Description of Specific Embodiments
[0018] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure will be more thorough and complete, and can fully convey the scope of the present disclosure to those skilled in the art.
[0019] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included hereinafter.
[0020] As described above, in a storage system, data can be stored in one or more pre-divided storage units (chunks). To ensure that data is not lost, each storage unit may include a plurality of data segments configured to store data and a plurality of coding segments configured to store check information (e.g., erasure code). The check information stored in the plurality of coding segments can be generated based on the data stored in the plurality of data segments. The data segments and coding segments of each storage unit can be stored in a plurality of storage devices. In this way, when a certain storage device fails, the data stored in that storage device can be recovered through the data and / or check information stored in other storage devices.
[0021] In a multi-copy data storage scenario (e.g., digital video recording), a user may need to store multiple copies of the same data. Multiple copies of the data can be stored in a large number of storage units. Typically, a single CPU core can be utilized to generate check information for data in a data segment in one storage unit. Multiple CPU cores can be utilized to generate corresponding check information for multiple storage units in parallel. In a multi-copy data storage scenario, due to the large amount of data, check information needs to be generated for a large number of storage units. When the number of CPU cores is insufficient, the check information can only be generated in multiple batches, which will consume a long time. In addition, generating check information requires reading the data segment data into memory. When generating check information for a large number of storage units, the memory consumed will be huge.
[0022] Embodiments of the present disclosure propose a solution for data storage to solve one or more of the above problems and other potential problems. In this solution, based on the target data to be stored and the number of its copies, multiple storage units for storing the copies of the target data are allocated. At least some of the multiple storage units can be configured to store the same number of copies of the target data. In this case, check information can be generated only for one of the at least some storage units, and the check information can be directly applied to other storage units in the at least some storage units (i.e., there is no need to perform the check information generation task for other storage units). In some cases, the multiple storage units may further include at least one storage unit configured to store another number of copies of the target data. Check information can be generated for the at least one storage unit. In this way, the number of required check information generation tasks can be greatly reduced, thereby improving the efficiency of multi-copy data storage while reducing memory consumption.
[0023] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Figure 1 A block diagram of an example environment 100 is shown. As Figure 1 shown, the environment 100 includes a host 110, a storage manager 120, and multiple storage devices 130-1, 130-2... 130-N (collectively or individually referred to as "storage devices 130", where N > 1). It should be understood that the structure of the environment 100 is described only for exemplary purposes and does not imply any limitation on the scope of the present disclosure. For example, embodiments of the present disclosure can also be applied to environments different from the environment 100.
[0024] The host 110 can be, for example, any physical computer, virtual machine, server, etc. that runs user applications. The host 110 can send input / output (I / O) requests to the storage manager 120, such as requests to read data from the storage device 130 and / or write data to the storage device 130, etc. In response to receiving a read request from the host 110, the storage manager 120 can read the stored data from the storage device 130 and return the read data to the host 110. In response to receiving a write request from the host 110, the storage manager 120 can write the data to be stored to the storage device 130. The storage device 130 can be any currently known or future-developed non-volatile storage medium, such as a disk, solid-state drive (SSD), or disk array, etc.
[0025] To store data in the storage device 130, one or more storage units for storing data can be partitioned. For example, each storage unit can be used to store a fixed amount of data. Figure 2 A schematic diagram of an example storage unit 210 according to an embodiment of the present disclosure is shown.
[0026] As Figure 2 shown, the storage unit 210 can include, for example, 12 data segments 210-1, 210-2... 210-12 configured to store data and 4 coding segments 210-13, 210-14... 210-16 configured to store check information. The check information (e.g., erasure code) stored in the coding segments 210-13, 210-14... 210-16 can be generated based on the data stored in the data segments 210-1, 210-2... 210-12 to ensure that when at most 4 segments among the segments 210-1, 210-2... 210-16 are lost (e.g., due to a failure of the storage device), the lost data can be recovered from the remaining segments. As Figure 2 shown, for example, the segments 210-1, 210-2... 210-16 in the data unit 210 can be separately stored in the storage devices 130-1, 130-2, 130-3, and 130-4. In this way, when a certain storage device among the storage devices 130-1, 130-2, 130-3, and 130-4 fails, the data in the failed device can be recovered using the data and / or check information in other storage devices.
[0027] It should be understood that Figure 2 the number of data segments and coding segments shown in
[0028] Figure 3FIG. 300 is a flowchart of an example method for storing data according to an embodiment of the present disclosure. Method 300 can be performed, for example, by a storage manager 120 as shown in Figure 1 . Method 300 will be described in detail below in conjunction with Figure 1 .
[0029] As Figure 3 shown, at block 310, the storage manager 120 receives a request to store target data, where the request includes the target data and indicates the number of copies of the target data to be stored.
[0030] In some embodiments, the storage manager 120 may receive the request from the host 110. For example, the request indicates to store 2000 copies of 4.5 MB of video data. That is, the amount of valid data to be stored is 4.5 MB × 2000 = 9 GB. For illustrative purposes only and not by way of limitation, various details of the embodiments of the present disclosure will be described in detail below in conjunction with this example.
[0031] At block 320, the storage manager 120 allocates a plurality of storage units for storing copies of the target data based on the target data and the number of copies. In some embodiments, each storage unit may include a plurality of data segments configured to store data and a plurality of coding segments configured to store parity information. In some embodiments, a set of the plurality of storage units are respectively configured to store a first number of copies of the target data.
[0032] Referring to the above example, assume that each allocated storage unit can be used to store 128 MB of valid data. Since some additional information can be stored when each copy is stored in the corresponding storage unit, the number of allocated storage units may be slightly greater than the ratio of the total amount of valid data to the amount of valid data that each storage unit can store.
[0033] Figure 4 FIG. shows a schematic diagram of storage units allocated for the total amount of valid data according to an embodiment of the present disclosure. As Figure 4As shown, for an effective data volume 410 of, for example, 9 GB, 72 storage units 420-1 to 420-71 and 430 can be allocated. Among them, the storage units 420-1 to 420-71 are respectively configured to store the same number (i.e., the first number) of copies of the target data. For example, 128 / 4.5 = 28 copies. The remaining storage unit 420-72 is configured to store the remaining number of copies. That is, 2000 - 28 × 71 = 12 copies. Taking the storage unit 420-1 as an example, it can include a plurality of data segments 421-1 and a plurality of coding segments 422-1... Taking the storage unit 420-71 as an example, it can include a plurality of data segments 421-71 and a plurality of coding segments 422-71. Taking the storage unit 430 as an example, it can include a plurality of data segments 431 and a plurality of coding segments 432. Hereinafter, the storage units 420-1 to 420-71 are collectively referred to as or individually become "storage unit 420", the data segments 421-1 to 421-71 are collectively referred to as or individually become "data segment 421", and the coding segments 422-1 to 422-71 are collectively referred to as or individually become "coding segment 422".
[0034] At block 330, the storage manager 120 writes the first number of copies to the data segments of each storage unit in the group of storage units. In some embodiments, each of the first number of copies can be encapsulated in a data block. The data block can include the corresponding copy and its additional information.
[0035] Figure 5 A schematic diagram of an example data block 500 according to an embodiment of the present disclosure is shown. As Figure 5 shown, for example, the data block 500 can include fields 510 to 540. The field 520 can store, for example, a copy of the target data (e.g., 4.5 MB of video data), and the field 510 can indicate, for example, the length of the data stored in the field 520. The field 530 can store, for example, a checksum calculated for the data stored in the field 520. The field 540 can store, for example, an identifier associated with the write request to identify which write request the data stored in the field 520 is associated with. In some embodiments, newly created storage units can reuse the data blocks used by the deleted storage units. The field 540 can be used to ensure that the old data in the data block is not regarded as valid data. For example, if the identifier stored in the field 540 does not match the current write request, it can be determined that the data in the data block is old data.
[0036] In some embodiments, the storage manager 120 may write a first number of data blocks to the data segments of each storage unit in the set of storage units. For example, the write operations for the data segments of each storage unit may be performed in parallel. In this way, the first number of replicas are written to the data segments of each storage unit in the set of storage units. In some embodiments, in response to the first number of replicas being written to the data segment of a storage unit, if it is determined that there is still unoccupied space in the data segment of the storage unit, the storage manager 120 may fill the space with invalid data (e.g., 0). In this way, the data stored in the data segments of each storage unit in the set of storage units is exactly the same.
[0037] For example, in Figure 4 the example shown, the storage manager 120 may write 28 data blocks to the data segment 421 of each storage unit 420 among storage units 420-1 to 420-71, and fill the remaining space in the data segment 421 of each storage unit 420 with 0. In this way, the data stored in the data segments of storage units 420-1 to 420-71 is exactly the same.
[0038] Return reference Figure 3 , at block 340, the storage manager 120 generates first check information based on the data stored in the data segment of one storage unit in the set of storage units. Then, at block 350, the storage manager 120 writes the first check information to the coding segments of each storage unit in the set of storage units. In some embodiments, the write operations for the coding segments of each storage unit may be performed in parallel.
[0039] For example, in Figure 4 the example shown, the storage manager 120 may generate an erasure code based on the data (including 28 data blocks and the filled invalid data) in the data segment 421 of any one storage unit 420 among storage units 420-1 to 420-71. The storage manager 120 may write the generated erasure code to the coding segment 422 of each storage unit 420. As described above, since the data stored in the data segments 421 of storage units 420-1 to 420-71 is exactly the same, the check information stored in their respective coding segments 422 is also exactly the same. In this way, it is possible to avoid generating respective erasure codes for 71 storage units.
[0040] In some embodiments, according to the target data size and the number of copies to be stored, the multiple allocated storage units may include only a set of storage units configured to store the same number (e.g., a first number) of copies. Additionally, in some other embodiments, the multiple storage units may further include at least one storage unit configured to store a second number of copies of the target data, where the second number is different from the first number. For example, in Figure 4 In the example shown, storage unit 430 is configured to store the remaining number of copies. That is, 2000 - 28×71 = 12 copies.
[0041] In this case, as Figure 3 shown, at block 360, storage manager 120 writes the second number of copies to the data segment of the at least one storage unit.
[0042] In some embodiments, each of the second number of copies may be encapsulated in a data block. Storage manager 120 may write the second number of data blocks to the data segment of the at least one storage unit. For example, each of the second number of data blocks may have a structure as Figure 4 shown. In some embodiments, in response to the second number of copies being written to the data segment of the at least one storage unit, if it is determined that there is still unoccupied space in the data segment of the at least one storage unit, storage manager 120 may fill the space with invalid data (e.g., 0).
[0043] For example, in Figure 4 the example shown, storage manager 120 may write 12 data blocks to data segment 431 of storage unit 430 and fill the remaining space in data segment 431 with 0.
[0044] At block 370, storage manager 120 generates second check information based on the data stored in the data segment of the at least one storage unit. Then, at block 380, storage manager 120 writes the second check information to the coding segment of the at least one storage unit.
[0045] For example, in Figure 4 the example shown, storage manager 120 may generate an erasure code based on the data (including 12 data blocks and the filled invalid data) in data segment 431 of storage unit 430. Storage manager 120 may write the generated erasure code to coding segment 432 of storage unit 430.
[0046] Although in Figure 5In the embodiment, blocks 360 to 380 are shown as being executed after block 350. It should be understood that blocks 360 to 380 may also be executed in parallel with blocks 330 to 350 to further reduce the time required for data storage. In addition, method 300 may also include Figure 3 Additional blocks not shown and / or may be omitted Figure 3 The scope of the present disclosure is not limited in this regard.
[0047] It can be seen from the above description that in the traditional solution, storing 2000 copies of 4.5M video data requires generating respective verification information for 72 storage units. That is, the number of verification information generation tasks is 72. In the solution according to the embodiment of the present disclosure, storing 2000 copies of 4.5M video data only requires generating respective verification information for 2 storage units. That is, the number of verification information generation tasks is 2. Therefore, the embodiment of the present disclosure can significantly reduce the CPU resources and time consumed by multi-copy data storage. In addition, in the traditional solution, storing 2000 copies of 4.5M video data requires more than 9GB of memory space. In the solution according to the embodiment of the present disclosure, storing 2000 copies of 4.5M video data only requires 2×128=256MB of memory space. Therefore, the embodiment of the present disclosure can significantly reduce the memory consumption of multi-copy data storage.
[0048] Figure 6 Schematic block diagram of an example device 600 that can be used to implement embodiments of the present disclosure is shown. Figure 1 The storage manager 120 shown may be implemented by the device 600. Figure 6 As shown, the device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 602 or computer program instructions loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0049] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0050] Each of the processes and treatments described above, such as method 300, may be executed by processing unit 601. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed onto device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by CPU 601, one or more actions of method 300 described above may be performed.
[0051] The present disclosure may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for performing various aspects of the present disclosure.
[0052] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed to be a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.
[0053] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to various computing / processing devices, or may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.
[0054] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.
[0055] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer - readable program instructions.
[0056] These computer - readable program instructions can be provided to a processing unit of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine, such that the instructions, when executed by the processing unit of the computer or other programmable data - processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions comprises a manufacture including instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.
[0057] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.
[0058] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.
[0059] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for data storage, comprising: Receiving a request to store target data, wherein the request includes the target data and indicates the number of copies of the target data to be stored; Based on the target data and the number of copies, allocating a plurality of storage units for storing the copies of the target data, wherein each storage unit includes a data segment configured to store data and an encoding segment configured to store check information, and a set of storage units among the plurality of storage units are respectively configured to store a first number of copies of the target data; Writing the first number of copies to the data segments of each storage unit in the set of storage units respectively; For one storage unit in the set of storage units, generating first check information based on the data stored in the data segment of the one storage unit; And Writing the first check information to the encoding segments of each storage unit in the set of storage units respectively.
2. The method according to claim 1, wherein at least one storage unit among the plurality of storage units is configured to store a second number of copies of the target data, and the method further comprises: Writing the second number of copies to the data segment of the at least one storage unit; Generating second check information based on the data stored in the data segment of the at least one storage unit; And Writing the second check information to the encoding segment of the at least one storage unit.
3. The method according to claim 1, wherein writing the first number of copies to the data segments of each storage unit in the set of storage units respectively includes: Generating the first number of data blocks based on the first number of copies, wherein each data block includes a copy and an identifier associated with the request; And Writing the first number of data blocks to the data segments of each storage unit in the set of storage units respectively.
4. The method according to claim 1, further comprising: For each storage unit in the set of storage units, In response to the first number of copies being written to the data segment of the storage unit, determining whether there is unoccupied space in the data segment of the storage unit; And If it is determined that there is unoccupied space in the data segment of the storage unit, filling the space with invalid data.
5. The method according to claim 1, wherein generating the first check information includes: Generating a first erasure code based on the data stored in the data segment of the one storage unit as the first check information.
6. The method according to claim 2, wherein writing the second number of copies to the data segment of the at least one storage unit includes: Generating the second number of data blocks based on the second number of copies, wherein each data block includes a copy and an identifier associated with the request; And Writing the second number of data blocks to the data segment of the at least one storage unit.
7. The method according to claim 2, further comprising: In response to the second number of copies being written to the data segment of the at least one storage unit, determine whether there is unoccupied space in the data segment of the at least one storage unit; and If it is determined that there is unoccupied space in the data segment of the at least one storage unit, fill the space with invalid data.
8. The method according to claim 2, wherein generating the second check information includes: Generating a second erasure code as the second check information based on the data stored in the data segment of the at least one storage unit.
9. An electronic device, comprising: At least one processing unit; At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit causing the electronic device to perform actions, the actions including: Receiving a request to store target data, wherein the request includes the target data and indicates the number of copies to be stored for the target data; Based on the target data and the number of copies, allocate a plurality of storage units for storing copies of the target data, wherein each storage unit includes a data segment configured to store data and an encoding segment configured to store check information, and a set of storage units among the plurality of storage units are respectively configured to store the first number of copies of the target data; Writing the first number of copies to the data segments of each storage unit in the set of storage units respectively; For one storage unit in the set of storage units, generating first check information based on the data stored in the data segment of the one storage unit; and Writing the first check information to the encoding segments of each storage unit in the set of storage units respectively.
10. The device according to claim 9, wherein at least one storage unit among the plurality of storage units is configured to store the second number of copies of the target data, and the actions further include: Writing the second number of copies to the data segment of the at least one storage unit; Generating second check information based on the data stored in the data segment of the at least one storage unit; and Writing the second check information to the encoding segment of the at least one storage unit.
11. The device according to claim 9, wherein writing the first number of copies to the data segments of each storage unit in the set of storage units respectively includes: Generating the first number of data blocks based on the first number of copies, wherein each data block includes one copy and an identifier associated with the request; and Writing the first number of data blocks to the data segments of each storage unit in the set of storage units respectively.
12. The device according to claim 9, wherein the actions further include: For each storage unit in the set of storage units, In response to the first number of copies being written to the data segment of the storage unit, determine whether there is unoccupied space in the data segment of the storage unit; and If it is determined that there is unoccupied space in the data segment of the storage unit, fill the space with invalid data.
13. The apparatus according to claim 9, wherein generating the first check information comprises: Generating a first erasure code based on the data stored in the data segment of the one storage unit as the first check information.
14. The apparatus according to claim 10, wherein writing the second number of copies to the data segment of the at least one storage unit comprises: Generating the second number of data blocks based on the second number of copies, wherein each data block comprises a copy and an identifier associated with the request; and Writing the second number of data blocks to the data segment of the at least one storage unit.
15. The apparatus according to claim 10, wherein the action further comprises: Determining whether there is unoccupied space in the data segment of the at least one storage unit in response to the second number of copies being written to the data segment of the at least one storage unit; and If it is determined that there is unoccupied space in the data segment of the at least one storage unit, fill the space with invalid data.
16. The apparatus according to claim 10, wherein generating the second check information comprises: Generating a second erasure code based on the data stored in the data segment of the at least one storage unit as the second check information.
17. A computer program product, the computer program product being tangibly stored in a non-transitory computer storage medium and comprising machine-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Data storage method and device, readable medium and data storage system
CN111273862A
Method and apparatus for storing data and computer program product for storing data
CN111488239A