Data storage method and device, storage medium and electronic equipment
By dividing data into data blocks and calculating verification blocks in a distributed storage system, the problem of low efficiency of traditional redundant storage is solved, and efficient data storage and rapid recovery are achieved.
Patent Information
- Application Number
- CN202411214881.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-13
AI Technical Summary
Existing distributed storage systems have shortcomings in improving data storage efficiency. Traditional data redundancy methods require a large amount of storage space, resulting in low data storage efficiency.
By dividing the target data into data blocks with preset data capacity, and using data redundancy algorithm to calculate verification blocks smaller than the number of data blocks, saving the data blocks and verification blocks to the server, and using verification blocks to lose data blocks in the data blocks Recovery when it is done.
Reduces storage requirements, improves data storage efficiency, reduces the storage pressure of the server, and can quickly recover data when data blocks are lost.
Smart Images
Figure CN119127077A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data storage, and more specifically, to a data storage method, device, storage medium, and electronic device. Background Art
[0002] With the development of storage technology, distributed storage technology has been widely used. For distributed storage systems, there is always the risk of data loss due to component failure. In order to reduce the risk of data loss, distributed storage systems use data redundancy to store data. The traditional data redundancy method usually copies the data into multiple copies for backup and preservation, and restores the data through backup files when it is lost or damaged. However, the above method requires a large amount of storage space, resulting in low data storage efficiency.
[0003] Therefore, in the related art, there is a problem of how to improve data storage efficiency.
[0004] Regarding the problem of how to improve data storage efficiency in related technologies, no effective solution has been proposed yet. Summary of the invention
[0005] The embodiments of the present application provide a data storage method, device, storage medium and electronic device to at least solve the problem of how to improve data storage efficiency in the related art.
[0006] According to an embodiment of the present application, a data storage method is provided, which is applied to a storage system, wherein the storage system includes a client and a server, wherein the client is used to initiate a data storage request, and the server is used to store data, including: upon determining that the client has received a request to store target data, dividing the target data into a first number of data blocks, wherein the data block size is a preset data capacity; calculating the first number of data blocks through a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the check block size is the preset data capacity, and the check block is used to restore the data block; and saving the first number of data blocks and the second number of check blocks to the server.
[0007] In an exemplary embodiment, the target data is divided into a first number of data blocks, including: obtaining the starting storage address of the target data and the data capacity of the target data; determining the first number according to the data capacity of the target data and the preset data capacity; determining the starting storage address of the first data block and the ending storage address of the first data block in the first number of data blocks according to the starting storage address and the preset data capacity, wherein the data size between the starting storage address of the first data block and the ending storage address of the first data block is the preset data capacity; determining the ending storage address of the first data block as the starting storage address of the next data block, determining the ending storage address of the next data block according to the starting storage address of the next data block and the preset data capacity, and obtaining the starting storage address of each data block in the first number of data blocks and the ending storage address of each data block; reading the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block; and storing the data of each data block in the memory space in the client to obtain the first number of data blocks.
[0008] In an exemplary embodiment, the first number of data blocks are calculated using a data redundancy algorithm to obtain a second number of check blocks, including: grouping the first number of data blocks according to a preset number to obtain multiple groups of data blocks; calculating each group of data blocks in the multiple groups of data blocks using the data redundancy algorithm to obtain the second number of check blocks, wherein one group of data blocks corresponds to one check block; and storing the second number of check blocks in a memory space in the client.
[0009] In an exemplary embodiment, saving the first number of data blocks and the second number of check blocks to the server includes: sending the first number of data blocks and the second number of check blocks from the memory space of the client to the server; and deleting the first number of data blocks and the second number of check blocks from the memory space.
[0010] In an exemplary embodiment, after reading the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block, the method further includes: calculating the data of each data block through the data redundancy algorithm to obtain the second number of check blocks, and storing the second number of check blocks in the memory space in the client.
[0011] In an exemplary embodiment, saving the first number of data blocks and the second number of check blocks to a server includes: sending the starting storage address of each data block and the ending storage address of each data block to the server so that the server directly reads the data of each data block to obtain the first number of data blocks; sending the second number of check blocks from the memory space of the client to the server; and deleting the second number of check blocks from the memory space.
[0012] In an exemplary embodiment, after saving the first number of data blocks and the second number of check blocks to the server, the method further includes: after determining that an instruction to read the target data has been received, reserving data capacity for the target data in the memory space of the client to obtain a reserved space; sending the starting storage address of the reserved space and the ending storage address of the reserved space to the server so that the server determines the storage address of each data block in the first number of data blocks in the reserved space; and receiving the first number of data blocks sent by the server to the reserved space to obtain the target data.
[0013] According to another embodiment of the present application, a data storage device is provided, including: a division module, for dividing the target data into a first number of data blocks when it is determined that the client has received a request to store the target data, wherein the data block size is a preset data capacity; a calculation module, for calculating the first number of data blocks through a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the check block size is the preset data capacity, and the check block is used to restore the data block; and a saving module, for saving the first number of data blocks and the second number of check blocks to the server.
[0014] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0015] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0016] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiment are implemented.
[0017] Through the present application, after the client receives a request to store the target data, the target data can be divided into a first number of data blocks, the first number of data blocks can be calculated by a data redundancy algorithm to obtain a second number of check blocks, the first number of data blocks and the second number of check blocks can be saved to the server, and the data blocks can be restored through the check blocks when the data blocks are lost. Therefore, the problem of how to improve data storage efficiency in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a hardware structure block diagram of a server device of a data storage method according to an embodiment of the present application;
[0019] Figure 2 is a flow chart of a data storage method according to an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of a data storage method according to an embodiment of the present application (I);
[0021] Figure 4 is a schematic diagram of a data storage method according to an embodiment of the present application (II);
[0022] Figure 5 is a schematic diagram of a data storage method according to an embodiment of the present application (III);
[0023] Figure 6 is a schematic diagram of a data storage method according to an embodiment of the present application (IV);
[0024] Figure 7 It is a structural block diagram of a data storage device according to an embodiment of the present application. DETAILED DESCRIPTION
[0025] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0027] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a data storage method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for target data, wherein the above-mentioned server device may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer programs corresponding to the data storage method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned data storage method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0029] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0030] In this embodiment, a data storage method is provided. Figure 2 is a flow chart of a data storage method according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:
[0031] Step S202, when it is determined that the client receives a request to store target data, divide the target data into a first number of data blocks, wherein the size of the data blocks is a preset data capacity;
[0032] Step S204, calculating the first number of data blocks by a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks;
[0033] Optionally, in the above embodiment, the data redundancy algorithm includes an ISA (Inter Storage Acceleration) erasure algorithm, which is implemented by an EC (Erasure Code) library provided by Inter, and can utilize Inter processor instructions to accelerate calculations.
[0034] Step S206: Save the first number of data blocks and the second number of check blocks to the server.
[0035] Through the above steps, after the client receives a request to store the target data, the target data can be divided into a first number of data blocks, the first number of data blocks can be calculated by a data redundancy algorithm to obtain a second number of check blocks, the first number of data blocks and the second number of check blocks can be saved to the server, and the data blocks can be restored through the check blocks when the data blocks are lost. Therefore, the problem of how to improve data storage efficiency in the related art can be solved.
[0036] In an exemplary embodiment, the target data is divided into a first number of data blocks, including: obtaining the starting storage address of the target data and the data capacity of the target data; determining the first number according to the data capacity of the target data and the preset data capacity; determining the starting storage address of the first data block and the ending storage address of the first data block in the first number of data blocks according to the starting storage address and the preset data capacity, wherein the data size between the starting storage address of the first data block and the ending storage address of the first data block is the preset data capacity; determining the ending storage address of the first data block as the starting storage address of the next data block, determining the ending storage address of the next data block according to the starting storage address of the next data block and the preset data capacity, and obtaining the starting storage address of each data block in the first number of data blocks and the ending storage address of each data block; reading the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block; and storing the data of each data block in the memory space in the client to obtain the first number of data blocks.
[0037] In an optional embodiment, if Figure 3As shown, for example, if the preset data capacity of the data block is 4Kb and the data capacity of the target data is 32kb, the target data can be divided into 8 (equivalent to the first number) data blocks, that is, Figure 3 K1.1, K1.2, K1.3, K1.4, K2.1, K2.2, K2.3, K2.4.
[0038] In an exemplary embodiment, the first number of data blocks are calculated using a data redundancy algorithm to obtain a second number of check blocks, including: grouping the first number of data blocks according to a preset number to obtain multiple groups of data blocks; calculating each group of data blocks in the multiple groups of data blocks using the data redundancy algorithm to obtain the second number of check blocks, wherein one group of data blocks corresponds to one check block; and storing the second number of check blocks in a memory space in the client.
[0039] Optionally, in the above embodiment, the data redundancy algorithm takes the ISA erasure algorithm as an example. Assuming the erasure ratio is 2:1, the data blocks are grouped according to a preset number of 2, with 2 data blocks forming a group. The ISA erasure algorithm is used to calculate each group of data blocks to obtain a check block.
[0040] Optionally, in the above embodiment, each group of data blocks may also correspond to multiple check blocks. The data redundancy algorithm takes the ISA erasure algorithm as an example. Assuming that the erasure ratio is 4:2, the data blocks are grouped according to a preset number of 4 pairs, with 4 data blocks forming a group. The ISA erasure algorithm is used to calculate each group of data blocks to obtain 2 check blocks.
[0041] In an optional embodiment, if Figure 3 As shown, taking the erasure ratio of 4:2 as an example, data blocks K1.1, K1.2, K1.3, K1.4, K2.1, K2.2, K2.3, K2.4 are grouped, every 4 data blocks are the first group, K1.1, K1.2, K1.3, K1.4 are a group, K2.1, K2.2, K2.3, K2.4 are the second group, and the first group of data blocks is calculated by the ISA erasure algorithm to obtain 2 check blocks, that is, Figure 3 M1.5, M1.6 in the second group of data blocks are calculated by the ISA erasure algorithm to obtain two check blocks, namely Figure 3 M2.5, M2.6.
[0042] Through the above embodiments, it is not necessary to copy all data contents for backup when storing data, which greatly reduces the storage pressure of the server and improves the data storage efficiency.
[0043] In an exemplary embodiment, saving the first number of data blocks and the second number of check blocks to the server includes: sending the first number of data blocks and the second number of check blocks from the memory space of the client to the server; and deleting the first number of data blocks and the second number of check blocks from the memory space.
[0044] In an optional embodiment, if Figure 3 As shown in the figure, after the erasure calculation is completed, the data blocks are saved to the OSS (Object Storage Service) server. In order to reduce the risk of data loss, each group of data blocks and the corresponding check blocks are stored separately in OSS.1, OSS.2, OSS.3, OSS.4, OSS.5, and OSS.6. When any data block or check block is lost, the data can be recovered through the remaining data blocks and check blocks.
[0045] In an optional embodiment, if Figure 4 As shown, when the client needs to read the target data, it needs to first read the data from OSS.1, OSS.2, OSS.3, and OSS.4 and then send it to the client's memory through the network to obtain data blocks K1.1, K1.2, K1.3, K1.4, K2.1, K2.2, K2.3, and K2.4. Then, the data blocks are restored through erasure calculation to obtain the target data.
[0046] In an exemplary embodiment, after reading the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block, the method further includes: calculating the data of each data block through the data redundancy algorithm to obtain the second number of check blocks, and storing the second number of check blocks in the memory space in the client.
[0047] In an exemplary embodiment, saving the first number of data blocks and the second number of check blocks to a server includes: sending the starting storage address of each data block and the ending storage address of each data block to the server so that the server directly reads the data of each data block to obtain the first number of data blocks; sending the second number of check blocks from the memory space of the client to the server; and deleting the second number of check blocks from the memory space.
[0048] In an optional embodiment, if Figure 5As shown, there is no need to divide the target data into data blocks and copy them to the client memory. Instead, the starting storage address of each data block and the ending storage address of each data block are directly transmitted to the OSS server through the network. OSS.1, OSS.2, OSS.3, and OSS.4 use the RDMA (Remote Direct Memory Access) technology to directly pull the data to be stored from the client target data address through the starting storage address of each data block and the ending storage address of each data block.
[0049] Verification blocks M1.5 and M1.6 need to be calculated based on the first group of data blocks K1.1, K1.2, K1.3, and K1.4, and verification blocks M2.5 and M2.6 need to be calculated based on the second group of data blocks K2.1, K2.2, K2.3, and K2.4. In this embodiment, independent data blocks are not copied in the memory, so the data content of each data block can be obtained according to the starting storage address of each data block and the ending storage address of each data block during calculation. The verification blocks obtained through the above calculation process are stored in the client memory, and then the verification blocks M1.5 and M1.6 are sent to OSS.5 for storage, and the verification blocks M2.5 and M2.6 are sent to OSS.6 for storage. The starting storage address and the ending storage address of each verification block can also be sent to the OSS server, and OSS.5 and OSS.6 directly pull the verification blocks to be stored from the client memory.
[0050] Through the above embodiment, the process of copying and sending in memory during data storage can be avoided. Instead, the server directly pulls the data to be stored from the memory according to the storage address, which simplifies the data storage process and speeds up the data storage speed.
[0051] In an exemplary embodiment, after saving the first number of data blocks and the second number of check blocks to the server, the method further includes: after determining that an instruction to read the target data has been received, reserving data capacity for the target data in the memory space of the client to obtain a reserved space; sending the starting storage address of the reserved space and the ending storage address of the reserved space to the server so that the server determines the storage address of each data block in the first number of data blocks in the reserved space; and receiving the first number of data blocks sent by the server to the reserved space to obtain the target data.
[0052] In an optional embodiment, if Figure 6As shown, when the client needs to read the target data, it first reserves space in the client memory according to the data capacity of the target data to be read, and then sends the starting storage address and the ending storage address of the reserved space to the server. The server will calculate the space required for each data block and the corresponding storage address respectively. After the calculation is completed, the RDMA technology is used to directly put each data block in the corresponding position of the reserved space to obtain the target data. After the placement is completed, the client can read the target data from the reserved space.
[0053] Through the above embodiment, when reading stored data, it is no longer necessary to read each data block separately and then merge them. Instead, the read data block is directly placed at a specified position in the reserved space to obtain the data to be read, thereby speeding up the data reading speed.
[0054] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0055] In the present embodiment, a data storage device is also provided, and the system is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions that have been made are not repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0056] Figure 7 is a structural block diagram of a data storage device according to an embodiment of the present application, such as Figure 7 As shown, the device comprises:
[0057] A division module 72, configured to divide the target data into a first number of data blocks when it is determined that the client receives a request to store the target data, wherein the size of the data block is a preset data capacity;
[0058] A calculation module 74, configured to calculate the first number of data blocks by using a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks;
[0059] The saving module 76 is used to save the first number of data blocks and the second number of check blocks to the server.
[0060] Through the above device, after the client receives a request to store the target data, the target data can be divided into a first number of data blocks, the first number of data blocks can be calculated by a data redundancy algorithm to obtain a second number of check blocks, the first number of data blocks and the second number of check blocks can be saved to the server, and the data blocks can be restored through the check blocks when the data blocks are lost. Therefore, the problem of how to improve data storage efficiency in the related art can be solved.
[0061] In an exemplary embodiment, the partitioning module 72 is also used to: obtain the starting storage address of the target data and the data capacity of the target data; determine the first quantity according to the data capacity of the target data and the preset data capacity; determine the starting storage address of the first data block and the ending storage address of the first data block in the first quantity of data blocks according to the starting storage address and the preset data capacity, wherein the data size between the starting storage address of the first data block and the ending storage address of the first data block is the preset data capacity; determine the ending storage address of the first data block as the starting storage address of the next data block, determine the ending storage address of the next data block according to the starting storage address of the next data block and the preset data capacity, and obtain the starting storage address of each data block in the first quantity of data blocks and the ending storage address of each data block; read the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block; save the data of each data block in the memory space in the client to obtain the first quantity of data blocks.
[0062] In an exemplary embodiment, the calculation module 74 is further used to: group the first number of data blocks according to a preset number to obtain multiple groups of data blocks; calculate each group of data blocks in the multiple groups of data blocks through the data redundancy algorithm to obtain the second number of check blocks, wherein one group of data blocks corresponds to one check block; and store the second number of check blocks in the memory space in the client.
[0063] In an exemplary embodiment, the saving module 76 is further used to: send the first number of data blocks and the second number of check blocks from the client's memory space to the server; and delete the first number of data blocks and the second number of check blocks from the memory space.
[0064] In an exemplary embodiment, the calculation module 74 is further used to: calculate the data of each data block through the data redundancy algorithm to obtain the second number of check blocks, and store the second number of check blocks in the memory space of the client.
[0065] In an exemplary embodiment, the saving module 76 is also used to: send the starting storage address of each data block and the ending storage address of each data block to the server, so that the server directly reads the data of each data block to obtain the first number of data blocks; send the second number of check blocks from the memory space of the client to the server; and delete the second number of check blocks from the memory space.
[0066] In an exemplary embodiment, the saving module 76 is also used to: after determining that an instruction to read the target data has been received, reserve data capacity for the target data in the memory space of the client to obtain a reserved space; send the starting storage address of the reserved space and the ending storage address of the reserved space to the server, so that the server determines the storage address of each data block of the first number of data blocks in the reserved space; receive the first number of data blocks sent by the server to the reserved space to obtain the target data.
[0067] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0068] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0069] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:
[0070] S1, when it is determined that the client receives a request to store target data, dividing the target data into a first number of data blocks, wherein the size of the data blocks is a preset data capacity;
[0071] S2, calculating the first number of data blocks by a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks;
[0072] S3: Save the first number of data blocks and the second number of check blocks to the server.
[0073] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0074] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0075] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0076] S1, when it is determined that the client receives a request to store target data, dividing the target data into a first number of data blocks, wherein the size of the data blocks is a preset data capacity;
[0077] S2, calculating the first number of data blocks by a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks;
[0078] S3: Save the first number of data blocks and the second number of check blocks to the server.
[0079] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0080] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:
[0081] S1, when it is determined that the client receives a request to store target data, dividing the target data into a first number of data blocks, wherein the size of the data blocks is a preset data capacity;
[0082] S2, calculating the first number of data blocks by a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks;
[0083] S3: Save the first number of data blocks and the second number of check blocks to the server.
[0084] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0085] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0086] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A data storage method, It is characterized in that Applied to a storage system, the storage system includes a client and a server, the client is used to initiate a data storage request, and the server is used to store data, including: In the case of determining that the client receives a request to store target data, dividing the target data into a first number of data blocks, wherein the size of the data blocks is a preset data capacity; Calculate the first number of data blocks by a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks; The first number of data blocks and the second number of check blocks are saved to the server.
2. The method according to claim 1, It is characterized in that Dividing the target data into a first number of data blocks includes: Acquire the starting storage address of the target data and the data capacity of the target data; Determining the first quantity according to the data capacity of the target data and the preset data capacity; Determining a starting storage address of a first data block and an ending storage address of the first data block in the first number of data blocks according to the starting storage address and the preset data capacity, wherein the data size between the starting storage address of the first data block and the ending storage address of the first data block is the preset data capacity; Determine the end storage address of the first data block as the start storage address of the next data block, determine the end storage address of the next data block according to the start storage address of the next data block and the preset data capacity, and obtain the start storage address of each data block in the first number of data blocks and the end storage address of each data block; Reading data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block; The data of each data block is stored in the memory space of the client to obtain the first number of data blocks.
3. The method according to claim 2, It is characterized in that Calculating the first number of data blocks by using a data redundancy algorithm to obtain a second number of check blocks includes: Grouping the first number of data blocks according to a preset number to obtain multiple groups of data blocks; Calculating each group of data blocks in the multiple groups of data blocks by using the data redundancy algorithm to obtain the second number of check blocks, wherein one group of data blocks corresponds to one check block; The second number of check blocks is stored in a memory space in the client.
4. The method according to claim 3, It is characterized in that Saving the first number of data blocks and the second number of check blocks to the server includes: Sending the first number of data blocks and the second number of check blocks from the memory space of the client to the server; The first number of data blocks and the second number of check blocks are deleted from the memory space.
5. The method according to claim 2, It is characterized in that After reading the data of each data block from the target data according to the starting storage address of each data block and the ending storage address of each data block, the method further includes: Calculating the data of each data block by using the data redundancy algorithm to obtain the second number of check blocks, The second number of check blocks is stored in a memory space in the client.
6. The method according to claim 5, It is characterized in that Saving the first number of data blocks and the second number of check blocks to the server includes: Sending the starting storage address of each data block and the ending storage address of each data block to the server, so that the server directly reads the data of each data block to obtain the first number of data blocks; Sending the second number of check blocks from the memory space of the client to the server; The second number of check blocks is deleted from the memory space.
7. The method according to claim 1, It is characterized in that After the first number of data blocks and the second number of check blocks are saved to the server, the method further includes: After determining that an instruction to read the target data is received, reserving data capacity of the target data in the memory space of the client to obtain a reserved space; Sending a starting storage address of the reserved space and an ending storage address of the reserved space to the server, so that the server determines a storage address of each data block in the first number of data blocks in the reserved space; The first number of data blocks sent by the server to the reserved space are received to obtain the target data.
8. A data storage device, It is characterized in that include: a partitioning module, configured to partition the target data into a first number of data blocks when it is determined that the client receives a request to store the target data, wherein the size of the data blocks is a preset data capacity; a calculation module, configured to calculate the first number of data blocks by using a data redundancy algorithm to obtain a second number of check blocks, wherein the second number is smaller than the first number, the size of the check blocks is the preset data capacity, and the check blocks are used to restore the data blocks; A saving module is used to save the first number of data blocks and the second number of check blocks to the server.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.