Node setting method, data storage method, electronic device, and storage medium
By setting up sparse files on storage nodes, a one-to-one correspondence is established between virtual storage addresses and physical storage addresses, thus solving the problem of data loss due to time differences during data backup and achieving highly reliable data storage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUBEI TIANRONGXIN NETWORK SECURITY TECH CO LTD
- Filing Date
- 2022-12-14
- Publication Date
- 2026-07-24
AI Technical Summary
In existing data backup methods, there is a time difference in data synchronization between the primary node and the backup node, which can easily lead to data loss if the primary node fails during this time difference.
By setting up corresponding sparse files on each storage node, the virtual storage addresses in the sparse files correspond one-to-one with the physical storage addresses. Data is first stored in the virtual storage address of the sparse file and then written to the corresponding physical storage address, thus avoiding the time difference between the primary node and the backup node.
Even if one storage node fails, other storage nodes can still store data, avoiding data loss and improving the reliability of data storage.
Smart Images
Figure CN115826867B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and more specifically, to node setup methods, data storage methods, electronic devices, and storage media. Background Technology
[0002] With the continuous development of internet-related technologies, the data generated by users' online activities has become extremely valuable.
[0003] The data has high mining value, so it is usually necessary to store it, and in order to prevent the loss of stored data due to reasons such as equipment failure, the data is often backed up.
[0004] Existing technologies typically back up data by setting up primary and backup nodes. This method, after acquiring data, first stores it on the primary node, and then synchronizes the data from the primary node to the backup node according to a preset time period. However, this method suffers from limitations due to the presence of primary and backup nodes.
[0005] There is a certain time difference (i.e., this time period) in the data synchronization between them. If the master node fails within this time difference, the data within this time difference will be lost. Summary of the Invention
[0006] The purpose of this application is to provide a node setting method, a data storage method, an electronic device, and a storage medium to solve the problem that existing data backup methods are prone to data loss.
[0007] The first aspect of this application provides a data storage method, wherein each storage node is configured with a corresponding sparse file, and each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node, the method comprising:
[0008] Obtain a data storage instruction, wherein the data storage instruction includes a data storage address and the data to be stored;
[0009] 5. According to the data storage instruction, write the data to be stored into each sparse file, where the data storage address indicates the virtual storage address;
[0010] In each sparse file, the data to be stored written in the virtual storage address indicated by the data storage address is written to the corresponding physical storage address in the corresponding storage node.
[0011] In one embodiment, the method further includes:
[0012] For each storage node, a sparse file corresponding to the storage node is created, wherein the storage amount of the virtual storage space of the sparse file is the same as the storage amount of the physical storage space of the storage node;
[0013] The physical storage space of each storage node is divided into multiple physical storage subspaces, and a corresponding physical storage address is set for each physical storage subspace.
[0014] Based on the partitioning method of each physical storage subspace, the virtual storage space of each sparse file is divided into virtual storage subspaces that correspond one-to-one with the physical storage subspaces, and a corresponding virtual storage address is set for each virtual storage subspace.
[0015] In one embodiment, the method further includes:
[0016] Obtain the data storage request input by the user, wherein the data storage request carries the data to be stored;
[0017] The data storage address is determined based on the amount of data to be stored and the storage capacity of the virtual storage subspace;
[0018] The data storage instruction is generated based on the data storage address and the data to be stored.
[0019] In one embodiment, the physical storage space of each storage node is divided into multiple physical storage subspaces, specifically including:
[0020] The physical storage space of each storage node is divided into multiple physical storage subspaces with a storage capacity of 4KB each.
[0021] In one embodiment, the method further includes:
[0022] Retrieve the command to add a new storage node;
[0023] For each newly added storage node and each existing storage node, a corresponding new sparse file is created. Each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node.
[0024] In one embodiment, the method further includes:
[0025] The data in the sparse file is synchronized to the new sparse file corresponding to the newly added storage node.
[0026] In one embodiment, after writing the data to be stored into each sparse file according to the data storage instruction, and the virtual storage address indicated by the data storage address, the method further includes: using an index file to record the data to be stored written to the virtual storage address of each sparse file.
[0027] The second aspect of this application provides a node configuration method, wherein each storage node is configured with a corresponding sparse file, and each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node, including:
[0028] Retrieve the command to add a new storage node;
[0029] For each newly added storage node and each existing storage node, a corresponding new sparse file is created. Each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. This ensures that when a second data storage instruction is received, the data to be stored in the second data storage instruction can be written to the virtual storage address indicated by the data storage address of the second data storage instruction in each new sparse file, and then written to the physical storage address in the corresponding storage node that corresponds one-to-one with the virtual storage address.
[0030] A third aspect of this application provides an electronic device, including:
[0031] processor;
[0032] A memory for storing processor-executable instructions; wherein the processor is configured to perform the method described in any one of the first aspects of the embodiments of this application.
[0033] A fourth aspect of this application provides a storage medium storing a computer program that can be executed by a processor to perform the method described in any one of the first aspects of this application.
[0034] The data storage method of this application involves setting corresponding sparse files for each storage node and aligning the virtual storage addresses in the sparse files with the corresponding storage nodes.
[0035] Each physical storage address corresponds one-to-one. During data storage, the data to be stored in the data storage instruction is first written to the virtual storage address in each sparse file. Then, the data to be stored written to the virtual storage address in each sparse file is written to the corresponding physical storage address in the corresponding storage node. Therefore, compared to the existing backup method using primary and backup nodes, since there is no primary and backup node...
[0036] The time difference between the two storage nodes means that even if one storage node fails, the other storage nodes can still store the data, so the data is not easily lost. Attached Figure Description
[0037] To more clearly illustrate the technical solutions of the embodiments of this application, the embodiments of this application will be described below.
[0038] The accompanying drawings used in this application are briefly described below. It should be understood that the following drawings only show some embodiments of this application 5 and should not be regarded as a limitation of the scope. For those skilled in the art,
[0039] For staff members, without expending any creative effort, they can obtain other related figures based on these figures.
[0040] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0041] Figure 2 A schematic diagram illustrating the interaction between an electronic device and a terminal, provided as an embodiment of this application; Figure 3 A schematic diagram illustrating the specific process of a node setting method provided in one embodiment of this application;
[0042] Figure 4 A schematic diagram of the data storage method provided in one embodiment of this application;
[0043] Figure 5 A schematic diagram illustrating the specific process of a node setting method provided in another embodiment of this application;
[0044] Figure 6 A schematic diagram illustrating a specific example of a node setting method provided for another embodiment of this application;
[0045] Figure 7 A schematic diagram illustrating the specific flow of a data storage method provided for another embodiment of this application; Figure 8 A schematic diagram of the specific structure of a data storage device provided in an embodiment of this application;
[0046] Figure 9This is a schematic diagram of the specific structure of a node setting device provided in an embodiment of this application. Detailed Implementation
[0047] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of this application, terms such as "first," "second," and "third" are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance or order.
[0048] As mentioned earlier, existing technologies typically back up data by setting up a primary node and a backup node. However, this method involves a time difference in data synchronization between the primary and backup nodes. After the primary node stores the data, it synchronizes the data to the backup node according to a preset time period (e.g., 1 hour, 1 day). If the primary node fails during this time difference, the data within that time difference will be lost. Therefore, this existing data backup method is prone to data loss.
[0049] In view of this, embodiments of this application provide a node setting method, a data storage method, an electronic device, and a storage medium, all of which can be used to solve the problem that existing data backup methods are prone to data loss. Figure 1 As shown, this embodiment provides an electronic device 1, which includes at least one processor 11 and a memory 12. Figure 1 Taking a processor as an example, the processor 11 and the memory 12 can be connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. The instructions are executed by the processor 11 to enable the electronic device 1 to perform all or part of the processes of the various methods in the following embodiments.
[0050] In practical applications, the electronic device 1 can be a mobile phone, laptop computer, desktop computer, etc. In conjunction with the application scenario of data backup in this application, the electronic device 1 can also be a server or a storage node in a server cluster for storing data.
[0051] like Figure 2 As shown, electronic device 1 can also interface with one or more terminals 2, and electronic device 1 can communicate with each terminal 2 separately. Terminal 2 can generate data; for example, terminal 2 can be a user's mobile phone, computer, or other type of terminal, so that the user can perform network activities on terminal 2, thereby generating data. Therefore, in one embodiment, after the user generates data through network activities on terminal 2, the data is sent to electronic device 1, and then stored by electronic device 1.
[0052] To facilitate understanding, the inventive concept of the embodiments of this application will be briefly explained first. The embodiments of this application provide multiple storage nodes, and each storage node is respectively configured with corresponding rare...
[0053] The sparse file is a type of file where each storage node has five virtual storage addresses that correspond one-to-one with each physical storage address in the corresponding storage node. In other words, for a virtual storage address in the sparse file, there is a corresponding physical storage address in the storage node corresponding to that virtual storage address. Similarly, for a physical storage address in a storage node, there is a corresponding virtual storage address in the sparse file corresponding to that storage node.
[0054] In sparse files, unlike regular files, each virtual memory address is considered empty and does not occupy storage space before data is written to it. Therefore, in practical applications, even if a sparse file has a virtual memory size of 10GB, if no data is written to that 10GB and the memory addresses are empty, the actual storage space occupied will be empty.
[0055] The storage space used is 0GB. Of course, if data is written to one or more virtual storage addresses in the sparse file, these virtual storage addresses that have been written with data will occupy the corresponding storage space, but other virtual storage addresses that have not been written with data will still not occupy storage space.
[0056] It should be noted that the individual storage nodes and their corresponding sparse files can be generated using the node setting method provided in this application's embodiments, such as... Figure 3 As shown, the method includes the following steps:
[0057] Step S31: For each storage node, create a sparse file corresponding to that storage node.
[0058] In this process, for each sparse file corresponding to a storage node, each virtual storage address in the sparse file needs to correspond one-to-one with each physical storage address in the corresponding storage node. Therefore, in step S31, the storage capacity of the virtual storage space of the created sparse file is related to the...
[0059] The physical storage space of the corresponding storage nodes is the same. For example, for a certain storage node, if its physical storage space is 1GB, then the virtual storage space of the sparse file corresponding to that storage node is also 1GB. Since the storage space of the two is the same, it provides the basis for the one-to-one correspondence between the virtual storage address and the physical storage address.
[0060] Step S32: Divide the physical storage space of each storage node into multiple physical storage subspaces, and set the corresponding physical storage address for each physical storage subspace.
[0061] In step S32, the physical storage space of each storage node can be divided into multiple physical storage subspaces, and then a corresponding physical storage address can be set for each physical storage subspace. The storage size of each physical storage subspace can be the same, for example, the storage size of each physical storage subspace can be 3KB, 4KB, 5KB or other values.
[0062] In particular, considering that the data processing format of the file system is usually 4KB, in step S32, the physical storage space of each storage node can be divided into multiple physical storage subspaces with a storage capacity of 4KB. That is to say, in step S32, the storage size of each physical storage subspace is 4KB, thus matching the data processing format of the file system, thereby improving data writing efficiency and reducing the waste of storage resources.
[0063] Of course, after dividing the physical storage space of each storage node into multiple physical storage subspaces, a corresponding physical storage address can be set for each physical storage subspace. For example, a corresponding number can be set for each physical storage subspace as the physical storage address.
[0064] Step S33: Based on the partitioning method of each physical storage subspace, divide the virtual storage space of each sparse file into virtual storage subspaces that correspond one-to-one with the physical storage subspace, and set the corresponding virtual storage address for each virtual storage subspace.
[0065] In step S33, based on the partitioning method of each physical storage subspace, for example, if the storage size of each physical storage subspace obtained in step S32 is 4KB, then in step S33, the storage size of the virtual storage space of each sparse file can also be partitioned into 4KB, thus corresponding one-to-one with the physical storage subspace. Furthermore, after partitioning into multiple virtual storage subspaces, a corresponding virtual storage address is set for each virtual storage subspace. In this way, each virtual storage address in the sparse file ultimately corresponds one-to-one with each physical storage address in the corresponding storage node.
[0066] Therefore, through steps S31 to S33 in this method, multiple storage nodes can be created, and each storage node is set with a corresponding sparse file. Moreover, for each storage node, each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node, thus providing a basis for the implementation of the data storage method provided in the embodiments of this application.
[0067] Based on the aforementioned storage nodes and the corresponding sparse files, embodiments of this application provide a data storage method. For example... Figure 4 The diagram shows the specific flow of this method, which includes the following steps:
[0068] Step S41: Obtain data storage instruction, which includes data storage address and data to be stored.
[0069] Step S42: According to the data storage instruction, write the data to be stored to each sparse file to the virtual storage address indicated by the data storage address.
[0070] Step S43: Write the data to be stored in the virtual storage address indicated by the data storage address in each sparse file to the physical storage address that corresponds one-to-one with the virtual storage address in the corresponding storage node.
[0071] Here we can provide a unified explanation of steps S41 to S43 above.
[0072] In practical applications, file systems for electronic devices typically use the iSCSI (Internet SCSI) mechanism for data writing. This process involves obtaining a data storage request input by the user, which carries the data to be stored. Then, a corresponding SCSI instruction is generated based on the data storage request. This SCSI instruction includes the physical storage address of the physical storage space in the storage node, and the data to be stored is then directly written to that physical storage address. Because the existing iSCSI mechanism in file systems does not provide a multi-copy mechanism, it is difficult to write to storage nodes simultaneously.
[0073] In the data storage method provided in this application embodiment, step S41 can specifically involve obtaining the data storage request input by the user, which carries the data to be stored, then determining the data storage address based on the amount of data to be stored and the storage capacity of the virtual storage subspace, and then generating a data storage instruction based on the data storage address and the data to be stored.
[0074] For example, a user can input the data storage request through the file system's interface, enabling the electronic device's file system to obtain the user's input data storage request. Then, the file system can determine the data storage address based on the amount of data to be stored and the storage capacity of the virtual storage subspace.
[0075] For example, taking a scenario where each physical storage subspace and each virtual storage subspace has a storage size of 4KB as an example, we can record whether data has been written to each virtual storage subspace. For instance, we can create an index file and use it to record whether data has been written to each virtual storage address of the sparse file. Thus, after receiving a data storage request, if the amount of data to be stored in the request is less than or equal to 4KB, we can query the index file to obtain any virtual storage subspace that has not yet been written to (this can be any virtual storage subspace that has not yet been written to). Then, we determine the virtual storage address of this virtual storage subspace as the data storage address. Conversely, if the amount of data to be stored is greater than 4KB, we can query the index file to first determine the first virtual storage subspace, then use offset + len(buf) to calculate the last virtual storage subspace, and then obtain the virtual storage addresses from the first to the last virtual storage subspace. Finally, we determine the virtual storage addresses from the first to the last virtual storage subspace as the data storage address.
[0076] After determining the data storage address, data storage instructions are generated based on the data storage address and the data to be stored. For example, corresponding SCSI instructions can be generated based on the data storage address and the data to be stored.
[0077] For step S42 above, based on the data storage instruction obtained in step S41, the data to be stored in the data storage instruction is written to the virtual storage address indicated by the data storage address in each sparse file. Then, in step S43, for each storage node, the data to be stored written to the virtual storage address in the sparse file corresponding to the storage node is written to the physical storage address corresponding to the virtual storage address, thereby writing the data to be stored to the storage node and realizing data backup.
[0078] Therefore, the data storage method of this application embodiment sets up corresponding sparse files for each storage node and makes each virtual storage address in the sparse file correspond one-to-one with each physical storage address in the corresponding storage node. In this way, during the data storage process, the data to be stored in the data storage instruction is first written to the virtual storage address in each sparse file, and then the data to be stored written in the virtual storage address in each sparse file is written to the physical storage address in the corresponding storage node that corresponds one-to-one with the virtual storage address. Therefore, compared with the backup method of the prior art through primary and backup nodes, since there is no time difference between the primary and backup nodes, even if one storage node fails, other storage nodes can still store data, so the data is not easily lost.
[0079] In addition, in practical applications, an index file can be created in addition to each sparse file, and the index file can be used to record whether data has been written to each virtual storage address of the sparse file. For example, after step S43 above, the index file can be used to record the data to be stored written to the virtual storage address of each sparse file, so that when a new data storage instruction is received, the index file can be used to determine the data storage address.
[0080] The node setting method provided in the above embodiments of this application can create multiple storage nodes and set sparse files corresponding to each storage node, thereby achieving data backup through the data storage method of the above embodiments. In practical applications, the embodiments of this application can also provide another node setting method, further adding new storage nodes when there are existing multiple storage nodes. The existing multiple storage nodes are each set with corresponding sparse files. As for how to add new storage nodes, it can be done through... Figure 5 The node setup method shown is implemented as follows, which includes the following steps:
[0081] Step S51: Obtain the instruction for adding a new storage node.
[0082] Step S52: Create corresponding new sparse files for each newly added storage node and each storage node.
[0083] Therefore, in step S52, the storage capacity of the virtual storage space of the newly created sparse file is the same as the storage capacity of the physical storage space of the corresponding storage node (or the newly added storage node).
[0084] like Figure 6 As shown, the existing storage nodes are storage node 1 and storage node 2, and each storage node 1 and storage node 2 is configured with a corresponding sparse file D1. Storage node 3 is a newly added storage node.
[0085] In step S51, the instruction to add a new storage node is obtained. This instruction can be input by the user through the interface. For example, due to increased workload or higher equipment failure rates, it may be necessary to add new storage nodes to the existing storage nodes 1 and 2. In this case, the user can input this instruction through the interface. Then, in step S52, corresponding new sparse files are created for storage nodes 1, 2, and 3 respectively. Figure 6 The sparse file D2 in the file.
[0086] Step S53: Divide the newly added storage node and the physical storage space of each storage node into multiple physical storage subspaces, and set the corresponding physical storage address for each physical storage subspace.
[0087] In step S53, the physical storage space of the newly added storage node and the existing storage nodes are divided into multiple physical storage subspaces, and a corresponding physical storage address is set for each physical storage subspace. The specific implementation of step S53 can be the same as step S32 above, and will not be repeated here.
[0088] Step S54: Based on the partitioning method of each physical storage subspace, divide the virtual storage space of each new sparse file into virtual storage subspaces that correspond one-to-one with the physical storage subspace, and set the corresponding virtual storage address for each virtual storage subspace.
[0089] The implementation of step S54 can be the same as step S33 described above, and will not be repeated here.
[0090] In practical applications, considering the continuity of business operations, when a new storage node needs to be added, another approach can be taken: suspending the service of the existing storage node and then adding the new storage node. However, this approach will affect the continuity of business operations. Alternatively, the sparse files of the existing storage node can be copied directly to the new storage node. However, because the existing storage node is continuously processing business operations during the copying process, the copied sparse file will still have data gaps, resulting in inconsistencies between the sparse files of the new storage node and the existing storage node.
[0091] The method provided in steps S51 to S54 of the embodiments of this application creates a new sparse file for each new storage node and each storage node. Then, it makes each virtual storage address in the new sparse file correspond one-to-one with each physical storage address in the corresponding storage node. After that, during the data storage process, it is only necessary to switch to writing to the new sparse file, which facilitates the data consistency between each sparse file.
[0092] Of course, after setting up the new storage node through steps S51 to S54 above, the method may further include synchronizing the data in the sparse file to the new sparse file corresponding to the newly added storage node. For example, in the above... Figure 6 In this process, data from sparse file D1 is synchronized to a new sparse file D2 corresponding to storage node 3. For example, a data sending service can be created on storage node 1 or storage node 2, and a data receiving service can be created on storage node 3. This allows data from sparse file D1 to be sent to storage node 3 and then written to the new sparse file D2. Even if storage node 1 fails, data from sparse file D1 on storage node 2 can still be sent to storage node 3 to achieve data synchronization. This is particularly effective when there are many storage nodes; data synchronization will only fail if all existing storage nodes fail, thus demonstrating a high fault tolerance rate.
[0093] It should be noted that during the data transmission process created in storage node 1 or storage node 2, considering that the data being transmitted is data from a sparse file and may contain a large number of gaps, the data transmission service can first determine whether the data is a gap before transmission. If it is, the data will not be transmitted (for example, it can be discarded); if not, the data will be transmitted, thereby reducing the amount of data transmitted.
[0094] Alternatively, for the new sparse file D2 corresponding to the existing storage nodes 1 and 2, one approach is to synchronize the data in the sparse file D1 to the new sparse file D2. Another approach is that, since both the sparse file D1 and the new sparse file D2 exist in the existing storage nodes (storage node 1 and storage node 2), snapshot technology can be used to layer the data of the sparse file D1 and the new sparse file D2 in the existing storage nodes. Through this data layering, it is not necessary to synchronize the data of the sparse file D1 in the existing storage nodes to the new sparse file D2 in the existing storage nodes, thus reducing the processing load.
[0095] Of course, after adding the new storage node based on the existing multiple storage nodes through steps S51 to S54 above, both the existing multiple storage nodes and the new storage node are each configured with a corresponding sparse file (i.e., the new sparse file), and each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. Thus, in the subsequent data storage process, data can be stored using the following data storage method, such as... Figure 7 The diagram shows the specific flow of this method, which includes the following steps:
[0096] Step S61: Obtain the second data storage instruction, which includes the data storage address and the data to be stored.
[0097] For a detailed explanation of step S61, please refer to step S41 above, which will not be repeated here.
[0098] Step S62: According to the second data storage instruction, write the data to be stored in the second data storage instruction to each new sparse file, where the data storage address in the second data storage instruction indicates the virtual storage address.
[0099] For a detailed explanation of step S62, please refer to step S42 above, which will not be repeated here.
[0100] Step S63: Write the data to be stored in the virtual storage address indicated by the data storage address in each new sparse file to the physical storage address that corresponds one-to-one with the virtual storage address in the corresponding storage node.
[0101] For a detailed explanation of step S63, please refer to step S43 above, which will not be repeated here.
[0102] Obviously, the data storage method in steps S61 to S63 can also solve the problems in the prior art, and will not be elaborated here.
[0103] Based on the same inventive concept as the data storage method provided in the embodiments of this application, the embodiments of this application also provide a data storage device. For any unclear aspects of this device embodiment, please refer to the corresponding content of the method embodiment. Each storage node is configured with a corresponding sparse file, and each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. For example... Figure 8 The diagram shows the specific structure of the device 70, which includes: a data storage instruction acquisition unit 701, a virtual memory address writing unit 702, and a physical memory address writing unit 703, wherein:
[0104] The data storage instruction acquisition unit 701 is used to acquire a data storage instruction, which includes a data storage address and data to be stored.
[0105] The virtual storage address writing unit 702 is used to write the data to be stored to each sparse file according to the data storage instruction, wherein the data storage address indicates the virtual storage address;
[0106] The physical storage address writing unit 703 is used to write the data to be stored in the virtual storage address indicated by the data storage address in each sparse file to the physical storage address that corresponds one-to-one with the virtual storage address in the corresponding storage node.
[0107] The device 70 provided in the embodiments of this application uses the same data storage method as the one provided in the embodiments of this application and has the same inventive concept. Under the premise that the method can solve the technical problem, the device 70 can also solve the technical problem, which will not be elaborated here.
[0108] In addition, in practical applications, the technical effects achieved by combining the device 70 with specific hardware devices, cloud technology, etc., are also within the scope of protection of this application. For example, different units in the device 70 can be deployed in different nodes of the distributed cluster to improve efficiency; or, some units in the device 70 can be deployed in the cloud to reduce costs.
[0109] The device 70 may further include a node setting unit, used to create sparse files corresponding to each storage node, wherein the storage capacity of the virtual storage space of the sparse file is the same as the storage capacity of the physical storage space of the storage node; to divide the physical storage space of each storage node into multiple physical storage subspaces, and to set a corresponding physical storage address for each physical storage subspace; and to divide the virtual storage space of each sparse file into virtual storage subspaces corresponding one-to-one with the physical storage subspaces according to the division method of each physical storage subspace, and to set a corresponding virtual storage address for each virtual storage subspace.
[0110] The device 70 may further include a data storage instruction generation unit, used to obtain a data storage request input by a user, the data storage request carrying the data to be stored; determine the data storage address based on the amount of data to be stored and the storage capacity of the virtual storage subspace; and generate the data storage instruction based on the data storage address and the data to be stored.
[0111] Specifically, dividing the physical storage space of each storage node into multiple physical storage subspaces can include dividing the physical storage space of each storage node into multiple physical storage subspaces with a storage capacity of 4KB.
[0112] The device 70 may further include a node addition unit for obtaining instructions to add a new storage node; and for each new storage node and each storage node, creating corresponding new sparse files, wherein each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node.
[0113] The device 70 may also include a data synchronization unit for synchronizing the data in the sparse file to the new sparse file corresponding to the newly added storage node.
[0114] The device 70 may further include: a recording unit, used to record the data to be stored by writing the virtual storage address of each sparse file into the index file.
[0115] Based on the same inventive concept as the node setting method provided in the embodiments of this application, the embodiments of this application also provide a node setting device. For any unclear aspects of this device embodiment, please refer to the corresponding content of the method embodiment. Each storage node is configured with a corresponding sparse file, and each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. For example... Figure 9 The diagram shown illustrates the specific structure of the device 80, which includes: an instruction acquisition unit 801 and a new sparse file creation unit 802, wherein:
[0116] The instruction acquisition unit 801 is used to acquire instructions for adding a new storage node;
[0117] The new sparse file creation unit 802 is used to create corresponding new sparse files for each newly added storage node and each existing storage node. Each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. This allows the data to be stored in the second data storage instruction to be written to the virtual storage address indicated by the data storage address of the second data storage instruction in each new sparse file, and then written to the physical storage address in the corresponding storage node that corresponds one-to-one with the virtual storage address.
[0118] This invention also provides a storage medium storing a computer program, which can be executed by a processor to complete all or part of the processes of the methods in this application embodiment. The storage medium can be a disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc. The storage medium may also include combinations of the above types of memory.
[0119] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.
Claims
1. A data storage method, characterized in that, In a system with multiple independent storage nodes including a primary node and a standby node, each storage node is configured with a corresponding sparse file, wherein each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. The method includes: Obtain a data storage instruction, wherein the data storage instruction includes a data storage address and the data to be stored; According to the data storage instruction, the data to be stored is written to each sparse file corresponding to all storage nodes, and the virtual storage address indicated by the data storage address; In each sparse file, the data to be stored written in the virtual storage address indicated by the data storage address is written to the corresponding physical storage address that corresponds one-to-one with the virtual storage address in the corresponding storage node. The virtual storage space of the sparse file has the same storage capacity as the physical storage space of the storage node; and the virtual storage space of the sparse file is divided into multiple virtual storage subspaces, and the physical storage space of the storage node is divided into multiple physical storage subspaces. The multiple virtual storage subspaces correspond one-to-one with the multiple physical storage subspaces and have the same size.
2. The method according to claim 1, characterized in that, The method further includes: For each storage node, create a sparse file corresponding to that storage node; The physical storage space of each storage node is divided into multiple physical storage subspaces, and a corresponding physical storage address is set for each physical storage subspace. Based on the partitioning method of each physical storage subspace, the virtual storage space of each sparse file is divided into virtual storage subspaces that correspond one-to-one with the physical storage subspaces, and a corresponding virtual storage address is set for each virtual storage subspace.
3. The method according to claim 2, characterized in that, The method further includes: Obtain the data storage request input by the user, wherein the data storage request carries the data to be stored; The data storage address is determined based on the amount of data to be stored and the storage capacity of the virtual storage subspace; The data storage instruction is generated based on the data storage address and the data to be stored.
4. The method according to claim 2, characterized in that, The physical storage space of each storage node is divided into multiple physical storage subspaces, specifically including: The physical storage space of each storage node is divided into multiple physical storage subspaces with a storage capacity of 4KB each.
5. The method according to claim 1, characterized in that, The method further includes: Retrieve the command to add a new storage node; For each newly added storage node and each existing storage node in the current system, a corresponding new sparse file is created. Each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node.
6. The method according to claim 5, characterized in that, The method further includes: The data in the sparse file is synchronized to the new sparse file corresponding to the newly added storage node.
7. The method according to claim 1, characterized in that, After writing the data to be stored to each sparse file corresponding to all storage nodes according to the data storage instruction, and the virtual storage address indicated by the data storage address, the method further includes: using an index file to record the data to be stored written to the virtual storage address of each sparse file.
8. A node setting method, characterized in that, In a system with multiple independent storage nodes, including a primary node and standby nodes, each storage node is configured with a corresponding sparse file. Each virtual storage address in the sparse file corresponds one-to-one with each physical storage address in the corresponding storage node, including: Retrieve the command to add a new storage node; For each newly added storage node and each existing storage node in the current system, a corresponding new sparse file is created. Each virtual storage address in the new sparse file corresponds one-to-one with each physical storage address in the corresponding storage node. This allows the data to be stored in the second data storage instruction to be written to each new sparse file corresponding to all storage nodes, based on the second data storage instruction. The virtual storage address indicated by the data storage address of the second data storage instruction is then written to the physical storage address in the corresponding storage node that corresponds one-to-one with the virtual storage address. The virtual storage space of the sparse file has the same storage capacity as the physical storage space of the storage node; and the virtual storage space of the sparse file is divided into multiple virtual storage subspaces, and the physical storage space of the storage node is divided into multiple physical storage subspaces. The multiple virtual storage subspaces correspond one-to-one with the multiple physical storage subspaces and have the same size.
9. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to perform the method as described in any one of claims 1-8.
10. A storage medium, characterized in that, The storage medium stores a computer program that can be executed by a processor to perform the method described in any one of claims 1-8.